LoadML has spiked for no reason

[Sycamore967]

Describe the bug
LoadML has spiked and the plan shows 6 times the load it should. I have plenty of history (21 days) and LoadML has been running for a week now across various updates. It's not done this before.

Expected behaviour
LoadML calculates correctly

Predbat version

8.33.10

Environment details

• Inverter and battery setup - GE 3.6 Hybrid and two batteries
• Standard HAOS
• No errors or warnings in PB at all.

Screenshots

Log file

predbat.log

Predbat debug yaml file

predbat_debug.yaml.txt

[gcoan]

Just looking at the chart, your ACTUAL load for today looks very much higher than it has for other days (at the same time), so looks like load ML is amplifying that high load and projecting it forward

worth checking what is in your house load energy sensor and any car energy sensors you have configured

maybe a problem with removing car energy?

[Sycamore967]

That's a fair point - it looks like Hypervolt have an issue their side with their servers and nothing was recorded for my car charge (which did happen - it's just not been recorded) - so that's the spike - I don't know of a way to remove that from the load history though.

[Sycamore967]

mmm - interesting Hypervolt are escalating my ticket - they don't know atm why the charge session wasn't reported. Not a PB issue, but I will obviously need to charge the car (for the rest of the week) - so how will ML now treat what it could potentially see as a huge house load? - or can a use something to exclude a nominal amount?

[Sycamore967]

Hypervault fixed their issues and car charging was fine last night. ML still seems to be taking the one occurrence of the high load and using that going forwards. Even though the car load was in the early hours of the morning, it's predicting a high load at 09:30 this morning.

With so much history to look back over, is it weighting the very recent history a little too much?

[gcoan]

Even though the car load was in the early hours of the morning, it's predicting a high load at 09:30 this morning.

where the 'now' line is, the forecast (red), +1 hour (amber) and actual (blue) lines look pretty convergent. Its just the +8 hours (purple) line is that was spiked, 8 hours ago, and if you look at the trend for it, it converges back to the red line pretty soon

[Sycamore967]

Hi, yeah I'm happy that things have converged, perhaps I just need to understand how ML is working things out. I'm expecting the weighting of any prediction to use the history, both in HA and now in the ML database to smooth the prediction curve, but in practice ML seems to have still assumed a larger than normal load mid morning (09:28 today) when I don't see the evidence for that in the previous week. Admittedly, the car charging (actual load) is skewing the figures (from the day before) but that was during the early hours, so why has ML calculated a predicted load that is so large 6hrs into the future in the case of Wednesday and 4hrs today (relative to the end of the car charging load that wasn't excluded).

You were spot on with the initial reason for the strange behaviour and I'd close this issue based on that, I'm just hoping to understand what I'm looking at and how its calculated a bit better and maybe help others if they read this.

[gcoan]

I don't have all the answers because I don't understand the inner workings of what load ML is doing

If I zoom the chart in, the now line appears to be at 9:30:

and zoom it in further:

it looks like whilst the +8 and +1 hour forecasts were very inflated initially for today (doubtless by yesterday's car charging energy sensor issue), by about 7am with the blue actual load tracking lower both the +1 and +8 hour forecasts had come down, so load ML was self-corrected

bear in mind that at 'now' 9:30, the purple line is what was forecast 8 hours ago, i.e. at 01:30, and yesterday that would have included EV charging The yellow and purple lines are the historic future projections not the current projections
I tried to explain this better in an update to the docs in my fork https://github.com/gcoan/batpred/blob/main/docs/load-ml.md#monitoring-and-troubleshooting

but its not easy without getting into a tongue-twister of words

[johnwb87]

but in practice ML seems to have still assumed a larger than normal load mid morning (09:28 today) when I don't see the evidence for that in the previous week.

That is what LoadML forecasted at 01:28 this morning. Did you charge the car last night? Did it start around 1:30? Is the energy sensor reported from your charger coming from a cloud API integration so it is delayed some what?

So the hypothesis is that LoadML saw a high load and thought blimey if this carries on that's where we will be in 8 hours time. Then the car sensor updated for subsequent runs and it corrected, hence the major drop back down.

This also explains Wednesday. It sees high load and starts taking that into account for the 8 hour ahead forecast. The car sensor doesn't update so LoadML is really sweating that it is going to carry on into the future. I guess the drop back coincides with roughly 8 hours after the car charging stops.