Memory-constrained forecasting
19% lower tail-forecast error at equal memory
Best budget: 2 learned slots beat 28 raw values — q99 pinball loss 0.01174 vs 0.00949.
Compare the two arms, not the levels: tail-oracle losses are inflated by a readout mismatch.
Same bytes, different memory
Drag the budget. Both bars always fill the same bytes — what changes is what the forecaster keeps in them.
75,848 bytes (75.8 KB)
Budget 2 of 6 · 75.8 KB · recency loss 0.01174 vs 0.00949 — 19% lower (p < 0.001).
Spikes: measured tail error by budget
Tail-forecast error for each arm at each budget, measured on the held-out test split. Lower is better. Tab to a budget, use the arrow keys to move, Enter to select.
Tail loss at this budget: 0.07763 vs 0.04269 — 45% lower (p < 0.001). Compare the arms; the levels are inflated by a readout mismatch.
All six budgets as a table
| Vector / raw slots | Bytes | Recency raw | Recency vector | Recency p | Tail raw | Tail vector | Tail p |
|---|---|---|---|---|---|---|---|
| 1 / 14 | 61,876 | 0.01161 | 0.01009 | 0.0000283 | 0.07552 | 0.04501 | 0.00142 |
| 2 / 28 | 75,848 | 0.01174 | 0.009486 | 7.66e-10 | 0.07763 | 0.04269 | 0.000364 |
| 3 / 42 | 89,820 | 0.01173 | 0.009649 | 3.57e-8 | 0.0776 | 0.04215 | 0.000289 |
| 4 / 56 | 103,792 | 0.01175 | 0.0101 | 0.0000365 | 0.07623 | 0.04161 | 0.000326 |
| 5 / 70 | 117,764 | 0.01176 | 0.01068 | 0.0112 | 0.07267 | 0.04111 | 0.000535 |
| 6 / 84 | 131,736 | 0.01157 | 0.01101 | 0.205 | 0.06187 | 0.04059 | 0.00153 |
The honest numbers
- 14×
- more raw history beaten at the same bytes, at every budget
- 12 of 12
- comparisons favour the learned summaries; 11 of 12 are significant on their own (p < 0.05)
- p = 0.2046
- the one exception: recency at 6 vector slots
- 1 market · 1 year
- of real electricity prices (499 nodes, 35035 intervals, 3504 held-out queries). Other markets are not tested.
How this works
What was compared
Two forecasters get exactly the same bytes. One spends them on a few 14-wide slots: a node’s latest price plus summaries of what the whole grid is doing. The other spends them on 14 times as many slots of that node’s own recent history. The comparison was written down before the test data was scored, and each budget gets its own Diebold–Mariano test, corrected for correlated errors.
What the numbers mean
Price spikes are rare and carry the risk, so forecasters are scored on the 99th-percentile tail, not the average. The headline 19% is the cut in q99 pinball loss at the best budget (2 vector slots vs 28 raw slots). Spikes are shown as per-budget tail error, measured on the held-out test split — every trace on this page is measured; nothing here is illustrative.
Where it does not hold
A learned memory keeper did not beat a simple baseline at any of three scales, and an earlier claim that streaming scenario selection would beat batch selection was not supported. Both are negative results, kept. One market, one year of prices, one machine.