Backtest: rolling out-of-sample forecasts, each made without that day’s prices, but made afterwards, plus the live days so far. The live forecasts are on the main page.
| Model | MAE (€/MWh) | RMSE (€/MWh) | rMAE | In MCS (10%) |
|---|---|---|---|---|
| Ensemble | 17.1 | 26.7 | 0.496 | yes |
| BMT-X | 18.3 | 28.9 | 0.531 | yes |
| XGBoost-X | 18.3 | 27.9 | 0.532 | yes |
| LEAR-X | 18.5 | 28.9 | 0.539 | |
| Naive (weekly mix) | 34.4 | 51.7 | 1.000 |
| Model | Mean error | MZ slope | MZ p | Cheapest hour ±1h | Peak hour ±1h | Rank correlation |
|---|---|---|---|---|---|---|
| Ensemble | 6.2 | 1.08 | <0.001 | 73.5% | 79.3% | 0.89 |
| BMT-X | 6.3 | 0.99 | <0.001 | 65.6% | 77.2% | 0.86 |
| LEAR-X | 7.9 | 1.10 | <0.001 | 72.7% | 76.4% | 0.88 |
| XGBoost-X | 4.5 | 1.02 | <0.001 | 71.1% | 79.3% | 0.88 |
| Naive (weekly mix) | -0.3 | 0.62 | <0.001 | 56.2% | 63.5% | 0.76 |
Bias correction (median error, last 28 days): mean error +6.0 to +0.6, MAE 17.1 to 16.3 (DM p = 0.008), significant for 3 of 4 models. It runs live as a challenger. a + b × forecast instead: MAE 16.8, worse (DM p = 0.005).
Rolling out-of-sample forecasts, 01 Oct 2025 to 19 Oct 2026, 381 days (375 backtest days, then 6 live days), 9,144 hourly forecasts, all models on the same hours.
Ensemble cuts the average error by 50% compared with the naive benchmark (the weekly naive). Ensemble is significantly more accurate than BMT-X (DM p < 0.001, GW p < 0.001), significantly better in 14 and worse in 0 of the 24 hours.
| Comparison | MAE diff | DM p | GW p | Hours better / worse |
|---|---|---|---|---|
| Ensemble vs BMT-X | -1.19 | <0.001 | <0.001 | 14 / 0 |
| Ensemble vs XGBoost-X | -1.23 | <0.001 | <0.001 | 9 / 0 |
| Ensemble vs LEAR-X | -1.48 | <0.001 | <0.001 | 20 / 0 |
| Ensemble vs Naive (weekly mix) | -17.34 | <0.001 | <0.001 | 24 / 0 |
| Method | Pinball | Skill vs naive | DM p vs best | 50% range covers | 90% range covers | Width 50% / 90% (€/MWh) | Calibration error |
|---|---|---|---|---|---|---|---|
| Error quantiles | 6.36 | 53.8% | best | 45.7% (p 0.002) | 85.0% (p <0.001) | 22.5 / 61.9 | 0.027 |
| Scaled by volatility | 6.45 | 53.2% | 0.019 | 48.0% (p 0.224) | 86.9% (p 0.005) | 25.2 / 68.9 | 0.017 |
| Adaptive (ACI) (used) | 6.47 | 53.1% | 0.066 | 49.5% (p 0.719) | 89.4% (p 0.535) | 26.0 / 79.1 | 0.005 |
Used: adaptive (ACI), the lowest pinball loss of the methods on target. 90% range: 89.4% of hours (p = 0.535), 50% range: 49.5% (p = 0.719), 333 days. Chance of a negative price: Brier score 0.0135 vs 0.0211 for the 28-day share, 36.0% better.
| Share of prices below the quantile | 5% | 10% | 25% | 50% | 75% | 90% | 95% |
|---|---|---|---|---|---|---|---|
| Adaptive (ACI) | 5.3% | 10.2% | 24.7% | 49.2% | 74.2% | 89.6% | 94.7% |
| Error quantiles | 7.0% | 11.7% | 25.0% | 46.7% | 70.6% | 86.5% | 92.0% |
| Scaled by volatility | 6.2% | 10.6% | 24.4% | 48.0% | 72.4% | 87.4% | 93.0% |
| Strategy | EUR per MW per year | Share of perfect | MAE (€/MWh) |
|---|---|---|---|
| Perfect foresight | 78,462 | 100.0% | |
| Ensemble | 74,063 | 94.4% | 17.1 |
| XGBoost-X | 73,444 | 93.6% | 18.3 |
| LEAR-X | 73,110 | 93.2% | 18.5 |
| BMT-X | 72,193 | 92.0% | 18.2 |
| Naive (weekly mix) | 69,133 | 88.1% | 34.4 |
Each day a 2 MWh / 1 MW battery, 90% round trip efficiency, max 1 full cycle per day plans tomorrow on the forecast and is paid the actual prices, over 375 backtest days, then 6 live days. Perfect foresight plans on the actual prices.
The ensemble earns significantly more than the weekly naive (p < 0.001). The ensemble and XGBoost-X are not significantly different in profit (p = 0.056). The ensemble is both more accurate and earns more.
No fees or battery wear.
| Method | Days below the floor | Kupiec p | Christoffersen p | Acerbi–Székely p | FZ0 score | DM p vs best | VaR / ES (€) | Live misses |
|---|---|---|---|---|---|---|---|---|
| Historical errors | 9.6% (32 of 333) | <0.001 | 0.274 | <0.001 | 5.31 | 0.045 | 98 / 137 | 0 of 6 |
| Scaled by price volatility | 7.5% (25 of 333) | 0.058 | 0.199 | 0.003 | 5.28 | 0.009 | 110 / 158 | 0 of 6 |
| Normal | 5.1% (17 of 333) | 1.000 | 0.024 | 0.050 | 5.28 | 0.053 | 127 / 159 | 0 of 6 |
| Scaled by recent errors (EWMA) (used) | 7.2% (24 of 333) | 0.102 | 0.149 | 0.082 | 5.04 | best | 96 / 158 | 0 of 6 |
Past days’ 24-hour errors, run through the day’s plan, in four ways: as they were (60 days), scaled by last week’s price volatility, a normal distribution, and scaled by an EWMA of recent errors (up to 250 days).
Rule set before running: pass Kupiec, Christoffersen and Acerbi–Székely at 5%, then the lowest FZ0 score. EWMA was added after the first three failed, so the live days are its real test.
Used: scaled by recent errors (EWMA). Below the floor on 7.2% of 333 days (Kupiec p = 0.102). The misses don’t cluster (Christoffersen p = 0.149) and aren’t worse than the ES says (Acerbi–Székely p = 0.082).
No fees or battery wear.