Accuracy & evaluation
Forecast value added: is the model worth the meeting?
Forecast value added (FVA) asks whether each layer — the naive, the model, the planner’s override — actually reduced error. If the override makes WAPE worse, the meeting is a cost center, not a control tower.
Updated Aug 26, 2026·8 min read
Every layer is a model
The naive is a model. The statistical champion is a model. The planner’s override is a model with a human in the loop. Forecast value added asks whether each layer reduced error on the same folds. If the override makes WAPE worse, the meeting is a cost center. It can still be politically useful. It should not get to pretend it is accuracy.
Pick the pain, then the metric
| Metric | Use it when | Be careful when |
|---|---|---|
| WAPE | Volume plans | Zeros, mixed SKU sizes |
| MAE | Every unit costs the same | Comparing different scales |
| RMSE | Rare spikes are the disaster | Noisy series, vanity 'stats' |
| MASE | Many series, one leaderboard | Wrong seasonal period |
| Bias | Always heavy or light | Using it alone |
How to measure FVA
Hold the origin dates still. Score naive → system → adjusted. Report the change in the metric ops feels (usually WAPE or MAE), and bias. A layer that cuts error in grocery and raises it in pharma is not “good.” It is a mix. Slice.
- If system FVA versus naive is negative, you do not have a forecasting problem. You have a data or grain problem.
- If planner FVA is negative on average, stop requiring overrides. Allow them where the planner has information the model cannot see.
- If only one person beats the system, that is a key-person risk, not a process.
Overrides that feel wise
A factory fire, a one-off bid, a regulation that starts Monday: those are reasons to touch the path. “I have a feeling Q4” is not. FVA will not stop a VP from editing a cell. It will make the hangover visible in April.
A baseline that never leaves production
Every Predict.ai goal keeps a naive on the same folds as the champion. That is FVA for the system layer, always on — not a slide someone forgot to refresh. If you add a human layer, compare it to the live path, not to memory.
The tournament is also FVA in disguise: a booster that cannot beat last week does not get to be the number in the API. That is the product doing the meeting’s job before the meeting starts.
FAQ
- What is forecast value added?
- The change in error when you add a step: statistical forecast versus naive, planner versus system. Positive FVA means that step earned its place. Negative means you paid for theater.
- Should planners stop overriding?
- No. They should override where they have information the model cannot see (a factory fire, a one-off bid). FVA tells you whether those overrides, in aggregate, help.
Keep going
Guide
What is good forecast accuracy?
Good is beating a naive baseline on the same grain, horizon, and folds — by enough to change a decision. Industry round numbers are gossip until they match your SKU mix and your clock.
Guide
How to backtest a forecasting model
Backtesting asks the model to forecast a past window it was not trained on, then scores the miss. Walk the window forward so you see many 'futures,' not one lucky test month.
Guide
Why one model never wins
A model that dominated last spring can lose in November. Series change, drivers change, and luck exists. Keep a challenger on the same folds, promote on evidence, and stay loyal to the score — not to the architecture that won a bake-off in March.
Use case
Retail demand forecasting
Anticipate demand by product, store, channel, and region before buying or allocation decisions are locked.
Use case
Revenue forecasting
Maintain a current revenue outlook grounded in bookings, pipeline, usage, renewals, pricing, and seasonality.