Accuracy & evaluation
Choosing a forecast accuracy metric
Pick the metric that matches the cost of being wrong. WAPE for volume, MAE when units are the pain, RMSE when big misses hurt more than small ones, bias when you always run heavy or light.
Updated Aug 19, 2026·10 min read
The metric is a job description
You do not “choose accuracy.” You choose which misses are allowed to dominate the meeting. Percent errors care about relative miss. Unit errors care about volume. Squared errors care about the rare disaster. Bias cares about the lean. Pick the one that matches the cost, then stop shopping when the bake-off gets uncomfortable.
Pick the pain, then the metric
| Metric | Use it when | Be careful when |
|---|---|---|
| WAPE | Volume plans | Zeros, mixed SKU sizes |
| MAE | Every unit costs the same | Comparing different scales |
| RMSE | Rare spikes are the disaster | Noisy series, vanity 'stats' |
| MASE | Many series, one leaderboard | Wrong seasonal period |
| Bias | Always heavy or light | Using it alone |
Percent errors
MAPE is the celebrity. WAPE is the one that survives a catalog. sMAPE was invented because MAPE kept exploding; there are two formulas, so paste yours in the footnote. None of these replace a signed bias. You can look “12% accurate” and still bleed cash if you are always heavy on yogurt.
Unit errors
MAE is the average miss in the original units. Ops can feel it. RMSE squares first, so a 10-unit miss is more than twice a 5-unit miss. Use RMSE when the tail is the product (an SLA, a stockout that takes down a line). Use MAE when every extra case is roughly the same headache.
Scale-free scores
MASE divides your MAE by a naive seasonal baseline. Below 1 means you beat last week (or last year). It is how you compare a big series and a small one without a fake percent. Get the period right — daily often 7, monthly 12 — or you will crown a model for beating a stupid baseline.
Don’t skip bias
Mean signed error, or percent bias. Slice it. Models that look unbiased at chain level are often drunk on one brand. You can debias with a multiplier if the lean is stable. If it flips every month, you have a regime, not a constant.
A practical pick
Volume plans: WAPE + bias. Unit-cost worlds: MAE + bias. Many series on one board: MASE, then glance at WAPE on the A items. Tail risk: look at a high quantile or RMSE, not only the average. And run a seasonal naive on the same folds so “good” has a floor.
FAQ
- Which forecast accuracy metric should I use?
- If the series has zeros or mixed scales, start with WAPE plus bias. If every unit costs the same, MAE is honest. If a rare spike is the disaster, look at RMSE or a tail pinball loss, not a single average.
- What is MASE?
- Mean Absolute Scaled Error. It divides your MAE by the MAE of a naive seasonal baseline. Below 1 means you beat 'same as last week/year.' It is handy when you compare series of different sizes.
- Should finance and ops share one metric?
- They should share a dashboard, not necessarily one number. Finance may care about revenue WAPE; ops may care about units on the A items. One metric that makes both groups shrug will not change a PO.
Keep going
Guide
WAPE, explained
WAPE (weighted absolute percentage error) is total absolute error divided by total actuals. It behaves when some periods are zero, and it does not let tiny SKUs dominate the score the way MAPE does.
Guide
What is good forecast accuracy?
Good is beating a naive baseline on the same grain, horizon, and folds — by enough to change a decision. Industry round numbers are gossip until they match your SKU mix and your clock.
Guide
How to backtest a forecasting model
Backtesting asks the model to forecast a past window it was not trained on, then scores the miss. Walk the window forward so you see many 'futures,' not one lucky test month.
Use case
Revenue forecasting
Maintain a current revenue outlook grounded in bookings, pipeline, usage, renewals, pricing, and seasonality.
Use case
Energy load forecasting
Forecast demand by interval, feeder, zone, and customer class with weather-driven uncertainty.