In production
Anomaly detection for business metrics: bands beat thresholds
A fixed threshold treats every Tuesday like every Saturday. A forecast band asks whether today is weird given what the series usually does at this clock. That is the difference between an on-call fire drill and a useful alert.
Updated Aug 5, 2026·8 min read
The threshold trap
“Page if traffic > 1,200” treats every Saturday like a crisis, or it ignores a quiet Wednesday that is actually on fire. Seasonal metrics have a clock. A frozen line does not. Teams either live with noise or they raise the bar until a real incident hides under it.
Bands vs a flat line
Use the forecast as the normal
A calibrated interval is “what this timestamp usually does.” When actuals leave p05–p95 more than they should, something changed — demand, tracking, or a broken feed. The forecast is the definition of normal. That is the whole trick, and it is why a stale forecast makes a bad anomaly detector.
You still need the forecast to be honest. If the band is swaggering, you will page on every peak. If it is timid, you will never page. Calibration first, alerts second.
Tuning so people still listen
- Require persistence: two intervals in a row, not one noisy hour.
- Minimum volume, so a SKU that sells two units cannot page the VP.
- Separate ‘pipe looks dead’ from ‘demand broke.’ Zeros in a file are not a season.
What you do after the page
Check the feed. Then the calendar (did someone forget a holiday). Then a scenario: did we pull a promo. Then, if the world really moved, refit. An anomaly is a question, not an automatic retrain. Automatic retrains on bad data are how you teach the model the outage.
FAQ
- Why do static KPI thresholds fail?
- Seasonality and growth. A threshold that was 'high' in February is normal in December. You either live with noise or you slowly raise the bar until a real incident hides under it.
- How do forecast bands detect anomalies?
- If actuals fall outside a calibrated interval (say p05–p95) more than they should, something changed — demand, tracking, or a broken feed. The forecast is the definition of 'normal for this timestamp.'
- Won't this alert every time the forecast is wrong?
- If the model is poorly calibrated, yes. Fix calibration first. Then require persistence (two intervals in a row) or a minimum volume so a single noisy hour does not page the team.
Keep going
Guide
Prediction intervals
A prediction interval is a range the future value is expected to fall in, not a promise. p50 is the middle path; p10–p90 is a band for planning stock, staff, and cash without pretending the future is a single line.
Guide
Why forecasts go stale in production
A forecast starts dying the moment the world moves and the model does not. New promotions, a competitor, a warehouse fire, a quiet change in how you record demand — live systems need a refresh cadence, drift checks, and a way to re-score the champion.
Guide
Forecasting with AI agents
An agent can create a forecast, ask why it moved, and push the number into the tools you already run — if the platform exposes those jobs as tools, not as a chat box glued to a screenshot. The forecast still has to be a real model with a score.
Use case
Fraud volume forecasting
Forecast fraud pressure by channel, geography, payment method, and attack pattern before queues spike.
Use case
Digital traffic forecasting
Forecast traffic, transactions, and concurrency before campaigns, launches, and seasonal peaks.