Models & methods

Ensemble

An ensemble combines more than one model — an average, a median, a weighted stack — so no single architecture has to be right on every week. Tournaments that pick a winner are a hard ensemble; blending is a soft one.

Simple averages of decent, uncorrelated models are embarrassingly strong. Complicated stacks overfit the bake-off.

In production, an ensemble is a serving contract: you must run every member, or distill. Latency and cost are part of the accuracy number, whether the paper said so or not.

A tournament that picks one winner is a hard ensemble. A median of three is a soft one. Use whichever you can actually operate at 6am.