Sample Size and the Minimum-Trades Threshold — When a Backtest Has Too Little to Say

Every number a backtest reports is an estimate, not a fact. A win rate of 80% is an estimate of the model's true win rate; an expectancy of 3 pips is an estimate of its true per-trade edge. Like any estimate, its reliability depends on how much data it rests on — and the most common way to be fooled by a backtest is to take a confident-looking number computed from a tiny number of trades at face value. Sample size is what separates a result that means something from one that only looks like it does.

Small Samples Lie Loudly

A model that wins eight of its first ten trades has an 80% win rate on paper. It is also entirely consistent with a model that has no edge at all, having simply had a good run — the way ten coin flips can easily land seven or eight heads without the coin being biased. With ten observations, the range of outcomes compatible with pure chance is enormous, so the headline statistic carries almost no information about what the model will do next.

The danger is that small samples do not announce their unreliability. An 80% win rate from ten trades and an 80% win rate from a thousand trades display identically — same number, same green cell on a dashboard — but they justify completely different levels of confidence. The first is noise that happens to look like signal; the second is evidence. Nothing in the figure itself tells you which you are looking at. Only the trade count does.

This is also how an optimiser gets fooled if you let it. A genetic algorithm scored on a thin sample will happily select the model that got lucky, because luck and edge are indistinguishable when there are too few trades to tell them apart. The fix is not a cleverer statistic; it is insisting on enough trades for the statistic to mean anything.

Why Thresholds Exist

This is why darwintIQ enforces minimum-trade requirements rather than trusting any result on its own. A result built on too few trades is treated as inconclusive rather than impressive — the system declines to act on a number it cannot stand behind.

The principle becomes especially important on the out-of-sample holdout. The holdout slice is, by design, only the most recent portion of the evaluation window, so it naturally contains fewer trades than the full history. The holdout selection gate therefore carries its own minimum-trade requirement: a model must produce at least a handful of trades on the unseen slice before its holdout result is allowed to count for or against it. A model that traded only twice in the holdout has not passed an out-of-sample test — it has barely taken one, and a pass or fail on two trades is closer to a coin flip than a verdict.

Thresholds were also rescaled when the evaluation window was extended to 40 hours. A minimum that made sense for a short window would be far too low for a longer one, where many more trades should occur; the gates were adjusted upward so that "enough trades" continues to mean enough relative to the opportunity the window provided.

Sample Size Is Not the Only Dimension

More trades make a statistic more reliable, but quantity does not substitute for the other qualities of a good result. A thousand trades with a marginally positive expectancy and a wild standard deviation is not automatically better than a few hundred steady ones. Sample size tells you how much to trust the estimate; it does not tell you whether the thing being estimated is any good. The right reading combines a sufficient trade count with healthy expectancy, stability, and an out-of-sample result that holds — sample size being the precondition that lets you take the rest seriously.

Final Thoughts

The instinct to be impressed by a high win rate or a clean equity curve is strong, and small samples are very good at producing both by accident. Before reading any backtest statistic as evidence, the first question is how many trades it rests on — because that number governs how much the others are worth. Minimum-trade thresholds, including the one on the holdout, encode that discipline into the pipeline so that a model has to earn its place on a sample large enough to mean something, not on a lucky afternoon.