How to test a strategy so it can fail.

What a backtest can and cannot tell you

A backtest answers one narrow question: if this exact rule had been followed on this exact data, with these assumptions about fills and costs, what would have happened? It does not tell you why it happened, whether it will happen again, or whether you would have stuck with it through the bad months.

Three things turn a backtest into evidence. A rule written down before the test, so the result cannot shape the rule. A control that shows what the same test produces when there is nothing to find. And data the rule has never seen. Without all three, a profitable backtest is a hypothesis, and usually a wrong one.

Every number on v0mma is stored with the data it came from, the period, the cost model and a set of warning flags the record computes itself: no out-of-sample segment, how many variants were tried on the same data, results before costs, a small sample. The flags travel with the number. A profit factor of 4 on twelve trades before costs is not a profit factor of 4.

Measure every trade in R

R is the profit or loss of a trade divided by the risk you took at entry, the distance to your stop times your size. A trade that risks $200 and makes $300 is +1.5 R. Measuring in R makes strategies comparable across markets, years and position sizes, and it puts the question where it belongs: how much does this rule earn for each unit of risk?

Expectancy is the only performance number that matters on its own. Win rate, profit factor and net profit are all consequences of it plus the shape of the exits and the number of trades. Report expectancy with the t-statistic and the number of trades, and you have said most of what can honestly be said.

The cost fraction decides what is worth testing

Every trade pays a round turn: commission plus the spread and slippage on the way in and out. In R, that toll is the round-turn cost divided by the stop distance. On a micro Nasdaq contract a round turn costs roughly 1.75 index points. With a 10-point stop the toll is 0.175 R per trade before the strategy has done anything; with a 50-point stop it is 0.035 R.

Most intraday edges, when they exist at all, are a few hundredths of an R. So a tight stop is not just "more aggressive". It can make the cost the largest number in the trade, larger than any edge you could hope to find. A useful rule of thumb from our own tests: if the stop is not at least about twenty times the round turn, costs will eat most edges.

Your exit sets your win rate

On a market with no edge, a trade with a stop of 1 R and a target of k R wins about 1 ÷ (1 + k) of the time. A target of a third of the stop wins about 75% of trades; a target of three times the stop wins about 25%. The expectancy is the same in both cases: zero, minus costs.

That is why a high win rate proves nothing. Profit-locking stops, small targets and averaging down all raise the win rate without creating an edge. The honest comparison is always the same exit applied to random entries. If random entries win as often, the win rate belongs to the exit, not to your signal.

Controls: what would have made the same money?

Medicine compares a drug with a placebo. Trading research almost never does, and that is where most false edges live. Four controls catch most of them:

Random entries. The same exits, costs and trade count, with entries at random times. The rule must beat the distribution of random results, not zero.

Placebo levels. If a rule trades a special price (a wall, a pivot, an expected move), run it at nearby prices that are not special. If the placebo earns the same, the level is decoration.

Placebo times. If a setup depends on a special time (10:00, the 8 o'clock range, a data release), run it at every other time.

Mirror trades. Take every signal in the opposite direction with the same distances. If the mirror does as well, the signal carries no direction, and the result came from the geometry of the bracket.

Selection bias: the best of many is lucky

Test one random rule and its t-statistic will rarely exceed 2. Test a thousand and the best one almost always will, on data that contains nothing at all. Every parameter you tune, every filter you add and every market you try is another test, whether you count it or not.

The fix is to judge a search by the null of the whole search. Run the identical search on data where the outcomes have been shuffled against the signals, many times, and record the best result each time. Your real winner has to beat the 95th percentile of those best-of-noise results. We ran an exhaustive search over 195,203 filter combinations and its proven optimum landed exactly on the median of the shuffled searches.

Out-of-sample: keep a vault

Data you have looked at is no longer a test. Before you start, lock away a block of history and do not touch it until the rule is final. Then look exactly once. If the rule fails, it is dead; re-tuning it on the vault turns the vault into training data.

Split the rest of your history too. A rule that works in 2015-20 and not in 2021-26 has told you something about 2015-20. Prefer the most recent years when you weigh evidence, because they are closest to the market you will trade, and published effects decay. Walk-forward testing (fit on the past, test on the next block, roll forward) formalises the same idea for rules with parameters.

Look-ahead bias and fill artifacts

Look-ahead bias is using information that did not exist at the moment of the decision: the high of a candle that has not closed, a daily ATR that includes today, an indicator computed on the bar you enter. It is usually one line of code, and it can turn pure noise into a fortune.

The best defence is to run every new engine on random-walk data first. A random walk has no edge by construction; if your strategy makes money on it, the engine is broken. The second defence is to check fills. Can a stop and a target both be hit inside the entry bar? Does a limit order fill on a touch or need a trade-through? Re-run the result with the strictest reasonable rule, for example fills only from the next bar. An edge that disappears is a fill artifact.

Fat tails: when a few trades carry everything

Breakout and trend strategies often lose on most trades and make their money on a handful of very large days. That can be a real edge, but it makes results fragile. If the best 5% of trades carry all of the profit, then the result depends on whether those few days happen again, and a filter that removes even one of them can look like a disaster.

Two checks keep you honest. Recompute the result without the best 5% of trades and see what is left. And be careful with filters on such strategies: with a fat-tailed outcome, a filter needs an enormous effect before a test can tell it apart from luck.

Prop-firm math

A funded-trader evaluation changes what a good strategy is. A typical $50,000 evaluation asks for $3,000 of profit before the account falls $2,000 below its highest end-of-day balance, and sometimes adds a consistency rule: no single day may be more than 40% of the total profit.

Three consequences follow. The trailing drawdown punishes losing streaks far more than the average trade rewards you, so the size of your losing runs matters as much as your expectancy. More trades buy speed (fewer days to the target), not a higher pass rate, and they often lower it. And inside an evaluation your downside is capped at the fee, so more variance can raise your chance to pass; once the account is funded, the same variance is simply risk.

The pre-flight checklist

Before any strategy trades money, it has to clear all of these. Almost nothing we test does, which is the point.

1. The rule, the costs and the pass mark are written down before the first run.
2. Costs are realistic for the contract you will trade, including slippage, and the stop is large compared with the round turn.
3. The engine has been run on random-walk data and makes nothing there.
4. The result beats random entries with the same exits, and placebo levels or times where they apply.
5. If more than one variant was tried, the result beats the null of the whole search.
6. Pooled over the full sample, t ≥ 2.5; most years are positive; the most recent two years are positive on their own.
7. The result survives the strictest reasonable fill rule and the removal of the best 5% of trades.
8. It holds on data that was locked away, or on a second market where the mechanism should also work.
9. The account rules have been simulated: drawdown, daily limits, consistency, and how long it takes to pass.
10. A forward period of live or paper trades agrees with the backtest before size goes up.