What Is Backtesting? And Why a Good Backtest Isn't a Promise

If quant trading is about rules, backtesting is how you check whether a rule was ever any good. You take a precise rule, run it across historical price data, and measure what it would have done. It’s the single most useful tool in systematic trading — and the single most misused.

What a backtest actually measures

A backtest replays history bar by bar and applies your rule mechanically: here it would have bought, here it would have sold, here’s the running profit and loss. Out comes a set of numbers — total return, win rate, worst drawdown, number of trades. For the first time, “I think this works” becomes “here’s what it did over 5,000 trades.”

That’s a real upgrade over guessing. But the numbers describe one specific past, not the future.

Why a beautiful backtest still fails live

This is the lesson that costs beginners the most money: a great backtest is not a promise. A curve that climbs smoothly upward can fall apart the moment it meets live markets, for reasons that have nothing to do with bad luck:

• Overfitting — you tweaked the rule until it fit history perfectly. You didn’t find an edge; you memorized the answer key. Markets then ask a different question.
• Lookahead bias — the test accidentally used information that wouldn’t have existed in real time (like today’s close to decide today’s open). The result is fantasy.
• Survivorship bias — testing only on stocks that still exist today quietly hides every company that went to zero.
• Ignored costs — drop in spreads, slippage, and fees and many “profitable” systems turn negative.

Each of these makes the past look kinder than the future will be.

The honest way to read a backtest

Treat a backtest as a way to disqualify bad ideas, not to confirm good ones. A rule that fails its backtest is almost certainly not worth trading. A rule that passes has merely survived one test — it still has to prove itself on data it never saw. The discipline is the same one this whole site preaches: evidence over hope, and no single result is a guarantee.

A realistic expectation

Backtesting tells you how a rule handled the past, honestly and in detail. It cannot tell you it will keep working. Used well, it filters out nonsense; used badly, it manufactures false confidence. Nothing here is financial advice.

Practice it

You don’t need code to feel how this works. Step through history one bar at a time in the Candlestick Replay drill — deciding before you see the next bar. That’s manual backtesting, and it teaches the same humility a good quant brings to a strategy.