Why a backtest lies a little
A backtest is a trading idea run on old data. Easy to say. Harder to use well.
We run them for two different jobs.
Job one is a decision. Which strategy. Which markets. How to split risk. Most people answer that with one question: which number looks better?
Job two is a picture of behaviour. How often it trades. What it costs. How much leverage it needs. That job gets skipped a lot.
Those two jobs want different tests.
The awkward bit
A backtest is the past. We then treat it as a map of the future. That does not quite fit.
Say you have strategy A and strategy B, and data back to 1970. Looking back, B is better. In 1970 you would not have known that.
If you want an honest picture of the past, you would have run both until the evidence at that time said B was ahead. The test starts as a mix and slowly becomes mostly B. That mix is what someone with no crystal ball could have earned.
That mix is not what you will trade on Monday. On Monday you will trade B. If you want to know how B behaves — trades, costs, leverage — test B on its own.
That B-only test will look better than the honest mix. You picked the winner with the whole story in hand. That is in-sample fitting.
Some of that is unavoidable. You cannot choose between A and B without looking. After you choose, you have to look at the thing you chose. It is fine if you remember: the number is too high.
The trap
The real mistake is dropping A completely because B won by a bit. Markets are noisy. A small win in a backtest is not a verdict.
The silly version: trade only on 26 January because that day had the best average return. The normal version: trade only one market because it topped the table.
Some history will not repeat. A long fall in interest rates made being long bonds look brilliant. That is not an edge you can count on forever.
A safer test starts with A and B together, and moves money toward the winner as evidence comes in. You often end with both, just more in B. Going forward, trade those weights. Then, if you want to study the mix, run a second test with the weights fixed.
That second test is still a bit optimistic. Less than “B only”. You did not put every egg in one basket.
How to use the number
Trust a result when it is strong and it has a reason. If you cannot explain it, it may just be luck — even if the statistic looks fine.
Assume the number is too high. Even a careful test, with no peeking, will still flatter what you get in the live book.