My backtest said 13.25×. After auditing it myself, 2.26×.

When a backtest and a live account disagree, the first explanation reached for is costs. Slippage, commission, borrow. Costs are real and they are usually not the whole story, because costs are symmetric and bugs are not: the ones that flatter a result survive longer than the ones that spoil it.

Mine disagreed by a factor of six. The audited figures against the originals:

Metric	Before audit	After audit
Total multiple	13.25×	2.26×
Annualised	+58.6%	+15.7%
Max drawdown	−14%	−45%
Per-trade t	—	1.40 (threshold 2.0, not passed)

None of the gap came from parameters. It came from three defects, none of which raised an exception, and all three pointing the same way.

That direction is the finding, not the numbers. A bug that makes a backtest look worse gets investigated the same afternoon. A bug that makes it look better gets treated as a result. Selection happens in the attention, not in the code.

One trade was 80.5% of the total return. Check that trade, do not celebrate it

Sort trades by contribution and look at the top one. Mine was +25,576%, carrying 80.5% of the entire result.

Its shape: 95 consecutive sessions with zero volume, then an open at 124.7× the prior price.

The backtest had no volume check, so it “bought” and “sold”. In the world, that is a stock resuming after a halt, and there was no counterparty at that price.

Halts are ordinary exchange machinery. The SEC’s investor material describes them as time for market participants to absorb material news (investor.gov, retrieved 2026-08-27). For a backtest only one consequence matters: during a halt there is no executable price, and the historical file still has a row for those days.

A row in the file is not proof you could have filled.

Removing that one trade took eighty percent of the return with it.

The check: if any single trade carries more than half the result, go and verify it filled. Look at the volume around it, not at the price.

Ticker as primary key: 13 renamed instruments bought twice

Thirteen instruments changed ticker inside the sample. The data source stored the old symbol and the new symbol separately, so the backtest saw the same company’s same price history as two independent opportunities and took both.

The crypto side of this site hit the same thing: a delisted contract relisted under a new symbol. Any market where the symbol is not the identity has this.

Maintaining your own rename table is not the fix. That table will miss one, and missing one raises nothing. The fix is a key that does not change. The academic standard is a permanent identifier. CRSP’s PERMNO exists for this, described as “fixed for the life of a security” (Morningstar Indexes, retrieved 2026-08-27).

Using ticker as the key assumed something mutable would not mutate. That assumption was never written down anywhere, which is exactly why nobody checked it.

The check: ask whether your key can change inside the sample. If it can, ask what happens when one instrument appears twice, and whether anything would tell you.

Equity marked only on exit days: −14% reported, −45% actual

The accounting function recomputed net value only when a position closed. Open-position drawdown never entered the equity curve at all.

reported max drawdown   −14%
recomputed daily        −45%

This is the one that matters most, and not because of return. Drawdown decides whether you can hold.

A statement will show −45%, and that is where a person turns the system off. Someone who planned around −14% has not planned for −45%. The number’s purpose was never comparing strategies; it is deciding in advance how far down you can still sit still — and a figure that never experienced a single day inside a position cannot answer that.

The check: print how many times the equity curve updates. If that count equals the number of closed trades rather than the number of sessions, the curve is not describing your account.

What the three have in common

#	What the backtest believed	What was true
1	A row in the file means a fill	No counterparty during a halt
2	The symbol is the identity	Symbols change, companies do not
3	Net value equals value at exit	The account moves every day

Each is an assumption nobody wrote down. Unwritten means unchecked, and unchecked meant all three drifted quietly in the same direction.

The dataset-level version of this has been measured. Elton, Gruber and Blake analysed survivor bias in mutual fund databases and found that keeping only the funds that survived systematically overstates performance (Review of Financial Studies 9(4), 1996, abstract, retrieved 2026-08-27).

That is bias the dataset hands you. Mine were written by hand, an order of magnitude larger, and no paper will catch them for you.

Bailey, Borwein, López de Prado and Zhu point the same way: try enough configurations and an attractive backtest is close to inevitable, whatever edge the strategy does or does not have (Notices of the AMS 61(5), 2014, SSRN, retrieved 2026-08-27).

Read together: the default state of a backtest number should be “false” until somebody shows otherwise, and the cheapest way to show otherwise is to audit it yourself first.

Five checks you can run

1. Sort trades by contribution and open the top one. Over half the result means go and check the volume, not the price.
2. Ask whether the key mutates. If it does, ask what happens when one instrument shows up twice, and whether anything reports it.
3. Count equity-curve updates. The count should be sessions, not exits.
4. Write down the dataset’s survival condition. Are only the instruments that survived to today in there? That condition shapes your conclusion.
5. Publish before and after side by side. Reporting only the audited figure asks the reader to trust you; the pair shows the size and direction of the bias.

The fifth gets skipped most, and it is the only one that is about credibility rather than arithmetic.

Why the conclusion was “keep going”, not “shut it down”

2.26× after the audit, t=1.40 against a threshold of 2.0. The line is still running, on money small enough not to hurt.

It is not repackaged beta: +0.21 beta, +0.13 correlation to the index, +12.8% annualised alpha. In the 2022 drawdown year the strategy was −5.1% against −18.2% for the index.

Unproven is not disproven. At t=1.40 the correct handling is to keep accumulating sample and watch which way t moves. Not to size up, and not to switch off. With one condition agreed in advance: losing money for the first six months is expected, and is not grounds for changing the spec.

The full per-item audit, the year-by-year breakdown and three figures that describe what holding it feels like are in My backtest said 13.25×, the audit left 2.26×.

FAQ

Are these beginner mistakes? The third one is. The first two are not. What they share is that the backtester had no reason to distrust the data: the row’s format, date and fields were all valid. Catching them requires assuming the data will lie to you.

Why did all three point the same way? Because a bug in the other direction gets chased the same day. In this audit, on 2026-08-18, all three flattered the result, and that is not chance. An error that makes the backtest look worse gets investigated; an error that makes it look better gets called a result.

Should you re-tune parameters after an audit? No. The point of the audit is knowing what the original number got wrong, not finding a configuration that restores 13.25×. If re-tuning is the first thing that happens afterwards, the audit did not happen.

What does the same disease look like one layer up? Five build gates broke on 2026-08-27 with none of them reporting anything; source and fixes are in Your data pipeline passes its own tests. Real money and the daily equity series are on the performance page.