Four mistakes distort most backtests: too few trades, overfitting, mixed P&L units, and annualizing R-multiples. Quantprove discounts small samples with a sqrt(n)/sqrt(500) confidence multiplier (a 20-trade log hits the 0.2 floor), surfaces overfitting through live retention and a KS/Anderson-Darling test, rejects mixed-unit pairs, and never scales R by sqrt(252).
A short trade log can post a high win rate and a clean equity curve by luck alone. With 20 trades, one or two outliers move the mean enough to fake an edge that vanishes over the next 50. Bailey and López de Prado document this directly: testing many strategy variants on short histories inflates the apparent Sharpe ratio, and their Deflated Sharpe Ratio adjusts for sample length and the number of trials.
Quantprove applies a confidence multiplier of max(0.2, min(1.0, sqrt(n)/sqrt(500))) to every Backtest raw score. A 20-trade log lands on the 0.2 floor, so a raw 80 reports as a 16 Edge Score. The multiplier reaches 1.0 only near 500 trades. The number tells you how much weight the sample earns, before you read the tiers on the Edge Score glossary page.
A 20-trade backtest hits the 0.2 confidence floor: a raw 80 reports as a 16 Edge Score.
Overfitting happens when you tune entries, exits, and filters until the backtest curve looks perfect on the data you fit it to. The Edge Score reads high. Live, the same rules meet trades they never saw, and the edge collapses.
Validation pairs a backtest log against a live log and scores how closely they match. EV Retention and Sortino Retention divide live performance by backtest performance, so a 0.50R backtest EV that drops to 0.10R live reports as 20% retention. A two-sample Anderson-Darling test (Anderson and Darling, 1954, which replaced the earlier KS check) then verifies whether the live return distribution plausibly came from the same process as the backtest. A wide gap is the overfitting signature. See the Stability Score glossary entry for how the buckets combine.
Quantprove reads one unit per log. The smart parser flags R-multiples when more than 10% of values equal exactly -1.0, or when the mean sits near 0 with a standard deviation of 1 to 2; high-variance values read as dollar P&L. A backtest in R and a live log in dollars are not on the same scale, so Validation rejects a mixed-unit pair rather than score a meaningless comparison.
Changing account size or adding deposits mid-stream distorts a dollar log the same way: a $200 trade at a $10k balance and a $200 trade at a $50k balance carry different risk, but the raw numbers look identical. Keep one unit across both logs. If account size changed, rescale to R so each trade reports as a multiple of risk, not an absolute amount.
R-multiples measure profit as a function of risk per trade, not a time-series percentage return. Multiplying them by sqrt(252), the convention for daily percentage returns, produces numbers with no meaning. The same error turns a strategy’s total result into a headline like "36R per year" that no underlying math supports.
Quantprove uses per-trade statistics throughout and never applies sqrt(252) to R data. The Sortino and EV figures behind the Edge Score are computed per trade. For aggregate results, read Total R or EV per trade. The metrics and tools guide covers the forward projection, which also runs on per-trade metrics rather than annualizing a return figure. If a yearly R figure shows up in a sales page, walk.
R-multiples scale with risk, not time. There is no valid sqrt(252) annualization for them; use Total R or EV per trade.
Run the trade log through Backtest first. If the Edge Score sits well below its raw value, the confidence multiplier is telling you the sample is thin; collect more trades before trusting the tiers. When you have live results, run Validation to read EV Retention and the distribution-test verdict against the backtest.
Before either run, confirm the unit. A log that mixes R and dollars, or spans a mid-stream account-size change, gives both modes a distorted baseline. The Best Practices guides cover sample size, validation, and unit handling in more depth, and the glossary defines each metric the modes report.