Statistics for trade data

IntermediateDuration ~30 min video + 35 min hands-onTools Python 3 + numpy

Why this lesson

A backtest hands you one number — an expectancy, an equity curve, a drawdown — and your brain treats it as fact. It is a single draw from a distribution you never see. The core skill of this level is refusing the point estimate: asking how many trades stand behind this number, and how different could it have been by luck alone? Get this wrong and you will promote noise to strategy. Get it right and most “winning” backtests reveal themselves as coin flips before they cost you a cent. This lesson is also the math behind the parent project’s cycle-speed limit: the number of trades you can gather per unit of time caps how fast you can honestly learn anything.

Watch

Watch for: A worked simulation, not hand-waving. With only 250 trades, the truly best parameter set (a real 10% edge) ranks about 17th out of 100 on average and captures only ~75% of its potential edge. At 1,000 trades that rises to ~85%; reaching the conventional 90% power threshold takes roughly 2,500 trades (around 13:03). The blunt conclusion at 16:15 — you cannot reliably extract the single best parameter set from a small optimization. This is the number to remember when a mentor shows you a 40-trade backtest.

Watch for: Full Python walkthrough. A 10/30 moving-average strategy on SPY looks great alone (~97% return, Sharpe ~0.8, max drawdown 11.7%). Then 5,000 historical-bootstrap Monte Carlo paths (resampling the log returns, code around 23:25) show simulated max drawdown often reaching ~65% versus the reported 11.7%, and a Sharpe distribution sitting mostly below the original. Watch the presenter visibly walk back his own enthusiasm — that is the correct reaction to a distribution. Note his honest caveat: simple bootstrap loses autocorrelation and regime structure.

The explainer

Two facts drive everything here. First, a trade result is a random variable, so any statistic you compute from a batch of trades has a variance — it wobbles from sample to sample even if the underlying edge is fixed. Second, the thing you care about, expectancy (the average R per trade), is a mean — and the uncertainty of a mean shrinks slowly, as one over the square root of the number of trades. That square root is the villain of the whole story: to halve your uncertainty you need four times as many trades.

Make it concrete. Suppose a strategy has a genuine expectancy of +0.1R per trade with a per-trade standard deviation of about 1R (typical for a stop-and-target system). The standard error of your measured expectancy after N trades is 1 / sqrt(N). To be about 95% confident the true expectancy is above zero, you need the measured mean to sit roughly 1.65 standard errors above zero, which means 0.1 > 1.65 / sqrt(N), i.e. N > (1.65 / 0.1)^2 ≈ 270 trades. And that is just to distinguish it from zero. This is why the Darwinex simulation finds that at 250 trades a real edge is routinely beaten by pure randomness, and that reliably ranking the best parameter set takes on the order of 2,500 trades — a small edge measured across a large parameter search needs a lot of evidence. A sample size of 30-40 trades, which is what most retail backtests and every “I tested it last week” claim rest on, has a standard error so large that noise wins constantly. The confidence interval around such an expectancy comfortably straddles zero.

The second tool is Monte Carlo resampling (bootstrapping). You have one historical sequence of trades. That sequence is one path the world happened to take; the trades could have arrived in a different order, and a slightly different draw of the same edge could have produced a very different equity curve. So you resample: draw trades from your own history, with replacement, thousands of times, and rebuild the equity curve each time. Now instead of one drawdown number you have a distribution of drawdowns. The payoff is brutal and useful. In David Bush’s account, a historical curve that showed a comfortable 15% drawdown revealed a 40% drawdown at 95% confidence once resampled — the 15% was luck of ordering. In the QuantInsti example, a reported 11.7% drawdown became ~65% across 5,000 simulated paths. If you sized your account for the historical number, the resampled number is the one that would actually have blown you up. This same distribution is how you compute a rough risk of ruin: the fraction of simulated paths that hit a loss you could not survive.

One honest caveat, straight from the QuantInsti presenter: simple resampling assumes trades are independent and identically distributed, which throws away autocorrelation and regime structure — losing streaks in real markets cluster more than a naive bootstrap admits. Block bootstrapping (resampling chunks of consecutive trades) is the standard fix, and Kevin Davey’s Monte Carlo drawdown work is the practitioner reference. But even the naive version is a massive upgrade over trusting one path.

The mental shift to carry out of this lesson: stop reading point estimates. An expectancy is a claim with an error bar. A drawdown is one sample from a fan of possible drawdowns. When someone shows you a single glorious equity curve, the correct question is not “how much did it make?” but “how many trades is that, and what does the distribution look like?” Most of the time, asking is enough to end the conversation.

Do it

You will build the two tools from scratch in about thirty lines of numpy: a power check (how many trades to tell +0.1R from zero) and a Monte Carlo resampler.

import numpy as np
rng = np.random.default_rng(42)

# --- 1. How many trades to distinguish +0.1R expectancy from zero? ---
# Model each trade's R-multiple as mean +0.1, std 1.0 (a realistic stop/target system).
TRUE_EXPECTANCY, TRUE_STD = 0.10, 1.0

def power_at(n, trials=5000, alpha_z=1.645):
    """Fraction of n-trade samples whose one-sided 95% CI clears zero."""
    samples = rng.normal(TRUE_EXPECTANCY, TRUE_STD, size=(trials, n))
    means = samples.mean(axis=1)
    ses = samples.std(axis=1, ddof=1) / np.sqrt(n)
    return np.mean(means - alpha_z * ses > 0)   # lower CI bound above zero

for n in (30, 100, 270, 500, 1000):
    print(f"{n:>5} trades -> {power_at(n):.0%} of samples prove an edge > 0")

# --- 2. Monte Carlo resample a real trade sequence ---
# Pretend these are your backtest's R-multiples (swap in your own trade log).
trades = rng.normal(TRUE_EXPECTANCY, TRUE_STD, size=250)

def max_drawdown(equity):
    peak = np.maximum.accumulate(equity)
    return (peak - equity).max()          # in R units

paths = 5000
dd = np.empty(paths)
for i in range(paths):
    resampled = rng.choice(trades, size=len(trades), replace=True)
    dd[i] = max_drawdown(np.cumsum(resampled))

hist_dd = max_drawdown(np.cumsum(trades))
print(f"\nHistorical drawdown:      {hist_dd:.1f}R")
print(f"Median resampled DD:      {np.median(dd):.1f}R")
print(f"95th-percentile DD:       {np.percentile(dd, 95):.1f}R")

Then answer for yourself:

1. From part 1, at what N does the power first clear ~90%? Compare it to the ~270-trade back-of-envelope and to Darwinex’s ~2,500 (theirs is higher because they are also ranking many parameter sets, not just testing one against zero — searching costs you evidence).
2. From part 2, how much larger is the 95th-percentile drawdown than the single historical number? That gap is what you would have under-provisioned for.
3. Swap trades for a real R-multiple list from your Level 1 trade log and rerun. If you have fewer than ~200 trades, note honestly that no statistic you compute is trustworthy yet — that is the finding, not a failure.

Terms introduced

Terms introduced in this lesson

• sample size
• variance
• confidence interval
• Monte Carlo resampling
• risk of ruin

Each term links to the glossary, where it is defined by the way you just used it.

Check

Check yourself

1. Roughly how many trades does the Darwinex simulation suggest you need before an optimization reliably picks out a genuine edge (≈90% power)?

   about 30about 250about 100,000about 2,500

2. Monte Carlo resampling of a trade sequence is useful mainly because it:

   predicts tomorrow's priceturns one lucky historical equity curve into a distribution of outcomes you could have hadremoves the need for out-of-sample testingincreases the number of real trades in your sample

3. A single historical backtest showing a 15% max drawdown is dangerous because:

   it is one path; resampling the same trades can reveal a 40%+ drawdown at 95% confidencedrawdown should be measured in pips, not percent15% is always too small to be realdrawdown never matters for FX

4. Why can a 30-40 trade backtest not be trusted even when it looks profitable?

   the standard error is so large that pure randomness routinely beats a real edge at that sample sizethe broker will not allow so few tradesyou need at least 50 trades for tax reasonsfewer than 100 trades cannot be plotted

Check answers

You can move on when you can… state roughly how many trades distinguish an expectancy from zero at realistic variance, resample a trade sequence Monte Carlo-style, and read a distribution instead of a point estimate.

Mark this lesson complete

Go deeper

• QuantStart articles — the best free corpus on the statistics of backtesting; search their bootstrap and Sharpe-uncertainty pieces.
• Better System Trader ep.111, “Managing Monte Carlo” — a full episode on applying Monte Carlo drawdown analysis in practice.
• The book that owns this topic: Kevin Davey, Building Winning Algorithmic Trading Systems — Monte Carlo drawdown analysis and position sizing for retail, from a real system trader.