Event-driven backtesting with honest costs

IntermediateDuration ~35 min video + 45 min hands-onTools Python 3 + pandas, backtrader

Why this lesson

You can now specify a strategy and reason about its statistics. Now you run it — and the single most important thing a backtest does is not tell you what you made, but tell you what you would have kept after costs. The retail graveyard is full of strategies that were real on gross numbers and dead on net. This lesson builds a strategy in an event-driven engine and then does the one thing every tutorial online skips: wires in the cost model you built in Level 1 and compares gross to net. The videos below are useful for syntax and honest about their own blind spot; the exercise is where the learning is.

Watch

The two primaries. The first is the fullest worked backtrader example available; the second explains why even “tick data” backtests are not the ground truth they appear to be. Both are cost-naive by the presenters’ own admission — treat every “the strategy worked” moment as gross.

Watch for: A complete opening-range-breakout in backtrader: pulls a year of NASDAQ-100 one-minute bars, buys on the OR-high break, exits at target / stop / end-of-day, runs across 100 symbols, then uses cerebro.optstrategy to compare 15/30/60-minute ranges. Two moments matter most. Around 16:35 he says out loud 'there's no commission and I haven't configured the slippage yet' — that is the exact gap your exercise closes. Around 55:30 he names the over-optimization risk of pushing the parameter search further. Ignore the AMD/Marriott 'winners' celebration; those are gross.

Watch for: Why a backtest can never perfectly match live even on identical data: a live engine processes one tick at a time and drops ticks that arrive mid-calculation (the 'Tick Delivery' chapter, ~1:35-5:10), while a backtester sees them all. Then the four MT5 price models — real ticks, 'every tick' (only ~4 real ticks per minute, the rest synthetically reconstructed, with the high/low order sometimes guessed wrong, ~9:12-10:01), M1 OHLC, and open-price-only. The fidelity gap generalizes to any bar-based engine including backtrader. This is where the OHLC-vs-tick and fill-model terms come from.

The explainer

There are two ways to run a backtest, and the difference is not cosmetic. A vectorized engine computes signals across the whole price series at once with array operations — fast, compact, a dozen lines. An event-driven engine steps through history one bar at a time, maintaining live order and position state: a signal creates an order, the order fills on a later bar at a modelled price, the position accrues cost. backtrader is event-driven, and that is exactly why it is the honest place to measure. Costs, partial fills, the one-bar delay between signal and execution, and the discipline of never using a bar’s close to make a decision you would have made at its open — all of these live in the order/fill lifecycle, and an event-driven engine forces you to handle them where they actually happen. A vectorized engine will happily let you multiply a signal column by a returns column and hand you a number that assumed instant, free, perfect fills.

That speed has a dark side the course names plainly: easy vectorized parameter sweeps are an overfitting accelerant. When testing one more variant costs you two seconds, you will test thousands and keep the best-looking one — which, as Lesson 2.3 showed, is very often drawn from the noise tail, not the edge tail. The vectorbt-versus-backtrader comparison even demonstrates this live, with a hyperparameter heat-map its own presenter concedes “some people might say is overfitting,” ending on a curve-fit reversal that outperforms in-sample as self-aware showmanship. Vectorized tools have a place — scanning, research, generating candidates — but the moment you want to believe a result, you move it to the event-driven engine and pay its honesty tax.

Now the fidelity ladder, because a backtest is only as honest as its worst assumption. Data quality comes in tiers: OHLC vs tick data. An OHLC bar tells you four prices per period and hides the path between them — so a backtester cannot know whether the high or the low came first, which matters enormously when both your stop and your target sit inside the same bar. Tick data records every quote, but as the Darwinex video shows, even MT5’s “every tick” mode reconstructs most ticks synthetically and can order the high and low wrong. And on a CFD platform there is a subtler issue, spread-in-data: your price feed is usually bid or mid, but you buy at the ask and sell at the bid, so the spread must be added by your fill model — the rule that decides what price an order actually gets. If your data is mid and your fills are mid, you have silently deleted the spread, which for an intraday breakout can be the entire edge.

Which brings us to the wiring that makes a backtest worth trusting. backtrader lets you set the two dominant costs directly on its broker. setcommission models your commission per round-turn (and, with a custom CommInfo, the FX contract math); set_slippage_perc or set_slippage_fixed models the gap between the price you wanted and the price you got — the natural home for half-spread and stop-order slippage. These are the same components you itemized in the Level 1 cost model: spread by session, commission per lot, and slippage that skews negative on stop orders (recall FXCM’s published stat that 57% of stop orders received negative slippage). The exercise below runs the same strategy twice — once with the broker’s default zero costs, once with your cost model attached — and the gap between the two final equity values, expressed in R, is the single most decision-relevant number a backtest can give you. If a strategy is a winner gross and a loser net, you learned that here for free instead of live for real.

Do it

You will build a small breakout strategy, run it gross, then attach costs and run it net. The centerpiece is the comparison, not the strategy.

import backtrader as bt

class RangeBreakout(bt.Strategy):
    params = dict(lookback=20, stop_atr=1.5)

    def __init__(self):
        self.hh = bt.ind.Highest(self.data.high(-1), period=self.p.lookback)
        self.atr = bt.ind.ATR(self.data, period=self.p.lookback)

    def next(self):
        if not self.position:
            if self.data.close[0] > self.hh[0]:          # break of prior N-bar high
                self.buy()
        else:
            # exit on a stop trailing the entry by stop_atr * ATR
            if self.data.close[0] < self.data.close[-1] - self.p.stop_atr * self.atr[0]:
                self.close()

def run(with_costs: bool):
    cerebro = bt.Cerebro()
    cerebro.addstrategy(RangeBreakout)

    # --- plug in your own OHLC feed here ---
    data = bt.feeds.GenericCSVData(dataname='eurusd_h1.csv', dtformat='%Y-%m-%d %H:%M:%S',
                                   timeframe=bt.TimeFrame.Minutes, compression=60)
    cerebro.adddata(data)
    cerebro.broker.setcash(10_000)
    cerebro.addsizer(bt.sizers.FixedSize, stake=10_000)   # notional units per trade

    if with_costs:
        # These are the Level-1 cost components, wired into the broker:
        # 1) commission per round-turn (fixed cash per trade here for simplicity)
        cerebro.broker.setcommission(commission=7.0,
                                     commtype=bt.CommInfoBase.COMM_FIXED,
                                     stocklike=False, mult=1.0)
        # 2) slippage as a % of price — models half-spread + stop-order slippage
        cerebro.broker.set_slippage_perc(perc=0.0002)     # ~2 pips on EUR/USD

    start = cerebro.broker.getvalue()
    cerebro.run()
    end = cerebro.broker.getvalue()
    return start, end

for label, costs in (('GROSS (no costs)', False), ('NET (cost model on)', True)):
    s, e = run(costs)
    print(f"{label:<22} {s:>10,.0f} -> {e:>10,.0f}   P&L {e - s:>+10,.0f}")

Then do the analysis that matters:

1. Read the gap. Run both and compute gross_pnl - net_pnl. Divide by your per-trade risk to express the cost drag in R. This is what the strategy pays the broker per unit of risk taken.
2. Find the flip point. Nudge the slippage up (0.0002 toward 0.0005, a wider or news-time spread) until a net-positive strategy goes negative. That slippage value is your strategy’s honesty threshold — beyond it, the edge is gone.
3. Confront the fidelity caveat. Your feed is H1 OHLC. Ask: does the exit rule ever have the stop and the target inside the same bar? If so, note that the OHLC path ambiguity from the Darwinex video means your fill could be optimistic — a known limitation to carry forward, not to paper over.
4. Contrast the engines. In one sentence, explain why you would prototype this with a vectorized sweep but only believe it after the event-driven, cost-loaded run.

Do not tune parameters to rescue a net loser. In Level 2 the honest outcome for most simple breakouts on FX net costs is “no surviving edge” — reporting that correctly is the skill, and it is exactly what the Level 2 capstone asks of you.

Terms introduced

Terms introduced in this lesson

• event-driven vs vectorized
• fill model
• OHLC vs tick data
• spread-in-data

Each term links to the glossary, where it is defined by the way you just used it.

Check

Check yourself

1. Every backtrader tutorial embedded in this lesson shares one defect the course exists to fix:

   they only test forex pairsthey use the wrong programming languagecommission and slippage are never configured, so results are gross, not netthey run too slowly to be useful

2. Why is an event-driven engine, not a vectorized one, the honest place to measure a strategy?

   it is always fasterit produces prettier chartsorder and fill state advance one bar at a time, so costs and look-ahead are handled where they actually happenvectorized engines cannot compute a Sharpe ratio

3. The lesson calls easy vectorized parameter sweeps an "overfitting accelerant" because:

   they cannot model slippage at allthey always overstate drawdownthe speed lets you test thousands of variants and keep the luckiest, mistaking noise for edgethey are mathematically incorrect

4. MetaTrader's "every tick" backtest model is a fidelity trap because:

   it is too slow to finishit only works on stocksmost of those ticks are synthetically reconstructed, and the high/low order can be guessed wrongit ignores commission

Check answers

You can move on when you can… implement a strategy in an event-driven engine, wire in the Level-1 cost model, and explain why event-driven order/fill state is where honesty lives — and why easy vectorized parameter sweeps are an overfitting accelerant.

Mark this lesson complete

Go deeper

• backtrader documentation — the commission and slippage pages are the reference for wiring costs; maintenance is frozen but the event-driven concepts are timeless.
• Part Time Larry’s backtrader series (Parts 1-3: UNkH1TQl7qo, 5VU3CJMuk0w, K8buXUxEfMc) — clean syntax scaffolding for Cerebro, data feeds, the Strategy class, sizers, and plotting. Use for mechanics only; none of them ever sets a commission.
• QuantStart on event-driven backtesting — why event-driven architecture is the honest choice, written for developers.
• The book that owns this topic: Andreas Clenow, Trading Evolved — full-code Python backtests of complete strategies for developers. Caveat taught honestly: its code is built on Zipline, now abandonware — treat the code as pedagogy, not as your toolchain. For the transaction-cost treatment specifically, Stefan Jansen’s Machine Learning for Algorithmic Trading ch.18 is the systematic reference.