Execution operations

ExpertDuration 40 minTools Python 3, MetaTrader5 package, An MT5 demo account, A logger (Python logging)

Why this lesson

This is the part no book covers well. Free material is rich on mechanics, backtesting, even overfitting theory, and thin-to-absent on live execution operations for retail MT5 — reconciliation, idempotency, disconnects, broker-side stops. The closest published treatments (Chan chapter 5, Jansen Part VI) are written for IBKR and Alpaca REST APIs and have to be translated to a terminal-IPC world. Practitioner post-mortems keep ranking “plumbing” — orphaned positions after restarts, no broker-side stops, residential internet — as an account-ending failure mode, unglamorous and quietly lethal. The demo code from Lesson 3.1 was happy-path only. This lesson adds the missing 80%.

Watch

There is no honest free video that teaches this for retail MT5 — a known gap, so the lesson leads with the explainer. For the professional framing of why operational and infrastructure risk sits alongside market risk, this segment earns its time:

Thumbnail for Risk Management with Robert Carver — five types of risk — **Watch for:** Carver naming infrastructure risk and counterparty risk as first-class risks — and the disasters (Knight Capital's algo bug) that came from execution plumbing, not bad strategy.

The explainer

Four disciplines separate a bot that survives its own crashes from one that quietly detonates.

1. State reconciliation on restart. Your process will restart — a VPS reboot, a crash, a deploy. The dangerous assumption is that your in-memory idea of “what I have open” survives the restart. It does not. On every startup, before doing anything else, you must reconcile: read the truth from the broker with positions_get() and orders_get(), load your own last-known intended state from disk, and diff them. Three cases fall out. A position the broker has and you intended — fine, adopt it. A position you intended but the broker does not have — it closed while you were down (stopped out, perhaps); update your book. A position the broker has that you did not intend, or cannot identify — an orphaned position, the classic bot account-killer. Orphans happen when an order fills during the exact window your process is dead, so no code ever recorded it. Reconciliation’s whole job is to make orphans loud instead of silent.

2. Identify your own orders. Reconciliation is only possible if you can recognise your trades among everything on the account. That is what the magic number (a strategy tag) and the order comment are for — you saw magic=424242 in Lesson 3.1. Give each strategy a distinct magic number and each intended order a client order id you generate and store before you send. Now the broker’s fills carry a fingerprint you can match against your own log.

3. Idempotent order submission. Networks time out mid-request. You send order_send, the connection drops before the reply arrives, and you genuinely do not know whether the order reached the broker. The naive reaction — resend — is how you end up with two positions where you wanted one. Idempotency means a retry of the same intended order cannot create a duplicate. The pattern: generate the client order id first, write “intended: id X, not yet confirmed” to disk, then send. On any retry, first check the broker (by magic plus comment, or by your stored ticket) for an order already bearing that id. If it is there, adopt it; only send if it is genuinely absent. A dedup guard around every submission is not optional in live trading.

4. Broker-side stops, always. A stop-loss that lives in your Python loop is not a stop — it is a hope that your process, your VPS, and your internet are all alive at the moment price hits it. Attach the stop at the broker, on the order itself (the sl field in the order_send request), so it survives your process dying. A software stop dies with the process; a broker-side stop does not. This is the operational twin of the Level 1 lesson that a software stop is not a stop — now it is a line of code you either wrote or did not.

5. Disconnect behaviour, decided in advance. When the terminal loses the broker, what should your bot do? The wrong answer is “keep sending orders into the void.” Decide the connectivity failsafe before you go live: detect the disconnect (via mt5.terminal_info() and last_error()), stop originating new entries, rely on the broker-side stops you already placed to protect open risk, and reconcile fully on reconnect rather than trusting stale in-memory state. Knight Capital lost around $440M in forty-five minutes because deployment and state assumptions were wrong, not because their strategy was — proof that this is where the money actually dies.

Put together: reconcile from the broker’s truth on every start, tag and pre-log every order so retries are idempotent, keep the real stop broker-side, and script the disconnect. None of it improves your edge. All of it decides whether you keep the edge you have.

Do it

Write the reconciliation skeleton. It diffs your intended book against the broker and logs orphans loudly. Run it against a demo account (open a manual trade in the terminal first to see an “orphan” light up).

import MetaTrader5 as mt5
import json, logging, pathlib

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
log = logging.getLogger("reconcile")

MAGIC = 424242
STATE_FILE = pathlib.Path("intended_state.json")


def load_intended() -> dict:
    """Our own last-known book: {client_id: {"symbol":..., "volume":..., "ticket":...}}."""
    if STATE_FILE.exists():
        return json.loads(STATE_FILE.read_text())
    return {}


def broker_positions() -> dict:
    """Positions the broker actually holds for THIS strategy, keyed by ticket."""
    out = {}
    for p in (mt5.positions_get() or []):
        if p.magic == MAGIC:            # only our own trades
            out[p.ticket] = {"symbol": p.symbol, "volume": p.volume, "sl": p.sl}
    return out


def reconcile() -> dict:
    """Diff intended vs broker state. Returns an action report; never trades automatically."""
    if not mt5.initialize():
        raise SystemExit(f"initialize() failed: {mt5.last_error()}")

    intended = load_intended()
    broker = broker_positions()
    intended_tickets = {v.get("ticket") for v in intended.values() if v.get("ticket")}

    report = {"adopt": [], "closed_while_down": [], "orphans": [], "missing_stop": []}

    # Positions the broker has but we never intended -> ORPHANS (the account-killer).
    for ticket, pos in broker.items():
        if ticket not in intended_tickets:
            report["orphans"].append(ticket)
            log.error("ORPHAN: broker holds ticket %s (%s %s) we did not intend - HALT and inspect",
                      ticket, pos["symbol"], pos["volume"])
        elif pos["sl"] == 0.0:
            report["missing_stop"].append(ticket)
            log.warning("ticket %s has NO broker-side stop - attach one", ticket)
        else:
            report["adopt"].append(ticket)

    # Positions we intended but the broker no longer has -> closed while we were down.
    for cid, meta in intended.items():
        if meta.get("ticket") and meta["ticket"] not in broker:
            report["closed_while_down"].append(meta["ticket"])
            log.info("ticket %s intended but gone at broker - closed while offline", meta["ticket"])

    log.info("reconcile summary: %s", {k: len(v) for k, v in report.items()})
    mt5.shutdown()
    return report


if __name__ == "__main__":
    reconcile()

The rule the code enforces: an orphan halts the bot for a human, it does not auto-trade. Reconciliation reports; it never guesses.

Terms introduced

Check

You can move on when you can… explain why orphaned positions are the classic bot account-killer, write a reconcile() that diffs your intended book against positions_get() and flags orphans, make order submission idempotent with a client order id and a dedup guard, and state why the real stop always lives broker-side.

Go deeper

Ernest Chan, Quantitative Trading, chapter 5 (Execution Systems) — the cleanest short book treatment; mentally translate its IBKR framing to terminal IPC.
Stefan Jansen, ML for Trading, Part VI — live trading systems and MLOps; written for Alpaca/IBKR, so read it as patterns, not copy-paste.
NautilusTrader docs — a production-grade engine that builds reconciliation and idempotency in; a reference design for when you outgrow scripts.
SEC order on Knight Capital — the canonical case study in how deployment and state plumbing, not strategy, ends firms.