Module 13: Trading Systems and Automation
Monitoring Systems and Knowing When They've Stopped Working
Chapter 11 of 12 · about 4 minutes
11.1 Broken, or just unlucky?
Six months into running the trend system live, Mint had a 9% drawdown. Seven losing trades in a row, then a small win, then four more losses.
"Is the system broken?" Mint asked. "Or is this just a bad stretch?"
"That's the hardest question in systematic trading," said Hash. "Stop too early, and you abandon a good system at its worst moment, often just before it recovers. Stop too late, and you keep feeding a system whose edge has disappeared. The answer is to decide in advance what 'broken' looks like."
11.2 Why systems stop working
| Cause | Example |
|---|---|
| Regime change | A trend system built in years of strong trends meets a long sideways market (Module 4, Chapter 6) |
| Crowding | Too many traders use the same edge, so it shrinks (Module 6, Chapter 12) |
| Market structure changes | ETFs, new derivatives, or changes in liquidity alter how prices behave (Module 10, Chapter 8) |
| Costs rise | Higher fees, slippage, or funding eat the edge (Chapter 4) |
| It never really worked | The backtest was overfitted (Chapter 4) |
| Operational issues | Bugs or execution problems degrade results (Chapter 10) |
11.3 Compare live results with expectations
Before going live, record what the backtest suggests is normal, then compare.
| Metric | Expected range from backtest (illustrative) | Live after 6 months (illustrative) | Status |
|---|---|---|---|
| Win rate | 30% to 40% | 28% | Slightly low, within reason given few trades |
| Average win | ~4R | 3.8R | Normal |
| Longest losing streak | Up to ~8 trades | 7 | Normal |
| Maximum drawdown | Up to ~10% | 9% | Normal |
| Average cost per trade | ~0.15% | 0.15% | In line with the updated assumption |
Mint's results, though painful, were within the range the backtest had shown. The system was behaving the way trend systems behave during choppy markets.
"So it's not broken," said Mint. "It's doing exactly what I tested, in a market it doesn't like."
"Yes," said Hash. "Which is why you wrote down the expected range before you started."
11.4 Define your stop rules for the system
Just as each trade has a stop, each system should have conditions for pausing or retiring it.
| Condition (illustrative) | Action |
|---|---|
| Drawdown exceeds the backtest maximum by a wide margin (for example, over 13% when the backtest maximum was 10%) | Pause and review |
| Losing streak longer than anything in the backtest, by a wide margin | Pause and review |
| Costs consistently double the backtest assumption | Reassess whether the edge survives costs |
| Expectancy negative over a large sample, such as 50 or more trades | Consider retiring |
| Market structure clearly changed in a way that breaks the hypothesis | Review the hypothesis (Chapter 2) |
Pausing means stopping new trades while you investigate: check for bugs, review the hypothesis, compare with similar strategies, and look at market conditions. It doesn't automatically mean the system is dead.
11.5 Don't tinker after every loss
When a system is losing, it's tempting to tweak it: add a filter here, change a parameter there. Doing this after every bad stretch leads straight back to overfitting (Chapter 4), now based on the most recent data.
| Healthy changes | Unhealthy changes |
|---|---|
| Made during scheduled reviews | Made after a painful loss |
| Based on a clear problem, like a bug or cost increase | Based on "it would have avoided last week's losses" |
| Tested out-of-sample before going live | Applied immediately |
| Documented | Undocumented |
11.6 Diversifying across systems
Since every strategy has conditions where it struggles, some traders run several systems that behave differently.
| System | Works best in | Struggles in |
|---|---|---|
| Trend following | Trending markets | Ranges |
| Mean reversion or grid | Ranges | Strong trends |
| Carry or funding | Positive funding markets | Funding reversals |
Combining strategies with low correlation can smooth overall results, as long as each has a real edge and total risk stays within your limits (Module 12, Chapter 5). It also adds complexity, so start with one system and add others only once the first is stable.
11.7 Try it yourself
For the system you've been developing, write down the expected range for win rate, average win and loss, longest losing streak, maximum drawdown, and costs, based on your backtest. Then define specific conditions for pausing and retiring the system. Keep this alongside your trading journal.
Key takeaways from this chapter
- Systems stop working because of regime change, crowding, structural shifts, rising costs, overfitting, or operational issues.
- Record expected ranges from your backtest before going live, and compare live results against them.
- Painful periods within the expected range are normal and don't mean a system is broken.
- Define in advance the conditions for pausing and retiring a system.
- Avoid tinkering after losses; make changes only in scheduled reviews, with out-of-sample testing.
- Running several low-correlation systems can smooth results, but start with one.