Module 13: Trading Systems and Automation
Backtesting Basics
Chapter 3 of 12 · about 5 minutes
3.1 Testing before risking
Mint had written precise rules for a trend-following system. "So now I trade it?"
"Now you test it," said Hash. "If you apply your rules to years of past data, you can see how they would have behaved: how often they won, how big the losses got, how long the bad periods lasted. It won't tell you the future. But it can tell you if an idea was never any good, without losing a cent."
3.2 What a backtest is
A backtest applies a strategy's rules to historical data to simulate how it would have performed. It's the systematic version of the pattern testing Node did by hand in Module 4, Chapter 4.
| Step | What you do |
|---|---|
| 1. Get data | Historical prices, and volume or funding if your rules need them |
| 2. Code or apply rules | Step through the data one period at a time, applying entries, exits, and sizing |
| 3. Include costs | Fees, slippage, and funding (Module 7, Chapter 11) |
| 4. Record every trade | Entry, exit, result, and account value over time |
| 5. Calculate metrics | Return, drawdown, win rate, expectancy, and more |
| 6. Compare with a benchmark | Usually simply holding BTC (Module 11, Chapter 12) |
Backtests can be done with spreadsheets, charting platforms with built-in strategy testers, or programming languages like Python.
3.3 Data matters
| Data issue | Why it matters |
|---|---|
| Source | Prices differ across exchanges (Module 1, Chapter 6); use a reliable source, ideally the market you'll trade |
| Length | Should cover bull markets, bear markets, and ranges |
| Granularity | Daily data for daily strategies; finer data for intraday strategies |
| Gaps and errors | Bad data points can create fake trades |
| Survivorship | Testing only tokens that still exist ignores those that died (Chapter 4) |
3.4 Key metrics
| Metric | What it tells you |
|---|---|
| Total return and annual return | How much the strategy made |
| Maximum drawdown | The worst peak-to-low fall (Module 12, Chapter 6) |
| Number of trades | Whether there's enough data to trust the results |
| Win rate | How often trades won |
| Average win and loss in R | Size of wins and losses relative to risk |
| Expectancy | Average result per trade (Module 4, Chapter 16) |
| Sharpe ratio | Return per unit of volatility; higher means smoother returns for the return earned |
| Time in market | Share of time the strategy held a position |
| Longest losing streak and longest drawdown period | How long the pain lasted |
A Sharpe ratio compares a strategy's return, above a risk-free rate, with how much its returns fluctuate. It's useful for comparing strategies with different volatility.
3.5 Mint's illustrative backtest
Here are illustrative results for Mint's daily BTC trend system from Chapter 2, over a hypothetical six-year period including two bull markets and one deep bear market, compared with simply holding BTC.
| Metric | Trend system (illustrative) | Buy and hold BTC (illustrative) |
|---|---|---|
| Annual return | 38% | 45% |
| Maximum drawdown | 10% | 77% |
| Number of trades | 48 | 1 |
| Win rate | 35% | |
| Average win | 4.2R | |
| Average loss | 1.0R | |
| Expectancy | ~+0.82R per trade | |
| Time in market | 52% | 100% |
| Longest losing streak | 7 trades |
Reading these results:
- The system earned less than buy and hold, but with a far smaller maximum drawdown, roughly one-seventh of buy and hold's.
- It lost on most trades, but the winners were much larger than the losers, the classic trend-following profile.
- It was out of the market almost half the time, which avoided much of the bear market.
- A seven-trade losing streak happened, which matters for psychology (Module 12, Chapter 6).
"So it's not better in every way," said Mint. "It's a trade-off: lower return, much smaller crashes."
"Exactly," said Hash. "For many people, avoiding a 77% drawdown is worth giving up some return. Especially if a 77% drawdown would make them panic sell."
3.6 Is it enough data?
Forty-eight trades is a modest sample. With small samples, results can be heavily influenced by a few lucky or unlucky trades.
| Number of trades | Confidence in results |
|---|---|
| Under 30 | Low; results could easily be luck |
| 30 to 100 | Moderate; look for consistency across periods |
| Over 100 | Higher, but still check different market conditions |
Long-timeframe strategies naturally produce fewer trades. One way to gain confidence is to test the same rules on other markets, like ETH, or other periods, and see whether the results are broadly similar.
3.7 Equity curves
An equity curve charts the account value over time in the backtest. Look at its shape, not just the end result:
| Equity curve pattern | What it suggests |
|---|---|
| Steady rise with modest dips | Consistent strategy |
| Most gains from one short period | Results depend on one event; fragile |
| Long flat or falling periods | Strategy may be hard to stick with |
| Very smooth, almost perfect line | Suspicious; check for errors or overfitting (Chapter 4) |
3.8 Try it yourself
Use a charting platform's strategy tester, or a spreadsheet, to backtest the simple rules from Chapter 2 on BTC daily data for as many years as available. Include 0.1% costs per trade. Record the metrics from section 3.4 and compare with buy and hold. Then test the same rules on ETH. Are the results similar?
Key takeaways from this chapter
- A backtest applies strategy rules to historical data to see how they would have performed.
- Steps include getting data, applying rules, including costs, recording trades, calculating metrics, and comparing with a benchmark.
- Good data covers different market conditions and comes from reliable sources.
- Key metrics include return, maximum drawdown, trade count, win rate, R-multiples, expectancy, Sharpe ratio, time in market, and losing streaks.
- Strategies involve trade-offs, such as lower return in exchange for smaller drawdowns.
- Small samples are unreliable; test across markets and periods.
- Look at the equity curve's shape, and be suspicious of results that look too perfect.