Practitioner course in algorithmic trading
Backtesting: Methods, Pitfalls and Validation
Why most backtests lie and how to build one that does not: the biases that inflate results, realistic execution, walk-forward and cross-validation designs, and statistics that account for how many ideas you tried.
Backtesting is the cheapest way to reject a trading idea and the easiest way to fall in love with a bad one. Traders on public forums say it bluntly: most backtests they have seen were useless. This course is about the methods that make a backtest worth reading.
What are the most common backtesting mistakes?
Using information that was not available at the time, such as revised data or a closing price to trade at the close. Testing only on companies that still exist today. Assuming orders fill at prices that would never have been offered. And trying hundreds of variations, then reporting the best as if it were the only one. Each has a fix, and the course works through all of them.
How do you know if a backtest is overfit?
There is no single test, but there are strong signs: performance that collapses out of sample, parameters that only work in a narrow range, results that depend on a few trades. The course adds a statistical answer, adjusting the Sharpe ratio for the number of trials, so you can say how likely a result is to be luck.
Who is this course for?
Anyone who tests trading strategies, from individuals to research teams. It counts toward the Certificate in Algorithmic Trading, the Certificate in Statistical Arbitrage and the Professional Diploma in Algorithmic Trading.
What you will be able to do
- Find and remove look-ahead, survivorship and selection bias in a backtest.
- Model fills, costs and slippage appropriate to the strategy's horizon.
- Design walk-forward and purged cross-validation tests.
- Adjust performance statistics for the number of strategies tried.
- Write a validation report that a reviewer can check.
Syllabus, module by module
Why backtests mislead
The gap between published backtests and live results, and its main causes.
Data biases
Look-ahead from revised and timestamped data, survivorship, and point-in-time universes.
Execution realism
Fill assumptions on bars, limit-order fills, slippage models and market impact.
Overfitting
Parameter search, degrees of freedom, and why a smooth equity curve is a warning.
Validation designs
Holdout sets, walk-forward analysis, and purged and embargoed cross-validation.
Multiple testing
The probability of backtest overfitting, deflated Sharpe ratios and honest reporting of trials.
Robustness
Parameter stability, subperiods, other markets and simulated data.
Project lab
Supervised work on the assessed backtest audit.
How the certificate is earned
An assessed project: you receive a flawed backtest, find and correct its biases, re-run it under a walk-forward design, and submit the corrected code with a validation report. On a pass you receive the Course certificate in Backtesting Methods, issued by the Institute with a verification code. Read how certification works.
Before you start
Working Python with pandas, statistics to regression level, and a first backtest of your own.
This course counts toward
- Certificate in Algorithmic TradingSix courses and a capstone that take a strategy from idea to a costed, sized and validated system, with performance reported the way a trading desk would read it.
- Certificate in Statistical ArbitrageFive courses and a capstone on market-neutral trading: time series foundations, pairs and baskets, cointegration and the validation that keeps spread strategies honest.
- Professional Diploma in Algorithmic TradingThe top credential on the practitioner path. Eight courses and a 40-hour capstone covering everything between a trading idea and a book you could defend to a risk committee.
Questions applicants ask
Is backtesting reliable?
A careful backtest is a useful filter that rejects bad ideas. It cannot prove a strategy will work, and a careless one actively misleads. The course is about the difference.
What is walk-forward analysis?
A test in which the strategy is fitted on one period, traded on the next, then refitted and moved forward, so every result comes from data the strategy had not seen when its parameters were chosen.
Which software does the course use?
Python with pandas and a small engine written in the course. The methods apply to any backtesting tool.