Look-Ahead Bias In Backtests And How To Detect It
Look-ahead bias in backtests is the result of using information that would not normally be available for the execution of a signal when it occurs. The information can be about price, past or future, or fundamental data. This is just one form of bias that can be found in backtest results but possibly the worst. There are other forms of bias that can affect results.
-Selection bias is due to the selection of a favorable result after many trials.
-Data-snooping bias is due to the reuse of data to optimize strategies.
-Overfitting bias due to optimizing strategy parameters.
While the three biases listed above as usually referred to as data-mining bias, look-ahead bias arises from errors in code, faulty data, or even from the lack of understanding of how trading strategies are used in the real world.
There is a false perception that look-ahead bias is rare. A look-ahead bias that went unnoticed for several years was detected in 2016 to be present in peer-reviewed publications about price series momentum, and even in one book. It is beyond the scope of the article to discuss the details.
Look-ahead is the worst type of bias because the results are wrong, as opposed to the other form of biases listed above. The reason is that look-ahead bias is immediately revealed in real-world execution, whereas the other form of biases are revealed after a regime change where the strategy becomes unprofitable. There will be a mismatch of trade entry and exit levels between real application and backtests when look-ahead bias is present; in the real world, a trader cannot execute a trade at past price levels or use future fundamental data.
I was surprised in 2016 to find out a hedge fund was advertising an index momentum strategy with exceptional alpha. I immediately thought something was wrong and after running a few backtests I identified the look-ahead bias. Six years ago people started using Python and R extensively for backtests and there were bugs in code that went unnoticed for a while. I wrote an article during that time to raise awareness about this type of bias.
How to detect look-ahead bias
If the equity curve of a backtest looks too good to be true, it is highly possible there is some form of bias, either over-fitting, selection bias, data-snooping bias, or even look-ahead bias. A very smooth straight line equity curve in a log chart may be problematic. Here is an example:
This particular equity curve was generated by a price series momentum strategy applied to SPY ETF, with a look-ahead bias. The entry was at the previous day’s close, something that is impossible in real life. The equity curve is very smooth, the annualized return is nearly 22% unleveraged and Sharpe is 1.81.
Below are some warning signs of look-ahead bias, or excessive bias in general:
- A very smooth equity curve, a straight line in a log chart.
- A very high annualized return, usually greater than 12%.
- A Sharpe ratio higher than 1.5.
- A MAR ratio greater than 1.
The above criteria are warning signs and do not imply every backtest that meets them is the result of bias. They tell you more work must be done to rule out bias. For example, running the strategy in forward mode and checking to see if the generated entry and exit levels by the backtest correspond to real-world entry and exit levels. Looking at the code is the best way but sometimes it is either not practical or the code is not available.
Below is what a realistic strategy equity curve looks like (backtest).
Real-world equity curves are volatile and the time to recover from drawdown is usually long in relative terms. For EOD strategies it may be weeks or months, for weekly strategies it may take months. Real trading is volatile. Smooth equity curves in most cases exist only in backtests.
The equity of the above strategy peaked in early January and then fell, remained slightly volatile, and has recently recovered. The strategy was lucky to be long AAPL and short INTC last week, gaining 2.5% from open to close of the week.
Strategies are often lucky, but also unlucky. This strategy was unlucky earlier this year with a long position in WMT before a crash due to an earnings report. The equity curve of unbiased strategies is volatile. A very smooth equity curve should serve as a warning sign and requires some serious work to decide whether bias is involved. The same holds for unrealistically high unleveraged annualized return and Sharpe ratio results.
