Quantopian Shutdown Offers Some Important Lessons

Quantopian will be remembered as an amazing effort by a small group of motivated and highly capable platform developers to crowd-source alpha in the quantitative trading space. A lot of hard work went into developing a platform where thousands of quant traders could code and test strategies hoping that they will receive an allocation and eventually profit. It did not work and in this article I present the reasons that caused this unexpected failure in my opinion.

The failure of Quantopian left me saddened not because I have used the free service and the software I have developed to search for anomalous behavior in market price action generates code also for that platform. The failure was bad for quantitative trading in general because people do not understand the crucial details but see it as a general problem.

In my opinion some of the reasons included in some articles about the failure of Quantopian are valid but are not primary but secondary. For example, data-mining bias due to repeated trials and data snooping was not only a problem of Quantopian; it is a problem of every quant strategy developer. After a certain point, the difference between having 100,000 users each testing their hypotheses or letting a genetic programming algo pick a strategy after testing one billion combinations of alternatives becomes immaterial. This is not the primary reason Quantopian failed. This is the reason any quant can fail in general.

Neither was lack of experience a problem that was particular to Quantopian; almost every hedge funds has this problem because many do not understand that experience and context are necessary in creating winning trading strategies and solely using Python libraries does not solve that problem.

In my opinion there were two main causes of the failure that every quant trader should udnerstand and avoid:

  1. Unrealistic goals and expectations coupled with restrictions

Realistic goals and expectations are the results of experience and proper analysis. Traders should understand what the market can offer in terms of returns and how. For example, using a market neutral equity long/strategy as a hedge to provide some convexity to a portfolio may be useful, if the strategy works well of course, but that cannot be a source of alpha because you are not going to get any alpha from such strategy nearly 90% of the time.

Quantopian figured out the above late in the game. Due to their process for testing market neutrality of strategies and strict requirements, Type-II error (false rejections) was high and they ignored many strategies that although not market neutral could have generated good returns.

In the course of the last two years (2019–2020), our weekly long/short strategy for Dow 30 stocks has generated 20.4% annualized return with only 7.5% maximum drawdown (Disclaimer: backtested results.)

CFTC RULE 4.41
Hypothetical or simulated performance results have certain inherent limitations. Unlike an actual performance record, simulated performance results do not represent actual trading. Also, since the trades have not actually been executed, the results may have under- or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity. Hypothetical trading results are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown.

Quantopian would not have used the above strategy because it was not market neutral since it adjusts exposure to capitalize on short-term direction. However, the strategy had 72 long and 84 short trades in the last two years. The main issue is that we would never reveal the way the strategy operates to anyone, included Quantopian. Our DLPAL LS customers can generate the strategy signals but the algorithm that achieves that is proprietary. In order to implement this in Quantopian we would have to reveal the algorithm and we would never do that.

2. Trusting the wrong people

We can all make the mistake of trusting the wrong people. Quantopian was a dream platform for many aspiring quants but also the target of some gamblers who had nothing to lose anyway by offering their skillfully over-fitted algos that passed the “AlphaLens” scrutiny.

I worked with Quantopian developers over a period of about a month to generate features using our DLPAL LS software for S&P 500 stocks, which they would then analyze further. They were very competent people with exceptionally polite manners. Due to the computational complexity of the task, I initially generated features for the 500 stocks for a period of about a year that they afterwards tested and decided that I should go ahead and generate a longer history since 2007. I had a few machines running for several days to complete the task only to find out after it was done that they changed the required format for the features. I recommended to them to take the files with the old format I had just generated and write a script to reformat them. They never replied as it was announced they had already offered allocations to a few people and they were probably busy with development.

There were people that from forum discussions it was evident they had next to zero market experience although they knew well how to program in Python. Python and trading as far from synonymous. One of the most successful traders I knew about 15 years ago did not even look at electronic quotes and had no account with direct order access but a full service broker. He would look at the open, high, low and close of the day in websites before the market open and then call the broker to place orders. That person probably made more money than several quant funds combined. Why? Simply because the person was very experienced and his focus was in identifying winners, not on avoiding software bugs.

If fact, many quant funds have performed worse that the average random trader in the last two years. Below is a simulation of 10,000 random traders that used a fair coin to go long SPY ETF if heads came up before the close and then reverse the position to short when tails came up. Commission is set to $0.01 per share and equity is fully invested. Simulation period is 01/2019–12/2020.

CFTC RULE 4.41
Hypothetical or simulated performance results have certain inherent limitations. Unlike an actual performance record, simulated performance results do not represent actual trading. Also, since the trades have not actually been executed, the results may have under- or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity. Hypothetical trading results are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown.

It may be seen, that 42.7% of the random traders made a profit in the last two years (01/2019 to 12/2020) and even 7.5% made more than the 55.2% buy and hold return. Mean return is about -0.5% but standard deviation is very high at 36.2%. There are some quite unlucky but also very lucky random traders. The distribution is leptokurtic with 2.14 excess Kurtosis and the skew is positive at 1.1, meaning that randomly trading SPY ETF may be more rewarding than going to a casino. Yet, some quantitative funds did not make any money and even were closed down. Why?

Trusting the wrong people can be detrimental to success in every respect and in any area. Usually good people show humility and low profile but wrong people are aggressive, they exaggerate their qualifications and even use Public Relation firms to boost their status. Choosing the right people can make the difference between being at the left tail or right tail of the above distribution. Unfortunately, most companies tend to choose the aggressive and pompous but not all.

About the author

Quant trader, blogger, trading book author and developer of DLPAL machine learning software. No investment advice. #trading #finance Website