Quantopian Shutdown Offers Some Important Lessons
--
Quantopian will be remembered as an amazing effort by a small group of motivated and highly capable platform developers to crowd-source alpha in the quantitative trading space. A lot of hard work went into developing a platform where thousands of quant traders could code and test strategies hoping that they will receive an allocation and eventually profit. It did not work and in this article, I present the reasons that caused this unexpected failure in my opinion.
The failure of Quantopian left me saddened not because I have used the free service and the software I have developed to search for anomalous behavior in market price action generates code also for that platform. The failure was bad for quantitative trading in general because people do not understand the crucial details but see it as a general problem.
In my opinion, some of the reasons included in some articles about the failure of Quantopian are valid but are not primary but secondary. For example, data-mining bias due to repeated trials and data snooping was not only a problem of Quantopian; it is a problem of every quant strategy developer. After a certain point, the difference between having 100,000 users each testing their hypotheses or letting a genetic programming algo pick a strategy after testing one billion combinations of alternatives becomes immaterial. This is not the primary reason Quantopian failed. This is the reason any quant can fail in general.
Neither was lack of experience a problem that was particular to Quantopian; almost every hedge fund has this problem because many do not understand that experience and context are necessary for creating winning trading strategies and solely using Python libraries does not solve that problem.
In my opinion, there were two main causes of the failure that every quant trader should understand and avoid:
- Unrealistic goals and expectations coupled with restrictions
Realistic goals and expectations are the results of experience and proper analysis. Traders should understand what the market can offer in terms of returns and how. For example, using a market-neutral equity long/strategy as a hedge to provide some convexity to a portfolio may be useful, if the strategy works well of course, but that cannot be a source of the alpha because you are not going to get any alpha from such strategy nearly 90% of the time.
Quantopian figured out the above late in the game. Due to their process for testing market neutrality of strategies and strict requirements, Type-II error (false rejections) was high and they ignored many strategies that although not market neutral could have generated good returns.
2. Trusting the wrong people
We can all make the mistake of trusting the wrong people. Quantopian was a dream platform for many aspiring quants but also the target of some gamblers who had nothing to lose anyway by offering their skillfully over-fitted algos that passed the “AlphaLens” scrutiny.
I worked with Quantopian developers over about a month to generate features using our DLPAL LS software for S&P 500 stocks, which they would then analyze further. They were very competent people with exceptionally polite manners. Due to the computational complexity of the task, I initially generated features for the 500 stocks for about a year that they afterward tested and decided that I should go ahead and generate a long history since 2007. I had a few machines running for several days to complete the task only to find out after it was done that they changed the required format for the features. I recommended to them to take the files with the old format I had just generated and write a script to reformat them. They never replied as it was announced they had already offered allocations to a few people and they were probably busy with development.
There were people who from forum discussions it was evident had next to zero market experience although they knew well how to program in Python. Python and trading as far from synonymous. One of the most successful traders I knew about 15 years ago did not even look at electronic quotes and had no account with direct order access but a full-service broker. He would look at the open, high, low, and close of the day on websites before the market open and then call the broker to place orders. That person probably made more money than several quant funds combined. Why? Simply because the person was very experienced and his focus was on identifying winners, not on avoiding software bugs.
Many quant funds have performed worse than the average random trader in the last two years. Below is a simulation of 10,000 random traders that used a fair coin to go long SPY ETF if heads came up before the close and then reverse the position to short when tails came up. The commission is set to $0.01 per share and equity is fully invested. The simulation period is 01/2019–12/2020.
Hypothetical or simulated performance results have certain inherent limitations. Unlike an actual performance record, simulated performance results do not represent actual trading. Also, since the trades have not been executed, the results may have under-or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity. Hypothetical trading results are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown.
It may be seen, that 42.7% of the random traders made a profit in the last two years (01/2019 to 12/2020) and even 7.5% made more than the 55.2% buy and hold return. Mean return is about -0.5% but standard deviation is very high at 36.2%. There are some quite unlucky but also very lucky random traders. The distribution is leptokurtic with 2.14 excess Kurtosis and the skew is positive at 1.1, meaning that randomly trading SPY ETF may be more rewarding than going to a casino. Yet, some quantitative funds did not make any money and even were closed down. Why?
Trusting the wrong people can be detrimental to success in every respect and in any area. Usually good people show humility and low profile but wrong people are aggressive, they exaggerate their qualifications and even use Public Relation firms to boost their status. Choosing the right people can make the difference between being at the left tail or right tail of the above distribution. Unfortunately, most companies tend to choose the aggressive and pompous but not all.
