Quantopian Shutdown Offers Some Important Lessons
Quantopian will be remembered as an amazing effort by a small group of motivated and highly capable platform developers to crowd-source alpha in the quantitative trading space. A lot of hard work went into developing a platform where thousands of quant traders could code and test strategies, hoping that they would receive an allocation and eventually profit. It did not work, and in this article, I present the reasons that caused this unexpected failure, in my opinion.
The failure of Quantopian left me saddened, not only because I have used the free service but also because of the software I have developed to search for anomalous behavior in price action generated code for that platform. The failure was bad for quantitative trading in general because people do not understand the crucial details but see it as a general problem.
In my opinion, some of the reasons included in some articles about the failure of Quantopian are valid but not primary. For example, data-mining bias due to repeated trials and data snooping was not only a problem of Quantopian; it was a problem of every quant strategy developer. I developed the first software program that identified strategies and generated code for trading platforms in the late 1990s, and I am very familiar with this issue. After a certain point, the difference between having 100,000 users each test their hypotheses or letting a genetic programming algorithm pick a strategy after testing one billion combinations of alternatives becomes immaterial. This is not the primary reason Quantopian failed. This is the reason any quant can fail in general.
Neither was lack of experience a problem that was particular to Quantopian; almost every hedge fund has this problem because many do not understand that experience and context are necessary for creating winning trading strategies, and solely using Python libraries does not solve that problem.
In my opinion, there were two main causes of the failure that every quant trader should understand and avoid:
- Unrealistic goals and expectations coupled with restrictions
Realistic goals and expectations are the results of experience and proper analysis. Traders should understand what the market can offer in terms of returns and how. For example, using a market-neutral equity long/short strategy as a hedge to provide some convexity to a portfolio may be useful if the strategy works well, of course, but that cannot be a source of alpha because you are not going to get any alpha from such a strategy nearly 90% of the time.
Quantopian figured out the above late in the game. Due to their process for testing the market neutrality of strategies and strict requirements, Type-II error (false rejections) was high, and they ignored many strategies that, although not market neutral, could have generated good returns.
2. Trusting the wrong people
We can all make the mistake of trusting the wrong people. Quantopian was a dream platform for many aspiring quants but also the target of some gamblers who had nothing to lose anyway by offering their skillfully overfitted algos that passed the “AlphaLens” scrutiny.
I worked with Quantopian developers for about a month to generate features using our DLPAL LS software for S&P 500 stocks, which they would then analyze further. They were very competent people with exceptionally polite manners. Due to the computational complexity of the task, I initially generated features for the 500 stocks for about a year that they afterward tested and decided that I should go ahead and generate a long history since 2007. I had a few machines running for several days to complete the task, only to find out after it was done that they changed the required format for the features. I recommended to them that they take the files in the old format I had just generated and write a script to reformat them. They never replied, as it was announced they had already offered allocations to a few people, and they were probably busy with development.
There were people who, from forum discussions, it was evident had next to zero market experience, although they knew well how to program in Python. Python and trading are far from synonymous. One of the most successful traders I knew about 15 years ago did not even look at electronic quotes and had no account with direct order access but only a full-service broker. He would look at the open, high, low, and close of the day on websites before the market opened and then call the broker to place orders. That person probably made more money than several quant funds combined. Why? Simply because the person was very experienced and his focus was on identifying winners, not on avoiding software bugs.
Many quant funds have performed worse than the average random trader in the last two years. Below is a simulation of 10,000 random traders that used a fair coin to go long SPY ETF if heads came up before the close and then reversed the position to short when tails came up. The commission is set at $0.01 per share, and equity is fully invested. The simulation period is 01/2019–12/2020.
Hypothetical or simulated performance results have certain inherent limitations. Unlike an actual performance record, simulated performance results do not represent actual trading. Also, since the trades have not been executed, the results may have under- or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity. Hypothetical trading results are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown.
It may be seen that 42.7% of the random traders made a profit in the last two years (January 2019 to December 2020), and even 7.5% made more than the 55.2% buy-and-hold return. The mean return is about -0.5%, but the standard deviation is very high at 36.2%. There are some quite unlucky, but also very lucky, random traders. The distribution is leptokurtic with 2.14 excess kurtosis, and the skew is positive at 1.1, meaning that randomly trading SPY ETF may be more rewarding than going to a casino. Yet, some quantitative funds did not make any money and even closed down. Why?
Trusting the wrong people can be detrimental to success in every respect and in any area. Usually, good people show humility and a low profile, but wrong people are aggressive, exaggerate their qualifications, and even use public relations firms to boost their status. Choosing the right people can make the difference between being at the left or right tail of the above distribution.
