In recent years, the Securities and Exchange Commission has focused on using quantitative analysis to identify statistical outliers and anomalies through programs like the Aberrational Performance Inquiry, which evaluates hedge fund returns,[1] and the Accounting Quality Model (informally known as “RoboCop”), which scours public company filings to estimate “peer-level risk metrics.”[2]  Using enforcement actions involving the allocation of securities as an example, we explore issues raised by the use of statistics in SEC enforcement actions and inquiries.

Recent trade allocation actions by the SEC

In the last few years, the SEC has announced multiple enforcement actions against investment advisers who allegedly engaged in self-serving and illegal allocations of trades to their personal accounts rather than their clients’ accounts.  This practice is known as “cherry picking.”  In most of these actions, the SEC presented evidence of higher returns in personal accounts and often took the extra step of purportedly proving, through statistical tests, that the different returns in those two types of accounts could not have happened by chance.

For years, the SEC has warned investment advisers about practices that could give rise to suspicions of improper trade allocation.  For example, in a 2008 Compliance Alert, SEC staff outlined best compliance practices with respect to trading in personal accounts and proprietary accounts.  Specifically, the SEC suggested determining trade allocations prior to or soon after trades are made, and, in the event that the allocation of a trade comes after execution, such allocations should be documented and reviewed by an appropriate individual.

However, not all post-execution trade allocations are improper.  In a 1995 pronouncement, the SEC’s Division of Investment Management did not object to post-execution trade allocation by an investment advisor who provided advice to a number of different types of investors, all of whom benefitted by aggregating their orders to obtain the best possible execution price and lower commissions.  Such aggregation and post-execution allocation is not inherently fraudulent or in breach of an advisor’s fiduciary duties toward its investors.  Nevertheless, there are many SEC cases against investment advisers involving cherry picking or favorable trade allocation.

What differentiates recent cases is the SEC’s application of statistics to analyze trades.  For example, in June 2015, the SEC announced an enforcement action alleging cherry picking that arose from an “initiative analyzing large volumes of investment advisers’ trade allocation data.”  This was supported by statistical tests purportedly showing that the investment adviser profited by entering certain options trades in a master account and then, after waiting to see how the positions fared during the course of the day, allocating profitable trades to his personal account and unprofitable trades to clients’ accounts in the afternoon.

In an April 2016 action, the SEC showed that “proprietary accounts averaged a first-day gain of 0.26% while client accounts averaged a first-day loss of 1.02%” and supported its allegations by showing that the “difference between the allocations [by another investment adviser] of profitable trades to proprietary accounts as compared to profitable trades allocated to the client accounts is statistically significant; the probability of observing such an uneven allocation of profitable trades by chance is less than one-in-one-million.”  Another April 2016 action provided similar arguments.

The novel implication of “intent” from statistical analysis

The SEC’s Division of Economic Risk Analysis determined in the June 2015 action described above that personal profits from these trades could not have resulted from “a coincidental or lucky combination of trades.”[3]  The commission investigated his profits from trading in options in an S&P 500 ETF called SPY and found that the profits in his personal accounts were higher than those of his clients, to a highly statistically significant extent.  The SEC used a simulation to calculate the chance that his personal accounts profit would be higher than his actual profit by random chance alone.  The SEC also found, using a simulation, that the likelihood of his personal accounts receiving their “high proportion of profitable trades by pure random chance is less than one in one trillion.”[4]

The SEC charged that investment adviser with fraud.  However, as with most statistically based enforcement efforts, the trade allocation initiative comes with risks and limitations.  The SEC’s announcements seem to imply that statistically eliminating chance as an explanation for the profitable trades proves a fraudulent motive.  We believe that is not always the case and that one should not base intent on statistical analyses, which can be helpful in showing an adviser was aware that his returns were higher than his clients’ but can also identify abnormal outcomes in an environment free from wrongdoing.  As the SEC’s former chief economist Craig Lewis explained in the context of the Accounting Quality Model, statistical analyses should be used to produce signals that may warrant further investigation.[5]

Statistics-based enforcement actions may be false alarms

Lessons can be drawn about statistical-based enforcement actions, even if the details on the statistical tests in this specific matter are scant.  Of particular interest is the possibility that statistical tests lead to false findings of suspected wrongdoing.  These are known as false positives and are the reason why further investigation is necessary to establish wrongdoing and intent.

First, before taking statistical results as definitive evidence, it may be useful to consider other explanations for those results.  Tests like the ones described by the SEC should also factor in the possibility that different trading strategies or trading constraints may explain different returns.  For example, an investment adviser may use certain types of options (puts/calls, out/at/in-the-money) for certain purposes for himself and different options for different purposes for his clients.

In contrast, in that same example, the SEC seemed inclined to aggregate any and all different types of options on SPY, despite their different average returns and the respondent’s use of options across different strategies.[6]  On any given day the respondent may have, for example, entered an aggressive strategy for himself using out-of-the-money calls and a conservative one for his clients using at-the-money calls.  However, the resulting different returns might have raised suspicion under the SEC’s analysis.

Second, and also related to trading strategies, it is important to examine portfolios instead of just one particular investment instrument.  Examining the returns in one specific asset may overlook how they are related to returns of other assets in the clients’ portfolios.  For example, if the options in a client account are a hedge against market exposure or against exposure to specific investment horizons, then it is important to notice offsetting returns and the impact of those on the overall returns to the client.

Third, it is important to examine the consistency of the returns behind the alleged wrongdoing.  If the statistical evidence aggregated over the whole period of alleged wrongdoing does not hold consistently at a higher frequency throughout that period, then the allegations may have to actually be concentrated in only a subset of the alleged period.

Fourth, and in the same spirit of the need to disaggregate returns to observe their distribution, care must be taken to ensure that statistical evidence on aggregate or average returns is not determined by a few outlier returns.  If a simulation is based on the average return of a fund or trading strategy over several years, the analysis may be swayed by a few very positive or very negative returns, absent which performance might be unremarkable.

Finally, there is often a temptation to label statistically significant results as relevant, but that is only the case if they have sufficient magnitude.  In the June 2015 matter described above, the personal accounts returned on average +6% in its SPY options trades on the first day, while the client accounts returned -5% on average.  This discrepancy seems large but would seem less unusual if compared with a typical fluctuation in the returns on SPY options.  One-day returns on these options can fluctuate by about 20 percentage points.  In addition, statistical tests of returns will inevitably find instances of abnormal returns, especially if run on a sufficiently large number of potential targets.  There are statistical tests that try to address this issue.

Conclusion

We recommend that investment advisers, asset managers, and even corporate defendants undergoing Office of Compliance Inspections and Examinations scrutiny or enforcement investigations become familiar with the SEC’s use of statistics.  When already faced with such actions, defendants countering statistics-based suggestions of fraud may require a sophisticated demonstration of why such suggestions may be misplaced.  These may involve statistical rebuttals as well as alternative explanations of the results.  Based on the approaches that the SEC has announced, these actions will likely become more frequent.

ENDNOTES

[1] See, e.g., http://www.sec.gov/News/PressRelease/Detail/PressRelease/1365171485332.

[2] See, e.g., http://www.sec.gov/News/Speech/Detail/Speech/1365171491988.

[3] http://www.sec.gov/news/pressrelease/2015-132.html.

[4] http://www.sec.gov/litigation/admin/2015/34-75319.pdf.

[5] “This is not to say that we’ve built a model that can ‘detect fraud.’  Far from it.  Rather, we hope to provide one more tool that the already sophisticated staff of the SEC can use in its efforts to ensure high quality financial statements.” in speech by then SEC Chief Economist Craig M. Lewis on December 13, 2012.

[6] The defendant “said that during the relevant time period, in investing his clients’ funds, he followed four investment models: conservative, moderate, aggressive, and options. [He added] that the options model traded only SPY options, but that he also traded SPY options in the other models.” Further details in http://www.sec.gov/litigation/admin/2015/34-75319.pdf.

This post comes to us from Tiago Duarte-Silva, PhD, an economist with Charles River Associates and adjunct professor at Boston College, and Nicolas Morgan, a partner at the law firm of Paul Hastings. The views expressed herein are those of the authors and do not necessarily reflect the views of Charles River Associates, Paul Hastings, or any of the other organizations with which the authors are affiliated.