Columbia Law School
Sky Blog
In his May 5 post, available here, Stanislav Dolgopolov states that the Securities and Exchange Commission’s recent settlement with Citadel “undermines the so-called ‘Berkeley Study’ which concluded that off-exchange market makers can neither profitably engage in data feed arbitrage by ‘filling marketable orders at (or within) the SIP-generated NBBO [National Best Bid and Offer] . . . at stale prices to the disadvantage of retail investors’ nor ‘choose as their pricing benchmark the slower SIP-generated NBBO to boost their performance metrics.’” Mr. Dolgopolov further states that our study “skirted the fact that relevant strategies do not rely on choosing either the SIP or direct data feeds but on cherry-picking individual trades.”
We take strong issue with these statements. We believe them to be plainly inaccurate and to misrepresent what our study concludes.
First, we fully account for cherry-picking. Panel C of Table 5 of our paper displays an upper bound to the profits available to liquidity providers seeking to take advantage of SIP latency. Across more than $3 trillion of SIP-priced trades in the 30 stocks comprising the Dow Jones Industrial Average, the total profits are about $500,000. For trades in non-exchange venues, the amount is $239,241. This is for 10 months of trading. We further estimate these same figures for the entire market over the course of a year. Our estimate for all non-exchange trades is between $2.9 million and $3.5 million. Again, these are gross profits to non-exchange liquidity providers assuming perfect cherry-picking of all marketable orders.
Second, the Citadel settlement covered the period between June 2008 and January 2010. We state clearly, “Nor do our results rule out the possibility that latency arbitrage arising from stale SIP quotes might have been prevalent in the quite recent past (e.g., 2014), for the simple reason that our data are not available until mid-2015.” We additionally state in our conclusion that, “Because our data commence in August 2015, we emphasize that these findings may very well reflect a new market environment in which the HFT strategies depicted in Flash Boys are less prevalent than in the past.”
Finally, the central research question we pose in our paper is whether SIP latency arbitrage strategies are sufficiently profitable that they can explain the current arms race in trading speed. Overall, our analysis suggests SIP reporting latencies generate remarkably little scope for exploiting the informational asymmetries available to subscribers to exchanges’ direct data feeds, regardless of whether trading is targeted at liquidity takers (at issue in Citadel’s settlement) or at liquidity providers (at issue in the recent wave of exchange proposals for speed bumps).
Even though our sample period commences after Citadel ceased using FastFill and SmartProvide, we fail to see how the SEC’s factual findings reveal a trading environment where FastFill and SmartProvide were sufficiently profitable to Citadel to justify large scale investments in trading speed. As Mr. Dolgopolov notes, total trades affected by these strategies were well under 1 percent of Citadel’s trading volume from June 2008 through January 2010. And total disgorged profits were $5 million on a firm with $23 billion in assets as of December 31, 2009. Assuming Citadel was a player in the high-speed arms race during this time-frame, the SEC settlement provides little reason to believe it was because of these two strategies.
Our paper is available here.
This post comes to us from professors Robert P. Bartlett III and Justin McCrary at the University of California, Berkeley – School of Law.
As a preliminary remark, I’d like to thank Professors Bartlett and McCrary for engaging in this debate. Moving to the subject matter itself, I wanted to point out that my own blog post (which sat in the publication queue for over a week) cited an earlier version of the empirical study in question, which is clearly indicated in the endnotes. Although the authors’ criticism of my own piece appears to be largely based on the most recent version posted on May 2 (e.g., “Panel C of Table 5”), these two versions are materially different, and I would certainly invite the readers to do their own comparison (e.g., by searching for “gross profits” or “picking-off”). I’ve just read the most recent version, and I do agree that it accounts for cherry-picking – absolutely no question about that! At the same time, I still don’t see how the cherry-picking scenario for an off-exchange market maker was spelled out in the earlier version. My sincerest apologies to the authors if I’m misreading the NBER version again! In any instance, this update is a much needed improvement (or clarification?) for this empirical study, but a reference to the SEC’s settlement with Citadel, as a clear illustration of latency arbitrage, would also be merited.
Overall, I still hold the opinion that the Citadel settlement is contrary to some of the *broader* conclusions of this empirical study, and I was certainly aware that the study covered a later period in time. Whether this settlement a mere data point going against the trend (or historical / discontinued practices) or very indicative of the trend remains to be seen. I have expressed some skepticism about the lucrativeness of pure data feed arbitrage by off-exchange market makers, but we really don’t know how the regulators came up with this specific disgorgement amount. Since the SEC pointed out that FastFill and SmartProvide had provided some value to directed orders under certain circumstances, one may speculate that the disgorgement amount had been adjusted for that. If that is the case, the magnitude of latency arbitrage could be even larger. Moreover, the Citadel settlement is an important example of how price improvement statistics could be manipulated.
Finally, as emphasized in my blog post, perhaps it’s not about SIP-direct feed arbitrage at all but rather about “gameable” speedbumps that are much longer-lived. While SIP-direct feed discrepancies served as a triggering event for SmartProvide (as described in the settlement), the actual timeframe for this algo was in a league of its own.
Stanislav,
Our original version discusses gross profits at p. 23. There, we note that net profits for all liquidity-taking trades in our sample were $11.1 million. In note 43, we specify that excluding gains to liquidity providers the amount is $11.6 million, indicating $500,000 of gross gains for liquidity providers. In response to comments, we revised the draft to emphasize these findings, so it’s good to know that our revisions had the desired effect.
We’re not sure what you mean by our “broader” conclusions. Our primary conclusion is that SIP latency arbitrage is not sufficiently profitable after August 2015 to explain the high-speed arms race. We also conclude that using the SIP NBBO vs. the Direct NBBO does not materially affect effective/quoted spread ratios (Table 8 in the current draft). We don’t view the Citadel settlement as contrary to either. As noted in our post, the settlement involved a timeframe that precedes our study. And in any event, the settlement does not reveal that the strategies affected a meaningful portion of Citadel’s trading volume or generated anything more than de minimis revenue for a firm its size.
Thank you for your suggestion that we reference the settlement, which we will do in our next revision.
Robert Bartlett & Justin McCrary
Many thanks for this clarification, and I applaud this much-needed revision! I think it’s reasonable to say that these references to the NBER version (p. 23 and footnote 44 – not 43) do not discuss the cherry-picking angle (once again, I would invite the readers to decide for themselves). My apologies to everyone who has been confused, but I don’t see how my interpretation of the earlier version was “misleading” in any way. The ultimate irony here is that our discussion is taking place precisely because I relied on an earlier version of the study (i.e., a stale reference point) and didn’t have an opportunity to see the most recent (and materially different) version (i.e., a fast reference point). My original concern was over the study’s comparison of “SIP NBBO” and “Direct NBBO” execution quality and the corresponding conclusion that customer order were not harmed by being priced off the SIP without explicitly mentioning Citadel-like abuses that rely on cherry-picking. This angle is in fact explored in my law review article that came out last fall, which was well before the date of the Citadel settlement.
I still think that the Citadel settlement, as a historical data point, *absolutely* falsifies a more *general* theory (based on a data set from a different time period of course – I never argued about that) that off-exchange market makers can neither profitably engage in data feed arbitrage by “filling marketable orders at (or within) the SIP-generated NBBO . . . at stale prices to the disadvantage of retail investors” nor “choose as their pricing benchmark the slower SIP-generated NBBO to boost their performance metrics.” I never questioned the data sample or the quantitative results of the original empirical study. I’ll leave it to the readers to make their own informed judgment about broader implications of the settlement for both past and current practices, while reminding everyone once again that pure data feed arbitrage by off-exchange market makers may not even be the main concern compared to longer-lived time delays. Another point is that when the “Berkley Study” originally came out, it attracted the attention of some people in the industry who used it to make a much wider claim that “latency arbitrage is a myth” instead of something like “a data sample for Dow Jones stocks starting in August 2015 shows that SIP-direct feed arbitrage is hardly profitable.” I can’t possibly blame the authors for that (and they probably have suffered enough), but that’s how distorted the public discourse became.
Many thanks to Professors Bartlett and McCrary for this debate and all the readers for their attention! I hope that it has clarified some key points and led us to a point where we’re left with only reasonable disagreements.