In Halliburton Co. v. Erica P. John Fund, Inc., 134 S. Ct. 2398 (2014), known as Halliburton II, the U.S. Supreme Court held that defendants may defeat the fraud-on-the-market presumption of reliance at the class-certification stage with evidence that the misrepresentation did not in fact affect the stock price. Securities litigants typically use event study methodology to detect and measure price impacts. Halliburton II will increase the ever-present role of event study methodology in securities litigation. Our new paper, Event Studies in Securities Litigation: Low Power, Confounding Effects, and Bias, explores the reliability of event studies in securities litigation.

Developed in the late 1960’s, the event study methodology was a breakthrough quantitative research method in financial economics and hundreds of articles in published peer-reviewed journals apply the methodology. Securities litigants have not been shy in asserting the event study’s impressive academic pedigree. What is curious about the use of the event study in litigation, however, is that the methodology used in court differs fundamentally from the methodology used in academic research. In particular, securities litigation event studies are almost always single-firm event studies that examine the price moves of the security of the single firm involved in the litigation, while almost all academic research event studies are multi-firm event studies that examine large samples of securities. Importing a methodology that financial economists developed for use with multiple firms into a single-firm context creates substantial difficulties, and review of the case law suggests that courts and litigants often have failed to recognize these problems.

First, compared with the multi-firm event studies that fill the published journals, single-firm event studies cannot detect economically meaningful price impact unless the price impact is quite large. Courts typically adhere to the same statistical significance levels that researchers use in multiple-firm event studies. But requiring conventional levels of statistical significance in the single-firm contexts effectively gives a “free pass” to economically meaningful securities fraud because the single-firm event study lacks the power to detect price impacts below a very high threshold. Accepting statistical insignificance at face value, courts conclude that some economically large price impacts are “immaterial.” Courts err because of a mistaken premise that statistical insignificance indicates the probable absence of a price impact. It does not. How important is this problem in practice? In our paper, we show, for example, that for the largest 10% of firms in 2014, a price impact would have to change the market capitalization by, on average, nearly $900 million to be detectable by the single-firm event study. Currently, the use of statistical significance concepts that are appropriate to multi-firm event studies implements a legal regime where the probability of incorrectly exonerating securities defendants is much higher than the probability of incorrectly finding securities defendants liable. This lowers the deterrent effect of the securities laws and may encourage more small- and mid-scale fraud on markets than is socially optimal given the costs of litigation.

Second, when a single-firm event study does detect a price impact, it reflects confounding effects that are unrelated to the alleged fraud. Financial economists have long understood that our ability to fully explain observed price moves is quite limited; much price movement occurs for reasons unrelated to news, including as a result of the liquidity trades of investors in the market seeking to raise funds for other purposes and the (at least short-term) impact of “noise traders” who trade for irrational reasons. In the multi-firm context of peer reviewed event studies, this is of less concern because averaging across firms tends to average away the effects of unrelated price movements leaving a better estimate of the mean price impact of the event under study. The matter is much more difficult in the single-firm context. When we observe a price impact of, say, -4.0%, we could have observed the actual price impact without contamination by confounding effects. But we also could have observed a combination of an event price impact of, say, -2.5% and an unrelated price impact of -1.5%. Or we could have observed a combination of an event price impact of -5.0% and an unrelated price impact of +1.0% in the other direction. In general, the price impact we observe is comprised of a component related to the event plus a component related to non-event movements, but we have no mathematically precise way to separate them from each other.

While the event study itself is a first step, an economic expert must then work to decompose price effects outside the framework of the event study methodology. Because that effort is necessarily partly subjective and less constrained by generally accepted quantitative methods, it has proven unsatisfactory to some courts, which have complained of such efforts that they are based on “unprovable and often unexplained assumptions,” that they are no “more than observations and market rumors,” and that they reflect little more than “a judgment call as to confounding information without any methodological underpinning.” Courts have excessively high expectations of the ability of litigants – whether plaintiff or defendant – to decompose an observed price impact into a component caused by fraud and a component caused by other factors. There simply is no fully reliable, mathematically precise way to do so.

Finally, single-firm event studies generate sizeable upward bias in detected price impacts and damages (i.e., overstating the magnitude of a price impact and damages). For example, suppose the true price impact is -2.0%, but the requirement of statistical significance is such that price impacts less severe than -2.94% will be rejected as statistically insignificant. In that case, a price impact will be detected only when there are confounding effects that push the observed price impact past -2.94%. The expected detected price impact, given that we observe a price impact large enough to be detected, may be substantially higher than the true price impact. This tells us that we cannot leave confounding effects unaddressed in the hope that they are as likely to be on one side of the true price effect as on the other. To the contrary, the bias in price impacts – and therefore in damages – is upward (i.e., in the direction of more severe price impacts) not downward.

We offer the following suggestions for improving the use of event study evidence in securities litigation.

1. Courts should require litigants and their expert witnesses to report the results of statistical “power analysis” for all event studies. Power is the ability to detect a true effect (e.g., a price impact due to fraud) when it exists. Financial economists using event studies in their academic work take power seriously. That same level of rigor should apply in litigation. A securities litigant should not be heard to say that a misrepresentation or corrective disclosure caused no price impact based on a test that had little or no power to detect a price impact that the court determines to be material.

2. To address the problem of confounding effects, courts should allow litigants considerable flexibility to present other evidence to prove that a price impact from misrepresentation or corrective disclosure did or did not occur. Such evidence may include, as it does now, intraday analyses and analyses of other news about the firm that day and quantitative analysis of its potential effect on value, but also evidence from multi-firm event studies of similar events that may shed light on the magnitude of price impacts of that type, estimates of the effects of liquidity and noise traders, and opinions based on other valuation methods such as discounted cash flow to value posited confounding effects.

3. Courts and litigants should recognize the possibility that statistically significant price impacts are biased estimates of true price impacts. Because of bias, detected price impacts are more likely to overestimate price impact than underestimate it. It is especially important to consider this possibility in evaluating securities damages.

The preceding post comes to us from Alon Brav, Robert L. Dickens Professor at Duke University Fuqua School of Business and National Bureau of Economic Research, and J.B. Heaton, partner with Bartlit Beck Herman Palenchar & Scott LLP. The post is based on their recent paper, entitled “Event Studies in Securities Litigation: Low Power, Confounding Effects, and Bias, forthcoming in the Washington University Law Review after presentation at the 21^st Annual Institute for Law and Economic Policy Conference to be held in April. The full paper is available here.