Effective corporate governance relies on models—business models, financial models, and risk models, among many others—the primary fuel for which is timely, reliable information. In this connection, statistics luminary George Box famously wrote, “Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.” Models can mislead because they are theoretically unsound or because they consume biased data: garbage in, garbage out, as the saying goes.

Traditional GAAP-based financial statements have historically served as a key corporate governance information source for boards and investors. Their reliability however, has been questioned by regulators, accounting practitioners, and researchers, for both theoretical and data-bias reasons.

In 2005, Don Nicolaisen, then the SEC’s Chief Accountant, testified before the PCAOB, “If I were to opine on a set of financial statements with my own views, there are few that I would find to be other than misleading.” In 2013, Lehman Brothers litigation plaintiffs won a $99 million settlement from audit giant Ernst & Young by alleging that the financial statements in a 2006 Lehman prospectus misled readers by following GAAP in accounting for so-called Repo 105 transactions. In 2009, the FASB quietly replaced the problematic standards, tacitly admitting that they were structurally biased.

Indeed, the FASB’s pattern of rejecting prior standards and promulgating new ones raises questions about the validity of its GAAP model. But even accepting the model as valid, a recent survey of public company CFOs suggests that public companies routinely manipulate data within the model in order to bias reported results.

Egregious cases are easy to spot in hindsight; Worldcom’s capitalization of fiber-optic line rental costs comes to mind, but the damage was done before the Worldcom fraud was exposed. But what about hidden or latent bias in the financial statements of firms that appear for the moment to be doing just fine, as did Worldcom before its catastrophic demise? The question is how unreliable are published financial statements and can we identify the worst, whether GAAP compliant or not, before they cause irreversible financial harm? Might there be a way to detect material financial statement bias in real time?

In 2012, SEC staff began touting the Accounting Quality Model (AQM), dubbed “Robocop” by commentators, which would in theory flag potential problem cases for early intervention. In early 2015, AQM was characaterized by the Director of the SEC’s Division of Economic and Risk Analysis (DERA) as one of “over a hundred custom metrics” delivered to SEC staff via an “intuitive dashboard.”

Little has been said about the practical results of these early detection initiatives, though the SEC recently credited DERA with preparing a statistical analysis that led to a 2015 “cherry-picking” enforcement against an investment advisor. Understandably, the statistical magic behind the dashboard remains hidden. However, it seems clear that at least one mathematical tool, Benford’s Law, will eventually find its way into the mix.

Benford’s Law is a simple mathematical rule governing the frequency with which the numbers 1-9 appear in leading digits (or pairs of digits) in naturally-occurring sets of numbers. Enter the lengths of all of the world’s rivers or the 2015 revenue figures of all U.S. public companies into a spreadsheet, count all of the numbers that begin with “1,” then divide this count by the total count of all of the numbers in the spreadsheet: first-digit 1s should account for roughly 30.1% of the total, whereas 9s should make up only 4.5%. Benford’s Law works because larger, heavier, wider, and taller things are more rare than smaller, lighter, narrower, and shorter ones. It also works for pairs of first digits such as 12 or 99, as shown in Figure 1.

Thus, if the financial statement figures of a particular company or large group of companies depart significantly from Benford’s Law, it suggests that some figures did not arise naturally in the ordinary course of business. Such anomalies might signal intentional deception, unintentional errors, or artificially imposed financial cutoffs within the organization. Whatever the reason, Benford’s Law can point to the need for deeper investigation. Benford’s Law has been greeted with what might be called skeptical enthusiasm by auditors and forensic accountants many of whom now use it to screen for possible irregularities.

The Law does come with limitations. For example, reliable Benford’s Law readings require large data sets and work only on the entire population of numbers, not on random samples drawn from the population. Some of these limitations are explored in our paper, Reassessing Benford’s Law, Sarbanes-Oxley & Dodd-Frank: A Non-Parametric Exploration of Financial Statement Numbers 1970-2013.

However, perhaps of greatest interest to corporate governance readers is our finding that despite all of the economic, political, and regulatory gyrations since 1970—including Black Monday (1987), Enron’s collapse (2001), implementation of the Sarbanes-Oxley Act (2003), and the 2008 financial crisis—the aggregate Benford’s Law conformity among public companies remained stable throughout the four-decade study period. The stability suggests that in the aggregate, regulatory initiatives like Sarbanes-Oxley and Dodd-Frank have had no discernible impact on the aggregate frequency of financial statement manipulation in the market as a whole.

Audit committees and investors should also note that Benford’s Law can be useful at the individual firm level. It can be used, for example, to test for anomalies in payroll, receivables, inventory, travel expense, or payables accounts containing thousands or millions of entries. At a slightly higher level of aggregation, a study published in December 2015 proposes a first-digit-based Benford’s Law metric called the Financial Statement Divergence (FSD) Score that may provide advance notice of misstated firm-specific financial statements.

In summary, corporate governance relies on a GAAP-based financial accounting model that is demonstrably flawed and embodies significant risk. Benford’s Law is a potentially powerful mathematical tool that can and should be used to help reduce this risk but only with proper training and professional judgment. When expertly deployed, it can strengthen the corporate governance model by serving as a high-level screen for potentially risky financial reporting.

The preceding post comes to us from Kurt S. Schulzke, Associate Professor of Accounting & Business Law and Director of Law, Ethics & Regulation at the Coles College Corporate Governance Center at Kennesaw State University. The post is based on a working paper he coauthored with Bradley Barney, Assistant Professor of Statistics, Kennesaw State University, “Reassessing Benford’s Law, Sarbanes-Oxley & Dodd-Frank: A Non-Parametric Exploration of Financial Statement Numbers 1970-2013”, which is available here.