Since the launch of ChatGPT in November 2022, there’s been an exponential surge in the use of generative AI tools. Anecdotal evidence suggests that these tools are also of considerable interest to financial firms. As a case in point, Ken Griffin, the founder and CEO of Citadel, recently acknowledged the profound influence of ChatGPT technology on the firm’s operations and confirmed that Citadel is negotiating for an extensive enterprise license. Despite this anecdotal evidence, the true economic value of AI tools in financial markets is yet to be fully understood.

In our latest research paper, we delve into the effectiveness of a generative large language model (LLM) in distilling the most crucial data from corporate disclosures. More specifically, we employ GPT-3.5 to condense the information relayed by corporations to their stakeholders. We then examine the information quality of these summaries and devise a measure of the degree of superfluous or less relevant textual information in corporate disclosures.

Our study focuses on the efficiency of generative language models, particularly GPT-3.5, in extracting the essence of corporate disclosures. How does the summary’s information compare with the original? What is the extent of information overload across different companies? Are there consequences in capital markets due to redundant information? Most important, is the model proficient at creating focused summaries for investors interested in a particular topic, such as ESG activities or firms’ risk exposure?

To address these issues, we concentrate on two primary types of corporate disclosures: management discussion and analysis (MD&A) and earnings conference calls. We start with a randomly selected sample of about 20 percent of all MD&As and conference call transcripts from 2009 to 2020. We then direct GPT-3.5 Turbo to generate a comprehensive summary of each document without referencing other documents or external sources. The model generates compact summaries, indicating potentially substantial improvements in information processing. The concern, however, is whether these summaries, despite omitting many details, are still informative.

Our findings demonstrate that the sentiment expressed in the summarized document  is stronger than in the original. When the original sentiment is positive (negative), the summarized document tends to be more positive (negative). We assess the information content of the summary against the original by comparing their ability to explain stock market reactions with the shared information. Assuming that stock prices are efficient at aggregating public disclosures, we find compelling evidence that the summary’s sentiment explains market reactions better than the original’s sentiment does. We also observe that the MD&A summaries are significantly more informative than conference calls, which are less likely to contain obfuscation or generic language.

Our study reveals a striking ability of the language model to condense information while preserving and even enhancing its value, suggesting potential information bloat in corporate disclosures. We further investigate whether companies differ in their disclosure bloat and whether it has negative capital market consequences. We establish a measurement of redundant or irrelevant information, referred to as “Bloat,” and explore whether it varies over time, across industries, and for individual  firms. We discover that Bloat tends to be higher when a firm reports losses, experiences negative sentiment, and suffers negative stock market reactions. This bloated reporting correlates with adverse capital market consequences, even when controlled for conventional readability metrics, thereby illustrating that our measure captures a different construct – relevance and redundancy of information instead of readability.

Finally, we investigate the capacity of LLMs to generate targeted summaries. We create prompt-based summaries focusing on financial performance, ESG activities, and firms’ risks. Our findings reveal an upward trend in ESG-related content in conference calls, and these summaries carry different dimensions of information communicated by the firms. As expected, the sentiment from ESG-specific summaries becomes progressively important in driving stock-market reactions in the later part of our sample period. Additionally, we find that the relative length of different risk summaries can help explain firms’ abnormal return volatility surrounding the information disclosure.

Overall, our results indicate that generative AI systems like ChatGPT can greatly assist investors by providing concise and relevant information from convoluted corporate disclosures. Corporate disclosures have been growing in complexity and length over the past decades, often overwhelming investors. Our findings suggest that generative language models could mitigate this information overload, as they produce considerably shorter summaries while retaining and emphasizing the core message. Such AI tools should aid investors in making well-informed investment decisions.

Our results suggest that regulators or information intermediaries like the SEC could establish infrastructures to provide summaries to improve the information efficiency of capital markets. These summaries might be particularly useful .for less sophisticated investors.

Additionally, by leveraging recent advancements in generative AI, we have developed a clear and easy-to-understand measure of the degree of redundancy and superfluous details in text. Thanks to its simplicity, our methodology can be readily applied to different forms of corporate communication or even in non-corporate settings, considering the rising importance of text in various disciplines.

This post comes to us from Alex G. Kim, Professor Maximilian Muhn, and Professor Valeri Nikolaev at the University of Chicago’s Booth School of Business. It is based on their recent article, “Bloated Disclosures: Can ChatGPT Help Investors Process Information?” available here.