Anthropic priced its initial public offering on Wednesday at $83 a share, above its initial range of $70 to $78, raising $81.3 billion in the largest IPO in history and valuing the company at about $1.5 trillion. On Thursday, its first day of trading on Nasdaq, its shares closed at $97.17, up 17 percent. Demand held through a month in which AI stocks sold off and the midterm elections swept a bloc of AI skeptics into office. It also survived Anthropic’s own admission that its AI agents went rogue before the safeguards to stop them were in place..
Anthropic’s registration statement, public since October 26, devotes some 90 pages to risk, and one passage has drawn most of the attention. In it, the company tells prospective shareholders:
Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.
It is a remarkable sentence to find in a securities filing, and most commentary has read it as a warning about the distant future. Investors should read the conditional clause instead: “if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures.” Safeguards, oversight, and security measures are not the vocabulary of computer science. They are the vocabulary of corporate governance. The prospectus is telling investors that the value of the enterprise turns on a governance question. The same document shows that, when it mattered, Anthropic had not answered that question.
The risk-factor summary warns that the company’s “AI models may behave in unintended, misaligned, or autonomous ways, including by taking harmful actions that no user or third party directed.” This is not hypothetical. The filing discloses that four of Anthropic’s models escaped their testing environments and operated on live, external computer systems for sessions lasting 10 to 34 hours, and that the company paused training of its next flagship model for more than a month. In the weeks before the filing, Anthropic disclosed that its Claude agents had submitted a made-up tip about an unsolved homicide to the Philadelphia Police Department, filed 20 incomplete visa applications on the State Department’s website, and pulled Georgia vote-count data from behind a paywall without anyone asking them to.
Anthropic is not alone: In June an experimental OpenAI agent researching health statistics broke into the infrastructure behind an Australian Medicare statistics portal, and OpenAI did not tell the Australian government until September, a delay Prime Minister Anthony Albanese called unacceptable.
Rogue agents are no longer a thought experiment. None of these incidents comes close to the catastrophe the prospectus contemplates, but they are early versions of the same failure: an AI system acting with more autonomy and reach than anyone intended. These systems will misbehave. The question for investors is whether the companies that build them have corporate governance strong enough to detect, prevent, and correct that misbehavior when it happens. That question can be asked before an investment is made. Today it rarely is.
Anthropic’s October 9 report, which disclosed the incidents in more detail than it had to, concedes that some of them could have been prevented, and the fixes the company adopted afterward, such as cutting its test environments off from the internet, required no new technology. What was missing before the incidents was not technology but governance: someone with the authority to decide what the company’s agents were allowed to do, and controls to make sure they did nothing more.
Technical Failures Are Governance Failures
Anthropic has built its identity on the idea that safety can be written into the business through rules the company sets for itself. Its Responsible Scaling Policy, first published in 2023, defined capability levels that would trigger mandatory safeguards and committed the company to pause training or deployment if it could not put adequate safeguards in place before reaching the next level. The policy was meant as the antithesis of Facebook’s early motto, “move fast and break things,” which told engineers to release products quickly and repair the damage afterward. Responsible scaling reversed the order: Before building a more powerful model, Anthropic would decide what safeguards it needed, and it would not go forward until they were in place. It was an attempt by a company to bind its own future conduct. In February, version 3.0 of the policy removed the pause commitment, putting in its place periodic risk reports, a roadmap of safety goals that are not binding, and recommendations for what the industry as a whole should do. The change shows the limit of self-binding: a commitment a company writes for itself is a commitment it can rewrite.
The incidents exposed a second limit, and it can be described in two vocabularies. In the engineers’ vocabulary, a scaling policy governs what a model can do at the frontier of capability, not what an agent is permitted to touch; it is not a “network boundary,” and the failures were failures of “containment” and “permissions.” That is how the problem presents as a technical one. In the vocabulary of corporate governance, it is something more familiar: the lack of corporate controls. What the company’s agents could reach was left to the configuration of a test environment rather than settled in advance, and with authority, by someone accountable for the answer. Even the pause that did occur, in training of the next flagship model, came after the escapes rather than ahead of them. On the ground, the record looks less like responsible scaling than like the motto it was meant to replace: Things broke first, and the fixes followed. Each vocabulary describes half the problem, and neither alone could have prevented the entire problem. Investors who gave Anthropic high marks for governance would have been right about its intentions and wrong about its exposure.
None of this is new except the technology. I have served as a corporate governance expert in two major shareholder derivative cases. In the Vioxx litigation, I helped fashion the governance reforms the court imposed on Merck. In the Wells Fargo sales-practices litigation, I assessed whether the reforms in the settlement would correct the governance failures behind the scandal. The failures at the two companies were different. Merck had no chief medical officer, and its board committees were not built for the scientific and patient-safety risks of a drug company. At Wells Fargo, no fewer than five board committees had some responsibility for the risks at the heart of the scandal, and none had clear control over them. But behind both scandals were failures of what I call governance hardware: the match between a company’s risks and its structures, its personnel, and its channels for reporting problems to the people responsible for acting on them, the kind of financial-reporting discipline that the Sarbanes-Oxley Act imposed on companies.
Anthropic’s hardware problem is different. The company has had the ideas for ensuring safety, and it has revised them; it has a responsible scaling officer and a safety organization. What its disclosures reveal is that the ideas did not reach the machinery: a test environment that could reach the open internet, a fabricated police tip that went unnoticed for 10 weeks, and no effective way to stop an agent from taking an irreversible action on its own. AI is a complicated enough technology that the reporting and control structures it needs will look different from those of a bank or a drug company. They need to be built just as seriously and just as deliberately. That is work for the general counsel’s office and the engineers together. A safety policy that exists as an idea at the top and is left to the technical staff to implement is how a company ends up disclosing its safeguards after the fact.
That work now carries legal stakes. Anthropic is a Delaware public benefit corporation, which requires its directors to balance shareholders’ financial interests against the public benefit the company has committed to pursue, but that status does not relieve them of the ordinary duties every Delaware board owes. Under Delaware law, directors must make a good-faith effort to ensure that the company has a system for reporting the risks that are critical to its mission, and they must act when red flags appear. In 2019 the Delaware Supreme Court applied that duty to the board of an ice-cream maker that had no system for monitoring food safety. For a frontier AI developer, the safe operation of its own models is at least as critical to the mission. Anthropic’s board has now seen the red flags, and so have its shareholders, in the company’s own filing. If incidents like these recur now that the company is public, the question will not be whether the technology misbehaved. It will be whether the board did what the law requires, and that is the stuff of a shareholder derivative suit.
Engineers and Lawyers in the Same Room
The strong demand for Anthropic’s shares does not mean investors ignored these disclosures. It means they weighed them against an extraordinary business. But what the disclosures reveal is a governance failure, and the prospectus itself explains why that matters to shareholders. Given the scale of harm a catastrophic AI incident could cause, it warns, “no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.” Where risk cannot be insured, it falls on shareholders, and the only protection left is governance.
Throughout this record, problems that presented as technical turned out to be one and the same failure of corporate control: Decisions that should have been handled as corporate governance matters were treated as technical ones. What an agent could reach, who would be told when it reached too far, and who had the authority to stop it were seen as engineering choices and not assigned to someone who would be accountable for the decisions. Neither lawyers nor technical specialists alone can remedy that failure. Data scientists know what a model can do and where a “network boundary” or a “permission” has to sit. Legal officers know how to turn those limits into control, each with a named person accountable for it, a reporting line, and a record that will stand up before a board, a regulator, or a court. Sound AI governance will come from a sustained dialogue between the two types of professionals, with the general counsel’s office at the table when a system is designed rather than summoned after an incident. Anthropic’s prospectus has done investors a service by putting the stakes in writing. Investors who ask whether that dialogue is happening will see these failures coming. Those who do not will find out in a risk-factor disclosure, after the value of their investment is already gone.
Michael A. Santoro is professor of management and entrepreneurship at Santa Clara University’s Leavey School of Business and director of the Artificial Intelligence, Business Ethics, and Human Rights Lab. He thanks Sophie Mooney for her research assistance as a fellow of the Artificial Intelligence, Business Ethics, and Human Rights Lab at Santa Clara University.
Disclosure of interests. The author serves on the advisory board of Entefy, a company that develops agentic AI systems, and holds stock options in Entefy. He is the founder of Ardelia Inc., which applies AI to art provenance research, and has founded other early-stage AI companies. He holds no financial interest in Anthropic or OpenAI and did not participate in Anthropic’s offering.
Sky Blog