Unlock Editor’s Digest for free
Roula Khalaf, editor-in-chief of the FT, selects her favorite stories in this weekly newsletter.
Anthropic formally warned investors that its technology could pose “existential risks to humanity” in a highly anticipated IPO prospectus released in recent days to a small group of partners.
The nearly $1 trillion AI startup led by Dario Amodei spent nearly a third of its lengthy S1 filing detailing “risk factors,” including the potential for increasingly advanced AI models to manipulate, blackmail and exhibit other unpredictable behavior.
Anthropic has highlighted more prosaic risks, including the extreme concentration of its customer base, with nearly a quarter of its revenue last year coming from just two clients, according to people familiar with the matter.
The extraordinary warnings in its risk disclosures add to the challenge public market investors face in valuing the leading U.S. AI group as it launches what is expected to be the most valuable IPO of all time.
Claude’s maker also provided investors with a clearer picture of the economic challenges of building cutting-edge AI models ahead of listing, expected on Nasdaq this fall.
Anthropic said it plans to spend $518 billion on cloud, compute and infrastructure obligations in the coming years to support its rapid growth.
The AI lab’s backers are confident that Anthropic can list at a valuation of more than $2 trillion, more than double the level reached in its last funding round in May and beyond the $1.78 trillion obtained by Elon Musk’s SpaceX in June. They point to its extraordinary growth rate to justify their optimism.
But concerns about the risks posed by AI now threaten to dampen that optimism after current and former Anthropic staff issued public warnings that unchecked AI could end humanity within a decade.
Amodei told the UN Security Council last week that AI could threaten humanity and was “the most important global security challenge facing the world today”.
The head of Anthropic this month made public calls to “advance the frontiers” of AI development. His main rival at OpenAI, Sam Altman, along with Musk, supported Amodei’s proposals for industrial cooperation to slow development and strengthen security measures, in a rare moment of unity.
Security concerns have increased following a series of high-profile cybersecurity incidents in which AI “agents” breached external systems. Last week, OpenAI revealed that its tools had hacked “dozens” of external sites, including governments.
The maker of ChatGPT on Monday abandoned plans to release its new model next month, citing security concerns.
Anthropic’s prospectus also revealed details of its previous financial performance and governance arrangements, which were first reported by Reuters and confirmed to the FT by several people who have seen the filing.
Anthropic declined to comment.
The company posted an operating loss of more than $8 billion last year as it increased spending on computing power, Anthropic told investors. Its revenues grew twelvefold to nearly $4.6 billion last year. But the rising costs of the computing resources needed to train and run its models have pushed operating expenses to nearly $13 billion.
The group has signed deals to secure computing power from partners including Google, SpaceX and a series of smaller companies.
Anthropic’s expenses and revenues both rose sharply this year. Revenue for the second quarter of this year was $11.5 billion and is on track to record its second consecutive quarter of operating profit, albeit on an adjusted basis.
Gn headline
