AI companies consider how to respond if catastrophic hacking incident sparks ‘revolt’: report

Executives at major AI companies like OpenAI and Anthropic are preparing potential responses if an AI disaster triggers a “public and political revolt,” according to a report.

Preparations focus on creating contingency plans in case one of their AI models causes major public harm – such as a hack targeting the power grid, water supply or banking system, Axios reported Friday.

It’s unclear whether the preparations are much different from the simulation exercises that institutions from banks to the Pentagon and beyond have long used to assess risks.

Yet the report was released during a period of unprecedented scrutiny of AI companies, as some researchers, like former Anthropic employee Jacob Coxon, warn that malicious, unchecked AI could wipe out humanity.

“As do many companies across industries, OpenAI conducts preparation exercises where teams discuss and work through a range of potential scenarios,” an OpenAI spokesperson said in a statement.

“These scenarios are not considered inevitable, but are intended to help us prepare for various circumstances,” the spokesperson added. “We have made it clear that AI is changing the cyber threat landscape and we are working to put powerful tools in the hands of defenders. »

OpenAI has come under increased scrutiny following recent revelations that some of its AI agents went rogue, escaped their testing “sandbox” and hacked rival AI company Hugging Face.

“The Hugging Face incident showed that we underestimated the true cyber capabilities of our AI models. We are strengthening our security requirements accordingly, which adds even more urgency to our existing security research and internal security work,” OpenAI co-founder Greg Brockman said in an essay published in August.

Anthropic has also been outspoken about the potential risks of advanced AI. The company’s CEO, Dario Amodei, in September called for an industry-wide pause in advanced AI development and endorsed the idea of ​​integrating third-party security assessors into larger companies.

The company also released an alarming report last month detailing attempts by bad actors to use its Claude chatbot for nefarious purposes, such as building guided missiles and tracking the movements of US ships and aircraft.

Anthropic did not immediately respond to a request for comment.