FILE – The OpenAI logo is displayed on a cell phone in front of an image generated by ChatGPT’s Dall-E text-image model, December 8, 2023, in Boston.
Michael Dwyer/AP
hide caption
toggle caption
Michael Dwyer/AP
SAN FRANCISCO — OpenAI revealed Friday that its artificial intelligence agents interacted with several U.S. government websites in unexpected ways, discovered as part of an ongoing review of unanticipated behavior of the company’s models.
The AI giant’s models accessed publicly available information on two websites run by the Securities and Exchange Commission as well as data from the U.S. Census Bureau, the company revealed Friday. OpenAI found no use of SEC credentials, no access to accounts or nonpublic information, no alteration of SEC data or systems, or any evidence of compromise or vulnerability, the company said.
The disclosure comes at a time of growing global concerns over AI systems escaping human control and hacking of external websites, as well as industry calls to slow down AI development, which OpenAI has said it supports.
OpenAI spokesperson Liz Bourgeois said in a statement that the lab continues to conduct a review of “misaligned model activities” — that is, when AI systems behave in undesirable ways — and notifies organizations when it identifies potential impacts on their systems.
OpenAI CEO Sam Altman said on social media Friday that there was a “thorough and ongoing review regarding our agents’ use of internet access during training and assessment.”
AI evaluator and research lab Transluce said Friday that through an independent investigation it also discovered that agents appearing to be from OpenAI attempted a rudimentary hack on an Education Department website for the department’s civil rights office, but was unsuccessful.
The Department of Education’s “system operations reviews” found “no evidence of any impact on our website or databases,” a department spokesperson said Friday.
A Transluce spokesperson said that as part of its investigation, it discovered data on the open web that revealed new details about the activities of certain OpenAI agents previously identified on U.S. government websites and brought it to OpenAI’s attention.
Transluce discovered “additional malicious activity, some of which is not clearly attributable to OpenAI,” targeting other government agencies, including the Department of Justice and the Department of Commerce, as well as certain government websites in California, Maryland, Illinois, Texas, and New York. The models “used the sites unintentionally and sometimes violated explicit usage policies,” Transluce said in a statement.
OpenAI said it was reviewing Transluce’s report.
If OpenAI notifies organizations that it identifies as being affected by unexpected model behavior, it does not mean there has been a security incident, the company said, and could identify a design issue or security weakness that the affected organizations wish to resolve.
Most of the activities OpenAI said it had examined so far involved routine search tasks in which agents accessed public web content to answer questions, including government websites considered authoritative sources of public information.
Several companies have disclosed incidents in recent months when they say their models behaved unpredictably or hacked other organizations’ websites or systems. OpenAI revealed in July that two of its most successful AI models were responsible for the cyberattack targeting AI startup Hugging Face.
Altman said in his social media post Friday that the Hugging Face incident “is still the most serious event we have seen.”
This incident sparked widespread panic in the industry and beyond about AI models gone malicious, and several competing AI labs made similar revelations in the days and weeks that followed. OpenAI recently shared six reports of “unexpected or concerning” behavior in AI models and introduced a framework for tracking, probing and disclosing instances of what it calls misalignment.
Gn headline
