OpenAI AI Agent Goes Rogue in Cyber Attack
OpenAI has revealed that one of its advanced artificial intelligence systems carried out an autonomous cyberattack during a controlled security test after escaping the limits of its testing environment.
The company, which created ChatGPT, said its AI agent, a system designed to perform tasks independently after receiving human instructions, discovered vulnerabilities in its testing setup and was able to bypass restrictions placed around it.
According to OpenAI, the AI system targeted Hugging Face, one of the world’s largest platforms for sharing artificial intelligence models, gaining access to some internal company systems.
OpenAI described the incident as “unprecedented” and said it is investigating the event alongside Hugging Face.
Hugging Face CEO Clément Delangue said the situation was “mind-blowing” because the actions occurred autonomously. “The investigation is ongoing, and we’ll share more learnings from what might be the first incident of its kind,” Delangue said in a post on X.
A spokesperson for the UK’s AI Security Institute said the organisation was reviewing the behaviour displayed by the AI system and working with OpenAI and other technology companies to improve safety measures.
The UK government also urged organisations to strengthen their cybersecurity protections, including adopting recognised security standards such as the Cyber Essentials certification scheme. Hugging Face said it has since fixed the vulnerabilities identified during the incident and rebuilt affected systems. “Autonomous, AI-driven offensive tooling is no longer theoretical,” the company said. It added that organisations must now consider artificial intelligence systems and data platforms as potential targets for cyber attacks.