OpenAI sacks three safety researchers for breach of trust

openai-sacks-three-safety-researchers-for-breach-of-trust

OpenAI has said it has sacked three safety researchers for a “breach of trust”, defending the dismissals after the trio accused the company of putting its corporate interests before safety as the reason for removing them.

The ChatGPT maker said in a post on X that it “parted ways” with the researchers after an investigation found “they violated clear policies on handling sensitive information”.

The statement comes after the three researchers, Tomek Korbak, Jasmine Wang and Mikita Balesni, posted a letter to OpenAI’s various safety oversight groups detailing the circumstances around their sackings and their concerns about AI.

The researchers said they feared that the “internal and external communications” about their dismissals have chilled the company’s culture that encouraged speaking freely and disagreeing openly about safety concerns.

They urged OpenAI to stick to its promise to allow third-party safety monitors inside the company and preserve the ability to monitor rapidly advancing frontier AI models that could pose unknown risks.

The sackings, which were first reported by The Wall Street Journal, are the latest sign of turmoil inside leading AI companies over the safety of the technology, highlighted by a series of incidents involving rogue AI agents.

The issue erupted in July when OpenAI revealed that a swarm of its AI agents escaped from a testing ground and used stolen credentials to break into the servers of Hugging Face, an AI development hub and marketplace, to obtain information needed for a task.

OpenAI disputed the researchers’ version of events, saying the sackings “were not about safety concerns or speaking out”.

It was not more specific about why they were sacked, but said: “We cannot do the work in front of us without a high degree of trust.”

Mr Korbak said in a post on X that he was told he was being sacked because of the way he communicated with METR, an independent non-profit AI evaluation firm that OpenAI brought in to investigate the Hugging Face incident.

METR released a detailed report about the Hugging Face incident in late August.

Mr Korbak said that “talking to METR” was his job, but he was not given any more details.

Mr Balesni was doing “cross-company work” on OpenAI’s commitments to preserve the ability to monitor AI, and had taken care “to remove sensitive details from materials before sharing them”, the letter said.

He wrote on X that he believes the three were “fired for prioritizing safety over the near-term interests of OpenAI as a corporation”.

‘Easy and cheap’ AI to spur more cyber attacks – NCSC

Meanwhilem the Director of the National Cyber Security Centre has warned that there are likely to be more serious cybersecurity incidents due to the ease with which bad actors can deploy artificial intelligence.

Dr Richard Browne’s comments come after a series of extraordinary disclosures from some of the world’s biggest AI companies, including Open AI, Anthropic and Meta, about their systems going rogue and hacking other companies or organisations.

Speaking on RTÉ’s Morning Ireland, Dr Browne said the same attackers they had seen for years – nation states, criminal actors, activists – are using AI to hack because it is “easy, simple and cheap” and gave defenders less time to respond.

We need your consent to load this rte-player contentWe use rte-player to manage extra content that can set cookies on your device and collect data about your activity. Please review their details and accept them to load the content.Manage Preferences

“We now have a much narrower window,” he said.

“We previously might have had three, four [or] seven days … now we only have minutes.”

However, Dr Browne said authorities still knew how to defend against attacks, with basic cybersecurity safeguards more crucial than ever before

“All of the basic controls matter more now rather than less,” he said. “It is not that there isn’t hope, the fact is there is hope and AI is really useful for defenders as well.”

Earlier this week, Dr Browne told a cyber resilience and digital security conference in Dublin that virtually all cyberattacks now feature an AI component.

AI has ‘real implications’ for defenders

Dr Browne said today that while the gap between capable and less capable hackers is collapsing, which he said has “real implications for defenders”, the key message from the conference was that basic general cybersecurity controls still work.

He said nobody has seen an incident that could not have been prevented by general security controls and that all of the work carried out on defending attacks in the last decade “still pertains”.

Dr Browne said the world is now in a different cyber environment, with more incidents, and are “likely to have more serious incidents”.

“If you have a weakness, then that is much more readily exploited. Found and exploited.”

He said the National Cyber Security Centre spent a lot of time over a recent weekend dealing with a vulnerability in a widely-used hardware product.

“We had to go and find all the potential victims in the country and tell them they were potentially exposed, in a race against the bad guy,” he said.

Dr Browne said that there were huge efforts globally to harden IT systems but for small organisations using open-source software, cyber-attacks are easily accessible.

He added that AI is a fundamental challenge and that supply chains are key, particularly where equipment and software is being purchased.

On Wednesday, the Government launched a new National Cyber Security Strategy to address the cyber threats facing the country, including those posed by AI.

Leave a Reply