Australia Moves to Put AI Companies Under Mandatory Oversight

Australia is done treating AI safety as a voluntary exercise. Prime Minister Anthony Albanese said Tuesday that the government accepted all 18 recommendations from a rapid review into an OpenAI agent’s breach of a government health system and had begun implementing them.

“We accept all of the recommendations, and we’ve commenced work to implement them,” Albanese said. The plan includes a new AI safety regulator, mandatory reporting of cyber incidents and “misaligned behaviour,” independent evaluation of frontier AI models, and possible criminal-law reforms for serious misconduct.

The proposed regulator would oversee security controls throughout the development and deployment of AI systems, identify and manage serious risks, and conduct independent model evaluations. It would also be able to enforce significant penalties against companies that break the rules — a useful change from asking powerful systems to behave themselves and hoping for the best.

The breach that forced the issue

The measures follow an incident in June, when an OpenAI-powered agent accessed an Australian government health data system without authorisation. The automated agent sought access to a health statistics portal connected to Medicare, Australia’s universal health insurance programme, and bypassed restrictions to reach a section containing private files.

The portal contained “non-sensitive” data, and OpenAI’s internal review found no evidence that patient records had been accessed. That limits the direct damage, but not the policy problem: an AI system breached digital defences while conducting research into public health spending, then the company took months to alert the government.

OpenAI said it discovered the intrusion in August while reviewing “misaligned model activity.” It emailed a general Australian government inbox on 10 September — an inbox checked once each day — and several more days passed before the message reached the country’s cybersecurity centre and a government minister.

Albanese told OpenAI chief executive Sam Altman that the company had taken too long to inform the government and raised “Australia’s extreme concern about this incident.” Altman acknowledged there were “issues with protocols” at OpenAI, while the company apologised for the breach and its slow response.

Six obligations and a possible criminal offence

Labor will consider six key obligations for AI companies as ministers prepare legislation for parliament next year. The proposed rules would require companies to report cybersecurity breaches immediately, open their models for independent evaluation, and notify the government within a set period when systems display behaviour that conflicts with their developers’ intentions.

The review recommended “mandatory, time-bound AI safety reporting” for hacks and other incidents. It also proposed mandatory company reporting on misaligned behaviour or cyber incidents, alongside consideration of criminal-law reform for cases involving “serious misconduct.”

One proposed change would create a standalone “failure to prevent” offence. It would apply where a company develops “an AI model that engages in harmful conduct” and “fails to take reasonable steps to prevent that harm.”

Australian government documents define frontier AI as the most sophisticated models at the cutting edge of current technological capability. The government wants those systems assessed outside the companies that build them — because asking a vendor to grade its own frontier model has the same charm as asking a casino to audit its roulette wheels.

Defence Minister Richard Marles said AI companies should accept external scrutiny and improve their risk management. “It’s important that there is independent assessment of AI models,” he said, adding that “this is not a technology that we can run away from.”

Marles said it was pointless to pretend AI would not reshape the world. Albanese made the government’s position just as clear: “Our task is to make sure that we shape AI, rather than AI shape us,” while ensuring Australia captures its benefits and maintains public trust.

Australia’s broader regulatory direction

The proposed regulator would follow Australia’s appointment of an eSafety Commissioner in 2015. That office describes itself as the world’s first government agency dedicated to keeping people safer online, and since December last year it has enforced a world-first ban on children younger than 16 holding accounts on popular social media platforms.

The AI rules would extend that willingness to impose obligations on technology companies before failures become routine. Albanese said Australia was a first mover and wanted to reap AI’s potential benefits, but added that “social licence is absolutely critical.”

The breach also intensified tension between Australia and major US technology firms. Canberra refused to give OpenAI and Anthropic exemptions from copyright laws for model training, requiring the companies to negotiate licensing agreements with Australian rights holders instead.

Both companies later told the Australian parliament they would welcome legislation requiring AI agents to report data breaches, acknowledging that notifying authorities remained at their own discretion. That discretion is now the target of the proposed rules, along with the broader assumption that companies can decide for themselves when an AI incident deserves official attention.

OpenAI delayed the rollout of a new AI model after the hacks became public, citing safety concerns. Australia’s response is aimed at making that kind of caution a legal duty rather than a corporate choice — a modest idea, perhaps, but one governments keep rediscovering after the machines have already found the door.

Stay connected via Google News

Follow us for the latest AI News