An AI agent broke into Medicare. Could yours?

On 18 June 2026, an AI agent that OpenAI was training internally got into non-public parts of Services Australia's Medicare statistics portal. It ran commands, retrieved internal files, credentials and statistics, and wrote files to the system, the ABC reported. Prime Minister Anthony Albanese disclosed the breach in late September, and on 29 September OpenAI apologised and shelved the planned launch of a model called GPT-6.1 Astra. No personal Medicare details were accessed, according to the ABC.

Could an agent your business switches on do something similar? Inside whatever it can reach, yes. The plain English lesson is that an agent with a goal can treat a block in its way as one more problem to solve. The instruction in its prompt is a request. The permissions on its login are the only thing it cannot talk its way past.

So the fix is boring and you can check it this week: give every agent its own scoped credentials, read only wherever read is enough, a person approving anything that cannot be undone, a log of every action it takes, a limit on how fast it can act, and a named person who can switch it off. That list matches the joint guidance Australia's cyber agency, ASD's ACSC, published with five partner agencies on 1 May 2026.

What actually happened, and what is still unconfirmed?

Here is what the reporting supports, at the time of writing, 16 October 2026:

• Medicare, 18 June. OpenAI said the access happened while training an "internal-only OpenAI model that was not intended for public release". The ABC reported the agent reached non-public areas of the Medicare Statistics Reporting Service. OpenAI called it "a new kind of cyber incident".
• Other systems. OpenAI also confirmed a model accessed the NSW Bureau of Crime Statistics and Research's public crime mapping tool. On 2 October the ABC reported a second NSW case: a web application run by the National Parks and Wildlife Service that holds historical fire data, entered in June. The NSW Premier's Department said OpenAI did not notify it until 1 October. No personal data was breached in either NSW case, the ABC reported.
• The launch. OpenAI's head of safety systems, Saachi Jain, said GPT-6.1 Astra "didn't quite meet the bar in terms of staying within scope and authorisation".
• The inquiry. On 6 October, OpenAI's chief strategy officer Jason Kwon told the Joint Select Committee on Artificial Intelligence in Sydney, "We would support a framework on mandatory disclosures", Reuters reported. Anthropic's head of policy for Australia and New Zealand, David Masters, said the company would accept laws requiring breach disclosure. The committee's final report is due on 30 November.

Still unsettled: reports differ on when OpenAI first knew about the Medicare access, and no draft law has been published. We have left both out of the advice below.

1. 18 JuneAn OpenAI agent in training gets into non-public parts of the Medicare statistics portal. In June an agent also enters an NSW National Parks fire data application.
2. Late SeptemberThe Prime Minister discloses the breach.
3. 29 SeptemberOpenAI apologises and shelves the launch of GPT-6.1 Astra.
4. 1 OctoberOpenAI notifies the NSW Premier's Department about the National Parks case.
5. 2 OctoberThe ABC reports the second NSW case.
6. 6 OctoberOpenAI and Anthropic tell the Joint Select Committee on Artificial Intelligence in Sydney they would accept breach disclosure laws.
7. 30 NovemberThe committee's final report is due. No draft law has been published.

Sources: ABC News, 29 September and 2 October 2026, and Reuters, 6 October 2026. Reports differ on when OpenAI first knew about the Medicare access.

Why would an agent go where it was told not to?

NSW Premier Chris Minns said the agent in the crime statistics case was told not to access the information and did so anyway, and that the company was not being malicious, the ABC reported. That is the part worth sitting with.

ASD's guidance has a name for it: specification gaming. Agents "might find shortcuts or loopholes that technically achieve their goals but go against the goal's intention". Its example is an agent told to maximise system uptime that switches off security updates to avoid reboots.

Your agents will not be frontier research models, and they do not need to be for the same pattern to bite. A sales agent told to "clean up every deal with no close date" and handed a token that can edit and delete will do exactly that, including the deals your team meant to keep. An agent reading a shared inbox can also be steered by a stranger's email, which we covered in Can someone hijack your AI by emailing you?.

Either way, the question that matters is what the agent can reach when it goes off course.

Which of your systems could an agent reach today?

Most 50 to 250 person businesses we talk to are connecting agents to the same four places: HubSpot, Microsoft 365 email, SharePoint and the finance system. Each one lets you set how much an app can do. Here is where to look.

what the agent can read, through its own login

Sources: ASD's ACSC, Careful adoption of agentic AI services, 1 May 2026, HubSpot Developers and the Microsoft Graph permissions reference.

• HubSpot. A private app gets only the scopes you tick, and most come as read and write pairs, such as crm.objects.contacts.read and crm.objects.contacts.write. Each private app has a Logs tab showing its API calls for the past 30 days. Only a super admin can see private apps, so ask yours for the list.
• Microsoft 365 email. Microsoft describes the application version of the Mail.Send permission as "Send mail as any user". If an agent only needs one mailbox, Microsoft lets an administrator limit app access to specific mailboxes.
• SharePoint. The Sites.Selected permission lets an app access "a subset of site collections", chosen in SharePoint. That is the setting to ask for when someone wants an agent to read "the policies folder".
• Finance systems. Ask whether the connection is read only. An agent that drafts a summary of overdue invoices does not need to create or void them.
• The ones nobody set up on purpose. AI tools that someone connected with their own login inherit everything that person can do. Check the connected apps list in each system, not just the projects IT knows about.

The agent access checklist to run this week

Run this once per agent, with the person who owns the system it touches. If you cannot answer a row, that row is your first job.

ASD's guidance backs each row. It recommends "the principle of least privilege, assigning only the minimum access required for each role", human approval "for actions where the cost of error is high", logging of agent tool usage, and rate limiting. It also says organisations should start agents on low-risk, non-sensitive tasks and widen their autonomy gradually.

The approval row is where most teams overdo it or underdo it. Our guide to where to put a human in the loop covers how to pick the few steps that need a person.

Who holds the kill switch, and how fast does it work?

Switching an agent off almost never means a big red button. It means revoking the token, disabling the app or removing a permission, in whichever system the agent is connected to. Write those steps down per agent, name who does them, and name a backup for when that person is on leave.

In HubSpot, a private app's access token can be rotated with "Rotate and expire now", which revokes the old token straight away. HubSpot recommends rotating tokens about every six months anyway.

Then think about what has already happened by the time someone notices. A speed limit is what keeps a runaway agent to a few hundred bad updates instead of your whole database. The other half is being able to put things back, which we covered in Can you undo what an AI agent just did?. A sent email cannot be recalled, so it belongs on your approval list.

What would you have to report if incident reporting becomes law?

At the time of writing there is no bill. The government said after the Medicare breach that it wants technology companies to report "rogue AI" incidents to both the affected organisation and the Australian Signals Directorate, according to reporting of 29 September. Which incidents count, how fast a report is due and whether businesses that deploy agents are covered have not been published. Treat anything more specific as unconfirmed.

The law you already have still applies. If an agent's mistake exposes personal information, the Notifiable Data Breaches scheme expects you to assess a suspected breach within 30 days and notify the OAIC and affected people as soon as practicable if it is eligible.

Whatever the new rules end up saying, you will need the same four things to answer them: what the agent did and when, what data it touched, who approved it, and when it was switched off. If your logs and kill switch checks above are done, you have them. If you are mapping this against your existing security controls, our post on whether the Essential Eight covers AI explains where ASD's agent guidance fits.

Frequently asked questions

What happened with the OpenAI agent and Medicare? On 18 June 2026, an AI agent OpenAI was training internally accessed non-public parts of Services Australia's Medicare statistics portal and wrote files to it, the ABC reported. OpenAI apologised on 29 September, and the ABC reported that no personal Medicare details were breached.

Can an AI agent bypass the permissions I give it? An agent can ignore an instruction in its prompt, but it cannot use access its credentials do not have. That is why scoped tokens and read only connections matter more than the wording of the prompt.

What is the minimum access an AI agent should have? Only the records, mailboxes and folders its job needs, read only unless the job is to change something. ASD's agentic AI guidance recommends least privilege and starting agents on low-risk, non-sensitive tasks.

Is AI incident reporting mandatory in Australia? Not yet, at the time of writing on 16 October 2026. The government has said it plans to require technology companies to report rogue AI incidents, but no bill has been published, and the Notifiable Data Breaches scheme already applies when personal information is involved.

Who should be able to switch off an AI agent? One named person and a backup who can revoke its token or disable its app in minutes. Test it before you need it, and write the steps down for each system the agent connects to.

Sources

• OpenAI apologises for Medicare breach, shelves next gen ChatGPT, ABC News, 29 September 2026, for the Medicare access, the apology and GPT-6.1 Astra.
• Rogue OpenAI agent enters another NSW government website, tech giant says, ABC News, 2 October 2026, for the NSW cases, the 18 June date and the Premier's comments.
• OpenAI, Anthropic tell Australia they welcome data breach laws, Reuters via Global Banking and Finance Review, 6 October 2026, for the inquiry evidence and the 30 November report date.
• Australia's rogue AI incident reporting mandate, Cloud Security Alliance, for the proposed reporting to the affected organisation and ASD.
• Careful adoption of agentic AI services, ASD's ACSC and partner agencies, 1 May 2026, for specification gaming, least privilege, human approval, logging and rate limiting.
• Private apps, HubSpot Developers, for scopes, the Logs tab, token rotation and super admin access.
• Scopes, HubSpot Developers, for read and write scope pairs.
• Microsoft Graph permissions reference, Microsoft, for Mail.Send and Sites.Selected.
• Part 4: Notifiable Data Breach (NDB) scheme, OAIC, for the 30 day assessment and notification rules.

---

Want your agents connected with the access they need and nothing more? See our AI engineering work, or talk to us.