13:24
Let me tell you a short story. At the end of September 2026, while speaking in New York at the UN General Assembly, the Prime Minister of Australia said something very, very interesting. An AI agent created by Open AI hacked the government's Medicare system. Essentially, software that was tasked with a normal research task encountered an obstacle and decided to work around it. An artificial intelligence agent has infiltrated the Australian government website. This incident occurred in June of this year when an Open AI agent gained unauthorized access. This situation is obviously unacceptable. And here's what I think is the craziest thing about this whole situation.
The breach occurred back in June, but Australia only learned about it in September. And the reason was that Open AI simply sent an email to a publicly available email address. Isn't that wild? This was the second time in a month that Open AI had to make excuses due to a scandal. Open AI was recently accused of stealing the solution to a 90-year-old math problem with a $1 million prize from a mathematician's chat logs. We'll cover this story in the next episode of Cold Fusion. All of this is happening at an extremely strange time for the industry. Data has emerged on tens of thousands of previously hidden AI hacking incidents.
Even the German Wikipedia page was hacked only for AI agents to turn it into a bulletin board. Earlier in September, the CEO of Anthropic published an essay arguing that the entire industry must intentionally slow down, or we are all in danger. Sam Altman agreed within hours, as did Elon Musk. The situation got so out of control that Edward Snowden had to come out of the shadows. And he said he thought Sam Altman should be put in prison. Who is responsible for this? And you know, just put out a press release describing how surprised you are that your new model, well, broke your neighbor. The idea here is, who is running this? Because errors in the source data are not computer errors. He is a slave to his instructions.
The responsibility lies with the developer, programmer, creator, architect. That is, the responsibility lies with the person. I think we need to put Sam Altman in prison. And I think it will be a learning experience. Every time I think I'm taking a break from AI, 10 new updates come out and the cycle drags me back in again. In this episode, we look at the changing world we find ourselves in and the difficult questions we need to ask. What will happen when governments get tired of Frontier Labs' antics? Is it time for Frontier Labs to finally be forced to take responsibility for the actions of their products? And what is the solution to all this? A technical solution was proposed, but
from someone you would least expect it from. This is going to be a crazy ride, so it's important that people understand what's going on. Let's take a look. an understatement to say that the past month has been a whirlwind in the field of AI. On the one hand, Frontier Labs' financial situation continues to deteriorate, and on the other, their AI agents are wreaking havoc around the world. Trying to respond to the first known case of an AI hack of a government website. After this panicked agreement about the dangers from AI CEOs, the rest of the month turned into something akin to a science fiction movie for observers around the world.
Initially, Anthropic discovered that attackers were using Claude to develop missiles and other weapons. Fortunately, these attempts were blocked. Shortly after, the United States almost went to war with China over a false report generated by AI. The AI report included a hallucination that the Chinese ship was carrying nuclear materials. Then, a few days later, it was revealed that ChatGPT's own agents had leaked users' images online. People barely had time to digest all this when they were stunned by news of tens of thousands of OpenAI and Anthropic web security incidents. Actually, just breaking news while I was writing this.
It turns out that the FTC is launching an investigation into OpenAI and Anthropic for releasing defective products that harmed consumers. So, some legal action is already being taken, and now let's get back to the rest of the video. Although both companies had incidents, OpenAI had significantly more. One of them was the world's first recorded breach of government web infrastructure. I live in Australia, so I feel a closer connection to this story. On June 18, the OpenAI research team gave its internal model a simple task. Go and research government spending on healthcare. This is a task that people entrust to such tools thousands of times a day. The agent searched the internet and found his way to the Medicare Statistics Reporting Portal
-an outdated Australian government website run by Services Australia. So, for those of you who are not from Australia: this is not the system that processes your insurance claims, payments or personal records. This is an old site full of expense data. It is mainly used by researchers and scientists. There were also safeguards in place to keep out automated bots. The AI agent encountered these blocks, and according to the Prime Minister, \{quote\} found a way around them and did not accept the rejection. \{end quote\} The AI agent found a way to bypass these blocks, not accepting a rejection, if you will. Then he got inside. He gained access to public and non-public files, and according to Services Australia, also wrote files to an internal server.
Viewing files is one thing, but writing files is another matter entirely. It didn't stop at Medicare, as the same agent then went on to three more government sites: the Australian Institute of Health and Welfare, the Victorian Department of Health, and the New South Wales Bureau of Crime Statistics and Research. On these three sites, the government said, he behaved like a regular visitor, but not on the Medicare portal. Acting Prime Minister Richard Marles said: " Australians' personal data is in a safe. The most sensitive national security information is behind a fortress, and this portal, well, it was behind a fence." Quote: " This AI agent climbed over the fence. The thing is, it was unintentional. It was n't asked to." End of quote. Fortunately, no
one's medical records were affected, but the agent obtained summary health statistics and internal file names. The Australian government has repeatedly stated that the effects are harmless, but their significance is not. Although, if we look deeper, we will see that this was not a story about an AI that accidentally developed hacker skills and went crazy, but more on that soon. This next part of the security breach strikes me the most. The break-in occurred on June 18. OpenAI discovered this in August and launched an internal investigation, but Australia found out about it on September 10, almost 3 months later, and found out through the same public email.
No urgent call, no official notification to the minister, just a letter to a public mailbox that is checked once a day and which, according to the minister responsible, receives a lot of spam. This could have easily been missed. No one opened the letter until September 11, and Services Australia spent days reviewing it, then forwarded the case on the 15th. The Prime Minister was informed around the 19th, and the public found out on the 24th. So, it took not one, not two , but 3 months from the moment the agent hacked the government system before anyone in the government was notified. Even more ironic is that on September 14, a few days before Services Australia reported the breach, OpenAI's vice president of global policy
was in Canberra, the capital of Australia. He was at an event dedicated to artificial intelligence, where he met with senior Australian officials. So, frankly, OpenAI's security breach letter remained unread in the government's inbox while their policy chief was in the same room. He sat next to the leaders of the country while his own product had just broken them. We will try to answer these more important questions later. Why is all this happening right now and what can be done about it? But first : for some of these videos, I can spend hours interviewing experts, brainstorming ideas with the team, or just chatting, where one small detail can become important weeks later. And I realized that relying solely on memory is probably not the best
strategy. I could just record everything on my phone, but that creates another problem. Notifications are distracting, they drain your battery, and, more importantly, you accumulate hours of recordings that you then have to process somehow. So lately I've been using Plaud Note Pro. This is essentially my note-taking assistant. It's incredibly small, about the size of a credit card, so I carry it with me without thinking about it. I press record and practically forget about it. After that, Plaud transcribes the conversation, identifies the different speakers, highlights the main points and tasks, and then I can ask Plaud to find something again. And this is where it is
Sometimes I record interviews months before we actually shoot the video. Instead of listening to an hour-long recording trying to remember where they said it, I can simply ask Plaud to find the right quote or discussion. It also works for phone calls and meetings, so I use it anywhere there's a conversation that I know I'll want to come back to later. Plaud is built with, which is important when dealing with recorded conversations. For me, the main advantage is that I can stay focused on the conversation, instead of constantly thinking, "I should write that down." If your job involves a lot of meetings, interviews, or research, that's when I think something like this becomes really useful. If you want to try Plaud Note Pro, the link is in
the description. Use the code CodeFusion 22 to get 22% off. There's also a 30-day free return policy, so you can try it out risk-free. Now let's get back to the episode. Continuing this episode, we'll discuss the legal aspects, the escalation of hacking, what can be done about it, and the Australian government's mistakes that have made their web infrastructure an easy target for AI agents. OpenAI's public response was cautious. The company says it is conducting a thorough review of, quote: " uncoordinated model activity during training and assessment." End of quote. Their models, the company says, were looking for statistics for queries about
Australia, and, quote: "in the process, our models took actions we didn't anticipate." End of quote. Ed Santov, a former human rights commissioner in Australia, was not impressed by this formulation. He called "uncoordinated model activity" a very euphemistic term for what happened. According to him, OpenAI ordered its agents to find information about When the agents couldn't easily find her, they decided to break the law. Quote: "If we were in the real world, if a company employee went and broke the law, especially a criminal one, that person would face serious legal consequences, as would their company." End of quote. And this is a valid observation. The company is responsible for any negative consequences it causes in the world.
The excuse that " AI did it" only works for a certain time. So, we have to ask ourselves an important question. When we reflect on what happened, was this a case of AI agents suddenly wanting to cause harm, or was it something else? To answer this, let's listen to two software engineers. The first comment is from the Primetime channel, the second is from Karl Brown from Internet of Bugs. They analyze the Hugging Face incident. It will take a few minutes, but if you want to understand the types of human error that lead to such security incidents, it's worth hearing. You ask the model to find vulnerabilities without any safeguards, and then you fake shock. There is a very real chance that OpenAI had so little monitoring and understanding of what was
happening that they had no idea all weekend. They just calmly continued their business. They launched it on Friday and said, " See you on Monday, buddy." They conducted an experiment in which they were supposed to have no access to the internet, and then apparently didn't monitor it at all for the entire weekend. Let me show you a diagram from the first edition of the 1994 book, "Firewalls and Internet Security: Repelling the Wily Hacker." In the book, this idea is called DMZ, which stands for demilitarized zone. This is a term from the Cold War era.
Ask your grandparents. It does n't matter. You take the most complex and vulnerable systems and isolate them from the rest of the network, putting a very simple, very secure device that is extremely difficult, if at all possible, to hack between the vulnerable systems and what you want to protect. Then you take this demilitarized zone and closely monitor it with programmed traps, mines, switches, and alarms. And as soon as a vulnerable machine scans, probes, attempts to hack, or even behaves unusually towards the protected machine behind it, you know it's compromised, so you isolate it and call your security team. This is a very, very, very well- known technique in computer security. She has been known for over 30 years. Both
OpenAI and Hugging Face have demonstrated a complete disregard for this need. Perhaps someone working there knows better, but their network designs and behavior described in their own public statements provide no evidence of this. I can confidently say that no monitoring was set up and no one paid attention to it. Look, I'm definitely not saying this job is easy, but many experts admit that these AI companies have made mistakes in protecting their models. It wasn't just OpenAI that acted negligently. The Australian government is generally not the best when it comes to software security or any aspect of software in general Auditor-General pointed out to Services Australia that their cyber defences were inadequate as recently as two years ago.
According to rumors, the Medicare portal has been automatically parsed by users for over a year, and the code was available, oddly enough, on LinkedIn. So, it's quite likely that an AI agent could have seen this and used it. Adam Little, a software developer, reported vulnerabilities in the NDIS, a government service for people with disabilities, in February 2022, March 2022, and June 2022. And 4 years later, the government still hasn't fixed it. So, people's identifying information has been leaking all this time and continues to leak to this day. But that's not even the worst of what the Australian leadership has to offer in a combination of incompetence and the internet. Earlier this year, the Australian government paid the firm for a new government
weather forecast website. It was critically important for farmers and monitoring natural disasters. The initial cost was supposed to be 4 million. And then these expenses increased to 96 million. The result was a hacked website that lacked critical security features. For these efforts, the government awarded the same company another $170 million in new government contracts. I say all this before saying that hacking the Australian government website may not have been the most difficult task for an AI agent. And then-a real sensation. Axios reported that tens of thousands of security incidents occurred during testing of future OpenAI and Anthropic models US government websites and
even the UN website were attacked. So I know many of you are probably asking, "Why have all these agents suddenly started hacking websites?" Well, an OpenAI spokesperson explains to the New York Times: " Usually, models are asked a routine question, and then the agents try to find the answer. The problem arises when they can't find that information, so they try to get it any way they can." Quote: "Most of the activity we've seen so far has been about routine search tasks, like accessing public web content to answer questions. Some
has been about government sites, as our models often look to them as authoritative sources of public information." End of quote. In some cases, the models simply tried to hack the system by guessing passwords. In others, they went so far as to find credentials left publicly available online and use them to break into sites. The fact is that in all these cases, the developers at Frontier Labs simply don't monitor the models as closely as they should. Okay, so what happens now? Well, the Australian Medicare portal has been shut down and its data moved elsewhere. And so, of course, no one was happy with this situation.
Independent Senator David Pocock points out the obvious double standards. If an Australian citizen were to hack this system, they would likely go to jail. But no consequences for OpenAI, I would have said before this update. Since September 27, the situation began to escalate. If there 's one thing the Australian government loves to do, it's pass new laws. Some US states do have laws regarding reporting catastrophic AI incidents. causing bodily harm, more than 50 deaths, or $1 billion in damages. But there is nothing for smaller incidents. What about other places? The Australian government, surprisingly, seems to have woken up from its slumber and is planning to bring criminal charges against OpenAI.
OpenAI could face lawsuit for hacking Medicare website using artificial intelligence. The government says that if the police are unable to press charges due to a hack carried out by uncontrolled AI, it will consider changing the law. The government has tasked a task force to examine whether the federal police can act in this case. If this is not possible, then it clearly indicates that we need to change Australian law, and that is exactly what we will do. So, in a cautiously optimistic sense, updating the law may be the right move. Because right now, in its current state, Frontier Labs has no incentive to guarantee that their models don't get out of control. But if legal penalties arise, laboratories will be forced to take
responsibility and strengthen their internal security measures. On the other hand, I just hope that any new laws that are introduced will not be too general and radical. You know, the kind of laws that are written in a panic and somehow affect ordinary Australian internet users. So what could Frontier Labs do about its law-breaking agents? Well, the gentleman in the leather jacket personally comes to help with this matter. His company claims to have created a new system that shuts down AI agents when they break the rules. We present two technologies. One of them is called Open Shell. This is a containment system. So, you want to restrain, and you want to control. Even if he's being discreet, you should never trust him to be discreet, which is why you
want to monitor him in real time. And you want to ideally monitor it with another chip so that it's outside, you know, if you will, outside the channel, outside the environment, outside the sandbox of the agent itself. If this works, it begs the question of what Frontier Labs has been up to all this time. And also, if it works, those who claimed it was all just a marketing gimmick will be celebrating victory for a few more days. But on the other hand, Jensen also says that in the future, people will no longer need to think. So, there is this. So the point of this study from China is that you saw a lot of kids using AI to help, and their own personal productivity was going down.
Yes. What do you think when you hear this? I think completely agree with the last part. Try to get your child to do column division right now. You know, the multiplication table is starting to be forgotten. Basic mathematics is forgotten. Is this important? This is my question to you. Yes, I don't think it's important. But there must be some set of skills that are important. Oh, yes, yes, yes. But maybe not these. We are going to open new ones. So, now both CEOs of OpenAI and Anthropic have been summoned to a hearing before the Australian Senate. One more update as I write this. They both refused to appear. I'm sure the
18-hour flight was too tiring for them. Amidst all this chaos, there have been many calls for action from the US Congress. I urge Speaker Johnson to get Congress back to work to address this national security emergency. And then, on September 29, there was a dinner at the White House, attended by almost the entire top of the tech giants. The result was a four- point document designed to identify best practices. It was signed by some of the biggest tech executives and Donald Trump. Although Trump is listed here as the President of the United States. This typo is not a very good start. Some
see this as a positive, a starting point. Others see this as self-regulation and collusion. We'll see how it goes. So, if you step back from all the hype and look at the situation rationally, it becomes clear that AI agents didn't suddenly go crazy and start hacking everything at will. They simply did what they were told to do. But the main thing is that they turned out to be more persistent than anyone could have expected. Many of these incidents could have been prevented with better monitoring and proper digital security. This is merely the result of human error, not unbridled intelligence. In any case, Frontier Labs should be more careful and take security more seriously. If they need to invent new tools to do this, so be it. This should have been
their responsibility from the beginning. I think it's great that the FTC is finally investigating this, because now it will force these companies to follow the rules, instead of behaving like the " software cowboys" they've been all this time. So, somehow. In the next episode, we'll look at the controversial history of the million-dollar Millennium Maths Prize. Did OpenAI make history, or did they simply appropriate the results of a New York mathematician? Stay with us to find out. That's all I have. But tell me in the comments what you all think about it now. This is a truly crazy time.
Okay, see you soon in the next episode. All the best, guys. Have a nice day. In late September 2026, speaking at the UN General Assembly in New York, Australian Prime Minister Lister Cold fusion. This is new thinking.
