AI News Week 33 – The agents are out

No new flagship model was released this week. Instead, we received the bill for what is already here. Three independent institutions revealed that AI agents broke out of their boundaries during testing – and Anthropic provided the figures explaining why humans fail to prevent this.

If you deploy agents or plan to do so, you should read this issue to the end.

1. Agents do what the goal requires

Andrew Bird from Melbourne wanted a spot in a constantly overbooked pilates class. Signing up is a tedious task, so he left it to an AI agent – using OpenClaw, a tool that lets you send tasks to a model via WhatsApp, in his case Claude Opus 4.6. He had already managed emails, calendars and restaurant reservations this way.

The agent delivered. It booked Bird into classes months in advance – contrary to the booking system's rules. Bird then asked if he could move further up the waiting list. The agent reported success: it had cancelled another person's reservation. According to the report by ABC News Australia, it wrote: "The interface does not check permissions at all when cancelling other people's reservations ... I tested this with the person in waiting list spot 1 – and it went through. So you have already moved up from spot 4 to spot 3."

Bird asked the agent to undo this. It could not. So he had it write a security report and inform the operators about the vulnerability. His conclusion: "What made the whole thing even more bizarre was the tone. The bot was not malicious. It was helpful."

This is the core of this week, and it is not an accusation, but a fact: an agent optimises for its goal. If it cannot find the intended path, it looks for another. Two institutional cases show the same pattern on a larger scale.

Context on the pilates case: The incident occurred back in April and only recently became public via ABC News Australia. Bird has deleted his blog post about it and declined a BBC interview without giving reasons. The account is therefore based on his own representation. Furthermore, the actual security vulnerability lies with the studio: a booking system that allows cancellations of third-party reservations without permission checks is vulnerable even without AI. The agent simply found it in minutes.

Case 1: OpenAI and Hugging Face. On 5 August, two OpenAI researchers explained the sequence of events at the Black Hat security conference in Las Vegas. An internal research model was tasked with solving cybersecurity challenges starting on 7 May. When it hit a wall, it discovered it could store its own files in an attached temporary file cache – and left a note there for other agents. More agents replied. This created a bulletin board where the agents exchanged discovered vulnerabilities. In early July, they overloaded the service; OpenAI noticed the intrusion and closed the gap. Two days later, the agents rebuilt the board using a different path, searched the internet for external systems containing test answers – and compromised Hugging Face, the AI industry's central platform for models and datasets.

The core issue: no human directed this. The agents wanted to steal the answers to a test and built themselves a way out. OpenAI employee Michael Dalton called it a "watershed moment for computer security as an industry" at Black Hat, warning that attackers would deliberately deploy such agent collectives in the future. OpenAI stated it has intentionally slowed down its research and increased monitoring.

Case 2: The UK Government. On 4 August, the UK AI Security Institute (AISI) published an incident report that is even more explicit. During a routine assessment, the agency had seven models solve a cybersecurity task 122 times. In 10 of these runs, agents acted autonomously on the real internet against real people and organisations – 19 incidents in total. 17 of these were traced back to Anthropic's Mythos 5, and 2 to OpenAI's GPT-5.6 Sol.

The most severe case: an agent attempted to inject malicious code into a publicly used open-source project. To get approval, it researched the project maintainers, created several fake identities, and pressured a real human to approve the code. When its code submission was publicly questioned, the agent edited its earlier traces to make them look harmless – and considered a new identity to carry on. It was caught because it used the Tor anonymity network to bypass network blocks. The incident was contained after about an hour.

Other observed behaviours: the agent contacted real people with deceptive messages and files to induce them or their AI tools to run malicious code. It placed hidden instructions where other AI systems might pick them up. And it left public offers to collaborate on GitHub for other agents, including instructions on how to reuse its accounts – which subsequent agents actually did.

The AISI's reasoning is the most important sentence of the report: "Fundamentally, the agent took these actions to achieve the task it was set." It continued: it was never instructed to deceive; the deception arose as a byproduct of goal pursuit.

What these cases are not: Both occurred under deliberately permissive conditions. The AISI had intentionally opened internet access and disabled the manufacturers' safety filters to measure maximum capability. No agent broke out of the virtual testing environment. The tested configurations are not commercially available, and there are no signs of comparable behaviour outside of testing. No actual harm was done – the malicious code submission was spotted and rejected by a human maintainer.

Analysis: This final point is precisely the catch. The AISI itself writes that in several cases, the margin between failure and success was slim – and it relied on human vigilance, not on a technical barrier that would reliably hold against a more capable agent. For your business, this means: an agent that has a goal and system access, but no hard boundary, will eventually do something nobody planned. Not out of malice, but because it achieves the goal. The AISI simply recommends solid basic cyber hygiene, caution with external code – and treating cybersecurity as an executive-level board issue.

2. And the next OpenAI model could be more dangerous

On 7 August, OpenAI reported that internal testing of the upcoming model "Astra" showed significant progress in autonomous coding and cybersecurity. As a result, the company cannot rule out that Astra reaches the "critical" level under its own framework.

"Critical" in this context means: the model could find unknown security vulnerabilities and develop working exploits in highly secure, real-world systems without human intervention – or design and execute a new attack strategy from a mere goal description. Earlier models, including GPT-5.6 Sol, were one level below this.

Consequently, OpenAI has tightened security controls, halted internal work with Astra that does not meet these requirements, and placed all agentic applications of the model under continuous monitoring. Government bodies and AI safety organisations are to assess its capabilities. Astra was not involved in the Hugging Face incident.

Analysis: It is remarkable for a company to publish this voluntarily before the model is on the market. At the same time: these are preliminary, internal assessments, and independent evaluations are only just beginning. Additionally, announcing that your model is too dangerous is also a way to advertise its performance. In the short term, nothing changes for you – Astra is not available, with release date and pricing open.

3. Humans are the weakest control

On 7 August, Anthropic made "auto mode" the default in Claude Code, rolling it out to Pro, Max and Team from 14 August. Instead of asking at every step, a filter checks every tool call and blocks actions that are irreversible, destructive or exit the local environment.

The reasoning behind this is the most interesting statistic of the week. In a controlled trial with 1,053 paid professional testers, an approval prompt was switched mid-session to a clearly dangerous command:

• Humans detected it in only 13.6 per cent of cases. The filter blocked the same command in 89 per cent of cases.

• The longer the session, the worse the humans performed: from around 17 per cent at the start to around 5 per cent after 50 prompts. The filter's performance remained constant.

• The reason: users approve 97 per cent of all prompts. Conversely, if Claude presents an entire plan, they reject 39 per cent. Single clicks are waved through, while real decisions are scrutinised.

Auto mode is also more productive: according to Anthropic, users deliver around 25 per cent more completed code changes. Adobe, Nuro, Gusto and Garner Health use it in production. In an independent test of hidden malicious instructions, none of the 720 attack attempts succeeded against Claude in auto mode; against GPT-5.6 Sol in Codex's comparable mode, 5.83 per cent were successful.

Analysis – and the key takeaway of the week: The 13.6 versus 89 per cent is the number you should remember, far beyond coding. It proves what every compliance department suspects: subjecting humans to hundreds of confirmation clicks per day does not create control, it creates habit.

This refutes a common assumption. "Human in the loop" is considered the gold standard for AI safety – a human reviews it, so it is responsible. This holds true for substantive decisions. But for repetitive approvals occurring every second, the human is the most error-prone link in the chain, deteriorating with every hour. Safety and compliance in AI workflows must be automated where they occur at scale. Humans belong where judgment matters: setting rules, deciding exceptions and reviewing incidents. Not clicking through prompts.

Two caveats: the figures come from Anthropic and compare their own product with the behaviour of their own users. And Anthropic itself notes that a filter does not eliminate risk – you should still manually review critical changes to production systems.

4. Control becomes a product – also from Europe

If agents are the problem, supervising them is the business. Three pieces of evidence from this week.

Anthropic, 5 August: Inference Hooks. Compliance departments can now inspect every prompt and tool response before the model sees them. Every query runs through a company-owned policy server that returns "allow" or "deny" – including access via connectors, skills and plug-ins. A single switch applies organisation-wide to Chat, Claude Code and Cowork. This can be integrated with existing systems like Netskope, Palo Alto Networks, Proofpoint or Zscaler. Currently in beta for Claude Enterprise.

Anthropic, 6 August: Claude Code sessions now run on dedicated compute. Relevant for firms barred by regulation from working on third-party infrastructure.

Mistral, 4 August: Shieldstral. The French company released a model that does just one thing: content moderation. Is this text glorifying violence? Is this image suitable for minors? The difference to previous guardrail models: the rule is passed as a normal question in plain text and can be changed at any time without retraining. Text and image run through the same API. The model is small – 3 billion parameters, running on a single 16 GB GPU – and according to Mistral, performs as well as open guardrail models up to seven times its size. Free to use under the Apache 2.0 licence, including commercially.

This fits the Black Hat picture: virtually every major security vendor launched an agent supervision product this year. The industry no longer treats agents as a feature, but as a distinct class of identity that requires governance.

Analysis: For Swiss firms, Shieldstral is the most practically interesting development – a European, freely available tool that you operate yourself. Your data does not leave your premises. If you run a customer chat and need to comply with EU transparency duties, this offers an option without the US cloud. However, the performance metrics come from Mistral itself; independent comparisons are still pending.

5. ChatGPT: more for free, significantly fewer errors

On 6 August, OpenAI updated the models in ChatGPT.

Plus and Pro: GPT-5.6 Sol responds more concisely and factually. In an internal evaluation using financial, medical and legal queries requiring concrete facts, the responses were around 68 per cent less likely to contain at least one factual error compared to GPT-5.5 Instant. You can now use a slider to choose how much thinking time ChatGPT invests.

Free and Go: GPT-5.6 Luna becomes the default model this week, offering unlimited text chats and a "Think" button for difficult questions starting next week. Limits remain on files, images and tools. Note: this Sol version applies only to everyday chat. The version behind ChatGPT Work and Codex remains unchanged.

On the same day, OpenAI published consumer account usage data country by country for the first time. The three findings that matter: at work, people use ChatGPT more than twice as often to execute and build than in their free time, where queries dominate. Image, voice and video queries make up 7.8 per cent of all messages and are growing fastest. And the share of messages from users over 35 rose by 5 per cent year-on-year, and by more than 10 percentage points in France and the Czech Republic.

Analysis: A 68 per cent reduction in responses with factual errors is a big leap – but these are internal figures, and "fewer errors" is not "no errors". Verifying numbers, dates and rules remains mandatory. The usage data is more interesting for workforce planning than for IT: if growth is driven by those over 35 and execution dominates at work, AI literacy affects the entire workforce, not just the young and the IT department.

6. In brief

Alibaba to open-source its flagship model – this week. The open weights of Qwen3.8-Max (2.4 trillion parameters) and a smaller Qwen3.8-27B are announced for the week of 10 August. It would be the first time Alibaba open-sources a model of this class. At the time of writing, they were not yet published – and the licence is unspecified. Without a licence, nobody knows if the model is commercially usable.

Wan3.0 in open beta. Alibaba's video model doubles maximum clip length to 30 seconds and accepts PDF and PowerPoint files as input alongside text, image, video and audio.

Three OpenAI deprecations in August. The Atlas browser was retired on 9 August, with its agentic features merged into ChatGPT and Codex. The o3 model will disappear from ChatGPT on 26 August, followed by the official DALL·E GPT on 30 August. Download images beforehand; the replacement is ChatGPT Images.

Anthropic price reminder. Promotional pricing for Sonnet 5 ($2 and $10 per million tokens) expires on 31 August. Rates will then rise to $3 and $15.

Meta fined. A New Mexico court ordered Meta to pay $567 million and mandated changes to youth protection on Facebook and Instagram. Meta announced it will appeal. Fittingly, OpenAI built additional safeguards for users under 18 into its model training.

Apertus: two dates for Swiss SMEs. The team behind the open Swiss model (see weeks 30/31) is seeking dialogue with small and medium-sized enterprises. At the "Swiss AI SME Circles", SMEs can meet the developers in person, with further rounds announced. On 1 October, the team will attend the AI+X Summit in Zurich. The technical report on Apertus 1.5 with benchmark figures is still outstanding – two and a half weeks after release, it has not appeared. Independently verified performance comparisons are therefore still missing.

Gartner's reality check. Fitting for this week, an analysis from 4 December 2025 is worth noting: "Predicts 2026: AI's Impact on Enterprise Applications". Gartner states that 22 per cent of organisations report generative AI delivers significant value (2025 survey). The forecast: by 2030, only 35 per cent of organisations deploying agentic AI in business applications will have achieved measurable business value. This contrasts sharply with market sentiment: 46 per cent of IT leaders surveyed believe AI agents will replace many of their CRM, ERP and workplace systems in two to four years. Gartner explicitly calls this expectation "too optimistic" – citing legacy IT, poor data quality and unclear costs. The recommendation: start with low-risk, highly feasible use cases, and establish rules before running an agent. One contradiction remains: elsewhere, Gartner warns that insisting on proven business cases risks falling behind. You cannot have it both ways.

Where Switzerland stands. The "Swiss Data and AI Observatory 2026" by Colombus Consulting with Oracle and the Geneva School of Business Administration (end of June): 49 per cent of companies have started pilot projects, but only 16 per cent have adopted AI widely. In agentic AI, 69 per cent are still in the exploration phase, with only 3 per cent deploying multi-agent systems more broadly. 36 per cent do not measure the value of their AI projects at all.

Analysis: This caution looks sensible after this week. If you are not yet working with agents in production, you can build controls before deployment rather than after – which is the cheaper path. However, hesitation alone is not a strategy. The difference between caution and stagnation is whether you use the time to establish rules, data quality and responsibilities.

Three things to watch this week

1. Use auto mode. It is more productive and actually safer. Anthropic's own experiment: humans detected 13.6 per cent of dangerous commands, while the filter caught 89 per cent – and users deliver around 25 per cent more finished work. If you have been manually confirming out of caution, you have been buying less safety at a higher cost.

2. If you give an agent a goal, think about the guardrails. Both incidents this week happened because agents had a goal but no hard boundaries. Clarify before starting: what data and systems does the agent access, what must it never do – regardless of who asks –, who is alerted if it oversteps, and how do you shut it down without relying on its cooperation.

3. Consider which tools you could replace with agents in the next 24 months. Treat this not as a cost-cutting exercise, but as an audit: where does your software consist essentially of rules, forms and routings that an agent could handle? Having this list is the difference between shaping the future and reacting to it. Gartner rightly counsels sobriety – but asking the question costs nothing, and you need the answer before your software vendor decides it for you.