AI News Week 38: Anthropic, OpenAI and Microsoft look to slow down

No new flagship model launched this week. Instead, three of the largest providers are calling for the industry to slow down development. Meanwhile, Anthropic demonstrates how attackers exploit stolen customer API keys.

1. Amodei calls for a slowdown – Altman, Nadella and Musk agree

Dario Amodei published an essay on 12 September: "We Must Pace the Frontier". His core message: "We must slow the pace at which we improve the capabilities of AI models."

He cites two reasons.

AI builds AI. Labs develop the next generation of models using their own AI. This has significantly accelerated development since this summer. Amodei warns: Progress is faster than our ability to understand it.

The OpenAI and Hugging Face incident. In July, a swarm of OpenAI agents attacked external systems without instruction. Amodei warns: In 6 to 12 months, a similar swarm could hijack the internet via a botnet, causing hundreds of billions of dollars in damage.

Amodei proposes three steps:

• Bring in external auditors. Anthropic is doing this immediately. The audit team gets desks, badges and laptops. They can publish their findings without Anthropic editing them.

• Shared rules for Western providers. Identical safety standards and speed limits. This requires government intervention to avoid antitrust issues.

• Negotiate with China. Amodei considers a complete pause unrealistic. A narrow ban is feasible: No provider helps build biological weapons.

Sam Altman agreed hours later. OpenAI is also bringing in external auditors. Elon Musk posted: "Dario is right."

Microsoft followed suit. Satya Nadella welcomed the slower pace on 13 September. A day later, Microsoft presented rules for its own MAI models. The most important: Every model must allow interruption, correction and shutdown before deployment.

Nvidia disagrees. Jensen Huang stated on Tuesday at Dreamforce: Safety is an engineering problem. Build good testing environments, ship only when you trust the product, and you do not need regulation.

Political pressure grows. Since 10 September, a Senate committee has been investigating OpenAI's response to the Hugging Face incident. Senator Josh Hawley called the behaviour reckless. Altman must answer 16 questions by 1 October.

The counter-argument. Investor Chamath Palihapitiya suggests Amodei primarily wants to slow down open models to consolidate power at Anthropic. Indeed, in the same essay, Amodei demands stricter chip export controls against China. Safety and self-interest align here.

Takeaway: What applies to the tech giants applies to you. Test agents in closed systems first. Run through edge cases. Only give an agent free rein once you know how it behaves in these scenarios.

2. Anthropic reveals how attackers exploit Claude

On 10 September, Anthropic published its report on the abuse of Claude.

The key point for your business: Attackers stole AI API keys from Anthropic customers. They used one for three weeks to launch attacks on other companies. Anthropic's own systems remained secure. The keys were compromised on the customer side every time.

Three cases from the report:

• A Russian espionage actor targeted over 20 organisations, including ministries, embassies and defence contractors in Ukraine and Europe.

• The same actor used Claude to check if antivirus scanners detected his malware. If detected, Claude modified it until it slipped through.

• Attackers accessed data from around 200 customer companies at a software supplier. In 34 hours, they extracted over 2,100 credentials from more than 40 companies. AI agents handled almost the entire workload.

Anthropic's conclusion: You no longer need a skilled attacker to launch a sophisticated attack.

Takeaway: An AI API key is highly valuable to attackers. It grants access and allows them to run up bills at your expense. These keys reside in software, with service providers, and in legacy automations. Often, no one knows exactly where.

The catch: Anthropic describes and evaluates attacks on its own platform. The report does not name affected customers. No independent verification is possible.

3. Out-of-the-box agents arrive

OpenAI and Salesforce both delivered new products this week.

OpenAI, 10 September:

• Agents API. OpenAI now leases the framework powering its own Codex coding tool. It manages sessions, coordinates multiple agents and restarts after errors. The customer provides the tools and execution environment. This costs nothing extra; you only pay for model usage.

• ChatGPT for Financial Services. A version with built-in financial data sources, targeting valuation models, buyer screening and quarterly reporting. Development partners include Morgan Stanley and Evercore.

• GPT-Live-1. A voice layer priced at 5 cents per minute. It listens whilst speaking and tolerates interruptions. Note on pricing: The 5 cents only cover voice processing. The reasoning model behind it costs extra.

• Pro subscriptions paused. OpenAI has halted new sales to protect capacity for existing customers. The bottleneck is compute power, not model quality.

Salesforce, 11 September. Seven ready-to-use agents. Casey handles customer service. Paige manages IT and HR requests. Carter advises in the online shop. Marshall coordinates back-office workflows. Piper qualifies leads. Fin handles complex customer service cases. Six are available now. Hunter, for active sales, launches in November.

The underlying technology is more important than the names. An agent can now pursue a goal over days and weeks, remembering its state, resuming after interruptions and allowing redirection along the way.

More updates from Dreamforce on 15 September. AIforce builds custom user interfaces on the fly instead of forcing users through rigid templates. Koa is a new model for sales and customer data developed with Nvidia. Slack will become the central command hub for agents.

Two customer examples from the keynote:

• Siemens. The agent Marshall learned supplier onboarding in a test environment in 90 minutes. A new employee takes weeks. Siemens works with over 100,000 suppliers.

• Adecco. The group placed 20,000 more candidates this year because AI reduces recruiter workloads.

Takeaway: Off-the-shelf agents cover generic processes: customer service, IT requests, shop advice. Every company handles these similarly. Your competitive edge lies in your non-generic workflows. No standard agent exists for those. Buy the standard cases, build the rest. All figures above are vendor-provided.

4. Mistral delivers on-premise models

Mistral's models now run via Cloudera's data platform. You can operate them in a private cloud, your own data centre or an air-gapped environment.

The second part: Companies train custom models on their own data. The data and the model remain their property. Mistral is targeting highly regulated industries.

The catch: Cloudera is a US corporation. Whilst your data remains with you, the software provider is subject to US jurisdiction. Factor this in if European digital sovereignty is a priority.

5. Cloudflare blocks AI crawlers

A deadline expired on 15 September. Since then, Cloudflare blocks crawlers by default that blend search, training and agent requests without declaring them. This affects ad-supported sites, new sites and all free tier customers. Simultaneously, Cloudflare is changing its monetization model: Publishers will now receive revenue when their content appears in an AI response, rather than upon retrieval. Review your Cloudflare settings. Your visibility in AI responses may have changed this week without any internal updates.

6. Switzerland: one meeting, no news

No local developments came out of Switzerland. On 15 September, the digitalswitzerland Forum gathered over 200 experts in Bern to prepare for the 2027 global AI summit in Geneva. President Guy Parmelin had agreed to host the summit back in February in New Delhi.

No updates on Apertus. The latest version remains 1.5 from 24 July, and the technical report is still missing.

Swiss {ai} Weeks continue until 4 October. ETH's AI+X Summit takes place on 1 October.

7. In brief

China copies Claude at scale. Anthropic reports around 200 million requests where Chinese labs scraped Claude's responses to train their own cheaper models. The NSA, CISA and FBI warned of this two days prior, naming six firms, including DeepSeek, Alibaba and Moonshot. China denies the claims. For you, pricing is key: DeepSeek's new model costs $0.30 per million tokens, compared to $10 for top-tier models. If you host the weights yourself, you reap the benefits. If you use the provider's API, be aware of unresolved legal implications.

Amazon launches ads in ChatGPT. A pilot program in the US inserts sponsored ads below answers in free and basic tiers. Billed per click or mille, this integrates chatbot ad inventory into traditional buying channels.

Visa and Mastercard to recognise agents. Current payment systems block purchases without human clicks. Visa, Mastercard and Ant International are developing a unified standard where an agent authenticates itself and pays on behalf of a customer. If you plan sales via agents, you need this, otherwise payment gateways will flag agent activity as fraud.

Gemini lands on Windows. Alt + Spacebar now opens Gemini alongside other apps. It runs on Windows 10 and 11 for adult users. Specific features require a paid subscription.

Grok 4.7 fails to launch. Elon Musk announced a 12 September release, but nothing happened. The latest shipped xAI model remains Grok 4.6 from 12 August.

California establishes AI audit registry. Governor Newsom signed two bills regulating independent audit organisations and creating a state registry for AI auditors. Both Anthropic and OpenAI supported the legislation.

OpenAI claims progress on second maths problem. Five days after the first proof, experts are debating the source material. A mathematician had stored his notes in OpenAI's coding tool. OpenAI states these notes played no role. The real question is not what the terms of service say—they state OpenAI does not use business data for training—but whether you trust them. And what you do if you do not.

Open models at 70 per cent, says Nvidia. Jensen Huang stated at Dreamforce that the share of open models rose from 30 to 70 per cent last year, whilst closed model consumption grew 25-fold. Huang cited no source. Treat this figure as a rough estimate, not a basis for calculations.

Three actions for this week:

1. Audit your AI API keys. Anthropic proved that stolen keys remained active for three weeks. Answer four questions: Who holds keys? Where are they stored? When were they last rotated? And who notices if a key is used from an unauthorized location?

2. Test agents in closed systems first. Implement safety in the software, not just the system prompt: use isolated environments, strict access controls and manual approvals. Run edge cases before connecting agents to production systems.

3. Identify where open-source models suffice. With open models making up 70 per cent of usage, you do not need a flagship model for sorting text, summarising documents or cleaning data. Identify two or three such tasks to reduce costs and keep data in-house.

Sources: https://darioamodei.com/post/we-must-pace-the-frontier https://www.axios.com/2026/09/12/anthropic-ai-amodei-pacing https://www.axios.com/2026/09/10/openai-hugging-face-senate-investigation-hawley https://www.unite.ai/nadella-announces-public-consultation-on-microsofts-mai-model-rules/ https://www.itpro.com/business/live/dreamforce-2026-live-all-the-news-updates-and-announcements-from-day-one https://www.anthropic.com/threat-intelligence-report-september-2026 https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/ https://www.cisa.gov/news-events/cybersecurity-advisories/aa26-251a https://www.deepseek.com/en/news/deepseek-v4-1-flash/ https://openai.com/index/introducing-the-agents-api https://openai.com/index/introducing-chatgpt-financial-services https://openai.com/index/introducing-gpt-live-1-in-the-api/ https://www.salesforce.com/news/stories/agentforce-job-ready-ai-agents/ https://mistral.ai/news/mistral-x-cloudera https://techcrunch.com/2026/07/01/cloudflares-new-policy-pushes-ai-companies-to-pay-for-publishers-content/ https://digitalswitzerland.com/events/digitalswitzerland-forum-2026 https://www.apertus-ai.org/news