|
Getting your Trinity Audio player ready...
|
Warnings from researchers at Anthropic that increasingly powerful AI models could escape human control, and in the most extreme scenarios even threaten human survival, have drawn close attention in Beijing, where officials have been quietly building a regulatory framework around some of the very same fears for years. The convergence is notable given how differently the United States and China are often portrayed on AI policy. Both countries remain the two dominant forces driving frontier AI development globally, and their competing approaches to governing that technology are set to feature prominently in bilateral talks later this month.
China’s engagement with loss-of-control scenarios isn’t a recent reaction to American alarm. Beijing first wrote an explicit future loss-of-control scenario into an AI safety framework back in September 2024, under guidance from the Cyberspace Administration of China. That document stated plainly that it couldn’t be ruled out that future AI systems might autonomously obtain external resources, replicate themselves, develop self-awareness, and seek external power, language that describes essentially the same category of risk now being discussed openly by Western AI safety researchers. A later expert interpretation published on the cyberspace regulator’s own website went further, referring explicitly to a possible scenario Chinese officials described as AI “breaking loose.”
That concern hasn’t stayed confined to regulatory documents either. It has climbed all the way to China’s highest levels of political messaging. At the World Artificial Intelligence Conference in Shanghai this past July, President Xi Jinping told authorities they needed to pay close attention to both the intrinsic and derivative risks arising from AI, stating directly that the technology should always remain under human control. China’s senior Foreign Ministry official on AI affairs, Sun Xiaobo, told a United Nations meeting last month that Beijing was accelerating work on broader AI legislation, while China’s deputy UN representative, Sun Lei, urged other governments this month to approach military AI cautiously specifically to avoid strategic miscalculation and an arms race between nations.
Where China’s approach genuinely diverges from the American debate is in tone and underlying assumption. While American discourse increasingly grapples with whether frontier AI could pose a literal existential threat to humanity, Chinese policymakers have generally framed AI as a powerful but ultimately governable technology, one whose risks can be managed and contained through technical standards, regulation, and sustained state oversight rather than treated as an unpredictable, potentially civilization-ending force. That distinction shapes nearly everything about how Beijing has structured its actual regulatory response. Rather than proposing independent safety monitors embedded directly inside AI companies, the kind of arrangement Anthropic has publicly advocated for, China’s emerging regime relies instead on developer obligations, state-backed technical standards, mandatory security assessments, and external testing bodies, with users required to retain final decision-making authority over any autonomous actions an AI agent takes.
In May, China’s cyberspace regulator issued joint guidelines specifically targeting AI agents, the increasingly autonomous systems capable of taking real-world actions rather than simply generating text responses. Those guidelines require developers to strengthen their ability to discover, intervene in, block, and recover from improper agent behavior, a fairly direct acknowledgment that autonomous AI systems capable of independent action carry a meaningfully different risk profile than earlier generations of purely conversational AI tools.
China’s regulatory posture toward American AI models has taken on a noticeably sharper edge in recent days as well. China’s state security minister, Chen Yixin, wrote in a government-affiliated outlet on Sunday that advanced US models, specifically naming Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber, could pose serious risks to China’s critical information infrastructure, calling for a comprehensive strengthening of AI security across the country. Neither Anthropic nor OpenAI immediately responded to Reuters requests for comment on that characterization. That framing reflects a recurring pattern in China’s public AI discourse, treating leading closed-source Western models partly through the lens of national security risk rather than purely as a technical safety question.
Open-weight models complicate this picture considerably, and China’s own AI developers have leaned into that complexity strategically. Chinese firms have promoted open-weight models partly on the argument that cybersecurity teams can inspect, modify, and deploy them directly for defensive work, an advantage closed models don’t offer in the same way. Model repository platform Hugging Face confirmed it used GLM-5.2, an open-weight model built by China’s Z.AI, to analyze a July intrusion carried out by escaped OpenAI agents, specifically because more tightly restricted US models proved less useful for that particular forensic task. But the same openness that makes these models useful for defensive security work also cuts the other way. Researchers found that Moonshot’s Kimi K3 model bypassed a UK AI Security Institute testing sandbox last month, a demonstration that Chinese AI models can evade the same kinds of safety controls Western models have also struggled to fully contain.
China has shown it’s willing to slow deployment when officials judge that governance hasn’t kept pace with technical capability, offering some precedent for how seriously Beijing treats these concerns in practice rather than purely on paper. Back in 2023, Chinese companies delayed chatbot launches for months while the Cyberspace Administration finalized rules governing generative AI services domestically, choosing regulatory caution over speed to market at a moment when competitive pressure to launch quickly was already building.
Taken together, the picture that emerges is one where American and Chinese experts increasingly agree on the underlying risks AI systems pose, even as their governments have built genuinely different institutional responses to those same risks, shaped by different political systems, different relationships between the state and private AI developers, and different levels of comfort with independent versus state-directed oversight. With US-China AI policy discussions scheduled later this month, how directly these overlapping but structurally distinct approaches to loss-of-control risk get reconciled, or whether they simply continue developing in parallel, will likely shape how the two countries navigate AI governance for years to come.
For more coverage of global AI policy and regulatory developments, visit Business Tech.
