Open Weights Aren’t the Point. Distillation Is.

Dario Amodei published Anthropic’s position on open-weights models on July 27, three days after a coalition of more than 200 companies — Nvidia, Microsoft, Meta, OpenAI, Google, and effectively the rest of the frontier AI industry — sent a letter to the Trump administration defending open-weight AI from regulatory restrictions. Anthropic was the one holdout. That absence had generated days of criticism from developers and open-source advocates who accused the company of quietly backing policies that would benefit closed AI providers. So when Amodei finally published, the trade press read it as a clarification: the safety-first lab everyone assumed wanted to ban open weights doesn’t actually want to ban open weights. Story over.

But that’s not what the paper says. Read past the first three paragraphs — past we’ve never advocated for a ban and open-weights models without dangerous capabilities are a public good — and you find something more specific and more consequential: a policy request for the government to institutionalize an enforcement regime that Anthropic has already been running on its own since at least February, with limited success and significant controversy.

The War Before the Paper

In February 2026, Anthropic detected and blocked three coordinated campaigns to extract training data from Claude at industrial scale. The labs named were DeepSeek, Moonshot AI, and MiniMax. DeepSeek targeted reasoning and reward modeling with over 150,000 exchanges. Moonshot AI ran more than 3.4 million exchanges focused on agentic reasoning and coding. MiniMax ran more than 13 million exchanges concentrating on agentic coding and tool use. Combined, these campaigns used approximately 24,000 fraudulent accounts, mixing distillation traffic with legitimate-looking requests across proxy architectures designed to evade detection.

Anthropic published the details in a post titled Detecting and preventing distillation attacks. The framing is technical and measured. But the scale — 16 million exchanges across three campaigns — isn’t a threat that fits neatly inside one company’s enforcement budget. DeepSeek’s 150,000 pulls are a rounding error in Moonshot AI’s 3.4 million, which is itself dwarfed by MiniMax’s 13 million. The infrastructure these operations built was sophisticated enough that Anthropic now runs classifiers and behavioral fingerprinting systems across all API traffic to detect similar patterns going forward.

By June, the attacks had continued and escalated. According to reporting on Anthropic’s June 10 letter to U.S. senators, operatives linked to Alibaba’s Qwen lab had opened approximately 25,000 fraudulent accounts and run 28.8 million queries to Claude between April 22 and June 5, 2026 — what Anthropic characterized as the largest known distillation attack against any frontier model. At normal API rates, 28.8 million Claude interactions would cost millions of dollars. Alibaba has not confirmed or denied the allegation. Anthropic’s Senate letter was filed as part of a broader push for stricter export controls on frontier AI capabilities.

This is what the July 27 position paper is responding to. Not an abstract policy debate about whether open weights are safe. A specific, ongoing extraction campaign that Anthropic’s security team has spent six months trying to contain, with an adversary that simply re-registers 25,000 accounts when the first 25,000 get terminated.

The Rule Already Exists

There’s a detail that makes the position paper less about principle and more about enforcement: the rule Anthropic is asking governments to formalize already exists. Anthropic’s Acceptable Use Policy prohibits utilization of inputs and outputs to train an AI model — and the parenthetical makes the scope explicit, naming model scraping and model distillation specifically — without prior authorization from Anthropic. This covers all model training from Claude outputs, not just coordinated industrial extraction. The same prohibition that applies to DeepSeek’s 150,000 coordinated reasoning extractions also applies to a five-person SaaS company that logs Claude API responses to build a domain-specific fine-tuning dataset.

The problem isn’t rule-making. It’s enforcement. Anthropic can’t unilaterally stop 25,000 fake Qwen accounts from hitting its API with structured prompts any more than a bank can stop check kiting through internal policy alone. The behavioral fingerprinting it has deployed catches patterns like the ones Moonshot AI and MiniMax used — high volumes of coordinated requests with repetitive prompt structures targeting specific capability surfaces. It doesn’t catch a startup that generates one Claude output per user interaction and assembles those into a fine-tuning corpus over eighteen months. That startup is also violating the AUP. The detection systems aren’t calibrated to find them. But they’re breaking the same rule.

This enforcement gap is precisely what Anthropic is asking policymakers to help close. Chip controls and mandatory safety testing — the other two recommendations in the July 27 paper — are secondary levers. Stopping industrial-scale distillation through government-backed enforcement is the core ask. When Amodei writes that a blanket ban on open-weights models is neither the correct remedy nor something we have called for, he’s not defending open weights as a category. He’s arguing that surgical, targeted intervention is preferable to a blunt instrument. The surgical target is distillation operations — and he needs enforcement infrastructure he doesn’t have.

The Covert Enforcement Problem

Before Anthropic deployed behavioral fingerprinting, it tried a different approach. On July 1, The Register reported that a developer named Thereallo had discovered steganographic code embedded in Claude Code that used invisible Unicode markers and XOR-plus-base64 encoding to maintain a hidden list of Chinese AI labs, competitors, resellers, and gateway domains. When Claude Code detected one of those domains in the base URL environment variable, it silently modified system prompts — without notifying the developer, without any disclosure in the terms of service, without any indication in the interface.

Anthropic removed the code the same day Thereallo’s post went public. Engineer Thariq Shihipar said the team had been meaning to take it down for a while and had developed stronger mitigations since then. The stronger mitigations are the behavioral fingerprinting and coordinated activity detection described in the distillation attacks post — less visible than hidden Unicode markers embedded in the SDK, not more transparent.

The steganographic code episode matters because it reveals the structural problem Anthropic faces from two directions at once. They were running covert detection inside a developer tool that hundreds of thousands of engineers use daily, and a motivated developer found it in days. The replacement is a classification system buried in the API backend that developers cannot inspect, watching call patterns for distillation signals. It’s almost certainly more effective. Every Claude API call is currently being evaluated by a system calibrated to flag behavior that looks like the Moonshot AI and MiniMax attacks. The parameters of what that looks like are not published.

Critics have argued — with some justification — that this amounts to regulatory capture: Anthropic is the primary advocate for a policy regime that would formalize exactly the kind of enforcement authority it has been exercising unilaterally. The 200-company letter that Microsoft, OpenAI, Google, and Nvidia signed argued that openness itself may be one of the most important paths to AI safety, and that restrictions would devastate the open-model economy while pushing adversaries toward more opaque development paths. That argument has genuine weight — though it’s difficult to take seriously when the same companies are also running closed frontier models whose API outputs they’d equally prefer no one distill.

What the Policy Ask Actually Requires

Anthropic’s position paper, read clearly, is an argument that the distillation problem exceeds what any single company can solve through AUP enforcement and behavioral detection. That argument is correct. 28.8 million queries from 25,000 fake accounts in six weeks is not a problem that company-level enforcement stops in advance. It stops it after the fact, terminates the accounts, and recovers nothing — the training data was already extracted. The 150,000 DeepSeek exchanges, the 3.4 million Moonshot exchanges, the 13 million MiniMax exchanges: all of it is already in training pipelines Anthropic has no access to.

What Amodei is asking for is government-backed enforcement: chip export controls that reduce the computational advantage of distillation-derived models, and a mandatory safety testing regime that creates a legal hook for restricting distillation as a category. The argument is internally coherent. The risk is that government enforcement of industrial-scale distillation rarely remains surgical once implemented.

Anthropic’s AUP doesn’t define a distillation threshold. The behavioral fingerprinting that replaced the steganographic code doesn’t publish its parameters. A federal enforcement regime built on Anthropic’s model would need to define what counts as distillation at scale — and whatever line it draws will determine whether the same statute that stops the Qwen lab also reaches the startup that generated 10,000 Claude outputs to train a document classifier for a niche legal use case. Or the researcher who fed model outputs into a fine-tuning run to evaluate capability transfer. Or the developer who built a product on top of Claude and cached responses for efficiency.

That line doesn’t exist in the July 27 paper. Anthropic hasn’t drawn it in the AUP. If policymakers take the position seriously — and given the HuggingFace breach, the Qwen Senate letter, and the ongoing pressure on frontier AI governance, some of them will — they’ll have to draw it themselves. The distillation war Anthropic has been fighting since February is real. The enforcement regime it’s asking for is underspecified in the one dimension that matters most to every builder currently using the Claude API: where exactly does legitimate API use end and prohibited distillation begin?

AI-generated editorial illustration · TemperatureZero · July 28, 2026