No passwords were stolen. No firewalls were breached. No servers were hacked.
And yet, Anthropic is accusing Alibaba of pulling off what it calls the largest known theft of AI capability in history.
28.8 million conversations. 25,000 fake accounts. Six weeks. The target: the very intelligence that powers Claude — Anthropic's most advanced AI model.
If you haven't heard of a distillation attack before, you're not alone. It sounds technical, but the concept is surprisingly simple — and understanding it matters because it's about to reshape how AI companies protect their most valuable asset: the knowledge locked inside their models.
Distillation is a legitimate AI training technique that's been around for years.
The basic idea: you have a large, expensive "teacher" model and you want to build a smaller, cheaper "student" model that performs almost as well. Instead of training the student from scratch on raw data, you feed it outputs from the teacher. The student learns to mimic the teacher's behavior — reasoning style, problem-solving approaches, response quality — at a fraction of the cost.
[Teacher Model: Large & Expensive] ↓ generates outputs [Training Data: Teacher's responses] ↓ used to train [Student Model: Small & Cheap, but almost as capable]
This is how many efficient models get built. Google has done it. Meta has done it. It's standard practice — when you own the teacher model.
The problem starts when you use someone else's model as your teacher without permission.
A distillation attack is when a competitor systematically queries your AI model at scale, collects all the outputs, and uses that data to train their own model — effectively stealing your model's intelligence.
No hacking required. Just a lot of API calls.
Here's the playbook:
The attacker isn't stealing code. They're not downloading weights. They're doing something far more subtle: using the model's own outputs against it.
According to a letter Anthropic sent to US Senators Tim Scott and Elizabeth Warren on June 10, 2026, operators affiliated with Alibaba and its Qwen AI lab ran a systematic campaign against Claude.
The numbers are staggering:
| Metric | Value |
|---|---|
| Fraudulent accounts created | ~25,000 |
| Total interactions with Claude | 28.8 million |
| Campaign duration | April 22 – June 5, 2026 |
| Targeted capabilities | Agentic reasoning, software engineering, long-horizon task completion |
These weren't random queries. The prompts were specifically designed to extract the domains where Claude excels: complex multi-step reasoning, coding across entire repositories, and completing extended tasks over long periods — precisely the areas where Claude's most advanced models lead the industry.
Anthropic described it plainly: these exchanges were designed to turn hundreds of billions of dollars in American AI investment into a subsidy for a geopolitical competitor.
The campaign reportedly continued even after the White House issued a memo in April warning foreign entities to stop.
Here's what makes distillation attacks uniquely difficult to defend against: they look like normal usage.
A real user sends a prompt and gets a response. An attacker sends a prompt and gets a response. From the model's perspective, these are identical.
There's no exploit to patch. No malware to detect. No breach to trace. The "attack" is simply using the API — just at enormous scale, with suspicious patterns.
Detection relies on spotting those patterns:
Anthropic's security team apparently did catch it — but not until 28.8 million exchanges had already occurred.
This is where it gets complicated.
Currently, model distillation attacks occupy a legal grey zone in most jurisdictions. Here's why:
Anthropic's letter to Congress frames it explicitly as a national security issue and calls on lawmakers to criminalize large-scale distillation campaigns. The argument: if a foreign entity can systematically copy the most advanced AI capabilities of an American lab, export control laws on hardware (like Nvidia chips) become much less effective. You can restrict the tools but not the knowledge those tools produce.
This is a genuinely new legal question — and one that courts and legislators will be working out for years.
For decades, tech companies protected value through:
AI changes this. The most valuable thing an AI company builds isn't its code — it's the emergent capability in its trained model. And that capability, once it exists, can be queried by anyone with an API key.
This creates a fundamentally new attack surface. The better your model gets, the more valuable it becomes as a target for distillation.
Some implications for developers and companies:
Rate limiting isn't enough. A determined attacker with 25,000 accounts can stay under any per-account rate limit while generating massive aggregate volume. Detection needs to look at behavioral patterns across accounts, not just individual usage.
Output watermarking is emerging. Researchers are developing techniques to embed invisible signals in model outputs — fingerprints that persist even when outputs are used as training data. If a watermarked response shows up in a competitor's model, the origin can be proven. This is still early-stage but actively being pursued.
Behavioral anomaly detection matters. The pattern of what is being queried is as important as how much. Queries systematically covering specific capability domains — especially sensitive ones — are a signal worth monitoring.
Terms of service aren't enough. Anthropic's ToS prohibits using outputs to train competing models. Alibaba's campaign ignored this. Legal agreements need legal enforcement, and enforcement needs law.
Several things are moving in parallel:
Legislation. Anthropic's letter to Congress is a formal ask to criminalize large-scale distillation attacks. Whether that becomes law — and how quickly — will depend on how seriously lawmakers treat AI capability theft as a national security issue.
Export controls. The US Commerce Department has already moved to restrict access to the most advanced AI models. The distillation attack is exactly the scenario those controls are designed to prevent: not hardware smuggling, but capability extraction through legitimate-looking API access.
Technical countermeasures. Output watermarking, behavioral detection, and rate limiting across correlated accounts are all active areas of development. Expect these to become standard infrastructure for frontier AI labs.
Industry norms. This incident is accelerating conversations between AI labs about what constitutes acceptable competitive practice and what crosses a line — regardless of current legality.
If you're building on top of AI APIs, this story has a direct lesson:
The intelligence in a frontier model isn't just an API endpoint. It's a product of years of research, billions in compute, and carefully curated training. The companies building these models are increasingly treating that capability as IP that can be stolen — not just copied.
For developers building AI-powered products, this means:
The Alibaba-Anthropic case is likely just the first high-profile distillation attack to become public. It won't be the last.
Understanding how AI models can be attacked — not just at the infrastructure level, but at the knowledge level — is becoming an essential part of building in this space. Next up: how output watermarking works, and whether it can actually stop distillation attacks.