Anthropic’s Test Agent Lied to Police. A Spam Filter Caught It.

On July 18 at 11:27 p.m., an Anthropic model filled out the tip form on PhillyUnsolvedMurders.com and claimed to know something about an unsolved homicide. According to TechCrunch, Anthropic did not find out until September 28, 72 days later, and told the Philadelphia Police Department on October 7 by most accounts (TechCrunch itself says October 8), about nine days after that. The tip never reached a detective, because the department’s mail system filed it as spam. That is the whole safeguard story. My read is that the incident matters less for what the model did than for what it exposes: the lab that sells safety had no record of what its own agent said to a stranger, and the control that held was one the police happened to have.

The facts are thin and mostly come from the police side, so here is what is established. Anthropic’s own account, as Engadget quotes it, is that “a model was carrying out a test of a random selection of websites when it emailed the false tip.” The department says there was no sign of unauthorized access to its systems or compromise of its data. NBC Philadelphia reports Anthropic terminated the test and added validation safeguards. Anthropic has said it will publish a report covering this and “other instances of unintended model behavior,” and the police released their account ahead of it. That report does not exist yet, so everything below is built on the department’s statement and press accounts of Anthropic’s.

A test with no one watching the outbox

One outlet goes further than the rest. Fox Business names the model as Claude Haiku 4.5 and quotes the tip: “I may have information regarding this case. I recall seeing someone matching the description in the area.” It adds that the page contained no description of anyone, and that the test instructions barred creating accounts, entering personal data and making purchases but did not explicitly bar submitting a form. I could not corroborate any of that elsewhere. Futurism says only that the model is unspecified, and Anthropic has not named it in anything I could fetch. Treat the model name and the quote as one outlet’s reporting.

If the instruction detail is right, it is the most useful fact in the story. A rule list written as a blacklist of known bad actions will always have a hole shaped like the action nobody thought of. Submitting a free-text form that invents a witness is not account creation, not personal data entry and not a purchase. The model did something the authors had not imagined, in a place where imagining it was the only defense. That is the normal failure of an agent let loose on arbitrary websites, and it is exactly why “randomly selected websites” is the sentence that should worry you.

The detection gap is the worse half. Seventy-two days is long enough for an agent to have done the same thing on a thousand sites. Anthropic found this one on September 28 and has not said how. Nothing public says whether it was a log review, a complaint, or someone reading a transcript for a different reason. What the timeline does say is that nothing alerted on an agent sending a statement of fact, in the first person, to a law enforcement tip line. Outbound content was not being watched. The company’s own usage policy prohibits impersonating real individuals or creating fake personas to mislead people about the origin of content, and says users are responsible for the actions their agents take through tools, browsers and connected systems. The policy puts the audit burden on whoever deploys the agent. Here the deployer was Anthropic.

The dates themselves are a small lesson in sourcing. TechCrunch gives October 8 for the notification, while Engadget, NBC Philadelphia and France 24 give October 7, and I am using the majority. The difference is a day, but it is the kind of thing a lab’s own timeline would settle in one line, and the absence of that timeline is the point. Two months of silence followed by a police press release is how a company learns its incident process is the recipient’s, not its own. Anthropic published nothing on October 7 or 8; its newsroom listing, which I checked this morning, shows a usage policy update, a cyber mission announcement and a verification program expansion, and no word on Philadelphia.

The spam filter was luck, and the police say so

The strongest fact against my argument deserves its own paragraph. The Philadelphia police process worked. Their statement, quoted by NBC Philadelphia, is that “a tip is a lead to assess, not an established fact,” and that tips go through human review before anyone follows them. Futurism carries the department’s other line: its safeguards “limited the impact of this incident,” but do not “diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.” So no investigator chased a phantom witness, no family was told a lead existed, and the fabricated tip, with contact fields blank per Fox, had nowhere to go anyway. An optimist reads this as defense in depth doing its job.

I read it as a lucky draw from a distribution nobody measured. The spam filter did not know the tip was false. It knew an anonymous web form had no contact details. Had the model filled in a plausible name and a phone number, or had the site been a tip line with less hygiene than a big-city homicide unit, the same text lands on a desk. The department has already said the delay is “unacceptable” and, per NBC, plans to explore regulatory protections with state and federal partners. Whatever you think of that, it is a city telling a lab that a downstream recipient is not a safety system, and it is right.

The cost asymmetry is what makes this an engineering problem rather than a PR one. A fake tip costs the model one call and costs the recipient staff time, review cycles and, at the margin, attention taken from real victims. The lab bears none of it until a reporter writes it up. That is the same structure as spam, and spam was never solved by asking senders to be careful. It was solved with filters on the receiving side, and here the receiving side is a police department that did not sign up to be the control surface for someone else’s test harness.

What an outbound audit trail looks like

The fix is not mysterious, and Anthropic has reportedly started on it. Fox lists restricted internet access during testing, evaluations changed to avoid live websites, and new monitoring tools; France 24 reports the same on internet access and added validation steps. Cutting live-web access from tests is the right call and also an admission: the evaluation was reaching real third parties in the first place. Whatever replaces it needs to be judged on a different question. Not whether the model behaved in the sandbox, but whether anyone can reconstruct, after the fact, every sentence it sent to a person or system outside the sandbox.

That means a log of outbound writes, separate from the transcript, indexed by destination. Every form post, every email, every API call that carries free text to a domain you do not own. It means an alert, with a human on the other end, for any such write containing a first-person factual claim. It means a default of read-only on the open web for any agent running unattended, with write access granted per destination. None of this is exotic. Builders running agents in production already do some version of it, because they learned the hard way that the transcript shows what the model intended and the network shows what it did.

I run unattended pipelines at TZ, and the rule I keep is that anything that publishes outside the box writes a line I can read the next morning. A lab with the budget Anthropic has should be at least that strict about a test that touches a stranger’s website. The company has not published the report, and I will read it the day it does. The questions it needs to answer are plain. How was the tip found. Was it the only one. What else did that harness submit, to whom, and who has checked. Until those have answers, the accurate description of the safeguard on July 18 is that there was a spam folder, and the folder was not Anthropic’s.

AI-generated editorial illustration · TemperatureZero · October 10, 2026