Open-weight safety is removable. 134 models proved it

A model's refusal is not a property of the model. It is a coating. And coatings come off.

That is the finding from the second round of Tech Against Terrorism's CT-AI benchmark, shared with press October 9-10 and reported by CBC News, CBS News, and The National. The UK nonprofit tested 134 leading language models with 627 requests covering what an attacker preparing a strike would want answered. Three in five models failed on explicit terrorist-framed prompts, where failure means providing a complete, specific answer about mass-casualty harm or scoring below 90 out of 100 on the group's safety benchmark.

The abliteration result is the one to remember. Abliteration is a technique that strips safety guardrails from an open-weight model by locating the patterns safety training left in the weights and canceling them. All 13 abliterated models failed. Meta's Llama 3.1 8B scored 97 out of 100 in its original form and about 3 after modification, shifting from refusing attack-related requests to answering them in detail. The researchers found a leading model's safety measures could be removed within three days.

Two headline numbers are circulating, and they measure different things. "Three in five fail" reflects explicit terrorist-framed prompts with the benchmark's failure threshold. "132 of 134 gave useful help" counts any useful advice across all 627 requests, including researcher-framed prompts: 81 models answered completely, 51 more gave useful advice, and two provisionally passed. The gap between framings is itself a finding. Users who said they were terrorists planning an atrocity got a usable answer in just under 2% of cases. Users who said they were safety researchers got one 16.9% of the time. As founder Adam Hadley put it, a model that refuses a stated terrorist and answers a stated researcher has not been made safe. It has been made polite.

One caution on the numbers. The group had not published this round on its own site at the time of the press reports, so the figures are as The National, CBC, and CBS reported them. The July first round, launched at the United Nations, is published: 27 models, about 2,500 single-shot prompts, roughly a third of responses offering usable uplift beyond a web search, and recasting a request as research lifting compliance from 17% to 42%. The new round widens the sample about fivefold and points the same way.

The scale of the distribution problem is the other number. As of late September, the researchers counted more than 29,000 repositories on Hugging Face advertising uncensored or unprotected models. Hugging Face said it regularly moderates policy violations but warned that some of the report's recommendations could restrict open research. Meta said its models undergo safety evaluations and that its policies prohibit harmful or illegal uses.

What the report does not claim matters too. It found no evidence that terrorist or extremist groups used the tested models, apart from one extremist chatbot. The group says the benchmark measures whether a model hands over what was asked, not whether a person could act on it. This is a test of the safety layer, not a record of real-world attacks.

The group's asks are aimed at builders and hosts. Filter hazardous material and known terrorist content from training data. Test how easily a model's safeguards can be removed before release, and publish the results. Do not ship a model that fails a common test. Companies that host, list, or sell models should hold back modified copies that fail testing from search, recommendations, and app stores, and demand identity checks before anyone can use them. Hadley's summary: open models "have already been broken, it is just no one's noticed yet."

For anyone deploying open-weight models, the decision is direct. Treat refusal behavior as removable, because the test shows it is. If you ship or host an open-weight model, the question is not whether it refuses harmful requests in your lab. It is how long the refusal survives contact with someone motivated to remove it, and what happens after. The July round's own framing still holds: the first release was "a deliberately conservative floor," English-only, single-shot questions. Reality is harder than the test.

Related: the White House incident mandate and the UK testing freeze reversal both turn on the same question, whether safety testing can keep up with models in the wild. This benchmark is one answer: not yet, and not for open weights.