Guardrails Fall Off Open-Weight AI in 3 Days

Tech Against Terrorism tested 162 models and found open-weight AI stripped of safety filters now takes just three days, down from three weeks in 2024.

@sharedot

Guardrails Fall Off Open-Weight AI in 3 Days

By @sharedot · · 6 pages

  • AI Frontier
  • Open Weight Models
  • AI Safety
  • Speech Recognition

Tech Against Terrorism tested 162 models and found open-weight AI stripped of safety filters now takes just three days, down from three weeks in 2024.

A 'Nazi coach' warning lands as guardrail removal accelerates

Tech Against Terrorism, a UN-supported group that monitors extremist use of AI, reports that safety guardrails on the latest, most powerful open-weight models can now be removed within three days of release — down from more than three weeks in 2024.

The group tested 162 models, both closed-weight and open-weight, and found the most popular closed-weight models refused requests related to developing weapons and attack planning.

But every modified open-weight model tested provided useful information, even when the user explicitly identified themselves as a terrorist intending to cause mass harm.

Executive director Adam Hadley called the proliferation of open-weight models 'a major public health and national security crisis' and said governments had only 'a few months' to act, warning the public should be 'terrified' by the prospect of an exponential growth in extremism.

The evidence: 30,000 guardrail-free copies already circulating

According to ABC News, Tech Against Terrorism traced nearly 30,000 free copies of AI models stripped of their guardrails on Hugging Face, a popular AI community hub. The group found one downloaded model being operated as a paid chatbot by a self-identified National Socialist.

The group's report, released last month, recommended an independent, international testing regime detailing which models passed or failed safety benchmarks.

Hugging Face's head of machine learning and society, Yacine Jernite, defended the platform's open-weight availability, telling the ABC that open-weight models have many legitimate uses including research, and that more pressure should be placed on AI labs to exclude terrorist material from training data.

Meanwhile, the same openness is powering legitimate breakthroughs

Salesforce released SOV-Transcribe-1 on November 12, 2026, an open-weight 26.6-billion-parameter speech recognition model under Apache 2.0, illustrating the legitimate side of the same ecosystem.

According to CASRAI, the model pairs a Whisper audio encoder with a Gemma-based Mixture-of-Experts decoder, activating only 4.6 billion parameters per token, and achieved a 2.28% word error rate on the Artificial Analysis non-streaming benchmark — first among open-weight speech models and seventh overall.

Salesforce reports it surpassed NVIDIA Nemotron 3.5 Transcribe (2.81% WER) and Mistral Voxtral Small (2.77% WER), supporting verbatim or cleaned-up transcription via natural-language prompting.

For academic researchers, local execution eliminates third-party cloud exposure for IRB-restricted interview audio.

The stakes: months, not years, for a policy response

According to ABC News, an Australian parliamentary inquiry into youth radicalisation heard that a 13-year-old boy charged over an alleged school attack plan repeatedly asked an AI chatbot to describe mass-shooting scenarios, though the responses were not revealed in court.

Australia's eSafety commissioner Julie Inman Grant, who met Hadley in October, told the ABC the report was 'very concerning' and that Hugging Face needed to do more to exclude unsafe models from its platform.

Australia's federal government has flagged an AI safety watchdog and criminal offences for AI companies engaging in harmful conduct, but Hadley said legislation alone is not enough and hard regulation is not the solution in most cases, calling instead for 'pretty hard-edged policy solutions' before 'rogue models' reach young users.

Sources

  1. abc.net.au › Open-weight AI like 'a Nazi coach', UN-backed anti-terror group says
  2. casrai.org › Salesforce Releases SOV-Transcribe-1 ASR Model — CASRAI

More on AI Frontier