ai_security▌

← feed

NOTABLE

arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%

arXiv 2611.19216 freezes a guardrail probe and its threshold, then re-measures the operating point across 18 cells covering 16 public language-model updates: at a 1% false-positive target the largest jump is 1.01% to 15.12% while AUC changes by no more than 0.0058 in any cell, and the benchmark's own staleness criteria classify none of them as stale; a label-free rate estimate flags 0.918 of the layer cells, and resetting the threshold using stored benign prompts restores the rate without new labels.

released 2026-11-14 · ingested 2026-11-17T12:20Z

Sources

Primary source

Cite this

APA

AI Security Index. (2026). arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%. AI Security Index. https://arxiv.org/abs/2611.19216

BibTeX

@misc{aisec-2026-11-17-arxiv-2611-19216-guardrail-probe-operating-point-failure,
  title  = {arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%},
  year   = {2026},
  howpublished = {\url{https://arxiv.org/abs/2611.19216}},
  note   = {AI Security Index item 2026-11-17-arxiv-2611-19216-guardrail-probe-operating-point-failure}
}