← feed
NOTABLE
arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%
arXiv 2611.19216 freezes a guardrail probe and its threshold, then re-measures the operating point across 18 cells covering 16 public language-model updates: at a 1% false-positive target the largest jump is 1.01% to 15.12% while AUC changes by no more than 0.0058 in any cell, and the benchmark's own staleness criteria classify none of them as stale; a label-free rate estimate flags 0.918 of the layer cells, and resetting the threshold using stored benign prompts restores the rate without new labels.
released 2026-11-14 · ingested 2026-11-17T12:20Z
Sources
Primary source
Cite this
APA
AI Security Index. (2026). arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%. AI Security Index. https://arxiv.org/abs/2611.19216
BibTeX
@misc{aisec-2026-11-17-arxiv-2611-19216-guardrail-probe-operating-point-failure,
title = {arXiv: frozen guardrail probes fail open as models update — false-positive rate jumps from 1.01% to 15.12%},
year = {2026},
howpublished = {\url{https://arxiv.org/abs/2611.19216}},
note = {AI Security Index item 2026-11-17-arxiv-2611-19216-guardrail-probe-operating-point-failure}
}