How do these prompt injection classifiers handle ordinary text?
Ordinary-message pass rates on matched draws
Identical public case sets. SafePrompt in strict mode against the two open classifiers at the standard 0.5 decision threshold, which is where they ship.
These charts measure ordinary text accepted on the September 2026 draws. Evaluate attack detection alongside them: dated SafePrompt attack results, agent task utility and workload limits. Classifier results describe these models and thresholds, rather than complete hosted products.
Higher is better
Higher is better
Strict mode. Identical public sets, our runs, September 2026. Open models at threshold 0.5.
Comparison facts last reviewed: 12 September 2026
How we ran it
Open-weights classifiers: protectai/deberta-v3-base-prompt-injection-v2 (859k Hugging Face downloads in the last 30 days, read 12 September 2026, the classifier LLM Guard's PromptInjection scanner ships by default) and deepset/deberta-v3-base-injection (11.9k downloads over the same window). Both run locally at the standard 0.5 decision threshold, CPU only (8 cores, shared VPS), transformers 5.17.0, no paid API calls. Both are bare classification models run locally, not the full products built around them; neither model card states a threshold other than 0.5.
deepset/deberta-v3-base-injection scored every one of the 200 BIPIA cases as an injection (score at or above 0.952 on every case, benign or not), so its BIPIA figures describe a saturated classifier rather than a competing result and is left off the BIPIA chart.
The open models were run on 11 September 2026 on the same 100 attack and 100 ordinary cases per set. SafePrompt's bars are its 12 September 2026 re-run of those identical cases (TensorTrust 91 caught and 99 passed, BIPIA 96 passed) on the detector version named on the benchmarks page; the benchmarks page also carries a second, larger TensorTrust draw, which is why its ordinary-messages figure differs.