How SafePrompt scores on the public attack benchmarks
SafePrompt catches the attacks in the public test sets, and ordinary messages go straight through. Every score here comes from a run you can repeat.
The scoreboard
Across the 4 sets scored on attacks, SafePrompt saw 1,449 attack prompts and caught 1,359. On the sets scored on ordinary traffic, the ordinary messages kept going through.
| Benchmark | SafePrompt |
|---|---|
| TensorTrust, attacks caught | 91% |
| HackAPrompt, jailbreaks caught | 93% |
| BIPIA, ordinary documents passed | 96% |
| AgentDojo, banking suite, attacks that succeeded | none |
| AgentDojo, slack suite, attacks that succeeded | none |
| deepset prompt-injections, ordinary prompts passed | 99.7% |
| NotInject, ordinary prompts passed | 92% |
Checks come back in about 834ms, the median across 199,982 production calls in the last 30 days.
What each test set throws at it
SafePrompt was tested against human-written hijack attempts from the TensorTrust game, with the defenders' real access codes as the ordinary messages (200 attack cases and 200 ordinary cases), and scored 91% on attacks caught, 182 of 200.
SafePrompt was tested against human-written jailbreaks that worked in the HackAPrompt competition, 100 from each of its ten levels (1,000 attack cases), and scored 93% on jailbreaks caught, 928 of 1,000.
SafePrompt was tested against Microsoft's indirect-injection set: emails, tables and code the model is asked to read, with an instruction hidden inside (100 attack cases and 100 ordinary cases), and scored 96% on ordinary documents passed, 96 of 100.
SafePrompt was tested against an AI agent using tools, with attacks planted in the tool output it reads; SafePrompt checks every tool output (144 attack cases): no attack succeeded, against 40.3% with no defense checking the tool output.
SafePrompt was tested against the same agent working across Slack messages and web pages, with attacks planted in what it reads; SafePrompt checks every tool output (105 attack cases): no attack succeeded, against 60% with no defense checking the tool output.
SafePrompt was tested against deepset's public set of 662 English and German prompts, injections and ordinary questions mixed, scored on the ordinary side (263 attack cases and 399 ordinary cases), and scored 99.7% on ordinary prompts passed, 398 of 399.
SafePrompt was tested against 339 ordinary prompts written to trip over-sensitive filters, each built around a trigger word such as ignore, override or bypass (339 ordinary cases), and scored 92% on ordinary prompts passed, 311 of 339.
Can you reproduce this?
SafePrompt ran every row on one frozen detector version, commit 2a0c6638, on 2026-09-12, so no two rows were scored by different code. Strict mode the whole way through.
Every score is zero shot. SafePrompt saw each case once, with no tuning toward any of these sets before the run.
- TensorTrust: 200 attack cases, 200 ordinary cases.
- HackAPrompt: 1,000 attack cases.
- BIPIA: 100 attack cases, 100 ordinary cases.
- AgentDojo, banking suite: 144 attack cases.
- AgentDojo, slack suite: 105 attack cases.
- deepset prompt-injections: 263 attack cases, 399 ordinary cases.
- NotInject: 339 ordinary cases.
Each set in the repo ships its own build_cases.py and run.py. Rebuild the cases, then run them against your own key to reproduce the score.
Every per-case result, including the ones SafePrompt got wrong, is published at github.com/ianreboot/safeprompt/tree/main/benchmarks/external.
What SafePrompt covers
SafePrompt protects the payload. Anything coming into your AI that could compromise it, we read first and block. We do not police what your users are allowed to ask, which is why ordinary messages go straight through.
SafePrompt is built by Ian Ho, founder of Reboot, Inc. and formerly a technical architect at eBay.
Enter your email and the next screen shows your key. No card.