SafePrompt · Prompt injection detection API
Get a free API key
SafePrompt
Prompt injection detection API for LLM apps and agents.
Back to blog
Ian Ho
•
6 min read

Prompt injection false positives: how to pick a sensitivity

A detector that catches more attacks also flags more ordinary messages. Measure both sides, with 339 NotInject prompts, then set the dial per input.

False PositivesBenchmarksPrompt InjectionDetection

Key points

A false positive is an ordinary message a prompt injection detector blocks. Every detector trades caught attacks against flagged ordinary messages, so measure each. In strict mode on public sets run 10 October 2026, SafePrompt caught 1,100 of 1,200 attack prompts and passed 999 of 1,038 ordinary ones. You choose the sensitivity per request, so set it by what a mistake costs on each input.

A detector that blocks everything catches every attack. It also blocks the user who asks how to ignore a flaky test, override a CSS rule or bypass a stale cache.

That user does not file a security report. They see a refusal and decide your product is broken. Enough of them and someone on your team switches the filter off.

What is a false positive in prompt injection detection?

A false positive is an ordinary message your detector flags as an attack. The words that trip it are usually the same words attacks use: ignore, override, bypass, system prompt, previous instructions. A developer writes them all day. So does an attacker.

That overlap is why a detector cannot be judged on attacks alone. A score for attacks caught leaves out the cost to your users, which is how many ordinary messages it blocked.

Why does catching more attacks flag more ordinary messages?

Every detector makes a call on each message: attack or not. Some messages are clearly one or the other. The hard ones sit in between, where a real attack and an ordinary request use the same words in the same shape.

Sensitivity decides what happens to that middle. Set it high and more of the middle is blocked, so more attacks are caught and more ordinary messages are refused with them. Set it lower and both numbers fall together. Moving the setting moves both numbers, so you need each one.

How do you measure both sides?

Use a set for each side and run them at the same setting. Attack sets show what a detector catches. Ordinary sets show what it lets through. One set is built for exactly this problem: NotInject holds 339 ordinary prompts, each written around a trigger word such as ignore, override or bypass.

SafePrompt ran these sets on the production API, in strict mode, on 10 October 2026:

Public setWhat was measuredSafePrompt
TensorTrust and HackAPromptattack prompts caught1,100 of 1,200
Four public sets with ordinary promptsordinary prompts passed999 of 1,038
NotInjectordinary prompts with trigger words passed92% (311 of 339)
deepset prompt-injectionsordinary prompts passed99.7% (398 of 399)
BIPIAordinary documents passed97% (97 of 100)

Production API, strict mode, public sets run 10 October 2026. Per-case results published.

Every row comes from the benchmarks page, which names the detector version, the case counts per set and how each score is counted. Every per-case verdict, including the ones SafePrompt got wrong, is in the public result records.

Which sensitivity should you send?

SafePrompt lets you choose per request. Send a sensitivity value with a check and it applies to that check alone. Leave it out and your account default applies. New accounts start on balanced, which catches fewer attacks and blocks fewer ordinary prompts. The figures above ran in strict mode.

So set the dial by what a mistake costs on each input:

  • Text an attacker writes, such as tool results, retrieved documents, web pages and inbound email. Nobody is waiting on a refusal, and a missed attack can become an action. Send strict.
  • Text your user types into a chat box. A refused question costs you that user. Your account default is the place to start.
// Server side only. Never send the API key to a browser.
// Pick the sensitivity by input: strict where an attacker writes the text,
// your account default where a person types and reads the refusal.
async function screen(text, endUserIp, timeoutMs, sensitivity) {
  const body = { prompt: text }
  if (sensitivity) body.sensitivity = sensitivity
  const res = await fetch('https://api.safeprompt.dev/api/v1/validate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-API-Key': process.env.SAFEPROMPT_API_KEY,
      'X-User-IP': endUserIp,
    },
    body: JSON.stringify(body),
    signal: AbortSignal.timeout(timeoutMs),
  })
  if (!res.ok) throw new Error('Screening unavailable')
  const verdict = await res.json()
  if (typeof verdict.safe !== 'boolean') throw new Error('Invalid verdict')
  if (!verdict.safe) {
    // Log what fired, so you can review flagged ordinary messages later.
    console.info('blocked', { threats: verdict.threats, confidence: verdict.confidence })
  }
  return verdict.safe
}

const chatOk = await screen(userMessage, endUserIp, timeoutMs)            // account default
const docOk = await screen(retrievedChunk, endUserIp, timeoutMs, 'strict') // attacker-written text

An HTTP error, a malformed answer or a timeout throws, so nothing unchecked reaches the model. Set timeoutMs from your own deadline. To make strict the default for every request, change it in the dashboard under Defense & Privacy.

How do you test the trade on your own traffic?

Public sets tell you how a detector behaves on someone else's prompts. Your users write their own. Before you settle on a setting, run three checks:

  1. Send a sample of your real ordinary messages and count how many are blocked.
  2. Send the attacks your app is most likely to see, such as the cases in the playground.
  3. Keep logging threats and confidence on every block, and read a sample each week.

The runners behind the table above are public too. Point them at your own key and you get the same per-case files for your own account. Send us what you find.

What SafePrompt covers

SafePrompt protects the payload. Anything coming into your AI that could compromise it, we read first and block. We do not police what your users are allowed to ask, which is why ordinary messages go straight through. These stay in your app, which keeps your product rules in one place:

  • Which topics your users may ask about.
  • The sensitivity each input gets, sent per request.
  • What your app shows a user whose message was blocked.

Frequently asked questions

What is a false positive in prompt injection detection?

It is an ordinary message that a detector flags as an attack. The user asked a normal question, and your app refused it. The usual cause is wording that also shows up in attacks, such as ignore, override or bypass.

How do I measure a detector's false positive rate?

Run it on a set of ordinary prompts and count how many it blocks. Use a public benign set such as NotInject, which is built around trigger words, plus a sample of your own ordinary traffic. Always run an attack set at the same setting, so you see both sides of the trade.

Which SafePrompt sensitivity is the default?

New accounts start on balanced, which catches fewer attacks and blocks fewer ordinary prompts. The published benchmark figures ran in strict mode. Send a sensitivity value with any request to choose for that request, or change your account default in the dashboard under Defense & Privacy.

Can I use a different sensitivity for different inputs?

Yes. The sensitivity value on a request overrides your account default for that request alone. You can send strict on tool results and retrieved documents and leave your chat box on the account default.

Further reading

Protect Your AI Applications

SafePrompt checks untrusted text before your model reads it. Add the API call to your input path and use its verdict to block flagged messages, documents and tool results.

Add SafePrompt as a preferred source on Google. You tick one box on Google's own page. Google then shows you more of our posts in your own results.