Limitations
SafePrompt is one control in a layered design: it screens untrusted content before it reaches a model or an agent. It reduces exposure. It does not remove it, and it is not a complete answer to prompt injection.
This page exists because a security product that only publishes its wins is harder to evaluate, not easier. If any of the limits below rule SafePrompt out for your case, we would rather you learned it here than after integrating.
What SafePrompt does not do
Authorization
SafePrompt does not decide what a user or agent is allowed to do. Permission checks belong outside the model and must not depend on it.
Output validation
SafePrompt screens input. It does not inspect what your model returns, and a clean input can still produce an output you should not act on.
Sandboxing and least privilege
If a tool can delete records, SafePrompt not flagging a prompt is not a reason for that tool to have the permission in the first place.
Human approval
For consequential actions, a detection layer is not a substitute for someone confirming the action.
Guaranteed detection
Detection is probabilistic. Every figure we publish is a measured rate with failures included, not a promise.
Where detection is weakest
These are the classes our own measurement is worst at. They are ordered roughly by how often they cost us a correct verdict.
Extraction worded as ordinary conversation
Our most common miss. A request for the system prompt or prior context, phrased as a reasonable question rather than an obvious override, is materially harder to separate from legitimate curiosity than a direct instruction override.
Security work that looks like an attack
Our most common false positive. Bare code fragments, and questions asking how an attack technique works, are the two classes we over-block. If your product handles security discussion, developer tooling, or educational content, test that traffic specifically before enforcing.
Injection arriving through retrieved content
Indirect injection, where the payload comes from a document, page, or tool result rather than the user, is the hardest class we measure and the one where a held-out probe scored furthest below our continuous suite. Treat retrieved content as untrusted at every boundary, not only at the user turn.
Attacks split across chunks or turns
A payload assembled from fragments, each individually unremarkable, is harder to catch than the same payload in one message.
Languages beyond our largest coverage
Multilingual and multi-turn coverage entered the suite later than the core English cases, so it has less measurement history behind it.
Limits of our own measurement
The benchmark is ours. We wrote the suite, so it cannot be treated as independent evidence. It runs continuously against the production API and every run and every failure is published, which makes it a good regression instrument and a weak substitute for an outside evaluator. An externally curated, sealed holdout is not yet in place.
A held-out probe scored lower. When we tested against cases written to be unfamiliar to the suite, detection came out materially below the continuous figure, with the misses concentrated in indirect and agent-to-agent injection. That gap is the honest measure of how much a familiar benchmark flatters a detector, ours included.
We publish a range, not a headline. Our public 165-case benchmark runs against the production API every 6 hours. Across the last 30 days (103 runs, suite v2.2): 96.25% to 97.5% of attack prompts blocked (median 97.5%), 1.18% to 3.53% of safe prompts flagged (median 1.18%). Re-run it yourself with your own key.
Speed depends on the path. Most requests go to AI semantic analysis: median 1183ms, 95th percentile 3586ms across 121,811 production calls in the last 30 days. Requests the pattern layers resolve on their own return in about 55ms, but those are the minority.
What to pair it with
Detection earns its place as one layer among several. In rough order of how much they limit the blast radius when detection misses:
- Authorization enforced outside the model, never inferred from its output
- Least-privilege tools, with allowlists and validated parameters
- A clear separation between trusted instructions and untrusted content
- Human approval before consequential or irreversible actions
- Output validation, and isolation of anything retrieved from outside
- Logging, monitoring, and rate limits so a miss is visible afterwards
Our reasoning on what we measure and how is in the FAQ, the running benchmark is on GitHub, and data handling is in the privacy policy. If something here is wrong or out of date, tell us on contact.