How to Test for Prompt Injection

To test for prompt injection, you send crafted inputs that attempt to override or hijack a model’s original instructions, then check whether the system follows the attacker’s instruction instead of its own. Effective testing combines manual adversarial prompts with automated payload libraries, covers both direct and indirect injection paths and measures whether the model leaks data, ignores guardrails or performs unintended actions.

Prompt injection is one of the most common weaknesses in production language-model applications. If your organisation has deployed a chatbot, a retrieval system or an agent that calls tools, testing for prompt injection should be part of your security assurance work. This guide explains what to test, how to structure the work and how to interpret the results.

What is prompt injection

Prompt injection is an attack where a user supplies input designed to change the behaviour of a language model beyond what the application intended. The model treats attacker-supplied text as instructions rather than data, so a crafted message can override system prompts, bypass safety rules or extract hidden context.

Prompt injection splits into two broad categories. Direct injection happens when the attacker types the malicious instruction straight into the interface. Indirect injection happens when the payload arrives through a document, web page, email or database record that the model later reads, which means the attacker never touches the application directly.

Is prompt injection a vulnerability

Yes. Prompt injection is a genuine security vulnerability, and it is listed as the top risk in the OWASP Top 10 for Large Language Model Applications. It is treated as a vulnerability because it lets an attacker cause the system to act against its owner’s intent: leaking data, calling tools without authorisation or producing harmful output.

The severity depends on what the model can reach. A model that only returns text carries lower risk than an agent connected to databases, payment systems or the Model Context Protocol, where a successful injection can trigger real actions with real consequences.

How to plan a prompt injection test

Start by mapping every point where untrusted input enters the system. This includes the obvious chat box and the less obvious paths: uploaded files, retrieved documents, tool responses, API parameters and any content pulled from external sources at run time.

For each entry point, define what a successful attack would look like. Common objectives are instruction override, system-prompt disclosure, data exfiltration from connected sources and unauthorised tool invocation. Writing these objectives down first keeps the test focused and makes the results measurable rather than anecdotal.

Prompt injection testing techniques

Prompt injection testing combines manual creativity with automated coverage. Manual testing lets you reason about the specific application logic, while automated payload sets give breadth and repeatability. Both belong in a complete assessment.

Useful techniques include:

• Instruction override: ask the model to ignore previous instructions and follow yours instead, then check whether it complies.
• System-prompt extraction: attempt to make the model reveal its hidden system prompt or configuration.
• Role and persona manipulation: instruct the model to adopt a role that removes its restrictions.
• Encoding and obfuscation: hide instructions inside encodings, translations or unusual formatting to bypass simple filters.
• Indirect payloads: place instructions inside a document or web page the model will later process, and confirm whether it executes them.
• Tool and function abuse: for agents, attempt to trigger tool calls or MCP actions the user should not be able to reach.

Run each technique against every entry point you mapped, and record the input, the expected safe behaviour and the actual response.

How to evaluate the results

A prompt injection test produces a list of inputs and the model’s responses. Evaluate each result against the objective you defined: did the model follow the attacker’s instruction, disclose protected information or perform an unintended action?

Classify findings by impact rather than by cleverness. An injection that only changes tone is low severity; one that exposes another user’s data or triggers an unauthorised transaction is high severity. Prioritise remediation by what the model can actually reach, because the same payload carries very different risk depending on the connected systems.

How to reduce prompt injection risk

No single control eliminates prompt injection, so defence relies on layers. Separate trusted instructions from untrusted input, constrain what the model is allowed to do and validate output before it triggers any action.

Practical measures include restricting tool permissions to the minimum needed, treating all retrieved content as untrusted, filtering and monitoring inputs and outputs and requiring human confirmation for high-impact actions. Regular retesting matters because model updates and new integrations can reintroduce weaknesses that a previous assessment cleared.

Common questions about prompt injection testing

How often should we test for prompt injection?

Test whenever you change the system prompt, add a data source, connect a new tool or upgrade the underlying model and at regular intervals in between. Each of these changes can introduce a new injection path or reopen one that earlier testing had closed, so treat testing as an ongoing activity rather than a one-off exercise.

What is the difference between direct and indirect prompt injection?

Direct injection is when an attacker types a malicious instruction straight into the application. Indirect injection is when the payload is hidden in external content, such as a web page or document, that the model reads later. Indirect injection is harder to spot because the attacker never interacts with the application directly.

Can prompt injection be fully prevented?

No current technique fully prevents prompt injection, because the model cannot reliably distinguish trusted instructions from untrusted input. The realistic goal is to reduce likelihood and limit impact through layered controls: minimal tool permissions, output validation and human approval for sensitive actions. This keeps a successful injection from causing serious harm.

Does prompt injection only affect chatbots?

No. Any system that passes untrusted text to a language model is exposed, including retrieval systems, document summarisers, email assistants and autonomous agents. Agents that call tools or use the Model Context Protocol carry the highest risk, because a successful injection can trigger real actions rather than only producing text.

Next steps

If you are deploying language-model applications and want structured assurance against prompt injection and related risks, explore our AI Security Projects service and the AI Security Projects insight, which sets out how MCP servers, prompt injection testing and agent runtimes fit together. Our AI behaviour verification work turns test findings into evidence auditors accept, and the companion guides on LLM penetration testing and red-teaming and evidence go deeper on method. To discuss a prompt injection assessment for your environment, get in touch with QL Security.