Context- and Query-Aware Detection of Indirect Prompt Injection (Research Defense)

Technical Analysis

The paper addresses indirect prompt injection (IPI), where malicious instructions are embedded in text consumed by an LLM-based agent (e.g., retrieved documents, tool outputs, web content) to hijack its behavior. The proposed defense performs sentence-level classification that is aware of both the surrounding context and the user's original query, improving over prior detectors that lack this granularity. To resist adaptive attackers who might paraphrase or perturb injected instructions to evade detection, the authors use two adversarial training approaches: embedding-space projected-gradient perturbations and LLM-based paraphrase simulation during training. This is a mitigation contribution for the model/planner attack surface rather than a description of a new attack technique or vulnerable system.

Detection Signatures

• N/A - this entry describes a defensive research method, not an attack signature.
• General IPI detection heuristics referenced in the domain: sentences containing imperative instructions embedded in otherwise descriptive/retrieved content, instructions that reference altering agent behavior/goals, out-of-context directives inconsistent with the surrounding document's topic or the user's original query intent.

Remediation Steps

1. 1

   Evaluate and adopt context-aware IPI detectors

   Consider integrating query- and context-aware malicious sentence classifiers (such as the CAD approach described) as a pre-processing filter before untrusted text reaches agent planning or tool-calling logic.

2. 2

   Adversarially harden detection models

   If deploying ML-based prompt injection detectors, incorporate adversarial training (embedding-space perturbation and LLM-paraphrase simulation) to reduce susceptibility to adaptive evasion.

3. 3

   Domain-tune detection thresholds

   Per the paper's findings, tune utility/robustness tradeoffs per application domain rather than using one-size-fits-all detection thresholds.

4. 4

   Layer defenses

   Use IPI detection as one layer alongside privilege separation, output filtering, and human-in-the-loop review for high-risk agent actions, since no single detector is guaranteed complete against novel adaptive attacks.