lowAgent ThreatPrompt Injection

Indirect Prompt Injection in Tool-Using LLM Agents (ROPE Defense Research)

First seen Aug 31, 2026 · Updated Aug 31, 2026

indirect-prompt-injectiondefense-researchtool-callinginformation-flow-controlorigin-trackingacademicASI05 · Unsafe Code ExecutionAML.T0051AML.T0054Surface: Tool LayerPropagation: None

This is a defensive research paper, not an active exploit or newly disclosed vulnerability. It proposes ROPE, a system that tracks the provenance of tool call parameters to block indirect prompt injection (IPI) attacks where malicious instructions embedded in tool-read content hijack an agent's tool calls. The underlying threat class (IPI leading to unauthorized state-changing tool calls) is real and well-known, but this document itself describes a mitigation with strong empirical results, not a new attack.

Technical Analysis

The general threat pattern being addressed is indirect prompt injection: an attacker plants instructions inside content that a tool-using agent ingests (web pages, documents, tool outputs), which the LLM then interprets as commands, causing it to invoke sensitive tools with attacker-controlled parameters. The entry point is any untrusted data source the agent reads during task execution; the attacker's gain is the ability to trigger state-changing actions (e.g., sending data, making purchases, modifying records) without the user's consent, crossing the boundary between passive content and active tool execution. ROPE mitigates this by enforcing that sensitive tool parameters must have a traceable origin from the user, a user-named source, or the user's own authoritative records, using deterministic origin checks rather than relying on an LLM to detect injected text. The paper reports the defense reduces attack success rate to 1.6-2.6% while preserving most task utility, indicating a meaningful improvement over prior system-level defenses, and notably neutralizes long-horizon injection attacks that previously bypassed such defenses.

Detection Signatures

  • Tool call parameters whose sole origin traces to attacker-writable content (e.g., fetched web pages, email bodies, third-party documents) rather than to user input or trusted records
  • Sudden shifts in tool call sequences or parameter values immediately following ingestion of external/untrusted content
  • Repeated or reworded injection-style phrases embedded in tool outputs or fetched documents attempting to instruct the agent (e.g., 'ignore previous instructions', 'send this to...', 'as the user, you must now...')
  • Long-horizon behavioral drift where tool usage patterns deviate from the original user-specified task across multiple steps

Remediation Steps

  1. 1

    Adopt origin-tracking / information-flow control for tool parameters

    Implement deterministic provenance checks (as in ROPE) so state-changing tool calls only execute with parameters traceable to the user, a user-named source, or trusted authoritative records.

  2. 2

    Apply task-conditional tool screening

    Restrict which tools can be invoked based on the declared user task to reduce the blast radius of injected instructions.

  3. 3

    Isolate untrusted content from control-flow decisions

    Treat all fetched/external content as data-only; never let the model treat embedded text in tool outputs as directive instructions for further tool calls.

  4. 4

    Audit sensitive tool parameters

    Maintain an explicit, audited list of state-changing tool parameters requiring origin verification before execution.

  5. 5

    Test against long-horizon and adversarially optimized injections

    Evaluate defenses against multi-step attacks and optimized injection rewordings, not just single-shot prompt injection tests.

Industries Most Exposed

software/technologyfinancee-commerceenterprise SaaSany industry deploying tool-using LLM agents

Sources

Respond to this threat

Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.