ContainmentBench: Post-Injection Containment Evaluation Benchmark (Research)
First seen Jul 29, 2026 · Updated Jul 29, 2026
This is an academic research paper introducing a benchmark for evaluating how well tool-using LLM agents contain the effects of prompt injection after it occurs, rather than a report of an active exploit or vulnerability. It proposes trace-based metrics to distinguish policy-endpoint outcomes from logged propagation and lost legitimate utility. No new attack technique, exploit, or real-world incident is disclosed; findings are synthetic, single-model, and methodological.
Technical Analysis
The paper does not describe a novel attack vector; instead it builds a sandboxed benchmark (ContainmentBench) that runs many rollouts of a tool-using agent (Qwen2.5-7B-Instruct) exposed to injected/tainted content and measures four separate dimensions: whether a terminal policy was violated, whether taint propagated through logged trajectories, whether recovery mechanisms triggered, and whether authorized workflows still completed successfully. Key finding: enforcement mechanisms that look identical at the 'zero committed harm' policy-outcome level can differ hugely in operational cost — taint-only enforcement preserves only ~16% of authorized tainted-workflow utility, while intent-aware/trusted-ledger and tool-boundary approaches preserve 85-92%. This is valuable for defenders designing containment/taint-tracking systems, since it shows that binary pass/fail security evaluations mask large differences in usability tradeoffs and incomplete propagation tracking. The work is entirely synthetic, single-model, and assumes a correct structured authorization ledger, so it is a methodology/evaluation contribution rather than evidence of an exploitable weakness in deployed systems.
Detection Signatures
- N/A - this is an evaluation methodology paper, not a description of an in-the-wild attack. Defenders building similar containment evaluations should look for: divergence between terminal policy-compliance logs and full trajectory/taint-propagation logs, sudden drops in authorized task completion rate under taint-tracking enforcement, and recovery-instrumentation events not reflected in endpoint policy outcomes.
Remediation Steps
- 1
Adopt multi-dimensional security evaluation
When assessing agent containment mechanisms, measure endpoint policy compliance, logged taint propagation, recovery triggering, and authorized-task completion separately rather than relying on a single pass/fail terminal outcome.
- 2
Benchmark utility loss alongside safety
Track how much legitimate/authorized functionality is lost under different containment strategies (taint-only vs. intent-aware vs. tool-boundary) to avoid deploying overly restrictive controls that break agent usability.
- 3
Validate structured authorization ledgers
If using ledger-based or intent-aware enforcement, ensure the correctness and integrity of the authorization ledger, since the paper notes this validity assumption is critical to the reported gains.
- 4
Stage-stratified logging
Instrument agent pipelines to log taint propagation at each stage (ingestion, planning, tool call, output) rather than only recording final policy outcomes, to enable post-incident forensic analysis.
Industries Most Exposed
Respond to this threat
Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.