ContainmentBench: Benchmark for Post-Injection Containment Evaluation in Tool-Using LLM Agents

Technical Analysis

The paper proposes ContainmentBench, a sandboxed evaluation framework that traces agent behavior after a prompt-injection event across four dimensions: endpoint policy compliance, logged taint propagation, recovery instrumentation, and completion of authorized tasks. Using Qwen2.5-7B-Instruct in a synthetic, single-model study, the authors show that different containment strategies (taint-only enforcement, trusted-ledger policy, tool-boundary baseline) can produce identical 'zero committed-harm' endpoint outcomes while differing substantially in logged trajectory and legitimate task completion rates (16% vs 86% vs 92%). No novel attack vector, exploit chain, or real-world incident is described; the work is a measurement/evaluation contribution intended to improve how researchers assess post-injection containment in agentic systems. The authors themselves caveat that results are synthetic, single-model, and assume a correct structured authorization ledger.

Detection Signatures

• N/A - this is a benchmark/evaluation methodology paper, not an active threat with observable indicators.
• Defenders building similar evaluations should log: taint provenance per tool call, authorization ledger state changes, and divergence between endpoint policy outcome and intermediate trajectory events.

Remediation Steps

1. 1

   Adopt trace-based evaluation for agent security testing

   When evaluating agent defenses against prompt injection, measure more than terminal harm/no-harm outcomes; instrument and log taint propagation, recovery actions, and legitimate task completion rates to avoid false confidence from equal endpoint labels.

2. 2

   Prefer tool-boundary and structured authorization controls over taint-only filtering

   The study's findings suggest tool-boundary enforcement and trusted authorization ledgers preserve significantly more legitimate agent utility than taint-only tracking while maintaining equivalent harm-prevention outcomes; prioritize these architectures in production tool-using agents.

3. 3

   Validate authorization ledger integrity

   Since the ledger-based policy's benefits depend on a correct structured authorization ledger, ensure any production implementation of similar controls has strong integrity guarantees on that ledger to avoid it becoming a single point of failure.

4. 4

   Extend evaluation beyond synthetic single-model studies

   Before relying on containment strategy comparisons for production decisions, validate findings across multiple models, real-world task distributions, and adversarial (not just synthetic) injection scenarios.