lowAgent ThreatPrompt Injection

BASIS: Attention-Probe-Based Selective Prompt Injection Defense

First seen Aug 11, 2026 · Updated Aug 11, 2026

prompt-injectiondefense-researchover-refusalattention-probesLLM-securitydetection-methodASI01 · Goal HijackingAML.T0051Surface: ModelPropagation: None

This is a defensive research paper, not an active exploit or vulnerability disclosure. It proposes BASIS, a method using attention-based linear probes to detect prompt injection attempts while reducing unnecessary refusals on inputs that robust models could safely handle. No new attack technique or vulnerability is introduced; the content describes a mitigation approach.

Technical Analysis

The paper addresses the known threat of prompt injection, where attacker-controlled text embedded in user or external data attempts to hijack LLM behavior. Rather than a novel attack, it introduces a defensive mechanism: an 'Attention Competition Ratio' feature extracted during prefill is fed into two lightweight linear probes (an existence probe and a breach probe) that decide, via cascaded gating, whether an injection attempt would actually compromise the specific model instance rather than blanket-refusing on any detected injection. This reduces false-positive refusals while maintaining detection of genuine breaches, evaluated across four tasks and six open-source LLMs. As a detection/defense technique, it does not itself create a cross-agent or cross-tool attack path, though it implicitly acknowledges that injection robustness varies by model and instruction template, which downstream defenders should account for.

Detection Signatures

  • N/A - this entry describes a proposed defense mechanism, not an attack signature.
  • General prompt injection indicators still apply: embedded imperative instructions in retrieved documents, unusual role-switching language in tool outputs, instructions attempting to override system prompts within user-supplied content.

Remediation Steps

  1. 1

    Evaluate BASIS-style probes for production LLM pipelines

    Consider integrating attention-based existence/breach probes as a lightweight pre-inference filter to reduce both missed injections and over-refusal, especially in high-throughput agent pipelines.

  2. 2

    Benchmark instruction robustness per model

    Since injection robustness varies significantly by model and instruction template, maintain model-specific robustness profiles rather than a one-size-fits-all refusal policy.

  3. 3

    Layer defenses

    Use such probes as one layer alongside input sanitization, output validation, and least-privilege tool access rather than as a sole defense.

  4. 4

    Monitor for over-refusal regressions

    Track false-positive refusal rates in production to ensure any deployed defense does not degrade legitimate agent functionality.

Industries Most Exposed

AI/ML platform providerssoftware developmentany industry deploying LLM-based agents with external data ingestion

Sources

Respond to this threat

Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.