Prompt Injection Evasion of LLM-based SOC Log Interpretation

Technical Analysis

The attack surface is the LLM's text-processing pipeline within log-analytics tools: an attacker who can influence log content (e.g., via user-agent strings, filenames, HTTP headers, or other logged fields) embeds contextual framing or explicit instructions designed to bias the model's classification judgment. Because the LLM treats untrusted log text as part of its reasoning context, injected content can override or dilute the indicators of compromise present in the same log trace, causing false-negative classifications. This crosses a trust boundary between raw untrusted telemetry and the model's decision-making process, effectively allowing an external, low-privilege actor (anyone who can generate logs) to manipulate a downstream security decision. The paper shows the effect is reproducible across multiple state-of-the-art LLMs and various optimization strategies (generic, refined, attack-specific), and identifies that model-generated explanations often leak evidence of manipulation, offering a viable detection signal.

Detection Signatures

• Log entries containing natural-language instructions, meta-commentary, or context-framing phrases embedded in fields normally expected to be structured/technical (e.g., user-agent, referrer, filename, process name)
• Unusually verbose or grammatically fluent text in fields typically populated by machine-generated values
• LLM explanation output that references instructions, benign framing, or reasoning inconsistent with raw log content
• Discrepancies between classifier confidence and presence of known IOCs in the same log entry
• Repeated attempts to inject phrases resembling '(ignore|disregard|this is a test|not malicious|authorized)' patterns within log fields

Remediation Steps

1. 1

   Sanitize and structure log inputs

   Strip or escape natural-language content from fields that should contain structured machine data before passing to the LLM; enforce schema validation on log fields.

2. 2

   Use explanation-based anomaly detection

   Implement automated scanning of LLM-generated rationales for injection indicators (e.g., references to instructions, unusual justification patterns) as a secondary detection layer, as proposed in the paper.

3. 3

   Ensemble and cross-validation

   Combine LLM-based classification with traditional signature/rule-based detection and require agreement or flag divergence for human review.

4. 4

   Adversarial robustness testing

   Red-team LLM-based log analysis pipelines using known injection generation/refinement techniques before production deployment.

5. 5

   Least-trust model integration

   Treat LLM verdicts as advisory rather than authoritative for SOC triage; retain human-in-the-loop review for classifications with low confidence or conflicting explanation signals.