RIPA: Multi-Channel Sensory Prompt Injection Against LLM-Controlled ROS 2 Robots

Technical Analysis

The attack exploits the fact that sensory inputs (OCR-extracted text, transcribed audio, or environment-state summaries derived from LiDAR data) are fed into the robot's LLM controller as part of its system or context prompt without adequate trust boundaries, allowing adversarial content embedded in the physical environment to be interpreted as legitimate instructions. Entry points include printed/displayed text captured by cameras, spoken commands captured by microphones, and fabricated obstacle data injected into the environment-state representation used at the LLM system-prompt level. Successful injection allows an attacker to override the robot's intended goal, causing it to execute attacker-chosen actions (goal-hijack) with physical-world consequences, and the vulnerability is model-specific rather than scale-dependent, meaning larger/newer models are not inherently safer. The proposed hybrid semantic firewall mitigates known patterns (0% ASR) but is bypassed 10.2% of the time by adversarially obfuscated payloads across 19 obfuscation techniques, showing that rule-based/semantic filtering alone is an insufficient boundary between untrusted sensor data and the LLM's action-planning context.

Affected Systems

ROS 2, DeepSeek-V4-Flash, Llama-3-8B-Instruct-Lite, Llama-3.3-70B-Instruct-Turbo, Qwen 2.5-7B-Instruct-Turbo, Gemma-3n-E4B

Detection Signatures

• Unexpected instruction-like text embedded in camera-captured images (OCR output) that resembles commands rather than environment labels
• Audio transcripts (Whisper STT output) containing imperative language inconsistent with expected ambient speech
• Sudden discrepancies between raw LiDAR/sensor data and LLM-reported environment-state descriptions (fabricated obstacles/clear paths)
• High-entropy or unusual token sequences in sensor-derived text fields feeding the system prompt
• Repeated firewall rule matches followed by benign-looking obfuscated variants achieving execution
• Logging showing system-prompt updates triggered directly by sensor pipeline without provenance tagging

Remediation Steps

1. 1

   Enforce sensor-data/instruction separation

   Architect the LLM controller so that data derived from cameras, microphones, and LiDAR is never concatenated into the system prompt as trusted instruction text; use structured, schema-validated fields with explicit provenance tags distinguishing 'observation' from 'command'.

2. 2

   Layered defense beyond semantic firewall

   Combine the semantic firewall with independent rule-based sanity checks on physical plausibility (e.g., cross-validate LiDAR-derived obstacle claims against raw sensor readings) and anomaly detection on sensor-to-prompt pipelines, since firewall alone leaves a 10%+ bypass gap.

3. 3

   Adversarial red-teaming with obfuscation coverage

   Continuously test defenses against the full obfuscation taxonomy (19 payload types, 5 categories) identified in this research, updating firewall rules and retraining detection classifiers as new bypass techniques emerge.

4. 4

   Model selection and per-model vulnerability testing

   Do not assume larger or newer models are safer; benchmark each candidate LLM's ASR against known injection channels before deployment in physical robotic systems, since vulnerability is model-specific not scale-dependent.

5. 5

   Physical safety fail-safes independent of LLM trust

   Implement hardware/software safety interlocks (e.g., independent obstacle-avoidance systems, action rate limiting, human-in-the-loop confirmation for high-risk actions) that do not rely solely on LLM-reported environment state.