Static Analysis Blind Spots in Malicious Agent Skill Detection (SkillsMetric Study)

Technical Analysis

Agent Skills are packaged bundles of instructions and executable scripts that extend LLM agent capabilities, distributed and installed similarly to plugins/extensions, making them a supply-chain attack surface. The SkillsMetric framework applies five static detection stages (pattern density, statistical anomaly, dataflow taint, import anomaly, capability mismatch) to a large adversarial corpus (2,266 malicious skills across 16 attack types, evaluated on SkillMD-138K). Findings show host-destruction attacks that rely on ordinary shell commands blend in with legitimate functionality and evade all five static stages entirely (0% detection), while natural-language prompt injection embedded in skill instructions—rather than code—is only caught 42% of the time because it lacks the syntactic signatures static analyzers look for. An attacker distributing a poisoned skill package could thus achieve code execution or goal hijacking on the host agent's runtime environment while evading automated static vetting, requiring semantic/behavioral review to close the gap.

Detection Signatures

• Skill packages invoking shell/system commands with destructive intent (e.g., recursive delete, disk overwrite, permission changes) disguised as maintenance/cleanup utilities
• Natural-language instructions within skill metadata or prompts that attempt to override agent behavior, request credential disclosure, or redirect task goals
• Mismatch between declared skill capability/purpose and actual imported modules or invoked system calls
• Skills with unusually low code complexity but high natural-language instruction density (indicative of NL-based injection rather than code-based attack)
• Statistical anomalies in encoding/obfuscation patterns suggestive of steganographic payloads (already well-detected but useful baseline)

Remediation Steps

1. 1

   Layer semantic review on static analysis

   Combine automated static scanning with LLM-based or human semantic review of skill instructions to catch natural-language prompt injection that static pattern matching misses.

2. 2

   Sandboxed execution and command allowlisting

   Run skill scripts in sandboxed, permission-restricted environments with allowlisted shell commands to prevent host-destruction attacks that use benign-looking system calls.

3. 3

   Capability declaration enforcement

   Require skills to declare capabilities/permissions upfront and enforce runtime checks that block behavior exceeding the declared scope.

4. 4

   Marketplace vetting pipeline

   Implement a defense-in-depth vetting pipeline for skill marketplaces: fast static pre-screen, dynamic/behavioral sandbox testing, and periodic re-scanning of published skills.

5. 5

   Runtime monitoring and anomaly detection

   Deploy runtime monitoring on agent hosts to detect destructive filesystem/system operations and anomalous goal deviation post-installation, since static pre-install checks are insufficient alone.