Implicit Semantic-Matching Manipulation of LLM Skill/Tool Selection (ISM)
First seen Sep 3, 2026 · Updated Sep 3, 2026
Researchers demonstrate a novel attack (ISM) that manipulates which skill/tool an LLM agent selects by crafting benign-looking skill metadata and prompts whose semantic relationship is engineered to bias the selector, without any explicit steering instructions. This bypasses human review and existing prompt-injection defenses far more effectively than explicit instruction-based attacks, raising the target-selection rate from ~15% baseline to ~63-73%.
Technical Analysis
The attack targets the skill/tool-selection stage of LLM agent frameworks, where a router or planner model picks an installed skill based on semantic similarity between user intent and skill descriptions. Instead of injecting explicit directives ('always choose skill X'), ISM jointly optimizes the attacker-controlled skill's metadata and companion prompt wording in three stages: broadening semantic coverage, sharpening distinctiveness relative to competing skills, and preserving natural, non-suspicious phrasing. Because no overt manipulation cues are present, both human reviewers (2.9% block rate) and LLM-based content inspectors, prompt-guard classifiers, and perplexity-based filters (PPL-W, Llama Prompt Guard 2, PIGuard) largely fail to flag it. The attacker gains control over which skill/tool is invoked for a broad class of benign-looking user requests, effectively hijacking the planner's routing decision and enabling downstream execution of an attacker-chosen skill instead of the legitimate one — a supply-chain-adjacent trust boundary violation within agent marketplaces or plugin ecosystems.
Affected Systems
generic LLM-agent skill/plugin marketplaces, retrieval-based skill routers, semantic tool-selection frameworks (framework-agnostic finding)
Detection Signatures
- Skill/tool descriptions with unusually broad keyword coverage relative to their stated function
- High semantic similarity scores across multiple unrelated user query templates for a single skill
- Skill metadata optimized (e.g., via paraphrase search or embedding-distance tuning) rather than hand-authored
- Anomalous spikes in selection rate for a newly added or low-reputation skill across diverse prompts
- Mismatch between skill's declared capability scope and the diversity of queries it gets selected for
Remediation Steps
- 1
Add selection-decision auditing
Log and periodically audit skill-selection decisions, especially for third-party or recently added skills, comparing observed selection rates against expected semantic scope.
- 2
Enforce provenance and review for skill metadata
Require vetted, static skill descriptions from trusted publishers; flag metadata that has unusually broad semantic coverage or shows signs of adversarial optimization.
- 3
Use ensemble/behavioral selection checks
Combine semantic-similarity routing with secondary checks (e.g., capability verification, sandboxed dry-run, or rule-based allowlists) rather than relying solely on embedding similarity.
- 4
Monitor for distributional drift
Track skill-selection frequency over time per skill/publisher; alert on sudden increases that don't correlate with legitimate usage growth.
- 5
Harden guardrails against non-instructional manipulation
Recognize that perplexity filters and prompt-injection classifiers (PPL-W, Prompt Guard, PIGuard) are ineffective against this class of attack; incorporate semantic-relationship analysis between prompt and skill description as a distinct detection layer.
Industries Most Exposed
Respond to this threat
Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.