Reconnaissance-Driven Pentesting Framework for AI Agents (KYA)

Technical Analysis

KYA performs black-box probing of a target agent to extract 'knowledge assets' such as system prompt structure, tool inventory, permission scope, and behavioral heuristics, then uses this profile to tailor indirect prompt injection payloads for higher success rates. The entry point is any interface the agent exposes to reconnaissance queries (chat interface, tool descriptions, error messages, or observable tool outputs), which the attacker uses to infer internal configuration without white-box access. The gain for an adversary is a significantly more targeted and effective injection payload, since generic injections often fail against unknown guardrails or tool restrictions. This crosses agent/tool boundaries in that reconnaissance findings about one agent's tool set or planner logic are used to craft attacks delivered through untrusted content later ingested by that same agent (classic indirect prompt injection vector), and the paper explicitly validates this against a real-world coding agent.

Detection Signatures

• Repeated or systematic probing queries attempting to enumerate system prompt contents, tool names, or permission boundaries
• Anomalous sequences of exploratory prompts followed by a targeted payload (recon-then-attack pattern)
• Agent logs showing unusual variance-testing queries (e.g., boundary-testing phrasing, meta-questions about capabilities/tools)
• High-frequency, low-payload interactions consistent with automated black-box profiling tools

Remediation Steps

1. 1

   Limit agent self-disclosure

   Prevent agents from revealing system prompt content, tool lists, or internal configuration details in responses to user or tool-originated queries.

2. 2

   Rate-limit and monitor probing patterns

   Detect and throttle sequences of exploratory queries that resemble reconnaissance behavior (e.g., repeated capability/boundary questions) before a payload is delivered.

3. 3

   Harden against indirect prompt injection

   Apply content provenance checks, input sanitization, and strict separation between trusted instructions and untrusted tool/content outputs to reduce injection success regardless of attacker profiling.

4. 4

   Adopt KYA benchmarks defensively

   Use the released KYA framework and benchmarks in internal red-teaming to proactively identify and patch agent weaknesses before adversaries can exploit the same reconnaissance techniques.

5. 5

   Least-privilege tool access

   Restrict agent tool permissions and scopes so that even a well-profiled attack has limited blast radius if injection succeeds.