Model-Specific Tool-Schema Drift in Coding Agent Edit Tools
First seen Jul 5, 2026 · Updated Jul 5, 2026
This report describes a reliability/compatibility quirk, not a security threat: newer Claude models (Opus 4.8, Sonnet 5) sometimes emit malformed tool call arguments with invented fields when used with third-party coding harnesses like Pi, likely due to RL training tuned specifically for Anthropic's own Claude Code edit tool. There is no malicious actor, injection, or exploitation involved—this is a model behavior/engineering problem causing failed tool calls and retries.
Technical Analysis
The issue arises because newer Anthropic models appear to have been fine-tuned via reinforcement learning to strongly prefer the specific edit-tool schema used internally by Claude Code (a search-and-replace based interface). When these same models are deployed in a different harness (Pi) with a differently-shaped but semantically similar edit tool schema, the model sometimes hallucinates extra, non-existent fields in the nested edits[] array, causing schema validation failures and requiring retries. This is a generalization/overfitting problem in tool-calling behavior, not an attack vector: there is no untrusted input, no cross-agent boundary being crossed maliciously, and no attacker-controlled data. The 'gain' here is purely incidental degraded reliability, not exploitation, though it does illustrate a broader supply-chain-adjacent concern that model providers' RL tuning on their own first-party tools can silently degrade compatibility with third-party tool schemas.
Affected Systems
Pi, Claude Code
Detection Signatures
- Repeated tool-call validation failures citing unexpected/unknown keys in edits[] array
- Elevated retry rates for edit-tool calls specifically on Opus 4.8 / Sonnet 5 compared to older Claude models
- Schema validation errors referencing invented fields not present in the tool's declared JSON schema
Remediation Steps
- 1
Add lenient schema handling
Have third-party harnesses tolerate or strip unexpected extra fields in tool call arguments rather than hard-rejecting the entire call, when the core edit intent is otherwise valid.
- 2
Align tool schemas with model training
Consider offering multiple edit-tool schema variants (e.g., a Claude-Code-compatible search/replace format and an apply_patch style) and select based on detected model family.
- 3
Monitor per-model failure rates
Track tool-call validation failure rates segmented by model version to detect regressions introduced by upstream model updates.
- 4
Report to model provider
File feedback with Anthropic about cross-harness generalization regressions so future RL tuning accounts for third-party tool schema diversity.
Industries Most Exposed
Respond to this threat
Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.