Memory Poisoning via Unsupported Provenance in Persistent Agent State
First seen Sep 3, 2026 · Updated Sep 3, 2026
This is a defensive research paper, not an active exploit or vulnerability disclosure. It formally describes how persistent AI agents can be poisoned when untrusted inputs (including prompt injections or unverified model inferences) get stored and later recalled as if they were verified agent history or user commitments, and proposes a typed provenance/guardrail system to prevent this. The core contribution is a mitigation architecture, evaluated against a small hand-authored conformance suite, not a report of a real-world attack.
Technical Analysis
The underlying threat class is memory poisoning: persistent agents that reflect, retrieve, and consolidate state can absorb untrusted inputs (prompt injections, hallucinated inferences, unverified retrievals) into long-term memory, after which that content is treated as trusted agent history or a user commitment with no re-verification of its evidential basis. The entry point is any write path into persistent state (reflection, retrieval, consolidation) that lacks provenance tracking, allowing unsupported claims to cross the boundary from 'stored' to 'asserted as true.' The paper's proposed defense—a typed provenance graph plus a generate-verify-revise mediator—gates release of such statements by checking accepted-evidence, temporal-validity, and disclosure policies before an assertion can be surfaced. In their conformance suite, naive flat/prior and source-tag comparison baselines leaked 19/19 and 18/19 unsafe assertions respectively, while the typed mediator blocked all 19 unsafe cases while preserving legitimate ones, demonstrating the practical severity of unmitigated memory poisoning and the effectiveness of provenance-based gating.
Detection Signatures
- N/A - this is an academic defensive framework, not an attack; no IOCs apply.
- Defensive indicator: agent memory writes lacking origin/epistemic-role metadata (no distinction between 'retrieved,' 'inferred,' and 'user-asserted' content).
- Defensive indicator: absence of temporal-validity or staleness checks before re-surfacing stored claims as current fact.
- Defensive indicator: agent responses citing 'memory' or 'prior conversation' without any traceable evidential source tag.
Remediation Steps
- 1
Implement provenance tagging on memory writes
Tag every stored item with origin (user input, tool output, model inference, external retrieval), dependency lineage, and epistemic role before persisting to long-term memory.
- 2
Gate assertion release with verify-before-surface checks
Before an agent presents a memory-derived statement as fact or user commitment, run it through a policy check for accepted-evidence status, temporal validity, and disclosure scope, similar to the paper's generate-verify-revise mediator.
- 3
Avoid flat/prior-only or naive source-tag comparisons
Do not rely solely on simple recency or shallow source-tag matching for trust decisions on stored content, as these were shown to leak the vast majority of unsafe assertions in testing.
- 4
Periodically audit and expire stale memory
Apply staleness and conflict-detection flags to persistent state so outdated or contradicted claims are not resurfaced as current agent history.
Industries Most Exposed
Respond to this threat
Pro subscribers get a full AI-generated incident-response playbook for this threat — detection, containment, eradication, and recovery steps — plus an unlimited AI Threat Advisor for questions about your environment.