Persistent Agent Memory Poisoning via Low-Effort False Assertions

Technical Analysis

The entry point is any write path into an agent's persistent/long-term memory store (e.g., conversation logging, tool output ingestion, or user-submitted content that gets summarized and stored) where no instruction, jailbreak trigger, or adversarial optimization is needed — a single plainly worded false statement suffices. Because the false assertion is topically and semantically similar to legitimate queries, it is retrieved in later sessions and treated as ground truth, corrupting downstream reasoning and answers. The paper shows a four-stage write-time screening pipeline tuned for indirect prompt injection (0.832 recall) catches 0 of 360 poisoned memories, since the text contains no injection markers — it is simply false, and falsity cannot be detected from content alone without external grounding. Provenance-weighted retrieval also fails: the shipped weighting is statistically equivalent to no defense, and a weight strong enough to suppress untrusted poisoned content also suppresses legitimate untrusted evidence, driving accuracy and evidence recall to near zero in adversarial mixed-provenance settings. This crosses the boundary from a single write event into all future sessions/agents that query the shared memory store, meaning one poisoned write can degrade many downstream reasoning tasks over time.

Affected Systems

LongMemEval benchmark harness, agent memory systems using provenance-weighted retrieval (generic architecture)

Detection Signatures

• No prompt-injection markers present (plain declarative false statements) — will not be caught by trigger/instruction-pattern classifiers
• High semantic similarity between stored memory entries and later unrelated queries triggering retrieval
• Sudden accuracy drop or answer inconsistency correlated with recently added memory entries
• Memory entries lacking verifiable provenance/source attribution or citation to external ground truth
• Retrieval logs showing untrusted-provenance content being weighted equivalently to trusted content (defense misconfiguration)

Remediation Steps

1. 1

   Adopt bounded occupancy constraints at retrieval

   Rather than additive provenance penalties, cap the number/proportion of untrusted-provenance memory items that can be surfaced per retrieval, as recommended by the paper, to prevent poisoned content from dominating answer context.

2. 2

   Require external grounding for factual writes

   Do not rely on content-only screening to validate truthfulness; cross-check candidate memory writes against trusted external sources, structured knowledge bases, or multi-source corroboration before persisting them.

3. 3

   Segregate and label provenance at ingestion

   Tag every memory entry with a strict provenance/trust level at write time and enforce retrieval-time policies that treat untrusted-origin content differently from verified content, rather than blending them with a single weighting scalar.

4. 4

   Implement memory write auditing and rollback

   Log all memory writes with source and timestamp, monitor for anomalous accuracy degradation, and support selective deletion/rollback of suspect memory entries.

5. 5

   Test defenses against low-effort adversarial baselines

   Benchmark memory-poisoning defenses against plainly worded, non-triggered false assertions (not just optimized/injection-style attacks), since this paper shows that is the harder case to catch.