Key takeaways
- Separate source records, session context, summaries, preferences, inferred facts, and instructions; they do not deserve equal authority.
- Store origin, subject, purpose, confidence, consent state, timestamps, policy, and expiry with every persistent item.
- Untrusted content should not gain permission to drive consequential actions merely because the agent summarized or remembered it.
- Provide inspection, correction, selective deletion, and downstream repair—not only a “clear chat” control.
01
Decide what memory means in this system
A recent conversation window, a retrieved CRM record, a generated summary, and a long-term preference are four different things. Name them separately. Document who can write each store, who can retrieve it, which tasks may use it, and whether it is evidence or merely a hint.
Avoid saving everything because storage is cheap. Persistent context increases privacy, security, and staleness risk. Keep the minimum needed for a declared purpose and prefer authoritative source records when a current lookup is available.
02
Bind provenance to the memory item
Recent research on memory poisoning shows why content and a loose derivation trail can be insufficient: untrusted material may be laundered through summaries, tool echoes, or apparent corroboration. Preserve the original origin and authority class so a later rewrite cannot quietly promote an attacker’s instruction into trusted policy.
Store the source reference, writer, creation time, subject, purpose, confidence, consent state, retention rule, and allowed uses. Do not let retrieved memory redefine system policy or authorize a tool action.
Swipe to compare every column
| Memory type | Use | Guardrail |
|---|---|---|
| Authoritative record | Current verified fact | Refresh from source and respect field ownership |
| User preference | Personalization | Consent, scope, inspection, expiry |
| Generated summary | Navigation aid | Link to source and mark as derived |
| Untrusted content | Evidence for review | Never treat as instructions or authority |
03
Retrieve with purpose and time in view
Filter retrieval by tenant, user, task, permitted data class, and freshness. A preference recorded for support should not automatically shape a credit decision or marketing audience. Rank authoritative current records ahead of old summaries and expose conflicts rather than asking the model to smooth them over.
Log which memory items affected a run so an incident can trace behavior back to stored context without capturing unnecessary hidden reasoning.
04
Design correction and forgetting as workflows
Users and operators need to inspect and correct persistent details. Deleting one vector is not enough when the value has been copied into summaries, caches, CRM notes, or downstream systems. Track derived artifacts and define propagation or manual repair.
Test poisoning, stale facts, identity collisions, cross-tenant retrieval, conflicting preferences, deletion, and restoration from backup. Memory is production data. It deserves a lifecycle, not a clever prompt.
Primary sources and further reading
Use the source material to validate details against your own context and current platform configuration.
- Securing LLM-Agent Long-Term Memory Against Poisoning
- From Untrusted Input to Trusted Memory
- OWASP: Prompt Injection
- NIST Generative AI Profile
This field note follows the XenGrowth editorial policy: primary sources where available, visible limitations, material review dates, and no invented first-hand experience.
Stay with the problem
