Skip to content
Agent Memory Governance

An AI Agent Should Not Remember Everything It Can Reach

Separate temporary context, working memory, verified customer facts and preferences with provenance, expiry, correction and deletion propagation.

Operations team separating records into distinct retention trays

Field note

By XenGrowth EditorialPublished Reviewed 12 min read

Key takeaways

  • Classify conversational context, task state, verified facts, preferences, derived summaries, and regulated records separately.
  • Attach source, confidence, purpose, owner, access policy, expiry, correction, and deletion state to every durable memory.
  • Keep write authority narrow; a model suggestion should not silently become a customer fact.
  • Test cross-account leakage, stale memory, prompt injection, correction, export, expiry, and downstream deletion.

01

A useful recollection can become a durable mistake

A prospect says, “I may move next spring.” The agent summarizes that as a confirmed relocation, writes it to the CRM, and uses it in every later conversation. Another workflow copies the summary into an audience. The system feels attentive until the customer has to correct a fact they never stated.

Memory is not one store. Define ephemeral turn context, temporary task state, approved operational history, customer-declared preferences, verified CRM facts, derived summaries, and records governed by retention rules. Give each class a different write path and lifetime.

Swipe to compare every column

Memory classDefault lifetimeWrite rule
Turn contextCurrent interactionNever promoted automatically
Working stateTask or workflow windowExpire after completion or failure
Customer factPurpose-based retentionVerified source or human approval
PreferenceUntil changed or expiredCustomer-declared or clearly inferred

02

Store provenance beside the memory

A durable item should carry the source event, exact or summarized value, actor, timestamp, purpose, confidence, transformation, access policy, owner, expiry, correction history, and systems that received it. W3C PROV offers a vocabulary for entities, activities, derivation, and responsibility; even a simpler schema should preserve that chain.

Keep the original evidence when policy allows, or store a controlled pointer. A free-form summary without source context is hard to contest and easy to over-trust. Treat model-generated memory as a proposal until a rule or person authorizes promotion.

03

Memory expands both privacy and security exposure

Minimize fields and retrieval scope by task, tenant, customer, and time. Use separate read and write permissions. OWASP’s excessive-agency guidance emphasizes reducing unnecessary functionality, permissions, and autonomy; an agent that only needs to read a preference should not have broad CRM mutation rights.

Assume retrieved text can contain manipulation. Do not let notes, emails, or web content instruct the agent to expose memory or change policy. Validate identity, object, field, action, and value outside the model before reading or writing consequential records.

04

Exercise forgetting before promising it

Test correction, expiry, account separation, export, deletion, cache invalidation, vector-index removal, backup policy, and every downstream copy. A record deleted from the CRM but still retrieved from an embedding index has not been forgotten in the experience that matters.

Give people a practical correction route and operators a lineage view. Monitor denied writes, promotion rates, stale retrievals, cross-tenant attempts, deletion lag, and repeated corrections. Good memory is selective, attributable, and willing to let go.

Primary sources and further reading

Use the source material to validate details against your own context and current platform configuration.

This field note follows the XenGrowth editorial policy: primary sources where available, visible limitations, material review dates, and no invented first-hand experience.

Stay with the problem

Explore AI & automation