How to test whether AI agent memory knows the current decision

The original RAG paper describes a model that combines generation with access to retrieved non-parametric memory. That is useful when the problem is finding evidence relevant to a query. It does not define a universal lifecycle for every record an application may retrieve.

Current agent frameworks also distinguish memory scopes. The LangChain documentation separates thread-scoped short-term memory from long-term stores shared across sessions. LangMem then describes profiles, collections, updates, consolidation, and reconciliation. The practical lesson is not that retrieval is defective. It is that retrieval, persistence, and lifecycle policy are separate design decisions.

Assume the project memory contains two statements:

Yesterday: Use SQLite for the application database.
Today: Replace SQLite with PostgreSQL.

Both fragments are relevant to a question about the database. A similarity search may reasonably return both. The operational question is different: which decision applies now, and what happened to the previous one?

The proof creates a disposable SQLite database, stores both fictional decisions, performs the guarded supersession operation, and checks the resulting current state, history, relation, and provenance. The stable JSON omits temporary paths, generated identifiers, timestamps, and preview hashes while still asserting that the guarded preview exists.

A timestamp can order events. It does not explain their semantic relationship.

• A correction may apply to a different project, environment, customer, or scope.
• A later sentence may be a proposal or opinion rather than an approved decision.
• Two sessions may update separate parts of the same project without replacing each other.
• Creation time does not prove that one record supersedes another.

A reliable current-state query therefore needs an explicit policy: record type, scope, lifecycle state, and the relation between old and new information. “Latest” may be one input to that policy, but it is not the policy itself.

The second reproducible scenario stores two fictional claims:

Operations runbook: Daily backup at 02:00 UTC
Incident review:   Daily backup at 04:00 UTC

Choosing the later record or the more similar fragment would hide the real issue: two named sources disagree. The MAPI proof preserves both records and both source references, creates the contradiction relation, marks the cluster unresolved, and does not select an automatic winner.

Provenance matters because the next step may depend on who made the statement and in what context. W3C PROV provides a general standards family for exchanging provenance information. MAPI uses its own product data model, but follows the same practical need: a record should remain traceable to its source evidence.

The table describes typical roles, not guaranteed capabilities of every implementation. A particular chat system or vector store may add versioning, metadata, conflict handling, or governance on top.

The two scenarios demonstrate a bounded set of product properties:

• Decision supersession produces one current state, preserves the previous decision, and keeps an explicit relationship.
• Conflicting sources remain as two records with provenance, a visible conflict, and no silent overwrite.
• Both scenarios use synthetic data and disposable databases.
• They run locally without an external LLM or API call.
• The runner emits stable, machine-readable JSON and a concise human report.

python scripts/run_product_proofs.py

Inspect the runner source and the methodology and expected results before relying on the summary above.

Supersession is not exclusive to MAPI. For example, current Cloudflare Agent Memory documentation also describes evolving facts through supersession while preserving older versions. The value of the MAPI scenarios is narrower: the repository exposes executable checks for its own claimed behavior.

MAPI can sit alongside retrieval, conversation history, a language model, an agent runtime, and an MCP client. It does not replace the model, RAG, or the rest of an agent architecture. Its focus is project-memory lifecycle: controlled writes, current-state resolution, history, relations, provenance, conflict review, and auditability.

That focus is useful only when the additional controls justify their operational cost. A simple file, transcript, or store can be the better choice when decisions rarely change, conflicts carry little risk, or nobody needs an audit trail.

You can reuse the two questions without adopting MAPI:

1. Store one decision, replace it explicitly, and ask for both the current state and its history.
2. Store conflicting claims from two sources and verify that neither provenance chain disappears before review.

If the system can only return similar text, the missing capability is not necessarily “better search”. It may be an explicit lifecycle and conflict policy. That is the boundary this article is meant to make testable.

The distinction used in this article is grounded in primary documentation and in executable MAPI scenarios:

• Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — the original RAG formulation combines parametric generation with a retrieved non-parametric memory.
• LangChain memory overview — current documentation distinguishes thread-scoped short-term memory from long-term memory shared across sessions.
• LangMem core concepts — collections and profiles illustrate different update models and the need to reconcile new information with previous beliefs.
• Cloudflare Agent Memory documentation — an example of a current memory product that also documents fact supersession and preservation of older versions.
• W3C PROV overview — the standards family for interoperable provenance information.
• MAPI GitHub, proof runner and proof methodology.