Two-Stage Agentic Memory Generation Architecture

The retrieve-at-query-time pattern dominates production memory systems for pragmatic reasons: it is simple to implement, it handles the conversational case adequately, and it places the memory burden entirely on the retrieval layer, which is a well-studied problem. For session-based assistants handling a bounded conversation, it is often sufficient.

It fails in three ways as systems scale:

Latency budget: Retrieving and reasoning over raw source material — emails, documents, prior conversations — inside the inference latency budget means the user waits for memory assembly on every query. The expensive work happens in the most latency-sensitive window.

Signal density: Raw source material is low-signal-density. Most content in an email thread, a document, or a conversation transcript is not relevant to any given query. Retrieval over raw sources requires the model to do significant filtering and synthesis work at query time — work that is duplicated on every subsequent query that touches the same source.

No amortization: If the same source material is relevant to 50 downstream queries, the retrieve-at-query-time pattern pays the synthesis cost 50 times. The two-stage architecture pays it once.

The separation is not merely an engineering optimization. It reflects an epistemologically correct framing of the enterprise memory problem: the reasoning required to extract signal from noise — determining what from a large source artifact is worth preserving — is too expensive to perform inside the inference latency budget at scale. That reasoning belongs in Stage 1, where it can be scheduled, cached, and amortized.

The two-stage architecture is model-forward by construction. The mechanism is precise: Stage 1 generation quality is a direct function of the foundation model's reasoning capability. When the foundation model improves — through a model update, a capability jump, or a fine-tuned variant — Stage 1 produces better memory entries from the same source material, without any change to the Stage 2 retrieval layer or the memory store schema.

This contrasts with architectures that embed memory logic in prompt templates or retrieval heuristics: those must be re-engineered when the model changes, because the logic is coupled to a specific model's behavior. The two-stage architecture decouples memory quality from model specifics. The model executes the generation contract; it does not define it.

The two-stage architecture is well-understood for the conversational case: a human-bot conversation produces a transcript, Stage 1 reads it and extracts structured memory entries. The transcript is the trace, and the trace is human-readable, relatively dense with signal, and structurally familiar.

The open engineering problem is what happens when the source stream is not a conversation.

For agentic artifact streams, Stage 1 cannot simply read and summarize. It must determine, agentically, what from this tool trace or plan execution record is worth preserving — and that determination requires understanding task intent, execution outcome, and downstream relevance across a heterogeneous stream.

This is a genuinely different retrieval problem from the conversational case. Enterprise-scale deployments that operate through non-human-bot infrastructure generate artifact streams that are orders of magnitude larger than conversation transcripts, with far lower signal density per token. The systems-level challenge — how to build a Stage 1 process that is both selective and comprehensive across these streams — does not yet have a canonical solution in the published literature.