AURVIZ O1 is designed to continuously perceive and understand the world around its wearer. Over weeks and months of use, this produces a substantial stream of conversational, visual, and contextual information that can grow to hundreds of thousands of tokens. A conventional language model cannot efficiently process this entire history for every query. Even when the complete history fits within a model’s context window, repeatedly processing large amounts of irrelevant information introduces unnecessary computational cost and latency.
We built Memtwin, the long-term memory system underlying AURVIZ O1, to address this problem. Rather than repeatedly exposing the language model to an entire interaction history, Memtwin transforms continuous interaction data into a compact, indexed memory representation and retrieves only the information relevant to a particular query. This creates a separation between memory retrieval—identifying the conversations or passages containing the required evidence—and answer generation, where the retrieved evidence is used to produce the final response. This separation is important because retrieval and reasoning represent distinct failure modes: a system can retrieve the correct information but fail to answer correctly, or generate a plausible answer despite retrieving incomplete evidence.
The Long-Term Memory Problem
Long-term conversational memory is fundamentally an information-retrieval problem. The LongMemEvalWu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR, 2025. arXiv:2410.10813. benchmark illustrates this directly. In the benchmark authors’ own evaluation, using GPT-4o as the reader, a model given only the conversations containing the information required to answer a question achieves 92.4% accuracy. When the same model must operate over the complete conversation history, accuracy falls to 64.0%. This 28.4 percentage-point difference represents the information-retrieval gap that a memory system must address.
The objective of a long-term memory system is therefore not simply to store more information. It is to recover the information that matters for a particular query and provide it to the reasoning model in a compact, relevant form. Memtwin operates primarily within this retrieval layer, transforming a large and continuously growing interaction history into a smaller set of high-value evidence.
Memory Architecture
AURVIZ is designed around a persistent multimodal memory architecture that enables the system to retain, organize, retrieve, and reason over information accumulated across interactions. Rather than treating each interaction as an isolated context window, the architecture continuously transforms multimodal observations and conversations into persistent memory representations that can be indexed and retrieved across sessions. The memory layer combines episodic context with a bi-temporal knowledge graph, providing both contextual fidelity and structured, temporally grounded representations of accumulated experience.
Memory Pipeline
The memory pipeline begins with multimodal inputs, including voice, text, and visual observations. These inputs are organized into coherent interaction episodes, which provide the contextual boundary for subsequent memory processing. An extraction layer identifies salient entities, relationships, facts, and events from each episode and transforms them into normalized memory representations.
Before information becomes persistent memory, the extracted representations pass through validation and consistency processing, including structural validation, deduplication, and state resolution. The resulting memories are then temporally grounded and incorporated into the persistent memory layer.
This pipeline establishes a controlled boundary between continuously generated interaction data and long-term memory. Rather than storing every observation indiscriminately, the system converts relevant information into representations that can be indexed, traversed, temporally resolved, and supplied to downstream reasoning.
Dual Representation of Memory
A central design principle of the architecture is maintaining two complementary representations of experience: episodic memory and structured memory.
Episodic memory preserves the original interaction context associated with a conversation, event, or observation. This provides access to contextual information that may be lost when an interaction is compressed into structured facts. Structured memory, in contrast, represents salient information through entities, relationships, facts, and events within a knowledge graph, enabling graph-based retrieval and reasoning over accumulated information.
The two representations are complementary rather than redundant. Episodic memory preserves contextual fidelity, while the structured representation provides a compact and traversable representation of knowledge. During recall, these representations can be combined to provide both the relevant fact and the context required to interpret it correctly.
Bi-Temporal Knowledge Graph
The structured memory layer is organized as a bi-temporal knowledge graph. Each memory relationship is associated with two temporal dimensions that capture different aspects of its history: transaction time and valid time.
Transaction time represents when a memory was recorded or updated within the system. Valid time represents the period during which the corresponding information was true or applicable in the user’s world. Maintaining these dimensions independently allows the system to distinguish between when the system learned something and when that information was actually valid.
This distinction is important for information that changes over time. Consider a user who mentions in June that they moved to a new city in March. The transaction time reflects when AURVIZ learned the information, while the valid time reflects the period to which the information refers. If the user’s location changes again later, the earlier state can remain part of the temporal graph rather than being overwritten as though it never existed.
The graph can therefore represent memory as a sequence of temporally grounded states.
This gives the memory layer the ability to reason over both current state and historical state, while retaining the temporal provenance of how that knowledge entered the system.
Memory Retrieval
When a user asks a question requiring information from previous interactions, the system first performs query understanding to determine whether persistent memory is required and what type of information is relevant. Memory-dependent queries are then processed through a retrieval layer that combines multiple signals, including semantic similarity, lexical relevance, conversational context, and structural relationships within the knowledge graph.
The retrieval process identifies candidate memories and ranks them according to their relevance to the query. Structured graph retrieval provides relationships between related memories, while episodic retrieval provides the surrounding interaction context.
The resulting evidence is assembled into the reasoning context rather than exposing the entire historical memory to the language model. This retrieval-first design reduces unnecessary context while preserving the information required to produce a grounded response.
Temporal Retrieval
Temporal reasoning is particularly important when a query refers to a historical state, event, or period. A conventional semantic retrieval system may identify several related memories without distinguishing which information was valid at the requested point in time. The bi-temporal knowledge graph introduces an additional temporal dimension to candidate selection and state resolution.
For a temporally constrained query, the system can first identify relevant memories and then resolve their temporal validity against the requested time context.
This allows the system to distinguish between information that is currently valid, information that was previously valid, and information that was recorded at a different point in the system’s history.
The distinction becomes particularly valuable for dynamic information such as preferences, plans, decisions, locations, projects, and other user state that evolves over time. Instead of treating all historical observations as simultaneously current, the retrieval layer can reason over the temporal state associated with each memory.
Continuous Memory Evolution
Long-term memory is treated as a continuously evolving knowledge representation rather than a static database. Each new interaction can introduce new entities and relationships, reinforce existing information, provide additional contextual evidence, or introduce a new state that supersedes an earlier one.
This lifecycle allows the memory representation to evolve without discarding its historical structure. New observations can therefore be integrated into an existing knowledge representation while preserving the temporal relationships between different states.
Time and Location as Memory Dimensions
The same architecture can extend beyond semantic and temporal information to incorporate spatial context. A memory can be associated with a location at which an interaction occurred, allowing spatial context to become an additional retrieval signal.
This enables queries such as:
“What did I talk about here?”
or:
“What did I discuss here last week?”
The latter combines two independent dimensions of memory—location and time—to significantly narrow the relevant memory space.
This provides a more natural interface to long-term memory because users can refer to memories using the same contextual cues they naturally use in everyday recall: what happened, when it happened, and where it happened.
Design Principles
The resulting architecture is built around five complementary properties: persistence, structure, context, temporal awareness, and spatial awareness.
Persistence allows information to remain available across interactions and sessions. Structure transforms salient observations into entities, relationships, facts, and events that support efficient retrieval and reasoning. Context preserves episodic information alongside structured representations. Temporal awareness, implemented through the bi-temporal knowledge graph, enables the system to distinguish when information was recorded from when it was valid. Spatial awareness allows memories to be associated with relevant locations and incorporated into contextual retrieval.
Together, these components transform memory from a passive storage layer into an active information-retrieval substrate for AI reasoning. The objective is not to retain every interaction indiscriminately, but to continuously organize accumulated multimodal experience into structured, contextual, temporally grounded, and spatially aware representations that can be retrieved when relevant.
In this architecture, long-term memory becomes part of the reasoning loop.
The resulting system is designed to move beyond assistants that simply retrieve previous text toward an AI system capable of maintaining a persistent, evolving model of what happened, when it happened, where it happened, and how that information changed over time.
Benchmarking Memtwin
We evaluate Memtwin using LongMemEval-S, a benchmark designed to measure long-term conversational memory. The benchmark contains 50 prior conversations and approximately 115,000 tokens of conversational history, with queries requiring the system to recall information from one or more earlier interactions. The complete evaluation contains 500 questions, and we run the full benchmark using the official grading rubric without modification.
Retrieval Evaluation
We evaluate retrieval independently using Recall@k and NDCG@k. Recall@k measures whether the conversation containing the required evidence appears within the top k retrieved results, while NDCG@k additionally considers the position of the relevant result within the ranking. This distinction matters in production systems because retrieving the correct memory at rank 1–3 is substantially more useful than finding it only after retrieving many irrelevant candidates.
On the reported comparison, Memtwin achieves 85.92 Recall@3, 89.84 NDCG@3, 90.14 Recall@5, 90.62 NDCG@5, 92.96 Recall@10, and 91.12 NDCG@10. Memtwin leads five of the six reported metrics, while MemGASXu, D., Wen, Y., Jia, P., Zhang, Y., Zhang, W., Wang, Y., et al. From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents. 2025. arXiv:2505.19549. achieves higher Recall@10.
The comparison systems come from a single independent evaluation on LongMemEval-S, and Memtwin’s figures were recomputed under that evaluation’s exact scoring rules — a strict all- gold-in-top-k recall rather than the partial-credit version, with abstention questions excluded — so the columns measure the same thing. Under our own more generous convention Recall@5 reads 92.13; under theirs it is 90.14, and 90.14 is the figure reported here. Two caveats on the rows: neither HippoRAG 2 nor SeCom evaluated themselves on LongMemEval, so those are third-party reproductions, and independent reproductions of HippoRAG 2 disagree with one another by around 5 NDCG points; and after excluding abstention questions our question set is 284 against their 470. The scoring rules match, the subsets do not.
The strongest separation occurs at small values of k. This is operationally significant because production memory systems typically retrieve a limited number of candidate memories rather than passing an entire lifetime of interaction history to the answering model. Consequently, ranking the relevant evidence near the top directly influences context utilization, latency, and downstream answer quality.
Retrieval quality alone does not guarantee correct answers. The retrieved evidence must ultimately support the language model in generating the correct response. Memtwin therefore also evaluates end-to-end answer accuracy on the same benchmark.
Under the reported evaluation conditions, Memtwin achieves 82.0% answer accuracy, compared with 50.6% for the full-history baseline without a memory system. Two of the compared systems score below that baseline — RecurSum at 35.40% and RAPTOR at 32.20% — which is enough to show that introducing a retrieval layer does not inherently improve long-context reasoning. If retrieval discards relevant information, the memory layer can actually make the available evidence worse.
Unlike the retrieval comparison, this one is not model-independent, and the difference should be read with that in mind. Every comparison row was measured by MemGAS with GPT-4o-mini as the answering model, including the full-history row at 50.6%. Memtwin’s figure uses Gemini 2.5 Pro. Part of the distance between them is the reader rather than the memory, and these numbers alone cannot separate the two. For scale, the benchmark’s own authors measured a full-history baseline of 64.0% with GPT-4o — 13.4 points above MemGAS’s 50.6% for what is nominally the same configuration. That difference is the reading model, which is why figures are not mixed across harnesses here.
The retrieval and answer figures above are drawn from one comparison, so they are collected here on a single axis: every metric down the side, every system across the top.
| Memtwin | MemGAS | HippoRAG 2 | SeCom | A-Mem | Contriever | MPC | MPNet | RecurSum | RAPTOR | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Retrieval | Recall@3 | 85.92 | 78.51 | 75.53 | 71.06 | — | 71.06 | 60.00 | 66.17 | 67.23 | — |
| NDCG@3 | 89.84 | 86.83 | 85.44 | 80.88 | — | 79.72 | 70.90 | 75.47 | 78.33 | — | |
| Recall@5 | 90.14 | 88.94 | 84.68 | 80.43 | — | 81.28 | 68.09 | 76.38 | 79.79 | — | |
| NDCG@5 | 90.62 | 88.77 | 87.32 | 83.08 | — | 82.47 | 73.27 | 78.29 | 81.76 | — | |
| Recall@10 | 92.96 | 94.47 | 91.28 | 89.15 | — | 90.00 | 80.00 | 85.11 | 87.66 | — | |
| NDCG@10 | 91.12 | 89.96 | 88.73 | 85.11 | — | 84.29 | 76.59 | 80.63 | 83.28 | — | |
| Answer | Answer accuracy | 82.0% | 60.20% | 57.60% | 56.00% | 55.60% | 55.40% | 53.80% | 53.20% | 35.40% | 32.20% |
Retrieval and answer accuracy on LongMemEval-S, every system on one axis. A dash is a figure the source evaluation did not report: the two systems compared only on answer accuracy have no retrieval numbers to give. Shaded cells are the best value in their row.
Performance Across Memory Tasks
Aggregate accuracy can conceal systematic weaknesses, so LongMemEval separates questions into different memory categories. Memtwin achieves above 77% accuracy across every evaluated question type, including single-session recall, preferences, knowledge updates, assistant-provided information, multi-session recall, and temporal reasoning.
| Question Type | Description | Memtwin |
|---|---|---|
| Single-session user | Information stated once by the user | 92.6% |
| Preference | User preferences, choices, or things they want to avoid | 83.3% |
| Knowledge update | A fact that changed over time | 83.3% |
| Single-session assistant | Information previously provided by the assistant | 77.8% |
| Multi-session | Information distributed across multiple conversations | 77.8% |
| Temporal reasoning | Determining when events occurred and their ordering | 77.8% |
Every question type the benchmark defines, scored separately, each bar drawn against a full 0–100 scale. No single easy category is carrying the average.
Preference and temporal reasoning are particularly important because they require more than semantic matching. A preference may be expressed indirectly or conditionally, while temporal reasoning may require reconstructing the ordering of events distributed across multiple conversations. These tasks require the system to preserve not only what was said, but also the contextual and temporal relationships surrounding it.
Evaluation Conditions
To make the reported results reproducible and interpretable, we report the complete evaluation conditions. Memtwin was evaluated on the full 500-question LongMemEval-S set using the official benchmark rubric, with Gemini 2.5 Pro as the reader model. The evaluation used a single pass without Best-of-N sampling, a single judge, no question-type hints, and an exact McNemar test. The reported statistical significance was p = 1.01 × 10−12, with an observed replication floor of approximately 0.77%.
We also score the same answers against a stricter judge of our own, which fails any answer that quotes a stale value beside the current one or drops a qualifier. Under it Memtwin scores 77.3%. The official-rubric figure is reported for comparability and this one because it is the harder test.
We intentionally do not provide the retriever with question-type labels. In a real interaction, users do not tell the system whether a question is temporal, preference-based, or multi-session. Retrieval therefore operates without this additional metadata.
Conclusion
A memory system for wearable AI operates under a fundamentally different constraint from a conventional conversational assistant. Instead of beginning with a small prompt and carefully selected context, it operates over an evolving stream of interactions accumulated across weeks and months. The objective of Memtwin is to transform this persistent history into retrievable, contextual, and temporally grounded evidence.
On LongMemEval-S, Memtwin achieves 90.14 Recall@5, 91.12 NDCG@10, and 82.0% end-to-end answer accuracy under the reported evaluation conditions.
For AURVIZ O1, this forms the foundation for a different kind of wearable AI: a system that does not simply respond to what is happening in the present, but can retrieve, contextualize, and reason over the user’s accumulated interactions.