Efficient Long Term Memory for LLM Agents

Summary & Methodology Analysis

Long-horizon LLM agents struggle to preserve historical information efficiently because eager turn-level consolidation makes memory construction expensive as conversations scale, while coarse summarization discards fine-grained details and larger retrieval contexts shift overhead to query time. LycheeMemory V2 solves this by using online semantic segmentation. This mechanism calculates semantic surprise, cohesion drop, length pressure, and turn-count pressure to batch multiple exchanges into coherent segments, followed by encoding those segments into context-independent typed memory records containing semantic statements, memory types, entities, topics, temporal scopes, and provenance links. Standard mechanics handle cosine similarity computations for incoming exchange embeddings and segment centroid updates. Encoded records go into an append-only store, and metadata updates indexes for entities, topics, entity-topic relations, temporal data, and event-frames without additional LLM calls. For retrieval, a plan-guided multi-route retrieval architecture uses an LLM planner to decompose questions into typed recall routes, followed by parallel recall channels, reciprocal-rank fusion, reranking, and diversity-aware selection.

Evaluation benchmarks include LoCoMo and LongMemEval-S alongside models such as GPT-4.1-Mini, text-embedding-3-small, bge-reranker-v2-m3, Mem0, MemoryOS, MemOS, Nemori, LightMem, TiMem, and MemU. The approach delivers substantial efficiency gains, reducing construction tokens by 86.0 percent on LoCoMo and 75.9 percent on LongMemEval-S compared with A-Mem, while reaching 89.22 percent and 92.20 percent accuracy on those respective benchmarks.

Despite these efficiency and accuracy gains, limitations remain. MemoryOS achieves higher accuracy on preference tracking on LongMemEval-S with GPT-4.1-Mini, indicating that dedicated user-profile modules can provide advantages for preference-intensive queries. Additionally, the evaluation focuses exclusively on text-only long-term conversational memory, leaving other modalities untested.