Back to Feed
Efficiency & Inference / Training & Fine-Tuning

Optimizing LLM Knowledge Updates and Memory

Original: Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination
By Tianyun Ji, Zhenya Huang, Jiayu Liu, Zirui Liu, Yu Su, Hongbin Pei Published 2026-08-02

Listen to the summary

Uses a voice available on your device

Audio options
On this page

Key Takeaways

  • COVE outperformed standard baselines on hybrid tasks by achieving a 24.1% success rate compared to 21.3% for parametric-only methods.
  • The system reduces training costs by requiring 86% fewer tokens than standard weight-based fine-tuning methods.
  • A volatility-aware learning strategy prevents performance degradation in API-renaming scenarios, maintaining 92.5% correctness where standard models dropped to 54.00%.
  • The architecture uses a router to decide whether to store new knowledge in an editable external memory or to bake it into the model weights.

Summary & Methodology Analysis

The paper introduces a unified coordination framework that splits LLM self-evolution into two channels: harness-based memory and parametric-based training. A task-aware router directs incoming information into harness-only, parametric-candidate, or hybrid channels based on feedback signals. This router triggers harness-side exploration for volatile data and parametric updates for stable, high-value knowledge. By managing these channels dynamically, the system avoids the overhead of constantly updating model weights while keeping the model updated with necessary information.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the primary problem this paper solves?

It addresses the trade-off between using external, editable memories and updating the model weights directly to incorporate new information during self-evolution.

Q2. Does this approach save computational resources?

Yes, COVE uses 86% fewer training tokens on average compared to traditional parameter-only learning.

Q3. How does COVE compare to standard fine-tuning for API changes?

In API-renaming tasks, COVE maintained 92.5% correctness compared to standard fine-tuned models which dropped from 96.50% to 54.00%.

Q4. How does the router determine where to store knowledge?

A feedback-conditioned router assigns tasks to specific channels based on task characteristics, feedback, and detected failure signals.

Q5. What is KnowledgePO?

KnowledgePO is a dual-modal knowledge optimization technique where harness-side memories assist in rollout generation while parametric learning selectively internalizes stable knowledge.

Q6. What is the role of the anti-recitation reward?

This reward is applied during fine-tuning to penalize models that attempt to use stale interface names or volatile knowledge, effectively preventing the internalization of unreliable data.

Q7. How are redundant entries removed from the system?

The system uses parametric-assisted harness memory release, where A/B evaluation identifies stable knowledge already absorbed into the model weights and deletes those redundant entries from the external retrieval set.

Q8. Are there any limitations to harness-based methods mentioned?

Yes, they can produce misleading results if the base model lacks the capability to interpret the feedback, and memory accumulation sometimes fails to improve specific tasks like formal proof construction.

Q9. What specific models were utilized in the evaluation?

The paper utilized several models including Qwen3-8B, GPT-4o-mini, and DeepSeek-v3.2.