Making AI Models Learn Their Own Context

Summary & Methodology Analysis

The paper introduces Latent On-Policy Self-Distillation (LOPD), which improves upon standard on-policy self-distillation (distillation is the process of training a smaller model to mimic a larger teacher model). Instead of using fixed, developer-defined heuristics to guide training, LOPD uses an experience bank containing successful past trajectories. A new component called the composer maps these retrieved experiences into continuous latent tokens (numerical representations used by the model for processing). These tokens condition the teacher, which re-evaluates the student's progress, creating a dynamic feedback loop that is learned rather than manually configured. The system utilizes a privileged-margin objective to ensure the teacher maintains an informative advantage over the student during training. Finally, reverse-KL distillation is used to align the student with this supervised teacher output, with only the student policy surviving for final inference. This end-to-end approach allows the model to scale its self-improvement capabilities without requiring constant human design updates. The method has been validated across models such as Qwen3-8B-LOPD and Olmo3-7B-LOPD, showing strong performance on tasks like code generation and agentic tool use. Current limitations include a tendency for distillation gradients to cause the composer to collapse if unconstrained, which results in the loss of useful latent context. Furthermore, while the learned representations are functional, their internal logic is not fully interpretable through direct decodability alone, leaving the exact mechanics of their success partially opaque.