2026年10月27日 (英语).
Ali Behrouz，工作人员研究科学家，Vahab Mirrokni,VP和Google研究员，Google研究
我们在持续学习中引入了睡眠阶段，使大型语言模型能够不断融合新的事实并增强推理，同时减轻灾难性的遗忘或静态的知识断层。
现代LLMs通过预训和内文学习(ICL)取得了令人印象深刻的表现。 然而，一旦部署，它们基本上保持静止状态，在固定的知识截止日期运作，使它们无法获得长期能力或适应新的信息。
虽然用新数据更新模型似乎是显而易见的解决方案，但持续的参数更新往往会引发灾难性的遗忘，而学习新任务会严重地降低以往学习过的绩效。 或者，严格依赖ICL将知识保留限制在直接上下文窗口：一旦上下文会话结束，收购的短期信息就会被遗忘。
受人大脑如何在休息期间处理并稳定记忆的启发，我们引入了持续学习中的睡眠框架：一种学习范式，使大型语言模型(LLMs)具有连续的周期性生命周期，使其能够不断学习，将脆弱的短期环境整合为稳定的长期参数。 通过这个框架，我们表明在活性互动和离线内存整合之间的交替，LLMs可以不断获得新的事实，将长文本知识保留到10M令牌，并改进复杂的推理。
传统的机器学习依赖于严格区分训练与推论。 在不断学习的框架内，这种区别变成人为的；聪明的agent必须不断学习，适应和处理信息。
在睡眠模式下，LLM运行于两个初级州：
活(醒)相：模型积极接收，处理，并与输入进行交互，将临时信息存储在快速，短期内存组件中。
- 睡眠阶段：模型暂停处理新的输入，并将其内部计算转向内存合并和自我改进。
虽然诸如"睡眠","梦想","自修"等术语借用神经科学，但我们的框架完全在固定的模型架构和人定义的训练规则中运作。 在离线睡眠阶段，模型会解锁预分配的Mixture-of-Experts (MoE)内存块并使用自生成的实践实例来更新其数值权重。 这个自动化的训练后周期有助于部署的模型吸收新的事实并提高其推理精度，同时使其核心代码库和设计保持不变。
睡眠范式通过两个阶段解决灾难性的遗忘和持续的适应：记忆巩固和梦想。
在人脑中，非快速眼动(NREM)慢波睡眠通过抽象和分馏将脆弱的记忆整合为长期结构。 我们提议的睡眠框架通过两个关键技术机制反映了这一自然现象：
- 参数扩展和连续内存结构：为了防止灾难性的遗忘，模型在睡眠时动态地缩放其容量。 新解锁的低级专家被整合到稳定而低频的内存区块中
- 知识出苗(上分馏)：被储存在高频层的脆弱短期记忆(如内分生记忆)被转移到新扩展的长期参数中。 这一点是通过通用知识分解(GKD)与基于强化学习(RL)的模拟学习相结合来实现的：
- 政策蒸馏：扩大的学生模式从教师分发中学习自采样序列产生的特定符号反馈。
- 学习模仿(LTI)：学生模式在完成由教师生成的文本前缀时，根据语义学和绝对相似性(Levenshtein starge)得到奖励。
- 突触穿行：一旦内存被整合为低频模块完成后，高频临时参数被重置出，释放出未来活会话的能力。
在REM睡眠期间，人类大脑会积极模拟各种情景，巩固情感/语义联系，并探索出新颖的神经途径。 例如，在“梦想阶段”中，睡眠框架执行一个自主的自我改进循环：
- 随机专家选择：在合成数据生成过程中，MOE路由器随机地激活了非标准专家。 这迫使模型将知识的不同领域混为一谈，并发现潜在的模式。
- 被破解的排练：该模型生成合成训练样品("梦想")并使用梯度数据选择进行过滤，以优先排序具有最大潜力的样品来提升整体模型能力。
- RL精细化：该模型在顶级梦样上执行有监督的微调，并评价该代是否改善了下游任务的表现。
除了实证结果之外，我们的睡眠范式还基于一个理论，该理论澄清了为什么其两个阶段(记忆巩固和梦想)能够支持持续学习，而不只是用新知识来换取旧能力。
对于内存整合，分析显示认为知识出苗可以被理解为结构化，低等压缩问题。 该模型没有被要求将它的短期痕迹的所有细节复制到它的参数中。 相反，它确定了一套紧凑的方向，以最好地保存模型本身的政策行为和学习目标所奖励的语义素质。
这一视角解释了两个信号的互补作用：政策分解使模型与模型在推断时实际产生的轨迹一致，减少只在教师生成的分布方面进行培训时可能出现的不匹配现象；同时，语义奖励改变了值得保留的内容，确定了保留意义和任务相关行为的方向，而不仅仅是匹配象征性概率。 然后，参数扩展提供了写出这些所选记忆而无需覆写先前所集成的知识所需的能力。
对于梦想，该理论将随机专家活化和基于奖励的样本选择定性为改进学习几何学的机制。 注入不同的MOE拓宽了合成生成期间所探索的梯度方向，使模型的更新收集量比普通的时钟数据本身所引发的更丰富。 渐入式滤波再强调带最有用的学习信号的合成经验，加强信息化方向，同时保留了底部梯度空间的结构。
这些步骤共同产生更均衡的优化地貌：学习较少被少数方向所支配，梯度条件较好，模型在完善能力时可以取得更稳定的进展。 在这种观点中，梦想不仅仅是自我生成的数据增强；它是一个故意使模型自我改进的更新多样化、选择和整合的过程。
就连续翻译而言，如满洲和卡拉芒等未见语言而言，睡眠框架在多阶段评价中保持了近零的性能下降，而ICL、Cartridges和受监督的微调(SFT)等标准基线则在相继学习过程中遭受了严重的灾难性遗忘。
在对 BABILong 基准测试的长文理解中，我们的睡眠方法展示了近乎完美的放大到10M令牌，表现优于GPT-4等强基线，而泰坦则通过使用睡眠时间整合将长文信息直接压缩成参数内存。
关于在SQuAD上评价的事实知识纳入和持续的培训前(CPT)，我们的睡眠方法在单任务上达到48.9%，在CPT评价上达到46.2%，表现优于基准方法，包括基础模型、SFT和自适应语言模型(SEAL)，成功地整合了新的事实信息，而没有贬低现有的模型陈述。
最后，根据数学推理基准测试，包括AIME 24/25和HMMT 25，睡眠增强基模分别实现了79.2%,69.0%和46.1%的精度分数。 这种表现超过了SFT，组类相对政策优化(GRPO)，以及政策上自我分解(OPSD)等基线，表明将内存整合与模仿学习相配驱动了上等推理性能。
我们的睡眠模式提供了一条通往自主，不断学习AI系统的道路。 通过超越静态的预训和脆弱的上下文windows，在主动学习和离线休息的交替下，LLMs增强能力来保留短期记忆，抽象出新技能入长期参数结构，并不断自我改进而不会被灾难性地遗忘。
这项研究由阿里·贝克鲁兹(Google和康奈尔大学)，法尔诺什·哈希姆(科内尔大学)，阿德尔·贾万马尔德(Google和南加州大学)和瓦哈布·米尔罗克尼进行。 我们感谢袁登和科林纳·科尔特斯审查这项工作及其宝贵建议。 我们也感谢马克·辛伯格(Mark Simborg)
×
❮
❯
×
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
October 27, 2026
Ali Behrouz, Staff Research Scientist, and Vahab Mirrokni, VP and Google Fellow, Google Research
We introduce a sleep phase in continual learning, enabling large language models to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
Modern LLMs achieve impressive performance via pre-training and in-context learning (ICL). However, once deployed, they remain largely static, operating with a fixed knowledge cutoff date that leaves them unable to acquire long-term capabilities or adapt to new information.
While updating models with new data seems like the obvious solution, continual parameter updates often trigger catastrophic forgetting, a fundamental issue where learning new tasks severely degrades performance on previously learned ones. Alternatively, relying strictly on ICL limits knowledge retention to the immediate context window: once the context session ends, the acquired short-term information is forgotten.
Inspired by how the human brain processes and stabilizes memory during rest, we introduce a sleep framework in continual learning: a learning paradigm that equips large language models (LLMs) with a continuous periodic lifecycle, enabling them to continually learn, consolidate fragile short-term context into stable long-term parameters. With this framework, we demonstrate that alternating between active interaction and offline memory consolidation allows LLMs to continuously acquire new facts, retain long-context knowledge up to 10M tokens, and improve complex reasoning.
Traditional machine learning relies on a strict separation between training and inference. In a continual learning framework, this distinction becomes artificial; an intelligent agent must constantly learn, adapt, and process information.
Under the sleep paradigm, an LLM operates across two primary states:
- Active (wake) phase: The model actively receives, processes, and interacts with inputs, storing temporary information in fast, short-term memory components.
- Sleep phase: The model pauses processing new inputs and redirects its internal computations toward memory consolidation and self-improvement.
While terms like "sleep," "dreaming," and "self-improvement" borrow from neuroscience, our framework operates entirely within a fixed model architecture and human-defined training rules. During the offline sleep phase, the model unlocks pre-allocated Mixture-of-Experts (MoE) memory blocks and updates its numerical weights using self-generated practice examples. This automated post-training cycle helps a deployed model absorb new facts and sharpen its reasoning accuracy while keeping its core codebase and design unchanged.
The sleep paradigm addresses catastrophic forgetting and continuous adaptation through two stages: memory consolidation and dreaming.
In the human brain, non-rapid eye movement (NREM) slow-wave sleep consolidates fragile memories into long-term structures via abstraction and distillation. Our proposed sleep framework mirrors this natural phenomenon through two key technical mechanisms:
- Parameter expansion & continuum memory hierarchy: To prevent catastrophic forgetting, the model dynamically scales its capacity during sleep. Using a Mixture of Experts (MoE) and Hope architecture , newly unlocked, low-rank experts are integrated into stable, lower-frequency memory blocks to store incoming knowledge without disrupting older parameters.
- Knowledge seeding (upward distillation): Fragile short-term memories stored in high-frequency layers (e.g., in-context memory) are transferred into the newly expanded long-term parameters. This is achieved through Generalized Knowledge Distillation (GKD) combined with reinforcement learning (RL)-based imitation learning:
- On-policy distillation: The expanded student model learns token-specific feedback from teacher distributions generated on self-sampled sequences.
- Learning to imitate (LTI): The student model is rewarded based on semantic and absolute similarity (Levenshtein distance) when completing prefixes of teacher-generated text.
- Synaptic pruning: Once the memory consolidation into lower-frequency modules is complete, the high-frequency temporary parameters are reset, freeing up capacity for future active sessions.
During REM sleep, the human brain actively simulates scenarios, consolidates emotional/semantic connections, and explores novel neural pathways. Mirroring this, during the “dreaming stage”, the sleep framework implements an autonomous self-improvement loop:
- Random expert selection: During synthetic data generation, the MoE router randomly activates non-standard experts. This forces the model to mix disparate domains of knowledge and uncover latent patterns.
- Curated rehearsal: The model generates synthetic training samples ("dreams") and filters them using gradient-based data selection to prioritize samples with the highest potential to boost overall model capability.
- RL refinement: The model performs supervised fine-tuning on top-rated dream samples and evaluates whether the generation improves performance on downstream tasks.
Beyond the empirical results, our sleep paradigm is grounded in a theory that clarifies why its two stages (memory consolidation and dreaming) can support continual learning without simply trading new knowledge for old capabilities.
For memory consolidation, the analysis shows that knowledge seeding can be understood as a structured, low-rank compression problem. The model is not asked to copy every detail of its short-term traces into its parameters. Instead, it identifies a compact set of directions that best preserve both the model’s own on-policy behavior and the semantic qualities rewarded by the learning objective.
This perspective explains the complementary roles of the two signals: on-policy distillation aligns consolidation with the trajectories the model will actually produce at inference time, reducing the mismatch that can arise when training only on teacher-generated distributions; meanwhile, the semantic reward changes what is worth retaining, prioritizing directions that preserve meaning and task-relevant behavior rather than merely matching token probabilities. Parameter expansion then provides the capacity needed to write these selected memories without overwriting previously consolidated knowledge.
For dreaming, the theory characterizes random expert activation and reward-based sample selection as mechanisms for improving the geometry of learning. Injecting different MoEs broadens the set of gradient directions explored during synthetic generation, exposing the model to a richer collection of updates than those induced by its ordinary wake-time data alone. Gradient-based filtering then emphasizes the synthetic experiences that carry the most useful learning signal, strengthening informative directions while preserving the structure of the underlying gradient space.
Together, these steps yield a more balanced optimization landscape: learning is less dominated by a small number of directions, gradients are better conditioned, and the model can make more stable progress when refining its capabilities. In this view, dreaming is not simply self-generated data augmentation; it is a deliberate process for diversifying, selecting, and consolidating the updates through which the model improves itself.
For continual translation, in unseen languages, such as Manchu and Kalamang, the sleep framework maintained a near-zero performance drop in multi-stage evaluations, whereas standard baselines like ICL, Cartridges, and supervised fine-tuning (SFT) suffered severe catastrophic forgetting during sequential learning.
In long-context understanding on the BABILong benchmark, our sleep method demonstrated near-perfect scaling up to 10M tokens, outperforming strong baselines like GPT-4, and Titans by using sleep-time consolidation to compress long-context information directly into parametric memory.
For factual knowledge incorporation evaluated on SQuAD and continued pre-training (CPT), our sleep method achieved 48.9% on single-task and 46.2% on CPT evaluations, outperforming baseline methods including the base model, SFT, and Self-Adapting Language Models (SEAL) by successfully integrating new factual information without degrading existing model representations.
Finally, on mathematical reasoning benchmarks including AIME 24/25 and HMMT 25, sleep-enhanced base-model achieved accuracy scores of 79.2%, 69.0%, and 46.1%, respectively. This outperformed baselines like SFT, group relative policy optimization (GRPO), and on-policy self-distillation (OPSD), showing that pairing memory consolidation with imitation learning drives superior reasoning performance.
Our sleep paradigm provides a path toward autonomous, continually learning AI systems. By moving beyond static pre-training and fragile context windows, alternating between active learning and offline rest empowers LLMs to retain short-term memories, abstract new skills into long-term parameter structures, and continuously self-improve without catastrophic forgetting.
This research was conducted by Ali Behrouz (Google and Cornell University), Farnoosh Hashemi (Cornell University), Adel Javanmard (Google and University of Southern California), and Vahab Mirrokni. We thank Yuan Deng and Corinna Cortes for reviewing the work and their valuable suggestions. We also thank Mark Simborg, Kimberly Schwede, and Tim Herrmann for their help in crafting this blog post.
×
❮
❯
×
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.