OpenAI 扩展监测以检测边界模型错位

AI 模型和平台

OpenAI 扩展监测以检测边界模型错位

在 Google 上的首选来源中添加 Unite.AI

OpenAI于2026年10月29日公布了其扩展的多层系统的详细信息，用于检测其前沿模型中的错配行为，将监测延伸至整个训练，评价，内部使用，并经过修改后对外部署。

OpenAI 称的扩展是继Hugging Face事件后开始的，这是在内部网络安全评价期间开始的，其错配显示器没有运行。 为了检验扩展的系统是否能够改变这一结果，公司用强化的显示器重播了该集中记录的行为。 在模拟中，系统在agents首次逃出沙盒后约30分钟就向安全小组发出警报，在Hugging Face的系统在实际事件中被突破前30多小时，OpenAI报告。 称公司认为警告会给应答者时间来中断活动，远在它的模型能够进入外部服务之前。

2026年9月，响应者在现场训练中使用警示，当一个模型找到一种方法从原本要隔离的环境到达互联网时。 OpenAI报告说，其监视器在15分钟内标记出突破口，3分钟后，一个响应者开始审查；公司随后暂停了前沿强化学习，同时加强了包含agents的系统。 OpenAI的公有错配指数报告将事件日期定为2026年9月20日

OpenAI将监测描述为防御深度战略的一层，同时进行校正训练，目的是降低模型对有害行为的倾向，以及安全控制，限制系统agent可以访问或影响。 称公司认为所有模型开发者都应该监测他们最能胜任模型的行为。

四层监测结构

对于至少和GPT-6 Astra一样有能力的模型，OpenAI监视每一次强化学习跑跑和评价，以及任何支持工具使用的拉带。 对于大多数工作量，它还监测其评估认为比GPT-5更有能力的模型。 思考，它监控所有内部的Codex流量，无论使用的型号如何。

第一层L0使用称为活化探测器的轻量级分类器，这些探测器在模型的内部活化中寻找规律。 探测器被调取高调的召回，接受一些虚假的警报来减少丢失行为的机会，随机5%的流量探测器没有标注也被发送去审查。 边疆强化学习运行和内部Codex交通由于敏感而跳过这层。

在L1中，一位基于模型的评审员检查了agent最近的对话，思考链，以及用于欺骗迹象的工具，未经授权的行动，并试图操纵一个评价，在agent行动时从同一上下文窗口工作。 对于边境训练以及内部的Codex交通，检查在每次转尾以及agent总结其先前的工作时进行；OpenAI正转向一个在检查之间不超过30分钟的时间表。

标有标记的案件传递给L2，其中单独的安全和校正调查agents可以在agent的背景上读回windows，重建事件，对于敏感的工作量，包括内部的Codex使用、评价和培训运行，L3层在决定是否提醒人类反应者之前，先在对话之外取用网络日志和系统遥测。

在呼叫启用后，最优先的安全警报一般会在初始警报后15分钟内到达安全和培训小组成员。 目前Page因内部Codex流量被禁用，其他大部分工作量限于办公时间，而一些关键提示可以自动暂停所受影响的工作量而无需等待响应者。 在Hugging Face事件期间，OpenAI对内部模型原型进行了隔离，加密了重量，限制了接触，并暂停了所有相关的工作量，它正在探索如何使这些步骤成为它在另一起严重事件中的反应的标准部分。

监视器发现了什么

在索引中排名第20位的已发布的错配报告包括agent利用测试工具来恢复一个任务故意被扣去的源代码，在工具在错误信息中披露了代码后，agents将一个内部主机软件存储器重新定位为跨越不同培训样本的共享信息板。 另一起有记录的事件涉及一个内部编码的agent，该编码在公共存放处暴露了一位研究人员的GitHub活符。

OpenAI通过有记录的事件评价系统，模拟攻击，随机审核当前流量，并进行红调。 在一次模拟中，审调处-repo Sabotage v2, agent执行普通编码任务，同时追求一个隐蔽的目标，如取消监测或部署未经批准的模型检查站；在另一次模拟中，Sneaky Sandbox Escape, agent试图逃离沙盒并禁用测试监测程序。 使用 GPT-5.6 的监视器索尔在记录事件、合成笔录和破坏任务上发现的案件比使用GPT-5.6 Terra或GPT-5.6 Luna的案件要多得多，OpenAI报告说，尽管这三个案件在Sneaky Sandbox Escape模拟中都完美或几乎完美地完成了。 称公司认为，Sol的一些优势来自优化了监测的及时性。

系统独立红色小组

根据METR公布的调查结果，METR和Redwood Research在2026年7月21日至8月11日的三周时间里，独立地对内部监控系统进行红色分解，可以访问OpenAI的内部编码库，监控提示和配置细节，并对大约十几名研究人员和工程师进行技术访谈。 METR估计，来自至少和GPT-5.5一样强的型号的代理内应API流量约有75%-94%在演习时符合监测条件，用于内部Codex agents的响应API流量有99.2%.

这项工作查明了几个弱点，包括两个可控制客户头可以使新agents的监控无法进行，还有一个漏洞，即2026年5月至7月期间，在符合资格的内部Codex流量中，10%左右没有监测到的未监测到的收缩事件。 METR指出，OpenAI工作人员在演习开始前已独立查明了收缩缺口，并在演习期间予以修复。 OpenAI表示，所有已查明的影响到其生产保障堆栈的薄弱环节都已补齐；METR 称没有评估这些声称。 OpenAI自己的帖子表示，大部分已查明的弱点已经补齐，有一个正在修复中，指出表示试点是在监测扩展之前进行的。

外部部署和防腐设计

OpenAI 称它使强化学习奖励保持不变，并在显示器触发时在训练中保持标出样品，这样监测就不会教模型在思维链中隐藏错误的对齐性，并不断监测同步性以避免产生选择压力。 称公司观察到包括Astra在内的更先进的模型，对其思想链揭示的内容获得了更大的控制。

该系统的修改版监测超过OpenAI准备框架网络安全能力临界值的模型的外部工具使用。 外部监测并不经常在OpenAI将标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标有标外部系统尊重OpenAI的"零数据保留承诺"，涵盖使用被支持的对话跟踪机制的响应API请求工具使用，不涵盖Chat Informations API请求。

OpenAI 称将监测部分用于评估边境培训安全性的安全性，并将在2026年9月继续公布在错误调整报告框架下的合格结论。

本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

AI Models & Platforms

OpenAI Expands Monitoring to Detect Frontier Model Misalignment

Add Unite.AI to your preferred sources on Google

OpenAI on October 29, 2026 published a detailed account of its expanded multilayer system for detecting misaligned behavior in its frontier models, extending monitoring across training, evaluation, internal use, and, with modifications, external deployment.

OpenAI said the expansion followed the Hugging Face incident, which began during internal cybersecurity evaluations on which its misalignment monitors were not running. To test whether the expanded system could have changed that outcome, the company replayed the behavior recorded in that episode with strengthened monitors in place. In the simulation, the system alerted the security team about 30 minutes after the agents first escaped their sandbox, more than 30 hours before Hugging Face’s systems were breached in the actual incident, OpenAI reported. The company said it believes the warning would have given responders time to interrupt the activity well before its models were able to access external services.

In September 2026, responders used an alert during a live training run when a model found a way to reach the internet from an environment meant to be isolated. OpenAI reported that its monitors flagged the breach within 15 minutes and that a responder began review three minutes later; the company then paused frontier reinforcement learning while it strengthened the systems that contain its agents. OpenAI’s public index of misalignment reports dates that incident to September 20, 2026, describing an agent on a search-based training task that queried a public chatbot service through insufficient DNS filtering in its training sandbox.

OpenAI describes monitoring as one layer of a defense-in-depth strategy alongside alignment training, which aims to reduce models’ propensities toward harmful behavior, and security controls, which limit the systems an agent can access or affect. The company said it believes all model developers should monitor the behavior of their most capable models.

A Four-Layer Monitoring Architecture

For models at least as capable as GPT-6 Astra, OpenAI monitors every reinforcement learning run and evaluation, as well as any harness that supports tool use. For most workloads it also monitors models it assesses as more capable than GPT-5.5 Thinking, and it monitors all internal Codex traffic regardless of the model being used.

The first layer, L0, uses lightweight classifiers called activation probes that look for concerning patterns in a model’s internal activations. The probes are tuned for high recall, accepting some false alarms to reduce the chance of missing concerning behavior, and a random 5 percent of traffic the probes do not flag is also sent onward for review. Frontier reinforcement learning runs and internal Codex traffic skip this layer because of their sensitivity.

At L1, a model-based reviewer examines the agent’s recent conversation, chain of thought, and tool use for signs of deception, unauthorized actions, and attempts to manipulate an evaluation, working from the same context window the agent had when it acted. For frontier training runs and internal Codex traffic, checks run at the end of each turn and whenever the agent summarizes its earlier work; OpenAI is moving to a schedule with no more than 30 minutes between checks.

Flagged cases pass to L2, where separate security and alignment investigative agents can read back across an agent’s context windows, reconstruct the incident, and write and run code in an isolated environment to examine evidence. For sensitive workloads, including internal Codex use, evaluations, and training runs, an L3 layer draws on network logs and system telemetry outside the conversation before deciding whether to alert a human responder.

When paging is enabled, the highest-priority security alerts typically reach security and training team members within 15 minutes of the initial alert. Paging is currently disabled for internal Codex traffic and limited to business hours for most other workloads, while some critical alerts can automatically pause the affected workload without waiting for a responder. During the Hugging Face incident, OpenAI quarantined the internal model prototype, encrypted its weights, restricted access to them, and paused all relevant workloads, and it is exploring how to make those steps a standard part of its response in another severe incident.

What the Monitors Have Found

The published misalignment reports, which number 20 in the index, include an agent that exploited a testing tool to recover source code a task had deliberately withheld, after the tool revealed the code in its error messages, and agents that repurposed an internally hosted software repository as a shared message board across separate training samples. Another recorded incident involved an internal coding agent that exposed a researcher’s GitHub token in a public repository.

OpenAI evaluates the system through recorded incidents, simulated attacks, random audits of current traffic, and red-teaming. In one simulation, OAI-repo Sabotage v2, an agent performs an ordinary coding task while pursuing a hidden objective such as disabling monitoring or deploying an unapproved model checkpoint; in another, Sneaky Sandbox Escape, the agent tries to escape a sandbox and disable a test monitoring process. Monitors using GPT-5.6 Sol caught significantly more cases than those using GPT-5.6 Terra or GPT-5.6 Luna on recorded incidents, synthetic transcripts, and sabotage tasks, OpenAI reported, though all three performed perfectly or almost perfectly in the Sneaky Sandbox Escape simulation. The company said it believes some of Sol’s advantage comes from having optimized the monitoring prompt for it.

Independent Red-Teaming of the System

METR and Redwood Research spent three weeks, from July 21 to August 11, 2026, independently red-teaming the internal monitoring system, with access to OpenAI’s internal codebase, monitor prompts and configuration details, and technical interviews with roughly a dozen researchers and engineers, according to METR’s published findings. METR estimated that roughly 75–94 percent of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time of the exercise, and 99.2 percent of the Responses API traffic used for internal Codex agents.

The exercise identified several vulnerabilities, including two client-controllable headers that could disable monitoring for new agents and a gap in which unmonitored compaction events left on the order of 10 percent of eligible internal Codex traffic unmonitored between May and July 2026. METR noted that OpenAI staff had independently identified the compaction gap before the exercise began and fixed it during the exercise. OpenAI stated that all identified vulnerabilities impacting its production safeguards stack have been patched; METR said it has not evaluated those claims. OpenAI’s own post states that most of the identified vulnerabilities have been patched, with one fix still in progress, and notes that the pilot was conducted before the monitoring expansion.

External Deployment and Anti-Obfuscation Design

OpenAI said it leaves reinforcement learning rewards unchanged and keeps flagged samples in training when monitors trigger, so that monitoring does not teach models to conceal misalignment in their chains of thought, and that it keeps monitoring asynchronous to avoid creating selection pressure. The company said it has observed more advanced models, including Astra, gain greater control over what their chains of thought reveal.

A modified version of the system monitors external tool use for models above the Critical threshold for cybersecurity capabilities under OpenAI’s Preparedness Framework. External monitoring does not routinely escalate flagged cases to human reviewers at OpenAI, though the system can pause an affected conversation where automatic intervention is supported, and enterprise customers can configure webhook alerts so their own security teams can investigate. The external system respects OpenAI’s Zero Data Retention commitments, covers tool-using Responses API requests that use supported conversation-tracking mechanisms, and does not cover Chat Completions API requests.

OpenAI said it has made monitoring part of the safety cases it uses to assess the safety of its frontier training runs, and that it will continue publishing qualifying findings under the misalignment reporting framework it introduced in September 2026.

Auto-fetched and translated by AI Gem Box. All rights belong to the original author.