Anthropic联合创始人杰克·克拉克将自我改进AI的预测推向了2027年。

Anthropic联合创始人杰克·克拉克将自我改进AI的预测推向了2027年。

一个新的基准测试称为TasteVal测量AI系统自行设计实验的多好并解释结果。 调查结果显示，改进曲线异常陡峭。

根据Anthropic联合创始人杰克·克拉克的说法，自2025年12月以来，这种能力大约每三个月翻了一番。 研究实验室"P-Zero Research"称其为"研究口味"，意为能够选择正确的实验并从中得出正确的结论。 测试的最佳模型以成本的一小部分超过了由24名人类专家组成的比较小组。

克拉克共同创办了Anthropic并撰写了"进口AI"通讯。 根据这些数字，他现在相信，在AI系统开发自己的接班人的递归自我改进(RSI)，早在2027年就有可能实现。 他之前曾预计2028年。

基准测试并不能证明AI系统已经在自我改进了，因为它只捕捉了狭义的一分科学活。 但是，它确实显示了辩论的方向，远离原始的模型性能，转向一个问题，即独立的研究工作语言模型能承担多少。

味道仅测量画面的一部分

P-Zero Research将研究口味定义为三种能力：确定哪些研究问题值得追求，设计实验来回答，以及正确解释结果。 口味Val只测试最后两个。 选择最有希望的研究方向往往被认为是科学直觉的核心，但在TasteVal中，已经为模型做出了决定。

八项任务中的每一项都从AI模型开发中明确了一个固定的研究目标，以及一个可衡量的分数。 任务包括培训数据选择、预训和微调语言模型、人类偏好培训以及防御对抗性提示。

该模型决定下一步运行的实验，并使用结果指导其下一步。 为了让编码技巧不扭曲结果，一个固定的编码agent运行在单一的Nvidia H100 GPU上的所有实验。 每次跑会经过40个GPU小时的计算或120小时的行走时后结束。

人类基线来自最近曾在OpenAI,Google DeepMind,Nvidia等地工作的24位专家。 他们执行同样的任务，每人至少有两人。 每个任务中最好的人类尝试是基准测试。

通过计算效率测定的研究品味

为了让研究口味量化，P-Zero Research将其转化为计算效率。 推理是，实验计算在AI研究中是很少的，因此选择正确的实验并正确读取其结果意味着运行较少，在同一预算上的进展更多。

实验室借鉴了它的"AI未来模型"，该模型预测，一旦编码自动化，实验计算就会成为最大的瓶颈。 因此，口味Val衡量一个模型的计算需要多少来匹配专家的分数。 一个用半个计算法到达的模型拥有这个定义的两倍的研究口味。 P-Zero Research称这个比值为计算乘数。

顶级型号为Anthropic的"Opus 5.5"，命中了2.3的计算乘数，以不到一半的计算分数达到专家级分数。 但统计范围为1.15至4.37。 在低端，模型几乎不会领先。 单次运行成本大约是专家努力平均成本的一分之一。

在整个几代模范中，计算乘数自2025年12月以来每三个月大约翻了一番。 这里的不确定性也很大，所报告的加倍时间从1.7到5.0个月不等。

在往年，模型研究口味增长得更慢。 该曲线仅于2025年转向2026年前后陡起。 这是否是一个持久的趋势不能从一个单基准测试中确定，作者自己承认他们的结果可能会过分地描述"研究品味是如何快速地改善". 任务可以快速和廉价地检查，这是AI实验室最容易优化的任务。 比较组也没有包括任何顶级研究人员。

克拉克把时间推向前进

克拉克长期以来称创造力是AI系统如何快速改善自身的关键因素。 今年早些时候，他写道，模型并没有以任何开创性的规模表现出这种能力。 他现在指出数学突破的进展，基准测试像TasteVal一样是这个变化的迹象。

在5月的一篇长篇散文中，克拉克提供了更谨慎的数字。 他在2028年末将独立训练更强接班人的系统的概率提高到了60% 到2027年达到30% 这个预测来自一个共同创立公司的人，他的模型超过是基准测试，新增对预测有一层偏差。

批评者正是以这种能力划出界线的

味道很重要，因为它衡量怀疑者所标出的确切能力，成为AI重大突破的障碍。 AI研究员赫比·布拉德利(Herbie Bradley)推倒了克拉克"梅"的作文，认为虽然模型可以令人信服地接管初级研究人员的工作，但像研究口味，创造力，以及连贯的长期研究规划等要求更高的技能却无处可见。

在规划和评价实验时，TasteVal反对这一评估，尽管称没有选择长期研究议程。

利用由富丹大学一个团队领导的AI agents对AI开发的研究中，实际差距有多大，显示仍然上升。 研究人员分析了自己模型的开发情况，发现AI agents贡献了高达55%的方法建议，但人类仍然做出了超过85%的最后决定。 作者指出，AI如何在结果出现之前产生不同的研究方向并评估其价值，目前还不清楚，这正是TasteVal取消模型盘的决定。

辩论正在改变

基准测试在一场辩论中到达主要的AI实验室踢出自己。 6月，Anthropic写道，递归自我改善还没有实现，但"可能比大多数机构准备的要快". AnthropicCEODario Amodei现在呼吁在AI开发中实现有控制的减速，而OpenAI则承认完全自主的RSI还没有实现，而称只有在人的监督得到保证的情况下才能追求。

品尝改变什么是进步。 长期以来，重点是模型如何很好地解决数学问题或编码挑战等定义明确的任务，但现在的问题是，它们能否做出推动研究前进的决定。

如果P-Zero研究在其他测试中测出的趋势维持下来，那么其影响就会同时减少。 AI的开发会更快，更便宜，但与此同时，越来越多的研究份额会更难被人类验证，因为模型会规划和评价自己的实验。

AI News Without the Hype – 由人类治愈

订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。

现在订阅

读取全相。

订阅无韵报道。

- 全面查阅THE DECODER的每篇文章

- 没有广告

- 加入评论和社区讨论

- 每周一次通过邮件转载AI新闻

- 6x/年："AI雷达"——对AI最重要的话题进行深潜。

- 每日AI新闻，总是最新的

- 我们完整的十年档案

- 由一个在AI工作10年以上的团队负责

订阅到 The Decoder

本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

Anthropic co-founder Jack Clark moves his prediction for self-improving AI forward to 2027

A new benchmark called TasteVal measures how well AI systems design experiments on their own and interpret the results. The findings show an unusually steep improvement curve.

That capability has roughly doubled every three months since December 2025, according to Anthropic co-founder Jack Clark. The research lab P-Zero Research calls it "research taste," meaning the ability to pick the right experiments and draw the right conclusions from them. The best model tested outperformed a comparison group of 24 human experts at a fraction of the cost.

Clark co-founded Anthropic and writes the Import AI newsletter. Based on the numbers, he now believes recursive self-improvement (RSI), where AI systems develop their own successors, could become possible as early as 2027. He had previously expected 2028.

The benchmark doesn't prove that AI systems are already improving themselves, since it only captures a narrow slice of scientific work. But it does show where the debate is heading, away from raw model performance and toward the question of how much independent research work language models can actually take on.

TasteVal only measures part of the picture

P-Zero Research defines research taste as three abilities: identifying which research questions are worth pursuing, designing experiments to answer them, and correctly interpreting the results. TasteVal only tests the last two. Choosing the most promising research direction is often considered the core of scientific intuition, but in TasteVal that decision is already made for the model.

Each of the eight tasks specifies a fixed research goal from AI model development, along with a measurable score. The tasks cover areas like training data selection, pretraining and fine-tuning language models, training on human preferences, and defending against adversarial prompts.

The model decides which experiment to run next and uses the results to guide its following step. To keep coding skill from skewing the outcome, a fixed coding agent runs all experiments on a single Nvidia H100 GPU. Each run ends after 40 GPU-hours of compute or 120 hours of wall-clock time.

The human baseline came from 24 experts who had recently worked at places like OpenAI, Google DeepMind, Nvidia, and Microsoft Research. They tackled the same tasks, with at least two people on each one. The best human attempt per task served as the benchmark.

Research taste measured through compute efficiency

To make research taste quantifiable, P-Zero Research translates it into compute efficiency. The reasoning is that experiment compute is scarce in AI research, so picking the right experiments and reading their results correctly means fewer runs and more progress on the same budget.

The lab draws on its "AI Futures Model," which predicts that experiment compute becomes the biggest bottleneck once coding is automated. So TasteVal measures how much compute a model needs to match the experts' scores. A model that gets there with half the compute has twice the research taste by this definition. P-Zero Research calls this ratio the compute multiplier.

The top model, Anthropic's Opus 5.5, hit a compute multiplier of 2.3, reaching expert-level scores with less than half the compute. The statistical range runs from 1.15 to 4.37, though. At the low end, the model would barely lead. A single run cost roughly one-thirtieth of what an expert attempt cost on average.

Across several model generations, the compute multiplier has roughly doubled every three months since December 2025. The uncertainty is large here too, with the reported doubling time ranging from 1.7 to 5.0 months.

In prior years, model research taste grew much more slowly. The curve only steepened around the turn of 2025 into 2026. Whether that's a lasting trend can't be determined from a single benchmark, and the authors themselves admit their results may overstate "how quickly research taste is improving." The tasks can be checked quickly and cheaply, which is the kind of task AI labs find easiest to optimize for. The comparison group also didn't include any top-tier researchers.

Clark pulls his timeline forward

Clark has long called creativity the key factor in how fast AI systems can improve themselves. Earlier this year, he wrote that models hadn't shown this ability at any groundbreaking scale. He now points to progress in mathematical breakthroughs and benchmarks like TasteVal as signs that this is changing.

In a long essay in May, Clark had offered more cautious numbers. He put the odds of a system that independently trains a stronger successor at roughly 60 percent by the end of 2028 and 30 percent by 2027. That forecast comes from someone who co-founded the company whose model tops the benchmark, which adds a layer of bias to the prediction.

Critics had drawn the line at exactly this capability

TasteVal matters because it measures the exact capability that skeptics had flagged as a barrier to a major AI breakthrough. AI researcher Herbie Bradley pushed back on Clark's May essay, arguing that while models could plausibly take over the work of junior researchers, more demanding skills like research taste, creativity, and coherent long-term research planning were nowhere in sight.

When it comes to planning and evaluating experiments, TasteVal pushes against that assessment, though it says nothing about choosing a long-term research agenda.

How wide the practical gap remains shows up in a study on AI development using AI agents led by a team at Fudan University. The researchers analyzed the development of their own model and found that AI agents contributed up to 55 percent of methodology proposals, but humans still made over 85 percent of the final decisions. The authors noted it remains unclear how AI can generate diverse research directions and assess their value before results are in, and that's exactly the kind of decision TasteVal takes off the models' plate.

The debate is shifting

The benchmark arrives in the middle of a debate the major AI labs kicked off themselves. In June, Anthropic wrote that recursive self-improvement hasn't been achieved yet but "could come sooner than most institutions are prepared for" Anthropic CEO Dario Amodei now calls for a controlled slowdown in AI development, while OpenAI acknowledges that fully autonomous RSI isn't happening yet and says it should only be pursued if human oversight is guaranteed.

TasteVal shifts what counts as progress. For a long time, the focus was on how well models solve well-defined tasks like math problems or coding challenges, but now the question is increasingly whether they can make the decisions that push research forward.

If the trend P-Zero Research measured holds up in other tests, the implications cut both ways. AI development would get faster and cheaper, but at the same time a growing share of the research would become harder for humans to verify, because models would be planning and evaluating their own experiments.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now

Read on for the full picture.

Subscribe for hype-free coverage.

- Full access to every article on THE DECODER

- No ads

- Join the comments and community discussions

- A weekly AI news recap via mail

- 6x/year: "AI Radar" — deep dives on the AI topics that matter most

- Daily AI news, always up to date

- Our full ten-year archive

- Covered by a team with 10+ years in AI

Subscribe to The Decoder

Auto-fetched and translated by AI Gem Box. All rights belong to the original author.