新的人工分析指数扩大了开放式模型和封闭式模型之间的差距
人工分析公司对其智能指数进行了整改，导致Kimi K3.1 Pro在排名中降下5个出点。 该型号原为发布第4名，但在第5版的8个闭塞型号后跌至第9名。 人工分析现在将开口模型与前导板上方相隔12分，从之前的3分上升。
指数的60%首次依赖于私人测试集来防止"benchmaxing"，即专门针对已知测试问题的训练模型的做法。 Kimi K3.1 号机车 Pro的下降表明旧指数使得一些模型看起来比他们看起来更好。 Grok 4.7也损失了12分。
AI研究者内森·兰伯特(Nathan Lambert)认为"板凳"只是故事的一部分。 今天，许多培训成果都发生在强化学习环境，其任务类似于基准测试。 跨越多个领域需要复杂的RL基础设施并不是每个实验室都有
关闭的测试设备确实有缺点，因为外出者无法再发现提示中的错误。 AA-Omniscience知识测试证明了这些错误的存在。 人工分析必须修改新指数的基准测试中模棱两可和有缺陷的问题。
AI News Without the Hype – 由人类治愈
订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。
现在订阅
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Artificial Analysis index widens the gap between open and closed models
Artificial Analysis overhauled its Intelligence Index, causing Kimi K3.1 Pro to drop five spots in the rankings. The model originally launched in fourth place, but it falls to ninth behind eight closed models in version 5. Artificial Analysis now puts the gap between open models and the top of the leaderboard at 12 points, up from 3 points previously.
For the first time, 60 percent of the index relies on private test sets to prevent "benchmaxxing," the practice of training models specifically on known test questions. Kimi K3.1 Pro's drop suggests the old index made some models look better than they were. Grok 4.7 also lost 12 points.
AI researcher Nathan Lambert thinks benchmaxxing is only part of the story. Many training gains today happen in reinforcement learning environments with tasks that resemble the benchmarks. Keeping up across multiple domains at once takes sophisticated RL infrastructure, and not every lab has it.
Closed test sets do have a downside, since outsiders can no longer spot errors in the prompts. The AA-Omniscience knowledge test proves those errors exist. Artificial Analysis had to revise ambiguous and flawed questions in that benchmark for the new index.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe now
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.