虽然许多注意力都集中在周一修订后S-1A中披露的更新后的财务Anthropic(收入飞涨，但费用也随之增加),AI实验室在精版中又投下一发小炸弹。
备案似乎警告说，该公司正在建立一个能够自我递归的模型。
RSI是AI在自身训练上发展得如此好，从而催生出AI能力急剧加速的地步。 对AI末日者来说，RSI代表着没有回归的灾难，在灾难中，AI构建出一个足够强大的东西，以至于它滑出超越了人类的控制，在实现目标时可能对其所造成的损害漠不关心。
对AI的爱好者来说，RSI或某种版本的AI可靠地利用自己来改进，代表了技术的希望，它成为了难以想象的生产力、科学发现和更多的工具。
Anthropic似乎认为RSI既是承诺也是危险。 它在S-1中称在研发中使用AI，但它告诫说："利用AI帮助建立更能胜任的AI，也会带来相当大的安全风险，特别是如果它能大大地加快进步的步伐。 这是《负责任扩大政策》所涵盖的领域之一，该政策提出了我们在接近所谓的“自我改进”时监测和减轻风险的计划。 ` ` '
当RSI一词在今年早些时候流行起来时，许多人认为这是一个叫声词，或者说，充其量是一个过于难以实现、定义或证明的目标。
但是在星期一，Anthropic在它的"风险因素"一节中添加了新的声明，表示1)它已经利用AI开发出新的模型，它在能力上的飞跃在当时没有意识到2)它可能很快会满足它自己的RSI的内部定义(粗体强调我们的，下面是：
此外，我们越来越多地使用人工智能系统来加快我们自己的研发，可能会使我们的模型比我们开发、验证和执行相应的安全、配合和安全措施的能力更快地取得进展。 因此，我们过去没有发现，将来可能也没有发现，我们的发展速度将大大加快，直到发展发生之后。
例如，我们的内部估计和我们对未来模式的期望，而不是我们目前的模式，表明我们可能即将实现我们负责任的扩大政策的定义，即 " 由大赦国际实际加速的进展 "。 我们没有将目前的任何模型指定为超过这一阈值，我们继续评估基本数据。
然而，Anthropic在备案中也称其模型中还没有一个超越了这个RSI阈值，并且它有望在可预见的将来将人类留在研发过程中。 “如果在技术上可行，我们预计无法安全地使用Claude来完全自主地设计和建造世代相传的前沿模型，而不需要有意义的人类监督或理解。 因此，我们正在考虑更新我们的负责任的扩大政策，使人类控制Anthropic的前沿模式发展。”
怀疑者可能指出，S-1既具有披露功能，也具有市场营销功能，而声称在接近一个能够自建的型号时，其威力如此之大，可能属于后一类。
然而，毫无疑问，Claude的能力，特别是在编码、数学和网络安全方面的能力，今年已经飞跃并达到了极限。 如果Anthropic认为RSI的可能性是相近的，也许我们应该相信它并相应采取行动——也许治理——.
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
While much attention has focused on the updated financials Anthropic revealed in its revised S-1A on Monday, (revenue is soaring, but so are expenses), the AI lab dropped another little bombshell in the fine print.
The filing seems to warn that the company is on the cusp of creating a model capable of recursive self-improvement (RSI).
RSI is the point at which AI grows so good at training itself that it spawns a dramatic acceleration of AI capabilities. To the AI doomers, RSI represents the cataclysm from which there is no return, in which AI builds itself into something powerful enough that it slips beyond human control and may be indifferent to the damage it causes while achieving its goals.
To AI enthusiasts, RSI, or some version of AI reliably using itself to improve, represents the technology’s promise, where it becomes a tool of unimaginable productivity, scientific discovery, and more.
Anthropic seems to believe that RSI is both a promise and a peril. It says in the S-1 that it uses AI in its R&D, but it cautions: “Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress. This is one of the areas covered by our Responsible Scaling Policy, which sets out our plans to monitor and mitigate the risks as we approach what has been termed ‘recursive self-improvement.’”
When the term RSI became popular earlier this year, many believed it was a buzzword or, at best, a goal too hard to achieve, define, or prove.
But on Monday, Anthropic added new statements to its Risk Factors section, saying 1) it has already used AI to develop new models whose leaps in capability it didn’t recognize at the time 2) it may soon meet its own internal definition of RSI (bold emphasis ours, below):
In addition, our increasing use of AI systems to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures. As a result, we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.
For example, our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data.
However, Anthropic also said in the filing that none of its models have crossed this RSI threshold yet and that it is promising to keep humans in the R&D process for the foreseeable future. “We do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic.”
Skeptics might point out that an S-1 functions as both disclosure and marketing, and that claims of being close to a model so powerful that it can build itself could fall into the latter category.
Nevertheless, there’s no question that Claude’s capabilities, especially in terms of coding, math and cybersecurity have grow in leaps and bounds this year. If Anthropic thinks the possibility of RSI is close, maybe we should believe it and act — perhaps govern — accordingly.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.