机器智能研究所（MIRI）负责人内特·苏亚雷斯近日在《大西洋月刊》CEO尼古拉斯·汤普森主持的访谈中表示，他认为先进人工智能在不到十年内导致人类灭绝的概率超过50%。他也透露，自己目前不再为退休储蓄：虽然还保留着在Google工作时开设的401(k)退休账户，但已经十年没有继续缴款。
苏亚雷斯长期研究人工智能对齐问题，也就是让强大AI系统的行为符合人类意图。他曾在Google和微软担任工程师，并与他人合著《如果有人造出它，所有人都会死》。他对风险的担忧并非认为AI会憎恨人类，而是认为超级智能系统可能根本不在意人类是否生存；如果人类妨碍其目标，或试图关闭它、开发竞争系统，人类就可能被视为障碍。
在他设想的风险路径中，关键并不只是AI能否发动攻击，而是它能否变得自给自足：如果系统能够自主获取能源、芯片和其他资源，并建立不再依赖人类的自动化基础设施，人类对它的制约能力就可能下降。苏亚雷斯形容，人们未必能预先知道超级智能会以哪种具体方式造成灾难，但仍可能判断自己正面对极强的对手。
他主张政府停止训练通用人工智能系统，把研发重点转向癌症研究、更安全的自动驾驶等具体用途，并表示灾难并非不可避免，自己仍在尽力阻止这种结果。他强调的解决办法，是不要造出可能杀死所有人的系统。
苏亚雷斯的判断是个人风险评估，并非已证实的科学预测，也不代表人工智能研究者之间存在统一共识。他停止增加退休储蓄的说法，反映的是他对未来风险的看法，不应被理解为对他人理财决策的建议。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Nate Soares, head of the Institute of Machine Intelligence (MII), recently stated in an interview with CEO Nicholas Thompson in Atlantic Monthly that he believed that advanced artificial intelligence was more than 50 per cent likely to lead to human extinction in less than a decade. He also revealed that he no longer saves for retirement: Although the 401 (k) retirement account, which was opened while working at Google, was maintained, no further contributions had been made for 10 years.
Soares has long studied the problem of artificial intellectual alignment, i.e. to make the behaviour of a strong AI system fit human intentions. He worked as an engineer in Google and Microsoft and co-authored " If someone builds it, everyone dies ". His fear of risk was not that AI would hate human beings, but that super-smart systems might not care whether or not human beings lived; human beings could be seen as obstacles if they obstructed their goals or tried to shut it down and develop a competitive system.
The key to the risk path he envisages is not just whether AI can launch an attack, but whether it can become self-sufficient: If the system is able to access energy, chips and other resources on its own and to build an automated infrastructure that is no longer dependent on human beings, the human capacity to constrain it may decline. Soares describes how it is not always possible to predict in advance the specific ways in which super-smarts can cause disaster, but it is possible to judge itself against a strong opponent.
He advocated for the Government to stop training in generic artificial intelligence systems and shift the focus of research and development to specific uses such as cancer research and safer autopilot, and stated that disasters were not inevitable and that it was still doing its best to stop such results. He stressed that the solution was not to create a system that could kill everyone.
Soares ' judgement is an individual risk assessment, not a proven scientific projection, nor does it represent a common understanding among artificial intelligence researchers. His argument that he had stopped increasing retirement savings reflected his vision of future risks and should not be understood as a recommendation for financial decision-making.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.