斯坦福研究员Chuyue Sun和她的计算机科学家同行开发了一个AI工具，在不到半天的时间里完成了一个工程项目，耗资不到500美元，相比之下，在五个月的时间里，三名高技能的专家手动完成了同样的工作。 太阳在最近的一篇文章中所报告的时间和成本的急剧减少，是唤醒一个被称为正式核查的模糊的编码和数学领域的早期成果之一。
几十年来，开发者一直设想用数学来证明他们的软件是无臭虫的。 这个过程是漫长而费力的，要求工程师写出关于其软件目标的精确声明，并将其代码转换为可以验证的公式。 虽然公司已经进行了复杂的测试，以尽可能多地识别其软件中的缺陷，但是它们很少为正式核查而外出，这可能会捕捉出更不稳定的行为。
现在，一波起步的浪潮正在打赌，基因AI将使正式的软件验证既实用又重要。 其中以"Truth"为主，"Sun"于8月加入并周四公开发布。 Sun的研究通过使用AI验证部分加密消息应用信号中对话的代码，证明了技术的潜力。
称 Ilya Sergey是Truth的首席科学官，“为什么我们对代码不照做呢？从数学角度讲，技术是完全存在的，只是难以衡量。”
&Truth从投资者那里获得了7000万美元的融资并旨在最终验证任何编程语言。 由Robinhood CEO弗拉德·特涅夫共同创立并获得2.95亿美元资金的Rival Harmonic，本周的宣布，其以数学为重点的AI agent Aristodle支持验证用C写的软件和用Verilog编写的硬件设计。 它正在与潜在的用户，包括芯片制造商Tensordyne和Nvidia评估技术，后者的风险资本臂是创业者的投资人之一。
就在上周，Anthropic 和验证启动定理联合宣布他们利用AI 来帮助在开源软件中找到万亿测试未能发现的bug. 他们还为Claude加装了推出来帮助其他开发者进行验证。 初始项目的费用没有披露。
大希望是软件运行更加可靠，安全头痛减少的未来。 但是，AI能否在NASA和Apple等组织最关键的系统之外，将正式核查工作带入广泛使用，仍然是一个尚未解决的问题。
该技术还处于原型化阶段。 虽然需求很高，但尚不清楚客户愿意支付多少，他们如何选择采用技术。 正式的核查依赖于对计算机和人员如何与软件互动的假设，一些专家对其能否捕捉出由用户出错和硬件故障而来的错误表示怀疑。
威尔·威尔逊(Will Wilson)在操作"猎虫AI"软件启动"Antithesis"时，称正式的验证是像自己这样的数学书呆子的"陷阱". Antithesis花费了数百万美元来正式验证它的一些网络安全防御，威尔逊描述的这个过程是“屁股里的一丁点". 然而起步器仍然使用自己的工具发现了更多的出错，而Antithesis也在软件中发现了一些被其客户正式核实的出错。 威尔逊·称。
以创造Ethereum密码货币而闻名的工程师Vitali克·布特林(Vitali克·布特林)今年早些时候在博客上写道，但是他和其他支持者仍然认为，这将成为防止AI系统发布网络攻击和造成破坏的重要防御手段。
核心安保人员
波音飞机，Amazon服务器，以及超过25亿个活跃的Apple设备至少包含一些经过正式验证的软件，根据公司披露。 有关守则通常与安全有关，其中的脆弱性如果被利用，可能造成严重损害。
Amazon Web Services的杰出科学家和副总裁拜伦·库克(Byron Cook)，称的正式核查缓解了客户对云安全的关注，并提前打碎了大虫子来帮助内部团队更快地前进。
但是，一些中型技术公司告诉WIRED公司，它们通常只核实其产品的一个关键特征，如果有的话。 他们提到缺乏核查工程师，费用超过潜在风险，也许最重要的是担心等待正式核查编码数周可能会使创新缓慢。
今年早些时候，Apple的安全小组称认为，在其用于加密的“corecrypto”图书馆中发现的一些问题是无法找到的。 但该公司补充说，在给Crecripto添加新的代码时，它必须"保守"，以确保它能够跟上审查的速度。
两位网络安全主管表示，但那些没有被授权公开演讲的主管们说，他们可以想象为AI生成的代码“核实”支付主要AI实验室的费用。
开源软件项目的开发者是许多应用软件的基石，他们越来越多地尝试AI辅助验证。 非营利慈善AI基金会利用Anthropic的一笔赠款，验证了今年与Sun在斯坦福所做的一样的加密代码。 被称为"曲线25519-Dalek"的图书馆维护者迈克尔·罗森伯格(Michael Rosenberg)希望持续核查工作能够出现可持续的资金。 也不应该仰赖AI公司的善意
一些渴望扩大核查的初创企业也不确定是直接向客户还是通过Anthropic等中介机构部署其工具。 他们的一般计划是使用AI来自动化很多涉及定义一个程序应该如何操作的工作。 例如，数据表的规格可能要求它按不同的顺序在排序前后包含相同的项目。 进一步的自动化努力，可能不涉及AI，会将这个代码转化为一种格式，使利恩，罗克和伊莎贝尔等数学工具可以用来检查代码是否如预期的那样有效。
最近进入正式核查领域的另一个人是Madan Musuvati，他在20多年后离开了Microsoft Research找到起动的特鲁伊迪克。 在OpenAI黑客Hugging Face等事件后，他感到迫切需要将他的学习商业化。 “攻击者手中的AI可能具有极大的威力。” Musuvati 称，但核查可以“使AI安全无虞
称对于这个过程越来越兴奋，公司首席执行官June Andronick和首席科学家Gerwin Klein说，兴趣比他们和他们的竞争所能提供的速度要快。
但双方都担心AI生成的规格会激发对结果的"无正当理由的信任". 验证者也渴望保留他们的作品带来。 Klein称它“令人上瘾，就像一个拼图游戏。” 若一切顺利
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Stanford researcher Chuyue Sun and her fellow computer scientists developed an AI tool that completed an engineering project in less than half a day for under $500, compared to the five months it took three highly skilled experts to do the same work manually. The dramatic reduction in time and cost, which Sun reported in a recent paper, is among the early results that are awakening an obscure field of coding and mathematics known as formal verification.
For decades, developers have envisioned using math to prove that their software is bug-free. This process is long and laborious, requiring engineers to write precise statements about the goals of their software and convert their code into formulas that can be validated. While companies already run sophisticated tests to identify as many flaws in their software as possible, they rarely shell out for formal verification, which can catch even more erratic behavior.
Now, a wave of startups is betting that generative AI will make formal software verification both practical and essential. Among them is &Truth, which Sun joined in August and publicly launched on Thursday. Sun’s research demonstrated the technology’s potential by using AI to verify part of the code that encrypts conversations in the messaging app Signal.
“Bridges, buildings, they don’t collapse because people did math before constructing them,” says Ilya Sergey, &Truth’s chief science officer. “Why don’t we do the same for code? The technology to mathematically reason was all there, just difficult to scale.”
&Truth has raised $70 million from investors and aims to eventually verify any programming language. Rival Harmonic, which was cofounded by Robinhood CEO Vlad Tenev and has received $295 million in funding, announced this week that its math-focused AI agent Aristotle supports verifying software written in C and hardware designs in Verilog. It is evaluating the technology with potential users, including chipmakers Tensordyne and Nvidia, whose venture capital arm is one of the startup’s investors.
And just last week, Anthropic and the verification startup Theorem jointly announced they had used AI to assist in finding bugs in open source software that trillions of tests had failed to detect. They also introduced an add-on for Claude to help other developers with verification. Costs of the initial project were not disclosed.
The big promise is a future where software runs more reliably with fewer security headaches. But whether AI can bring formal verification beyond the most critical systems at organizations like NASA and Apple and into widespread use remains an open question.
The technology is still in the prototyping stage. Though demand is high, it's unclear how much customers are willing to pay and how they prefer to adopt the technology. Formal verification relies on making assumptions about how computers and people interact with software, and some experts are skeptical about its ability to catch bugs stemming from user error and hardware breakdowns.
Will Wilson, who runs the bug-hunting AI software startup Antithesis, calls formal verification “a trap” for math nerds like himself. Antithesis spent millions of dollars to formally verify some of its cybersecurity defenses, a process Wilson describes as a “pain in the ass.” Yet the startup still uncovered additional errors using its own tool, and Antithesis has also found bugs in software that several of its customers had formally verified. “It can give you some assurance, but only robust exhaustive testing can prove your assurance is true,” Wilson says.
Or as Vitalik Buterin, the engineer known for creating the Ethereum cryptocurrency, put it in a blog post earlier this year, “formal verification is not a panacea.” But he and other proponents of the approach still believe it will be an important defense against AI systems launching cyberattacks and wreaking havoc.
Core Security
Boeing planes, Amazon servers, and more than 2.5 billion active Apple devices contain at least some formally verified software, according to company disclosures. The code in question is typically security-related, and vulnerabilities in it could cause serious harm if exploited.
Byron Cook, a distinguished scientist and vice president at Amazon Web Services, says formal verification has eased customer concerns about cloud security and helped internal teams move faster by squashing big bugs early on.
But several midsized tech companies tell WIRED they have typically verified only one key feature of their products, if any at all. They cite a lack of verification engineers, costs that outweigh the potential risks, and, perhaps most importantly, concerns that waiting weeks for code to be formally verified could slow innovation.
Earlier this year, Apple’s security team said that some of the issues formal verification uncovered in its “corecrypto” library, which is used for encryption, could not have been found any other way. But the company added that it has to be “conservative” when adding new code to corecrypto to ensure it can keep pace with reviewing it.
Two cybersecurity executives at major companies that haven’t formally verified any code say they aren’t sure whether automation will make it worthwhile. But the executives, who weren’t authorized to speak publicly, say they could imagine paying major AI labs for “verified” AI-generated code.
Developers of open source software projects, which are the bedrock of many apps, are increasingly experimenting with AI-aided verification. The nonprofit Beneficial AI Foundation used a grant from Anthropic to validate the same encryption code this year as Sun did at Stanford. Michael Rosenberg, a maintainer of the library known as curve25519-dalek, hopes sustainable funding emerges for ongoing verification efforts. “I have no ideas as to how that’s going to happen, though, and we certainly shouldn’t rely on the goodwill of the AI companies,” he says.
Some of the startups eager to expand verification also aren’t sure whether to deploy their tools directly to customers or through intermediaries like Anthropic. Their general plan is to use AI to automate much of the work involved in defining how a program is supposed to behave. For example, specifications for a data table might require that it contain the same items before and after sorting, just in a different order. Further automation efforts, which may not involve AI, would translate that code into a format that math tools like Lean, Rocq, and Isabelle can use to check whether the code works as intended.
Another recent entrant to the formal verification field is Madan Musuvathi, who left Microsoft Research after more than 20 years to found the startup Truidic. After incidents like the OpenAI hack of Hugging Face, he felt there was an urgent need to commercialize his studies. “AI in the hands of attackers can be extremely powerful,” Musuvathi says, but verification can “make AI safe and secure for everyone, everywhere.”
Proofcraft, one of the few companies that has long offered traditional verification, says growing excitement about the process is translating into a flood of inquiries from potential customers. June Andronick, the company’s CEO, and Gerwin Klein, its chief scientist, say interest easily outpaces what they and their competition can provide.
But both worry that AI-generated specifications could inspire “unjustified trust” in the results. Verifiers are also eager to preserve the joy their work brings them. Klein calls it “pretty addictive, like a puzzle game.” If all goes well, he’ll be doing more solving than ever.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.