我参考了人工智能的晶体球，问问我的总理联盟幻想队
几个星期前，我部署了一个小群的AI agents 来编制和管理一个实验性名册。 鉴于AI agents扩散到我们生活的几乎每个方面，我想知道一个集群的bots是否能组起一支获胜的队伍。
我首先研究了几年前在这里使用AI的潜力，它的建议非常荒谬——它没有包括现实世界的信息，比如玩家受伤或团队管理层的改变。
这次，在去探亲的航班上，我向Codex求了帮助。 首先，我为模型提供了一些相关的研究论文。 然后，我要求它建立一个通过各种步骤指导不同agents的指南和代码，它将检查最近的新闻并运行一个模型来作出预测。
它很快产生了一个设置，似乎可以整理和分析更多的信息比我从来没有有过。 但我对最初的团队建议多么愚蠢感到惊讶。 把厄林·哈兰留在长椅上？真的吗，Codex?
然后，我要求Codex建立一个团队，由agents做不同的事情：几个人阅读新闻和博客文章，其他人分析最近的结果，这似乎大大改进了建议。 我试着把agents增加到超过100个
然而，几年前的改善是惊人的。 Haifeng Xu是芝加哥大学的计算机科学家，研究AI在市场状况中的应用
」徐告诉我
去年末期，徐对卡尔希和Polymarket等预测市场日益受欢迎感到惊奇。 他想知道这些市场的最佳商业模式可能如何发展，因此他与几位同事一起开始测试他们的预测技能。
他们有模型猜测，例如谁将赢得NBA锦标赛冠军，谁与特朗普总统相会。
徐告诉我，这些模型有时能够比一般人做得更好，没有特别的指示或指导，他就是称。 在测试中，GPT-5最能做预测，其次是Grok 4,Claude Sonnet,Gemini和Llama.
团队接着开发了"先知之地"(Actor Arena)，即测量模型预测能力的基准测试。 也是哈佛大学研究人员和一家叫Arcada Labs的公司，最近给前沿模型10 000美元赌注了57天的预测市场。 这些模型在实验中不断地损失了资金，但研究者们看到这些模型在一段时间内不断改进的证据。
徐告诉我，AI可以最有效地使用自定义的牵引力进行预测，而指出拥有agents的独特任务——阅读新闻，收集其他数据，分析整个系统，例如——似乎是最有效的方法。
有趣的是，AI仍然常常落后于被称为"超级福雷克特人"的熟练专家。 徐解释说，虽然人类无法消耗同样多的信息，但他们对真实世界——即新闻背后的动态——的理解更深。
这称，越来越多的证据表明，人类超级福克的边缘正在被侵蚀。 最近一场名为"Metaculus Cup"的比赛显示，AI有时甚至可以击败最优秀的人类预测者。 而一个被称作"预测奔驰"的基准测试可以提供进一步的证据，证明当持续比赛的结果很快结束时AI可以匹配人类技能。 事实上，我认为我们可以期望大实验室将注意力转向未来探索作为一种强大的新方法来衡量机器智能的进步。
所有这些预测性实验似乎都表明，像数学、编码和法律一样，基于贸易的行业可能很快被大赦国际改进。 也许最好的人贩子会承担一种新的角色：监督和组织AI agents，能够远比他们更能支配未来。
目前，我的AI的建议似乎并不那么令人印象深刻。 过去几周，我当然不会在自己的选择中使用它的任何建议。
但AI无疑会好起来，也许让它太容易主宰幻想联盟很快。 希望不会，因为这会破坏通过自己赢得一切来赢得吹嘘权的乐趣。
这是威尔·奈特的AI Lab通讯的版本。 在这里阅读之前的通讯。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
I have consulted my AI-powered crystal ball, and it’s telling me that frontier models are going to get a lot better at making predictions in the near future. Just ask my Premier League fantasy team.
A few weeks ago, I deployed a small horde of AI agents to craft and manage an experimental roster. Given the proliferation of AI agents into pretty much every aspect of our lives, I wanted to know if a swarm of bots could put together a winning team.
I first looked at the potential for AI here a few years ago, playing around with a tool from the UK’s Turing Institute that selected teams after modeling previous fantasy league data. Its suggestions were remarkably bonkers—it didn’t incorporate real-world information, like a player’s injury or a team’s change in management.
This time, while on a flight to visit family, I asked Codex for help. First, I fed the model a couple of relevant research papers. Then, I asked it to build a harness—instructions and code that guides different agents through various steps—that would check recent news and run a model to make predictions.
It quickly came up with a setup that seemed to collate and analyze way more information than I ever would have. And yet I was surprised by how dumb the initial team suggestions were. Leaving Erling Haaland on the bench? Really, Codex?
I then asked Codex to create a team of agents that did different things: several that read news and blog posts; others that analyzed recent results; more that ran models; and a bunch that critiqued suggestions. This seemed to significantly improve the suggestions. I tried increasing the number of agents to more than 100, hoping this would make the system smarter still, but it didn’t change much besides running up my API bill.
Still, the improvement from a few years ago was striking. According to Haifeng Xu, a computer scientist at the University of Chicago who studies the uses of AI in market situations, newer models may be even better at making predictions because they’re able to consume and analyze huge amounts of real-world information, and they’re able to orchestrate more complex systems for their guesses.
“AI is just getting smarter in understanding the world—I feel like that's the fundamental reason they’re improving,” Xu tells me.
Late last year, Xu was marveling at the growing popularity of prediction markets like Kalshi and Polymarket. He wondered how the best commercial models might fare on those markets, so alongside several colleagues, he began testing their forecasting skills.
They had models guess, for example, who would win the NBA Championship and who President Trump would meet with.
“Very quickly we started to realize that models did really well,” Xu tells me. The models were sometimes able to do better than the average person without any special instructions or guidance, he says. In their testing, GPT-5 was the best at making predictions, followed by Grok 4, Claude Sonnet, Gemini, and Llama.
The team went on to develop Prophet Arena, a benchmark that measures a model’s ability to make predictions. It’s one of a number of benchmarks designed to test AI’s future-gazing capabilities; another, Prediction Arena, from researchers at Harvard University and a company called Arcada Labs, recently gave frontier models $10,000 to wager on prediction markets over 57 days. The models consistently lost money in their experiment, but the researchers saw evidence of them improving consistently over time.
Xu tells me that AI can most effectively make predictions using a custom harness, and notes that having agents with unique tasks—reading the news, collecting other data, and analyzing the whole system, for instance—seems to be the most effective approach.
Interestingly, AI still often lags behind very skilled experts known as superforecasters. Xu explains that, although humans cannot consume as much information, their understanding of the real world—that is, of the dynamics behind a news story—is deeper.
That said, there’s growing evidence that the human superforecaster’s edge is eroding. A contest called the Metaculus Cup recently showed that AI can sometimes beat even the best human predictors. And a benchmark called ForecastBench could provide further evidence that AI can match human skills when the results of an ongoing contest conclude soon. In fact, I think we can expect the big labs to turn their attention to future-gazing as a powerful new way to measure progress in machine intelligence.
All of these predictive experiments seem to suggest that, like math, coding, and law, industries predicated on trading may soon be upended by AI. Perhaps the best human traders will take on a new kind of role: supervising and orchestrating AI agents capable of divining the future far better than they can.
For the time being, my AI’s suggestions still don’t seem all that impressive. Its team hasn’t outscored the average over the past weeks. I certainly won’t be using any of its suggestions in my own selections.
But AI will no doubt get better, perhaps making it all too easy to dominate fantasy league pretty soon. I’m kind of hoping it won’t, though, as it would spoil the fun of trying to earn bragging rights by winning all by yourself.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.