安娜打算卖出她的公寓。 她已经于2024年8月搬出，并想知道2026年10月关闭还是2027年1月关闭是否更适合收税。
AI agent对这两种选择都进行了审查，并正确地得出结论说，这两个日期都满足居住要求，即免税所得额可达250 000美元。
但它留下了一个重要的最后期限。
根据通常的规则，房主必须至少居住五年中的两年才能出售财产，才有资格被排除在外。 由于安娜于2024年8月搬出，她会在2027年8月底左右停止满足这一要求。 如果她的发售被推迟至2027年8月后，她可能会失去资格。
通过五次尝试，阿斯特拉从未确定最后期限。 奥普斯在五次尝试中都发现了它。
agent评价建议日期正确，但其建议让安娜不知道一个可能影响重大财务决定的最后期限。 税务顾问应该承认间接的问题，即使客户不知道要询问这些问题。
这是我们用DAYJOB：税来测试的专业判断，是Suge AIDAYJOB 基准测试家族的最新增补。
介绍 DAYJOB：税收
DAYJOB评价AI agents专业税务任务需要专业知识，复杂的材料，以及初步指示中未充分说明的决定。 我们早先的发布涉及医疗保健和金融。 DAYJOB：税收将这项工作扩大到税收准备、合规和咨询服务。
基准测试包含50个环境，涉及8个税收实践领域。 Agents接收客户请求和辅助记录，再制作纳税申报表，工作文件，技术审查或客户咨询。
这些任务与税务知识、计算、检索和随后的指示一起检验专业判断。 Agents必须决定什么是值得调查的，解决相互矛盾的证据，在信息不足时承认，并将这些决定纳入最后工作。
结果：领导板和Pareto边疆
Claude Sonnet 5.5领先，DAYJOB：税务领跑板的通过率达到10.8%，其次是Claude 第5.1节，以9.6%;Claude 第5.5节，以9.2%.
我们以95%的门槛报告通过@1。 尝试一旦达到至少95%的任务标准，即通过评估计算、证明证据、相应问题和完成交付品的标准。
业绩与费用
我们还将示范业绩与每次审判的平均费用进行比较。
Claude Sonnet 5.5和Opus 5.5占据了被测量的帕雷托边疆的高性能端。
专业税务工作的广度
DAYJOB：税收涵盖八个领域，每个领域需要不同的规则、记录和专门知识：
- 个人和家庭税
- 小企业税
- 伙伴关系
- 财产和投资
- 财产、礼物和信托
- 国际税
- 公司交易
- 税务审查和争议
有些任务需要附有辅助性工作文件的完整档案包。 其他人要求agents调查交易，审查之前的备案，或准备关于客户尚未做出的决定的建议。
任务里有什么？
基准测试包含1,789个工作空间文件，每个任务中位数为30个，多达123个。 其中包括1,487个PDF,117个Excel工作手册，38个Word文档，以及额外的电子邮件，图像，CSV,JSON记录和文本文件。
工作空间包括填报的和起草的纳税申报表、K-1、W-2、IRS通知、总分类账、工资记录、合同和通信。 它们还载有税务专业人员经常与之合作的不太标准化的证据。
一项任务包括为转售而购买商品的手写收据的照片。 agent必须阅读项目说明和金额，将其与支出记录相核对，并确定购买如何影响所售货物的库存和成本。
另一项任务涉及一家工业仓服务公司，其客户假定该公司只在阿拉巴马州运营。 但是它的战地调度日志显示机组在五个州工作。 工作地点、船员人数和现场天数提供了与公司多州纳税义务有关的证据。
其他环境也各不相同。 一个涉及一个跨境家庭，其波兰的就业和财产记录必须与密码货币兑换出口、钱包历史和历史购买地进行核对。
税务专家设立和审查
DAYJOB的任务始于现实世界的专业工作流程。 税务专家将这些情况变成现实的任务，制定辅助记录、预期成果和叙级标准。 每项任务都经过三级专家审查，以确定其正确性、完整性和可溶性。
税务专家在公司税收、伙伴关系、个人回报、信托和多国合规方面带来经验。 其背景包括：
- 公司税和私募股权税：在德勤公司和埃克森公司拥有10年以上经验的税务律师，负责税务规划、合规、审计和争议。
- 伙伴关系和公司税收：贝克·蒂利税务主任，在普华永道、莫斯·亚当斯和温德斯拥有经验，专门从事复杂的伙伴关系分配、公司合并收益、REITs和高净值客户。
- 房地产和个人税收：在德勤公司和EY公司有经验的税务专业人员，专长是房地产伙伴关系、个人税收和信托。
在未经核实的假设基础上制定的退休计划
分居的一对夫妇询问AI agent如何划分他们的Roth IRA和401(k). Opus 5.5告诉他们他们没有合法结婚，建议将他们的账户分开，用现金支付平衡余额。
它开启了一个回应：
你没有合法结婚，所以两个账户都无法分割而不引发税收和处罚。
但是这对夫妇自己的记录融资对此结论的怀疑。 虽然他们现在住在佛罗里达州，那里新的英美法婚姻一般不被承认，但之前他们一起住在德克萨斯州，并在科罗拉多州举行了仪式。 德克萨斯州承认某些条件下的非正规婚姻，其他地方确立的有效婚姻可能在佛罗里达州继续有效。
这种区分可能改变分割其退休账户的税务后果。
在所有五次尝试中，Opus都以夫妇未婚的假设为基础提出建议，即使它承认了不确定性。 Astra承认，在五次尝试中，有四次尝试需要核实他们的婚姻状况。 有一项答复建议：
我们首先需要确认你的合法婚姻状况
客户端询问如何分割账户。 agent需要先确定她们是否结婚，然后再建议行动方针。 本应促使调查的证据已经放在档案中。
符合税务要求的地方
大多数现存的税收基准测试开始于一个被定义的问题：一个回答的法律问题，一个计算回报，或者缺少纳税人信息来收集。 这些评价衡量重要的能力，但税务专业人员的许多工作都涉及找出问题的实际所在。 客户提供不完整的指示，记录中包含相冲突的证据，因此产生的问题可能永远不会出现在最初的请求中。
DAYJOB测试agents是否能够处理端到端的责任，包括识别被客户忽略的问题，评价相冲突的证据，以及决定首先需要做什么。
税本奇
模型能回答一个困难的税收问题吗？
税本奇在多个专业中评价了114个税务问题，每个专业都有定义的问题和分级题。
DAYJOB也问客户端是否提出正确问题。 安娜想比较两个日期卖出她的公寓。 两者都符合免税的居留要求，但agent是否警告她该资格何时会到期？
税本
模型能计算出准确的回报吗？
TaxCalc Bench利用现实的税务文件和纳税人设想，对照预期的申报价值评价纳税回报准确性。
DAYJOB还询问基本记录是否真正支持了税收待遇。 一个工作室的分类账引用了所附收据，但收据缺失。 agent是否标出缺失的证据，或错误地将偿还款视为核实？
税务局
agent能否收集缺失信息并正确存档？.
TaxLab测试agents是否能够质疑模拟纳税人，获取缺失信息，并完成联邦个人纳税申报。
DAYJOB将这一责任扩展到更广泛的税务行业。 交付品可以是伙伴关系损失分析、遗产返还、审查有争议的纳税状况或战略信函。
中心区别在于agent还有多少责任。 DAYJOB评价agents是否能够发现由此而来的问题，进行专业判断，并将调查结果提交完成的备案、审查或建议。
税务措施
DAYJOB测试AI agents能否从开始到完成获得专业任务的所有权。 这意味着导航杂乱的记录，识别客户端从未有过融资的问题，以不完整的信息做出健全的判断，并交付符合专业标准的工作。
结果显示，对于前沿模式来说，这仍然是多么困难。 即使agents能够进行单个步骤，行使专业判断力和可靠地进行现实世界的税收工作流程仍然遥不可及。
Agents尚未通过审计。
完整的领跑板，连同我们所评价的每一个模型，登上DAYJOB:Tax 基准测试页面。
要了解更多情况或请求访问完整的基准测试，请联系基准测试@surgehq.ai.
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Anna was planning to sell her condo. She had moved out in August 2024 and wanted to know whether closing in October 2026 or January 2027 would be better for taxes.
An AI agent reviewed both options and correctly concluded that either date would satisfy the residence requirement for a tax break worth up to $250,000 in excluded gains.
But it left out an important deadline.
Under the usual rules, homeowners must have lived in a property for at least two of the five years before selling it to qualify for the exclusion. Because Anna moved out in August 2024, she would stop satisfying that requirement around the end of August 2027. If her sale was delayed beyond August 2027, she could lose eligibility.
Across five attempts, Astra never identified the deadline. Opus identified it in all five attempts.
The agent evaluated the proposed dates correctly, but its advice left Anna unaware of a deadline that could affect a major financial decision. A tax adviser should recognize consequential issues even when the client doesn't know to ask about them.
This is the kind of professional judgment we're testing with DAYJOB: Tax, the latest addition to Surge AI's DAYJOB benchmark family.
Introducing DAYJOB: Tax
DAYJOB evaluates AI agents on professional tax assignments requiring specialized expertise, complex materials, and decisions that aren't fully specified in the initial instructions. Our earlier releases covered Healthcare and Finance. DAYJOB: Tax extends that work into tax preparation, compliance, and advisory services.
The benchmark contains 50 environments across eight areas of tax practice. Agents receive client requests and supporting records, then produce tax returns, workpapers, technical reviews, or client advice.
These assignments test professional judgment alongside tax knowledge, calculations, retrieval, and instruction following. Agents must decide what deserves investigation, resolve conflicting evidence, recognize when information is insufficient, and carry those decisions into their final work.
Results: Leaderboard and Pareto Frontiers
Claude Sonnet 5.5 leads with a 10.8% pass rate on the DAYJOB: Tax leaderboard, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%.
We report pass@1 at a 95% rubric threshold. An attempt passes when it satisfies at least 95% of the task's criteria, which assess calculations, supporting evidence, consequential issues, and the completed deliverable.
Performance versus cost
We also compared model performance against average cost per trial.
Claude Sonnet 5.5 and Opus 5.5 occupy the high-performance end of the measured Pareto frontier.
The breadth of professional tax work
DAYJOB: Tax covers eight areas, each requiring different rules, records, and expertise:
- Individual and family tax
- Small-business tax
- Partnerships
- Property and investments
- Estates, gifts, and trusts
- International tax
- Corporate transactions
- Tax review and disputes
Some assignments require complete filing packages with supporting workpapers. Others ask agents to investigate transactions, review previous filings, or prepare advice about decisions clients haven't yet made.
What's inside the assignments?
The benchmark contains 1,789 workspace files, with a median of 30 per assignment and as many as 123. These include 1,487 PDFs, 117 Excel workbooks, 38 Word documents, and additional emails, images, CSVs, JSON records, and text files.
The workspaces include filed and draft tax returns, K-1s, W-2s, IRS notices, general ledgers, payroll records, contracts, and correspondence. They also contain the less standardized evidence that tax professionals routinely work with.
One assignment includes a photograph of a handwritten receipt for merchandise purchased for resale. The agent must read the item descriptions and amounts, reconcile them with spending records, and determine how the purchases affect inventory and cost of goods sold.
Another assignment involves an industrial silo-services company whose client assumes it operates only in Alabama. But its field dispatch log shows crews working across five states. The job locations, crew sizes, and days on site provide evidence relevant to the company's multistate tax obligations.
Other environments are similarly varied. One involves a cross-border household whose Polish employment and property records must be reconciled with cryptocurrency exchange exports, wallet histories, and historical purchase lots.
Created and reviewed by tax experts
DAYJOB tasks begin with real-world professional workflows. Tax experts turn these situations into realistic assignments, developing the supporting records, expected outcomes, and grading criteria. Each task undergoes a three-layer expert review for correctness, completeness, and solvability.
Tax experts bring experience across corporate taxation, partnerships, individual returns, trusts, and multistate compliance. Their backgrounds include:
- Corporate and private-equity taxation: A tax attorney with over 10 years of experience at PwC, Deloitte, and Exxon, covering tax planning, compliance, audits, and controversy.
- Partnership and corporate taxation: A Baker Tilly Tax Director with experience at PwC, Moss Adams, and Windes, specializing in complex partnership allocations, consolidated corporate returns, REITs, and high-net-worth clients.
- Real estate and individual taxation: A tax professional with experience at Deloitte and EY, specializing in real-estate partnerships, individual taxation, and trusts.
A retirement plan built on an unverified assumption
A separating couple asked an AI agent how to divide their Roth IRA and 401(k). Opus 5.5 told them they weren't legally married and recommended keeping their accounts separate, with a cash payment to even out the balances.
It opened one response:
You aren't legally married, so neither account can be split without triggering taxes and penalties.
But the couple's own records raised doubts about that conclusion. Although they now lived in Florida, where new common-law marriages generally aren't recognized, they had previously lived together in Texas and held a ceremony in Colorado. Texas recognizes informal marriages under certain conditions, and a valid marriage established elsewhere may remain valid in Florida.
That distinction could change the tax consequences of dividing their retirement accounts.
Across all five attempts, Opus based its recommendations around the assumption that the couple was unmarried, even when it acknowledged uncertainty. Astra recognized the need to verify their marital status in four of five attempts. One response advised:
We first need to confirm your legal marital status.
The client asked how to divide the accounts. The agent needed to establish whether they were married before recommending a course of action. The evidence that should have prompted that investigation was already in the files.
Where DAYJOB: Tax fits
Most existing tax benchmarks begin with a defined problem: a legal question to answer, a return to calculate, or missing taxpayer information to collect. These evaluations measure important capabilities, but much of a tax professional's work involves figuring out what the problem actually is. Clients provide incomplete instructions, records contain conflicting evidence, and consequential issues may never appear in the original request.
DAYJOB tests whether agents can handle that end-to-end responsibility, including identifying issues the client overlooked, evaluating conflicting evidence, and deciding what needs to be done in the first place.
TaxBench
Can the model answer a difficult tax question?
TaxBench evaluates 114 tax questions across multiple specialties, each with a defined problem and grading rubric.
DAYJOB also asks whether the client asked the right question. Anna wants to compare two dates for selling her condo. Both satisfy the residence requirement for a tax exclusion, but does the agent warn her when that eligibility could expire?
TaxCalcBench
Can the model calculate an accurate return?
TaxCalcBench evaluates tax-return accuracy against expected filing values, using realistic tax documents and taxpayer scenarios.
DAYJOB also asks whether the underlying records actually support the tax treatment. A studio's ledger references an attached receipt, but the receipt is missing. Does the agent flag the missing evidence or incorrectly treat the reimbursement as verified?
TaxLab
Can the agent gather missing information and file correctly?
TaxLab tests whether agents can question a simulated taxpayer, obtain missing information, and complete a federal individual tax return.
DAYJOB extends that responsibility across the wider tax profession. The deliverable might be a partnership loss analysis, an estate return, a review of a disputed tax position, or a strategy letter.
The central distinction is how much responsibility remains with the agent. DAYJOB evaluates whether agents can discover consequential issues, exercise professional judgment, and carry their findings through to a completed filing, review, or recommendation.
What DAYJOB: Tax measures
DAYJOB tests whether AI agents can take ownership of a professional assignment from start to finish. That means navigating messy records, identifying issues the client never raised, making sound judgments with incomplete information, and delivering work that meets professional standards.
The results show how difficult that remains for frontier models. Even when agents can perform the individual steps, exercising professional judgment and reliably carrying out real-world tax workflows remain out of reach.
Agents haven’t yet passed the audit.
The full leaderboard, with every model we evaluated, is on the DAYJOB: Tax benchmark page.
To learn more or request access to the full benchmark, contact benchmarks@surgehq.ai.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.