思想领袖
为什么Token成本不告诉你企业AI是否工作
在 Google 上的首选来源中添加 Unite.AI
企业AI正在经历一个熟悉的周期。 起初，各种组织纷纷走入，因为技术令人振奋，法案尚未到来，因此达到了最大和最有能力的模型。 当它到达时，反应是同样可预测的：队伍开始配给准入，减少实验，并寻找更便宜的模型。
Databricks的共同创始人兼首席执行官Ali Ghodsi将这一周期的早期阶段描述为“tokenmaxing”，其中消耗更多的代币被视为进步的证据。 更重要的阶段是“价值最大化”，这意味着了解支出产生什么，并引导支出用于重要的工作。
这不仅仅是成本优化的新名称。 成本优化询问一个组织如何能减少支出，而价值放大则询问它如何从支出中获得更多。 随着AI从孤立的实验转向日常企业业务，这种区分将变得日益重要。
最大的模型并不总是正确的模型
许多组织开始他们的AI方案时，使用了几乎每一项任务可用的最强大的模型。 这种方法在早期的试验阶段是有意义的，当时各小组正在试图了解技术可以做些什么，而模型能力比可预测性更重要。
然而，随着部署的增加，对每项请求使用前沿模式的费用变得难以解释。 一些组织的反应是，限制准入或要求小组少做一些工作，但广泛的限制可抑制有益的试验和真正的浪费。
正在出现一种更为成熟的办法。 团队开始将较小的专业化模式与日常工作相匹配，同时为需要复杂推理或承担高昂失败成本的任务保留更能胜任的模式。 随着时间的推移，一些组织也可能在自己的基础设施内部署模型，以获得对成本、业绩和敏感数据的更大控制。
重要的问题不是大模型还是小模型更好。 问题在于模型是否适合工作，结果是否证明生产模型的总成本是合理的。
Tokens 显示活动，不显示值
大多数企业AI仪表板都是围绕消费建造的。 它们显示使用了多少道具，哪些模式处理了道具，以及该组织花费了多少。 这些数字是必要的，但并不表明工作是否产生了有益的结果。
一个消耗了两倍多的代币的团队不一定会产生两倍多的价值。 它可能处理更困难的工作，使用一种不必要的强大模型，或者依赖agent来反复重复失败的任务。 模型成本高的工作量如果能防止停电，解决客户问题，或让雇员返回宝贵时间，仍可能是一种出色的投资。
同样的问题在可观察性上存在了多年，只有千兆字节而不是符号。 团队开始收集一切，因为最大的能见度似乎是最安全的策略。 当存储和处理账单增加时，它们开始取样数据，减少留存，并降低上下文来控制支出。
在这两种情况下，对费用上升的反应往往是减少。 各组织需要的是能够确定哪些活动是有价值的，哪些活动是浪费的，哪些活动可以在不牺牲结果的情况下提高效率。
实际成本超出推论
模式调用很少是AI的全部工作量。 agent可以在生成答案之前检索文档，查询数据库，搜索遥测，调用外部服务，并生成新的应用程序或基础设施活动。
考虑agent调查为何在部署后延长离职时间。 它可以检查日志、比较衡量标准、检查痕迹、检索部署记录并分析下游服务。 如果它的第一个理论失败，它可能会调整时间窗口，在不同服务名下搜索，并遵循几个替代原因。
因此，一项人类指令可以产生数百起机器产生的操作。 这些业务消耗了计算、存储输入和产出、网络容量和第三方API配额，但这些都不得出现在本组织的模式支出仪表板上。
这意味着看似廉价的模型可以支持昂贵的工作流程。 更有能力的模型可能花费更多的费用，但用更少的步骤得出正确的结论，需要较少的复述和支持性基础设施。 进行这种比较的唯一途径是观察完成的任务。
连接活动、费用和结果
价值交流要求各组织将经常单独测量的三类信息连接起来。
第一个是AI活性，包括模型选择，符号消耗，耐久性，工具调用，重复，以及平行分支。 第二是这项活动产生的基础设施和财务成本，包括计算、储存、网络使用、数据库负荷和外部API收费。 三是成果，应说明工作是否顺利完成，是否有人认为成果有用。
并不是每个结果都需要转化为准确的美元数额。 一项实际措施可能就足够了，例如，是否接受一个答案，一个人是否必须重新做工作，事件是否得到更快的解决，或者自动工作流程是否在没有干预的情况下完成。
例如，如果能够帮助工程师提前30分钟恢复关键服务，那么引发上百个询问的事件调查可能是值得的。 如果一个小得多的工作流程能够产生一个没有人读过的报告，那么这个流程可能仍然是浪费。 仅靠消费无法区分两者。
一旦各组织能够将活动与成果联系起来，它们就能够审慎地决定在哪里使用前沿模型，哪里较小模型足够，哪里需要重新设计工作流程。
可见度不应要求移交控制
理解AI价值需要访问敏感背景。 提示、完成、源代码、客户信息和生产遥测都可能是同一工作流程的一部分。
各组织不应仅仅为了了解其人工智能系统的运作情况而向外部服务机构输出这种信息。 有助于衡量的提示和生产环境可能是安全、隐私和遵守小组不能允许离开本组织环境的同样信息。
因此，企业领导人应当询问他们的AI和可观察性数据处理在哪里，谁可以访问，保留多长时间，以及分析是否能够在其现有的安全边界内进行。 善政取决于能见度，但这种能见度绝不能造成新的数据控制问题。
使分析贴近基础数据也便于将AI活动与它所影响的应用、基础设施和业务流程联系起来。 这种全面的看法使各队不仅能够了解他们花费了多少资金，而且能够了解开支的回报。
优化值，不为量
在企业AI的下一阶段取得成功的组织不一定是那些使用最多标志或花费最少的组织。 他们将看到哪些模式和工作量产生有意义的结果，然后利用这些知识指导部署。
这需要超越孤立的模型成本仪表板。 各小组需要通过模式呼叫、工具、基础设施和最终结果，从用户请求中观察整个路径，同时将敏感信息置于适当的组织控制之下。
Tokenmaxing是一个有用的阶段，因为它鼓励了实验，并揭示了AI可以做些什么。 价值交流是必须遵循的学科，将能见度、成本和结果结合起来，这样各组织就可以继续试验，而不失去对支出或数据的控制。
企业AI不需要仅仅因为第一批账单已经到来而变小。 它需要更加周密，努力与困难相匹配，模式与任务相匹配，支出与可衡量的价值相匹配。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Thought Leaders
Why Token Costs Don’t Tell You Whether Enterprise AI Is Working
Add Unite.AI to your preferred sources on Google
Enterprise AI is moving through a familiar cycle. At first, organizations went all in, reaching for the largest and most capable models because the technology was exciting and the bill had not yet arrived. When it did arrive, the response was equally predictable: teams began rationing access, reducing experimentation, and searching for less expensive models.
Ali Ghodsi, cofounder and CEO of Databricks, has described the early stage of this cycle as “tokenmaxxing,” where consuming more tokens is treated as evidence of progress. The more important stage is “valuemaxxing,” which means understanding what that spending produces and directing it toward the work that matters.
This is not simply a new name for cost optimization. Cost optimization asks how an organization can spend less, while valuemaxxing asks how it can get more from what it spends. That distinction will become increasingly important as AI moves from isolated experiments into everyday enterprise operations.
The Largest Model Is Not Always the Right Model
Many organizations started their AI programs by using the most powerful model available for almost every task. This approach made sense during the early experimental phase, when teams were trying to understand what the technology could do and model capability was more important than predictability.
As deployments grew, however, the cost of using a frontier model for every request became difficult to justify. Some organizations reacted by limiting access or asking teams to do less, but broad restrictions can suppress useful experimentation along with genuine waste.
A more mature approach is emerging. Teams are beginning to match smaller, specialized models to routine work while reserving more capable models for tasks that require complex reasoning or carry a high cost of failure. Over time, some organizations may also deploy models within their own infrastructure to gain greater control over cost, performance, and sensitive data.
The important question is not whether a large or small model is better. It is whether the model is appropriate for the work and whether the result justifies the total cost of producing it.
Tokens Show Activity, Not Value
Most enterprise AI dashboards are built around consumption. They show how many tokens were used, which models handled them, and how much the organization spent. Those numbers are necessary, but they do not reveal whether the work produced a useful result.
A team that consumes twice as many tokens is not necessarily creating twice as much value. It may be handling more difficult work, using an unnecessarily powerful model, or relying on an agent that repeatedly retries failed tasks. A workload with high model costs may still represent an excellent investment if it prevents outages, resolves customer issues, or gives employees back hours of valuable time.
The same problem has existed in observability for years, only with gigabytes instead of tokens. Teams began by collecting everything because maximum visibility appeared to be the safest strategy. When storage and processing bills increased, they started sampling data, reducing retention, and dropping context to control the expense.
In both cases, the response to rising cost has often been to do less. What organizations need instead is the ability to identify which activity is valuable, which is wasteful, and which could become more efficient without sacrificing the outcome.
The Real Cost Extends Beyond Inference
A model call is rarely the complete AI workload. An agent may retrieve documents, query a database, search telemetry, call external services, and generate new application or infrastructure activity before producing an answer.
Consider an agent investigating why checkout latency increased after a deployment. It might examine logs, compare metrics, inspect traces, retrieve deployment records, and analyze downstream services. If its first theory fails, it may adjust the time window, search under different service names, and follow several alternative causes.
One human instruction can therefore create hundreds of machine-generated operations. Those operations consume compute, storage input and output, network capacity, and third-party API quotas, none of which may appear in the organization’s model-spending dashboard.
This means a seemingly inexpensive model can support an expensive workflow. A more capable model might cost more per call but reach the correct conclusion in fewer steps, requiring fewer retries and less supporting infrastructure. The only way to make that comparison is to observe the complete task.
Connect Activity, Cost, and Outcomes
Valuemaxxing requires organizations to connect three types of information that are often measured separately.
The first is AI activity, including model selection, token consumption, latency, tool calls, retries, and parallel branches. The second is the infrastructure and financial cost created by that activity, including compute, storage, network use, database load, and external API charges. The third is the outcome, which should indicate whether the work was completed successfully and whether anyone found the result useful.
Not every outcome needs to be translated into an exact dollar amount. A practical measure may be enough, such as whether an answer was accepted, whether a person had to redo the work, whether an incident was resolved more quickly, or whether an automated workflow completed without intervention.
For example, an incident investigation that generates hundreds of queries may be worthwhile if it helps engineers restore a critical service 30 minutes sooner. A much smaller workflow may still be wasteful if it produces a report that nobody reads. Token consumption alone cannot distinguish between the two.
Once organizations can connect activity with outcomes, they can make deliberate decisions about where to use frontier models, where smaller models are sufficient, and where a workflow needs to be redesigned.
Visibility Should Not Require Surrendering Control
Understanding AI value requires access to sensitive context. Prompts, completions, source code, customer information, and production telemetry may all be part of the same workflow.
Organizations should not have to export this information to an outside service simply to understand how their AI systems are performing. The prompts and production context that make measurement useful may be the same information that security, privacy, and compliance teams cannot allow to leave the organization’s environment.
Enterprise leaders should therefore ask where their AI and observability data is processed, who can access it, how long it is retained, and whether the analysis can happen within their existing security boundary. Good governance depends on visibility, but that visibility must not create a new data-control problem.
Keeping analysis close to the underlying data also makes it easier to connect AI activity with the applications, infrastructure, and business processes it affects. That complete view is what allows teams to understand not only what they spent, but what the spending returned.
Optimize for Value, Not Volume
The organizations that succeed in the next phase of enterprise AI will not necessarily be those that use the most tokens or spend the least. They will be the ones that can see which models and workloads produce meaningful outcomes, then use that knowledge to guide deployment.
This requires moving beyond isolated model-cost dashboards. Teams need to observe the full path from a user request through model calls, tools, infrastructure, and the eventual result, while keeping sensitive information under appropriate organizational control.
Tokenmaxxing was a useful phase because it encouraged experimentation and revealed what AI could do. Valuemaxxing is the discipline that must follow, combining visibility, cost, and outcomes so organizations can keep experimenting without losing control of their spending or their data.
Enterprise AI does not need to become smaller simply because the first bills have arrived. It needs to become more deliberate, with effort matched to difficulty, models matched to tasks, and spending matched to measurable value.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.