几十卡、数百卡、上千卡，AI 算力基础设施的规模竞赛，正在进入一个新的阶段。
过去，大模型厂商展示算力实力，最直观的方式是拥有多少张 GPU、建设了多大规模的计算集群。如今，随着超节点成为 AI 基础设施领域的热门方向，厂商展示实力的方式又多了一个：能够把多少颗 AI 芯片连接起来，让它们像一台计算机一样协同工作。
从英伟达的 GB200 NVL72，到华为的数百卡、千卡级超节点，再到各家国产厂商陆续推出的紧耦合计算系统，超节点的规模不断扩大。72 卡、384 卡、1024 卡，这些数字也越来越频繁地出现在产品发布和技术讨论中。
乍看之下，这似乎是过去算力竞赛的延续：以前比谁拥有的 GPU 更多，现在比谁能把更多 GPU 连接在一起。
但超节点真正试图解决的，恰恰是过去那种依靠增加机器数量扩充算力的方式所暴露的问题。
当模型规模越来越大，计算不再是唯一的瓶颈。芯片之间的数据交换、不同计算任务之间的同步，以及内存资源能否被高效使用，都开始影响最终的运行效率。增加更多芯片固然能够提高理论算力，但如果大量时间被消耗在数据搬运和等待上，硬件投入就未必能够转化为相应的性能收益。
超节点由此受到关注。它希望通过更高带宽、更低时延的互联，以及计算、内存和软件系统的协同设计，让更多芯片成为一个紧密协作的计算整体。
不过，一个值得追问的问题也随之出现：当超节点本来是为了提高计算效率而出现，为什么卡数又成为最受关注的竞争指标？连接的芯片越多，就一定意味着模型运行得越快吗？
从“堆机器”到“重构连接”，超节点为什么突然火了？
超节点并不是简单地把过去的多卡服务器做得更大，它在试图改变计算资源的组织方式。
传统 AI 集群通常以多卡服务器为基本单元。以常见的八卡服务器为例，服务器内部通过 NVLink 等高速互联技术实现 GPU 之间的数据交换；当模型或训练任务需要更多资源时，再将多台服务器通过 InfiniBand 或以太网连接起来。
这种架构具有很强的扩展能力，也支撑了过去几年的大模型训练。但问题在于，服务器内部和服务器之间的通信并不处于同一个性能层级。
当模型能够在一台服务器内部完成主要计算时，数据可以依靠相对高速的内部互联进行交换。一旦模型规模超过单台服务器的承载范围，就需要把计算和数据分布到更多服务器上，频繁跨越节点边界。
对于需要大量同步和数据交换的任务而言，这些边界会直接影响效率。GPU 一方面负责执行计算，另一方面还要等待其他芯片完成计算、传回数据，才能继续下一步工作。
这就意味着，计算集群拥有多少理论算力，与模型实际能够获得多少有效算力，并不是一回事。
2024 年 3 月，英伟达在 GTC 发布 Blackwell 平台及 GB200 NVL72，为这道难题提供了一个醒目的新解法。
GB200 NVL72 将 72 颗 Blackwell GPU 连接进同一个 NVLink 域，把紧耦合计算的范围从传统多卡服务器进一步扩大到整个机柜。
它的出现，改变了增加资源的组织方式。过去，模型超出一台服务器的承载范围，就要更多地依赖服务器间网络；如今，更多 GPU 可以进入同一个高速互联域，在更紧密的连接范围内协作。超节点希望减少的，正是模型跨越这些边界时产生的通信和数据搬运开销。
对于大模型，这种变化有直接吸引力：更多芯片参与计算，也提供更多显存资源；更快的互联则为模型并行、专家间数据交换等操作创造了更好的条件。能否将这些条件转化为性能，仍要依靠模型切分、算子和调度优化，但系统设计已经有了新的空间。
后来公开的模型研发实践，进一步解释了这种需求。
2025 年 5 月，DeepSeek 团队发表架构论文，披露其使用的 H800 系统中，按论文的有效带宽口径，节点内 NVLink 通信约为 160GB/s，跨节点 InfiniBand 通信约为 40GB/s，相差约四倍。为控制混合专家模型的通信成本，团队限制了每个 Token 所访问的专家跨越多少个节点，并利用更快的节点内连接转发数据。
这项实践说明，芯片怎样连接，已经会反过来影响模型怎样设计。
超节点之所以受到关注，正是因为行业需要把更多资源用起来，同时减少资源之间的协作负担。
厂商很快沿着这一方向推出产品。
华为、浪潮信息、阿里云……几乎各大厂商都在布局超节点。2025 年 4 月，华为云发布 CloudMatrix 384，并宣布已在芜湖数据中心规模上线，将超节点推向数百卡规模。2026 年 7 月，华为公开展示 1024 卡配置的 Atlas 950 SuperPoD，提供 256TB 全局统一内存编址空间。浪潮信息近期也发布了元脑 SD200 Ultra，以 128 芯本土 AI 芯片构建紧耦合系统，提供 8TB 统一编址显存，并围绕通信和模型算子进行优化，单机承载运行 2.8 万亿参数的 Kimi K3 大模型，Token 生成时延首次突破 5.85 毫秒，相当于单用户速度 170 Tokens/s。
不同产品的芯片能力、互联机制和部署形态并不相同，但它们共同推动了一种变化：AI 基础设施的竞争，从采购多少芯片，进一步走向如何组织芯片。
只是，超节点火起来之后，卡数又成为最醒目的数字。几十卡、数百卡、上千卡，高速互联域的规模不断扩大，发布会上最容易传播的，依然是“能连多少卡”。
超节点真正的较量是什么？
超节点火起来，是因为更多芯片需要更高效地协作。但如果把“高效协作”简化成“连接更多芯片”，很容易又回到过去的规模竞争。因为，支撑大参数量模型究竟能“跑多快”才是超节点更值得较量的能力。
以 Kimi K3 为例，作为全球最大开源模型，K3 拥有 2.8 万亿总参数、104B 激活参数，官方推荐 64 卡甚至更大规模的 AI 服务器才能装得下。可“装得下”并不意味着“跑得快”。有业内人士表示，一般未经优化的超节点运行 Kimi K3，初始性能可能仅在 10 tokens/s 左右。
这也暴露出超节点规模扩张背后的另一个问题：芯片多了，性能是否真的同步增长？
72 卡、384 卡、1024 卡，数字越大，越容易成为发布会的主角。但对于用户，一个更大的计算域也确实意味着更多资源，可以容纳更大的模型、更多中间数据，并为并行计算提供空间。问题是，这些资源要付出多少代价才能被用起来？新增一倍芯片，能否换来相应的性能提升？
所以，超节点真正需要解决的，并不是“还能连接多少芯片”，而是如何让更多芯片高效协同，把堆叠起来的算力真正变成全球前沿模型的推理速度。
当被问及 Scale-up 域是否越大越好时，浪潮信息首席 AI 战略官刘军给出了一个直接的回答：“规模不重要。核心是你的衡量标准是你要跑什么样的模型？你为这样的模型所匹配的计算架构是不是能让这个模型发挥出效果。”
这句话需要放在他的技术路线和产品定位下理解，但它提出了一个关键判断：超节点的价值，最终要回到具体模型的实际性能。
一个模型需要多少芯片，并不能只看参数量。权重采用什么精度、上下文有多长、同时承接多少请求，都会影响内存需求；采用怎样的并行策略、计算过程中需要交换多少数据，又会影响互联需求。对于混合专家模型，还要看专家如何分布、每个 Token 需要跨越哪些芯片。
能把模型放进一个计算域，只是选型的起点。放进去之后能否达到目标时延、支持所需并发，才决定这套配置是否合适。为了让单个请求更快，可能需要更多芯片共同处理；为了承接更多请求，也可能更适合把资源分成多个实例。两种需求，并不一定指向同样大的计算域。
如果模型在较小的域内已经能够高效运行，继续扩大单域，未必比增加服务实例更划算。反过来，如果模型的容量和通信需求已经超出较小系统的承载范围，更大的紧耦合域就可能是必要选择。不存在一个脱离模型和业务、普遍适用的“最佳卡数”。
规模扩大，还会把协作本身变成更复杂的工程。
刘军在采访中指出，规模大了，通信成本及问题的放大效应也会上升。如果数据和计算相隔过远，通信、调度的开销可能超过新增计算资源带来的收益。
高速互联可以降低这些开销，却不会让它们消失。即便资源已经统一寻址，访问远端显存也不意味着与访问本地显存具有相同成本。数据经过多少跳、链路是否拥塞、计算能否与通信重叠，仍然影响实际性能。统一地址空间让资源更容易被访问，软件还需要决定怎样访问才高效。
超节点越大，对拓扑和软件的要求也越高。 它需要证明的，不只是所有卡都能连接起来，还包括资源能否灵活分配。大任务需要整个域时，系统能否高效协同；小任务只需要一部分资源时，剩余资源能否服务其他业务？不同任务之间是否会相互干扰？如果只能用满配跑出好成绩，却难以经济地承接日常负载，规模优势就可能变成闲置成本。
因此，超节点是否好用，不能只看它能容纳多大的模型，更要看目标模型在其中能否达到预期的时延和吞吐。
可靠性是另一笔不能略过的账。
刘军特别提到，长程 Agent 任务可能持续数小时甚至数天，运行中断会带来更大的损失。但这并不意味着大超节点必然不可靠，也不意味着一次底层故障一定让任务从头开始。最终影响取决于故障能否被隔离、任务状态是否保存，以及服务能否重试、迁移和恢复。
对用户来说，更值得追问的是：一条链路或一颗芯片出问题，会影响多少资源？维护一部分设备时，其他业务能否继续运行？恢复需要多久，会丢失多少进度？这些问题决定了规模扩张的风险能否被控制，不能用一次短时间性能测试代替。
最后，规模还要落实到机房和预算上。更多芯片需要相应的供电、散热和互联设施，也带来软件适配与运维成本。采购价只是投入的一部分。如果为了获得有限的性能提升，需要大幅改造机房，或者增加长期维护负担，那么更大配置未必具有更好的回报。
“超节点，难在统一寻址”
超节点真正的吸引力，在于把原本分散的计算和存储资源组织成一个更紧密的协作整体。问题是，“像一台计算机一样工作”是一种目标描述，背后仍有大量架构差异。
刘军在采访中特别强调统一寻址。他认为，如果只是将模型拆成多个部分，分别放到不同服务器上运行，那仍然属于节点集群，浪潮信息所理解的超节点，需要首先解决显存统一寻址，让多颗加速器之间能够更直接地访问数据。
这是理解其技术路线的一个重要入口，但不能把它直接当成所有厂商共同遵循的定义。判断一套系统，仍需要拆开几个经常被混用的概念：资源是否具有统一的地址表达，远端访问能否由加速器直接发起，访问路径经过多少层交换，以及软件如何管理数据放置和同步。
统一寻址首先解决的是“能否找到、如何访问”的问题，并不自动意味着远端显存与本地显存具有相同的访问成本，也不自动保证所有资源都能高效共享。即便从软件看来，它们属于同一个地址空间，物理距离、拓扑和链路拥塞仍然存在。数据放在哪里、什么时候搬、能否与计算重叠，依然决定最终性能。
刘军用“串门”解释浪潮信息的对称内存方案：过去访问另一颗 GPU 的数据，像先去物业拿门条；现在则希望直接到邻居家。他强调，在统一寻址之后，还要进一步减少通信中的调度和中转开销。
这个比喻的价值，在于把超节点真正要解决的问题从“连接更多卡”拉回到“让这些卡更有效地协同”。如果扩大了资源池，却没有降低访问和同步成本，软件看到的空间可能更大，计算单元等待数据的时间也可能更长。
因此，“能放下模型”只是第一道门槛，“放下之后能否高效运行”才是更难的一道。
超节点如何选型：先看业务，再定规模
还有个问题值得关注，不同模型对系统的需求，并不随着参数量按同一种比例增长。
权重的存储需求与参数量、数据精度有关；推理时还要考虑上下文、并发请求及 KV Cache 等额外开销。对采用混合专家架构的模型而言，总参数量、每个 Token 实际激活的参数量，以及专家之间的通信，也分别约束着容量、计算和互联。仅凭“支持多少万亿参数”，很难推导真实的服务能力。
训练和推理的规模选择同样不能混为一谈。训练通常更关注一次训练任务的扩展效率；推理还要同时兼顾首 Token 时延、后续 Token 生成速度、并发和总吞吐。在一个大域内让单个请求更快，与将资源拆成多个服务实例来承接更多请求，可能是不同的优化方向。
在这次采访中，刘军把需求分为“能力型”和“容量型”。前者用于支撑前沿模型，追求复杂问题的处理能力；后者用于大量日常任务，追求充分、经济的智能供给。
在他的观察中，客户往往两者都需要：探索、规划和关键判断希望用更强的模型，经过验证的重复流程则希望更快、更便宜地执行。
企业需要一个能力上限，但未必需要每项任务都使用同样昂贵的资源。能够运行前沿模型的配置，应当验证复杂任务的质量和完成时间；承接日常服务的配置，则要验证在既定质量与时延下，能够持续处理多少请求。
因此，超节点选型应从业务往下推：先确定目标模型、精度、上下文、并发和时延要求，再结合并行与部署策略，确定需要多大的紧耦合域。测试也应使用接近真实业务的请求长度和到达节奏，观察高峰期间的尾部时延，而不能只看少量请求下的最快生成速度。
随后，再逐步增加资源，检查新增芯片究竟改善了什么。如果模型已经满足要求，扩大单域却没有明显提高有效吞吐，就应该重新考虑资源分区或增加服务实例；如果瓶颈来自容量和跨芯片通信，更大的域才可能具有明确价值。
超节点的合理规模，应当从业务测试中推导出来。 厂商给出的最大配置，说明系统能够扩展到哪里，用户自己需要验证的，是自己的业务在哪个规模上，能够以合理成本获得稳定、合格的产出。
超节点兴起，是因为行业逐渐发现，买来的芯片再多，也需要解决它们如何协作的问题。如今，当超节点本身进入规模竞赛，行业需要警惕的是：曾经希望通过架构创新解决的效率问题，会不会再次被更大的数字遮住。
超节点走向成熟的标志，或许不是“能做多大”，而是“能跑多快”。
当行业能够清楚解释为什么需要增加一张卡，也能坦然说明为什么不必再加，算力建设才真正从规模竞赛走向了系统效率判断。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Dozens of cards, hundreds of cards, thousands of cards, the size of the AI computing infrastructure is entering a new phase.
In the past, LLM manufacturers demonstrated their ability to calculate, most intuitively through the number of GPUs and large-scale computing clusters. Now, as the hypernodes become the popular orientation of the AI infrastructure, there is an additional way for manufacturers to show their strength: how many of the AI chips can be connected to work together like a computer.
From GB200 NVL72 in Weida, to the hundreds of cards and kilocal-class hypernodes in China, to the fast-lined computing systems introduced by various domestic manufacturers, the size of the hypernodes is growing. 72 cards, 384 cards and 1024 cards are also increasingly present in product publish and technical discussions.
At first glance, this seems to be a continuation of the old calculus: there were more GPUs than who had them, and now there are more GPUs than who can connect them.
But what the hypernodes are really trying to solve is precisely the kind of problem that has been exposed in the past by increasing the number of machines to expand the calculus.
When the model is growing in scale, computing is no longer the only bottleneck. Data exchange between chips, synchronization between different computing tasks and the efficient use of memory resources begin to affect the ultimate operational efficiency. While the addition of more chips would improve the theoretical calculation, hardware input may not translate into corresponding performance gains if a significant amount of time is spent on data handling and waiting.
Supernodes have thus received attention. It hopes to make more chips a closely coordinated whole of calculations through the interconnection of higher bandwidth and lower latency, as well as co-design of computing, memory and software systems.
However, a question worth asking follows: Why is the number of cards the most interesting indicator of competition when the hypernodes were created to improve the efficiency of calculations? The more chips you connect, the faster the model goes?
From "stacking machines" to "reconstructing connections," why is the hypernodes suddenly on fire?
The hypernodes do not simply do more with the former Doca servers, but are trying to change the way resources are organized.
Traditional AI clusters are usually based on Doc servers. In the case of a common eight-card server, data exchange between GPUs is achieved within the server through high-speed interconnected technologies such as NVLink; multiple servers are connected through InfiniBand or with a web when more resources are required for a model or training mission.
This structure has a strong outreach capacity and has supported LLM training over the past few years. The problem, however, is that communications within and between servers are not at the same level of performance.
When the model is able to complete major calculations within a server, data can be exchanged on relatively high-speed internal interconnections. Once the size of the model exceeds the carrying range of a single server, calculations and data will need to be distributed to more servers, often crossing node boundaries.
These boundaries have a direct impact on efficiency for tasks requiring considerable synchronization and data exchange. The GPU, on the one hand, is responsible for the implementation of the calculations and, on the other hand, is awaiting the completion of the calculations and the return of the data for the continuation of next step.
This means that it is not the same thing to calculate how much the cluster has in theory and how much effective the model can actually achieve.
In March 2024, Ying Weidar provided an interesting new solution to this challenge at the GTC publish Blackwell platform and GB200 NVL72.
GB200 NVL72 connects 72 Blackwell GPUs to the same NVLink domain, further extending the range of compacting calculations from the traditional Doc server to the entire cabinet.
Its emergence has changed the way resources are organized. In the past, the model went beyond the carrying of a server and became more dependent on inter-server networks; today, more GPUs can enter a high-speed interface and collaborate within a closer connection. It is the communication and data removal costs that the model incurs when it crosses these boundaries that it wants to reduce.
For LLM, this change is directly attractive: more chips are involved in computing and more VRAM resources are available; faster connectivity creates better conditions for the operation of parallel models, data exchange among experts, etc. The ability to convert these conditions into performance remains dependent on model splitting, algorithms and scheduling optimization, but new space is available for system design.
This need was further explained by subsequent public modelling development practices.
In May 2025, the DeepSeek team published a framework paper disclosing its use of the H800 system, with NVLink communications at the node of approximately 160 GB/s, and the Infini Band communication at the cross node of approximately 40 GB/s, which is about four times different. To control the communication costs of the hybrid expert model, the team limited the number of nodes per Token-visited expert to cross and used a faster in-line connection to transmit data.
This practice suggests that how the chip is connected has, in turn, affected how the model is designed.
The focus on hypernodes is precisely because industry needs to devote more resources while reducing the burden of collaboration between resources.
The manufacturer quickly launched the product in this direction.
Wigand, Wave, Alibaba Cloud. Almost every major manufacturer is set up over the nodes. In April 2025, the Chinese cloud publish Cloud Matrix 384 was declared operational at the size of the Sun Lake data centre, moving the node to hundreds of cards. In July 2026, China provided a common global memory site space for Atlas 950 SuperPoD, configured to publicly display 1024 cards. The wave information has also recently included publish meta-brain SD200 Ultra, which constructs a compact system with 128 core indigenous AI chips, provides 8TB integrated site VRAM, and optimizes around communications and model algorithms, with a single machine carrying Kimi K3 LLM, Token generating latency for the first break of 5.85 milliseconds, equivalent to 170 Tokens/s per user.
The capabilities, interconnectivity mechanisms and deployment patterns of chips vary from product to product, but together they drive a change: AI infrastructure competition, moving from how many chips are purchased to how to organize them.
It's just that, after the super-lighting, the number of cards becomes the most visible. Dozens of cards, hundreds of cards, thousands of kilocalories, the increasing size of the high-speed interconnectivity, and the most easily distributed at publish meetings, remain “how many cards can be connected”.
What's the real fight over the node?
The hyper-lighting is due to the need for more effective collaboration of the chips. But if the term “efficient collaboration” is simplified to “more chips connected”, it is easy to return to the scale of the past. Because the ability to support a large parameter count model to “run fast” is more of an ability to compete.
In the case of Kimi K3, which is the largest open source model in the world, K3 has 2.8 trillions of total parameters, 104B activated parameters, officially recommended 64 cards and even larger AI servers. But “can” does not mean “run fast”. According to industry sources, Kimi K3 is generally not optimized and the initial performance may be around 10 tokens/s.
This also exposes another problem behind the expansion of the size of the hypernominal nodes: are there really synchronized increases in the number of chips?
72 cards, 384 cards, 1024 cards, the larger the number, the easier it is to become the main character of the publish conference. For users, however, a larger range of calculations does mean more resources that can accommodate larger models, more intermediate data and provide space for parallel calculations. The question is, at what cost are those resources to be used? Double the amount of chips. Can you do that with the upgrade?
So what really needs to be addressed is not “how many chips can still be connected”, but how much more chips can work together, making the stacking of numbers really the pace of reasoning for global frontier models.
When asked if the Scale-up domain was as big as possible, the Chief of Wave Information AI Strategic Officer Liu Jun gave a direct answer: “Size is not important. The core is your measure. What kind of model are you running? Is the computational architecture that you match for this model to make it work?
This sentence needs to be understood in the context of his technical route and product positioning, but it provides a key judgement: the value of the hypernodes, and ultimately the actual performance of the specific model.
How many chips a model requires, not just parameter count. The accuracy of the weights, the length of context and the number of requests that are received at the same time will affect memory needs; the parallel strategy used, the amount of data to be exchanged in the calculation process, and the interconnection needs. For hybrid expert models, it also depends on how the experts are distributed and what chips each Token needs to cross.
It's only the starting point of the selection. Whether or not the target latency can be reached when it is in place, and whether the support needs to be combined will determine whether the configuration is appropriate. In order to expedite individual requests, more chips may be needed to work together; in order to take on more requests, it may also be more appropriate to divide resources into multiple examples. Two types of requirements do not necessarily point to the same large range of calculations.
If the model is already operating efficiently in smaller domains, it is not necessarily more cost-effective to continue to expand the uni-domain than to increase the number of service examples. In turn, if the capacity and communication requirements of the model are beyond the carrying range of the smaller system, a larger compaction may be a necessary option. There is no one “best card” that is universally applicable, separate from models and operations.
The expansion would also turn collaboration itself into more complex work.
In an interview, Liu Jun noted that, given their size, the cost of communication and the magnification of problems would also increase. If data and calculations are too remote, the cost of communications and movement control may exceed the benefit of the additional computing resources.
High-speed interconnections can reduce these costs, but they will not disappear. Even if resources have been located in a single location, access to remote VRAM does not mean the same cost as access to local VRAM. The extent of data jumps, the congestion of links and the ability to calculate overlaps with communications still affect performance. Unified address space makes resources more accessible and software needs to decide how to access them in order to be efficient.
The larger the nodes, the higher the requirements for pouncers and software. It needs to prove not only that all the cards are connected, but also that there is flexibility in the allocation of resources. Can the system be synergistically efficient when a large mission requires an entire area; can the remaining resources serve other operations when a small mission requires only a portion of resources? Will there be interference between different missions? If it is difficult to carry the day-to-day load on an economic basis, with only a full complement of good results, size advantages can become idle costs.
Therefore, the usefulness of the hypernodes cannot be determined only by the size of the model they can accommodate, but also by whether the target model can achieve the expected latency and throughput.
Reliability is another one that cannot be omitted.
In particular, Liu Jun mentioned that the Agent mission could last for hours or even days, and that the disruption would result in greater losses. This does not mean, however, that the supernodes are necessarily unreliable, nor does it mean that a bottom failure must lead to the start of the mission. The ultimate impact depends on the isolation of the malfunction, the preservation of the mission status and the ability of the service to retry, relocation and restoration.
For users, it is even more interesting to ask: how much resources can be affected by problems with a link or a chip? Can other operations continue to operate while some equipment is maintained? How long will it take to recover and how much progress will be lost? These questions determine whether the risk of scale expansion can be contained and cannot be replaced by a short time performance test.
Finally, the scale would have to be applied to the aircraft room and budget. More chips require corresponding power supply, heat diffusion and interconnection facilities, which also entails software adaptation and transportation costs. The procurement price is only part of the input. If, in order to achieve limited performance enhancement, there is a need for substantial upgrading of aircraft rooms, or for a longer-term maintenance burden, the larger configuration may not have better returns.
"Supernodes, difficult to locate in a uniform"
The true attraction of the hypernodes is to organize the previously dispersed computing and storage resources into a more collaborative whole. The problem is that “working like a computer” is a description of a goal and that there are still a number of structural differences behind it.
In their interviews, Liu Jun placed particular emphasis on a unified location. In his view, if the model were to be broken down into multiple parts and operated on separate servers, it would still be a nodal cluster, the hypernodes understood by the wave information, and would need to first address VRAM ' s unified location to allow more direct access to data between accelerators.
This is an important entry point for understanding its technical route, but it cannot be taken directly as a definition shared by all manufacturers. To judge a system, there is still a need to break down several concepts that are often mixed: Whether resources have a uniform address expression, whether remote access can be directly initiated by accelerators, how many layers of access paths are exchanged, and how software manages data placement and synchronization.
The first solution to a unified location is the question of “possibility and access”, which does not automatically mean that the remote VRAM has the same access costs as the local VRAM, nor does it automatically ensure that all resources are efficiently shared. Even in the case of software, they belong to the same address space, and physical distances, pounces and chains still exist. Where, when and whether the data will overlap with the calculations will still determine the final performance.
The Symmetric Memory Program of Liu's Army's “Six Doors” to explain the wave's information: The data that used to go to another GPU is like going to the property first to get a note; now you want to go directly to the neighbor's house. He stressed that, following a unified location, movement control and transit costs in communications would be further reduced.
The value of this metaphor lies in bringing the problem that the hypernodes are really about to solve from “to connect more” to “to make these cards work more effectively”. If the pool is expanded without reducing access and synchronization costs, the software may see more space and the computing unit may wait longer for data.
Therefore, “can put the model down” is only the first threshold, and “can work efficiently after putting it down” is the more difficult one.
How to select a hypernodes: start with business, then scale
There is also a concern that the need for systems in different models does not increase at the same rate as parameter count.
The storage requirement for weights is related to parameter count, data accuracy; additional expenses such as context, the simultaneous request and KV Cache are also considered in the reasoning. For models using hybrid expert structures, total parameter count, parameter count, which is actually activated by each Token, and communication between experts are also limited by capacity, computing and interconnection, respectively. It is difficult to extrapolate real service capacity simply by supporting the trillion parameters.
The choice of the size of training and reasoning should likewise not be confused. Training tends to focus more on the effectiveness of the expansion of a training mission; reasoning also takes into account the speed at which the head Token latency, the follow-up Token generation, combined hair and total throughput. Allowing individual requests to be faster within a wide area and taking on more requests with multiple service examples of resources may be in different directions of optimization.
In this interview, Liu Jun classified needs as “capacity” and “capacity”. The former are used to support front-line models and to pursue the capacity to deal with complex issues, while the latter are used for a large number of day-to-day tasks in pursuit of adequate and economical intellectual supply.
In his observations, clients often need both: exploration, planning and critical judgement want a stronger model, while proven duplicate processes want faster and cheaper implementation.
An enterprise needs a capacity cap, but it is not necessarily necessary to use equally expensive resources for each task. The configuration of the front-line model, which is capable of running, should validate the quality of complex tasks and the completion time; the configuration of the day-to-day service should verify how many requests can be processed on a continuous basis under the established quality and latency.
Therefore, the supernode model should be rolled down from business: first, to determine the target model, precision, context, co-head and latency requirements, and then, in conjunction with the parallel and deployment strategy, to determine how close the field is required. The test should also use the request length close to the real business and the pacing to observe the end of the peak period latency, and not only the fastest generation rate at a small number of requests.
Subsequently, resources were gradually increased to examine what had improved the new chip. If the model has met the requirements and the expansion of the uni-domain has not significantly increased its effective ingestion, then there should be a reconsideration of the case of resource partitioning or additional services; if bottlenecks come from capacity and cross-chip communications, a larger area may have a clear value.
The reasonable size of the hypernodes should be extrapolated from the operational tests. The largest configuration given by the manufacturer indicates where the system can be extended, and what the users themselves need to verify is the scale of their operations, which can achieve stable and qualified outputs at reasonable cost.
The super-nodes have emerged because the industry has come to find that as many chips as possible have been bought, the question of how they work together also needs to be addressed. Today, as the hypernodes themselves enter the scale competition, the industry needs to be wary of whether the efficiency issues that were once sought to be addressed through structural innovation will again be overshadowed by larger numbers.
The sign of supernodes moving to maturity may not be “how much can be done”, but “how fast can they run”.
When the industry can clearly explain why an additional card is needed, it can also make it clear why it is no longer necessary to add more, so that it can truly move from a scale competition to a system-efficiency judgement.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.