Rack-scale AI：戴尔和AMD如何建设基础设施
Dell Technologies Inc. 及其合作伙伴Advanced Micro Decembers Inc. 面临类似的竞争压力。 这两所技术发电所的运作环境都是在商业环境下进行的，现在通过中央处理单元、图形处理单元、架平面系统、内存容量、网络带宽和每美元信使衡量成功。
在这些措施之间竞争的压力凸显了CUBE研究所称的“工程速度 ”。 自从5月的戴尔技术世界会活动以来，AMD和戴尔并没有减缓合作和公告的速度。
AMD和戴尔正在努力建设企业AI基础设施，能跟上不断变化的工作量和客户需求。
称是CUBE研究的Dave Vellante。 “工程系统学习、验证和改进整个平台的速度可能比任何其他人都要快。”
这个特征是SiliconANGLE Media对建筑转向动力连续，生产级AI的探索的一部分。 以及今年的Dell Technology World的报导。 (* 以下披露。 )
小规模合并
Dell和AMD关系的基石是服务器巨头的PowerEdge组合。 戴尔于5月扩展了它的PowerEdge服务器行，并跟踪了AMD处理器集成的其他细节。
这两家公司采取了一项旨在促进服务器合并的战略，因为代理AI推动对通用计算的需求激增。 Agents严重依赖CPU，然而数据中心并没有太多额外空间
由AMD处理器提供动力的系统将带来70%的基因超原性能收益，并且可以将戴尔第14个PowerEdge世代的13个服务器折叠成一个，据戴尔的产品策划者和高级顾问Pradyumna Narendra说。
“每更换10到13个服务器，你就能得到如此多的生产率收益，因为现在你可以用一个服务器做到这一点，” “当你在数据中心中扩展它，这就像一个加速效应，你可以得到利用。”
Dell和AMD还必须满足大量的AI使用案例和伴随的计算需求。 戴尔的三机架单元服务器冷却所需的功率比两机单位等同的要小，让客户将更多的计算功率装入每个机架，根据纳伦德拉的说法。
他解释说，“通过部署包体稍大一些的服务器，在机架级别上交付密度更高，有时似乎反感”。 “随着大赦国际的加速、激起和推动性AI的周而复始，它的速度将加快并增长得更快。 随着这些使用案例的扩大，你得到指数增长的计算需求。”
模块部署和验证
除了服务器合并，戴尔和AMD一直在稳步加强模块化解决方案来简化AI的部署和缩放：戴尔AI平台与AMD.
即成型企业AI解决方案将AMD Instinct加速器和EPYC处理器与Dell PowerEdge服务器，PowerSwitch网络和PowerScale存储用于模型培训，微调和高通量推论相融合。 该平台的设计是让客户从满足当前工作量并随时间推移而扩展计算和GPU密度的配置开始，而不后置。
客户想要模块化解决方案。 称 Varun Chhabra在与CUBE讨论时，“计算、存储、联网[和]GPU。 上面的软件框架，模型：他们都接受了测试吗？它们是否都经过验证？”
戴尔将AMD ROCm软件堆栈整合到平台中，以支持培训，微调，并在-presises上服务大型语言模型。 根据Chhabra的说法，模块化平台允许客户从较小的部署开始，并在相同的基础设施上扩展。
“你可以从这些堆放、计算、联网、预先与AMD Instinct和EPYC CPU合并的存储、计算和联网的可堆放单元开始，然后用ROCm软件以及坐在上面的开放生态系统来进行小的整合。” “我们测试它，我们验证它，客户可以使用它，一旦它们开始看到它的价值，它们就知道它们可以用同样的框架以模块方式进行规模化。”
利用PCIe的惠益
从7月开始，Dell 宣布在其PowerEdge XE7745和R7725服务器中支持AMD Instinct MI350P PCI Express GPU，为企业提供了另一种缩放代理和基因AI部署的选项。 这也是对内存带宽的重要促进，在时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到到时间到时间到时间到时间到时间到时间到时间到时间到时间到到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间到时间
MI350P提供144千兆字节的HBM3E内存，根据业务开发高级经理"Instinct Data Center GPUs"的迈克·达比(Mike Darby)在CUBE上出场时的描述，这可以是推论性能的关键差异。
他的称表示，“这是许多记忆，“主要用例是推论：这都是暂时性的，是首先表示[和]用户体验的时候了，这样你的注意力就不会在等待你的反应时转移到别的地方去。 ”
AMD和戴尔在PCIe集成上的合作还有另一个值得注意的角度。 AMD基于PCIe的GPU支持空气冷却部署，为用户提供了一个在现有的数据中心基础设施内部署AI的路径。
科技行业最近几个月在全美建立大型数据中心时遇到阻力，另外，戴尔发现，由于它部署了下一代的PowerEdge服务器，一些企业客户仍然没有准备好在数据中心内采用液冷。
“我们与戴尔合作，在GPU Instinct的可空气冷却行进图中增加了PCI卡，使其适合更多Dell PowerEdge服务器，这些服务器将进入更广泛的企业”，称 Robert Hormuth，在与CUBE谈话时，担任位于AMD的数据中心解决方案组的架构和战略公司副总裁。 “并非每个人都采用直接液体冷却。”
开放市场的合作
Dell今年与AMD的合作已经超越了联合产品公告的范围。 6月，两家公司宣布与剑桥大学合作，在联合王国建立了主权AI创新实验室。
合作旨在帮助推进下一代AI基础设施和AI带动的科学突破。 它建立在剑桥扩展的AI研究基础设施上，包括由第五代AMD ESYC处理器和AMD Instinct MI355X GPU加速器所提供动力的Zenith AI超级计算机被整合入戴尔基础设施。
Dell和AMD最近几个月还就电信倡议与AT&T公司进行了合作。 电讯数据对于AI来说是一种重载，AT&T已经处理过超过一万亿个符号来训练模型。 AMD为通过Open Telco AI倡议开发的OTel 2.0模型的培训和推论工作量提供了高性能硬件基础。
Dell将这一技术与用于电信环境的AI-优化服务器相融合，为跨数据密集，真实世界网络环境的运营商创造了生产级基础。 OTel 2.0的训练基础设施使用戴尔的PowerEdge XE9785服务器，配置有8个MI355X GPU.
Dell和AMD的联盟凸显了AI基础设施市场增长中的一个关键发展：竞争的机会很多。 Dell和AMD似乎都致力于建设在AI处理解决方案中提供企业选择的组合。
" AMD没有将其新产品。作为 " 值得尊敬的替代品 "。 “它声称直接领导了CPU、GPU、机架级性能、内存能力、网络带宽和每美元信使。 它将这一论点扩展到企业AI、机器人、超低相关性推论和迅速扩展的伙伴生态系统。”
(*披露：CUBE是戴尔AI数据平台事件的付费媒体合作伙伴。 CUBE活动报道的发起人Dell或其他赞助者对CUBE或SiliconANGLE上的内容都没有编辑控制。 ).
图片：SiliconANGLE/ChatGPT
SiliconANGLE联合创始人John Furrier的留言：
支持我们的使命，通过与CUBE社区接触，保持内容的开放和自由。 加入CUBE的校友信托网络，其中技术领导者相互连接，分享情报并创造机会。
- 15M+观看CUBE视频，为跨越AI、云、网络安全等的对话提供动力
- 11.4k+CUBE校友——通过一个独特的信任网络，与11,400多位技术和企业领袖建立联系，塑造未来
你是AWS客户吗？在财务上支持SiliconANGLE，从我们的市场门户页面和链接购买您的AWS服务：https://siliconangle.com/aws-marketplace/
关于 SiliconANGLE 媒体
SiliconANGLE Media由技术视觉家John Furrier和Dave Vellante所创立，构建了产业领先数字媒体品牌的动态生态系统，达到了1500+百万精英技术专业人士。 我们的新专有CUBE AI Video Cloud正在观众互动中突破地步
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Rack-scale AI: How Dell and AMD are building infrastructure
Dell Technologies Inc. and its partner Advanced Micro Devices Inc. face similar competitive pressures. Both tech powerhouses operate in a business climate where success is now measured across central processing units, graphics processing units, rack-scale systems, memory capacity, network bandwidth and tokens per dollar.
The pressure to compete across those measures underscores what theCUBE Research has termed “Engineering Velocity.” Since the Dell Technologies World event in May, AMD and Dell have not slowed their pace of collaborations and announcements.
AMD and Dell are working to build enterprise AI infrastructure that can keep pace with changing workloads and customer requirements.
“In AI infrastructure, the winner may not be the company with the best benchmark on keynote day,” said theCUBE Research’s Dave Vellante. “It will more likely be the company whose engineering system learns, validates and improves the entire platform faster than anyone else.”
This feature is part of SiliconANGLE Media’s exploration of the architectural shifts powering continuous, production-grade AI. Be sure to check out theCUBE’s “Dell AI Data Platform Event: From Ambition to AI at Scale” and its past coverage of this year’s Dell Technologies World. (* Disclosure below.)
Rack-scale consolidation
A cornerstone of the Dell and AMD relationship is the server giant’s PowerEdge portfolio. Dell expanded its PowerEdge server line in May and has followed that up with additional details surrounding the integration of AMD processors.
The two companies have pursued a strategy designed to facilitate server consolidation as agentic AI drives a surge in demand for general-purpose compute. Agents rely heavily on CPUs, yet data centers don’t have a lot of extra room to add servers in rooms already stacked with GPUs.
Systems powered by AMD processors will deliver 70% gen-over-gen performance gains and can fold 13 servers from Dell’s 14th PowerEdge generation into one, according to Pradyumna Narendra, product planner and senior consultant at Dell.
“For every 10 to 13 servers you replace, you get so much of that productivity gain back because now you can do that with one server,” he told theCUBE. “When you expand that across data centers, that’s like an accelerating effect that you get to capitalize from.”
Dell and AMD must also accommodate a wealth of AI use cases and the accompanying demand for compute. Dell’s three-rack-unit server requires less power for cooling than a two-unit equivalent, allowing customers to pack more computing power into each rack, according to Narendra.
“By deploying servers that are slightly larger in package, you deliver more density at a rack level, which seems sometimes counterintuitive,” he explained. “As AI accelerates, inferencing and agentic AI around it will accelerate and grow faster. As these use cases scale out, you get exponential growth in the need for compute.”
Modular deployment and validation
Beyond server consolidation, Dell and AMD have been steadily enhancing a modular solution to simplify AI deployment and scaling: the Dell AI Platform with AMD.
The on-premises enterprise AI solution integrates AMD Instinct accelerators and EPYC processors with Dell PowerEdge servers, PowerSwitch networking and PowerScale storage for model training, fine-tuning and high-throughput inference. The platform is designed to let customers begin with configurations that meet current workloads and expand compute and GPU density over time, without rearchitecting.
“What we’re finding is customers want modular solutions. They want to think about [solutions] across the whole platform,” said Varun Chhabra, senior vice president of product marketing and infrastructure solutions group at Dell, in a discussion with theCUBE. “Compute, storage, networking [and] GPU. The software framework on top of it, the models: Have they all been tested? Have they all been validated?”
Dell has integrated the AMD ROCm software stack into the platform to support training, fine-tuning and serving large language models on-premises. The modular platform lets customers begin with smaller deployments and expand on the same infrastructure, according to Chhabra.
“You can start small with these composable units of storage, compute, networking, pre-integrated with AMD Instinct and EPYC CPUs with the ROCm software as well as the open ecosystem that sits on top of it,” he said. “We test it, we validate it, and the customers can use it and once they start seeing value in it, they know they can scale in a modular way with the same frameworks.”
Leveraging benefits of PCIe
Dell announced support for AMD Instinct MI350P PCI Express GPUs in its PowerEdge XE7745 and R7725 servers beginning in July, giving enterprises another option for scaling agentic and generative AI deployments. It also represents an important boost in memory bandwidth which matters in time-to-first-token latency.
The MI350P offers 144 gigabytes of HBM3E memory, which can be a key differentiator for inference performance, according to Mike Darby, senior manager of business development, Instinct Data Center GPUs, at AMD, during an appearance on theCUBE.
“It’s a lot of memory, but it’s also HBM3E, so [it has] an amazing amount of bandwidth,” he said. “The main use case being inference: It’s all about that latency, time to first token [and] the user experience, so that your attention span doesn’t move somewhere else while you’re waiting for your response.”
AMD and Dell’s collaboration on PCIe integration has another angle worth watching. AMD’s PCIe-based GPU supports air-cooled deployments, providing users with a path to deploy AI within existing data center infrastructure.
As the tech industry has encountered resistance in recent months to the buildout of large data centers across the U.S., concerns have centered on the environmental impact and use of water-based cooling systems. Separately, Dell found that, as it deployed its next-generation PowerEdge servers, some enterprise customers were still not prepared to adopt liquid cooling within data centers.
“Working with Dell, we added a PCI card to our air-coolable roadmap for GPU Instinct so that it fit in more Dell PowerEdge servers that go to the broader enterprise,” said Robert Hormuth, corporate vice president of architecture and strategy for the Data Center Solutions Group at AMD, during a conversation with theCUBE. “Not everybody is adopting direct liquid cooling.”
Collaboration in an open market
Dell’s work with AMD this year has extended beyond joint product announcements. In June, the two companies announced a partnership with the University of Cambridge to establish a Sovereign AI Innovation Lab (SAIL) in the United Kingdom.
The collaboration is designed to help advance the next generation of AI infrastructure and AI-powered scientific breakthroughs. It builds on Cambridge’s expanded AI research infrastructure, including the Zenith AI supercomputer, powered by fifth-generation AMD EPYC processors and AMD Instinct MI355X GPU accelerators integrated into Dell infrastructure.
Dell and AMD have also collaborated in recent months on a telecom initiative with AT&T Inc. Telecom data is a heavy lift when it comes to AI, with AT&T having processed over one trillion tokens to train models. AMD has provided the high-performance hardware foundation for both training and inference workloads involving the OTel 2.0 models developed through the Open Telco AI initiative.
Dell integrated this technology with AI-optimized servers for telecom environments to create a production-grade foundation for operators across data-intensive, real-world network environments. OTel 2.0’s training infrastructure uses Dell’s PowerEdge XE9785 server configured with eight MI355X GPUs.
The alliance between Dell and AMD highlights a key development in the growing AI infrastructure market: There are plenty of opportunities for competition. Both Dell and AMD appear committed to building portfolios that offer enterprise choices in AI processing solutions.
“AMD didn’t present its new products … as ‘respectable alternatives,’” Vellante said. “It claimed outright leadership across CPUs, GPUs, rack-scale performance, memory capacity, network bandwidth and tokens per dollar. It extended the argument across enterprise AI, robotics, ultra-low-latency inference and a rapidly expanding partner ecosystem.”
(* Disclosure: TheCUBE is a paid media partner for the Dell AI Data Platform Event. Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
Image: SiliconANGLE/ChatGPT
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.