OCT. 8, 2026 (英语).
Google AI Edge Team兴奋地宣布了开源发布ML Drift，我们的高性能，跨平台，端侧 GPU 计算引擎专门为端侧 AI/ML推论所制造，在Apache 2.0许可下。 通过抽象硬件以及跨越OpenGL ES,OpenCL,Metal，和WebGPU的端侧 GPU的低等API复杂性，ML Drift授权开发者构建实时，从高级视频效果到基因AI的交互ML体验跨越多个平台。 作为LiteRT内部的核心GPU加速引擎，ML Drift也作为自定义图形和推论运行时间的独立库，为各地提供峰值性能提供统一的基础。
ML Drift 跨平台的 GPU API 支持矩阵
与数据中心推论不同的是，模型运行在同质，可预见加速器集群上，将GPU-加速AI部署到边缘设备是由显著的硬件多样性定义的。 开发者面临着广泛的GPU架构，驱动程序版本，以及低等API，不知道应用程序将运行在哪个特定的硬件上。
TensorFlow Lite GPU代表建立了基础GPU加速，但生态系统后来有所演变：目前的端侧工作量现在包含一个宽谱，从实时计算机视觉，音频，和深度处理模型到高参数基因AI. 这些建筑将消费者硅推向极限，制造出遗留的运行时间无法解决的计算和内存瓶颈。
这种转变需要有一个普遍的工程基础，以确保古典和下一代模型的可移植性和性能。 借鉴"Speed is all you need"中所讨论的优化原理，我们构建了一个单一，统一的框架，为传统的ML提供稳定性，为最先进的基因AI提供峰值性能，确保了你的模型运行在我们最高效，最前瞻性的GPU计算引擎上。
为了实现峰值性能和扩大古典和基因结构的模型覆盖，ML Drift 推出核心建筑的改变和结构升级都超越了遗留的TFLite GPU代表。
通过Tensor虚拟化实现的统一阴影：历史上，为TFLite GPU代表维护优化的阴影，需要将硬编码逻辑拉伸绘图直接到物理GPU对象(如纹理和缓冲)上，分别跨越OpenGL,OpenCL，和金属后端。 ML Drift 推出收音机的收音机虚拟化，一个核心建筑范式，将收音机的逻辑表达与它在GPU上的物理分配脱钩。 通过在编译器初始化阶段拥有动态的遮荫器模板来解析和翻译坐标，这种统一的遮荫器模型消除了保持相去甚远的后端特有遮荫器代码库的需要，在保留跨平台模型可移植性的同时增加了最小的运行时间。
广泛自定义Op框架：ML Drift采用现代化自定义Op框架，提供低等阴影语言访问的直接注册API. 为加快速度，ML Drift包含一个代理 SKILL.md 向导，允许编码 agents 在分钟内编写，注册，并验证表演自定义阴影。 这为开发者提供了颗粒控制，可以直接将专门的模型块集成到执行图中，大大地减少了部署专有架构的障碍。
5D Tensor Support:TFLite GPU代表机在结构上被硬码为4D分机，迫使开发者对需要5D分机的复杂模型使用布局黑客。 ML Drift在新的LiteRT ML Drift GPU加速器中反映了这个已逾期很久的功能，以允许5D的收发式支持。 以此为例，LiteRT执行3D等复杂工作量，用于量子/空间AI和spatiopal模型(例如：YOLO 11n,MobileViT v2和Swin Transformer v2)，直接在边缘GPU上。
经典模型性能升级：作为我们遗留下来的GPU后端的直接接班人，ML Drift为现有的古典工作量提供了严格的性能升级。 通过将运行时间与硬件特定限制脱钩，提高内核执行效率，在TFLite GPU上已经运行的模型获得即时性能改进。 移徙的设计是直截了当的，遗留的工作量维持或超过以前的基准测试。
相位感知优化 Edge LLMs：相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相位相相位相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相相货币根据活性执行阶段，漂移动态切换内核和布局配置。 在解码阶段，它使用自定义，相接的KV缓存布局，并应用有侵略性的内核活化量子化来绕过冗余内存回转。
Prefill 512，解码128，上下文长度为640,4-位重(块大小为32),fp16 KV缓存，argmax采样，同步每个令牌。
预填(固条；左取出地块)和解码(厚条；右取出地块)的吞吐。
Gemma4 E2B在支持交叉KV共享的框架上表现较好。
扩展跨平台( Desktop Previews)：在当今新兴的代理编码时代，高度响应的桌面推论正变得对为本地编码agents和工作站工作流程提供动力至关重要。 虽然我们最初为浏览器内加速设计了ML Drift的WebGPU后端，但Dawn(Chrimium的WebGPU实施)却允许我们在浏览器外编译出完全相同的编码库。 这个统一的API模型使得ML Drift能够运行在Windows和Linux上，利用WebGPU的现代，本土硬件抽象来绕过DirectX和Vulkan分解，同时补充了我们现有的高性能本土金属在macOS上的后端。
我们的主要重点仍然是在资源限制最紧的地方提供轻量级生产推断：移动和边缘设备。 然而，地方发展需要各开发商机器的灵活性。 下面是ML Drift在工作站硬件上运行的Gemma模型的早期快照，说明我们的统一运行时间尺度如何跨越环境。
桌面 LLM 性能。 预填 8192，解码 1024，上下文长度 9216
4-位重(块大小为32),fp16 KV缓存，正格马克斯采样，同步每个令牌。
跨越边缘和工作站环境，高峰吞吐量只讲述了一半的故事：记忆足迹决定着哪些工作量可以同时运行。 在我们的Gemma 基准测试中，ML Drift实现了比其他框架更低12%的内存管理费，将内存释放用于其他本地任务。
ML Drift已经每天在数百万个设备上运行，为Google生态系统的关键功能提供动力，包括Chrome,YouTube Shorts，相片，和Meet. 在前沿，它充当了先行端侧能力的运行时间，为Google AI Edge apps提供动力，如 Gallery, Foresight，和 Eloatent.
YouTube Shorts迁移出基于分块的效应，使用ML Drift. 这在Android和iOS都实现了平均帧延缓40%的降低，使创造者能够在不降相机帧的情况下实时应用高真能视觉效果。
为了提供快速，无缝的端侧相片编辑，Google相片团队将ML Drift集成在其计算摄影和分片管上。 与遗留下来的GPU代表相比，这提供了长达2秒的加速，使先进的照片增强感觉瞬间。
Google 通过ML漂流加速相片Unblur
Google Chrome将ML Drift整合到其AI运行时基础设施中，为Built-In AI APIs(包括即时，即时和作家API)的本地硬件加速Gemini Nano模型提供动力。 通过桌面平台，ML Drift可以实现高效益的端侧 AI，跨越从电子商务审查汇总到企业客户工作流程的网络应用不断增长的生态系统：
Cafe24通过ML Drift使用即时API来加速标记生成。
除了Google产品之外，主要开发方合作伙伴也采用了ML Drift，为移动用户带来桌面等级能力：
Adobe灯光室和Adobe Photoshop升级了关键AI功能，包括Select Object,Select Sky，以及可适应的外观，并配有ML Drift，在移动上实现最快达到30%的端侧性能，用于支持质量的相片编辑。
Adobe 灯光室通过移动 LiteRT ML Drift GPU 加速器运行端侧
Snap 集成LiteRT为动力，通过Android设备对GPU进行端侧推论。 ML Drift GPU加速器为Snapchat镜头中的ML动力面部和样式生成器效果带来了30%的模型延迟改进。 货币漂流不仅可以改善基于现有模型的用户体验，还可以提高扩展运行甚至大型扩散模型的能力。
使边缘AI效率最大化需要软件架构和物理硅之间的深度对接。 在整个ML Drift开发过程中，Google与硅和IP伙伴密切合作，实现优化，使硬件在现代GPU中得到最大利用：
- 臂：协同优化马里和Immortalis GPU的计算内核和纹理缓存位置，确保高通量、高功率的推论跨移动设备。
- Intel:ML Drift利用了本地的WebGPU计算和Xe矩阵扩展(XMX)来加速客户端LLM和用Xe3图形的Intel® CoreTM Ultra处理器的视觉工作量。
- Qualcomm Technologies, Inc.:ML Drift优化了最高峰ALU和内存带宽利用的OpenCL内核，使Qualcomm-AdrenoTM GPU上的有效边缘AI加速成为可能。
ML Drift通过与硅相接，确保了最先进模型优化转化为电池意识，高帧在用户设备上的性能。
随着ML Drift的发布，遗留下来的TFLite GPU代表将不再收到新的功能更新。 我们鼓励所有开发者向 LiteRT ML Drift GPU 加速器过渡，它为您现有的模型提供完全倒向相容性，同时立即解锁跨越Android,iOS,Web，和桌面的现代性能收益。 对于Android开发者使用未捆绑的运行时间来将app 二进制大小最小化，ML Drift加速在今天以独立的LiteRT包提供，并很快会来到Google Play Services的LiteRT.
无论您正在将端侧模型集成到一个应用程序中，还是正在编写自定义的 GPU 阴影，ML Drift 提供了两种直通路径：
- 对于应用程序开发者：使用LitERT ML Drift加速器取出箱外模式分区，收音机虚拟化，并优化了LLM加速。
- 对于GPU & System Engineers：通过ML Drift GitHub寄存器直接用ML Drift的独立的C++ API来构建自定义的"GpuModel"计算图表并写入自定义的遮蔽器(通过 Dawn快速启动开始在Android和WebGPU上使用我们的OpenCL).
社区反馈和贡献
在Apache 2.0许可下，ML Drift完全为开源。 我们欢迎各方的贡献、区域渔业委员会和社区反馈：
- 来源代码：github.com/google-ai-edge/ml-drift
- 问题跟踪器和特性请求：在 GitHub 上文件一个问题
- 讨论 & QQA: Google AI边缘社区论坛
- 贡献：审查我们的贡献指南，以添加新的UCL操作器或硬件特定优化。
承认：
安德烈·库利克，安基特·高雅尔，鲍勃·刘，金秉哲，陈冠英，长明孙，克里斯·麦克拉纳汉，克拉克·杜瓦尔，科马克·布里克，丹尼尔·何，大卫·内多，爱卡捷琳娜·伊格纳谢瓦，冯武·姚，弗兰克·班，杰拉尔多·卡兰扎，格兰特·詹森，雅各布·多尔穆特，贾加德什·帕卡拉沃尔，詹姆斯·普莱斯，杰里米·克姆普克，静江·利，唐继强，开尔文·马，肯·罗素，肯吉·巴赫克，林·陈克，罗科·孔克，卢旺，玛丽萨·伊科诺米迪斯，马特·克莱德尔，马蒂亚斯·格伦德曼，马蒂亚斯·西蒙松，帕韦瓦·安德鲁什基维奇，拉曼·萨罗金，罗马里克·乔丁，萨钦·克特瓦尼，索姆达特克达特
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
OCT. 8, 2026
The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML inference, under the Apache 2.0 license. By abstracting hardware and low-level API complexities of on-device GPUs across OpenGL ES, OpenCL, Metal, and WebGPU, ML Drift empowers developers to build real-time, interactive ML experiences from advanced video effects to generative AI across multiple platforms. Serving as the core GPU acceleration engine within LiteRT, ML Drift is also available as a standalone library for custom graphics and inference runtimes providing a unified foundation to deliver peak performance everywhere.
ML Drift's GPU API support matrix across platforms
Unlike datacenter inference, where models run on homogeneous, predictable accelerator clusters, deploying GPU-accelerated AI to edge devices is defined by significant hardware diversity. Developers are faced with a wide range of GPU architectures, driver versions, and low-level APIs, not knowing a priori on which specific hardware the application will run on.
The TensorFlow Lite GPU delegate established foundational GPU acceleration, but the ecosystem has since evolved: current on-device workloads now encompass a broad spectrum, ranging from real-time computer vision, audio, and depth processing models to high-parameter generative AI. These architectures push consumer silicon to its limits, creating compute and memory bottlenecks that legacy runtimes weren't designed to solve.
This shift necessitates a universal engineering foundation of ensuring portability and performance across both classical and next-generation models. Drawing on optimization principles discussed in "Speed is all you need", we've built a single, unified framework that delivers stability for traditional ML and peak performance for the most advanced generative AI, ensuring that your models are running on our most efficient, forward-looking GPU compute engine.
To achieve both peak performance and expanded model coverage across classical and generative architectures, ML Drift introduces core architectural changes and structural upgrades over the legacy TFLite GPU delegate.
Unified Shaders via Tensor Virtualization: Historically, maintaining optimized shaders for the TFLite GPU delegate required hardcoding logical tensor mappings directly to physical GPU objects (such as textures and buffers) separately across OpenGL, OpenCL, and Metal backends. ML Drift introduces tensor virtualization, a core architectural paradigm that decouples a tensor's logical representation from its physical allocation on the GPU. By having dynamic shader templates resolve and translate coordinates during the compiler’s initialization phase, this unified shader model eliminates the need to maintain disparate, backend-specific shader codebases, adding minimal runtime overhead while preserving cross-platform model portability.
Extensible Custom Op Framework: ML Drift adopts a modernized custom op framework that provides direct registration APIs with low-level shading language access. To accelerate this, ML Drift includes an agentic SKILL.md guide, enabling coding agents to author, register, and verify performant custom shaders in minutes. This provides developers with granular control to integrate specialized model blocks directly into the execution graph, drastically reducing the barrier to deploying proprietary architectures.
5D Tensor Support: The TFLite GPU delegate is structurally hardcoded to 4D tensors, forcing developers to use layout hacks for complex models requiring 5D tensors. ML Drift reflected this long overdue functionality in the new LiteRT ML Drift GPU accelerator to enable 5D tensor support out-of-the-box. With this, LiteRT executes complex workloads like 3D convolutional networks for volumetric/spatial AI and spatiotemporal models (e.g., YOLO 11n, MobileViT v2, and Swin Transformer v2) directly on edge GPUs.
Performance Upgrades for Classical Models: As the direct successor to our legacy GPU backend, ML Drift provides a strict performance upgrade for existing classical workloads. By decoupling the runtime from hardware-specific constraints and improving kernel execution efficiency, models that were already running on TFLite GPU gain immediate performance improvements. Migration is designed to be straightforward, with legacy workloads maintaining or exceeding previous benchmarks.
Stage-aware Optimizations for Edge LLMs: Autoregressive LLMs have two different computational workloads during inference: the compute-bound KV cache prefill stage and the memory-bandwidth-bound decode stage, generating tokens one by one. ML Drift dynamically switches kernels and layout configurations depending on the active execution phase. During the decoding stage, it utilizes a custom, convolution-aligned KV cache layout and applies aggressive in-kernel activation quantization to bypass redundant memory roundtrips.
Prefill 512, decode 128, context length 640, 4-bit weights (block size 32), fp16 KV cache, argmax sampling, sync every token.
Prefill (solid bars; left ord.) and decode (hollow bars; right ord.) throughputs are presented.
Gemma4 E2B performs better on frameworks supporting cross KV sharing.
Broadening across Platforms (Desktop Previews): In today’s emerging agentic coding era, highly responsive desktop inference is becoming vital for powering local coding agents and workstation workflows. While we originally engineered ML Drift’s WebGPU backend for in-browser acceleration, Dawn (Chromium's WebGPU implementation) allowed us to compile that exact same codebase natively outside the browser. This unified API model enables ML Drift to run on Windows and Linux, leveraging WebGPU's modern, native hardware abstraction to bypass DirectX and Vulkan fragmentation, while complementing our existing high-performance native Metal backend on macOS.
Our primary focus remains delivering lightweight, production-grade inference where resource constraints are tightest: mobile and edge devices. However, local development requires flexibility across developer machines. Below is an early snapshot of ML Drift running Gemma models on workstation hardware, illustrating how our unified runtime scales across environments.
Desktop LLM performance. Prefill 8192, decode 1024, context length 9216,
4-bit weights (block size 32), fp16 KV cache, argmax sampling, sync every token.
Across edge and workstation environments alike, peak throughput only tells half the story: Memory footprint dictates what workloads can actually run concurrently. In our Gemma benchmarks, ML Drift achieves up to 12% lower memory overhead than other frameworks, freeing up memory for other local tasks.
ML Drift is already running in production across millions of devices daily, powering key features across Google's ecosystem, including Chrome, YouTube Shorts, Photos, and Meet. At the frontier, it acts as the runtime for pioneering on-device capabilities, powering Google AI Edge apps like Gallery, Foresight, and Eloquent.
YouTube Shorts migrated its segmentation-based effects to use ML Drift. This delivered up to a 40% reduction in average frame latency across both Android and iOS, enabling creators to apply high-fidelity visual effects in real time without dropping camera frames.
To deliver fast, seamless on-device photo editing, the Google Photos team integrated ML Drift across its computational photography and segmentation pipelines. This delivered up to 2 sec speedup compared to the legacy GPU delegate, making advanced photo enhancements feel instantaneous.
Google Photos Unblur accelerated via ML Drift
Google Chrome integrated ML Drift into its AI runtime infrastructure to power native hardware-accelerated Gemini Nano models for Built-In AI APIs (including the Prompt, Summarize, and Writer APIs). Across desktop platforms, ML Drift enables high-efficiency on-device AI across a growing ecosystem of web applications from e-commerce review summarization to enterprise customer workflows:
Cafe24 using Prompt API for tag generation accelerated via ML Drift.
Beyond Google products, leading developer partners have adopted ML Drift to bring desktop grade capabilities to mobile users:
Adobe Lightroom and Adobe Photoshop upgraded key AI features, including Select Subject, Select Sky, and Adaptive Portrait, with ML Drift, achieving up to 30% faster on-device performance for pro-quality photo editing on mobile.
Adobe Lightroom running on-device via mobile LiteRT ML Drift GPU accelerator
Snap integrated LiteRT to power on-device inference on GPU across Android devices. ML Drift GPU accelerator brought 30% model latency improvements for ML-powered face and style generator effects in Snapchat lenses. ML Drift not only improves user experiences based on existing models, but also expands capabilities to run even large diffusion models.
Maximizing edge AI efficiency requires deep alignment between software architectures and physical silicon. Throughout ML Drift's development, Google worked closely with silicon and IP partners to enable optimizations that maximize hardware utilization across modern GPUs:
- Arm: Co-optimized compute kernels and texture cache locality for Mali and Immortalis GPUs, ensuring high-throughput, power-efficient inference across mobile devices.
- Intel: ML Drift harnessed native WebGPU compute and Xe Matrix Extensions (XMX) to accelerate client LLM and vision workloads on Intel® Core™ Ultra processors with Xe3 graphics.
- Qualcomm Technologies, Inc.: ML Drift optimized OpenCL kernels for peak ALU and memory bandwidth utilization, enabling efficient edge AI acceleration on Qualcomm® Adreno™ GPUs.
By working hand-in-hand with silicon leaders, ML Drift ensures that state-of-the-art model optimizations translate into battery-conscious, high-frame rate performance on user devices.
With the launch of ML Drift, the legacy TFLite GPU delegate will no longer receive new feature updates. We encourage all developers to transition to the LiteRT ML Drift GPU accelerator which offers full backwards compatibility for your existing models while immediately unlocking modern performance gains across Android, iOS, Web, and desktop. For Android developers using unbundled runtimes to minimize app binary size, ML Drift acceleration is available today in standalone LiteRT packages and is coming soon to LiteRT in Google Play Services.
Whether you are integrating on-device models into an app or authoring custom GPU shaders, ML Drift provides two straightforward paths:
- For Application Developers: Use the LiteRT ML Drift accelerator to get out-of-the-box model partitioning, tensor virtualization, and optimized LLM acceleration.
- For GPU & System Engineers: Build directly with ML Drift's standalone C++ APIs via the ML Drift GitHub repository to construct custom `GpuModel` compute graphs and write custom shaders (get started with our OpenCL on Android and WebGPU via Dawn quickstarts).
Community feedback and Contributions
ML Drift is fully open source under the Apache 2.0 license. We welcome contributions, RFCs, and community feedback:
- Source Code: github.com/google-ai-edge/ml-drift
- Issue Tracker & Feature Requests: File an Issue on GitHub
- Discussions & Q&A: Google AI Edge Community Forum
- Contributing: Review our Contributing Guide to add new UCL operators or hardware-specific optimizations.
Acknowledgments:
Andrei Kulik, Ankit Goyal, Bob Liu, Byungchul Kim, Champ Yen, Changming Sun, Chris McClanahan, Clark Duvall, Cormac Brick, Daniel Ho, David Neto, Ekaterina Ignasheva, Fengwu Yao, Frank Ban, Gerardo Carranza, Grant Jensen, Jacob Dormuth, Jagadeesh Pakaravoor, James Price, Jeremy Kemp, Jingjiang Li, Jiuqiang Tang, Kelvin Ma, Ken Russell, Kenji Baheux, Lin Chen, Loko Kung, Lu Wang, Marissa Ikonomidis, Matt Kreileder, Matthias Grundmann, Mattias Simonsson, Paweł Andruszkiewicz, Raman Sarokin, Romaric Jodin, Sachin Kotwani, Somdat
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.