NVIDIA拥有发布 Nemotron 3.5 Trancis，一款为抄写而建的开放权重语音LLM. 一个型号处理40种语言，录音时间可长达1小时。 文本指令选择每个请求的输出样式。 选项包括逐字记录、书面形式、发言者属性、背景和翻译产出。 对于开发者来说，它可以取代单独的ASR，分化和对齐工具。
TL; DR 数据
- 大小：共34B个，约4.3B活性参数；65 536个上下文；音频可长达1小时。
- 运行在：1x H100 80 GB在BF16. FP8(约35 GB)和NVFP4(约21 GB)检查站；NVFP4运行在RTX 5090上。 GGUF文件目标为NeMo-Speech.cpp.
- 性能：英语和多语种开放ASR Leaderboards上的最佳开放权重模型；顶级商业API仍然领先。
- 最佳：4.71%在9个公开的英语开放ASR集(下一个开放型号：4.94%)上表示WER.
- 最糟的是：34.95%的cpCER在Ali Meeting 普通话远地会议上。
- 底线：最好的事情：1个开放型号覆盖了文字记录，扬声器标签，时间戳和翻译。 最糟糕的事情：只下线，普通话会议准确性差。
什么是尼莫特龙3.5转写？
尼莫特龙 3.5 Translax是来自NVIDIA的语音增强语言模型(SALM)，将音频和指令转化为文本。 一个音频编码器将嵌入到LLM解码器中，这使得转录可以被迅速。 NVIDIA的技术研究报告认为，没有一份正确的记录。 数字，填充器和域名词取决于谁读取输出。 NVIDIA将模型定位为抄写专家，而非通用助理。
建筑如何运作？
它有三个部分：
- 平行专家编码器：一个可列车的0.6B变形器ASR编码器在冷冻的尼莫特龙3 Diarization模型旁运行。 平面仪追踪到8个喇叭它的演讲活动每80分就加入ASR的特色。 LLM因此看到谁说话的时间，没有架构变化。
- LLM骨干：Nemotron 3.5闪电30B-A3B，一种混合马巴-Transformer MoE. 它有52个地层：23个曼巴-2,23个MOE，只有6个关注地层。 教育部的每个层都通往128名专家中的6名，加上1名共同的专家。 很少的注意层将 KV 缓存保持为小的小时音频。
- 添加：用于投机解码的共享多键预测层草稿。 50M参数的CTC适配器将完成的文字记录与音频对齐，用于文字时间戳。
型号名称为称 30B-A4B，但检查站持有约34B个参数。 额外重量来自1.3B MTP模块和音频组件。
在256个H100GPU上使用1个受监督的微调级的训练，共28,500步。 随后是短暂的RL阶段：400个GSPO更新，奖励为1减WER. 该型号看到了大约1.72M独特时段的音频，或6.5M时段有重复。
基准测试的得分如何？.
以下所有数字由NVIDIA在技术报告中自报。
- 英文开放ASR(9套公开版):4.71%表示WER. 下届公开赛车型为Qwen3-ASR-1.7B，得分4.94%. 7个商业系统仍然领先，被OtterAI speech-v3.1所控制，占3.88%.
- 多语言开放ASR(7种语言):4.62%为WER，而Qwen3-ASR-1.7B为6.49%。ElevenLabs Scribe v2达到3.56%。
- 长式ASR:10.98%为WER,Parakeet-TDT-0.6B-v3为13.37%。
- 多语种，全会话：15.80%为CpWER,16.83%为DiCoW v3.3与DiariZen。 NVIDIA 称是这里第一个击败级联系统的SALM.
- 背景 ASR：一个实体在背景ASR-Bench上列出削减命名实体WER，从12.21%到2.20%.
你怎么部署尼莫特龙3.5转写器？
BF16,FP8,NVFP4和GGUF的重量船。 一个 DFlash2 草案模型是可选的。 OpenMDW-1.1许可证允许商业使用。
运行时间包括vLLM 0.32.0，并配有 NeMo Speech 插件，变形器和 SGLang-Omni 02.0. vLLM 曝光相容的 OpenAI API.
NVIDIA型号的主要速度来自其Nemotron Trancis NIM. 与NVFP4和DFlash2, 1 RTX PRO 6000 Blackwell Server Edition转录了每秒2,238秒的音频。 它在0.13秒后返回了30秒的剪辑。 有个发现：那些服务优化的还没有开源。 NVIDIA预计发布日 vLLM 路径大约会慢2x. 在用MTP2的BF16中的1H100上，NVIDIA估计约有410个RTFx.
对于微调，NVIDIA测试了17个任务。 每架单相机用上约0.5至2.3GPU小时。
与其他开放的ASR模型如何比较？
*竞争对手在目前9集主板上的得分来自NVIDIA的报告，引用了从2026年10月起的Hugging Face主板结果。 领跑板设定自竞争对手自发卡后有所改变，因此2个WER行无法互换。
关键外卖
- 一个开放模式处理40种语文的文字记录、演讲者、时间戳和翻译。
- 4.71%的英语WER是最佳开放权重分数；商业API仍然领先。
- 大约4.3B活性参数；BF16装入1H100,NVFP4装入RTX 5090.
- 一个实体列出削减命名实体WER从12.21%到2.20%.
- 只下线；普通话会议和密码交换仍然是薄弱环节。
请检查access-date=中的日期值 (帮助) 技术报告，模型权重，项目页面和现场演示。 并加入我们的通讯。 等等，你在电报里吗？现在你也可以加入我们的电报。
需要和我们合作推广您的 GitHub Repo OR Hugging Face Page OR Product Release OR Webinar等吗？联系我们
Asif Razzaq是Marktechpost AI Media Inc.的首席执行官。 作为一个有远见的企业家和工程师，阿西夫致力于利用人工智能的潜力为社会造福。 他最近的努力是人工智能媒体平台的发布，该平台以深入报道机器学习和深层学习新闻而出名，这些新闻在技术上既可靠又为广大观众所容易理解。 该平台每月有超过200万次观看，表明其受观众欢迎。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
NVIDIA has released Nemotron 3.5 Transcribe, an open-weights speech LLM built for transcription. One model handles 40 languages and recordings up to 1 hour. A text instruction picks the output style for each request. Options include verbatim, written form, speaker-attributed, contextual and translated output. For developers, it can replace separate ASR, diarization and alignment tools.
TL;DR
- Size: 34B total and about 4.3B active parameters; 65,536-token context; audio up to 1 hour.
- Runs on: 1x H100 80 GB in BF16. FP8 (about 35 GB) and NVFP4 (about 21 GB) checkpoints exist; NVFP4 runs on an RTX 5090. GGUF files target NeMo-Speech.cpp.
- Performance: Best open-weights model on the English and multilingual Open ASR Leaderboards; top commercial APIs still lead.
- Best: 4.71% mean WER on the 9 public English Open ASR sets (next open model: 4.94%).
- Worst: 34.95% cpCER on AliMeeting Mandarin far-field meetings.
- Bottom line: Best thing: 1 open model covers transcripts, speaker labels, timestamps and translation. Worst thing: offline only, with weak Mandarin meeting accuracy.
What is Nemotron 3.5 Transcribe?
Nemotron 3.5 Transcribe is a speech-augmented language model (SALM) from NVIDIA that turns audio and instructions into text. An audio encoder feeds embeddings into an LLM decoder, which makes transcription promptable. NVIDIA’s technical research report argues there is no single correct transcript. Numbers, fillers and domain terms depend on who reads the output. NVIDIA positions the model as a transcription specialist, not a general-purpose assistant.
How does the architecture work?
It has three parts:
- Parallel Expert Encoder: a trainable 0.6B Transformer ASR encoder runs beside a frozen Nemotron 3 Diarization model. The diarizer tracks up to 8 speakers. Its speaker activity is added to the ASR features every 80 ms. The LLM therefore sees who spoke when, without architecture changes.
- LLM backbone: Nemotron 3.5 Lightning 30B-A3B, a hybrid Mamba-Transformer MoE. It has 52 layers: 23 Mamba-2, 23 MoE and only 6 attention layers. Each MoE layer routes to 6 of 128 experts plus 1 shared expert. Few attention layers keep the KV cache small for hour-long audio.
- Add-ons: a shared multi-token prediction (MTP) layer drafts tokens for speculative decoding. A 50M-parameter CTC adapter aligns the finished transcript to audio for word timestamps.
The model name says 30B-A4B, but the checkpoint holds about 34B parameters. The extra weight comes from the 1.3B MTP module and the audio components.
Training used 1 supervised fine-tuning stage on 256 H100 GPUs for 28,500 steps. A short RL stage followed: 400 GSPO updates with a reward of 1 minus WER. The model saw about 1.72M unique hours of audio, or 6.5M hours with repeats.
How does it score on benchmarks?
All numbers below are self-reported by NVIDIA in the technical report.
- English Open ASR (9 public sets): 4.71% mean WER. The next open model, Qwen3-ASR-1.7B, scores 4.94%. Seven commercial systems still lead, topped by OtterAI speech-v3.1 at 3.88%.
- Multilingual Open ASR (7 languages): 4.62% mean WER, versus 6.49% for Qwen3-ASR-1.7B. ElevenLabs Scribe v2 reaches 3.56%.
- Long-form ASR: 10.98% mean WER, versus 13.37% for Parakeet-TDT-0.6B-v3.
- Multi-speaker, full sessions: 15.80% mean cpWER, versus 16.83% for DiCoW v3.3 with DiariZen. NVIDIA says it is the first SALM to beat cascaded systems here.
- Contextual ASR: an entity list cuts named-entity WER on ContextASR-Bench from 12.21% to 2.20%.
How do you deploy Nemotron 3.5 Transcribe?
Weights ship in BF16, FP8, NVFP4 and GGUF. A DFlash2 draft model is optional. The OpenMDW-1.1 license allows commercial use.
Runtimes include vLLM 0.32.0 with the NeMo Speech plugin, Transformers and SGLang-Omni 0.2.0. vLLM exposes an OpenAI-compatible API.
NVIDIA model’s main speed comes from its Nemotron Transcribe NIM. With NVFP4 and DFlash2, 1 RTX PRO 6000 Blackwell Server Edition transcribes 2,238 seconds of audio per second. It returns a 30-second clip in 0.13 seconds. There is a catch: those serving optimizations are not open source yet. NVIDIA expects the release-day vLLM path to be roughly 2x slower. On 1 H100 in BF16 with MTP2, NVIDIA estimates about 410 RTFx.
For fine-tuning, NVIDIA tested 17 tasks. Each took about 0.5 to 2.3 GPU-hours on a single GB300.
How does it compare with other open ASR models?
*Competitor scores on the current 9-set leaderboard come from NVIDIA’s report, citing Hugging Face leaderboard results from October 2026. The leaderboard set changed since the competitors published their own cards, so the 2 WER rows are not interchangeable.
Key Takeaways
- One open model handles transcripts, speakers, timestamps and translation across 40 languages.
- 4.71% English WER is the best open-weights score; commercial APIs still lead.
- About 4.3B active parameters; BF16 fits 1 H100, NVFP4 fits an RTX 5090.
- An entity list cuts named-entity WER from 12.21% to 2.20%.
- Offline only; Mandarin meetings and code-switching remain weak spots.
Check out the technical report, model weights, project page and live demo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Asif Razzaq is the CEO of Marktechpost AI Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.