Nvidia的开放尼莫特龙3.5 译名结合了抄录、翻译和发言者的表彰
关键点
- Nvidia有发布尼莫特龙3.5 Transis，一种遵循文本命令并结合了最多可达40种语言的转录，翻译，和扬声器识别的开放式语音模型。
- 该模型有300亿个参数，但每步只有40亿个活动，所以运行在一个单一的GPU上。 它的排名接近基准测试的顶部，以获得准确性和扬声器归属。
- Nvidia还发布Granary v2，一个有84.4万小时音频的数据集，以及高效地使模型适应医学等专业领域的工具。
Nvidia拥有发布的可公开语音识别模式取出文本指令，而公司称则领导了多个基准测试。 该版还包括一个大型的培训数据集和工具来使模型适应专门领域。
简单的笔录不会告诉你称是什么它也不会翻译出任何东西或拼写技术术语。 尼莫特龙 3.5 Translax是用来用一个单一的模型来处理所有这些问题的，它也可以回答关于录音内容的问题。 Nvidia 称该模型转录了40种语言，将其中的31种语言译成或从英语翻译，并用13种语言识别出扬声器。 它处理长达一小时的录音。 他们最多可以包括8名发言者。
用户用简单的指令控制模型。 视速率而定，同一会议录音可成为带有发言者标签的记录稿、英文译文或所作决定的摘要。 用户还可以通过诸如药物名等技术术语来作为提示。 模型只有在录音中实际出现时才应该使用。
单个 GPU 的近端精度
该模型有300亿个参数，但每个处理步骤只有约40亿个活跃。 因此，它运行在单一的Nvidia GPU上。 这涵盖了从紧凑DGX Spark到数据中心硬件的所有内容。 Nvidia 称该模型在一款RTX PRO 6000 Blackwell服务器版上以NVFP4精度在三分钟内处理100小时音频。
在Hugging Face的英语Open ASR Leaderboard上，Transtics在开放量级模型中排名第一，平均字差率为5.33%. 在人工分析的比较中，它得分是2.81%. 这接近最佳的开放模型 Voxtral Small 以2.77%的成绩，但Trancis只使用大约六分之一的活性参数。 Nvidia表示，后面有几条被广泛使用的商业语音服务线索。
领先者在演讲者识别方面是最大的。 VoiceArena测试了14个对话系统，发言者有时相互交谈，Trancis语排在第一位。 也少了几个字，少了8.32%，而中位数约为31%。 在大卫AI的基准测试上，它测量了21种语言的演讲者归属，它贴出最低出错率。 它在Nvidia自己专为任务设计的模型之前刚刚完成。
开放的数据集允许开发者为自己的领域提供培训
除了模型之外，Nvidia正在发布Granary v2数据集。 它包含约84.4万小时的48种语言的语音，比上个版本的约64.3万小时的25种语言有所上升。 每张录音都呈现出几种抄录风格。 一个版本是逐字，有"uh"等填词和拼出数字。 另一个被清理了，没有填充器和数字，而不是用文字。 Nvidia也正在分享它用来构建数据集的工具，因此开发者可以在自己的录音上运行。 称公司的数据集虽然并不完全符合模型的训练数据。
Nvidia测试的目标是微调对印度医生和医科学生的录音。 在完整的基准测试上，更精确的即时将出错率从30.8%降低到23.8%. 微调之后，该比率下降到了11.8%. 培训数据主要是合成语言。 对演讲者来说，这个模型以前没有听说过，Nvidia分别报告从22.44%下降到12.17%，相对改善45.8%. 这使得调整后的模型领先于开放的医学演讲模型，其得分达到13.79%. Nvidia 称全过程在单个Nvidia GB300上花费了约14个GPU小时。
专门化要付出代价一般英语口语的误差率略有上升，而医学模型在品牌药物上保持更准确。 Nvidia建议在构建自定义版本时至少预留100个真实，已核实的录音进行测试。
基础模型也有限度。 特征因语言而异，而Nvidia 称音频问答功能则被用英语培训。 该模型也仅为收录的音频而建。 Nvidia 称用户应当抽查姓名，扬声器更改，回答。
该模型和数据集位于Hugging Face上。 还有一个可以尝试的演示。 对于微调，Nvidia提供向导和扩展，让AI编码助理运行培训工作流程。
Nvidia直到最近才于9月份将专用的扬声器识别模型推出 Nemotron 3 Diarization. 它可以分辨出多达8个扬声器，甚至在直播中也可以分辨出，但是它需要像Nvidia's Parakeet这样的单独的模型来进行实际的抄录。 尼莫特龙 3.5 Transis既结合了工作，又结合了新增翻译和音频理解。 它不支持实时处理，虽然。 Nvidia已经将Granary数据集用于其多式Nemotron 3 Nano Omni。
竞争者正在确定不同的优先次序。 Mistral的Voxtral家族也结合了转录和音频理解。 随着近期的宣布 Voxtral 3，公司主要关注实时语音agents.
AI News Without the Hype – 由人类治愈
订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。
现在订阅
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Nvidia's open Nemotron 3.5 Transcribe combines transcription, translation, and speaker recognition
Key Points
- Nvidia has released Nemotron 3.5 Transcribe, an open speech model that follows text commands and combines transcription, translation, and speaker recognition for up to 40 languages.
- The model has 30 billion parameters, but only 4 billion are active per step, so it runs on a single GPU. It ranks near the top of benchmarks for accuracy and speaker attribution.
- Nvidia is also releasing Granary v2, a dataset with 844,000 hours of audio, along with tools for efficiently adapting the model to specialized fields like medicine.
Nvidia has released an open speech recognition model that takes text instructions, and the company says it leads several benchmarks. The release also includes a large training dataset and tools for adapting the model to specialized fields.
A plain transcript doesn't tell you who said what. It also won't translate anything or spell technical terms correctly. Nemotron 3.5 Transcribe is designed to handle all of that in a single model, and it can answer questions about a recording's content as well. Nvidia says the model transcribes 40 languages, translates 31 of them to or from English, and identifies speakers in 13. It handles recordings up to an hour long. They can include up to eight speakers.
Users control the model with simple instructions. Depending on the prompt, the same meeting recording can become a transcript with speaker labels, an English translation, or a summary of the decisions made. Users can also pass along technical terms like drug names as hints. The model is only supposed to use them if they actually appear in the recording.
Near-top accuracy on a single GPU
The model has 30 billion parameters, but only about 4 billion are active per processing step. As a result, it runs on a single Nvidia GPU. That covers everything from the compact DGX Spark to data center hardware. Nvidia says the model processes 100 hours of audio in under three minutes on an RTX PRO 6000 Blackwell Server Edition at NVFP4 precision.
On Hugging Face's English Open ASR Leaderboard, Transcribe ranks first among open-weight models with an average word error rate of 5.33 percent. In Artificial Analysis' comparison, it scores 2.81 percent. That's close to the best open model, Voxtral Small at 2.77 percent, but Transcribe uses only about a sixth of the active parameters. According to Nvidia, several widely used commercial speech services trail behind.
The lead is biggest in speaker recognition. VoiceArena tested 14 systems on conversations where speakers sometimes talk over each other, and Transcribe took first place. It also dropped the fewest words, missing 8.32 percent compared to a median of about 31 percent. On David AI's benchmark, which measures speaker attribution across 21 languages, it posted the lowest error rate. It finished just ahead of Nvidia's own dedicated model for the task.
An open dataset lets developers train for their own fields
Alongside the model, Nvidia is releasing the Granary v2 dataset. It contains about 844,000 hours of speech in 48 languages, up from roughly 643,000 hours in 25 languages in the previous version. Each recording comes in several transcription styles. One version is verbatim, with filler words like "uh" and spelled-out numbers. Another is cleaned up, with no fillers and digits instead of words. Nvidia is also sharing the tools it used to build the dataset, so developers can run them on their own recordings. The company says the dataset doesn't fully match the model's training data, though.
Nvidia tested targeted fine-tuning on recordings of Indian doctors and medical students. On the full benchmark, a more precise prompt cut the error rate from 30.8 to 23.8 percent. After fine-tuning, the rate fell to 11.8 percent. The training data was mostly synthetic speech. For speakers the model hadn't heard before, Nvidia separately reports a drop from 22.44 to 12.17 percent, a relative improvement of 45.8 percent. That put the adapted model ahead of an open medical speech model, which scored 13.79 percent. Nvidia says the whole process took about 14 GPU hours on a single Nvidia GB300.
The specialization comes at a cost. The error rate rose slightly on general English speech, and the medical model stayed more accurate on brand-name drugs. Nvidia recommends setting aside at least 100 real, verified recordings for testing when building custom versions.
The base model has limits too. Features vary by language, and Nvidia says the audio question-answering feature was trained in English. The model is also built only for recorded audio. Nvidia says users should spot-check names, speaker changes, and answers.
The model and dataset are on Hugging Face. There's also a demo you can try. For fine-tuning, Nvidia offers guides and an extension that lets an AI coding assistant run the training workflow.
Nvidia only recently introduced Nemotron 3 Diarization, a dedicated speaker recognition model, in September. It can tell apart up to eight speakers, even in live conversations, but it needed a separate model like Nvidia's Parakeet for the actual transcription. Nemotron 3.5 Transcribe combines both jobs and adds translation and audio understanding. It doesn't support real-time processing, though. Nvidia had already used the Granary dataset for its multimodal Nemotron 3 Nano Omni.
Competitors are setting different priorities. Mistral's Voxtral family also combines transcription and audio understanding. With the recently announced Voxtral 3, the company is focusing on real-time voice agents.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe now
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.