Edge0拥有发布 Audio8-TTS-Realistic，为零镜头语音克隆的开口量文本转语音型。 Audio8-TTS-Realistic是一个0.6B参数模型，从1个参考剪辑和誊本中复制出一个语音。 它支持23种语言，比它的前身多12种。 Apache 2.0下方的重量和密码船。 对于开发者来说，捕获量是竞争性多语种精度，其大小对微调来说是便宜的并服务。
TL; DR 数据
- 大小：601M主模型参数(不包括代码)；最多2 048个装入的文字/上下文的音频位置。
- 运行：建议使用CUDA GPU；在BF16的NVIDIA H20上进行烟雾测试。 论文在Apple M2 CPU(INT4 ONNX)上报告了1.1至1.2 GiB峰值内存。 没有现实的 ONNX 导出是公开的。
- 性能：领先于CV3上较大的开放型号，落后于MiniMax上4B级型号。
- 最佳：9种语言平均CV3出错4.86%,Qwen3-TTS-1.7B为7.73%。
- 最糟的是：48.57%的WER在泰语(CV3)上，31.35%的CER在广东语(MiniMax)上。
什么是Audio8-TTS-Realistic? (英语).
这是Edge0的Audio8 TTS家族的第二个0. 它以Audio8-TTS-Preview-0.6b为基础，并保持了相同的变形器推论接口。 寄存器捆绑了检查站，一个44.1千赫神经音频编码器，活化器，处理器和自定义代码。 它也可以在没有参考声音的情况下生成语音。
新语言有阿拉伯语，芬兰语，印地语，印尼语，马来语，挪威语，葡萄牙语，俄语，瑞典语，泰语，土耳其语和越南语。
建筑如何运作？
该型号采用了由Fish Audio S2 Pro所启发的双自旋(DualAR)设计。 根据技术研究报告，慢分行是建立在Qwen2.5 0.5B上。 它有24层，宽度896，运行速度约为每秒21.5帧。 每一步都会预测出文本符号外加第一个音频代码本。
一个浅的4层快分行接下来会同时预测剩下的9个代码本。 因此，顺序计算只在帧一级进行。
Edge0如何训练现实主义？
团队认为，单词出错率(WER)和扬声器相似度(SIM)错失了节奏，暂停和时间。 一个模型在两者上都能得分良好，并且仍然是声音合成。
培训分为三个阶段：
- 多语种监督微调(SFT)。
- GRPO 附加奖励：WER0.7重，SIM0.3重。
- GRPO由一位有学问的现实主义评论家指导。
评论家根据同一句子的相匹配的人类记录来评分每个输出。 它接受了由6个开源 TTS系统所建造的优质梯子的训练。 这些作品被评为"私自盲目听力研究"，以人类录音为最。 合成暂停压缩新增精细跑向梯子。
失去身份后，评论家无法奖励发电机指纹。 评论家头部在被冷冻的WavLM大型编码器上方有大约3M参数。 持续时间的处罚还比参考长度高1.35倍。 这阻止了政策以较慢的言论来吸引批评者。
决赛阶段仅使用中英文提示。 它仍然改进了13种低资源语言的总衡量标准。 他们的错误从22.53%下降到20.77%,SIM从0.637上升到了0.679.
在基准测试上表现如何？.
以下所有数字均为自报出错率，较低者较好。
-种子-TTS (英文/中文):1.25%平均出错，SIM 0.704.
- CV3(9种语言)：平均误差4.86%,SIM 0.661.
- MiniMax(24种语言)：平均误差5.89%,SIM 0.754.
在CV3上，Audio8击败了VoxCPM2(5.35%),CosyVoice3-1.5B(6.89%)和Qwen3-TTS-1.7B(7.73%). 在MiniMax上，Qwen3-TTS-1.7B的成绩为27.41%，但官方只支持24种测试语言中的10种。
较大的模型仍然领先于其他地方。 Higgs TTS 3在CV3上报告4.41%，在MiniMax上报告2.74%. Qwen3-TTS-1.7B在种子-TTS上的平均比例为1.01%. Edge0本身称其表为参考比较，因为评价管道和文本正常化有所不同。
它如何与其他开放的TTS模型进行比较？
VoxCPM2的CV3平均值是从其每种语言表中计算的。
你怎么部署的？
安装 PyTorch 2.5+和变形器 4.57+，再以信任-remote code=True来加载。 语音克隆需要一个干净的参考片段及其准确的笔录。 GitHub repo 新增 a CLI，分批并监督微调脚本。
对于GPU服务来说，一个SGLang Omni适配器提供呼号关注，动态分批和OpenAI相容/v1/audio/speech API. Edge0 指出吞吐量和流出量没有基准。 Hugging Face ZeroGPU上的免费游戏场运行。
关键外卖
- 601M参数，23种语言，Apache 2.0重。
- 一个有学问的现实主义评论家推动最后的GRPO阶段。
- 4.86% CV3出错比Qwen3-TTS-1.7B和VoxCPM2.
- 4B级的Higgs TTS 3仍然领先于CV3和MiniMax.
- 低资源语言仍然薄弱，泰国语接近49%。
看看报纸，模型重量，Repo和样本并加入我们的通讯。 等等，你在电报里吗？现在你也可以加入我们的电报。
需要和我们合作推广您的 GitHub Repo OR Hugging Face Page OR Product Release OR Webinar等吗？联系我们
Asif Razzaq是Marktechpost AI Media Inc.的首席执行官。 作为一个有远见的企业家和工程师，阿西夫致力于利用人工智能的潜力为社会造福。 他最近的努力是人工智能媒体平台的发布，该平台以深入报道机器学习和深层学习新闻而出名，这些新闻在技术上既可靠又为广大观众所容易理解。 该平台每月有超过200万次观看，表明其受观众欢迎。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Edge0 has released Audio8-TTS-Realistic, an open-weight text-to-speech model for zero-shot voice cloning. Audio8-TTS-Realistic is a 0.6B-parameter model that clones a voice from 1 reference clip and transcript. It supports 23 languages, 12 more than its predecessor. The weights and code ship under Apache 2.0. For developers, the catch is competitive multilingual accuracy at a size that is cheap to fine-tune and serve.
TL;DR
- Size: 601M main-model parameters (codec excluded); up to 2,048 packed text/audio positions of context.
- Runs on: a CUDA GPU is recommended; smoke-tested on NVIDIA H20 in BF16. The paper reports 1.1 to 1.2 GiB peak memory on an Apple M2 CPU (INT4 ONNX). No Realistic ONNX export is public yet.
- Performance: ahead of larger open models on CV3, behind 4B-class models on MiniMax.
- Best: 4.86% average CV3 error across 9 languages, versus 7.73% for Qwen3-TTS-1.7B.
- Worst: 48.57% WER on Thai (CV3) and 31.35% CER on Cantonese (MiniMax).
What is Audio8-TTS-Realistic?
It is the second 0.6B release in Edge0’s Audio8 TTS family. It builds on Audio8-TTS-Preview-0.6b and keeps the same Transformers inference interface. The repository bundles the checkpoint, a 44.1 kHz neural audio codec, tokenizer, processor and custom code. It can also generate speech without a reference voice.
New languages are Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish and Vietnamese.
How does the architecture work?
The model uses a dual-autoregressive (DualAR) design inspired by Fish Audio S2 Pro. According to the technical research report, the slow branch is built on Qwen2.5 0.5B. It has 24 layers at width 896 and runs at about 21.5 frames per second. Each step predicts text tokens plus the first audio codebook.
A shallow 4-layer fast branch then predicts the remaining 9 codebooks in parallel. Sequential compute therefore happens only at the frame level.
How does Edge0 train for realism?
The team argues that word error rate (WER) and speaker similarity (SIM) miss rhythm, pauses and timing. A model can score well on both and still sound synthetic.
Training runs in 3 stages:
- Multilingual supervised fine-tuning (SFT).
- GRPO with a combined reward: 0.7 weight on WER, 0.3 on SIM.
- GRPO guided by a learned realism critic.
The critic scores each output against a matched human recording of the same sentence. It was trained on a quality ladder built from 6 open-source TTS systems. These were ranked in a private blind listening study, with human recordings at the top. Synthetic pause compression adds finer rungs to the ladder.
An identity-erasure loss stops the critic from rewarding generator fingerprints. The critic head has about 3M parameters on top of a frozen WavLM-large encoder. A duration penalty also kicks in beyond 1.35x the reference length. This blocks the policy from gaming the critic with slower speech.
The final stage used only Chinese and English prompts. It still improved both aggregate metrics on the 13 lower-resource languages. Their error fell from 22.53% to 20.77%, and SIM rose from 0.637 to 0.679.
How does it perform on benchmarks?
All numbers below are self-reported error rates, where lower is better.
- Seed-TTS (English/Chinese): 1.25% average error, SIM 0.704.
- CV3 (9 languages): 4.86% average error, SIM 0.661.
- MiniMax (24 languages): 5.89% average error, SIM 0.754.
On CV3, Audio8 beats VoxCPM2 (5.35%), CosyVoice3-1.5B (6.89%) and Qwen3-TTS-1.7B (7.73%). On MiniMax, Qwen3-TTS-1.7B scores 27.41%, but it officially supports only 10 of the 24 test languages.
Larger models still lead elsewhere. Higgs TTS 3 reports 4.41% on CV3 and 2.74% on MiniMax. Qwen3-TTS-1.7B averages 1.01% on Seed-TTS. Edge0 itself calls its table a reference comparison, since evaluation pipelines and text normalization differ.
How does it compare with other open TTS models?
VoxCPM2’s CV3 average is computed from its per-language table.
How do you deploy it?
Install PyTorch 2.5+ and Transformers 4.57+, then load with trust_remote_code=True. Voice cloning needs a clean reference clip and its exact transcript. The GitHub repo adds a CLI, batching and supervised fine-tuning scripts.
For GPU serving, an SGLang Omni adapter provides paged attention, dynamic batching and an OpenAI-compatible /v1/audio/speech API. Edge0 notes throughput and streaming were not benchmarked. A free Playground runs on Hugging Face ZeroGPU.
Key Takeaways
- 601M parameters, 23 languages, Apache 2.0 weights.
- A learned realism critic drives the final GRPO stage.
- 4.86% CV3 error beats Qwen3-TTS-1.7B and VoxCPM2.
- 4B-class Higgs TTS 3 still leads on CV3 and MiniMax.
- Lower-resource languages remain weak, with Thai near 49% WER.
Check out the Paper, Model Weights, Repo and Samples. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Asif Razzaq is the CEO of Marktechpost AI Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.