Jack Dorsey's Block 发布是公司研究实验室的一个新语音模型，该型号为"德萨"(Dessa)的称公司完全接受了人语训练并拥有一款NVIDIA GPU.
德萨是一款全复式模型，这意味着它可以同时说话和收听，就像今年早些时候的GPT-Live模型。 一般情况下，语音模型与用于知识检索的大型语言模型相配。 Dessa的知识来源于对话数据。
Block 称认为，目前的模型可以听起来很不自然和令人烦恼，因为它们在管道中有能驱动语音生成的文本。 ElevenLabs v4或"Fish Audio Drama 3"等多款模式在语音生成中使用语音标记来驱动情感。 Dorsey的声称公司将情感植入其模式。 并不清楚这种情绪是否由某种标签所左右。
“与当前声音的交流，AI会很烂。 将你的声音变成文字，世界上最强大的AI无法听到你的感受。 称公司在博客文章中表示
agent有不同的声音、节奏和情感。 例如，在谈话中，机器人称因工作而感到疲惫，此时听起来很疲惫。
在下面的例子中，"机器人"指的是在发出情感和几乎崩溃的声音的同时，让自己感到舒适的人。
另一个显示是Block团队成员与语音助理一起玩游戏。
该型号可在当地运行于消费级计算机如MacBook M4 Max等的称公司或更佳。 然而，目前公众无法进入德萨，而布洛克已发出呼吁，要求安全研究者测试该模型。 这有可能检验一个声音agent在另一端可以欺骗人的情况。
Block从事金融业务，并向小企业出售其解决方案。 Square已经使用语音AI处理订单。 但凭借自己的能运行端侧的语音模式，它可以以更低的成本为更多的商业和消费者体验提供动力。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Jack Dorsey’s Block released a new voice model out of the company’s research lab that runs locally and sounds strangely human with its expressions like laughs, pauses, and cries. The company said the model, called Dessa, is trained fully on human speech and a single NVIDIA GPU.
Dessa is a full-duplex model, which means it can talk and listen at the same time, just like the GPT-Live model OpenAI released earlier this year. Typically, speech models are paired with a large language model for knowledge retrieval. Block said that Dessa’s knowledge stems from the conversational data it has.
Block said that the current models can sound unnatural and annoying because they have text in the pipeline that drives speech generation. A lot of models like ElevenLabs v4 or Fish Audio Drama 3 use voice tags to drive emotion in speech generation. Dorsey’s company claims that emotion is built into its model. It’s not clear if that emotion is being steered by some kind of tags.
“Interacting with current voice AI can just plain suck. It turns your voice into text and throws away an incredible amount of information text alone can’t carry. The most powerful AIs in the world can’t hear how you feel. We think that’s ridiculous, leads to a worse experience, and is potentially dangerous,” the company said in a blog post.
The company’s researchers posted multiple examples of conversations with Dessa where the agent had different voices, cadences, and emotions. For instance, in the conversation, the bot said it was tired from work, and at that point it sounded tired.
In the example below, the bot was talking about the person making it comfortable while sounding emotional and almost breaking down.
Another one shows a Block team member playing a game with the voice assistant.
The company said that the model can run locally on a consumer-grade computer such as a MacBook M4 Max or better. However, there is no public access to Dessa at the moment, and Block has put out a call for safety researchers to test the model. This is likely to test out cases where a voice agent could trick people on the other end.
Block is in the business of finance and selling its solutions to small businesses. Square already uses voice AI to handle ordering. But with its own voice model that can run on-device, it can power a lot more business and consumer experiences at a lower cost.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.