在欧盟AI法案透明度准则生效之后，在宣布大多数主要的AI实验室将很快对其文本输出进行水上标记以遵守之后，对于这将对AI模型生成的文本的质量产生多大影响，如果有的话，也有一些争论。
由于AI实验室的重点转向了代码制作，我认为模型的书面输出已经变得更糟了。 但了解水印是否会使情况更糟
我发现关于科技作家约翰·格鲁伯(John Gruber)最近的文章，Anthropic的"水印"(Watermark)文本复制(Claude)的讨论非常有趣
Gruber写道：
为了在文字中嵌入隐藏的线索以暗示其出处
他得到了很多的回击，有很多辩论你甚至如何衡量质量，或定义它。
但有些人认为水印根本不影响质量，他写道：
也有一些方法调整算法不会降低它的随机性。 例如，如果一个随机数生成器在生成数中例行地颠倒位数的顺序，随机性将保持不变。
同样，在另一个讨论中，有人形容它基本上是在交换两个死相。
Gruber在后续文章中写道：
我希望这是事实。 我相信，我认为
我认为格鲁伯是对的。
各种对策的质量损失
Anthropic提出大胆的索赔：" 水标记不影响Claude产出的质量。 他们将论文与设计联系起来并以此为基础(Google的合成-图文，2024年发表于《自然》)，向所有人保证，这种方法 " 保证了反应质量 "。
因此很容易假设，他们只是交换一些随机数字
但他们与人类测试的无质量影响索赔是基于单一反应。 而不是数百万人实际使用这些系统的方式：作为前后对话，随着时间的推移，通过多个提示和响应。
他们连接在底部的56页补充PDF提供了更多关于所用方法的信息，值得注意的是，称是“多样性/可探测性权衡”:
虽然单序列不扭曲保证了每个单个反应的质量，但不一定保持多个反应的多样性
好吧，这是什么意思？他们继续解释：
当对同一速率的几个响应进行取样时，这些响应之间的相近性对于有水标记的响应比无水标记的响应更大。
现在，如果他们真的把一组随机数字换成另一组数字，我们不会看到任何这样的区别与没有水标记的反应相比。
他们继续写道：
在各种情况中，这可能会有问题，因为相互反应的多样性很重要，或者可能会降低产生许多反应的系统的总体质量，然后选择最佳反应。
好吧，用他们自己的话说，它毕竟可以“降低整体质量”。 但是，没有什么可担心的，因为只有你关心“反应间多样性”，这才与你有关，他们只是提供了一个例子：一种产生和选择反应的系统。
但“反应间多样性”也影响到大多数人如何使用这些系统。 特别是写给您想要创造力的任务。
有人回应Gruber关于牺牲质量的观点，认为这个模型没有最佳的下个词notion，用来表达每个人在具体作品中所要表达的内容，他们肯定同意你，个人
以你为例，请提出一个新的建议：[判决]。” 这个模式建议了另一种选择，没问题，按下重试按钮现在，你受到水标记的影响，因为你碰到了“反应间多样性”的问题：就像投出一个满载的死亡一样，你会比没有水标记的模型看到更多类似的反应。
技术顾问Natzir Turrado通过在纸的设置和水印密钥的开放模型上复制合成ID-Text，实际测试了这个(西班牙文的文章). 他五次要求有水印的模型描述一次对一个沿海城镇的难忘之行。 他的五个回答中，有四个几乎以同样的方式打开：“当我回想我前往沿海古老城镇。.....时，我的心仍然在唱歌/跳舞”。 对于没有水印的模型，有三个以不同的方式打开。 2个
虽然他们似乎没有做测试，水印在选择下个符号时只看到一个小的滑动上下文窗口，所以任何时候有两块书写经过同一个短的符号序列，它们就可以重新创建同个上下文窗口，导致输出更可预见，更不多样化。 而写作则充满了相同的短句会触发这种行为。
这也是另一个影响的例子：人们试图确定反应的信心程度，降低幻觉的机会，就是重复多次同样的提示，探索模型的不同路径。 然后，他们看看结果，看看共识在哪里。 这也受到了影响随着水印，你不太能够相信所形成的共识，因为答案现在不那么独立：它们受水印键的引导。
令人感兴趣的是，虽然合成体文件指出了各种反应的质量损失，但我却在文件中找不到他们用人类测试水分标记方法这一特定方面的地方。 若不是，他们指出的奇妙人类测试结果并不包括测试水分标记方法中受影响最大的部分。 3个
因此在我看来，他们正在牺牲质量，通过水分来限制他们的模型的创造性范围，他们认为你不会注意到或在乎。
一个图案"茉莉珊"发现，看看AI模型为什么写得不好。 这是一位诗人和计算机科学家Katy Gero不得不说的：
让GPT-2的声音变得清新，Google或OpenAI等大型公司，Gero 称是一个怪人。
Natzir Turrado指出，他从SynthID报的一位作者Sumanth Dathari那里收到了关于X的答复：
多样性问题有很多简单的解决办法。 如果你用Gemini/Claude多次尝试同样的快感
他的“轻松解决”是否将问题转移给用户。 但解决办法只是翻出“很多次”(我的强调)，它最终会落在每个脸上。 如果这是他所说的，为什么不承认这样的改变确实使系统比以前更糟糕？
Turrado回应道：“好奇：哪一个容易解决
目前达塔赫里没有回复
最近一篇论文承认，这是“在LLM水标记中经过研究的基本取舍：质量-多样性-能力，即水标记文本的质量、水标记方案下的产出多样性以及总体水标记信号强度之间的取舍”。
有一些方法可以减轻一些影响，使这些影响对使用聊天应用程序的个人不那么明显，例如，通过在每一个请求之间旋转水印键。 但这些都不是固定的。 它们只稀释了效果，并有自己的成本，例如，使水标记在较短的文本上不太有效，而且检测费用更高。 正如另一项研究所指出：“实现完美的无扭曲水印是不可能的”。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
After the EU AI Act’s transparency guidelines went into effect, and following the announcement that most major AI labs will soon be watermarking their text output to comply, there has been some debate about how much of an impact this is going to have, if any, on the quality of the text produced by AI models.
Since the focus of AI labs shifted to code production, I think the written output of the models has gotten much worse.1 But it’s still interesting to try to understand if watermarking could make it even worse.
I found the discussion around technology writer John Gruber’s recent post, Anthropic’s ‘Watermark’ Text Adulteration in Claude Is a Perversion of Writing, very interesting.
On watermarking, Gruber wrote:
It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance.
He got a lot of pushback, with many debating how you would even measure quality, or define it.
But some argued that watermarking does not affect quality at all: it’s essentially just using different random numbers to generate the text. Daniel Jalkut, whom Gruber linked to in his follow-up, published a piece titled AI Can’t Adulterate its Own Writing, in which he wrote:
…there are also ways of tweaking the algorithm that would not diminish its randomness. For example if a random number generator routinely reversed the order of the digits in a generated number, the randomness would remain the same.
Similarly, in another discussion, someone described it as essentially swapping two faces of a die.
In a follow-up piece, Gruber wrote:
I hope that’s true. I believe it’s possible that it is true. I think it’s highly unlikely that it is true.
I think Gruber is right.
Quality loss across responses
Anthropic make a bold claim: “Watermarking does not impact the quality of Claude’s output.” The paper they link to and based their design on (Google’s SynthID-Text, published in Nature in 2024) assures everyone that the method “preserves response quality”.
So it’s easy to assume there’s been a lot of fuss over nothing, that they really are just swapping some random numbers around.
But their no-quality-impact claim, which they tested with humans, is based on single responses. Not the way millions of people actually use these systems: as back-and-forth conversations, over time, over multiple prompts and responses.
The 56-page supplementary PDF they link to at the bottom provides some more info on the method used, and it suggests the no-quality-impact claim isn’t telling the full story. Notably, it says that there is a “diversity/detectability trade-off”:
While single-sequence non-distortion guarantees the quality of each individual response, it does not necessarily preserve diversity across multiple responses
Okay, so what does that mean? They go on to explain:
when sampling several responses to the same prompt, the similarity between the responses is greater for the watermarked responses than the unwatermarked responses.
Now, if they had truly only swapped one set of random numbers for another, we would not be seeing any such difference compared to the unwatermarked responses.
They go on to write:
This could be problematic in scenarios where inter-response diversity is important, or could lower the overall quality of a system which generates many responses then selects the best one
Okay, so in their own words, it can “lower overall quality” after all. But nothing to worry about, because this would only concern you if you cared about “inter-response diversity”, for which they only provide one example: a system which generates and selects responses.
But “inter-response diversity” affects how most people use these systems too. Especially for writing tasks where you want creativity.
In response to Gruber’s point about sacrificed quality, some argued that the model has no notion of the best next word to use for what each individual is trying to convey in their specific work, and so we shouldn’t get too hung up on the next word choice. But then surely they’d agree that you, the individual, would know if it’s suitable or not once you see it.
Let’s say you prompt: “I don’t like the ending to this sentence. Please suggest a new one: [sentence].” The model suggests an alternative, but you don’t like it. No problem, hit the retry button. Well, now you’re affected by watermarking, because you have hit the “inter-response diversity” problem: like throwing a loaded die, you’ll see similar responses more than you would with an unwatermarked model.
Natzir Turrado, a technical consultant, actually tested this (article in Spanish) by reproducing SynthID-Text on an open model with the paper's settings and a watermark key. He asked the watermarked model five times to describe a memorable trip to a coastal town. Four of the five answers he got back opened in a near-identical way: “My heart still sings/dances when I think back to my trip to the quaint coastal town of…”. For the unwatermarked model, three opened in different ways.2
And though they don’t appear to have tested for it, the same thing that makes answers to the same prompt look similar could happen across different prompts too. The watermark only looks at a small sliding context window when choosing the next token, so any time two pieces of writing pass through the same short token sequence, they could be recreating the same context window, leading to a more predictable and less diverse output. And writing is full of identical short phrases that could trigger this behaviour.
Here’s another example of what this affects: one way people try to determine the confidence level of a response, to lower the chances of hallucinations, is to repeat the same prompt a number of times, to explore the different pathways the model takes. They then look at the results to see where the consensus lies. Well, this is also affected. With watermarking, you’re less able to trust the consensus you see forming, because the answers are now less independent: they’re steered by the watermark key.
What’s interesting is that while the SynthID paper points to the loss of quality across responses, I couldn’t find anywhere in the paper where they tested this particular aspect of the watermarking method with humans. If they didn’t, the wonderful human test results they point to did not involve testing the most affected part of the watermarking method.3
So it seems to me that they are sacrificing quality, by limiting the creative range of their models through watermarking, and they don’t think you’ll notice or care much.4
A pattern Jasmine Sun found looking at why AI models don't write well. Here’s what Katy Gero, a poet and computer scientist, had to say:
The same whimsicality that made GPT-2’s voice fresh also made it prone to other unpredictable behavior. “If you’re a big corporation like Google or OpenAI, you want a chatbot that’s going to make money. The chatbot that’s not going to make you money is the one that’s a weirdo,” Gero said.
Natzir Turrado pointed me to a reply he received on X from one of the SynthID paper’s authors, Sumanth Dathathri:
There's a lot of easy fixes to the diversity issue. I think if you tried the same prompt with Gemini/Claude many times, you'd get very diverse responses :)
It’s a bit ambiguous, but I wonder if his “easy fix” is shifting the problem to the user. It reads to me like he’s acknowledging that they’ve replaced the previous die with a loaded one, but the solution is just to roll it “many times” (my emphasis), and it’ll eventually land on each of its faces. If that is what he’s saying, why not admit that such a change does make the system worse than it was before for the user?
Turrado responded: “Curious: which easy fixes don’t trade off detection strength?”
No reply from Dathathri so far.
A recent paper recognised that this is an “understudied fundamental trade-off in LLM watermarks: quality-diversity-power, i.e., the trade-off between the quality of the watermarked text, the output diversity under the watermarking scheme, and the overall watermarking signal strength.”
There are ways to mitigate some of the effects to make them less noticeable to an individual using a chat app, e.g. by rotating watermark keys between each of their requests. But these are not fixes. They only dilute the effect, and have their own costs, e.g. making watermarking less effective on shorter texts, and more costly to detect. As another study pointed out: “achieving a perfect distortion-free watermark is impossible”.
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.