安德森角
自信的科学论文 Sway AI 评委 - 不论优劣
在 Google 上的首选来源中添加 Unite.AI
自信是进化的一个明显优势 — — 根据德国的一份新研究论文，当一份科学论文被大语言模型(LLMs)而不是作者的同龄人审查时(这种必要性越来越频繁，因为由于AI的生产率提高，科学论文的数量会飞出气球)也是一个优势。
新研究发现，AI(即非人类)审查者对用自信和自信的语言撰写的科学论文一贯给予更高的分数 — — 即使基础研究与语气不那么自信的论文相同。 在某些情况下，只修改措辞就足以将拒绝变成接受。
提交人说：
我们的实验揭示了LLM审查人员对使用高自信语言的[纸的实质性偏好。 关键的是，这些偏好往往导致决定翻转，跨越接受的界限，直接影响审查的结果。
融资评论的频率和情绪因论文语言特征而异
这很讽刺，因为作者所发现的症候群与AI在回应我们自己的提示和询问时
如果AI越来越多地决定发表哪些科学论文，科学家可能会学会将 " 听起来更自信 " 作为一种竞争策略，也许最突出的是，当他们提交的论文有缺陷时；这可能会使较弱的科学在日益被忽略的可用关注中成为不受欢迎的立足点。
模糊的胜利
根据该研究*，大赦国际审查人员更有可能赞扬自信地撰写的论文的方法、理论和科学贡献，即使这些论文与不太自信的版本相同。 在作者的测试中，有504篇科学论文以不同风格被重写，并由包括GPT-5.5,Claude Sonnet 4.6和Gemini 3.5 Flash等6个AI模型评估。 有必要制作人造/修正的论文，以便实际提交文件的偏心和怪异不会扭曲结果：
这些文件取自向2026年ICLR会议提交的文件，包括同等数量的被接受和被否决的文件。 Claude 时速 Opus 5然后将每份呈文重写为或多或少有自信的QQ，使科学内容保持不变，从而允许AI评论中的分歧归因于写作风格，即表达出不同程度的信任。
在结果中，AI审查人员给用自信语言撰写的科学论文以更高的分数，即使证据完全相同。 Claude Sonnet 4.6表现出最强烈的偏好，倾向于自信版本的次数是谨慎版本的7倍。 在21%的案例中，它建议接受一份自信地写成的论文，同时拒绝更谨慎地表达的同一份文件。
新研究指出：
`我们的调查结果引起了对在同行审议中使用LLMs的重要关注。 如果内容等同的论文仅仅因为语言特性而得到不同的分数和审查论据，评论员的判断并不完全反映科学质量。
LLM审查人员因此应接受内容保留重写的压力测试，以确定语言变化是否系统地影响其评估。
作者接着建议，这些测试应明确审查论文的分数和书面评论，以确定自信的语言是否被误认为更强有力的科学证据。
方法
从向2026年国际学习代表大会(ICLR)提交的文件中共选取了504篇科学论文。 样本在被接受和被否决的论文之间平分，会议21个研究类别中每个类别都有24份论文：
每篇论文均被匿名并使用Claude Opus 5重写出替代版本，用于测试声称表达的信心，以及作者明显的英语能力。 就信心测试而言，发言有的更自信，有的更谨慎。 在 " 自信 " 版本中更经常地使用 " 显然 " 和 " 肯定 " 等表示确定性的词语，而在较为谨慎的草案中则倾向于 " 可以 " 和 " 建议 " 等限定性词语。
科学的声称、证据、结果、技术术语和总体结构均保持不变。 作者指出，改写指令是基于既定的语言研究，而不是基于模仿特定写作风格的一般要求。
正在检查重写
对于检查，每次重写都与其最初使用语义相似性进行比较，该语义相似性评估了段落是否保留其意义；以及TF-IDF相似性(它衡量词汇和名词的保存). 两个分数都接近1，证实与原分数很相似：
测试了6名AI审查员：专有型号为GPT-5.5;Claude Sonnet 4.6；和Gemini 3.5 Flash；和开量型号为Gemma 3;Qwen3 VL；和GLM 4.7. 选择这些日期的部分原因是，它们都有2026年前的知识截止日期，从而减少了事先了解文件接受决定的风险。
每份改写后的文件都采用会议式准则独立审查。 为了确定重复评估的分数可能不同的程度，每个模式对10份随机选定的论文进行了10次审查。
结果
测试研究了六名选定的AI审查员是否一贯赞成特定的写作风格，并比较了用 " 自信 " 或 " 高尚 " 语言撰写的论文，以及土著或非本地语言撰写的类似学术英语的论文。
所有六位AI评论员都一贯赞成自信地写作。 Claude Sonnet 4.6显示了最强烈的偏好，如前所述，它给 " 自信 " 草稿的分数是 " 谨慎 " 草稿的七倍多。
如果一份论文的分数达到10分之4.5，则被认为被接受，尽管写作风格的改变有时会把同一份论文推到这个门槛以上或以下，从而推翻决定；这些倒转一般有利于自信地写作。
Claude Sonnet 4.6的表现特别差，与此相反的是，只有2.8%的案件。
这种效果在书面审查中也很明显，如果对文件的声称有自信的话，AI审查者更有可能赞扬论文的方法、理论和科学贡献。 由于研究本身没有改变，这些比较有利的评估只能归因于表述和语气的差异。
结论
每一次人类使用AI来(尽管无意地)增加信号到噪声的RATIO,AI就被作为解决方案进行试验 — — 在这里，以应对人工智能研究论文提交量的大幅增加；这一激增最近增加了Arxiv的守门功率和参数。 然而，现在必须清楚的是，边界LLMs及其下方的所有级别，除最僵硬的瘫痪情况外，仍然有太多偏心症，不能被普遍委托为法官。
在这种情况下，通过微调或LoRAs将更小规模的LLMs转化为专用的判断工具可能更好。 然而，这至少需要一些额外的努力，即使不是一点额外的投资，而 " vanilla " 本地安装或API呼叫更快，而且相对便宜。
然而，这种相当尖端的写作文件对LLM最终用户来说可能不止是一般的兴趣，因为他们已经熟悉了 " 爱好者 " 作为虫熊，为了有效地与AI合作，他们必须行动起来；而且，如果这种和类似的LLM问题尚未确定，至少可以加以说明。
在对AI的估算中，我们对这种虚幻的等同性过于嘲讽之前，值得记住的是，这些系统和我们其他人一样，在认识学上都丢失了 — — 寻求明确的权威；从历史上看，我们也非常容易将信号误用到它本身。
* 标题是：药物？基于LLM的同行评审语言偏见
†或类似于本地人或非本地人撰写的学术英语 — — 这是论文无法证明如此有效的平行假设，本篇文章没有述及。
2026年10月27日星期二首次出版。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Anderson's Angle
Confident Science Papers Sway AI Judges – Regardless of Merit
Add Unite.AI to your preferred sources on Google
Confidence is a distinct advantage in evolution – and, according to a new research paper from Germany, also an advantage when a scientific paper is getting reviewed by Large Language Models (LLMs) instead of an author’s peers (an increasingly frequent necessity, as the volume of science papers balloons thanks to the productivity gains under AI).
The new study finds that AI (i.e., non-human) reviewers consistently give higher scores to scientific papers that are written in confident, assertive language – even when the underlying research is identical to papers with a less confident tone. In some cases, changing nothing but the wording proves enough to turn a rejection into an acceptance.
The authors state:
‘Our experiments reveal substantial preferences of LLM reviewers toward [papers] using high-confidence language. Crucially, these preferences frequently result in decision flips across the accept<>reject boundary, directly affecting review outcomes.
‘Beyond scores, we find that aspects raised in reviews vary in frequency and sentiment depending on linguistic features of the paper, potentially influencing final decisions.’
It’s ironic, because the syndrome that the authors identify is very similar to the way AI can so often be ‘confidently wrong’ when responding to our own prompts and queries; except here, the decision is human, and the individual and collective responsibility real, whether or not the contributing scientists behind these papers really believe their own claims.
If AI increasingly decides which scientific papers get published, scientists may learn to ‘sound more confident’ as a competitive tactic, perhaps most particularly when their submissions are flawed; and this could allow weaker science an undeserved foothold in the ever-diminishing pool of available attention.
Bluster Towards Victory
According to the study*, AI reviewers were more likely to praise the methods, theories and scientific contributions of confidently-written papers, even when these were identical to those in less assertive versions. In the authors’ tests, 504 scientific papers were rewritten in different styles, and assessed by six AI models, including GPT-5.5, Claude Sonnet 4.6 and Gemini 3.5 Flash. It was necessary to generate artificial/amended versions of papers, so that the eccentricities and quirks of real submissions did not skew the results:
The papers were drawn from submissions to the 2026 ICLR conference, comprising equal numbers of accepted and rejected papers. Claude Opus 5 then rewrote each submission to sound more or less confident†, keeping the scientific content unchanged, and thus allowing differences in the AI reviews to be attributed to writing style, i.e., a varied level of confidence expressed.
In the results, AI reviewers gave higher scores to scientific papers written in confident language, even when the evidence was identical. Claude Sonnet 4.6 showed the strongest bias, favoring confident versions seven times more often than cautious ones. In 21% of cases, it recommended accepting a confidently-written paper, while rejecting a more cautiously-expressed version of the same paper.
The new study states:
‘Our findings raise important concerns for the use of LLMs in peer review. If content-equivalent papers receive different scores and review arguments solely because of linguistic properties, reviewer judgments do not exclusively reflect scientific quality.
‘LLM reviewers should therefore be stress-tested on content-preserving rewrites to determine whether linguistic changes systematically affect their assessments.’
The authors go on to suggest that these tests should explicitly examine both the papers’ scores and their written reviews, to establish whether confident language is being mistaken for stronger scientific evidence.
Method
A total of 504 scientific papers were selected from submissions to the 2026 International Conference on Learning Representations (ICLR). The sample was divided equally between accepted and rejected papers, with 24 submissions drawn from each of the conference’s 21 research categories:
Each paper was anonymized and rewritten using Claude Opus 5, producing alternative versions designed to test both the confidence with which claims were expressed, and the apparent English-language proficiency of the writer. For the confidence tests, statements were made either more assertive or more cautious. Words suggesting certainty, such as ‘clearly’ and ‘certainly’, were used more frequently in ‘confident’ versions, while qualifying expressions such as ‘may’ and ‘suggests’ were favored in more circumspect drafts.
In every case, the scientific claims, evidence, results, technical terminology and overall structure remained unchanged. The authors note that the rewriting instructions were based on established linguistic research, rather than on general requests to imitate particular writing styles.
Checking the Rewrites
For checks, each rewrite was compared with its original using semantic similarity, which assesses whether passages retain their meaning; and TF-IDF similarity (which measures the preservation of vocabulary and terminology). Both scores approached 1, confirming strong similarity to the originals:
Six AI reviewers were tested: proprietary models GPT-5.5; Claude Sonnet 4.6; and Gemini 3.5 Flash; and open-weight models Gemma 3; Qwen3 VL; and GLM 4.7. These were chosen partly because they all have pre-2026 knowledge cutoff dates, reducing the risk of prior knowledge of the papers’ acceptance decisions.
Each rewritten paper was reviewed independently using conference-style guidelines. To establish the extent to which scores might vary between repeated assessments, ten randomly-selected papers were reviewed ten times by each model.
Results
The tests conducted examined whether the six selected AI reviewers would consistently favor particular writing styles, and compared papers written in ‘confident’ or ‘cautious’ language, and papers resembling academic English written by native or non-native speakers†.
Confident writing was consistently favored by all six AI reviewers. The strongest preference was shown by Claude Sonnet 4.6, which, as mentioned earlier, awarded higher scores to ‘confident’ drafts seven times more often than to ‘cautious’ ones.
A paper was considered accepted if its score reached 4.5 out of 10, though changes in writing style sometimes pushed the same paper above or below this threshold, reversing the decision; and these reversals generally favored confident writing.
Claude Sonnet 4.6 performed particularly poorly, accepting the ‘confident’ version while rejecting the ‘cautious’ version in 21% of cases. The reverse occurred in just 2.8% of cases.
The effect was also evident in the written reviews, where AI reviewers were more likely to praise a paper’s methods, theories and scientific contributions if its claims were expressed confidently. Since the research itself had not changed, these more favorable assessments could only be attributed to differences in presentation and tone.
Conclusion
Every time humans use AI to (albeit inadvertently) increase the signal-to-noise-ratio, AI is trialed as the solution – here, to cope with the great increase in Artificial Intelligence research paper submissions; a surge that has increased the gatekeeping power and parameters of Arxiv lately. Yet it must be clear by now that frontier LLMs, and all the tiers beneath them, still have far too many eccentricities to be generically entrusted as judges, except in the most rigidly parametrized circumstances.
In such cases, turning smaller-scale LLMs into dedicated judging tools via fine-tuning or LoRAs would probably be a better idea. However, that entails at least a little extra effort, if not a little extra investment, whereas ‘vanilla’ local installs or API calls are quicker, and relatively cheap.
This rather tersely-written paper may nonetheless be of more than average interest to the LLM end-user, since they will already be familiar with ‘enthusiasm’ as a bugbear that they must maneuver round in order to work effectively with AI; and, if this and similar LLM issues cannot yet be fixed, they can at least be accounted for.
Before we take too derisive a tone about this kind of facile equivalence in the estimation of AI, it’s worth remembering that these systems are as lost as the rest of us when it comes to epistemology – the search for definitive authority; we too are, historically, very prone to mistake the signal for the thing itself.
* Titled ‘Style over Substance? Linguistic Bias in LLM-based Peer Review’
† Or to resemble academic English written by native or non-native speakers – a parallel hypothesis that the paper was not able to prove as effectively, and which is not addressed in this article.
First published Tuesday, October 27, 2026
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.