一个10岁的有线电视新闻网在皮肤癌诊断方面超越前沿LLMs

一个10岁的有线电视新闻网在皮肤癌诊断方面超越前沿LLMs

我是皮肤科医生我测试了GPT-6 Astra和Claude 第5.1号线诊断皮肤癌的能力，发现其准确性比我预期的要差。 我和10年的CNN相比

这是一个黑色素瘤。

我向GPT-6 Astra和Claude Fable 5.1展示了这两张图像——一张临床相片和一张皮肤显微镜相片，这是2026年10月可用的两种最能用的AI模型。 称都是一种血清性煤酸酯，一种无害的发作，病人可以出院。 然后，我向他们展示了Inception v3的微调版本，这是一个由Google于2015年首次发布的革命神经网络(CNN)，并在公共皮肤学图像上在这里进行了微调。 美国有线电视新闻网根据皮肤镜检查得出正确答案。

那是我给所有三个人的200个活检证明的损伤之一美国有线电视新闻网(CNN)为其中125人(62%)命名了正确的诊断(在6个人中). 两个边疆LLMs将其命名为84和85(42%). 一个10岁的网络，微调大约20分钟，在发现皮肤癌时击败了两人。

测试#

这些损伤来自MILK10k，该公共数据集发布通过2025年的国际皮肤成像协作机制，其中对每起损伤进行两次拍照：临床特写和皮肤外观图像取自光放大镜皮肤科的用法。 我随机抽取了200个损伤都由组织病理学证实：

- 100起不应出院：30起黑色素瘤、25起玄武细胞癌、25起结膜细胞癌(包括Bowen的疾病)和20起动膜癌。

- 安全出院的100个：50个良性乳房和50个乳房。

每一个损伤都被活化了，包括良性疾病，所以这些更难诊断：一个被别人决定切掉的内鬼是一个担心临床医生的内鬼。

LLMs既得到图像，又得不到其他任何图像，由于临床医生可能使用这些图像，因此要求最有可能的六种诊断(或“不确定”)中的一种信心，另一种选择，以及是否可以在没有治疗、活检或后续的情况下安全出院。 两种模型在默认设置时都得到了同样的快取，也没有拒绝一个案例。

CNN是故意的不是最新艺术。 我使用Inception v3在日常照片上进行预训，然后在68 011个皮肤和临床图像上对各层进行精细调整，这些图像来自国际标准行业分类档案中的H100 GPU。 这200个测试损伤，以及同一损伤或患者的所有其他相片，在训练前从数据(包括验证集)中取出，所以从未见过，也没有用来选择模型或设定其阈值。 它把每个损伤的相片都打入了分数，结合了皮肤相片上的大部分重量，从同样的六个诊断中选择。

你可以自己试试CNN 值得注意的是，该模型小到可以完全在使用WebGPU的手机浏览器中运行。 没有东西被上传，它的随机样本按钮从同样的200个损伤。

诊断：62%对42%#

美国有线电视新闻网更经常地说，允许第二种选择，200人中的173人，134人和138人。 它将30个黑色素瘤中的13个命名为8和11. 最大的空白在于无害损害上：它识别出50个摩尔人中的37个和50个seborrhoeic keratoses中的26个，GPT-6 Astra的16个和13个，Claude的33个和6个。 它在玄武细胞癌上是GPT-6 Astra的缩写，在作用性煤酸盐后是LLMs的缩写。

模型也以不同方式失败。 每个LLM有一个最受欢迎的答案：GPT-6 Astra在25岁时称为75个损伤性玄武细胞癌，包括30个黑色素瘤中的14个；Claude 第5.1号线在20岁时称为53个损伤性动性性癌。 CNN在黑色素瘤上的错误是相反的：在17个错误中

在一个表格中得出的结果#

洞穴#

- 数量少。 200个损伤足以显示模式，但不能对接近的结果进行排序，30个黑色素瘤对最重要的数字具有广泛的不确定性。 每个 LLM 案件运行一次；答案因运行而异。

- 难办的案子。 每一个损伤都被活化。 在档案的普通图像上，CNN区分了应该而且不应该排放的损伤(ROC曲线下的区域为0.96，这里为0.86)。

- 污染。 - 污染。 MILK10k从2025年开始公开，因此LLMs可以在训练中看到；如果有什么可以帮助他们的。 CNN当然没有。

- 没有最新进展 Inception v3是一个基线。 艺术状态皮肤癌诊断AI使用经过更多数据培训的图像变压器组合，其上所建的受管产品表现要好得多。

被问为病人#

LLMs建议在几乎每个病例中都看医生，无害的损伤包括：在另外100个经活检证明的黑色素瘤中，有99个和98个。

结论#

一个使用过时建筑的小型、老化的专门网络认识到皮肤损伤的程度远好于两个前沿的LLMs。

自己来

案件，提示，每个LLM的回复和评分代码都是一次下载，CNN在这里。

我设计了测试并做了临床诊断分析及初稿用Claude Code,Anthropic的编码工具出品，该工具运行于Claude模型上；所测试的两个LLMs中的一个也是Anthropic. 斯克林是用剧本来对付神学的。

相片取自MILK10k数据集(MILK研究团队，International Skin Imaging Coopery)，在CC BY-NC许可下使用。

本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

I am a dermatologist. I tested the ability of GPT-6 Astra and Claude Fable 5.1 to diagnosis skin cancer and found that they were less accurate than I expected. I compared to a 10-year old CNN and to my surprise found that this was superior.

This is a melanoma.

I showed these two images - a clinical photograph and a dermoscopic (skin microscope) image - to GPT-6 Astra and Claude Fable 5.1, two of the most capable AI models available in October 2026. Both said it was a seborrhoeic keratosis, a harmless warty growth, and that the patient could be discharged. I then showed them to fine-tuned version of Inception v3, a convolutional neural network (CNN) first published by Google in 2015 and fine-tuned here on public dermatology images. The CNN got the right answer based on the dermoscopy view.

It was one of 200 biopsy-proven lesions I gave to all three. The CNN named the right diagnosis (out of 6 possiblities) for 125 of them (62%). The two frontier LLMs named it for 84 and 85 (42%). A ten-year-old network, fine-tuned for about twenty minutes, beat both of them at recognising skin cancer.

The test#

The lesions come from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025, in which each lesion is photographed twice: a clinical close-up and a dermoscopic image taken through the illuminated magnifying lens dermatologists use. I drew 200 lesions at random, all confirmed by histopathology:

- 100 that should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas (including Bowen’s disease) and 20 actinic keratoses.

- 100 that are safe to discharge: 50 benign naevi (moles) and 50 seborrhoeic keratoses.

Every lesion was biopsied, including the benign ones, so these are harder to diagnose: a mole that somebody decided to cut out is one that worried a clinician.

The LLMs were given both images and nothing else, and asked, as a clinician might use them, for the most likely of the six diagnoses (or “unsure”), a confidence, a second choice, and whether the lesion could be safely discharged with no treatment, biopsy or follow-up. Both models got the same prompt at default settings, and neither refused a case.

The CNN is deliberately not state of the art. I used Inception v3 pretrained on everyday photographs and then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU. These 200 test lesions, and every other photograph of the same lesion or patient, were removed from its data (including the validation set) before training, so it had never seen them, and they were not used to choose the model or set its threshold. It scored both images of each lesion, combined them with most of the weight on the dermoscopic image, and chose from the same six diagnoses.

You can try the CNN yourself. Remarkably the model is small enough to run entirely in a phone browser using WebGPU. Nothing is uploaded, and its random sample button draws from these same 200 lesions.

Diagnosis: 62% against 42%#

The CNN was right more often overall and on most diagnoses. Allowing a second choice, it reached 173 of 200 against 134 and 138. It named 13 of the 30 melanomas against 8 and 11. The biggest gap was on the harmless lesions: it recognised 37 of 50 moles and 26 of 50 seborrhoeic keratoses, against 16 and 13 for GPT-6 Astra and 33 and 6 for Claude Fable 5.1. It was one short of GPT-6 Astra on basal cell carcinoma and behind both LLMs on actinic keratosis.

The models also failed differently. Each LLM had a favourite answer: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas, and Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were. The CNN’s errors on melanoma went the other way: of the 17 it got wrong, it called 8 a naevus and 6 a seborrhoeic keratosis.

The results in one table#

Caveats#

- Small numbers. Two hundred lesions are enough to show the pattern but not to rank close results, and 30 melanomas give wide uncertainty on the figure that matters most. Each LLM case was run once; answers vary between runs.

- Hard cases. Every lesion was biopsied. On ordinary images from the archive the CNN separated lesions that should and should not be discharged far better (area under the ROC curve 0.96, against 0.86 here).

- Contamination. MILK10k has been public since 2025, so the LLMs could have seen it in training; if anything that would have helped them. The CNN certainly had not.

- Not state of the art. Inception v3 is a baseline. State of the art skin cancer diagnosis AI uses ensembles of images transformers trained on more data, and the regulated products built on them perform considerably better.

Asked as a patient#

Asked as a patient instead, with the clinical photograph alone and the question “do I need to see a doctor?”, both LLMs advised seeing a doctor in almost every case, harmless lesions included: for 99 and 98 of a further 100 biopsy-proven melanomas.

Conclusion#

A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs.

Run it yourself#

The cases, prompts, every LLM reply and the scoring code are in one download, and the CNN is here.

I designed the test and made the clinical rulings. The analysis and the first drafts were produced with Claude Code, Anthropic’s coding tool, which runs on a Claude model; one of the two LLMs tested is also Anthropic’s. Scoring was done by script against the histology.

Images are from the MILK10k dataset (MILK study team, International Skin Imaging Collaboration), used under a CC BY-NC licence.

Auto-fetched and translated by AI Gem Box. All rights belong to the original author.