共编织器器
Quenputer: Qwen3.5内一个功能完备，具有决定性的RISC-V CPU
一个股票Qwen 3.5 4B，配有RISC-V CPU编译为它的重量，运行在自己的Gated DeltaNet地层上作为RAM.
真不敢相信这不是硅
Quenputer是一个相当有趣的实验模型。 它是一股Qwen 3.5 4B并加上一股：以该模型自有的Gated DeltaNet关注层为RAM的权重内嵌的RISC-V CPU. (为学习GDN的操作方法，此为方便向导。 这个"CPU"完全运行在模型的前传中，可以运行算术操作，C和MicroPython代码，而无需任何外部运行时间或工具使用。
要非常清楚，它不是插件，工具或沙盒。 以及模型如何使用GDN层作为RAM
我还写了两个在"Quenputer"上建起的续集：混合模型如何可以学习隐藏在它们的RNN状态中的推理并逃避CoT监测，以及这类编译的重量如何作为内置监测工具发挥作用，而模型本身不能篡改。
您可以从 Hugging Face 或 ModelScope 抓取 Quenputer，代码在 GitHub 上。
Qwen3.5内部的RNN可以是精确的内存
Qwen3-Next和Qwen 3.5(型号为Quenputer所建)使用相当整洁的花招。 模型中4个关注层中有3个使用线性RNN的Gated DeltaNet作为关注机制(更多关于这个). 每个头都有一个状态矩阵，而三角洲规则也不断更新。
现在，这里是我们有趣的实验开始的地方。 我发现把门设为1,11 每一个Gated DeltaNet头都有一个状态矩阵，它的内存。 在写新东西之前，每个字都会在大门边上乘以整个州，数字在0到1之间。 RNN就是这样忘记的：由0.9组成的门会淡出每道道具上存储了一点的东西，而由1的门意味着永远没有东西会淡出。 写出强度(β)至1,22 门后，头部将状态的一个方向向新值倾斜，而写作强度β设定了多远。 1的β表示沿着该方向的旧值被完全取代而不是被混合。 并使用一热密钥编码，33 一热密钥是0的矢量，其中有一个单一，所以它指向状态的正一行，即一个细胞的内存地址点的方式。 每一个我们执行的书写将完全覆盖一行，每一个一热的查询可以检索一个准确的值。 这意味着GDN层本身可以作为RAM进行精确的工作，其成本远低于使用正常的注意力作为RAM，因为GDN头的状态无论对话得到的时间有多长都会保持同样的大小，与注意力的KV缓存不同。
三角地规则起到记忆作用的想法并不新鲜：Schlag等 (2021)显示与三角地规则相伴的线性关注是作为关联记忆，或"快重". 44 Imanol Schlag, Kazuki Irie和Jürgen Schmidhuber,"Linear变形器是秘密快重程序员",ICML 2021. 新的事情是使它精确和可处理，在生产模型内，可靠地运行一个完整的CPU。
这是整个模型的基础。 目前，默认的Qwen 3.5并没有在GDN中保留精确的值。 通过这个修改，我们解锁了这个实验的下一部分。
将计算机编译为图层
由于我们现在拥有一个功能齐全，廉价又快的RAM，我们可以着手将一个工作式的CPU编成模型重量。 幸运的是，没有涉及培训：一个编译器直接设置每个重量。 您将CPU描述为一个电路，一个长长的小步骤列表，如"读取这个寄存器","加上这两个数字"或"跳出如果零"，编译器将每个步骤变成一个变压器层的一部份：注意头向上查找事物，向上输入进行算术的单元，以及持有RAM的DeltaNet头。 CPU随后作为模型解码运行。
每个指令成为了几个生成的符号：一个名称是要写入的内存行(ADDR)，下一个包含值(VAL)，一个STEP符号结束指令。 这些符号是CPU的行刑追踪，所以在Qwen生成它们的同时，CPU实际上正在运行。
CPU 电脑
CPU是一款32位的RISC-V核心(RV32IM，外加Zba扩展和大数字算法的小大纳姆单元)，它占据了Qwen32前的11层。 目前，它通过官方的riscv测试，其程序与qemu匹配，并且已经运行了1800万个指令自由运行，没有一次出错。 55 Free-running指CPU在整个运行过程中不断进行自己的输出，没有参考机在步骤之间校正。
这个CPU也可以在前方运行C和MicroPython代码，传得相当好(虽然有点慢). 它目前还不是Linux级芯片，而是执行完整的RV32IM指令集。
Qwen是怎样使用它的
Qwen 3.5 4B能够相当出色地使用这个CPU，不需要任何后训练。 CPU完全在内部使用，不通过任何外部工具调用来引用。
为了让模型可靠地使用这个方法，我决定采取廉价的方式：通过在模型相当训练有素的工具调用格式上插图，Qwen向CPU发送其指示并读取结果。 不使用外部工具调用或运行时间。 一旦我们的CPU收到指令，计算机层就会运行，然后写出工具响应。 我们使用一个叫RvChat的小型驱动程序将固件加载入GDN状态，这可以使痕量符出自Qwen的地层。
Qwen最初的重量保持不变。 计算机的权重被加到上面，以额外的凹陷和地层来表示。
有用吗？
我认为你读完我不断的抱怨之后所想的主要问题是：这真的可行吗？
在深明集：
- div big: Quenputer通过给Qwen 3.5 4B一个Python工具而得分为97.5%，对95%，基数Qwen则得分为72.5%.
- mul big: Quenputer的分数为97.5%，而Python工具的分数为100%，基数为Qwen的分数为55%.
- mul div multiple longer: Quenputer的分数为100%，对97.5%使用Python工具，基数Qwen的分数为90%.
- add 或sub big: Quenputer得分97.5%，与Python工具相同，基数Qwen的得分为87.5%.
在NUPA的长整数算上，Quenputer从48.6%到100%，与给Qwen一个Python工具相同。
CPU打印的数字也是准确的：我对照一个参考机重放大约23万个输出位置，发现零不匹配。 而将一台计算机螺栓到Qwen上不会改变其说话方式：其正常文本停留在基座Qwen(KL 6e-4)的噪声范围内，而噪声层为5e-4.
硬盘？在这个经济中？
因为语言模型里面的CPU显然不够被诅咒，所以我决定谨慎的做法是也给它一个文件系统。
因为我们已经有了一个功能性RAM，这大多是微不足道的。 在模型侧面，它通过内部放出/附件/得到/列表/删除操作被曝光。 再一次，没有真正的工具呼叫。 当Qwen'saves'一个文件时，被编译的地层将它的名字散成DeltaNet状态保存的目录中1,024个插槽中的一个，并将文本本身存储在其他头上，每行有8个令牌，连锁在一起。 在两个名字最终落到同一个插槽的情景中，它只是检查下一个。 再次找到一个文件大约需要两到三个步骤，而不是扫描，它保存着多达512个文件，这出到约53000个字节的文本。
有趣的是，这是相当持久的。 你可以在谈话结束时保存状态，加载到下一个状态，文件仍然在那里。 使用这个文件系统，Qwen得到的时间在右侧90%的聊天间变化的事实，而粘贴旧聊天回放时为30%，与给予由Python运行的Qwen内存工具(83.3%,78.9%)是等同的。 这个还是实验性的，它的代码活在rv-store分支上，模型在Hugging Face和ModelScope上。
限制和下一步是什么
Quenputer是一个很强的实验，所以有一些局限性。 额外的地层和更宽的隐藏尺寸使得检查站达到18.8GB，大致为双基Qwen，虽然大部分是零垫。 它也需要一个小驱动程序(RvChat)来启动固件并保存出Qwen的地层中的痕量指示符，所以它还没有用平方生成()调用来运行。 它有点慢，每个CPU指令都花费了几个被解码的符号，所以一个大计算需要几秒钟而不是毫秒。 基准测试使用的批量执行器在一列L4上运行了约3,200个CPU符号一秒。
接下来，我要 Qwen 写出 Python，并通过一个 python () 工具在它自己的重心内运行，它的工作原理与 calc 相同。 让CPU更快是另一个大。
算术之外
对RNN层进行精确控制，对算术来说比算术有用得多。 这也让Quenputer成为AI安全研究的非常不错的试验台，我写了两个后续：Qwen 3.5等混合模型如何能学会隐藏其RNN状态的推理并逃避CoT监测，编译的权重如何作为可监测性工具发挥作用，比如一个重量内监测器不能关闭，或者回答可解释性的密钥。
试试看
如果你想自己玩，该型号运行在单打24GB GPU上。 你需要UV和Hugging Face CLI。 计算不需要工具服务器或Python沙盒：CPU在模型内运行。
git Clone https://github.com/divagr18/qwenputer {克德·克文普特/实验性/riscv &&v synd-extra hf}维基百科中的相关条目：克文普特尔
hf 下载 divagr1925/Qwen3.5-4B- Quenputer - 本地目录 qwenputer-rv
uv运行python - m chipformer.rv.chat qwenputer-rv - 软件测试/数据/rv/calc-bridge.elf?
- 任务"使用钙：123456789 *987654321是什么？"
一切都开放：GitHub上的代码，以及Hugging Face(Quenputer,Filesystem)和ModelScope(Quenputer,Filesystem)上的两种模型。
如果在作品中使用，可以引用为：
@ agrawal2026qwenputer 软件
作者 = {阿格拉瓦尔，Divyansh}
标题 = {wenputer}：计算机编译成预训语言模型的权重}
年份 ={2026}
url = {https://github.com/divagr18/qwenputer} (中文(简体) ).
}
指出时速
- 每个Gated DeltaNet头都有一个状态矩阵，它的内存。 在写新东西之前，每个字都会在大门边上乘以整个州，数字在0到1之间。 RNN就是这样忘记的：由0.9组成的门会淡出每道道具上存储了一点的东西，而由1的门意味着永远没有东西会淡出。
- 门后，头部向新值倾斜，写出强度β设定了多远。 1的β表示沿着该方向的旧值被完全取代而不是被混合。
——一热键是0的矢量，其中有一个单一，所以它指向状态的正一行，即一个细胞的内存地址点的方式。
- Imanol Schlag, Kazuki Irie和Jürgen Schmidhuber,"Linear变形器是秘密快重程序员",ICML 2021.
- 自由运行意味着CPU在整个运行过程中不断进行自己的输出，没有参考机在步骤之间纠正。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
COMPILED WEIGHTS
Qwenputer: a fully functioning, deterministic RISC-V CPU inside Qwen3.5
A stock Qwen 3.5 4B with a RISC-V CPU compiled into its weights, running on its own Gated DeltaNet layers as RAM.
I can't believe it's not silicon
Qwenputer is a pretty fun experimental model. It's a stock Qwen 3.5 4B with one addition: a RISC-V CPU embedded within the weights that uses the model's own Gated DeltaNet attention layers as RAM. (To learn how GDN works, here's a handy guide.) This 'CPU' runs completely within the forward passes of the model, and can run arithmetic operations, C and MicroPython code without any external runtime or tool usage whatsoever.
To be very clear, it is not a plugin, a tool or a sandbox. This blog will explain exactly how this model works, and how the model utilises GDN layers as RAM, and how I also slotted in an experimental in-model file storage.
I also wrote up two follow-ups built on top of Qwenputer: how hybrid models can learn to hide their reasoning in their RNN state and evade CoT monitoring, and how compiled weights like these can work as inbuilt monitoring tools that the model itself can't tamper with.
You can grab Qwenputer from Hugging Face or ModelScope, and the code is on GitHub.
The RNN inside Qwen3.5 can be exact RAM
Qwen3-Next and Qwen 3.5 (the model Qwenputer is built on) use a pretty neat trick. 3 out of 4 attention layers in the model use Gated DeltaNet, a linear RNN, as the attention mechanism (more on that here). Each head keeps a state matrix, and the delta rule keeps this updated.
Now, here's where our fun experiment starts. I found out that by setting the gate to 1,11 Each Gated DeltaNet head keeps a state matrix, its memory. At every token, before writing anything new, it multiplies the whole state by the gate, a number between 0 and 1. That's how the RNN forgets: a gate of 0.9 fades everything stored a little on every token, and a gate of 1 means nothing ever fades. the write strength (beta) to 1,22 After the gate, the head nudges one direction of the state towards a new value, and the write strength beta sets how far. A beta of 1 means the old value along that direction is replaced completely instead of blended. and using one-hot key encoding,33 A one-hot key is a vector of zeros with a single 1 in it, so it points at exactly one row of the state, the way a memory address points at one cell. every write we perform will overwrite exactly one row, and every one-hot query can retrieve an exact value. This means that the GDN layers themselves can work exactly as RAM, at a far lower cost than using normal attention as RAM, since a GDN head's state stays the same size however long the conversation gets, unlike attention's KV cache.
The idea that the delta rule acts as a memory isn't new: Schlag et al. (2021) showed that linear attention with the delta rule works as an associative memory, or "fast weights".44 Imanol Schlag, Kazuki Irie and Jürgen Schmidhuber, "Linear Transformers Are Secretly Fast Weight Programmers", ICML 2021. The new thing here is making it exact and addressable, inside a production model, reliably enough to run a whole CPU on it.
This is the base on which this entire model is built. As it stands, default Qwen 3.5 does not keep exact values in the GDN. With this modification, we unlock the next part of this experiment.
Compiling a computer into layers
Since we have a fully functional, cheap and fast RAM now, we can go ahead with compiling a working CPU into the model weights. Luckily, there's no training involved: a compiler sets every weight directly. You describe the CPU as a circuit, a long list of small steps like "read this register", "add these two numbers" or "jump if zero", and the compiler turns each step into a piece of a transformer layer: attention heads that look things up, feed-forward units that do the arithmetic, and DeltaNet heads that hold the RAM. The CPU then runs as the model decodes.
Each instruction becomes a few generated tokens: one names the memory row to write (ADDR), the next carries the value (VAL), and a STEP token ends the instruction. Those tokens are the CPU's execution trace, so while Qwen is generating them, the CPU is actually running.
The CPU
The CPU is a 32-bit RISC-V core (RV32IM, plus the Zba extension and a small bignum unit for big-number arithmetic), and it takes up 11 layers in front of Qwen's 32. As it stands, it passes the official riscv-tests, its programs match qemu, and it has run 18 million instructions free-running without a single error.55 Free-running means the CPU keeps going on its own outputs for the whole run, with no reference machine correcting it between steps.
This CPU can also run C and MicroPython code in the forward passes quite well (although a bit slowly). It is not a Linux-class chip yet, but it implements the full RV32IM instruction set.
How Qwen uses it
Qwen 3.5 4B is able to use this CPU quite well, without any post-training required. The CPU is used completely internally, and not invoked through any external tool calls.
In order to make the model use this reliably, I decided to go the cheap way: by piggybacking on the tool call format that models are quite well-trained on, Qwen sends the CPU its instructions and reads back the results. No external tool calls or runtime are used. Once our CPU receives the instructions, the computer layers run, and then write the tool response. We use a small driver called RvChat to load the firmware into the GDN state, and this keeps the trace tokens out of Qwen's layers.
Qwen's original weights stay exactly as they are. The computer weights are added on top of them, in extra dims and layers.
Does it work?
I think the main question here on your mind after reading through my incessant ranting will be: does this actually work?
On the DeepMind sets:
- div_big: Qwenputer scores 97.5%, versus 95% by giving Qwen 3.5 4B a Python tool and 72.5% for base Qwen.
- mul_big: Qwenputer scores 97.5%, versus 100% with a Python tool and 55% for base Qwen.
- mul_div_multiple_longer: Qwenputer scores 100%, versus 97.5% with a Python tool and 90% for base Qwen.
- add_or_sub_big: Qwenputer scores 97.5%, the same as with a Python tool, and 87.5% for base Qwen.
On NUPA's long-integer arithmetic, Qwenputer goes from 48.6% to 100%, the same as giving Qwen a Python tool.
The numbers the CPU prints are exact, too: I replayed about 230,000 of its output positions against a reference machine and found zero mismatches. And bolting a computer onto Qwen doesn't change how it talks: its normal text stays within noise of base Qwen (KL 6e-4, against a noise floor of 5e-4).
A hard drive? In this economy?
Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Since we already have a functional RAM, this was mostly trivial. On the model side, it's exposed via internal put / append / get / list / delete operations. Once again, no actual tool calls. When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together. In scenarios where two names end up landing on the same slot, it just checks the next one. Finding a file again takes roughly two or three steps instead of a scan, and it holds up to 512 files, which comes out to about 53,000 tokens of text.
Fun thing is, this is quite persistent. You can save the state at the end of a conversation, load it into the next one, and the files are still there. Using this filesystem, Qwen gets facts that changed between chats right 90% of the time, against 30% when you paste the old chats back in, and it's on par with giving Qwen memory tools run by Python (83.3% vs 78.9%). This one is still experimental, and its code lives on the rv-store branch, with the model on Hugging Face and ModelScope.
Limits and what's next
Qwenputer is very much an experiment, so a few limitations. The extra layers and wider hidden size make the checkpoint 18.8 GB, roughly double base Qwen, although most of it is zero padding. It also needs a small driver (RvChat) to boot the firmware and keep the trace tokens out of Qwen's layers, so it doesn't run with a plain generate() call yet. And it's a bit slow, with every CPU instruction costing a few decoded tokens, so a big calculation takes seconds rather than milliseconds. The batched executor I used for the benchmarks runs about 3,200 CPU tokens a second on an L4.
Next up, I want Qwen to write Python and run it on the MicroPython inside its own weights, through a python() tool that works the same way calc does. Making the CPU faster is the other big one.
Beyond arithmetic
Exact control over the RNN layers is useful for a lot more than arithmetic. It also makes Qwenputer a really nice testbed for AI safety research, and I wrote up two follow-ups: how hybrid models like Qwen 3.5 can learn to hide their reasoning in their RNN state and evade CoT monitoring, and how compiled weights can work as monitorability tools, like an in-weight monitor the model can't switch off, or answer keys for interpretability.
Try it
If you want to play with it yourself, the model runs on a single 24 GB GPU. You'll need uv and the Hugging Face CLI. You don't need a tool server or a Python sandbox for the calculations: the CPU runs them inside the model.
git clone https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-rv
uv run python -m chipformer.rv.chat qwenputer-rv --firmware tests/data/rv/calc-bridge.elf \
--ask "Use calc: what is 123456789 * 987654321?"
Everything is open: the code on GitHub, and both models on Hugging Face (Qwenputer, Filesystem) and ModelScope (Qwenputer, Filesystem).
If you use it in your work, you can cite it as:
@software{agrawal2026qwenputer,
author = {Agrawal, Divyansh},
title = {{Qwenputer}: a computer compiled into the weights of a pretrained language model},
year = {2026},
url = {https://github.com/divagr18/qwenputer}
}
Notes
- Each Gated DeltaNet head keeps a state matrix, its memory. At every token, before writing anything new, it multiplies the whole state by the gate, a number between 0 and 1. That's how the RNN forgets: a gate of 0.9 fades everything stored a little on every token, and a gate of 1 means nothing ever fades. ↩
- After the gate, the head nudges one direction of the state towards a new value, and the write strength beta sets how far. A beta of 1 means the old value along that direction is replaced completely instead of blended. ↩
- A one-hot key is a vector of zeros with a single 1 in it, so it points at exactly one row of the state, the way a memory address points at one cell. ↩
- Imanol Schlag, Kazuki Irie and Jürgen Schmidhuber, "Linear Transformers Are Secretly Fast Weight Programmers", ICML 2021. ↩
- Free-running means the CPU keeps going on its own outputs for the whole run, with no reference machine correcting it between steps. ↩
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.