Stalwart v1.0：星球上最快的邮件服务器
5年前，2021年10月3日，第一个承诺降落在一个名为"mail-parser"的小型仓库。 这是一个电子邮件解析图书馆用Rust，仅此而已。 在接下来的五年里，它发展成为了与JMAP,IMAP,POP3,SMTP,CalDAV,CardDAV和WebDAV的完整邮件和协作服务器，一个垃圾邮件过滤器，集群，并支持了半打存储后端。 今天，经过了五年的持续发展，我们自豪地宣布了Stalwart v1.0，这是第一个稳定的发行。
1.0版本从来就不是另一个功能发布。 它有两个目标。 第一个是性能：使服务器的每个部分尽可能快地，用数字来证明。 第二是数据本身的稳定性：最终确定数据库计划，以便从现在起，升级Stalwart意味着安装一个新的版本，而仅此而已，而不需要一些较早的发布所需的破坏性迁移。 在此过程中，一些新功能也找到了它们的方式，包括PDF和办公室文件的全文搜索。
让我们走过去。
每微秒追逐几个月
题为“每月追逐每一微秒”的一节
本版中的大部分作品发生在一个剖析器下。 几个月来，我们测量了Stalwart的 CPU 周期、内存和数据库的圆行情况，改写出名之处，并再次测量。 释放中包含有近200项承诺触及了1300多起档案，其中绝大多数都存在，以制造出更快或更轻的东西。
用 SIMD 和 SWAR 重写解析器
题为“用SIMD和SWAR重写纸张”的章节
一个邮件服务器花费了一个惊人的时间来寻找特定的字节：一个信头行的结束，一个MIME边界，一个被引用的可打印体中的等同符号，一个结束一个IMAP标记的空间。 一次完成一个字节，对于一个 10 MB 消息，有很多循环重复。 两个技术让一个CPU一次做这个工作很多字节：
- SIMD(单一指令，多数据)使用现代处理器(SSE2,SSSE3和AVX2在x86-64上，NEON在ARM上)的宽矢量登记册，用单一指令来比较或转换16或32字节。
- SWAR(SIMD Into A Register) 用普通的64位整数来应用同样的想法：几个精心选择的算术和比特操作一次测试一个字的所有八个字节。 它不需要特殊的指示，所以在每个CPU上都有效。
斯达尔瓦特现在使用两者，在任何地方都很重要。 堆栈底部坐落着两个与服务器一起达到1.0的库：
- 邮件-parser 1.0, Stalwart 开始的图书馆，从头开始被改写。 SIMD内核定位MIME边界和头字段，SWAR处理头字名，解析后的消息活在少数平面向量中，而身体只有在有人实际读取时才被解码。 CPU所特有的内核在运行时被取出，每个都被模糊地与平地的scalar参照。 比起其他分析器，它比流行的Rust mailparse箱子快3倍，比多维科特自己的分析器快9倍，比libEtPan快15至31倍！
- 编码是一个新的编码库，为Base64,Base32提供SIMD内核并可以被引用的可打印的编码，即电子邮件中几乎每个附件和非ASCII头的编码。 在AMD EPYC服务器上解码一个MIME被包装的 Base64附件比标准Rust Base64 箱子快22倍，被引用的可打印解码比被引用的可打印箱子快9倍。 它现在处理服务器中每个 Base64，可引用的可打印和IMAP UTF-7转换。
在它们上面，协议层得到同样的处理。 IMAP命令解析器现在由表格驱动，扫描整行字节，而不是每个字符踩一个状态机器。 SMTP命令解析器，JMAP后方的JSON解析器，垃圾邮件滤波器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器活化器。 即使是管理员用于路由和过滤决定的表达式语言，现在也编译为有专门指令的紧凑字节码来进行共同比较，并在再生内存竞技场对其进行评价。
访问数据库的速度更快
题为“Fewer，更快地访问数据库”的章节
快速解析被浪费如果每一个请求然后等待在数据库，所以我们花了同样长的时间关于如何 Stalwart 谈论它的存储。 身份识别器现在被分配到使用它们的同一写入交易中，从每封信件的发送中去除整个回程。 读取-修改-写入更新，如在搜索索引中添加关键字或附加，作为后端内的合并操作运行。 被存储的对象使用zstd使用专门受过电子邮件、日历和联系数据培训的词典进行压缩。
每个后端也分别调制：
- RocksDB：每一种数据现在都生活在一个为如何访问而配置的柱状家族中，有开花的滤波器用于点检，大块用于扫描，而收缩则用于调制数据。 排队配额等热量计数器被分布在最多256个档位来避免写出冲突。
- PostgreSQL和MySQL：预先生成和缓存准备的语句，用自动重试来块化迭代，每作回回行会减少。
- FoundationDB：读取：交易需要先在平行的 windows 中进行预接，且大值以块组装。
- SQLite：页面大小、缓存、内存映射和WAL设置，用于邮件工作量。
存储层上方坐落了重新设计的缓存子系统。 每个账户的信息索引以紧凑的打包形式保存在内存中，IMAP的排序，线程，取旗，搜索消息属性和再同步，现在从该缓存中解答，读取的数据库很少或根本没有。 非常大的账户会用内存预算获得自己的缓存级，缓存快照可以交换到本地磁盘，Redis或blob商店来代替重建。
一个新的搜索引擎
题为“新搜索引擎”的一节
内置的全文搜索引擎从地上重写。 上个设计每字存储一个位图，没有字位的notion. 新的引擎以65 536个文件块储存张贴，并保留每条消息的文字位置。 这使得真正的短语搜索成为可能，让一个信箱内部的搜索只读取信箱所触及的块。 它索引词源和前缀，用zstd来压缩其数据，通过背景队列来索引新邮件，并带有货币控制，重试和重建等功能。 PostgreSQL和MySQL搜索后端也进行了重修。 在一个25万条消息的邮箱里对每个消息体的全文搜索现在返回不到4毫秒
对数百个VCPU进行了压力测试
题为“对数百个VCPU进行压力测试”的一节
优化代码而不在现实的负载下进行测量是猜想，所以我们建立了一个专门的基准和压力测试实验室来进行这个发布。 它运行于Microsoft Azure上，跨越了30多个区域，多达18个完整的测试实验室平行运行，几百个vCPU同时繁忙。
试验目录包括35种试验，分布在550多个单个试验细胞，它们包括：
- 每个主要协议：IMAP、JMAP(包括推出事件源和WebSocket)、SMTP入境、提交和中继、CalDAV、CardDAV和WebDAV。
- PostgreSQL, FoundationDB和RocksDB作为数据存储器，Redis作为内存存储器，S3和文件系统作为blob存储器，以及混合负载下的三节点集群。
- 大型数据集：500个账户的160万条信息、75万个日历和联系项目、1 200万个和500万条信息的个人邮箱、1 000个单一服务器上的账户、1 000万条信息的邮件排队、DMARC和TLS报告100万份报告的洪水。
- 暴风雨和虐待：每秒有2万个新连接，大量登录，每个账户有数千个IDLE和推开订户，有敌意和畸形的信息，慢读客户端和超大字号。
- 6小时的浸泡测试，寻找内存增长，缓存漂移和缓漏，包括一次通过MTA推出超过一千万个消息。
整个运行情况都有所描述。 CPU时间被用perf进行采样并转化为火焰图，离CPU分析显示线程在锁和I/O上等待的地方，堆积被以jemalloc和heaptrack来描述，一个特殊的仪器堆积计算每一次存储回程，每个指令，这样我们就能看到究竟哪个请求比他们应该的更点击数据库。 每种固定装置在被接受之前，先用相配的机器来比照以前的装置进行测量。
这一周期的第一轮全轮共出现64个问题，这些问题被固定并重新测试。 第二轮重排一切，再修33个。 实验室捕捉到的一些例子：
- 重开一个500万条留言的邮箱从61米到0.21米
- 一个有500万条信息的账户的JMAP查询从3.3秒到12分
- RocksDB的高峰入境邮件吞吐量几乎翻了一番，从每秒9 690通到18 394通。
- 一个内存问题，即当实际堆积低于0.4 GB时，过程维持了9.6 GB，发现并固定了。 在6小时的浸泡测试中记忆力保持平平
这个实验室不是一次性的释放努力。 现在，这是Stalwart如何在内部进行测试的一部分：今后每一次释放都要经过同一种基准测试和应力测试后再出船，所以性能收益保持得到。
比 Postfix 和 Dovecot 更快
题为“快于后修和多维科特”的一节
内部数字只告诉你斯塔尔瓦特是否比它快。 我们还想知道它如何与世界上大部分地区的情况相比较。 因此，我们把Stalwart v1.0头放在最广泛部署的开源邮件堆：SMTP的后缀3.11和IMAP的多维可特2.4.
两人运行在相同的Azure服务器上 (8 AMD EPYC Zen 5 vCPUs, 32 GiB of RAM，本地NVMe存储)，装入相同的1000个邮箱外加3个非常大的信箱，最多可达25万个消息，并收到完全相同的工作：接收邮件，发送邮件并提供给IMAP客户端。 加密，密码散列和全文搜索在两者的配置上都相同，两个系统在同一硬件上交替测量。
Stalwart在38个标题比较中的35个比较快。
其中一些结果：
- IMAP:Stalwart在15个日常客户业务中的14个比多维科特快，比1.4到19倍。 完全TLS握手的登录需要5.1 ms而不是28 mss，取出1 MB消息的速度快了6倍，将INBOX的线程快了5倍，移动或删除100个消息的速度快了18到19倍。
- SMTP:Stalwart接受比Postfix快3.6到12倍的电文，在Postfix的723上维持每秒5,445个接发电文。
- 交付：一个消息提前16次对IMAP客户端可见，等待IDLE的客户端以3.3 ms而不是半秒通知。
- 装入：Stalwart每秒同时服务5.4倍的IMAP会话，并在邮件开始减速前转发3.8倍。
差距随着邮箱的大小而扩大。 在一个25万话信箱上，每个话文机体的全文搜索在斯塔尔瓦特需要3.75毫秒，多维科特需要136毫秒，移动或删除2.5万个话文在大约十分之一秒内完成，而不是24至34秒：快了200多倍。
三次测试 Dovecot won都是相同的操作：将每条消息都标记在一个邮箱中，同时读取。
只有Postfix和Dovecot进行了测试，但它们并不仅仅是两种产品。 它们几乎是每个受欢迎的自寄邮件服务器下的引擎：每台开源多颗恒星在GitHub上的多发邮件服务器，包括docker邮件服务器和mailcow，在引擎盖下运行Postfix和Dovecot,Mail-in-a-Box也一样。 在几乎每个基准测试上击败这些组合意味着在互联网上多数自办邮件后面击败了堆叠。 我们是
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Stalwart v1.0: The Fastest Mail Server on the Planet
Five years ago, on 3 October 2021, the first commit landed in a small repository called mail-parser. It was an e-mail parsing library written in Rust, and nothing more. Over the following five years it grew into a complete mail and collaboration server with JMAP, IMAP, POP3, SMTP, CalDAV, CardDAV and WebDAV, a spam filter, clustering, and support for half a dozen storage backends. Today, after five years of continuous development, we are proud to announce Stalwart v1.0, the first stable release.
Version 1.0 was never meant to be another feature release. It had two goals. The first was performance: making every part of the server as fast as it can be, and proving it with numbers. The second was stability of the data itself: finalising the database schema so that, from now on, upgrading Stalwart means installing a new version and nothing more, without the disruptive migrations some earlier releases required. Along the way a few new features found their way in too, including full-text search inside PDF and office documents.
Let’s walk through it.
Months spent chasing every microsecond
Section titled “Months spent chasing every microsecond”
Most of the work in this release happened under a profiler. For months we measured where Stalwart spends its CPU cycles, its memory and its database round trips, rewrote whatever stood out, and measured again. The release contains close to 200 commits touching more than 1,300 files, and the vast majority of them exist to make something faster or lighter.
Parsers rewritten with SIMD and SWAR
Section titled “Parsers rewritten with SIMD and SWAR”
A mail server spends a surprising share of its time looking for specific bytes: the end of a header line, a MIME boundary, an equals sign in a quoted-printable body, the space that ends an IMAP tag. Done one byte at a time, that is a lot of loop iterations for a 10 MB message. Two techniques let a CPU do this work many bytes at a time:
- SIMD (Single Instruction, Multiple Data) uses the wide vector registers of modern processors (SSE2, SSSE3 and AVX2 on x86-64, NEON on ARM) to compare or transform 16 or 32 bytes with a single instruction.
- SWAR (SIMD Within A Register) applies the same idea with ordinary 64-bit integers: a few carefully chosen arithmetic and bitwise operations test all eight bytes of a word at once. It needs no special instructions, so it works on every CPU.
Stalwart now uses both, everywhere it matters. At the bottom of the stack sit two libraries that reach 1.0 together with the server:
- mail-parser 1.0, the very library where Stalwart began, has been rewritten from scratch. SIMD kernels locate MIME boundaries and header fields, SWAR handles header names, the parsed message lives in a handful of flat vectors, and bodies are only decoded when someone actually reads them. The CPU-specific kernel is picked at runtime, and every one is fuzzed against a plain scalar reference. Against other parsers it is 3 times faster than the popular Rust mailparse crate, 9 times faster than Dovecot’s own parser, and 15 to 31 times faster than libEtPan!, VMime and GMime on a full parse.
- encodify is a new encoding library with SIMD kernels for Base64, Base32 and quoted-printable, the encodings that carry nearly every attachment and non-ASCII header in e-mail. Decoding a MIME-wrapped Base64 attachment is 22 times faster than with the standard Rust base64 crate on an AMD EPYC server, and quoted-printable decoding is 9 times faster than with the quoted_printable crate. It now handles every Base64, quoted-printable and IMAP UTF-7 conversion in the server.
Above them, the protocol layer got the same treatment. The IMAP command parser is now table-driven and scans whole runs of bytes instead of stepping a state machine per character. The SMTP command parser, the JSON parser behind JMAP, the spam filter tokenizers, the HTML-to-text converter, the variable-length integer codec and the log serializers all use SWAR scanning. Even the expression language, which administrators use for routing and filtering decisions, now compiles to a compact bytecode with specialised instructions for common comparisons and evaluates them in a recycled memory arena.
Fewer, faster trips to the database
Section titled “Fewer, faster trips to the database”
Fast parsing is wasted if every request then waits on the database, so we spent just as long on how Stalwart talks to its storage. Identifiers are now assigned inside the same write transaction that uses them, removing whole round trips from every message delivery. Read-modify-write updates, such as adding a keyword or appending to a search index, run as merge operations inside the backend. Stored objects are compressed with zstd using dictionaries trained specifically on e-mail, calendar and contact data.
Each backend was also tuned individually:
- RocksDB: every kind of data now lives in a column family configured for how it is accessed, with bloom filters for point lookups, large blocks for scans and compaction tuned for data that churns. Hot counters, such as queue quotas, are spread across up to 256 slots to avoid write conflicts.
- PostgreSQL and MySQL: pre-generated and cached prepared statements, chunked iteration with automatic retry, and fewer round trips per write.
- FoundationDB: reads that a transaction will need are prefetched in parallel windows, and large values are assembled in chunks.
- SQLite: page size, cache, memory mapping and WAL settings tuned for a mail workload.
On top of the storage layer sits a redesigned cache subsystem. Each account’s message index is held in memory in a compact packed form, and IMAP sorting, threading, flag fetches, searches on message properties and resynchronisation are now answered from that cache with few or no database reads. Very large accounts get their own cache tier with a memory budget, and cache snapshots can be swapped out to local disk, Redis or the blob store instead of being rebuilt.
A new search engine
Section titled “A new search engine”
The built-in full-text search engine was rewritten from the ground up. The previous design stored one bitmap per word and had no notion of word positions. The new engine stores postings in blocks of 65,536 documents and keeps word positions for every message. That makes real phrase searches possible, and lets a search inside one mailbox read only the blocks that mailbox touches. It indexes word stems and prefixes, compresses its data with zstd, and indexes new mail through a background queue with concurrency control, retries and rebuilds. The PostgreSQL and MySQL search backends were reworked as well. On a mailbox of 250,000 messages, a full-text search of every message body now returns in under 4 milliseconds.
Stress tested on hundreds of vCPUs
Section titled “Stress tested on hundreds of vCPUs”
Optimising code without measuring it under realistic load is guesswork, so we built a dedicated benchmarking and stress-testing lab to go with this release. It runs on Microsoft Azure, across more than 30 regions, with up to 18 complete test labs running in parallel and several hundred vCPUs busy at once.
The test catalogue covers 35 kinds of test, spread over more than 550 individual test cells, and they cover:
- Every major protocol: IMAP, JMAP (including push over EventSource and WebSocket), SMTP inbound, submission and relay, CalDAV, CardDAV and WebDAV.
- PostgreSQL, FoundationDB and RocksDB as data stores, Redis as the in-memory store, S3 and the filesystem as blob stores, and a three-node cluster under mixed load.
- Large datasets: 1.6 million messages and 750,000 calendar and contact items across 500 accounts, individual mailboxes of 1, 2 and 5 million messages, 100,000 accounts on a single server, mail queues of 10 million messages, and DMARC and TLS report floods of a million reports.
- Storms and abuse: 20,000 new connections per second, mass logins, thousands of IDLE and push subscribers per account, hostile and malformed messages, slow-loris clients and oversized literals.
- Six-hour soak tests that look for memory growth, cache drift and slow leaks, including one that pushed over 10 million messages through the MTA.
Runs were profiled throughout. CPU time was sampled with perf and turned into flame graphs, off-CPU analysis showed where threads waited on locks and I/O, the heap was profiled with jemalloc and heaptrack, and a special instrumented build counted every storage round trip, per command, so we could see exactly which requests hit the database more than they should. Each fix was then measured against the previous build on matched machines before it was accepted.
The first full round of this cycle turned up 64 problems, which were fixed and retested. The second round re-ran everything and fixed 33 more. Some examples of what the lab caught:
- Reopening a mailbox with 5 million messages went from 61 ms to 0.21 ms.
- A JMAP query on an account with 5 million messages went from 3.3 seconds to 12 ms.
- Peak inbound mail throughput on RocksDB nearly doubled, from 9,690 to 18,394 messages per second.
- A memory issue where the process held up to 9.6 GB while the actual heap was under 0.4 GB was found and fixed. In six-hour soak tests memory now stays flat.
This lab is not a one-off effort for the release. It is now part of how Stalwart is tested internally: every future release goes through the same benchmark and stress-test rounds before it ships, so performance gains stay gained.
Faster than Postfix and Dovecot
Section titled “Faster than Postfix and Dovecot”
Internal numbers only tell you whether Stalwart got faster than it was. We also wanted to know how it compares to what most of the world runs. So we put Stalwart v1.0 head to head with the most widely deployed open-source mail stack: Postfix 3.11 for SMTP and Dovecot 2.4 for IMAP.
Both ran on identical Azure servers (8 AMD EPYC Zen 5 vCPUs, 32 GiB of RAM, local NVMe storage), loaded with the same 1,000 mailboxes plus three very large ones of up to 250,000 messages, and received exactly the same work: receiving mail, delivering it and serving it to IMAP clients. Encryption, password hashing and full-text search were configured the same way on both, and the two systems were measured alternately on the same hardware.
Stalwart was faster in 35 of the 38 headline comparisons.
Some of the results:
- IMAP: Stalwart is faster than Dovecot in 14 of 15 everyday client operations, by 1.4 to 19 times. A login with a full TLS handshake takes 5.1 ms instead of 28 ms, fetching a 1 MB message is 6 times faster, threading the INBOX 5 times faster, and moving or deleting 100 messages 18 to 19 times faster.
- SMTP: Stalwart accepts a message 3.6 to 12 times faster than Postfix, and sustains 5,445 incoming messages per second against Postfix’s 723.
- Delivery: a message is visible to an IMAP client 16 times sooner, and a client waiting in IDLE is notified in 3.3 ms instead of half a second.
- Load: Stalwart serves 5.4 times more concurrent IMAP sessions per second and relays 3.8 times more mail before it starts to slow down.
The gap grows with the size of the mailbox. On a 250,000-message mailbox, a full-text search of every message body takes 3.75 ms in Stalwart and 136 ms in Dovecot, and moving or deleting 25,000 messages finishes in about a tenth of a second instead of 24 to 34 seconds: more than 200 times faster.
The three tests Dovecot won are all the same operation: marking every message in a mailbox as read at once.
Only Postfix and Dovecot were tested, but they are not just any two products. They are the engine under nearly every popular self-hosted mail server: every open-source dockerized mail server with more than 10,000 stars on GitHub, including docker-mailserver and mailcow, runs Postfix and Dovecot under the hood, as does Mail-in-a-Box. Beating that combination on almost every benchmark means beating the stack behind most of the self-hosted mail on the internet. We are
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.