2美元微控制器上的SIMD加速计算机视觉
我为ESP32-S3写了加速FAST特性探测器，利用了它的128位SIMD指令。 它大约实现参考执行的性能的两倍，并需要约~6ms来处理一个QVGA(320x240)帧。 您可以在这里找到源代码。
就其价格而言，ESP32-S3是微控制器的动力所。 在其不假定的塑料包中，有一个最多240MHz的双核心CPU，其外围部分包括WiFi和蓝牙低能无线电。 在挖掘其技术参考手册时，我发现芯片支持一套有限的SIMD指令。 对于比普通咖啡便宜的硅来说，这很酷。 让我们看看我们能用它做什么令人兴奋的事吧！
在撰写SIMD加速的FAST角探测器的过程中，我在ESP32-S3上学习了组装的基本原理，围绕其古怪的局限性而工作，最终写出我自己的基本收存器，活塞。
我意识到这些指示并非完全秘密，我想用他们自己做实验写出比我在网上找到的更先进的代码由于我最近对计算机视觉非常感兴趣，我决定为ESP32-S3编写SIMD加速执行FAST特性探测器。
我开始寻找使用SIMD指令的代码在编写本报告时，我只能找到三个来源：
- Larry Bank的博客
- Espressif的esp-dsp库，用于DSP-sque特性(进取、FFT等)
- Espressif的esp-dl库，用于芯片AI加速。
我感到很欣慰地发现ESP32-S3上的组装并没那么糟糕。 我决定这个总体结构：
.c节.rodata
# 这里只读数据
. 节。文本
对齐 4
. 全球添加
. 类型添加，@ 函数
# 一个简单的函数，新增两个数字
# int32 t 添加( int32 t a, int32 t b)
添加：
条目 sp, 48 # sp (stack 指针) 是 a1 的别名
# 定义 r a 2
# 定义 r b a3
# 这里的代码
添加 a2, r a, r b # a2 = a +b
retw.n # 返回，其中a2包含返回值。
从那里开始，我开始写一个SIMD加速版的FAST角预测试。 基本上，它在每个主方向取样一个像素，如果至少有三个“极端”像素，则检查。 在这种情况下，一个极端像素就是其与中像素的绝对相差在一定阈值上。
我决定了一个函数，可以计算一次十六像素的极限数，因为每个矢量寄存器可以存储十六个八位值。 这将通过取样每个主要方向的4个像素的“圆通”并把它们与各自的中像素作比较来实现。
但这是我遇到的第一个限制，在中像素对齐的假设下，东西两块会错相：
幸好芯片有指示，可以将一分两个相接的收发机提取入另一个收发机。 利用这个功能，我能够通过阅读和取出与中区相邻的两个区块的切片来取出东西两块。
好吧，现在我有像素数据。 是时候做一些比较了吧？错。 结果ESP32-S3只执行签名的八位数的比较。 这使得处理更加困难，因为我所有的像素数据都被存储为未签名的8位数字。 如果你只是把未签名的数据重新解释为签名数据，你会得到一些奇怪的结果：
np.int8(np.uint8(范围(0,256,8)))).
数组([0、8、16、24、32、40、48、56、64、72、80]
88, 96, 104, 112, 120, -128, -120, -112, -104, -96, -88,
-80, -72, -64, -56, -48, -40, -32, -24, -16, -8],
d类型=int8)
范围[128-255]得到非线性地映射到[-128-1]，打破了所有比较操作。 为了做到这一点，我需要把数值范围从[0,255]转移到[-128,127]. 这应该是一个简单的减法，但指令集再次受阻：在ESP32-S3上，所有SIMD的算术操作都是饱和的，这意味着所有超过127的像素值最终都会被夹在-128，为了克服这个问题，我不得不在不使用减法指示的情况下，从每个数字平行地减去128个。 对我来说，这起初似乎是一项艰巨的任务，但最终我找到了解决办法。
让我们假设x是小品。 以下是它的位数值：
-128 64 32 16 8 4 2 1
发现什么有趣的事了吗？减法与-128的加法相同，因此计算出x-128，我们只要在x的最显要数字中再加一到一到一到一到一二八，就可以"递增"数字。 我们可以通过翻出x的最高位来计算这个。 由于用一个来擦去任意一分，与取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取取因此，x-128 == ^ 0x80.
现在，当未签名的八位值被加载时，它们会被用 0x80 的 XOR'd 转换成一个适当的(线性)范围，使算术和比较再次有意义。
有了这两个把戏，我能够绕过ESP32-S3的限制，并写出SIMD加速的FAST角预测试。 我也用类似的操作来写一个SIMD加速评分功能
最终，我得以将FAST特性探测器的吞吐量提高约220%，从5.1MP/s到11.2MP/s在我的测试中。 这完全在实时计算机视觉任务的可接受性能范围内，使得ESP32-S3能够轻松地处理出30fps VGA流。 两块还不错！
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
SIMD-accelerated computer vision on a $2 microcontroller
I wrote an accelerated FAST feature detector for the ESP32-S3 that leverages its 128-bit SIMD instructions. It achieves roughly double the performance of the reference implementation, and takes around ~6ms to process a QVGA (320x240) frame. You can find the source code here.
For its price, the ESP32-S3 is a powerhouse of a microcontroller. Within its unassuming plastic package lies a dual-core CPU running at a maximum of 240MHz with a slew of peripherals, including WiFi and Bluetooth Low Energy radios. While digging through its technical reference manual I discovered that the chip supports a limited set of SIMD instructions. For silicon that’s cheaper than the average coffee, that’s pretty cool. Let’s see what exciting things we can do with it!
In the process of writing a SIMD-accelerated FAST corner detector, I learned the basics of assembly on the ESP32-S3, worked around its eccentric limitations, and ended up writing my own basic register allocator, basm.
After some more web browsing, I realized that these instructions weren’t exactly secret, but rather relatively unknown. I wanted to experiment with them myself and write more advanced code than what I could find online. As I’ve been really interested in computer vision lately, I decided on writing a SIMD-accelerated implementation of the FAST feature detector for the ESP32-S3.
I started by looking for example code that uses SIMD instructions. At the time of writing, I could only find three sources:
- Larry Bank’s blog, where he explores the feature and provides a minimal example
- Espressif’s esp-dsp library, used for DSP-esque features (convolutions, FFT, etc.)
- Espressif’s esp-dl library, used for on-chip AI acceleration.
After perusing through them, I was relieved to find that assembly on the ESP32-S3 isn’t that bad. I settled on this general structure:
.section .rodata
# read-only data here
.section .text
.align 4
.global add
.type add,@function
# a simple function that adds two numbers
# int32_t add(int32_t a, int32_t b)
add:
entry sp, 48 # sp (stack pointer) is an alias for a1
#define r_a a2
#define r_b a3
# code here
add a2, r_a, r_b # a2 = a + b
retw.n # return, with a2 containing the return value.
From there, I started writing a SIMD-accelerated version of the FAST corner pre-test. Essentially, it samples one pixel in each cardinal direction and checks if there are at least three “extreme” pixels. In this case, an extreme pixel is one whose absolute difference from the center pixel is above a certain threshold.
I decided on a function that would calculate the extrema count of sixteen pixels at a time, since each vector register can store sixteen eight-bit values. This would be achieved by sampling four “chunks” of pixels in each cardinal direction and comparing them to their respective center pixels.
However, this is where I ran into my first limitation- the ISA doesn’t allow for direct misaligned reads. Under the assumption that the center pixel would be aligned, the east and west blocks would be misaligned:
Thankfully, the chip has an instruction that allows for extracting a slice of two concatenated registers into another one. Using this functionality, I was able to extract the east and west blocks by reading and extracting slices from the two blocks adjacent to the center block.
Okay, so now I have the pixel data. Time to do some comparisons, right? Wrong. Turns out the ESP32-S3’s only implements comparisons for signed eight-bit numbers. This made processing a little more difficult, because all my pixel data was stored as unsigned eight-bit numbers. If you just reinterpreted the unsigned data as signed data, you get some weird results:
>>> np.int8(np.uint8(range(0, 256, 8)))
array([ 0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80,
88, 96, 104, 112, 120, -128, -120, -112, -104, -96, -88,
-80, -72, -64, -56, -48, -40, -32, -24, -16, -8],
dtype=int8)
The range [128 255] gets nonlinearly mapped to [-128 -1], breaking all compare operations. To make this work, I needed to shift the range of the values from [0, 255] to [-128 127]. This should’ve been a simple subtraction, but the instruction set got in the way again: on the ESP32-S3, all the SIMD arithmetic operations are saturating, which meant that all pixel values above 127 would end up being clamped at -128 instead of underflowing to their appropriate values. To overcome this problem, I had to somehow subtract 128 from each number in parallel without using the subtraction instruction. This seemed like a daunting task to me at first, but I eventually arrived at a solution.
Let’s assume that x is the minuend. Here is its digits’ place values:
-128 64 32 16 8 4 2 1
Notice anything interesting? Subtraction is the same as addition by -128, so to compute x - 128, we can just “increment” the number by -128 by adding one to x’s most significant digit, whose place value is -128. We can compute this by simply flipping the highest bit of x. Since XORing any bit with one is the same as taking the bitwise NOT, we can selectively flip the highest bit of x by XORing it with the bit mask 0b10000000, which equal to 0x80. Therefore, x - 128 == x ^ 0x80.
Now, whenever unsigned eight-bit values are loaded in, they are XOR’d with 0x80 to convert them an appropriate (linear) range where the arithmetic and comparisons make sense again.
With those two tricks, I was able to circumvent the limitations of the ESP32-S3 and write the SIMD-accelerated FAST corner pre-test. I also wrote a SIMD-accelerated scoring function using a similar set of operations but I won’t be writing about that here.
In the end, I was able to improve the throughput of the FAST feature detector by about 220%, from 5.1MP/s to 11.2MP/s in my testing. This is well within the acceptable range of performance for realtime computer vision tasks, enabling the ESP32-S3 to easily process a 30fps VGA stream. Not bad for $2!
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.