Hardwired Distributed Processing Unit: Pushing Neural Network Decisions to the Sensor

Traditional AI inference typically relies on high-performance processors or GPU clusters, but a recent study demonstrates a completely different approach: using a hardwired distributed processing unit (HDPU) to make neural network decisions directly at the sensor, thereby fully freeing the host. Using crosswalk detection as a case study, the research implements a complete pipeline from pixels to decisions on a low-cost Artix-7 FPGA, offering an ultra-low-power, ultra-low-latency hardware solution for edge AI.

From Preprocessing to Decision: The HDPU Paradigm Shift

Early HDPUs mainly handled preprocessing tasks such as edge detection, corner extraction, or steering control. This study moves the decision itself to the sensor. The unit completes inference within a fixed number of clock cycles per frame, and the entire detection path—from camera pixels to the final decision—occupies only 1,814 LUTs, requiring no processor, operating system, or full-resolution frame storage. This means inference no longer needs host involvement; the sensor itself can output structured results.

The key to this design is hardwiring a binarized convolutional neural network (CNN) into FPGA logic. The network has four layers, 93,408 one-bit weights, and a pipelined 80x60 pixel input. Through bit-exact comparison from the Python model to RTL simulation and then to board-level verification, the team ensured that hardware behavior matches the software model exactly. On all 639 test images, the board returned scores and decisions identical to the bit-exact software model.

Binarized Neural Network (BNN) is a class of neural networks that constrain both weights and activations to +1 or -1. Since multiplication in the binary domain can be reduced to XNOR and popcount operations, BNNs can be implemented very efficiently on FPGAs or ASICs—large numbers of one-bit weights can be packed into very small logic resources, with no need for floating-point multipliers or large on-chip storage. The cost is usually lower expressiveness and accuracy than full-precision networks, so they are often used in extremely resource-constrained edge scenarios with relatively fixed tasks. The 93,408 one-bit weights in the article are exactly a manifestation of this property: weights are固化 in logic as bits, and inference incurs almost no storage bandwidth overhead.

Performance and Accuracy: Reliable Decisions at Fixed Latency

The unit completes its decision within a fixed 2,269,477 clock cycles per frame; this latency depends on the FPGA clock frequency but always falls within the frame period. When the unit gives a definite answer (rather than "uncertain"), its accuracy on street-view photos reaches 95.7%. As a detail, the hardware contrast enhancement stage raises the accuracy on dashcam-like camera frames from 47.7% to 78.2%, showing the significant gain of hardware preprocessing in real-world scenarios.

Compared with running full-precision ResNet-18 on the host, the HDPU's advantage is not only latency. The host saves 8.13 milliseconds of core time per frame, while full-precision ResNet-18 even with 16 threads needs more than one frame period. In addition, the host receives only 16 bytes of data per frame instead of 614,400 bytes—a 38,400-fold reduction in data volume. This order-of-magnitude reduction is crucial for edge devices sensitive to bandwidth and power consumption.

LUT (Look-Up Table) is the most basic programmable logic resource in an FPGA, typically implemented as a 4-input or 6-input arbitrary Boolean function, and often combined with flip-flops and carry chains into logic cells. The number of LUTs used is a common metric for judging whether an FPGA design is "lightweight," because it directly reflects chip area consumption. Artix-7 belongs to the low-cost, low-power branch of the Xilinx 7 series, with a single chip typically providing tens of thousands to hundreds of thousands of LUTs. The entire detection path in the article occupies only 1,814 LUTs, meaning it consumes only a small fraction of the chip resources, leaving room for integrating more sensor processing units or more complex logic on the same FPGA.

Bit-exact comparison is a strict standard for verifying numerical consistency in hardware implementations: it requires not only that the hardware output matches the software model in classification results, but also that intermediate activations, scores, and even the final decision are compared bit by bit and are completely identical. Since all operations in a binarized network are integer or Boolean logic, there is theoretically no floating-point rounding error, so bit-exactness is an achievable and verifiable goal. The team exported parameters from the Python model, reproduced them in RTL simulation, and then measured on the board; all three layers of comparison passed, ruling out common pitfalls such as quantization errors, timing races, and bit-width truncation. This is key evidence for the trustworthiness of a hardware accelerator.

Hardware Contrast Enhancement: A Small Design Solves a Big Problem

The accuracy improvement on dashcam frames reveals the value of hardware preprocessing. On a resource-constrained FPGA, a simple contrast enhancement stage can raise accuracy from less than half to nearly 80%. This design approach shows that hardware optimization for specific sensor characteristics is often more effective than simply increasing network complexity.

The design philosophy of this unit is "good enough is good": the 80x60 pixel input resolution is far lower than modern cameras, but combined with a binarized network and hardware enhancement, it can still achieve usable accuracy on specific tasks. This offers a new idea for low-cost, low-power edge vision applications.

Implications for Edge AI: Moving Decisions Forward and Freeing the Host

The significance of this research is that it proves neural network decisions can be completely decoupled from the host. In scenarios such as autonomous driving, intelligent surveillance, and industrial inspection, outputting decisions directly at the sensor can greatly reduce system complexity and response latency. The host no longer needs to run inference models and only receives a small amount of structured results, freeing up compute for other tasks.

From a broader perspective, HDPU represents a trend toward "hardwired intelligence": solidifying a trained neural network into dedicated hardware logic to perform specific tasks with extremely low power and deterministic latency. Although less flexible than general-purpose processors, for fixed tasks this approach has significant advantages in energy efficiency and real-time performance. As FPGA and ASIC costs decline, sensor-side inference is expected to become one of the mainstream paradigms of edge AI.

The study also points to future directions: how to hardwire more types of networks (such as object detection and semantic segmentation) to the sensor side, and how to balance the tension between hardware solidification and model updates. In any case, the idea of pushing decisions to the sensor has opened a noteworthy new path for edge computing.

One-line Summary

A hardwired distributed processing unit enables sensor-side neural network decisions on an FPGA, completely freeing the host.

Paragraph Summary

This study proposes using a hardwired distributed processing unit (HDPU) to complete neural network decisions directly at the sensor. Using crosswalk detection as a case study, it implements a complete pipeline from pixels to decisions on a low-cost Artix-7 FPGA. The core is solidifying a four-layer binarized CNN (93,408 one-bit weights, 80x60 input) into FPGA logic; the entire detection path occupies only 1,814 LUTs, requiring no processor, operating system, or full-resolution frame storage. The unit completes inference within a fixed 2,269,477 clock cycles per frame, achieving 95.7% accuracy on street views for definite answers, and hardware contrast enhancement raises dashcam frame accuracy from 47.7% to 78.2%. Compared with running ResNet-18 on the host, it saves 8.13 milliseconds of core time per frame, and the host receives only 16 bytes instead of 614,400 bytes of data. All 639 test images are bit-exact, proving the feasibility of moving decisions forward and freeing the host.