ESP32-C3 Adblocker Stores Blocklist Hashes in Flash, Eliminating PSRAM Requirement

New Ad-Blocking Approach for ESP32-C3

A new ad-blocking solution has been developed for the ESP32-C3 microcontroller that rethinks how domain blocklists are stored. Instead of loading full domain strings into RAM, which often necessitates additional PSRAM, this project stores domains as sorted 40-bit hashes directly in the device's flash memory.

This technique allows the ESP32-C3 to manage a blocklist of over 140,000 domains using approximately 0.7 MB of flash and only 50 KB of RAM. Domain matching is performed via a binary search of the flash-based hash table, completing in about 10 milliseconds.

Technical Implementation Details

The system processes DNS queries by extracting the domain, generating an FNV-1a hash, and then performing a binary search against the flash-stored hash table. If a hit occurs, the query is sinkholed by returning 0.0.0.0; otherwise, it is forwarded to an upstream resolver.

The choice of 40-bit hashes is a balance between flash usage and collision probability. At 141,000 domains, collisions are negligible. While 32-bit hashes would save flash, they would increase collision rates, and 64-bit hashes would waste space without providing significant benefit for this scale.

Broader Applicability and Hardware Requirements

This flash-based hashing method is not exclusive to the ESP32-C3 and can be applied to other microcontrollers. For example, on a 16 MB ESP32-S3, this approach could store approximately 2.7 million domains, compared to about 466,000 domains if strings were stored in 8 MB of PSRAM.

The project requires any ESP32-C3 board with 4 MB of flash and no PSRAM. It has been tested on a C3 SuperMini. The device can be powered from a stable USB source, such as a phone charger or a router's USB port, and can be housed in a 3D-printed enclosure designed to maintain antenna performance and ventilation.

Setup and Maintenance

Initial setup involves a USB connection for flashing the firmware. Subsequent firmware and blocklist updates can be performed over Wi-Fi. Users are advised to use a current version of PlatformIO for compilation and flashing.