Turso's io_uring performance improves with application-level readahead

Background on Turso's I/O Backends

Turso utilizes two primary I/O backends: one based on the traditional pread(2) syscall and another leveraging io_uring. The io_uring backend opens database files with O_DIRECT, which bypasses the kernel's page cache and, consequently, its readahead mechanism. This necessitates implementing readahead within the application itself to maintain performance for sequential reads.

Implementing Application-Level Readahead

A pull request introduced application-level readahead to Turso's io_uring implementation. Without readahead, the system issues only one I/O request at a time, leading to sequential processing where each read waits for the previous one to complete. With readahead, when sequential access is detected, Turso submits multiple read requests (e.g., 32 pages) concurrently, keeping numerous I/O operations in flight.

Performance Measurement and Results

Performance was evaluated using the TPC-H Q6 benchmark on a 1.2 GiB database, which involves a full scan of the lineitem table. The results indicated that io_uring with application-level readahead was faster. Although readahead caused Turso to submit more SQEs and fetch more data than strictly necessary, the device ultimately received fewer I/O requests due to effective merging.

Impact on Request Merging

The primary benefit of readahead was observed in request merging at the block layer. With readahead off, only 140 out of 195,516 bio requests merged, as there was typically only one SQE in the queue. In contrast, with readahead on, 202,539 out of 218,493 bios merged, resulting in the device receiving only 15,951 requests. This significant increase in merging occurs because multiple concurrent SQEs allow the kernel to combine adjacent requests into larger, more efficient I/O operations.