BrevFeed
Sign in
← All stories
● Covered by 1 source · 2 reportsMedium impact2 neutral

Tailscale Outages Traced to 16-Year-Old SQLite WAL-Reset Bug, Now Fixed

🔄 Updated 1d ago
Aggregated by BrevFeed dev · updated 3d ago
🔖 Save
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Tailscale experienced multiple outages due to database corruption.
  • The cause was a 16-year-old SQLite WAL-Reset bug.
  • The bug, a data race, existed since 2010.
  • SQLite version 3.51.3 includes the fix for this bug.
  • Tailscale has fixed the issue and shared findings.

Outages Linked to Long-Standing SQLite Bug

Tailscale reported experiencing shaky uptime at the end of last year and into the new year, with many outages caused by a single bug deep within SQLite. The company spent months on forensics to identify the root cause.

The issue was traced to a 16-year-old bug in SQLite's Write-Ahead Logging (WAL) reset mechanism. This bug, a data race with tight timing constraints, had been present since 2010 but was previously unknown to the SQLite development team.

Internal System Architecture

Tailscale's control plane, which clients interact with as a single public endpoint, is internally split into coordination servers or "shards." Each tailnet resides on one internal shard and can migrate between them. Each shard utilizes an SQLite database to store data.

Bug Identification and Resolution

The SQLite team released version 3.51.3, which includes a fix for the WAL-Reset bug. Developers noted that the bug was difficult to reproduce organically and required special testing logic to trigger the circumstances for verification.

A developer, using the Antithesis testing tool, was able to quickly identify the WAL-Reset bug in SQLite version 3.51.2, confirming its existence and the effectiveness of the fix in version 3.51.3.

Impact and Community Contribution

Tailscale has implemented a fix for the issue and is sharing details of their findings. This contribution aims to assist the broader SQLite community in understanding and addressing this long-standing database vulnerability.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

Keep reading

dev Optimizing SQLite for Production App Servers by Tuning WAL Mode, Concurrency, and VFS Layers 18d ago dev Benchmarking DuckDB vs. SQLite for Observability Data on Low-Cost Hardware 17d ago dev Meta Introduces Hybrid Asset Classification for Privacy-Aware Infrastructure 22h ago dev BriskDB Alpha Released, Sharding SQLite Files for Parallel Writes and PostgreSQL Compatibility 13h ago dev Developer recounts months of work undone by hidden database commit within transaction 14d ago

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~7 min · 6 stories · Aug 15

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Primary sources

GitHub tailscale/sqlite GitHub antithesishq/sqlite GitHub antithesishq/hands-on-tutorial-1 GitHub antithesishq/antithesis-skills

How outlets covered it

Hacker News Front Page — Breaking the WAL 💬 Discuss on HN 3d ago →

A developer used the Antithesis testing tool to quickly identify the WAL-Reset bug in SQLite version 3.51.2, a data race issue that had existed since 2010. The bug was confirmed fixed in SQLite 3.51.3, demonstrating Antithesis's capability in uncovering complex database issues.

Hacker News Front Page — Tailscale Traces Database Corruption to 16y/o SQLite WAL-Reset Bug 💬 Discuss on HN 3d ago →

Tailscale experienced multiple outages due to database corruption, which was traced to a 16-year-old bug in SQLite's write-ahead logging (WAL) reset mechanism. The company has implemented a fix and is sharing details of the issue to aid the broader SQLite community.