Experimental Sharded, Encrypted Storage Developed for Yggdrasil Overlay Network

Decentralized Storage Concept

An experimental system for sharded, encrypted file storage has been implemented on the Yggdrasil overlay network. This proof-of-concept enables a group of users to pool their disk space, distributing file fragments across their machines. Each file is encrypted and split into pieces, which are then spread among the participating nodes. This design ensures that data remains accessible even if individual machines go offline.

Technical Implementation

The system uses AES-256-GCM encryption for each 4 MiB file chunk, with a unique key per file and a fresh nonce per chunk. Reed-Solomon erasure coding is applied to each chunk, creating data and parity shards. The number of shards varies based on the number of online machines, ensuring that no single machine holds multiple shards of the same chunk for redundancy. Node IDs are derived from public keys, and nodes only interact with group members.

Features and Capabilities

Beyond storage, the system includes features such as version control, allowing restoration of older file versions. It also supports direct and topic-based messaging between nodes, with messages delivered even if nodes were offline. The system can host websites by publishing folders across multiple machines, maintaining availability if one server fails. Email to a user's domain can also be routed and encrypted within the group, accessible via a dashboard.

Accessibility and Status

Users can set up a storage box, with step-by-step guides available for Windows users. A gateway service allows non-box owners to pay a monthly fee to access group storage via S3-compatible programs, with members choosing whether their boxes store customer data and earning credit. The project is currently experimental, and its cryptography has not undergone independent review, so it is not recommended for storing irreplaceable data.