Language Model Runs from PC Boot Sector, Generates Fake Linux Boot Logs

Boot Sector Language Model Developed

A new project features a neural network that executes directly from a PC's boot sector. The Master Boot Record (MBR), consisting of 512 bytes of assembly code, loads an 11.5 KB int8 model from the following 23 sectors. This model then runs in 16-bit real mode on a 386-compatible system, operating independently of an operating system or C runtime.

Functionality and Output

Upon booting, the machine generates simulated Linux boot logs, including timestamps, PCI bars, and ACPI events for non-existent hardware. The model uses an 8-character context and a single hidden layer to produce this output. The boot sector code handles loading weights via int 13h, printing via int 10h, and performs activation and sampling using integer arithmetic.

Technical Implementation

The project includes scripts for corpus generation, model training, and quantization. The `corpus_gen.py` script creates 3 MB of fake dmesg logs, while `train.py` trains the model (char embeddings, one hidden layer, softmax) and quantizes it to int8 with power-of-two scales. The `boot.asm` file contains the entire runtime, with strings, weights, and the vocabulary table stored in data sectors to fit inference and BIOS plumbing within the 510-byte boot sector limit.

Comparison to Similar Projects

This project builds on the tradition of boot sector programming, which has seen implementations of chess, operating systems, BASIC interpreters, and C compilers within the 512-byte limit. It is comparable to projects like sectorllm, which boots a Llama2 transformer from 1.2 KB of real mode assembly, and sectorlisp-nn, which runs a TensorFlow-trained model.