Soup CLI v0.72.4 Enables Fine-Tuning 8B LLMs on 4GB Laptop GPUs with Layer Streaming

Accessible LLM Fine-Tuning

Soup CLI, a tool designed to simplify large language model (LLM) fine-tuning, has released version 0.72.4. This update focuses on making LLM training more accessible by enabling users to fine-tune 8B parameter models on GPUs with limited memory, specifically 4GB laptop GPUs.

Layer Streaming Technology

The core feature of this release is layer streaming, which allows the frozen base model to be kept out of VRAM and fed to the GPU one decoder layer at a time. This technique significantly reduces memory requirements, making it possible to perform complex training tasks on less powerful hardware. Previously, layer streaming only supported supervised fine-tuning, but now it extends to preference losses.

Expanded Preference Loss Support

Version 0.72.4 now supports various preference losses, including DPO (Direct Preference Optimization), ORPO, SimPO, and KTO. For DPO, Soup addresses the memory challenge of a reference model by using the same streamed base with its adapters switched off, effectively making the reference model 'free' in terms of memory. This approach resulted in DPO peaking at 0.914x the supervised-fine-tuning peak on an RTX 3050 4GB GPU, compared to an additional 730MB if a separate model copy was used.

Configuration and Usage

Users can enable layer streaming and 4-bit quantization through a simple YAML configuration file. The `soup train` command then executes the fine-tuning process. The tool also includes VRAM pre-flight checks to account for paired losses, which require twice the rows. While memory is optimized, the reference model in DPO does incur a time cost, reading the layer stack 1.52 times as often per step as supervised fine-tuning.