Whistle Speech Recognition Model Released for On-Device Applications

Introduction of Whistle Model

Whistle is a newly released speech recognition model specifically engineered for on-device applications. It targets a range of devices including mobiles, wearables, robots, smart home systems, automotive platforms, and microcontrollers. The model is distributed as a single 16.9 MB file.

On-Device Functionality

The model performs three primary functions directly on the device: transcription, word timestamps, and speech embedding. Transcription handles 16 kHz mono audio clips up to 30 seconds long in one pass, supporting English, German, French, Spanish, Italian, Dutch, and Polish, with automatic language detection. Word timestamps provide start, end, and probability for each word. Speech embedding outputs encoder data per 80 ms frame without generating a transcript.

Technical Architecture

Whistle runs on the CPU without external dependencies and utilizes the same C++ engine and quantization as the Needle model. Its front end processes 16 kHz mono audio into 80 log-mel bins. The encoder consists of eight Simple Attention blocks, similar to Needle, using mHC residual lanes and a Monarch Hadamard MLP. The decoder features eight Laddered Simple Attention blocks with a different layer count compared to Needle's block list. Each decoder layer incorporates a gated cross attention to read from the encoder.

Decoding Process

Decoding employs five beams scored by length-normalized log probability. Keyword biasing is implemented using an Aho-Corasick automaton to boost the log probability of specified phrases. The transcript is limited to 320 tokens, and the vocabulary includes 8,192 text pieces plus seven language-specific tokens, allowing detected language to be emitted as a token.