Raspberry Pi 5 Runs Qwen3.6-35B-A3B Locally for Offline Car AI Assistant

Offline AI Assistant for Vehicles

A new project demonstrates an AI assistant operating entirely offline within a car. The system utilizes a Raspberry Pi 5 to host the Qwen3.6-35B-A3B language model, enabling local processing for various in-car functionalities. This setup aims to provide AI capabilities without reliance on cloud services or internet access.

Technical Implementation and Features

The Raspberry Pi 5, equipped with 16 GB of RAM, runs the 35-billion parameter model, achieving a generation speed of 3.5 tokens per second. Key features include answering questions based on the car's 745-page owner's manual, providing page citations, and refusing to answer queries outside its knowledge base. It also offers grounded self-knowledge, reporting live data such as temperature, memory, and network status.

Hands-free voice interaction is supported through a continuous listener using whisper.cpp, processing speech locally without cloud-based speech-to-text. The system is autonomous, with all services starting automatically on boot, and can pull updates from a repository, maintaining reachability even behind a phone hotspot's NAT.

Impact and Connectivity

This project showcases the potential for powerful, localized AI applications in environments with limited or no internet connectivity. By operating fully offline, it addresses privacy concerns and eliminates subscription costs associated with cloud AI services. The system includes a three-tier connectivity strategy (phone hotspot, home Wi-Fi, fallback access point) to ensure the phone can always connect to the car's AI.