Cactus Hybrid Introduces Gemma 4 E2B Model with Self-Assessed Answer Confidence

Introducing Confidence Scoring for On-Device AI

Cactus has developed a new technique for on-device AI models to determine the reliability of their own answers. Small, local models offer speed and privacy but can sometimes produce incorrect information. The new method addresses this by integrating 'probes' within the model's checkpoint.

How the Hybrid Routing System Works

These probes score every answer with a confidence level between 0 and 1, returned as structured data. If a model's confidence in an answer is below a set threshold (e.g., 0.85), the query can be automatically rerouted to a larger, more capable model. This allows for efficient local processing of high-confidence queries while leveraging more powerful external models for uncertain cases.

Gemma 4 E2B Hybrid Model Release

The first implementation of this technology is the Gemma 4 E2B Hybrid model, available in the Cactus Hybrid collection on Hugging Face. This model, the smallest Gemma variant, demonstrates performance comparable to Gemini 3.1 Flash-Lite on most benchmarks. It achieves this by processing the majority of queries itself and routing only 15–35% of them to the Gemini 3.1 Flash-Lite for validation or more complex responses.

Developer Access and Benchmarking

Developers can access and integrate the Gemma 4 E2B Hybrid model using the `cactus-compute` library or through `mlx-lm` and `transformers` with provided code examples. Cactus encourages independent benchmarking for various quantization methods like Unsloth, GGUF, and MLX to assess performance.