During private evaluations, Gemini 4 Carbon demonstrated a significant performance lag, completing coding tasks in approximately an hour that competitor models like Opus and Astra finished in a few minutes. This observation aligns with similar user feedback regarding Carbon's real-world usability due to its speed.
The benchmark task involved an agent receiving a codebase with a pre-existing bug and a symptom-based bug report. The agent's objective was to diagnose and fix the bug without introducing unrelated changes. The task emphasizes engineering judgment, such as tracing symptoms to code paths and correctly placing fixes, rather than raw intelligence.
Analysis identified two primary factors contributing to Carbon's slowness: slower inference speed and a higher volume of work per task. Slower inference, encompassing queueing, network latency, and processing/generation speed, accounts for 16-30% of the extra time compared to Astra and 45-49% compared to Opus. The remaining delay stems from Carbon performing more steps, tool calls, and processing more tokens.
Specifically, Carbon takes about seven times more steps than Astra and eleven times more actions (e.g., reading files, running commands). It also writes seven times more code and processes over sixteen times more input tokens due to the increasing length of conversational context with each step.
Before concluding on inference and work volume, simpler explanations were ruled out. Testing Carbon within other coding agents, opencode and mini-SWE-agent, showed no meaningful difference in performance, indicating the issue was not with the testing harness. Additionally, while effort settings vary across providers, running Carbon on a 'low' setting often resulted in task failure, suggesting the 'high' setting was necessary for task completion.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
Benchmarking reveals Gemini 4 Carbon takes significantly longer to complete code debugging tasks compared to models like Opus and Astra. The slowness is attributed to both slower inference speeds and a higher number of steps, tool calls, and tokens processed per task.