← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

LLMs Solve and Optimize Puzzles in Zachtronics' TIS-100 Programming Game

🔄 Updated 10d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • 16 LLMs attempted 55 TIS-100 programming puzzles.
  • Opus 5.5 and GPT-6 Astra solved all puzzles and found new optimization points.
  • Models were given the game manual and real-time feedback.
  • The experiment measured solving ability and optimization skills.

LLMs Tackle TIS-100 Challenges

Sixteen large language models, including frontier and open-weight models, were tasked with solving and optimizing puzzles in Zachtronics' TIS-100. This game requires players to program a simulated assembly language machine with severe constraints on nodes, registers, and code lines. Each model attempted all 47 in-game puzzles and eight community tournament puzzles, with 10 to 45 minutes allocated per puzzle for finding a working program and improving it.

Top Performers: Opus 5.5 and GPT-6 Astra

Opus 5.5 and GPT-6 Astra emerged as the leading models, both successfully solving all 55 puzzles. These models also identified new frontier points on 11 puzzles, indicating novel optimization strategies. Opus 5.5 found 72 new points, while GPT-6 Astra found 32. GPT-6 Astra showed a slightly higher record proximity at 78.0 compared to Opus 5.5's 77.2, measuring closeness to community records. GPT-6 Astra also led in all six tested hypervolume settings, which quantify the space covered by each model's optimization tradeoffs.

Solving vs. Optimizing Skills

The experiment differentiated between solving and optimization capabilities. Gemini 3.8 Flash solved 53 puzzles and found one new optimization point. Fable 5.1 solved 52 puzzles but found new points on five, illustrating that solving ability does not always correlate directly with optimization skill within this constrained programming environment.

Methodology: Harness Design

The testing environment provided models with the game manual, visible test inputs, register states, and game feedback, similar to what a human player would experience. Models could read puzzle descriptions and language references, edit programs, and run them in the actual game. Diagnostic information, such as mismatches for wrong outputs and blocked ports for deadlocks, was provided to aid in debugging and improvement. Community code or exact record scores were withheld; models only received coarse score histograms.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

Sixteen large language models (LLMs) were tested on Zachtronics' TIS-100, a low-level assembly programming game, to assess their ability to solve and optimize complex programming puzzles. Opus 5.5 and GPT-6 Astra demonstrated the highest performance, solving all 55 puzzles and discovering new optimization frontiers. This experiment highlights LLMs' capabilities in code generation and optimization within constrained, interactive environments.