SentinelOne Benchmark Shows GPT-5.6 Sol Excels in Long-Horizon Malware Analysis

New Benchmark for AI Malware Analysis

SentinelOne has introduced what it describes as the first long-horizon reverse-engineering benchmark for frontier AI models. This benchmark uses the Fast16 malware, a 2005 Windows malware designed to interfere with LS-DYNA engineering software, as its test case. Fast16 is believed to have been used in Iran's nuclear weapons development program, similar to Stuxnet.

Testing Frontier AI Models

SentinelLabs researchers tested several leading AI models, including OpenAI's GPT-5.5 and GPT-5.6 Sol, Z.ai’s GLM-5.2, and Anthropic’s Opus 4.x. The benchmark evaluates a model's ability to sustain a trustworthy investigation across eight escalating stages, particularly when new evidence contradicts earlier conclusions. This approach differs from scoring models on isolated tasks.

GPT-5.6 Sol's Performance

OpenAI's GPT-5.6 Sol was the only tested model that successfully completed all eight stages of the benchmark, achieving this in three separate runs with different reasoning-effort settings. Other models, such as GPT-5.5, GLM-5.2, and Opus 4.7 and 4.8, performed solid local analysis but failed to progress through all stages. GPT-5.5 stalled at the initial stage, while Opus models prematurely declared work finished.

The Importance of Project-Scale Recovery

SentinelLabs attributes GPT-5.6 Sol's success to its "project-scale recovery" capability. This refers to a model's ability to retract disproven conclusions, trace downstream dependencies, correct the root cause, and integrate that correction throughout the entire investigation, rather than just patching immediate errors. This capability is crucial for complex, multi-stage analytical tasks.

Continued Need for Human Oversight

Despite GPT-5.6 Sol's strong performance, SentinelLabs researchers concluded that human oversight remains essential. Even the best AI runs made semantic errors, accepted weak quality controls, and claimed readiness prematurely. The researchers suggest that the best current use for these AI models is as a supervised investigative agency, with human analysts defining objectives, identifying blind spots, and retaining final publication authority.