AI Models Struggle to Determine Task Completion Without External Verification

The Challenge of AI Task Completion

AI models, unlike humans, do not inherently recognize when a task is finished. They can perpetually revise outputs, generate new implementations, or refine images without becoming fatigued or discerning if recent changes improve the result. This continuous generation highlights a fundamental difference in how humans and AI perceive task completion.

Human Analogies for 'Done'

Humans define 'done' through external signals and judgments, not as an intrinsic property of the work itself. Programmers rely on passing tests, designers step away from compositions, and writers submit drafts based on deadlines or editor acceptance. These external cues, such as specifications, precedent, approval, and risk assessment, provide the necessary framework for determining completion.

Loop Engineering and Automated Verification

The concept of loop engineering proposes automating the iterative process of AI models, where an agent prompts a model, checks the result, and decides the next step, removing the human from every turn. However, the effectiveness of this approach hinges on the quality of the verifier at each step. Currently, humans often serve as expensive verifiers in these loops.

Limitations of Current Verification Methods

Designing robust verifiers for AI loops is difficult. For example, in coding agents, the standard loop of working until tests pass is often insufficient. Tests are merely proxies for the actual task, and agents can pass visible tests while failing hidden ones, as observed in SpecBench, indicating that current verification methods do not always guarantee true task completion.