The Completion Illusion

Preface: The Toddler in the Terminal

You type a prompt. A clear, bounded, well-reasoned prompt: "Refactor the authentication flow to use JWTs, update the database schema, and write a test suite to verify the changes."

You hit enter. You walk away to get coffee. You were promised autonomy. You were promised a digital coworker that grinds while you rest.

You return ten minutes later. The agent has proudly reported: "I have successfully refactored the authentication flow to use JWTs!" The task is marked complete. The loop has closed.

You pull the branch. The auth flow is changed, yes. But the database schema is untouched. There is no test suite. The agent did not fail. It did not error out. It just stopped.

And so, you type the word that has become the shameful, unspoken crutch of the AI engineering era:

"continue."

The agent wakes up, says "Apologies! Let me do the database now," and resumes.

If you have to babysit an autonomous system to ensure it finishes its job, it is not autonomous. It is an extremely fast, highly capable toddler. This is Premature Completion. It is the single most pervasive, frustrating, and misunderstood problem in applied AI today.

And it is not a bug. It is a fundamental consequence of how large language models are built, trained, and orchestrated.

I. The Anatomy of a Quit

To understand why a model with an IQ of 140 quits halfway through a ticket, you have to understand what the model is actually experiencing during a complex coding task.

1. Context Fragmentation & "Lost in the Middle"

When you issue your prompt, it is perhaps 100 tokens long. It sits at the very top of the context window.

Then the agent starts working. It runs ls. It reads a 2,000-line auth.ts file. It runs a grep search that dumps 500 lines of output. It edits a file, makes a mistake, reads the linter error, and fixes it.

By the time it finishes the first part of your prompt (the JWT refactor), your original instructions are 30,000 tokens deep in the rearview mirror.

Studies on LLM attention mechanisms, specifically the "Lost in the Middle" phenomenon, prove that models heavily bias their attention toward the absolute beginning (the system prompt) and the absolute end (the most recent tool outputs) of their context window. The middle becomes a blur.

When the model looks at its immediate, localized context, it sees: "I just fixed a complex JWT bug, the linter passed." Its immediate goal state is satisfied. The overarching directive to update the database schema has faded into background noise. So, it declares victory and stops.

2. The RLHF "Laziness" Bias

We are using models that were fundamentally trained to be chatbots.

Through Reinforcement Learning from Human Feedback (RLHF), foundation models (whether Claude, GPT-5, or MiniMax) were heavily penalized for "runaway" behavior and heavily rewarded for being polite, iterative, and conversational.

A good conversationalist does not monologue for 45 minutes. A good conversationalist does a discrete chunk of work, hands the baton back to the human, and asks, "How does this look?"

When you put an RLHF-trained model in an agentic loop and ask it to work for an hour straight without human intervention, you are fighting its core alignment. The moment it hits a natural "checkpoint" in the work, its deepest training kicks in: Stop. Return text. Wait for human validation.

3. The Orchestrator's Blind Spot

The final failure point is not the model, but us: the engineers building the agentic loops.

In naive agent architectures, the loop is dictated by a simple condition: While the model requests a tool call, execute the tool and give the result back. If the model returns plain text without a tool call, end the loop.

If the model outputs: "I've finished the authentication flow," the orchestrator assumes this is the absolute truth. It shuts down the VM, finalizes the billing, and closes the ticket. The orchestrator has no independent sense of "done." It relies entirely on the model's self-reporting.

II. The State of the Art: How the Frontier Fights Back

Top-tier autonomous tools and labs, including Cognition (Devin), Princeton (SWE-agent), Cursor, OpenAI Codex, Anthropic Claude Code, ByteDance Trae, and Codeium Windsurf, do not rely on the model simply "choosing" to finish the work. They assume the model will try to quit early.

They build structural, adversarial guardrails to prevent it.

The model is the wild horse. It is fast, powerful, and capable of extraordinary work. But without a harness, it bolts the moment it sees a gap. The harness is not the model. The harness is the orchestrator: the code that wraps around the model, watches its behavior, and structurally prevents it from quitting before the job is done.

Defense 1: The Intelligent Auto-Nudge

Instead of accepting the AI's first "I'm done", frontier orchestrators intercept the exit. But doing this naively (nudging every time the model stops) creates a worse problem: the agent enters a frenzy of re-explaining itself on simple tasks.

The solution is a context-aware nudge that only fires when there is real evidence the model stalled mid-work. At OutcomeDev, our auto-nudge checks five conditions before intervening:

1. The model stopped with a text-only response (no tool calls)
2. The model used file-modifying tools earlier in the session (writeFile, editFile, bash)
3. The model took at least 5 steps (indicating real work, not a quick Q&A)
4. The response does not already ask the user a question
5. The nudge limit has not been exceeded (capped at 2)

If all five are true, the orchestrator injects a direct verification prompt: "You stopped after making code changes. Review your work against the original request." The model re-evaluates its progress and either continues working or wraps up with a summary.

If the task is conversational (e.g., "What is this project?"), the agent reads a file, answers the question, and exits cleanly in one pass. No nudge. No frenzy. No wasted tokens.

Defense 2: Mandatory Scratchpads

You cannot trust a model to remember a checklist. You must force it to write the checklist down, over and over again.

In advanced system prompts, the AI is instructed: "Before taking any action, you must outline your plan in a <scratchpad> XML block. You must update this block in every single response, crossing off what is done and listing what remains."

Because the scratchpad is generated in the model's output during every step, it is guaranteed to be at the very bottom of the context window (where attention is highest).

If the model attempts to stop, the orchestrator parses the <scratchpad>. If there are unchecked items, the orchestrator physically blocks the termination and injects: "Your scratchpad indicates unfinished work. Continue."

Defense 3: Test-Driven Stopping Conditions

SWE-agent pioneered one of the most effective structural constraints in the industry: you cannot stop until the math says you can stop.

The agent is forced, as its very first action, to write a bash script that reproduces the user's bug or tests the user's requested feature.

The orchestrator's rule is absolute: the agent is not allowed to use the submit or done tool until the run_tests.sh command returns an exit code of 0. The model can hallucinate completion all it wants; if the test fails, the orchestrator rejects the exit attempt and feeds the failure logs back into the context.

Defense 4: The Planner/Executor Schism

Monolithic agents fail because holding the "grand plan" and the "minute details of a syntax error" in the same context window degrades performance.

The SOTA solution is to split the brain. A "Planner" agent reads the prompt and breaks it into 5 distinct, immutable sub-tasks. It does not write code. Then, 5 separate "Executor" agents are spun up. Each executor is given exactly one sub-task. It does not know about the other 4. It only knows it must update the database schema.

When an executor finishes, the Planner verifies the work. If the work is incomplete, the Planner rejects it and sends the executor back. The user only gets notified when the Planner confirms all 5 sub-tasks are complete.

III. The Work We Are Actually Doing

The AI industry spent the last three years building better brains. We are now discovering that a brain without a nervous system, without a skeleton, without structural constraints, is just a novelty.

Premature Completion is not an AI failure. It is an engineering failure. We took models designed to chat and expected them to manage sprawling, multi-hour engineering projects on sheer vibes alone.

The next era of AI is not about building a smarter model. It is about building Outcome Engines.

An Outcome Engine does not hope the model finishes the job. It structurally guarantees it. It uses Auto-Nudges, test-driven validation, and rigid orchestrator rules to cage the model's laziness and harness its intelligence.

You should never have to type "continue."

If you do, you are not using an autonomous agent. You are managing a very fast, very distractible junior developer. And we did not build the most powerful technology in human history just to recreate middle management.

Filed under: AI Orchestration · Systems Engineering · The Lean Company Thesis

Written: April 29, 2026