The 128KB Wall

Preface: The Ghost in the Shell

Something was wrong with our production system, and we had not noticed.

Users would start a task. The AI agent (Claude Code, running inside an ephemeral Linux sandbox) would execute flawlessly. It would read files, edit code, install dependencies, run tests, push changes. Everything worked.

Then the user would send a follow-up message. "Now add auth." Or "Fix the mobile layout." Or just "continue."

The agent would reconnect to the warm sandbox. It would load the conversation history. It would begin executing.

And then it would die. Instantly. Silently. No error message. No stack trace. No crash report. The logs would show:

Sandbox reconnected — skipping clone and install
Continuing with conversation context
Claude Code execution failed
Agent execution failed
Task failed to continue

Five lines. Zero information. The CLI process started and immediately exited with a non-zero code. No stdout. No stderr. No output of any kind.

Our auto-nudge system would detect the stall and try to restart the agent. It would send the same prompt. The agent would fail again. The auto-nudge would fire again. A loop of recovery mechanisms fighting a failure they could not see.

We had built an entire immune system to fight a disease we had never diagnosed.

I. The Archaeology of a Silent Failure

To understand what killed our agent, you need to understand how a modern AI coding platform actually works at the system level. Not the marketing level. Not the "just prompt it" level. The actual, physical, metal-touching level where software meets operating system meets hardware.

The Architecture

OutcomeDev runs AI coding agents inside ephemeral Linux sandboxes, fresh virtual machines provisioned on demand. When a user submits a task, the system:

1. Creates a sandbox (a real Linux VM with Node.js, Git, and a full filesystem)
2. Clones the user's repository into it
3. Installs dependencies
4. Executes the AI agent inside the sandbox
5. The agent reads files, writes code, runs commands, and pushes changes

When the user sends a follow-up message, the system tries to reconnect to the existing sandbox (a "warm window") rather than creating a new one from scratch. This turns a 60-second cold start into a 2-second reconnect.

The problem lives in step 4.

Two Execution Paths

OutcomeDev supports two ways to run an AI agent:

Path A: The AI SDK (Native). The server calls Anthropic's API directly using the Vercel AI SDK. The prompt is sent as a JSON body over HTTP. The model responds with structured tool calls. The server executes those tools against the sandbox. This is clean, modern, and inherently robust. The prompt never touches a shell.

Path B: Claude Code CLI. The server runs the Claude Code command-line tool inside the sandbox as a subprocess. The prompt is passed as a shell argument. The server captures the CLI's stdout as a JSONL stream and parses it in real-time.

Path B exists because Claude Code has its own optimized tool implementations, extended thinking support, and a user experience some developers prefer. It is a legitimate technical choice.

It is also the path that killed us.

The Prompt Gets Bigger

On the first message, the prompt is small. "Build me a dashboard." Maybe 50 characters. The shell command works perfectly.

Then the user sends a follow-up. "Now add authentication." The system loads the conversation history, every previous user message and every agent response, and prepends it to the new prompt so the agent has context.

Here is the thing about agent responses: they are not short. They contain every tool call, every file read, every shell command, every code block the agent wrote. A single agent turn can produce thousands of characters. After three turns, the conversation history alone can exceed 50,000 characters.

All of this gets stuffed into a single shell argument.

And that is when we hit the wall.

II. ARG_MAX: The 50-Year-Old Trap Door

In 1971, when Ken Thompson and Dennis Ritchie were building Unix at Bell Labs, they had to make a decision about the exec system call, the kernel function that starts a new process. Specifically: how big can the argument list be?

They picked a number. It was small, because memory was expensive and PDP-11 machines had 64KB of RAM total. Over the decades, as Unix evolved into Linux and macOS and every cloud server on the planet, this number grew. But it never went away.

On modern Linux, the limit is typically 128KB for the combined size of all command-line arguments and environment variables passed to a process. It is defined by a kernel parameter called ARG_MAX.

If you try to execute a command whose arguments exceed this limit, the kernel does not give you an error message. It does not throw an exception. It does not log a warning. The exec syscall returns E2BIG ("Argument list too long") and the shell process exits with a non-zero code.

No stdout. No stderr. No output of any kind.

Sound familiar?

The Timeline of Failure

Here is exactly what was happening in production:

Turn 1: User sends "Build me a dashboard." The shell command is ~200 bytes. ARG_MAX is 128KB. The command executes perfectly. Claude builds the dashboard, writes 15 files, runs npm install, pushes to GitHub. The user is delighted.

Turn 2: User sends "Add authentication." The system loads Turn 1's history: the user's message plus Claude's 8,000-character response containing every file edit and tool call. After shell escaping (backslashes for quotes, dollars, backticks), the argument is ~20KB. Still under the limit. Claude adds auth. Works fine.

Turn 3: User sends "Now add role-based access control." The system loads Turn 1 + Turn 2 history. Two user messages. Two agent responses. Each agent response contains file edits, shell commands, npm outputs, test results. After shell escaping, the argument is ~80KB. Getting close.

Turn 4: User sends "Fix the mobile layout." The system loads Turn 1 + Turn 2 + Turn 3 history. Three user messages. Three agent responses. Shell escaping inflates special characters. The argument hits 135KB.

The kernel rejects it. The process never starts. The exit code is non-zero. No output is produced. Our error handler catches the non-zero exit code and logs: "Claude Code execution failed."

The auto-nudge system detects the stall. It sends the exact same prompt again. The kernel rejects it again. The auto-nudge fires again. And again. And again. A 50-year-old kernel limit created a livelock in our 2026 AI orchestration system.

III. The Recovery Infrastructure We Built Around a Symptom

This is the part of the story that haunts me. Because we did not find this bug immediately. Instead, we built increasingly sophisticated recovery systems to work around a problem we did not understand.

Layer 1: The Auto-Nudge

When we first noticed agents stalling, we built an auto-nudge system. If no SSE updates arrive for 60 seconds, a warning banner appears. After 30 more seconds, the system automatically stops and restarts the task. No user intervention required.

This was elegant. It was well-tested. It worked beautifully for the problem it was designed to solve (model laziness and premature completion).

It was also completely useless against ARG_MAX.

The auto-nudge would detect the stall, stop the task, and re-send the same prompt. The prompt was still too big. The kernel would still reject it. The nudge would fire again. We had built a defibrillator for a patient who had been shot. We kept shocking the heart, but the bullet was still in the chest.

Layer 2: Stall Detection

We then added more granular stall detection. We tracked stream chunk counts, content accumulation, session IDs. We could distinguish between "the model is thinking slowly" and "the model has stopped producing output."

This helped us see the problem more clearly. But it did not help us fix it, because the diagnostic information we logged was: "Claude Code execution failed." No exit code details. No argument size measurement. No kernel error.

Layer 3: The Context-Aware Nudge

We made the auto-nudge smarter. It would only fire if the model had been actively editing files (not just answering questions). It checked five conditions before intervening. It capped itself at two attempts to prevent infinite loops.

This was genuinely good engineering. It solved a real problem (premature completion). But it was solving a different problem. The ARG_MAX failure looked identical from the outside: the agent stops, no output appears, the task stalls. But the root cause was entirely different.

The Lesson

We spent weeks building recovery infrastructure for a problem that required zero recovery infrastructure. The fix was not a smarter nudge, a better timeout, or a more resilient retry loop.

The fix was four lines of code:

# Instead of this (breaks at ~128KB):
claude -p "entire conversation history here..." --output-format stream-json

# Do this (no size limit):
cat /tmp/.claude-prompt | claude -p --output-format stream-json

Write the prompt to a file. Pipe it via stdin. The data flows through a pipe, which is stream-based and has no size limit. The shell never has to construct the argument vector. The kernel never sees it.

This is the Unix way. This is how grep, awk, sed, jq, and every serious Unix tool handles large input. It has been the correct answer since 1973.

IV. The Five Words That Cracked It Open

I want to dwell on this, because it is the crux of the essay.

The AI model I was debugging with is capable of extraordinary reasoning. It knows how Unix processes work. It knows what ARG_MAX is. It knows how shell argument expansion differs from stdin pipes. All the facts needed to diagnose this bug exist inside its training data.

But facts are not diagnosis. Diagnosis requires direction. And direction came from five words.

The Wild Goose Chase That Did Not Happen

Imagine I had opened a debugging session and said: "My Claude Code agent fails on follow-up messages. Here are the logs." The model would have seen:

• Non-zero exit code
• No stdout or stderr output
• Sandbox is healthy (reconnection works)
• API key is valid
• History is loaded correctly

A dozen plausible hypotheses would have followed. Check the API key. Verify network connectivity. Ensure the CLI binary is installed. Check if the sandbox filesystem is corrupted. Maybe the model is rate-limited.

All reasonable. All wrong. And each one would have spawned sub-investigations: reading config files, testing endpoints, checking permissions. Burning tokens and time chasing ghosts.

Instead, I said: "This was not happening before. I started experiencing it after the conversation got a lot of turns."

Two sentences changed everything.

Context Engineering in Action

Those two sentences are not prompts. They are constraints. Together they instantly eliminated 90% of the hypothesis space:

1. The core architecture is sound (it worked before)
2. The API keys are valid (they were working recently)
3. The sandbox infrastructure is fine (same VMs, same provider)
4. Something changed, and it correlates with conversation length

The first sentence told the model to stop looking for fundamental breakage. The second pointed it directly at conversation size as the variable. The model did not need to check the git log or trace recent commits. It went straight to the data path: conversation history gets loaded, concatenated, passed as a shell argument. Shell arguments have a size limit. That limit is ARG_MAX.

From there, the fix was obvious. The model knew stdin pipes bypass argument vectors. It just needed to know where to look.

The Real Skill

The industry talks about prompt engineering, crafting the perfect instruction. But the harder skill is context engineering: knowing which pieces of information to surface, when to surface them, and how they constrain the problem space.

"It fails on follow-ups" is a prompt. "This was not happening before, it only started failing after longer conversations" is context. The difference between those two statements is the difference between a six-hour wild goose chase and a 45-minute diagnosis.

The full reasoning chain looked like this:

1. Human context: "This was not happening before" → the system is fundamentally sound
2. Human pattern recognition: "It started after the conversation got a lot of turns" → the variable is conversation size
3. Systems knowledge: The prompt is passed as a shell argument → shells have size limits
4. OS knowledge: ARG_MAX is ~128KB on Linux → the history exceeds this after several turns
5. Unix knowledge: $(cat file) still expands inline → command substitution does not help
6. Unix knowledge: Stdin pipes bypass argument vectors entirely → this is the correct fix
7. CLI knowledge: Claude Code supports cat file | claude -p natively → the fix is trivial

Steps 3-7 are facts the model already knew. Steps 1-2 were human contributions. The compass that pointed the model at the right mountain instead of letting it wander the entire range.

V. The Industry Pattern

We are not the first team to hit this wall. And we will not be the last. The pattern is universal:

GitHub Copilot CLI

GitHub's CLI tool uses HTTP APIs directly. Prompts are sent as JSON bodies over HTTPS. The shell is used only to invoke the CLI binary; the prompt data never flows through shell arguments. This is not an accident. It is a deliberate architectural choice by engineers who understood that shells are not data transport mechanisms.

Devin (Cognition)

Devin runs its agent loop server-side with direct API calls. The agent communicates with models via HTTP, not CLI subprocesses. The shell is used only for executing user-facing commands (running tests, installing packages), never for passing prompts.

Cursor

Cursor's agent mode communicates with models through its own internal IPC mechanism. Prompts are passed as structured data, not shell strings. The editor never constructs a shell command containing the full conversation history.

The Pattern

Every mature AI tool converged on the same answer independently: do not pass prompts through the shell. Use HTTP. Use stdin pipes. Use structured IPC. Use anything except shell arguments.

The shell is a user interface. It was designed for humans typing commands. It was never designed to transport 100KB of conversation history containing embedded code blocks, JSON fragments, and special characters.

When you use the shell as a data transport layer, you inherit every limitation and quirk of the shell. ARG_MAX is just the most dramatic. There are also quoting issues, encoding issues, null byte handling, locale-dependent behavior, and signal handling races.

VI. The Fix, and What It Teaches Us

The actual fix was implemented in under an hour. It has three components:

1. Stdin Pipe

# Write prompt to a temp file (quoted heredoc = no expansion)
cat > /tmp/.claude-prompt << 'PROMPTEOF'
[entire conversation history, verbatim, no escaping needed]
PROMPTEOF

# Pipe via stdin — no size limit, no shell escaping
cat /tmp/.claude-prompt | claude -p --output-format stream-json

The prompt flows through a Unix pipe, which is a kernel-level stream with no size limit. The shell never parses it, never escapes it, never constructs an argument vector from it. It just flows.

2. History Condensation

Even with the pipe removing size limits, we also condense the conversation history:

• Strip tool activity noise: Lines like "Editing src/foo.tsx" and "Running: npm install" are UI indicators, not meaningful context. They are stripped from agent messages before history assembly.
• Last 10 messages: Only the most recent 10 messages are included (down from 20). Older context is available in the codebase itself.
• 1,500 character cap: Each message is truncated to 1,500 characters. If the model wrote a 10,000-character response, only the first 1,500 characters are included in the history.

This reduces token cost, improves response quality (models perform better with focused context), and keeps the history well under the 10MB stdin pipe limit.

3. Error Diagnostics

The old error handling:

Claude Code execution failed

The new error handling logs (server-side):

{
  "exitCode": 1,
  "stdoutChunks": 0,
  "stderrChunks": 0,
  "contentReceived": false,
  "contentLength": 0,
  "isResumed": true,
  "historyLength": 8
}

And surfaces differentiated messages to the user:

• Zero output → "Claude Code failed to start, no output received"
• Output but no content → "Claude Code exited without producing a response"
• Content but non-zero exit → "Claude Code execution failed"

These three categories cleanly separate the three classes of failure: process-level (ARG_MAX, missing binary), API-level (auth failure, rate limit), and model-level (hallucination, context overflow).

VII. Context Engineering Is the New Moat

This experience crystallized something I have been thinking about for a while.

The frontier models know almost everything. They have read the Linux kernel source code. They understand ARG_MAX. They know how Unix pipes work. They can write a heredoc in their sleep. The raw knowledge is there.

What they lack is direction. They do not know which facts are relevant to your problem, in your system, at this moment. They cannot observe that something used to work and stopped. They cannot feel the pattern ("this only fails after several turns") that a human accumulates through lived experience with the system.

The skill that matters is not telling the model what to do. It is telling the model where to look. That is context engineering: the discipline of surfacing the right constraints, at the right time, to collapse a vast hypothesis space into a tractable one.

Two sentences turned a potential six-hour investigation into a 45-minute fix. Not because the model was incapable, but because without them, it had no compass.

This is the real multiplier. A human who understands their system deeply, paired with a model that knows everything about computer science broadly, can move at a speed that neither could achieve alone. The human provides the contextual constraints. The model provides the exhaustive search within those constraints. Together, they converge fast.

The engineers who thrive in this era will not be the ones who memorize APIs or grind LeetCode. They will be the ones who develop deep intuition for their systems, who can smell when a failure is architectural vs. environmental vs. kernel-level, and who know how to translate that intuition into context that an AI can act on.

That is the job now. Not writing code. Not prompting. Providing the signal that makes everything else converge.

VIII. The Takeaway

We built auto-nudge, stall detection, timeout banners, and context-aware retry logic. All of it is necessary. The auto-nudge catches premature completion, a real and persistent problem where models quit before the job is done. The stall detection catches hung processes and network timeouts. The retry logic recovers from transient API failures. These systems save users every single day.

But none of them could reach this bug. They operate at the orchestration layer: watching model behavior, monitoring output streams, managing task state. The ARG_MAX failure was happening one layer below, at the kernel level, before the model ever got a chance to run.

The fix was four lines of bash. Write the prompt to a file. Pipe it to stdin. Clean up the file. Done.

But those four lines required a chain of reasoning that spanned five decades of computing history, from Ken Thompson's PDP-11 at Bell Labs to a serverless sandbox running Claude Code in 2026. And that chain was unlocked not by a smarter model or a better prompt, but by a human who said, "This was not happening before."

Software engineering is not writing code. It is understanding systems deeply enough to know which five words change everything.

Filed under: Systems Engineering · Context Engineering · AI Orchestration

Written: May 7, 2026