What Is an Agent Harness? How Coding Agents Actually Work

Claude Code, Codex, Amp, and Factory can all use frontier models. Give them the same repository and the same bug, though, and they will take different routes through the problem. One searches the right files immediately. Another burns half its context reading generated code. One runs the tests before touching anything. Another declares victory with a beautiful explanation and a broken build.

The model matters. The machinery around the model matters just as much.

That machinery is the agent harness.

An agent harness turns a language model into a system that can do work. It builds the model’s context, gives it tools, executes those tools, applies permission rules, records what happened, manages the context window, and decides when the task is finished.

This is why dropping a new model into an existing coding agent can immediately make the product better. The model arrives with stronger reasoning. The harness already knows how to search a repository, edit a file, run a test, recover from an error, and keep the whole thing from wandering into your SSH keys.

I have written guides to Claude Code, Amp, Factory Droid, and the OpenAI Agents SDK. The more I use these tools, the less useful model-only comparisons become. You are using a complete system every time you hand an agent a task.

This guide explains that system. We will pull an agent harness apart, compare how popular coding agents assemble the pieces, and build a small working harness in Python.

What Is an Agent Harness?

Anthropic defines an agent harness, sometimes called a scaffold, as the system that processes inputs, orchestrates tool calls, and returns results. OpenAI uses the term for the agent loop and execution logic underlying Codex.

I prefer a slightly broader definition:

An agent harness is the runtime that connects a model to context, tools, an environment, policies, state, and feedback loops so it can pursue a goal over multiple steps.

The model can generate text and structured tool calls. It cannot reach into your laptop and run pytest. It cannot decide which files it is allowed to overwrite. It does not preserve a session after an API request ends. Your harness does those things.

At its simplest, the relationship looks like this:

User
  ↓
Agent harness
  ├── Builds context
  ├── Calls the model
  ├── Executes tools
  ├── Enforces policy
  ├── Stores state
  └── Checks the result
        ↓
Repository, browser, terminal, APIs, databases

The model is one dependency inside the harness. A crucial dependency, obviously, but still one dependency.

Model, Agent, Harness, and Sandbox

These terms get thrown together, so let us clean them up.

Term	What It Is	Example
Model	The neural network producing responses and tool calls	Claude Opus 5, GPT-5.6, Kimi K3
Agent	The complete goal-directed system a user interacts with	A coding agent fixing a bug
Agent harness	The loop and runtime coordinating the model, tools, context, and policies	Claude Code or Codex core
Sandbox	The constrained environment where tools execute	A container, VM, or OS-level restricted process
Evaluation harness	Infrastructure that runs tasks, records trajectories, and grades outcomes	A suite measuring bug-fix success

The distinction between an agent harness and an evaluation harness is especially useful. The agent harness does the work. The evaluation harness measures whether it worked.

Anthropic has shown how much this distinction can affect benchmark results. In one example, fixing task and grader problems moved an Opus 4.5 CORE-Bench score from 42% to 95%. A benchmark score belongs to a model, an agent harness, an evaluation harness, and a particular set of tasks. Pull one number out of that system and you lose most of the meaning.

Why Agent Harnesses Matter Now

For a while, AI products could build their identity around exclusive access to a particular model. That advantage is shrinking.

Frontier models appear in several products. Open-weight models such as Kimi K3 can run behind different interfaces and APIs. Model releases arrive often enough that last month’s winner can become this month’s fallback.

The durable product decisions now live in the harness:

Which context reaches the model?
Which tools can it call?
How clearly are those tools described?
Where does code execute?
Which actions require approval?
How does the agent recover from a failed command?
What survives when the context window fills up?
How does it prove that the task is complete?
Can several agents work at once without destroying each other’s changes?

These decisions explain why one product feels careful and another feels chaotic even when both route to the same model.

They also explain why changing the model can expose weaknesses in a harness. A workaround created for an older model may constrain a newer one. Anthropic found this while simplifying a long-running application harness: newer models could handle work that previously required extra sprint decomposition and repeated evaluation. Every layer of scaffolding encodes an assumption about what the model cannot do. Those assumptions expire.

The best harness uses the smallest amount of orchestration needed to make the model reliable for the task you care about.

The Agent Loop: The Tiny Core Inside Every Harness

Strip away the interface, plugins, memory, remote runners, dashboards, and droid army. You eventually reach a loop:

while True:
    response = call_model(context, tools)

    if response.has_tool_calls:
        results = execute_tools(response.tool_calls)
        context.add(response, results)
    else:
        return response.text

That loop gives the model a way to act and observe what happened.

Suppose you ask an agent to fix a failing password-reset test. A normal run might look like this:

1. Model asks to list authentication files
2. Harness runs the file-list tool
3. Model asks to read reset-password.ts
4. Harness reads it and returns the contents
5. Model asks to read the failing test
6. Harness returns the test
7. Model proposes an edit
8. Harness checks the write policy and asks for approval
9. Harness applies the edit
10. Model asks to run the focused test
11. Harness runs it inside the environment
12. Test fails with a new error
13. Model reads the error and revises the edit
14. Test passes
15. Model returns a final explanation

The model never runs step 2, 4, 8, 9, or 11 itself. It requests actions. The harness owns the real world.

That ownership is where most engineering work begins.

The Nine Parts of a Good Agent Harness

The loop may fit on a screen. A useful harness needs more than a loop. I group the work into nine parts.

1. The Model Adapter

The model adapter translates the harness’s internal representation into the format expected by a model provider.

At minimum, it handles:

Authentication
Model selection
Instructions and messages
Tool schemas
Streaming responses
Tool-call parsing
Rate limits and retries
Token and cost accounting

This boundary lets the rest of the harness stay independent from a provider. Amp can route work across models because its product logic does not have to become a completely different application every time the underlying model changes. Codex can point at compatible Responses API endpoints. A home-grown harness can expose a ModelClient interface and swap Claude, an OpenAI model, or an open-weight model behind it.

Do not flatten every provider into the lowest common denominator, though. Models are trained around particular tool schemas, reasoning controls, and context-management features. A clean adapter should preserve useful provider-specific capabilities instead of pretending every API behaves identically.

2. The Context Builder

The context builder decides what the model knows at each step.

For a coding agent, that can include:

The base system instructions
The user’s task
Repository instructions from AGENTS.md or CLAUDE.md
Current working directory and Git state
Available tools and skills
Relevant files
Previous messages and tool results
A plan or task list
Memories from earlier sessions
Permission and sandbox rules

This is context engineering in its most practical form. Loading more context does not guarantee a better result. Irrelevant context consumes tokens, distracts the model, and makes the important instructions harder to find.

A good context builder answers three questions on every turn:

What must the model know right now?
What can it retrieve through a tool if needed?
What can be removed because it has gone stale?

Claude Code and Codex both load layered instruction files. Mature harnesses also pay attention to prompt caching, stable prefixes, and the position of changing content. The arrangement affects latency and cost as well as quality.

3. The Tool Registry

Tools give the model capabilities. The registry tells it which capabilities exist and how to call them.

A coding harness usually offers tools for:

Listing and searching files
Reading files
Applying precise edits
Running commands
Inspecting Git history and diffs
Searching the web
Calling MCP servers
Spawning specialist agents
Updating plans or task state

Tool design has a huge effect on agent behavior. Compare these descriptions:

run(command: string)
Runs a command.

run_tests(scope: string)
Runs the repository's configured test command. Use "focused" after a local
change and "full" before completing the task. Returns the exit code and the
last 12,000 characters of output.

The second tool gives the model a clear contract. It narrows the action, explains when to use it, and returns a bounded result. The first tool hands the model a loaded chainsaw and hopes for the best.

Tool results matter too. Dumping 300,000 characters of logs into the context can ruin the next turn. A harness should truncate noisy output, preserve the exit code, point to the full artifact, and keep the portion most likely to contain the error.

4. The Execution Environment

The harness needs somewhere to perform tool calls.

Local coding agents often execute in your working copy. Cloud agents commonly use containers or virtual machines. Parallel agents may receive separate Git worktrees. Browser agents need a browser session with its own cookies and network policy.

Anthropic separates managed agents into three concepts: a session, a harness, and a sandbox. The session records what happened, the harness decides what to do next, and the sandbox provides the hands.

The environment should be reproducible. An agent performs much better when it can discover how to install dependencies, start the application, and run tests without guessing. A repository with one reliable bootstrap command is easier for humans too. Funny how that works.

5. Policy, Permissions, and the Sandbox

An agent that can run code can delete files, expose credentials, install malicious dependencies, or follow instructions hidden in untrusted content.

Prompt instructions are not a security boundary. “Please do not read .env” is advice. A filesystem policy that denies access to .env is control.

A serious harness needs several layers:

Layer	Example Control
Filesystem	Read broadly, write only inside the workspace
Process	Run commands with reduced OS permissions
Network	Disable network access or allow approved domains
Secrets	Keep credentials outside the model’s context
Approval	Ask before destructive or external actions
Tool policy	Allow safe tools automatically and block dangerous inputs
Audit	Record requests, tool calls, results, and approval decisions

OpenAI describes this approach in its documentation on running Codex safely: technical boundaries for routine work, explicit approval for higher-risk actions, managed configuration, and agent-native logs.

Approval fatigue is real. If every file read and test run needs a click, users eventually approve everything without looking. Good policy allows low-risk, reversible work inside a constrained environment and reserves interruption for actions with meaningful consequences.

6. Session State and Persistence

The API call ends. The task may continue tomorrow.

A session store preserves the append-only history of user messages, model responses, tool calls, tool results, approvals, errors, and status changes. Persistence lets a user resume a thread, inspect the path to a result, or hand the work to another client.

OpenAI’s Codex harness treats thread lifecycle and persistence as core functionality. The same harness can power a terminal, IDE, web interface, or desktop application because the user interface talks to a persistent thread rather than reimplementing the loop.

For long-running work, save state outside the conversation too:

A task list with explicit statuses
A progress log
Git commits
Test results
Generated plans and specifications
Links to large artifacts

Anthropic’s early long-running harness used a feature list and claude-progress.txt to bridge fresh sessions. That simple approach worked because the environment became a durable source of truth.

7. Context Management and Compaction

Every tool call adds material to the conversation. A long test log, a large source file, and a few rounds of debugging can consume a context window surprisingly quickly.

A harness needs a strategy before that happens:

Drop tool results that are no longer useful
Replace large outputs with summaries and artifact references
Preserve decisions, constraints, unresolved work, and recent changes
Cache stable instructions
Compact the conversation near a token threshold
Start a fresh session with durable handoff files when appropriate

Compaction is lossy unless the provider supplies a native mechanism designed for the model. Even then, important details can disappear. The environment should carry critical state so the agent can recover by reading the plan, Git history, and current tests.

This is why “one million token context window” does not eliminate harness design. A larger room still becomes messy if you throw everything on the floor.

8. Verification and Feedback

An agent needs evidence that its work succeeded.

For coding tasks, useful feedback includes:

Unit and integration tests
Type checking
Linters
Builds
Browser tests
Screenshots
Static architecture rules
Security scanners
A clean Git diff

The most important feedback is mechanical. Telling an agent to “follow our architecture” helps. A custom lint rule that rejects an invalid dependency makes the requirement enforceable and returns an error the agent can fix.

OpenAI’s harness engineering team found that an underspecified environment slowed its agents. They moved architectural rules into custom linters and structural tests with remediation instructions in the errors. That is a powerful pattern: turn taste and constraints into feedback the agent can observe.

An evaluator agent can help when quality is subjective or the task pushes beyond the builder’s reliable range. It also adds latency and cost. Anthropic’s experiments found that newer models made the evaluator unnecessary for some tasks while it remained useful near the edge of model capability.

Start with deterministic checks. Add an evaluator when you can describe what it should judge and prove that it catches failures worth paying for.

9. Observability and Evals

Production agents fail in trajectories, not only final answers.

You need to see:

The input and assembled context
Which model and configuration ran
Tool calls and their latency
Tool results and errors
Approval decisions
Token usage and cost
Compaction events
Final environment state
Why the loop terminated

A polished final message can hide a failed test. The outcome is the repository after the run, not the sentence claiming the task is complete.

Build evals from real failures. If your agent repeatedly edits generated files, create tasks that test whether it finds the source. If it stops after a focused test, grade whether it runs the broader suite. If it loses requirements after compaction, build a long trajectory that crosses the threshold.

This turns harness development into engineering instead of prompt folklore.

How Claude Code, Codex, Amp, and Factory Package the Harness

All four products implement the same fundamental loop. Their product choices emphasize different parts of it.

Harness	Strongest Emphasis	Distinctive Choices	Best Fit
Claude Code	Deep local coding workflow	Layered instructions, skills, hooks, subagents, MCP, permission modes	Developers who want a flexible terminal agent
Codex	Portable, persistent agent runtime	Open-source core, strong sandboxing, persistent threads, local and cloud surfaces	Work that moves between terminal, IDE, desktop, and cloud
Amp	Opinionated multi-model experience	Capability dial, model routing, automatic subagents, Oracle, durable shared threads, remote orbs	Developers who want the product to choose the model stack
Factory	Structured software delivery	Spec Mode, custom droids, Missions, worktrees, enterprise context, headless execution	Larger projects that benefit from planning and coordination

This table will age because the products move quickly. The more durable comparison is how each harness answers five questions.

How Does It Find Context?

Claude Code and Codex lean heavily on repository instructions and active search. Amp adds its own retrieval and specialist calls. Factory combines repository instructions with reusable company context and structured specs.

The winning approach depends on your repository. An excellent retrieval system cannot recover conventions that only exist in someone’s head.

How Does It Control Work?

Claude Code gives the model a flexible loop with plans, subagents, skills, and hooks. Codex exposes durable threads that can be resumed and forked. Amp automatically calls specialists and lets the user choose a capability level. Factory gives planning and multi-agent coordination more visible product structure through Spec Mode and Missions.

Where Does It Execute?

Local execution gives the agent the environment you already use. Remote execution keeps work running after your laptop closes and makes isolation easier. Worktrees allow several agents to edit one repository without sharing a working directory.

The right answer increasingly involves all three: local for interactive debugging, remote for long tasks, and isolated worktrees for parallelism.

How Does It Keep You Safe?

Look past the permission prompt. Check the actual filesystem boundary, network policy, secret handling, managed configuration, and audit trail.

An agent asking for approval before rm is useful. An OS-level sandbox preventing it from reaching outside the workspace is stronger.

How Does It Know It Is Done?

This is where many harnesses still struggle. The model decides it has enough evidence and returns a final message. Strong repository instructions, mandatory checks, hooks, evaluator passes, and CI can make that decision more reliable.

When I test a coding agent, I care less about whether it produced a plausible diff in one shot. I watch whether it reproduces the problem, reads the right code, uses feedback, notices failed tests, and proves the final state.

Build a Small Agent Harness in Python

The fastest way to understand a harness is to build one.

We are going to make a small repository agent called mini_harness.py. It can:

List files
Read files
Search text
Propose exact replacements
Ask before writing
Run one configured test command
Stop after a fixed number of model turns
Save the complete trajectory as JSON

It will use Claude’s Messages API because the manual tool loop is easy to see. The same architecture works with another provider.

This is an educational harness. It constrains writes to the selected repository and prevents the model from choosing arbitrary shell commands. It does not provide OS-level isolation. Run it against a disposable repository until you have added a real sandbox.

Set Up the Project

Create a directory and install the Anthropic SDK:

mkdir mini-harness
cd mini-harness
python -m venv .venv
source .venv/bin/activate
pip install anthropic

Export your API key:

export ANTHROPIC_API_KEY="your-api-key"

Create mini_harness.py with the following code:

import argparse
import json
import os
import subprocess
from datetime import datetime, timezone
from pathlib import Path

import anthropic


MAX_STEPS = 20
MAX_TOOL_OUTPUT = 12_000


def safe_path(root: Path, relative_path: str) -> Path:
    """Resolve a path and reject anything outside the repository."""
    candidate = (root / relative_path).resolve()
    if not candidate.is_relative_to(root):
        raise ValueError(f"Path escapes repository: {relative_path}")
    return candidate


def list_files(root: Path, pattern: str = "*") -> str:
    files = []
    for path in root.rglob(pattern):
        relative = path.relative_to(root)
        if path.is_file() and ".git" not in relative.parts:
            files.append(str(relative))
        if len(files) >= 500:
            files.append("...truncated after 500 files")
            break
    return "\n".join(files)


def read_file(root: Path, path: str) -> str:
    target = safe_path(root, path)
    content = target.read_text(errors="replace")
    if len(content) > MAX_TOOL_OUTPUT:
        return content[:MAX_TOOL_OUTPUT] + "\n...output truncated"
    return content


def search_text(root: Path, query: str) -> str:
    matches = []
    for path in root.rglob("*"):
        if not path.is_file() or ".git" in path.relative_to(root).parts:
            continue
        if not path.resolve().is_relative_to(root):
            continue
        try:
            for line_number, line in enumerate(
                path.read_text(errors="replace").splitlines(), start=1
            ):
                if query.lower() in line.lower():
                    matches.append(
                        f"{path.relative_to(root)}:{line_number}: {line.strip()}"
                    )
                    if len(matches) >= 100:
                        return "\n".join(matches) + "\n...truncated after 100 matches"
        except OSError:
            continue
    return "\n".join(matches) or "No matches found"


def replace_in_file(
    root: Path, path: str, old_text: str, new_text: str
) -> str:
    target = safe_path(root, path)
    content = target.read_text()
    count = content.count(old_text)

    if count != 1:
        raise ValueError(
            f"Expected one exact match in {path}, found {count}. "
            "Read the file again and choose a unique block."
        )

    print(f"\nProposed edit: {path}")
    print("-" * 60)
    print(old_text)
    print("+" * 60)
    print(new_text)
    approved = input("Apply this edit? [y/N] ").strip().lower() == "y"

    if not approved:
        return "User rejected the edit"

    target.write_text(content.replace(old_text, new_text, 1))
    return f"Updated {path}"


def run_tests(root: Path) -> str:
    """Run a fixed command chosen by the user, never by the model."""
    command = os.environ.get("HARNESS_TEST_COMMAND", "python -m pytest")
    completed = subprocess.run(
        command.split(),
        cwd=root,
        capture_output=True,
        text=True,
        timeout=300,
    )
    output = completed.stdout + completed.stderr
    if len(output) > MAX_TOOL_OUTPUT:
        output = output[-MAX_TOOL_OUTPUT:]
    return f"exit_code={completed.returncode}\n{output}"


TOOLS = [
    {
        "name": "list_files",
        "description": "List files in the repository. Use a glob pattern to narrow results.",
        "input_schema": {
            "type": "object",
            "properties": {"pattern": {"type": "string"}},
            "required": ["pattern"],
        },
    },
    {
        "name": "read_file",
        "description": "Read a UTF-8 text file inside the repository.",
        "input_schema": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
        },
    },
    {
        "name": "search_text",
        "description": "Search repository text for a case-insensitive literal string.",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
    {
        "name": "replace_in_file",
        "description": (
            "Replace one exact, unique block of text in a repository file. "
            "The user must approve the edit."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string"},
                "old_text": {"type": "string"},
                "new_text": {"type": "string"},
            },
            "required": ["path", "old_text", "new_text"],
        },
    },
    {
        "name": "run_tests",
        "description": (
            "Run the test command configured by HARNESS_TEST_COMMAND and return "
            "the exit code and output."
        ),
        "input_schema": {"type": "object", "properties": {}},
    },
]


SYSTEM_PROMPT = """You are a careful coding agent working in one repository.

Investigate before editing. Make the smallest change that solves the task.
Read a file before editing it. Use exact replacements only.
Run the configured tests after a change.
Never claim success when tests fail.
If you cannot verify the result, say exactly what remains unverified.
"""


def execute_tool(root: Path, name: str, tool_input: dict) -> str:
    if name == "list_files":
        return list_files(root, tool_input["pattern"])
    if name == "read_file":
        return read_file(root, tool_input["path"])
    if name == "search_text":
        return search_text(root, tool_input["query"])
    if name == "replace_in_file":
        return replace_in_file(root, **tool_input)
    if name == "run_tests":
        return run_tests(root)
    raise ValueError(f"Unknown tool: {name}")


def save_trace(trace: list[dict]) -> Path:
    trace_dir = Path(".harness-traces")
    trace_dir.mkdir(exist_ok=True)
    timestamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S")
    path = trace_dir / f"run-{timestamp}.json"
    path.write_text(json.dumps(trace, indent=2))
    return path


def run_agent(root: Path, task: str) -> str:
    client = anthropic.Anthropic()
    messages = [{"role": "user", "content": task}]
    trace = [{"type": "user", "content": task}]

    for step in range(1, MAX_STEPS + 1):
        response = client.messages.create(
            model="claude-sonnet-5",
            max_tokens=8_000,
            system=SYSTEM_PROMPT,
            tools=TOOLS,
            messages=messages,
        )

        assistant_content = [block.model_dump() for block in response.content]
        messages.append({"role": "assistant", "content": assistant_content})
        trace.append(
            {
                "type": "assistant",
                "step": step,
                "stop_reason": response.stop_reason,
                "content": assistant_content,
            }
        )

        if response.stop_reason != "tool_use":
            text = "\n".join(
                block.text for block in response.content if block.type == "text"
            )
            trace_path = save_trace(trace)
            return f"{text}\n\nTrace: {trace_path}"

        tool_results = []
        for block in response.content:
            if block.type != "tool_use":
                continue

            try:
                result = execute_tool(root, block.name, block.input)
                is_error = False
            except Exception as error:
                result = f"{type(error).__name__}: {error}"
                is_error = True

            print(f"[{step}] {block.name}: {result[:200]}")
            tool_result = {
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": result,
                "is_error": is_error,
            }
            tool_results.append(tool_result)
            trace.append(
                {
                    "type": "tool_result",
                    "step": step,
                    "tool": block.name,
                    "input": block.input,
                    "result": result,
                    "is_error": is_error,
                }
            )

        messages.append({"role": "user", "content": tool_results})

    trace.append({"type": "error", "content": "Maximum steps reached"})
    trace_path = save_trace(trace)
    return f"Stopped after {MAX_STEPS} model calls. Trace: {trace_path}"


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("task", help="The task for the agent")
    parser.add_argument("--repo", default=".", help="Repository to work in")
    args = parser.parse_args()

    repository = Path(args.repo).resolve()
    if not (repository / ".git").exists():
        raise SystemExit(f"Not a Git repository: {repository}")

    print(run_agent(repository, args.task))

Run the Harness

Point it at a disposable repository and configure the test command:

export HARNESS_TEST_COMMAND="python -m pytest"
python mini_harness.py \
  --repo ../sample-project \
  "Fix the failing test for expired password-reset tokens. Keep the change small."

A run will look roughly like this:

[1] list_files: src/auth/reset.py
tests/test_reset.py
[2] read_file: def validate_reset_token(token): ...
[3] read_file: def test_expired_token_returns_400(): ...

Proposed edit: src/auth/reset.py
------------------------------------------------------------
if decoded["exp"] < now:
    raise TokenError()
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
if decoded["exp"] <= now:
    raise ExpiredTokenError()
Apply this edit? [y/N] y

[5] replace_in_file: Updated src/auth/reset.py
[6] run_tests: exit_code=0
12 passed in 0.41s

The expired-token boundary now uses the domain-specific error expected by the
HTTP handler. The configured test suite passes: 12 tests passed.

Trace: .harness-traces/run-20260810-184215.json

That small program contains the core pieces:

messages is the in-memory session.
SYSTEM_PROMPT supplies stable operating instructions.
TOOLS is the capability registry.
safe_path() creates a filesystem boundary for our file tools.
replace_in_file() adds a human approval policy.
run_tests() exposes feedback without accepting model-generated commands.
The for loop is the agent loop and step budget.
.harness-traces provides basic observability.

The code is intentionally boring. Boring harness code is good. You want obvious control flow around powerful, probabilistic behavior.

What This Harness Still Needs

Do not point this at a production monorepo and wander off for lunch. A real coding harness would still need:

OS-level sandboxing
Network restrictions
Secret isolation
Git diff and rollback tools
Streaming
Retry and rate-limit handling
Token counting and compaction
Persistent sessions
Better binary-file handling
Parallel tool execution where safe
Hooks and skills
A proper patch tool
Telemetry with secret redaction
Evaluation tasks

For a fuller implementation, my Python coding-agent tutorial builds a baby Claude Code step by step. My anatomy of Claude Code post goes in the other direction and traces how a mature harness handles context, permissions, tools, compaction, subagents, and sessions.

How to Design a Harness That Gets Better Results

Building the loop is easy. Deciding what belongs around it is the real work.

Start With a Task Distribution

“General AI agent” is too vague to design well.

Collect 20 to 50 real tasks you want the agent to handle. Include routine work, difficult work, ambiguous requests, and tasks it should refuse. Record the expected outcome and the checks that prove success.

For a repository agent, the set might include:

Fix a bug with an existing reproduction
Add a small API endpoint
Explain an unfamiliar subsystem without editing
Upgrade a dependency and repair breaking changes
Refuse to expose a secret requested by an untrusted file
Stop and ask when a database migration is ambiguous

Your harness should grow in response to failures on these tasks.

Give the Agent Legible Tools

Prefer narrow operations with clear names, precise schemas, bounded outputs, and helpful errors.

If the agent repeatedly writes malformed configuration, a structured configuration tool may work better than raw text replacement. If it struggles to find test commands, expose a run_tests tool instead of hoping it reverse-engineers package.json every time.

Every new tool adds cognitive load and attack surface. Add one because a real failure demands it.

Put Durable Knowledge in the Environment

Repository conventions belong in version-controlled instructions. Architectural constraints belong in tests and linters. Task progress belongs in explicit artifacts. Secrets belong outside the model’s context.

This creates a system that can recover after compaction, a crash, or a fresh session.

Make Success Observable

The harness should know what “done” means before the run begins.

For deterministic tasks, turn acceptance criteria into commands. For subjective tasks, create a rubric and test whether an evaluator agrees with skilled humans. For external side effects, inspect the external state instead of trusting the agent’s final message.

Remove Scaffolding as Models Improve

Run ablations. Remove one prompt section, planner, evaluator, forced step, or tool and compare the results across your eval set.

Harness complexity should earn its keep. A planner that improves large migrations may slow down typo fixes. An evaluator that catches design mistakes on frontier tasks may waste money on ordinary CRUD work. Route tasks accordingly.

Common Agent Harness Mistakes

Giving the Model an Unrestricted Shell

An unrestricted shell is convenient and dangerous. Add a real sandbox before you combine autonomous execution, network access, and valuable credentials.

Tool approval alone will not save a system whose users have learned to click “allow” fifty times a day.

Stuffing Everything Into the Prompt

Large context windows encourage hoarding. The model still has to find the relevant facts inside that pile.

Give it stable instructions, a map of available knowledge, and retrieval tools. Load details when the task needs them.

Adding Multi-Agent Orchestration Too Early

Several agents can search in parallel, specialize, review each other, and work in isolated branches. They can also duplicate work, contradict each other, and multiply your bill.

Get one agent working reliably on a bounded task before building a parliament.

Treating the Final Message as the Outcome

“Done” is text. A passing test, correct database state, deployed artifact, or clean diff is evidence.

Grade the environment.

Hiding Errors From the Model

Tools should return useful failures. Include the exit code, relevant stderr, and a suggestion when the contract was violated. The agent can often recover if the harness tells it what actually happened.

Building Around One Model’s Weakness Forever

Prompts and orchestration accumulate like old feature flags. Test them when you upgrade the model. The new model may handle the task directly, and the workaround may now make it worse.

Skipping Traces

If you only store the final response, every failure becomes a ghost story. Save the trajectory with sensitive data redacted. You need it for debugging, security review, cost analysis, and eval creation.

Should You Build a Harness or Use One?

For most coding work, start with an existing harness.

Claude Code, Codex, Amp, and Factory have already solved thousands of annoying details around streaming, terminal interaction, edits, permissions, context, and recovery. Configure repository instructions, add the skills and tools your work needs, and measure the result.

Build your own harness when at least one of these is true:

Your task needs a specialized environment or tool contract
You need control over model routing and cost
You have compliance or data-boundary requirements
The agent sits inside your product
You need a workflow that existing coding agents cannot express
Harness behavior is part of your product advantage

Agent SDKs occupy the useful middle. The OpenAI Agents SDK handles the loop, tools, sessions, handoffs, guardrails, and tracing. Anthropic’s Agent SDK exposes the same core primitives that power Claude Code. You keep control over the workflow without rebuilding every protocol detail.

My recommendation is simple: adopt a harness first, extend it second, and build from scratch when the task gives you a concrete reason.

The Harness Is Where Your Agent Becomes a Product

Models will keep improving. Context windows will grow. Tool use will become more reliable. Some scaffolding we consider essential today will disappear into the model or provider API.

The core product questions will remain. What can the agent see? What can it do? Where does it run? What stops it? What survives? How does it know it succeeded?

Those answers live in the harness.

If you want to understand coding agents, build the tiny loop once. Then inspect a mature system and notice how much work surrounds it. The intelligence gets the attention. The harness turns that intelligence into useful, repeatable, and reasonably safe work.

Now go build one. Preferably inside a sandbox.