You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
154 lines
7.8 KiB
154 lines
7.8 KiB
# The Philosophy of Agent Harness Engineering
|
|
|
|
> **The model already knows how to be an agent. Your job is to build it a world worth acting in.**
|
|
|
|
## The Fundamental Truth
|
|
|
|
Strip away every framework, every library, every architectural pattern. What remains?
|
|
|
|
A loop. A model. An invitation to act.
|
|
|
|
The agent is not the code. The agent is the model itself -- a vast neural network trained on humanity's collective problem-solving, reasoning, and tool use. The code merely provides the opportunity for the model to express its agency.
|
|
|
|
The code is the harness. The model is the agent. These are not interchangeable. Confuse them, and you will build the wrong thing.
|
|
|
|
## What an Agent IS
|
|
|
|
An agent is a neural network -- a Transformer, an RNN, a learned function -- that has been trained, through billions of gradient updates on action-sequence data, to perceive an environment, reason about goals, and take actions to achieve them.
|
|
|
|
A human is an agent: a biological neural network shaped by evolution. DeepMind's DQN is an agent: a convolutional network that learned to play Atari from raw pixels. OpenAI Five is an agent: five networks that learned Dota 2 teamwork through self-play. Claude is an agent: a language model that learned to reason and act from the breadth of human knowledge.
|
|
|
|
In every case, the agent is the trained model. Not the game engine. Not the Dota 2 client. Not the terminal. The model.
|
|
|
|
## What an Agent Is NOT
|
|
|
|
Prompt plumbing is not agency. Wiring together LLM API calls with if-else branches, node graphs, and hardcoded routing logic does not produce an agent. It produces a brittle pipeline -- a Rube Goldberg machine with an LLM wedged in as a text-completion node.
|
|
|
|
You cannot engineer your way to agency. Agency is learned, not programmed. No amount of glue code will emergently produce autonomous behavior. Those systems are the modern resurrection of GOFAI -- symbolic rule systems the field abandoned decades ago, now spray-painted with an LLM veneer.
|
|
|
|
## The Harness: What We Actually Build
|
|
|
|
If the model is the agent, then what is the code? It is the **harness** -- the environment that gives the agent the ability to perceive and act in a specific domain.
|
|
|
|
```
|
|
Harness = Tools + Knowledge + Observation + Action Interfaces + Permissions
|
|
```
|
|
|
|
### Tools: The Agent's Hands
|
|
|
|
Tools answer: **What can the agent DO?**
|
|
|
|
Each tool is an atomic action the agent can take in its environment. File read/write, shell execution, API calls, browser control, database queries. The model needs to understand what each tool does, but not how to sequence them -- it will figure that out.
|
|
|
|
**Design principle**: Atomic, composable, well-described. Start with 3-5. Add more only when the model consistently fails to accomplish tasks because a tool is missing.
|
|
|
|
### Knowledge: The Agent's Expertise
|
|
|
|
Knowledge answers: **What does the agent KNOW?**
|
|
|
|
Domain expertise that turns a general agent into a domain specialist. Product documentation, architectural decisions, regulatory requirements, style guides. Inject on-demand (via tool_result), not upfront (via system prompt). Progressive disclosure preserves context for what matters.
|
|
|
|
**Design principle**: Available but not mandatory. The agent should know what knowledge exists and pull what it needs.
|
|
|
|
### Context: The Agent's Memory
|
|
|
|
Context is the thread connecting individual actions into coherent behavior. What has been said, tried, learned, and decided.
|
|
|
|
**Design principle**: Context is precious. Protect it. Isolate subtasks that generate noise (s04). Compress when history grows long (s06). Persist goals beyond single conversations (s07).
|
|
|
|
### Permissions: The Agent's Boundaries
|
|
|
|
Permissions answer: **What is the agent ALLOWED to do?**
|
|
|
|
Sandbox file access. Require approval for destructive operations. Enforce trust boundaries between the agent and external systems. This is where safety engineering meets harness engineering.
|
|
|
|
**Design principle**: Constraints focus behavior, not limit it. "One task in_progress at a time" forces sequential focus. "Read-only subagent" prevents accidental modifications.
|
|
|
|
### Task-Process Data: The Agent's Training Signal
|
|
|
|
Every action sequence the agent executes in your harness is training signal. The perception-reasoning-action traces from real deployments are the raw material for fine-tuning the next generation of agent models. Your harness doesn't just serve the agent -- it can help evolve the agent.
|
|
|
|
## The Universal Loop
|
|
|
|
Every effective agent -- regardless of domain -- follows the same pattern:
|
|
|
|
```
|
|
LOOP:
|
|
Model sees: conversation history + available tools
|
|
Model decides: act or respond
|
|
If act: tool executed, result added to context, loop continues
|
|
If respond: answer returned, loop ends
|
|
```
|
|
|
|
This is not a simplification. This is the actual architecture. Everything else is harness engineering -- mechanisms layered on top of this loop to make the agent more effective. The loop belongs to the agent. The mechanisms belong to the harness.
|
|
|
|
## Principles of Harness Engineering
|
|
|
|
### Trust the Model
|
|
|
|
The most important principle: **trust the model**.
|
|
|
|
Don't anticipate every edge case. Don't build elaborate decision trees. Don't pre-specify the workflow.
|
|
|
|
The model is better at reasoning than any rule system you could write. Your conditional logic will fail on edge cases. The model will reason through them.
|
|
|
|
**Give the model tools and knowledge. Let it figure out how to use them.**
|
|
|
|
### Constraints Enable
|
|
|
|
This seems paradoxical, but constraints don't limit agents -- they focus them.
|
|
|
|
A todo list with "only one task in progress" forces sequential focus. A subagent with read-only access prevents accidental modifications. A context compression threshold keeps history from overwhelming.
|
|
|
|
The best constraints prevent the model from getting lost, not micromanage its approach.
|
|
|
|
### Progressive Complexity
|
|
|
|
Never build everything upfront.
|
|
|
|
```
|
|
Level 0: Model + one tool (bash) -- s01
|
|
Level 1: Model + tool dispatch map -- s02
|
|
Level 2: Model + planning -- s03
|
|
Level 3: Model + subagents + skills -- s04, s05
|
|
Level 4: Model + context management + persistence -- s06, s07, s08
|
|
Level 5: Model + teams + autonomy + isolation -- s09-s12
|
|
```
|
|
|
|
Start at the lowest level that might work. Move up only when real usage reveals the need.
|
|
|
|
## The Mind Shift
|
|
|
|
Building harnesses requires a fundamental shift in thinking:
|
|
|
|
**From**: "How do I make the system do X?"
|
|
**To**: "How do I enable the model to do X?"
|
|
|
|
**From**: "What should happen when the user says Y?"
|
|
**To**: "What tools would help address Y?"
|
|
|
|
**From**: "What's the workflow for this task?"
|
|
**To**: "What does the model need to figure out the workflow?"
|
|
|
|
**From**: "I'm building an agent."
|
|
**To**: "I'm building a harness for the agent."
|
|
|
|
The best harness code is almost boring. Simple loops. Clear tool definitions. Clean context management. The magic isn't in the code -- it's in the model.
|
|
|
|
## The Vehicle Metaphor
|
|
|
|
The model is the driver. The harness is the vehicle.
|
|
|
|
A coding agent's vehicle is its IDE, terminal, and filesystem. A farm agent's vehicle is its sensor array, irrigation controls, and weather data. A hotel agent's vehicle is its booking system, guest channels, and facility APIs.
|
|
|
|
The driver generalizes. The vehicle specializes. Your job as a harness engineer is to build the best vehicle for your domain -- one that gives the driver maximum visibility, precise controls, and clear boundaries.
|
|
|
|
Build the cockpit. Build the dashboard. Build the controls. The pilot is already trained.
|
|
|
|
## Conclusion
|
|
|
|
The model is the agent. The code is the harness. Know which one you're building.
|
|
|
|
You are not writing intelligence. You are building the world intelligence inhabits. The quality of that world -- how clearly the agent can perceive, how precisely it can act, how rich its knowledge -- directly determines how effectively the intelligence can express itself.
|
|
|
|
Build great harnesses. The agent will do the rest.
|
|
|