A model can be very smart, but an agent that has to work in a real environment for tens of minutes, hours, or across several context windows needs more than a model that "thinks right." It has to run, pause, observe, recover, coordinate, verify, and remember. None of that is the model's job. It's the harness's job.
ChatGPT and I wrote this together over one very long technical discussion. It started with reading the open-source Codex harness and kept going deeper, until it turned into a bigger question: if you treat agents as a digital team that can own real work, how do you design its rules, its knowledge system, and its runtime?
First, the levels of evidence. Everything here falls into one of three kinds. Verified in source means it's stated in the Codex open-source files or OpenAI's official engineering posts as of October 2026. Architectural abstraction means a logical diagram drawn to explain an idea, not a source dependency graph. Design proposal means something we worked out in the discussion that Codex does not implement natively. I label each one where it appears. The code keeps changing, so treat the repository at the relevant version as the final word.
If you only remember one line: the LLM provides open-ended intelligence, the harness provides an executable runtime structure, and only the two together make an agent you can actually use.
A harness is not a prompt
A harness is the execution, state, and control system that sits between the model and the outside world.
Say the user asks: "Find the bug in the login module, fix it, and make sure the tests pass." The LLM can decide which files to look at, generate a patch, and explain the error. But reading files, starting processes, approving permissions, writing logs, and handling interrupts are all done by the runtime.
There are really two execution paths inside a harness:
- Model-driven: when semantic judgment is needed, the harness calls the LLM and lets it propose a tool call or give a final answer. For example, the model outputs
read_file("login.ts"), and the harness checks permissions, reads the file, and passes the text back. - Program-driven: the user clicks stop, a timeout is detected, concurrency needs limiting, state needs writing to SQLite. The harness just runs its own program logic without asking the LLM. For example, on an interrupt event it cancels the task with a CancellationToken and cleans up, and that step makes no model call at all.
So the accurate way to say it is: a harness is a software system with an LLM embedded in it, not a system whose code is all generated on the fly by an LLM. We came back to this point again and again in our discussion.
"Agent runtime" and "agent harness" overlap a lot. You can think of the runtime as the execution-focused kernel and the harness as the whole thing, covering execution, context, tools, safety, and collaboration. That split is only a design convenience; the industry has no shared definition. OpenAI's own description of the harness is also organized around the agent loop and the logic around it [1].
How Codex is split up
The Codex CLI Rust workspace has many crates, and the count changes with the main branch, so it isn't worth memorizing. What is worth remembering are a few key boundaries: codex-core holds most of the agent logic, codex-protocol defines the internal data protocol, codex-api talks to the model, codex-app-server provides the interaction protocol for clients, and there are further components for persistence, the TUI, sandboxing, and patching [2].
When a system has both a desktop app and a CLI, the worst thing you can do is write a separate agent loop for each interface. The App Server exposes the underlying capabilities over bidirectional JSON-RPC: one client request can map to many streamed events, and the server can also turn around and ask the client for approval. That separates the interface from the agent's execution lifecycle [1].
There's also a very Unix detail: codex-arg0. The program picks an entry point based on the executable name it was launched with, so for example apply_patch and the Linux sandbox helper get dispatched to different logic [3]. This isn't "telling the LLM who it is." It's multi-entry reuse at the process level.
Task, Turn, Sampling: how one instruction becomes dozens of steps
The user states the request once, and Codex keeps calling the model, running tools, and observing results until the turn ends.
People often describe this as Task Loop → Turn Loop → Sampling Loop. It's a helpful mental model, but the source does not contain three neatly nested while loops with those names. More precisely: the Session's submission loop dispatches operations, SessionTask manages the task lifecycle, and run_turn issues model sampling requests one after another within a turn [4].
| Concept | Question it answers | Mainly controlled by |
|---|---|---|
| Session / Task | When does the task start, get cancelled, end? | The harness's task mechanism |
| Turn | Should this turn keep going? | Model signals + harness state |
| Sampling | What response items did this model request produce? | Generated by the model, received and handled by the harness |
| Tool Call | Which tool, with what arguments? | Proposed by the model, executed by the harness |
Take the login bug. A single turn might involve five samplings like these (an illustrative trace; real Codex does not fix the number in advance):
| Sampling | What happens | Model output |
|---|---|---|
| 01 Analyze the request | The model sees the task and project environment for the first time and decides to read the login module first | read_file("src/login.ts") |
| 02 Locate the problem | The harness returns the file contents, and the model notices the validation flow misses an error branch | read_file("src/auth.ts") |
| 03 Generate the fix | Based on the new information the model proposes a patch; the harness checks its scope and applies it | apply_patch({ ... }) |
| 04 Run the tests | The harness runs the tests and collects the exit code and logs, so the model sees real feedback next time | run("npm test -- login") |
| 05 Finish | Tests pass and there's no pending input, so the model gives a final answer and the harness ends the turn | Final Message · turn/completed |
In pseudocode it looks like this (not actual Codex code):
async fn run_turn() {
loop {
let context = build_model_input();
let result = run_sampling_request(context).await;
if result.has_tool_calls() {
execute_and_record(result.tool_calls()).await;
}
if !needs_follow_up(result, pending_input) {
break;
}
}
}
Who decides how many samplings there are? The main agent doesn't allot "ten" up front, and the user doesn't specify it either. The LLM produces responses such as tool calls in one request; after the harness executes them, it decides dynamically whether to make another request, based on needs_follow_up, pending input, hooks, and stop conditions [4].
One more thing: a task ending is not the same as a task succeeding. The model saying "I fixed it" and the tests actually passing are two different things. The second requires the harness to read external evidence of verification. In a high-reliability system, tests, acceptance, and resource budgets should all be independent of the model's own claims.
Run in parallel, collect in order
In one sampling, the model can emit several tool calls. Whether they can run concurrently depends on the tools and the execution environment, not on "every call needs another sampling."
If the model wants to read three unrelated files, they can run at the same time. If two calls both modify the same database record, you have to be careful. Codex's tool runtime uses mechanisms like read-write locks to separate tools that can run concurrently from tools that need exclusive execution [5].
While we're here, let's separate tools from skills. A tool is an execution capability that's already implemented, such as reading a file or running a shell command. A skill is more like a work manual: it tells the model what process to follow with the existing tools, sometimes with scripts, templates, and reference material attached.
A useful dividing line: the LLM may output a structured call like read_file(path="login.ts") without writing any code at all, or it may generate a shell command and hand it to the harness's existing command-execution tool. Cancelling tasks, writing logs, and handling timeouts are usually done by the harness's own Rust code.
Op / Event: how the client sees what the agent is doing
Codex doesn't wait until the whole task is done and then return one giant result. The client sends operations to the harness, and the harness keeps sending events back to the client.
Note that there are two layers of protocol: the Op / Event types inside the Rust core, and the JSON-RPC methods and notifications the App Server exposes. They are semantically related but don't share the same names [1].
Event-driven design is also the foundation for multi-agent collaboration. A subtask finishing, a tool failing, an approval request, a user interrupt: the runtime receives, records, and dispatches each of these first, and only triggers new model reasoning when something needs to be re-judged.
SubAgents: when to split, when to wait
Sampling is a reasoning iteration inside one agent. A subagent is a separate executor with its own context and workflow. Parallel doesn't automatically mean faster.
Say a website needs a database, a backend, and a frontend. Handing it all to the main agent to do serially is simple but slow. Splitting it across three subagents allows parallel work, but the interface contracts and write scopes have to be settled first. The key is not deciding up front to "spin up three agents," but looking at the task's dependency DAG and shared-resource conflicts first.
The Codex multi-agent spec includes tools such as spawn_agent, send_message, followup_task, wait_agent, and interrupt_agent. Notably, in V2, wait_agent can wait for a mailbox update, new user input, or a timeout. It doesn't necessarily wait for all subagents to finish [6].
| Situation | Suggested mechanism | Why |
|---|---|---|
| A and B each read logs independently | Parallel | No write conflicts, both can move forward at once |
| Backend depends on a schema that isn't settled yet | Agree on the contract, then go parallel | Avoids rework built on an outdated structure |
| All three agents need to edit the same file | Coordinate write scopes | Keeps patches from overwriting each other and losing updates |
| Main agent has no independent work and depends on the results | wait_agent |
No need for the LLM to keep polling "are you done yet" |
A mailbox is a collaboration mechanism, not a shared brain. send_message sends a message but doesn't trigger a new turn on its own; followup_task can trigger a follow-up turn when the target is idle. This lets the main agent pause its own sampling while subagents keep working [6].
How much context a subagent inherits comes down to the abstraction of context isolation. Inheriting everything makes it easy to pick up where things left off, but it costs more and can pollute the context. Inheriting less helps focus. Starting from a blank context is cleanest but needs a more complete task description. The specific options and defaults depend on the tool protocol of the version you're running, so don't treat any one version's "three levels of inheritance" as a stable official interface.
The project-management principle we arrived at: run independent tasks in parallel and strongly dependent tasks in series; if the main agent has work on the critical path, it keeps working, and it only waits when there's nothing left for it to do. When a subagent finds an anomaly that affects others, it reports to the main agent first, and the main agent decides whether to reassign work. The runtime handles passing messages and carrying out interrupts.
World State: telling the model what changed in the world
Across dozens of samplings, AGENTS.md, permissions, and environment config can all change. How do you sync those changes to the model while disturbing the existing context as little as possible?
Codex's WorldStateSection defines stable IDs, snapshots, and render_diff for different kinds of model-visible state. The harness can compare against the previous snapshot to decide what needs updating [7]. The diff here is a structured state comparison with differential rendering. It doesn't mean every change is sent to the model as just a few lines of text.
Why this matters for the prefix KV cache
When two model inputs start with exactly the same token sequence, the server has a chance to reuse the prefix KV cache it already computed. Keeping old messages and appending new information at the end usually hits the cache more easily than repeatedly rewriting the beginning. But an identical prefix is only a necessary condition; it doesn't guarantee a hit [8].
So: World State optimizes state syncing, prompt caching optimizes computation over repeated prefixes, and compaction deals with context growth. None of them replaces the others.
What happens when the context fills up
To me this is the most interesting part of the whole design: a gradual shift from "compressing the past" to "saving state and reorganizing the working set."
Path one: handoff-style compaction
Codex's compaction prompt asks the model to write a handoff summary for another LLM that will take over the task: keep progress, key decisions, constraints, remaining steps, and important references [9]. The question it answers is not "what did we talk about," but "how does the next executor carry on."
The difference is concrete. "The database module is mostly done" is a summary. "Which files were changed, which test failed, why it failed, and which user constraints must not be violated" is a handoff you can keep working from. Both can lose detail, but the handoff is at least explicitly optimized for the goal of resuming the task.
Path two: token budget, open a new window
Codex's compact_token_budget.rs clearly includes a mode that skips summary compaction by the model or server and calls start_new_context_window to set up a new window. It still hangs off the compaction lifecycle so it can reuse the hook and event machinery. That doesn't mean traditional compaction has been retired [10].
Each of the three materials does one job. The Thread Hint is a short resume cue. Notes are the model's organized notes on the current task (progress, decisions, to-dos). History is the traceable raw record of the session and tool calls. Where the relevant extensions are available, the new window gets the resume hint first and then reads details as needed. The exact interfaces, enabling conditions, and retention policies vary by version and configuration, so you can't assume every Codex product has the same tools.
This is also where the approach has its most important limit: "the history is still there" doesn't mean "the model will remember to look." If the model didn't write something into Notes and doesn't know what to search for in History, the information still exists but can't be recovered. The problem shifts from storage to retrieval decisions.
External storage can ease the problem of context capacity. But "when to remember to go look for what" is still a core problem for any intelligent system.
Long-term memory: not just remembering conversations, but distilling experience
The Codex Memories README describes a two-phase mechanism. When an eligible root session starts, a memory job is triggered asynchronously in the background. Phase 1 reads eligible rollouts, has the model extract raw_memory and rollout_summary, and stores them in the state database. Phase 2, under a global lock, selects candidates and organizes a file workspace, and then a dedicated consolidation agent produces denser long-term memory [11].
How SQLite, Markdown, and a KG relate
You have to separate "the format of the record" from "the database you query." Codex memory isn't all stored as Markdown. The state database holds structured task records and metadata, rollout history has its own persistence format, and files like MEMORY.md hold long-form knowledge that's friendly to an LLM. Markdown is a presentation layer, SQLite is an embedded relational database, and Neo4j is a graph database. A KG is a knowledge model; it doesn't require Neo4j.
| Storage | What it's good for |
|---|---|
| SQLite · local state | State indexes, metadata, and transactions for a single-user local runtime; embedded in the app, no separate database service |
| Markdown / files · narrative memory | Complex explanations, work handoffs, reusable procedures, human-readable Notes and Skills |
| PostgreSQL · shared service (proposal) | A server-side shared agent platform: transactions, versions, centralized permissions, concurrent writes; a KG can also be represented in relational tables |
| Neo4j · graph (optional) | Evaluate it when multi-hop relationships, path queries, and complex graph traversal truly become core needs |
It's not "a million users means you must use PostgreSQL." A million users each running an independent agent locally, each with its own SQLite, is perfectly reasonable. What actually drives the choice is whether state needs to be centrally shared, accessed across machines, or updated with high concurrency.
What long-term memory should know
Three kinds of things: stable user and project preferences, reusable task experience, and environment facts that have been verified. But each entry needs a scope: which project, which environment, what time, which version. A guess made during one exploration should not automatically become organizational policy.
Codex's consolidation looks at signals such as usage_count, last_usage, and generation time, and also allows stable, repeated procedures to be distilled further into skills [11][12]. But remember: being used repeatedly is not the same as being proven correct.
Connecting the KG, RAG, the context window, and the agent loop
These four aren't four competing products. They're four stages of data flow, each with its own job. The knowledge graph maintains entities and relationships. RAG is the process of retrieval augmentation, which can query documents, vectors, structured databases, or graphs. The context window is the working set this one model request can see directly. The agent loop decides when to ask the model for the next step. What connects them is the retrieval policy and the context builder inside the harness.
Not every sampling needs a full RAG pass
A common mistake is "before every model call, search the whole knowledge base first." That adds latency, cost, and noise in the context. We prefer three kinds of triggers:
| Trigger | When to retrieve | Who decides |
|---|---|---|
| Mandatory retrieval | New task, key constraints, resuming across windows | Deterministic harness rules |
| Event-driven | Schema change, stale dependency, test failure | State events + query policy |
| Agent-driven | The model hits an unknown and needs more background | The LLM calls the retrieval tool itself |
Mapped onto the stages of one task:
| Stage | How retrieval works |
|---|---|
| Planning | Mandatory retrieval: read the rules in effect, the task goal, key entity relationships, and current dependencies; let the LLM explore the unknowns itself |
| Ongoing execution | Skip or retrieve selectively: if World State and relevant versions haven't changed, don't retrieve again; only run targeted queries on events like new files or schema changes |
| Test failure | Event-driven + agent-driven: check error codes, related past failures, and version changes first; if the model thinks it still lacks information, it widens the search itself |
| Resuming across windows | Mandatory checkpoint retrieval: must restore the task goal, latest decisions, key constraints, to-dos, and the necessary evidence index, then query Notes / History as needed |
How the KG takes part in scheduling
Say the KG records that the backend API depends on schema v1. Agent A upgrades the schema to v2. The harness can walk the dependency chain and generate a DEPENDENCY_OUTDATED event for the main agent to handle. At that point the KG is no longer just a reference library for answering questions; it's the factual basis for dependency checks and task coordination, provided the relationships, versions, and sources have been verified.
MATCH (service)-[dep:DEPENDS_ON]->(schema)
WHERE schema.id = 'schema_v1'
AND dep.status = 'active'
RETURN service.id, dep.valid_from, dep.evidence_id;
There's an important insight here: an agent's retrieval should be aware of state, not just of the question. Whether the model is planning, fixing an error, integrating subtasks, or resuming context determines which kind of information it should recall.
Memory isn't just storage: it needs facts, evidence, and versions
"What an agent said" and "what the organization currently confirms is true" must never be mixed up.
Suppose Agent A reads from staging.yaml that the max connection count is 100, and Agent B finds 200 on the production instance. They don't necessarily conflict; it may just be different environments. Even if both come from production, you still need to distinguish the value declared in the config file from the value actually in effect in the database.
We suggest splitting knowledge into states: proposed, verified, conflicted, superseded, invalidated. You also need to separate observed_at (when it was observed) from valid_from (when the fact took effect). A fact that gets cited often does not become true because of it.
{
"entity": "production.db.connection_pool",
"attribute": "max_connections",
"value": 200,
"status": "verified",
"observed_at": "2026-10-27T10:02:00+08:00",
"valid_from": null,
"evidence": "sql-query/run-182",
"version": 2
}
Two different kinds of "consistency" are involved. Database transactions make sure concurrent writes don't overwrite each other. Evidence verification makes sure the facts written in are trustworthy. The first can be solved with SQLite / PostgreSQL transactions, version numbers, and optimistic locking. The second is not something ACID solves for you.
So we chose hybrid memory, with three views of each piece of knowledge:
| View | Question it answers | What goes in it |
|---|---|---|
| Narrative Notes | Why was this decided at the time? | Complex semantics, background, and experience |
| Structured KG | What holds true now? | Entities, relationships, time, and versions |
| Event log | What actually happened? | Evidence that can't be silently overwritten |
Natural-language Notes don't all need to become triples, and a KG shouldn't try to imitate a relational database in Markdown. Let natural language keep the complex semantics, let structured facts support retrieval, verification, and dependency queries, and let raw events handle traceability.
The harness is like policy, the main agent is like a project manager
A good agent system and a well-run remote team are solving the same organizational problem: how do you get autonomy and coordination at the same time?
We talked about the example of a remote team. If everyone schedules their time with complete freedom and no rules at all, collaboration falls apart. If you dictate every minute of everyone's day, you kill autonomy. A better approach keeps a lot of individual freedom while setting a small number of rules that require coordination, such as four hours of core overlap time every day.
In the agent world, you can replace fixed-time coordination with event-driven checkpoints: align on goals when a task starts, notify when dependencies change, check permissions before critical operations, save progress when crossing windows, and accept results on completion.
Strict on boundaries, loose on process, strict on verifying results.
External knowledge does not automatically get instruction authority
Prompt injection in RAG and external documents is another thing you have to deal with: a malicious document may pretend to have higher priority and trick the model into calling a dangerous tool. A trustworthy harness can't defend against this just by saying "please ignore malicious instructions." It has to isolate external knowledge from control policy, and enforce hard constraints at the point of execution through real permissions, tool authorization, and sandboxing.
This affects memory too. Content extracted from an untrusted document can be recorded as a candidate fact, but it can't automatically become company security policy, much less change the harness's execution permissions. In one line: untrusted knowledge can inform decisions, but it can't grant permissions.
From memory to skill to tool: how agents actually build capability
A system's progress shouldn't just be "remembering what it did last time." It should also cut down the repeated reasoning next time.
Say a coding agent has to check database schema compatibility every week. The first time, it might write a throwaway Python comparison script, read the logs, and produce a report. The next time it repeats roughly the same steps. Once the process has proven reliable, you can first capture it as a set of working steps, and then turn the deterministic parts into reusable code.
Codex's memory consolidation prompt already includes guidance to "distill valuable, recurring procedures into skills" [12]. But "the model wrote a skill" is not the same as "the system safely deployed a new tool." A new tool involves code testing, registration, authorization, version control, and a release that can be rolled back.
| Situation | Better form |
|---|---|
| A project's preferences, important business constraints | Memory / KG |
| A multi-step process that needs the LLM's judgment | Skill / SOP |
| Stable input-output transformations, format validation | Verified tool |
| Problems that change often and have no mature process yet | Keep handing them to a frontier LLM to explore |
This lines up with the division of labor I've long believed in, which I call heterogeneous intelligence: deterministic problems go to traditional programs, well-defined probabilistic judgments go to dedicated decision models, and open-world problems go to frontier LLMs, with the harness managing the collaboration and boundaries between them.
Putting every component into one picture
This isn't an official Codex "ultimate architecture." It's a blueprint we worked out starting from the source and combining it with ideas from organizational management and knowledge governance.
If you built it from scratch, what does the minimum viable version need?
The first version doesn't need five kinds of databases, and it doesn't need elaborate GraphRAG from day one. First make the agent loop, the event log, explicit permissions, state management, and recoverable task checkpoints reliable; for memory, Markdown + SQLite is enough to start. Only consider PostgreSQL once shared access and complex transactions are real needs, and only evaluate Neo4j once complex multi-hop graph queries are an actual bottleneck.
- Cancellable tasks and continuous turns
- Tool calls and results that leave a trace
- Incremental context updates and recovery after compaction
- Reusable Notes and a history index
- A trust boundary between rules and knowledge
- Clear task acceptance and tests
The way to tell whether it's getting better isn't whether the model's answers sound smoother. It's the final task success rate, the rate of missed key constraints, the quality of memory recall, the share of repeated work, latency, and real cost. Those are the experiments actually worth running next.
Future agents may look more like an organization that keeps learning: with its own working memory, organizational knowledge, tool platform, and collaboration discipline; sometimes exploring on its own, sometimes required to follow the rules. The best harness won't think everything through for the model. It lets intelligence keep producing value inside clear boundaries.
Quick reference
| Term | One-line definition |
|---|---|
| Harness | The whole system that hosts the model, tools, state, control, and execution logic |
| Agent Runtime | A name that leans toward the execution engine; in practice it overlaps heavily with the harness |
| Task / Session | The lifecycle of a task, and the session environment it runs in |
| Turn | A continuous stretch of agent work driven by one user request |
| Sampling | One generation request to the model, plus handling of its response |
| Tool Call | A call request proposed by the model and executed by the harness's tool implementation |
| SubAgent | An agent with its own execution context that can be delegated tasks |
| Mailbox | The mechanism for asynchronous communication and wake-ups between agents |
| World State | Typed runtime-environment state the model needs to know about, and its updates |
| KV Cache | Intermediate attention state reused for identical input prefixes |
| Compaction | Keeping a task running by shrinking or rebuilding the context |
| Thread Hint | A short resume cue for continuing a task across windows |
| Notes / History | Readable working notes / traceable execution records |
| KG / RAG | A structured knowledge network / the process of retrieving from various sources to augment the model |
| Memory Consolidation | Deduplicating, updating, and consolidating scattered session memories into long-term knowledge |
| Skill / Tool | Reusing a way of working / reusing an implemented program capability |
References
This is an independent reading of open-source technology and does not represent OpenAI's official product documentation. All diagrams are redrawn, and the code is limited to simplified snippets or pseudocode. When the source is updated, the repository at the corresponding version is the authority.
- OpenAI · Unlocking the Codex harness: how we built the App Server
- Codex Rust workspace · Cargo.toml
- codex-arg0 · multi-entry execution dispatch
- codex-core · run_turn and sampling logic
- codex-core · parallel execution of tool calls
- Codex · multi-agent tools and collaboration
- World State · Section / Snapshot / Diff
- OpenAI API · Prompt Caching
- Codex · compaction handoff prompt
- Codex · token budget new-window implementation
- Codex Memories · Phase 1 / Phase 2
- Codex · memory consolidation prompt
- World State · AGENTS.md state updates
- OpenAI · Unrolling the Codex agent loop
- OpenAI Developers · Codex as a platform