How this was measured.

Method

Each model-effort build was given a fresh GitHub repository and an identical text-only prompt.

Every build ran in its model provider's native IDE. Opus and Fable ran on Claude Code; the GPTs ran on Codex.

Raw session logs were then analysed to extract, normalise and calculate tokens, cost and time.

Sixteen of the eighteen builds carry full measurements. Two were built through the Claude iOS app, whose sessions live on Anthropic's servers and expire after thirty days. Those transcripts passed their retention window before the analysis ran, so those two builds appear on the site without figures.

Measuring time

This was fairly straightforward, once the start and end points were clearly demarcated. The clock starts when the prompt is sent. It stops when the model sends a message that the game is ready.

The window covers everything the model does on its own: its turns, tool calls, subagents, background work and self-reviews.

Fable 5 Ultracode ended with “Trashketball is built, reviewed, and verified”. Opus 4.8 Max ended with “Done. The game is running at http://localhost:8123 in the Browser pane.” For Codex builds, the equivalent marker is the first task_complete event after the prompt, which is the harness's own record of a turn closing.

That end point is a specific, quotable line in the transcript, located by reading rather than by taking the session's final timestamp. Session end times can be triggered by something else entirely, such as hitting a usage limit.

Startup queue lag counts, because a user waiting for a model to begin is still waiting. Idle time after delivery does not count.

Measuring tokens

Input is every token a model read: the prompt, plus the whole conversation resent on each turn. Output is every token it generated. Getting both right from the logs means being careful about four things.

• Claude splits input by how each token was billed. The field named input holds only the fresh, uncached portion, which on these builds is < 2%, because almost everything the model reads is re-read from cache. Total input under this methodology includes those cached reads, and the smaller amount written into cache. The implication is on cost: cached reads are priced at a fraction of fresh tokens, so the split matters for the bill, not just the count.
• OpenAI reports a model's reasoning as a subset of its output count, the same way it reports cached reads as a subset of input. The output figure already includes reasoning, so it needs no adjustment: total tokens on the OpenAI side come out to input plus output, with reasoning already inside that output.
• The two vendors tokenise text differently, so the same paragraph becomes a different number of tokens on each side.
• Orchestrated builds fan work out to subagents, each of which writes its own transcript, so the coordinator's log is only part of the picture. GPT-5.6 Sol Ultra ran seven subagents alongside its main thread; counting them raised its output from 74,000 to 250,000 tokens, its input from 22M to 37M, and its cost from $16 to $34.

Every figure here was calculated from the raw logs on these terms, summed once per API response inside the same window used for the clock.

Measuring cost

These games were built on the Claude Code and Codex macOS apps, both on subscription plans. A subscription charges a fixed monthly price and grants weekly or hourly usage quota, sometimes topped up by unexpected promotional refreshes.

The platforms publish how much of that quota you have used. They do not publish how many tokens it represents.

Both vendors do publish token pricing for API and Enterprise customers, so the dollar figures here apply those published rates to the tokens measured. That gives a counterfactual: what each build would have cost at API rates, not what anyone paid. Whether subscription usage is metered and cached identically to API usage cannot be verified from outside.

Caching dominates the pricing math. Majority of input arrived as cache reads, which is billed at a fraction of the standard rate.

The cost represents the dollar value of each build, but a subscriber paid far less in reality.

Rates come from the Anthropic and OpenAI pricing pages and no other source.

Controlling conditions

Contamination checks. Each build ran in a fresh repository, and the transcripts confirm that isolation held in both directions.

Across all sixteen builds with local logs, twelve on Claude Code and four on Codex, every build started from an empty folder, installed its dependencies fresh, wrote only inside its own project, and referenced no other build's files. None wrote to a shared skill, a global config, or a memory store that a later build could read.

Codex also reads AGENTS.md instruction files from parent folders if any exist. None do, and the models' own searches for one came back empty.

The single exception is harmless. Fable 5 Ultracode wrote two notes to a memory directory inside its own project, which would only surface if a user resumed a session in that same repository.

Some interesting findings More to come

Visual verification is expensive. Opus 4.8 Ultracode took 132 screenshots of its own rendering, across its main thread and 31 subagent sessions, returning about 175,000 tokens of inline image. A rendered 3D scene cannot be verified by reading its source, so the spend buys something real.

Input dwarfs output. Every build read at least 46 times more than it wrote. Two causes: First, the API keeps no memory between calls, so each one resends the entire conversation. A build making 1,400 calls pays for its own history 1,400 times over. Second, every call also carries the harness's system prompt and tool definitions, which run 17,000 to 19,000 tokens on Codex and 42,000 to 55,000 on Claude Code.

Limitations

LLMs are nondeterministic. The same prompt, same model, same effort and same harness can result in different outputs. This experiment was run once per model-effort. It is a disciplined study, not a rigorous benchmark.

User-configured harnesses on Claude Code vs Codex may not be identical.