External/oc
1
0
Fork 0
mirror of https://github.com/only-cli/oc.git synced 2026-09-15 10:40:56 +02:00
Code Issues Packages Projects Releases Wiki Activity

readme: codex numbers from the six task run

Browse Source
This commit is contained in:
only-cli
2026-08-18 13:26:40 -04:00
parent babd5959d9
commit 3738f41825
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -113,7 +113,7 @@ playwright-mcp ################################## 1,575,695 tokens 48 tu
Each session gets a skill documenting its tool, so every condition runs at its best. Two results are worth stating plainly. oc and lynx were the only tools that returned real content on every task: Reddit served Jina Reader and Playwright MCP a 403, so both "answered" that task by reporting the block, and raw curl burned its full turn budget there and on the GitHub search, roughly 400k tokens each, and returned nothing. But lynx, not oc, took the token and cost columns this round, and the reason is a missing feature. The compact view leaves link URLs out to save tokens, and `oc do <n>` does not ship until v0.2, so an agent following a link has to re-fetch the page as `oc open --json` or `oc raw` just to learn where `[15]` points, while `lynx -dump` prints a references list for free. That tax is most of the gap on the multi-step tasks. Activatable numbered actions are the fix and the point of v0.2.
The same six tasks run through OpenAI's Codex CLI (`codex exec`) as well; totals there are much smaller because Codex's per-session overhead is, so the two agents are compared within their own tables. The benchmark repo has per-task numbers, per-tier breakdowns, methodology, and instructions for adding other tools and models.
The same six tasks run through OpenAI's Codex CLI (`codex exec`) as well, where oc comes out ahead: of the two tools that returned real content on all six tasks, oc spent 287,862 tokens and lynx 408,548, and Playwright MCP spent 699,810 on the Reddit thread alone. Codex's per-session overhead is much smaller than Claude Code's, so the two agents are compared within their own tables rather than against each other. The benchmark repo has per-task numbers, per-tier breakdowns, methodology, and instructions for adding other tools and models.
## Status