Closes#3.
An API answer is a page: jsonToHTML turns a JSON body into one article per
item, and everything downstream (numbering, budget, do, read, next, raw)
treats it as an ordinary document. No per-site logic and no new dependency.
The compact view is the hard part, since a search response carries far more
fields than fit in 500 tokens. So the renderer scores each field by how much
it varies across items against how wide it prints, penalises fields flattened
out of a sub-object (owner.reputation describes the asker, not the answer),
and spends about 60 characters per item on the winners. What every item
shares is stated once at the bottom instead of repeated, empty fields are
named rather than printed, and what was cut says so and points at oc raw,
which keeps every field.
On the Stack Exchange search endpoint that is 30 results in ~960 tokens
against ~5,500 for the raw body, with each title a link and question_id
visible.
Also here:
- clis/stackoverflow.com.json gains search <query>, which is what #3 was
blocking. Results carry question_id, and the question feed reads one in
full, so search now completes without touching the challenged HTML page.
- fetch: the native-fetch path rejected anything that was not HTML or XML.
It now accepts JSON, which also makes the two transports render one URL
the same way, since the impers path never checked the type at all.
- raw threads the URL through so its view of an API response can be titled
and, unlike the compact view, keeps every field.
Deliberately not done, from the notes on the issue: pagination in the
actions line, and API metadata on stderr. There is no stderr channel at the
distill seam, so response-level fields (has_more, quota_remaining) render as
one footer line instead. A columns hint in the clis specs and a --json
passthrough both looked like the wrong trade: the first needs per-site
tuning for something the scoring already handles, the second would break the
machine-stable Page contract.
Addresses the indirect-prompt-injection risk category flagged by
skills.sh's Snyk audit — the tool's job is fetching third-party web
content for an agent to read, so that caveat belongs in the docs
agents actually load. Also trims SKILL.md's own prose for token cost.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Lets users run /plugin marketplace add only-cli/oc and install the
existing skill as a plugin, alongside the skills.sh and manual-copy
distribution paths.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ossf/scorecard-action only publishes point-release tags (v2.4.4, etc),
no floating v2 major tag, so the workflow failed to even resolve the
action and never published results -- which is why the README badge
showed "invalid repo path". Pin to the v2.4.4 commit SHA instead.
Adds free security tooling for a public npm CLI: CodeQL static analysis
on push/PR plus a weekly scan, a Dependency Review check that blocks PRs
introducing vulnerable or malicious packages, --provenance on npm publish
(cheap given existing OIDC trusted publishing), and a weekly OpenSSF
Scorecard run with a README badge.
Trims the README to what a skimming user needs: it turns websites into
a CLI for AI agents, saves tokens, and gets past blocks on some sites.
Drops the repeated bar chart and three full result tables in favor of
one small table and a link to only-cli/benchmarks, which already carries
the full methodology and per-task numbers.
PR #5's guard pattern-matched the URL's hostname against a regex, which
both under- and over-blocked: IPv4-mapped IPv6 loopback ([::ffff:127.0.0.1]),
0.0.0.0, and any DNS name that merely resolves to a private address all
sailed through, while a legitimate public hostname like 10.example.com was
wrongly rejected because it starts with "10.". It also only checked the
original URL, so a public URL that redirects to an internal address was
never re-validated.
This replaces the regex with net.isIP + dns.lookup: IP literals are checked
directly (including decoding IPv4-mapped/-compatible IPv6), and DNS names
are resolved first so every address they point to is validated before
connecting. The same check now reruns on every redirect hop for both the
impers and native-fetch transports. Resolving before connecting doesn't
pin the address for the actual connection (neither impers nor fetch expose
that here), so a name that re-resolves differently between this check and
the real connect remains a known, documented residual gap.
Watch pages need client JS to become interactive, but the title,
description, view count, and caption tracks already ship inline in
the initial HTML as ytInitialPlayerResponse, so this reads that
directly instead of waiting on the v0.3 headless fallback. Each
caption track becomes a numbered link, and oc do on it fetches the
timedtext transcript, collapsed into one block so it pages through
oc next/oc read like any other long document instead of costing one
block per caption line.
Adds youtubeToHTML and transcriptToHTML alongside feedToHTML in the
distiller, a youtube.com.json shortcut, and offline tests against a
fixture watch page.
Fixes#6
Scattered best-per-column checkmarks made the table hard to scan;
the prose right after it already calls out which tool wins which
column, so the marks were redundant noise on top of that.
The AAPL rows repeated the same $310.03/$302.25 price-of-AAPL figure
in the answer column, which was noise. Drop it and add an est. cost
column instead, with a footnote on how each figure is derived since
no real Codex billing data exists in this repo.
The teaser only ever showed the Reddit case. Adding the single-page AAPL
task on the same token scale shows both sides honestly in one place: oc
wins the multi-hop task outright and Claude Code's own default fails it,
but on a single static page Claude Code's default WebFetch is the
cheapest correct answer in the whole comparison.
The teaser bar only showed tokens and a word for the answer. Same numbers
as the Reddit row in the defaults table below, now with turns and a
check mark, so the bar reads standalone without needing the table to
know a run passed. The failing row also moves to the bottom, matching
how the fuller table below already orders it.
Merged the Reddit and AAPL default-tools tables into one, with a task
column, since they shared every other column and split the same
comparison across two tables for no reason. Dropped their ASCII bars too:
the teaser bar at the top of the README already draws the Reddit numbers,
so repeating them twice more added nothing. Also dropped the bar under
the Claude five-tool agent table, since it only restated that table's own
tokens and turns columns, and did so with a raw curl number that had
drifted from the table next to it.
The stock quote task made a fine single-page example but understated the
real gap: on a task with a couple of hops, Claude Code's own WebFetch and
WebSearch are blocked from reddit.com outright and it has no default shell
fallback, so it fails the task rather than just costing more. Codex's
default gets there through its own web search plus curl, at 2.7x oc's
tokens. That comparison now leads the README, with the stock quote case
kept below it as the honest counterexample where a default tool wins.
Also corrects the earlier claim that Codex reads raw HTML with curl by
default: it reaches for its own web search tool first and, on the AAPL
page, never opens the page at all, which is why it reported a stale price.
Found while chasing an anomalous benchmark rerun: under a sandbox that
blocks writes outside its workspace (codex exec's default), 'oc open'
still renders fine but its session save throws, and the bare catch {}
swallowed it completely. The next 'oc do' then resolves against whatever
session last saved successfully, silently, with no sign the state is
stale. In one repro this served a Hacker News agent a Yahoo Finance page
under 'oc do 5' and it answered from that instead of erroring, which is
how a multi-step task ballooned to 250k tokens without a single wrong
tool call visible in the log.
This does not fix the staleness itself, that needs a design for detecting
a save failure across separate 'oc open' and 'oc do' invocations. It makes
the failure visible on stderr instead of invisible, which is the cheap
half of 'fail loud and cheap' and better than nothing while the real fix
gets designed. Verified manually against an unwritable OC_HOME; cli.js's
main() always calls the network fetcher, so an automated test here would
need fetch-mocking infrastructure this project has none of yet, and none
of cli.js has test coverage today.
Yahoo Finance quote, news, history, lookup, markets, gainers, losers, and
trending pages all arrive server rendered (the AAPL quote page carries the
closing price in about 456 tokens of view against 325,000 of raw HTML), so
a definition ships for all eight. The README grows the comparison people
actually ask for: the same stock price task run through Claude Code and
Codex as they ship versus with oc, one live session each, wrong answer and
all. Codex read raw HTML and reported a price that is not on the page;
WebFetch's digest was cheaper than oc this run and the README says so
plainly rather than hiding it. Also a banner, badges, and an llms.txt
pointer, so both people and models skimming the repo can tell what it is.
Three changes with one motive: a second command costs an agent more than
the lines it saves. 'oc do' on a heading or text block now prints the read
instead of refusing, since refusing spends a whole turn naming the command
that should have run. A page that would finish within about four times the
budget is printed whole rather than cut, because the cut moves tokens into
a second command instead of saving them. And social timelines stopped
rendering as one fused paragraph: linkedom splits text nodes around
apostrophes, so fragments are merged back by parent node and edge
whitespace, block elements now end lines, and repeated button labels are
trimmed sooner than links because a button label is never the content.
With that, x.com profiles and posts read without a login, so a six line
cli definition ships for the two x.com pages that work.
The budget was being spent on whatever came first in the document, which on
most pages is menus. On the Reddit thread the benchmark uses, all 500 tokens
went to nav, sidebar, and moderator boxes, so an agent that opened the page to
read the discussion had to escalate to oc raw to see a single comment.
distill now finds the content (main, role=main, a single article, else the
densest run of prose) and emits it first, with the rest of the page after it
under a divider. Nothing is dropped, so do <n> still reaches every link.
Link labels that repeat down a page go with it: a thread stamps permalink,
save, and report onto every comment, which cost more than the comments. The
count and three examples are printed in place, and oc raw still has them.
Same thread, before and after: 660 blocks to 254, whole compact page 4,184
tokens to 2,712, and the first view now holds comments instead of a sidebar.
Pages small enough to print whole keep document order.
find searches the distilled page the session already holds, prints one line
per match with the number to read it by, and costs no fetch. It matches the
query as a phrase, case insensitive, and falls back to matching the words
separately when the phrase is not there.
On the reddit thread from the benchmark: 'oc find w3m' is 115 tokens against
9,670 for oc raw, and it lands on the numbers to read.
The compact view was all or nothing: an agent that needed more than the 500
token budget had only oc raw, ten to twenty times the price. Now open saves
the distilled page, next continues it where the view stopped, and read <n>
prints one region in full. Headings and text blocks long enough to be cut are
numbered so they can be addressed, and the marker prices what it left behind.
On one Reddit thread: open 475 tokens, next 455, read 88, raw 9,670.
The compact view hides link URLs because printing them is most of what
makes a page expensive, which left an agent re-fetching the same page as
--json or raw just to learn where [15] pointed. oc open now saves the
handles it numbered to a small JSON file per session under ~/.only-cli
(OC_HOME relocates it), and oc do <n> resolves one and renders the target
exactly as open would. Search engine tracking redirects are unwrapped so a
result link opens the destination instead of a script page.
Errors name the command that fixes them, since agents read them: an
out-of-range number reports the valid range, an input says to use fill, a
button says the page handles it itself.
Two hops on Hacker News cost about 3k characters this way, against roughly
23k for the re-fetch route.
Each claude -p session now loads a skill documenting its tool, so every
condition runs at its best. The multiplier on the page-view table is the
precise one: all six pages for less than half the tokens of the cheapest
rival, a single-screenshot floor.
Page-view table now includes Playwright MCP snapshots, Browser Use state
messages, rendered HTML via Playwright and Selenium, and the computer-use
screenshot floors. New table for the end-to-end claude -p benchmark: the
agent finished all three tasks with oc on the fewest tokens and lowest
cost of any full-success condition, while raw curl, Jina Reader, and
Playwright MCP each burned 13 turns on the Reddit task and returned
nothing.
The feed fixture covers Atom entries (escaped bodies, self-closed
categories, bylines) and an RSS item with a CDATA body. Raw markdown of
a feed exposed an old quirk: cleanDocument removed the head before
toMarkdown read the title, so any page whose body lacked a matching h1
lost its title in raw mode. cleanDocument now captures the title first
and returns both.
Sites behind hard bot challenges often leave their feeds open. distill()
now detects feed XML, converts entries (title, byline, link, escaped HTML
body) into a plain HTML document, and everything downstream is unchanged.
fetch accepts xml content types. Ships a stackoverflow.com spec mapping
question/tag/user onto the feed URLs.
Live: oc open on a Stack Overflow question feed renders ~493 tokens vs
~29k for the page HTML, HTTP 200 where the HTML page is challenged.
setup-node's registry-url writes an npmrc auth-token line with a
placeholder token, which npm used instead of OIDC and got a 404 from
the registry. Dropping registry-url lets trusted publishing work.
New clis: github.com (repo, user, search, trending, issues) and
linkedin.com (profile, company, jobs; public guest views work through
the Chrome fingerprint). Both verified live before shipping.
With the package scoped, owner/repo and scope/name line up exactly:
github.com/only-cli/oc is npm @only-cli/oc is the oc command. The skill
installs with npx skills add only-cli/oc.
npm rejects the unscoped name only-cli as too similar to the existing
onlycli package, so the package lives in the org scope with the binary
name. The command is still oc and the repo is still only-cli/only-cli;
install snippets across README, SKILL.md, llms.txt, and the benchmark
docs now say npx @only-cli/oc.
The README now lists the tuned site shortcuts (Hacker News, Reddit,
DuckDuckGo, Bing) next to the generic engine, and cites the live
benchmark numbers from only-cli/benchmarks. llms.txt gives AI assistants
a one-page summary to index. npm keywords added for search.
The internal build spec moves out of the published repo.
npm ci in CI requires package-lock.json, so stop ignoring it; it pins CI
and contributor installs only, consumers never see it.
publish.yml now maps release channels to npm dist-tags: latest, beta,
alpha (from the version suffix of a GitHub release) and dev (manual runs
stamp a unique 0.x.y-dev.N version per build). Publishing a prerelease
version to latest fails loud.
Generic distillation engine (no per-site adapters): fetch via impers
impersonating Chrome with a firefox-fingerprint retry, distill to an
interaction tree, render under a hard token budget with numbered action
handles. Raw mode emits markdown via turndown or cleaned HTML. Per-site
CLI definitions for HN, Reddit, Bing, DuckDuckGo. Offline test suite,
agent skill, OIDC publish workflow.
only-cli