The title and every heading are the page's to write, and both skipped
truncate(), so one hostile scalar could print unbounded output whatever
the budget said. The compact view now cuts titles, headings, and input
names at the same cap and marker every other block gets, and read cuts
even a first block bigger than its whole budget, since 'up to N tokens'
is a promise the page must not be able to break. The distilled page
keeps the full values: --json stays the machine-stable view, bounded by
the fetch cap, and machines cut for themselves.
Fixes#28
contentFailure called every render under 25 tokens a failure, which
made exit 2 claim a terse page was JavaScript-only, gated, or
challenged when it had simply arrived terse. The verdict now needs
evidence: nothing extracted is empty whatever the page weighed, and a
short render is only a failure when the markup behind it was far too
big to have carried only that. A status endpoint or a one-line answer
now exits 0, script-only shells and consent walls still exit 2, and
raw applies the same rule before refusing. The --json empty field
follows the same policy, so callers keep a machine-stable distinction
between empty and merely short.
Fixes#29
Both transports buffered a text response whole before the distiller saw
it, so a hostile URL was an unbounded allocation. Every transport now
enforces one cap, 25MB decoded, three times the largest body oc fetches
on purpose. Content-Length is checked first to fail before the bytes
arrive, and because that header is optional and untrusted, the stream
is counted as it lands too: native fetch and the proxy transport abort
the transfer at the cap, and impers, which buffers inside its own
binding, is checked before its body travels any further. Decoded bytes
are what get counted, which is what stops a decompression bomb.
Fixes#27
Node 24 has bundled an npm new enough for trusted publishing since
24.4, so upgrading npm at publish time only added an unpinned install
to the release path. If a runner ever serves an older 24.x, npm publish
fails loudly rather than shipping anything.
Every GitHub action is now pinned to a full commit SHA with its version
in a trailing comment, verified against each upstream repo's tags, so a
retagged release cannot swap code under CI. The publish workflow pins
npm to an exact version instead of @latest, the experiment Dockerfile
pins its base image by digest, and codeql.yml gets a top-level read-all
permissions block so the default token grants nothing the analyze job
does not ask for.
SECURITY.md points reports at GitHub private vulnerability reporting,
which is now enabled, and scopes out experiments/. Branch protection on
main now blocks force pushes and deletions.
package.json points bin.oc at src/cli.js, so an npm link resolves the
global oc to this file directly and running it needs the bit. Without
it every oc invocation dies with Permission denied while node src/cli.js
keeps working, which is easy to miss.
skills.sh renders skills/web-browsing-cli/SKILL.md straight from GitHub, and
the release checklist said there was nothing to do for it because the skills
CLI reads that file live off main. That is true of the install path and false
of the page: the page showed the 0.2.0 pin from 2026-08-20 while main had
already shipped 0.3.0 and 0.4.0 the same day, so every reader was handed a
two-release-old install command.
The site offers exactly one lever. Its documented API is read only, with no
refresh or re-index endpoint, and the skills CLI has no publish or sync
command; a repository is re-read after the telemetry service sees an install
from it, and repo pages are cached on top of that. So a stable publish now
runs one `skills add` against the repo, which is the invocation the
install-remove-loop experiment already proved out. It costs one install on the
counter per release, which is the price of the only mechanism there is. The
job is continue-on-error and runs after npm publish has already succeeded: a
page that catches up late is a smaller problem than a red release.
The refresh is worthless if the pin it publishes is stale, which is the actual
root cause here, so a latest publish now fails when SKILL.md disagrees with
package.json. Beta and dev skip the check, because a pin moves when a release
is stable rather than when it enters beta, which is the rule 0.3.0-beta.1
already followed. Verified both ways against the current tree: it passes on
0.4.0 with a matching pin and refuses a 0.4.0 release still pinning 0.2.0.
The channel the earlier step resolves is now a job output, so the refresh job
can gate on it instead of re-deriving it from the version string.
The feature landed with one sentence in the install paragraph, which named
the three environment variables and nothing else. Anyone actually putting oc
behind a corporate proxy had to read src/fetch.js to learn that an https
target prefers HTTPS_PROXY and falls back to HTTP_PROXY while an http target
uses HTTP_PROXY only, that a bare host:port is read as http://, that a socks
URL is refused rather than ignored, or that NO_PROXY takes suffix, wildcard,
host:port, and CIDR entries. All of that is now in a Proxies section, and
every claim in it was checked against the merged code rather than written
from the diff.
Two limits are documented instead of left to be discovered. oc does not read
ALL_PROXY, but the impers transport is libcurl underneath and reads it on its
own, so a request oc treats as direct can still leave through a proxy; the
same holds for the *.suffix, host:port, and CIDR forms of NO_PROXY, which
libcurl does not parse. Verified live against a third party proxy by watching
the egress IP: with only ALL_PROXY set, or with NO_PROXY=*.host naming the
target, oc reported a direct fetch and the request went through the proxy
anyway. And an IPv6 literal over HTTPS cannot work through a proxy today,
because URL.hostname keeps the brackets, so net.isIP reads 0 and the SNI and
certificate check both treat [2606:...] as a DNS name.
The security properties a reader would otherwise have to assume are stated:
the CONNECT tunnel still verifies the origin certificate (confirmed against
expired, self-signed, and wrong-host endpoints through a real proxy),
credentials in the proxy URL reach the proxy and nothing else including
across redirects, private and internal targets stay refused, and a name that
resolves publicly for oc and internally for the proxy is not something oc
can detect, so the proxy is trusted for its own egress policy.
The skill gets the short version, since an agent needs two things: that no
flag or setup is required, and that a "proxy failed" or "blocked" line is a
transport problem to report rather than a page to retry. llms.txt gets one
fact next to the existing transport fact.
Version bump across package.json, the lockfile, the plugin manifest, the
marketplace entry, and the npx pins in the agent skill.
Adds CHANGELOG.md covering what landed since 0.3.0: dispatched site
shortcuts, the Wikipedia shortcuts, proxy env var support, the loud exit 2
on a page with no readable content, and the MIT LICENSE file.
The skill gains a site shortcuts section, since 0.3.0 documented the
shortcuts in the README but the skill never mentioned them, plus the exit 2
contract and the proxy note. llms.txt gains Wikipedia and the proxy line.
README gains the end to end benchmark: five Wikipedia lookups run as whole
tasks in Claude Code with one tool each. All three tools answered every
task correctly, so it reports cost rather than accuracy.
Article, search, and non English wiki lookups via clis/wikipedia.org.json,
reachable as oc wiki, oc wikipedia, or oc wikipedia.org.
The article and lang commands use ?action=render, which returns the article
HTML without the interlanguage sidebar and Tools menu that otherwise eat
about half of a 500 token budget before any prose. Its links stay root
relative, so oc do <n> still follows them; the Parsoid endpoints render just
as clean but emit ./Title hrefs that resolve against the API path and break
link following.
Search uses the normal results page. The api.php JSON search endpoint is
cheaper on paper but distills to nothing today, since its results sit in a
nested query.search array.
Closes#21
The shortcuts table promised `sub <name>`, `item <id>`, `repo <owner> <name>`
and the rest for 13 sites, and clis/*.json shipped in the published `files`
list, but nothing ever read those files: `oc reddit sub ClaudeAI` answered
`unknown command 'reddit'`. The help text labels `fill` and `submit` as
planned, so an agent reading the table had every reason to treat the
shortcuts as shipped, construct one, and fall back to raw fetching when it
failed, which is the outcome this tool exists to prevent.
src/sites.js resolves `oc <site> <verb> [args]` against clis/*.json at
runtime and hands the URL to the existing open path, so a shortcut cannot
change what a page costs or how it renders, and a new definition needs no
wiring. A site answers to its domain, its bare name, and a short alias
(hn, gh, so, ddg, yt, aws, gcp, learn, finance, twitter), because an agent
that has to guess the spelling is back to guessing URLs.
Two details are worth naming. The last declared argument takes every word
after it, so `oc aws search s3 lifecycle rules` needs no quoting. And a value
filling a path segment keeps its slashes while a value in a query string does
not, so `oc learn doc azure/aks/what-is-aks` reaches that page instead of
asking the site for one impossible segment.
`oc sites` lists every site with its verbs, one line each, so discovery costs
less than a wrong guess. reddit's {sub} and github's {repo} are renamed to
{name} so the usage lines print what the README documents.
Fixes#16
README and package.json both said MIT, but with no LICENSE file the claim was
not a grant: default copyright applies, so vendoring oc, shipping it inside a
corporate toolchain, or passing a license audit were all blocked, and GitHub's
license detection reported null. The published package already carries
"license": "MIT" in its metadata, so the repository and the tarball disagreed
about what users received.
The badge now links to the file instead of an in-page anchor, which is also
what the OpenSSF Scorecard License check reads.
Fixes#15
A JS-only page, a consent wall, and a bot challenge all answer HTTP 200
with markup that carries no text, and oc reported those renders as
successes: a title, an actions line, and "100% saved" in verbose mode,
which is true of a render that saved every token by extracting none.
From the output alone an agent could not tell that from a page that is
genuinely empty, so it never fell back to anything heavier and the empty
result travelled on as evidence.
oc now prints one line on stderr and exits 2 in that case, and --json
carries the same verdict as an always-present 'empty' field, so a caller
can branch on "nothing on this page" vs "oc could not read this page"
without parsing prose. Exit 2 is distinct from the exit 1 every other
failure uses. It sets process.exitCode rather than calling process.exit
so whatever did render still finishes printing.
The thresholds in render.js are measured, not guessed. contentTokens
counts text the page wrote (prose, headings, and link or button labels
over 25 chars, which is what separates a headline from nav chrome), so a
link-list page like Hacker News or a search result still reads as
content. Against live pages the failures land at 47 and 51 tokens
(reddit.com/r/*, instagram.com) while the thinnest page the README
claims support for carries 463 (an X profile), so the floor at 25 and
the thin-vs-HTML-weight rule at 100 tokens against 2500 of markup both
sit in a wide gap. Verified with no false positives on feeds, the Stack
Exchange API, Microsoft Learn RSS, a YouTube watch page, AWS and GCP
docs, a one-line HN item, and example.com.
'oc raw' fails only on genuinely blank output, since raw is the fallback
the compact view's failure line names and must not refuse the same pages.
Closes#14
The test for revalidating redirect hops drove httpbin.org, so httpbin being
down failed the suite. It is down now, returning 503, which fails CI on main
and would have failed the release: the publish workflow runs npm test before
it ships, so a stable version could not have reached npm while a third
party's app tier was unwell.
It was also testing less than it looked. Each transport carried its own copy
of the redirect loop, and the live test only ever exercised whichever one was
installed, so the guarantee held in one copy and was unproven in the other.
A check that matters twice is a check a change can fix once.
Both transports now share one loop that takes the request as a callback,
which is what makes the hop check provable against a transport that never
leaves the process. Three offline tests replace the live one: a hop to a
private address is refused and never asked for, a hop to a public address is
still followed (a loop that rejected everything would have passed the first
test and broken every redirect on the web), and a cycle gives up. Removing
the hop check fails the first of them, which is more than the httpbin test
could say for the transport it did not run.
Verified live afterwards on both paths: an http to https chain through
impers, a plain page, and a literal and a resolved private address both
still refused.
A syntax highlighter gives every token of a command its own element, so
`s3://bucket/` reached the walk as `s3`, `:`, `//`, `bucket`, `/`, and the
rule that reassembles text fragments only glues the ones sharing a parent.
The rest were space-joined. The AWS CLI reference handed over
aws s3 cp test . txt s3 : // amzn - s3 - demo - bucket / test2 . txt
and Node's fs docs handed over `console . log` and `fd ?. close ()`. Any
command or snippet an agent took from a docs page was wrong, and nothing in
the output said so. Highlighting is not an edge case: of 172 pre elements on
the AWS CLI reference, the Rust book, the Node API docs and the Python
library docs, 159 are split this way.
A pre or code subtree is now read as one string. That costs nothing to
follow, because not one of those 172 blocks contains a link, and it also
fixes inline code, which was inserting a space into `Byte ( u8 only)`.
Two things fall out of it.
Page furniture had to stop riding along. Node puts a language label beside a
copy button inside every code block, so the subtree's text ended in
`javascriptcopy`. A control is chrome, and so is the block-level element
holding it, which is how the label leaves with the button it sits beside.
The test stays on block-level wrappers because a highlighter's own elements
are inline, so a stray control can never take a line of code out with it.
Code blocks keep their lines. Collapsing them was survivable while the code
was already wrecked; once it reads correctly, `// read it back` in front of
the statements that followed it is worse, because the output now looks
trustworthy. Blank runs and the indentation the whole block shares carry no
meaning and still go. A cut lands on a line end for the same reason it
already lands on a sentence end, and a snippet stays one line, because that
is what find's index promises.
Measured over five real pages, the compact view moves by -15, -7, +208, -45
and 0 characters, about 35 tokens in total, all of the growth being Python's
pretty-printed output getting its indentation back.
A tool call inside an agent session costs 23,000 to 33,000 tokens of
overhead whatever it prints, so the page-view win only reaches the
session total if answering a task takes fewer commands. Three places
were charging a command to say what the next command should be, each
found by capturing the command stream of a real agent run rather than
by reading the code.
A search result title is a link. Every engine puts it in an anchor
filling an <h2>, and the walk took the heading's text and returned,
dropping the href, so `do` on the most obvious number on a results page
printed the title back. The agent then spent a second command finding
the number that navigates. The href now rides along when the anchor is
the whole heading, which is the test documentation fails on purpose:
every heading in the Rust book and on an AWS CLI reference page carries
a permalink to its own id, and following one would refetch the page the
agent is already reading.
`find` pointed at its answer. With a single match it printed the block
and a number, and the agent's next command was always the `read` on
that number, so it now prints the region. With several matches it
showed a 200 character snippet of each even when the budget had room
for them whole, so it spends that room, on the same terms `FINISH`
already documents for a page that nearly fits.
A truncated block ended mid sentence. Asked for the first sentence of a
page, an agent was handed it complete, followed by a marker saying 302
characters were cut, and spent a command on `read` to find out whether
the sentence went on. The cut now falls on the last sentence that
finished inside the cap, and measured across five real pages it costs
nothing: four came out within three characters of before.
The package-lock name field catches up with the scoped package name,
which npm rewrites on any install.
Three faults in the JSON renderer, all of which the npm registry's package
endpoint hits at once, where the render came out as a truncated blob titled
after the package's two maintainers.
mainArray took the longest array of objects at the top level, so `maintainers`
became the subject and the package itself was pushed into the metadata line. A
root carrying its own name is the resource, and an array hanging off it
describes that resource rather than standing in for it. Conventional container
keys are checked first, so a named collection is still read as a collection.
The metadata line capped nothing. One long scalar there, a readme in npm's
case, cost more than the rest of the page put together; a summary line has to
stay a line, so a long one becomes its own block.
An oversized markup field distilled into more blocks than the item it hangs
off had fields. Under BODY_CAP a body is still rendered in place with its links
followable, which is what the Stack Exchange withbody shape wants; over it, one
numbered line, with `oc raw` still holding the whole thing.
The release commit was made from a scratch worktree whose node_modules was a
symlink to another checkout, and .gitignore listed node_modules/ with a
trailing slash, which matches a directory and not a symlink. So the link
itself went into the tree, pointing at an absolute path on one machine.
Anyone cloning main got a dangling node_modules before npm was ever run. The
npm tarball is unaffected: the files whitelist decides what ships, and npm
never packs node_modules, which the 0.3.0-beta.1 pack listing confirms.
The ignore rule loses its trailing slash so it matches either shape.
Testing the 0.3.0-beta.1 build against live URLs turned up a gap the beta
notes claimed was closed: only the native-fetch path checked the content
type, and impers is the default whenever the optional dependency installs.
So 'oc open' on a PNG rendered eight kilobytes of mojibake as a page, with
numbered blocks, an actions footer, and a straight face.
The check now lives in one exported assertReadableType that both transports
call, so a refusal cannot depend on which client happened to get the page.
While the gate was being written down it also grew a correct allow list.
The old one matched the substring html, xml, or json anywhere in the header,
which let application/vnd.ms-htmlhelp through and, worse, refused text/plain:
a robots.txt or an llms.txt is exactly the kind of small text file an agent
asks for, and the fetch path was answering that it was not a page. Readable
now means any text/* type plus the application/* types that are really text,
including the +json and +xml families a feed answers with. A missing header
stays readable, since small servers omit it and the page behind it is fine.
Tested offline against the header strings themselves rather than the network.
Ships JSON API rendering (#3) and the AWS, Google Cloud, and Microsoft Learn
documentation shortcuts (#11) to the beta channel.
Documentation caught up with what the code actually does while it was open:
- Status said the remaining actions land in v0.2, which shipped without them.
fill, submit, and back are now marked planned in the help and the README,
a label that cannot go stale the way a version number does.
- llms.txt names the cloud documentation shortcuts and the JSON rendering.
The skill keeps its npx pin on 0.2.0. A pin is what agents actually execute,
so it moves when a release is stable, not when it enters beta.
Closes#3.
An API answer is a page: jsonToHTML turns a JSON body into one article per
item, and everything downstream (numbering, budget, do, read, next, raw)
treats it as an ordinary document. No per-site logic and no new dependency.
The compact view is the hard part, since a search response carries far more
fields than fit in 500 tokens. So the renderer scores each field by how much
it varies across items against how wide it prints, penalises fields flattened
out of a sub-object (owner.reputation describes the asker, not the answer),
and spends about 60 characters per item on the winners. What every item
shares is stated once at the bottom instead of repeated, empty fields are
named rather than printed, and what was cut says so and points at oc raw,
which keeps every field.
On the Stack Exchange search endpoint that is 30 results in ~960 tokens
against ~5,500 for the raw body, with each title a link and question_id
visible.
Also here:
- clis/stackoverflow.com.json gains search <query>, which is what #3 was
blocking. Results carry question_id, and the question feed reads one in
full, so search now completes without touching the challenged HTML page.
- fetch: the native-fetch path rejected anything that was not HTML or XML.
It now accepts JSON, which also makes the two transports render one URL
the same way, since the impers path never checked the type at all.
- raw threads the URL through so its view of an API response can be titled
and, unlike the compact view, keeps every field.
Deliberately not done, from the notes on the issue: pagination in the
actions line, and API metadata on stderr. There is no stderr channel at the
distill seam, so response-level fields (has_more, quota_remaining) render as
one footer line instead. A columns hint in the clis specs and a --json
passthrough both looked like the wrong trade: the first needs per-site
tuning for something the scoring already handles, the second would break the
machine-stable Page contract.
All three providers render their own docs search client-side, so
distilling the search page yields only nav chrome. Microsoft Learn is
the exception underneath: its public RSS search endpoint serves real
results as a feed, which the engine already renders (same route as
Stack Overflow). AWS and Google Cloud expose search only as JSON (#3),
so until the engine renders JSON their search goes through DuckDuckGo
HTML with a baked-in site: filter, the same endpoint the duckduckgo.com
CLI already relies on. Bing was tried first for that job and rejected:
it silently drops the site: operator on some queries.
Verified live: the Learn RSS endpoint returns titled results for
"app service deploy"; the DuckDuckGo route returns real
docs.aws.amazon.com pages for "lambda timeout" (it can answer with a
rate-limit challenge under rapid-fire automated use, noted in README).
Cloud provider docs are the pages agents hit most while writing
infrastructure code, and they carry the heaviest chrome: nav trees,
version pickers, feedback widgets. One config per provider gives them
tuned shortcuts instead of raw URLs.
The Google config points at docs.cloud.google.com because
cloud.google.com 301s every docs path there; skipping the redirect
saves a round trip, same trick as reddit.com going via old.reddit.com.
All six URL templates were verified against the live sites with
fetch + distill (56 to 4126 blocks, real titles, HTTP 200).
Closes#11
Addresses the indirect-prompt-injection risk category flagged by
skills.sh's Snyk audit — the tool's job is fetching third-party web
content for an agent to read, so that caveat belongs in the docs
agents actually load. Also trims SKILL.md's own prose for token cost.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
RonCodes88