CONTRIBUTING described a command as a URL template only, so a contributor
adding a docs site had no way to learn that a search verb can also name a
static index (sphinx, rdoc, nodedoc), a JSON endpoint (api), or fall back
to DuckDuckGo with a site: filter, or how to pick between them. It also
still counted six files in src/.
The rust, java, ts, and php search verbs had no test; only cpp's did. #25
asked for both verbs per language to be covered offline.
Dependabot opened separate PRs for init (#35) and analyze (#34). Each
failed because the two steps must run the same version of the action.
Bump init, analyze and upload-sarif together and add a Dependabot group
so future codeql-action bumps arrive as one PR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The title and every heading are the page's to write, and both skipped
truncate(), so one hostile scalar could print unbounded output whatever
the budget said. The compact view now cuts titles, headings, and input
names at the same cap and marker every other block gets, and read cuts
even a first block bigger than its whole budget, since 'up to N tokens'
is a promise the page must not be able to break. The distilled page
keeps the full values: --json stays the machine-stable view, bounded by
the fetch cap, and machines cut for themselves.
Fixes#28
contentFailure called every render under 25 tokens a failure, which
made exit 2 claim a terse page was JavaScript-only, gated, or
challenged when it had simply arrived terse. The verdict now needs
evidence: nothing extracted is empty whatever the page weighed, and a
short render is only a failure when the markup behind it was far too
big to have carried only that. A status endpoint or a one-line answer
now exits 0, script-only shells and consent walls still exit 2, and
raw applies the same rule before refusing. The --json empty field
follows the same policy, so callers keep a machine-stable distinction
between empty and merely short.
Fixes#29
Both transports buffered a text response whole before the distiller saw
it, so a hostile URL was an unbounded allocation. Every transport now
enforces one cap, 25MB decoded, three times the largest body oc fetches
on purpose. Content-Length is checked first to fail before the bytes
arrive, and because that header is optional and untrusted, the stream
is counted as it lands too: native fetch and the proxy transport abort
the transfer at the cap, and impers, which buffers inside its own
binding, is checked before its body travels any further. Decoded bytes
are what get counted, which is what stops a decompression bomb.
Fixes#27
Node 24 has bundled an npm new enough for trusted publishing since
24.4, so upgrading npm at publish time only added an unpinned install
to the release path. If a runner ever serves an older 24.x, npm publish
fails loudly rather than shipping anything.
Every GitHub action is now pinned to a full commit SHA with its version
in a trailing comment, verified against each upstream repo's tags, so a
retagged release cannot swap code under CI. The publish workflow pins
npm to an exact version instead of @latest, the experiment Dockerfile
pins its base image by digest, and codeql.yml gets a top-level read-all
permissions block so the default token grants nothing the analyze job
does not ask for.
SECURITY.md points reports at GitHub private vulnerability reporting,
which is now enabled, and scopes out experiments/. Branch protection on
main now blocks force pushes and deletions.
The rest of the languages from #25, each probed for the cheapest shape
that works:
- go: pkg.go.dev renders search on the server, so 'search' is a plain
URL and 'pkg <path>' opens a package page.
- php: manual-lookup.php sends an exact function name straight to its
manual page, so 'fn array_map' lands on the function doc; fuzzy
search goes through DuckDuckGo since the suggestions render behind
the manual's own TOC.
- cpp: cppreference's new static site has clean /cpp/ and /c/ paths;
its native search page is bare links oc refuses as unreadable, so
'search' goes through DuckDuckGo.
- rust: std and doc page opens; rustdoc's search-index.js format is
version-fragile, so 'search' goes through DuckDuckGo.
- java: Javadoc opens pinned to javase/26, the current GA (27
redirects there); the typeSearchIndex file lacks the module names a
URL needs, so 'search' goes through DuckDuckGo.
- ts: handbook page opens; the site searches via Algolia client-side,
so DuckDuckGo again.
- learn gets a 'dotnet <api>' verb, since .NET API docs already live
there.
README table and prose and the skill notes cover the new sites.
RDoc publishes a site's whole search index as one static JS file, the
same way Sphinx and the Node.js docs do, so 'oc ruby search' fetches
docs.ruby-lang.org's index (en/3.4, where the file exists), caches it
under the shared day cache, and ranks it with the same scorer the Node
backend uses. Class methods print as Class.name(params), instance
methods as Class#name(params), and every entry links straight to its
method anchor. 'oc ruby class Array' opens a class page directly.
cli.js now dispatches the local search backends through one map, and
the site-resolution guard keys on the query field, which only a search
shape ever carries.
nodejs.org has no search results page: the site's search box is a
JavaScript modal asking a third-party service, so the search verb went
through DuckDuckGo. But the API docs publish their entire reference as
one static JSON file, all.json, much the way a Sphinx site publishes
its search index, so a new nodedoc backend ranks that file locally:
every module, class, method, property, and event heading becomes a
result linking to its own anchor, and oc node search prints them as a
normal numbered page.
The file is ~8MB (~1MB over the wire) and static, so the day cache the
Sphinx backend used moves to a shared cache module both backends call:
one directory per backend, one file per host, parsed before written so
a block page never poisons it.
A typical result list costs under 100 tokens and answers from disk in
under 100ms once cached.
MDN renders its search results client-side, so 'oc mdn search' has gone
through DuckDuckGo with a site: filter. But the page it renders gets its
results from a public JSON endpoint, /api/v1/search, which returns the
site's own ranking with a summary per result in one ~11KB response. A new
'api' shortcut shape names such an endpoint and which response fields
hold the list, title, URL, and snippet; the response becomes the same
synthetic results page a Sphinx search produces and rides the normal
render path, so 'do <n>' follows a result. First view is ~520 tokens and
works for any site whose search answers as JSON.
CodeQL cannot see that the optional '>' already keeps every bracket out
of the flattened title, and it re-flags each shape of the regex. The
character walk does the same thing in a form whose completeness is plain
to read: nothing between or including brackets survives, closed or not.
CodeQL flagged the tag strip in plainTitle: requiring a closing '>' means
a title whose markup never closes keeps its '<script' fragment. The '>'
is optional now, so every '<' starts a strip and none reaches the results
page, and the output is trimmed. Both call sites already HTML-escape the
result; this hardens the flattening itself. Covered by a new test.
docs.python.org has no search server: its search page downloads the
whole Sphinx index (searchindex.js, ~900KB gzipped) and ranks matches
in the browser. oc now does the same ranking itself, so 'oc py search'
answers from the site's own index instead of DuckDuckGo.
The index is cached on disk for a day and never printed; what renders
is a numbered result list that rides the normal open path, so 'do <n>'
follows a result. A query that names a symbol exactly (json.dumps)
links straight to its anchor. A site definition opts in with a
'sphinx' field naming the docs root, so any Sphinx site can use the
backend later.
First three languages from #25. Each is one definition in clis/:
- oc py library json, oc py doc reference/datamodel (the .html is
appended, matching how the aws shortcuts spell pages)
- oc mdn js Array/map, oc mdn css grid-template-columns, and a generic
oc mdn doc <path> for everything else under /en-US/docs
- oc node api fs
Python renders search client-side (Sphinx), MDN serves its search page
as a client-rendered shell with only the nav in the HTML, and nodejs.org
has no search, so all three get the DuckDuckGo site: filter fallback the
aws and gcp shortcuts already use. Short names py, mdn, and node join
the alias table; python, mozilla, nodejs, and the full domains resolve
as before through the generic rules.
package.json points bin.oc at src/cli.js, so an npm link resolves the
global oc to this file directly and running it needs the bit. Without
it every oc invocation dies with Permission denied while node src/cli.js
keeps working, which is easy to miss.
