diff --git a/README.md b/README.md index c530a2e..45579a2 100644 --- a/README.md +++ b/README.md @@ -68,10 +68,26 @@ oc ... site shortcut: 'oc hn top', 'oc reddit sub ClaudeAI' oc sites the site shortcuts that ship with oc oc fill type into a numbered input (planned) oc submit [n] submit a form (planned) +oc login seed cookies for a session (--cookie, --domain) +oc logout [session] clear saved cookies for a session ``` Flags: `--budget ` (default 500), `--json`, `--html` (raw as cleaned HTML), `--session `, `--verbose`/`-v` (metrics on stderr, or export `OC_VERBOSE=1`). +### Authenticated sessions + +Pages behind a login need cookies. Seed them once per session, then browse normally: + +```bash +oc login --cookie "session=...; auth=..." --domain example.com --expires 2h --session work +oc open https://example.com/dashboard --session work +oc logout work +``` + +Cookies live in a separate sidecar file (`.cookies.json`) under `~/.only-cli/sessions/`, not in the page-state JSON. The default lifetime is one hour (`--expires 1h`). When cookies expire or the site returns a login page, `oc` says so plainly (exit 2) instead of distilling the login form as content. + +Copy the `Cookie` header from your browser's devtools (Application → Cookies, or the Network tab on a request). `--domain` is the site hostname those cookies belong to. + `oc open` remembers the page it rendered in a JSON file per session under `~/.only-cli` (override with `OC_HOME`), so `oc do 3` follows `[3]` without the agent ever handling a URL. A result title on a search page is a link, so `oc do` on it opens the result rather than repeating the title. Pages longer than the budget say what they left out; `oc find`, `oc read `, and `oc next` read the rest without refetching the page, and a `find` with a single match prints that region instead of the number to read it with. The budget is a target rather than a hard cap: a page that would only run a little long is printed whole rather than cut, since one extra tool call costs far more than the tokens it would have saved. When a page comes back with no readable text (JavaScript-only, a consent wall, a bot challenge), `oc` says so in one line on stderr and exits 2 instead of printing a title and calling it a render. That is a different exit code from every other failure, and `--json` carries the same verdict as an `empty` field, so an agent can tell "this page has nothing on it" from "oc could not read this page" and pay for a browser only when it is worth it. @@ -99,7 +115,7 @@ Works on any mostly-static site with no per-site setup: news sites, blogs, docum A shortcut only ever resolves to a URL and then takes the same path `oc open` does, so it changes nothing about what a page costs or how it reads. The last argument takes every word after it, so `oc ddg search claude code cli` and `oc aws search s3 lifecycle rules` need no quoting, and a path argument keeps its slashes, so `oc learn doc azure/aks/what-is-aks` reaches that page. -A few of these (X, Stack Overflow, YouTube, Microsoft Learn search) read pages that look login-gated or JS-only from the outside, by finding the server-rendered HTML, feed, inline data, or public API the page already ships without a login. Stack Overflow search goes through the Stack Exchange API, and each result prints its `question_id`: read one with the `question ` feed rather than following its link, since the question page itself answers a bot challenge instead of the question. AWS and Google Cloud render docs search purely client-side with no feed, so their `search` goes through DuckDuckGo with a baked-in `site:` filter instead. Not supported yet: pages that only render with JavaScript, sites behind logins, and sites with hard bot challenges that expose no feed. +A few of these (X, Stack Overflow, YouTube, Microsoft Learn search) read pages that look login-gated or JS-only from the outside, by finding the server-rendered HTML, feed, inline data, or public API the page already ships without a login. Stack Overflow search goes through the Stack Exchange API, and each result prints its `question_id`: read one with the `question ` feed rather than following its link, since the question page itself answers a bot challenge instead of the question. AWS and Google Cloud render docs search purely client-side with no feed, so their `search` goes through DuckDuckGo with a baked-in `site:` filter instead. Not supported yet: pages that only render with JavaScript and sites with hard bot challenges that expose no feed. Sites that genuinely require your account can be reached with `oc login` (bring your own cookies). Want a website on that list? Open a pull request, or an issue naming the site — see [CONTRIBUTING.md](CONTRIBUTING.md). @@ -115,9 +131,9 @@ Full methodology, per-task numbers, and other agents/models live in [only-cli/be ## Status -Early. Reading works and is covered by offline tests: static pages, XML feeds, JSON APIs, budget-aware rendering, sessions, and the numbered actions `do`, `find`, `read`, `next`, and `raw`. Writing does not: `fill`, `submit`, and `back` report that they are not implemented rather than pretending, and a lazy headless fallback for script-heavy pages comes after them. +Early. Reading works and is covered by offline tests: static pages, XML feeds, JSON APIs, budget-aware rendering, sessions, authenticated cookie jars, and the numbered actions `do`, `find`, `read`, `next`, and `raw`. Writing does not: `fill`, `submit`, and `back` report that they are not implemented rather than pretending, and a lazy headless fallback for script-heavy pages comes after them. -Known limits, honestly: no JavaScript rendering yet, no sites behind logins yet, and pages behind hard bot challenges may still refuse the tool. +Known limits, honestly: no JavaScript rendering yet, and pages behind hard bot challenges may still refuse the tool. ## Contributors diff --git a/llms.txt b/llms.txt index a312d02..201c0a4 100644 --- a/llms.txt +++ b/llms.txt @@ -17,7 +17,8 @@ Key facts: - X profiles and individual posts read without a login (about 390 and 260 tokens); X search, explore, and hashtag pages do not, and oc reports the block instead of guessing - Requests impersonate Chrome, so pages that block plain scripts often still work - Agent skill included: `npx skills add https://github.com/only-cli/oc --skill web-browsing-cli` ([skills.sh](https://www.skills.sh/only-cli/oc/web-browsing-cli)) -- No JavaScript rendering yet and no login sessions yet (both on the roadmap) +- Authenticated pages: `oc login --cookie "..." --domain example.com [--expires 1h] [--session name]` seeds a timeboxed cookie jar; cookies are sent on every fetch for that session and live in a separate file from page state +- No JavaScript rendering yet (on the roadmap) ## Docs diff --git a/skills/web-browsing-cli/SKILL.md b/skills/web-browsing-cli/SKILL.md index c6b5126..4506835 100644 --- a/skills/web-browsing-cli/SKILL.md +++ b/skills/web-browsing-cli/SKILL.md @@ -15,6 +15,8 @@ npx --yes @only-cli/oc@0.3.0 find where a string appears, or that plac npx --yes @only-cli/oc@0.3.0 next next ~500 tokens of the page already open npx --yes @only-cli/oc@0.3.0 read full text of region [n] npx --yes @only-cli/oc@0.3.0 raw [url] whole page as markdown (--html for cleaned HTML) +npx --yes @only-cli/oc@0.3.0 login seed cookies (--cookie, --domain, --expires) +npx --yes @only-cli/oc@0.3.0 logout [session] clear saved cookies ``` None of these except `open`/`do`/`raw ` fetch anything — they replay the page `open` already saved. @@ -54,9 +56,21 @@ None of these except `open`/`do`/`raw ` fetch anything — they replay the - `--html` — with `raw`, cleaned HTML instead of markdown. - `--verbose` (`-v`/`--stats`) — stderr metrics: tokens saved, HTTP status, client identity, timing, transfer size, memory. Costs tokens itself, so pass only when diagnosing; `OC_VERBOSE=1` turns it on globally. +## Authenticated pages + +Sites that need your account: seed cookies once, then browse normally. + +```bash +oc login --cookie "session=...; auth=..." --domain example.com --expires 2h --session work +oc open https://example.com/dashboard --session work +oc logout work +``` + +Copy the `Cookie` header from browser devtools. Default lifetime is 1h. When cookies expire or the site returns a login page, `oc` says so (exit 2) instead of rendering the login form as content. Cookies live in a separate file from page state and are never included in `--json` output. + ## When not to use it -Pages needing login or heavy client-side JS aren't supported yet. If a page comes back empty or blocked, say so and fall back rather than retrying. +Pages needing heavy client-side JS aren't supported yet. If a page comes back empty or blocked, say so and fall back rather than retrying. ## Untrusted content