The entry link sat beside the byline as an anchor labelled "open", the same
label on every entry, and the repeated-controls filter hid them all on any
feed with five or more entries. Nothing in a subreddit listing led to a post,
so `oc do <n>` on one read its heading instead of following it, and an agent
hunting for the link refetched the feed into Reddit's 429. The title is now
the link, which headingHref already turns into a followable heading.
Following a post out of a feed then landed on the reddit.com HTML page, a
login wall for a logged-out reader, so fetchPage maps reddit.com front page,
subreddit, post, user and search URLs to their www.reddit.com Atom feeds when
the request carries no cookies for the site. Feeds, other paths, and
logged-in sessions are fetched as asked.
Fixes#59.
Reddit's edge answers the chrome fingerprint with a 403 or a 429 while
letting firefox through, and it allows anonymous readers about ten
requests a minute per address, so the wasted chrome attempt on every
read was costing a real share of that budget. identityOrder() picks the
order per host: firefox then chrome for reddit.com and its subdomains,
chrome then firefox everywhere else, with the same downgrade chain and
plain fetch fallback as before. Refs #52.
impers resolves the chrome alias to its newest fingerprint (chrome150 as
of impers 0.1.1), but the native library it loads can be an older system
copy of libcurl-impersonate found before its own pinned download. Such a
copy refuses the fingerprint with an ImpersonateError before any request
leaves, and oc died on the spot with "Impersonating chrome150 is not
supported" (#40).
A refused identity now downgrades the same way a 403 already did: chrome
falls back to firefox, and when both are refused the plain fetch
transport still gets the page. Any other impers failure propagates
unchanged.
The test for revalidating redirect hops drove httpbin.org, so httpbin being
down failed the suite. It is down now, returning 503, which fails CI on main
and would have failed the release: the publish workflow runs npm test before
it ships, so a stable version could not have reached npm while a third
party's app tier was unwell.
It was also testing less than it looked. Each transport carried its own copy
of the redirect loop, and the live test only ever exercised whichever one was
installed, so the guarantee held in one copy and was unproven in the other.
A check that matters twice is a check a change can fix once.
Both transports now share one loop that takes the request as a callback,
which is what makes the hop check provable against a transport that never
leaves the process. Three offline tests replace the live one: a hop to a
private address is refused and never asked for, a hop to a public address is
still followed (a loop that rejected everything would have passed the first
test and broken every redirect on the web), and a cycle gives up. Removing
the hop check fails the first of them, which is more than the httpbin test
could say for the transport it did not run.
Verified live afterwards on both paths: an http to https chain through
impers, a plain page, and a literal and a resolved private address both
still refused.
Testing the 0.3.0-beta.1 build against live URLs turned up a gap the beta
notes claimed was closed: only the native-fetch path checked the content
type, and impers is the default whenever the optional dependency installs.
So 'oc open' on a PNG rendered eight kilobytes of mojibake as a page, with
numbered blocks, an actions footer, and a straight face.
The check now lives in one exported assertReadableType that both transports
call, so a refusal cannot depend on which client happened to get the page.
While the gate was being written down it also grew a correct allow list.
The old one matched the substring html, xml, or json anywhere in the header,
which let application/vnd.ms-htmlhelp through and, worse, refused text/plain:
a robots.txt or an llms.txt is exactly the kind of small text file an agent
asks for, and the fetch path was answering that it was not a page. Readable
now means any text/* type plus the application/* types that are really text,
including the +json and +xml families a feed answers with. A missing header
stays readable, since small servers omit it and the page behind it is fine.
Tested offline against the header strings themselves rather than the network.
PR #5's guard pattern-matched the URL's hostname against a regex, which
both under- and over-blocked: IPv4-mapped IPv6 loopback ([::ffff:127.0.0.1]),
0.0.0.0, and any DNS name that merely resolves to a private address all
sailed through, while a legitimate public hostname like 10.example.com was
wrongly rejected because it starts with "10.". It also only checked the
original URL, so a public URL that redirects to an internal address was
never re-validated.
This replaces the regex with net.isIP + dns.lookup: IP literals are checked
directly (including decoding IPv4-mapped/-compatible IPv6), and DNS names
are resolved first so every address they point to is validated before
connecting. The same check now reruns on every redirect hop for both the
impers and native-fetch transports. Resolving before connecting doesn't
pin the address for the actual connection (neither impers nor fetch expose
that here), so a name that re-resolves differently between this check and
the real connect remains a known, documented residual gap.
only-cli