mirror of
https://github.com/safedep/pmg.git
synced 2026-08-03 07:24:09 +02:00
3031665ac765ac0b881161f0e1dab2f7897bd23d
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b19473945b |
Add experimental Go module proxy support (#358)
* feat: add experimental Go module support via pmg go Adds Go modules as a proxy-guarded ecosystem, opt-in only: the command runs solely when invoked explicitly as `pmg go ...` and is deliberately excluded from setup aliases and PATH shims so existing users are unaffected. - packagemanager: goPackageManager with fail-safe command classification (vet/fix excluded from non-download since they can fetch on a cold cache) and pinned-version extraction where only canonical semver counts as explicit. - GOPROXY normalization (fail-closed): effective GOPROXY read via `go env` (honors go env -w), rebuilt comma-joined with `direct` dropped so a 403 block is terminal and nothing silently falls back to unanalyzed VCS fetches. GOPRIVATE/GONOPROXY surface a warning; GOINSECURE is cleared. Contributed to the proxy flow through a new ProxyRoutingProvider hook (extra child env + dynamic MITM hosts). - Go interceptor with dynamic host matching from the user's effective GOPROXY via InterceptorContext.GoProxyHosts. Malware analysis runs on .zip only (the sole endpoint that delivers code); .info/.mod/@latest/ list pass through; /sumdb/ traffic and sum.golang.org are never touched so checksum-db verification stays intact; golang.org/toolchain is allowed on Go's own checksum verification. - Dependency cooldown: publish time captured from .info responses (body unmodified), in-window .zip blocked with 403; fails open for cooldown only when the publish time was never observed. - Cert gate: on macOS/Windows `pmg go` fails fast with actionable guidance unless the persisted PMG CA is OS-trusted (Go ignores SSL_CERT_FILE there); Linux works via the injected bundle. - proxye2e: GOPROXY-protocol mock registry, Go driver and 10 hermetic cases (allow/block/confirm, case-escaped paths, cooldown block and fail-open, toolchain, sumdb passthrough). Verified end-to-end on Linux: `pmg go get github.com/google/uuid@v1.6.0` MITMs proxy.golang.org, analyzes the decoded module at the .zip fetch, and go.sum verification succeeds through the tunneled checksum db. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014xuhBeTVpfU4SdqVaarvuK * fix(go): address review findings on experimental Go support - Drop fmt/clean from NonDownloadCommands: both load packages via go list and can download modules on a cold cache, which would bypass the proxy under install_only. - Support GOPROXY entries with a base path (e.g. corp Athens/JFrog at https://corp/goproxy): the interceptor now receives host -> base URL and strips the path prefix before parsing module URLs, so verdicts and cooldown key on the real module path. - Default unschemed GOPROXY entries to https, matching go's own behavior, so corp mirrors configured as bare hosts are intercepted instead of silently unanalyzed. - Memoize the final verdict per module zip: go re-requests a failed zip during go get's load phase, which double-recorded stats (the report showed the same blocked module twice) and would have re-prompted on Confirm verdicts. - Fetch .info out-of-band on a cooldown cache miss: go serves .info from its local module cache on any machine that used go before PMG, which silently disabled cooldown. Failure of the side-fetch still fails open for cooldown only. - Move the noop package resolver into packagemanager. Verified live: cold-cache cooldown block now records once; warm-cache rerun is blocked via the side-fetch instead of failing open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014xuhBeTVpfU4SdqVaarvuK * docs: collapse Go proxy-mode details by default Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014xuhBeTVpfU4SdqVaarvuK --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
648adcbda4 |
feat: add uvx (uv tool run) package executor (#357)
* feat(uvx): add uvx (uv tool run) package executor Adds support for `uvx`, implemented as a PyPI Executor alongside pipx. uvx is an alias for `uv tool run`: it installs a tool into an ephemeral environment and runs it, so it has no install/list subcommand and the first positional argument (or --from) is the package to audit. Parsing highlights: - --from overrides the positional command as the package to audit - --with packages are audited as additional environment dependencies - name@version shorthand (ruff@0.3.0, ruff@latest) is normalized - flag parsing stops at the tool name so the tool's own flags are not misread as uvx options; uvx's value/boolean flags are registered so none greedily consume the package positional - VCS/URL/local-path specs are skipped for registry auditing Wires up command registration, analytics, shell alias/shim, cloud audit mapping, a dedicated `uvx` sandbox profile (UV_*/PIP_* env, uv cache and tool dirs), config policy, docs, unit tests and an E2E workflow step. Closes #326 https://claude.ai/code/session_011hyLxq7oWJX5Dp4tCEfG19 * chore(uvx): align docs and base profile with uvx support Incorporates the low-risk, non-parser improvements from the community PR #345 (author non-responsive) into our implementation: - list uvx (and the previously-missing pipx) as PyPI managers in the pypi-restrictive base profile package_managers and its README, so the base profile applies directly when selected via --sandbox-profile - document uvx in docs/github-action.md and docs/proxy-mode.md - add version / IsExplicitVersion assertions to the uvx parser tests Our pflag-based parser is kept as-is: unlike #345 it audits --with packages and handles all uvx short flags (e.g. -w), both of which the community PR misses. * fix(uvx): skip interpreter requests; use require in tests Addresses review feedback on PR #357: - uvx interpreter requests (`uvx python`, `uvx python@3.12`, `uvx pypy`, ...) launch an isolated interpreter rather than installing a PyPI tool. Treating the positional as a package made the guard flow resolve/analyze pkg:pypi/python (and python==3.12), which could wrongly block or fail a valid invocation. Skip these for the positional; --with packages on the same command are still audited. - Use require.NoError / require.Len for fatal assertions in the uvx tests, matching the repo's testing convention, so a failure stops the subtest before a nil dereference instead of panicking. * docs(uvx): document fail-open and --with-requirements trade-offs Record the two deliberate parsing decisions raised in review as in-code trade-off comments (no behavior change): - unknown flags are tolerated (fail open), consistent with the other executors; the residual gap only affects non-proxy guard mode since the default proxy flow intercepts every registry download. - --with-requirements / --with-editable values are consumed but not expanded into audit targets; expanding them needs manifest-extractor and guard changes, tracked as follow-up. Proxy mode still covers them. * docs(uvx): drop --with-requirements limitation note Per maintainer review: guard mode is being deprecated and auditing the contents of an existing requirements file is a scanner's responsibility, not PMG's. Remove the "known limitation / follow-up" note; the flags stay registered only so their values are not mistaken for the tool positional. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
c47776db27 |
feat(proxy): persistent proxy server mode (#351)
* refactor(flows): extract SetupCACertificate for reuse Move the CA load/generate/merge logic out of proxyFlow into an exported flows.SetupCACertificate so the persistent proxy server can reuse it. * feat(proxy): add persistent proxy server with start/stop/env/status Introduces 'pmg proxy' commands backed by internal/proxyserver: a long-lived MITM proxy that intercepts package managers via env vars (no shims). Supports --daemon (Unix), --state, --port; generic 'env' output that skips cert vars when the CA is OS-trusted; opt-in 'stop --fail-on-violation' (fail-closed on crash) with a synchronous cloud event flush; and the malysis analysis cache. * feat(action): add server-mode for persistent proxy When server-mode=true the action starts the proxy daemon and injects proxy env vars into the job instead of installing shims. * test(proxy): add persistent proxy server E2E workflow * docs(readme): document persistent proxy server mode * fix(proxy): create cache dir before writing state file and daemon log On a fresh CI runner the cache directory does not exist yet; os.OpenFile and os.WriteFile do not create parent dirs, so 'pmg proxy start --daemon' failed with 'no such file or directory'. MkdirAll the parent before writing. * docs: add persistent proxy server architecture doc * refactor proxyserver * fix(proxy): always emit cert env vars instead of skipping on OS-trust status npm/pip/yarn/requests trust the MITM CA inconsistently across tools, versions, and configs; many still use bundled CA stores. Always emitting the cert-path env vars is the conservative choice that works regardless, and is harmless for tools that read the OS store (they ignore the vars). Skipping them when a system CA exists would silently break any tool still on a bundled store. * refactor(proxy): drop redundant audit init in daemon; rely on main.go main.go's PersistentPreRun already initializes the audit pipeline for every command (including the daemon's re-exec'd child) and closes it at process exit. Re-initializing in proxyserver.Run created a second auditor and a second cloud-sync WAL connection, orphaning the first. Removing it makes the daemon consistent with the normal proxy flow, which never self-initializes audit. * fix(proxy): bypass proxy env when flushing events to cloud on stop pmg proxy stop inherits HTTP(S)_PROXY (injected by 'pmg proxy env') pointing at the PMG proxy it just shut down. The cloud sync gRPC client honored those vars and routed api.safedep.io through the dead proxy, failing with 'connection refused' so no events were delivered. Clear the proxy env vars before the sync so PMG's own cloud traffic goes direct. * chore(proxy): address review feedback - configurable bind host via proxy.server.listen_host (default loopback) - proxy commands use ui.ErrorExit instead of returning errors to cobra - rename errcode to ProxyPolicyViolation (covers malware + cooldown) - share cloud sync via audit.DrainToCloud (de-dup with cmd/cloud/sync) - centralize proxy CA bundle path in certmanager - docs: persistent proxy cert trust + bind address * fix(proxy): show real message on fail-on-violation error stopExitError set only WithMsg, but ui.ErrorExit renders HumanError, so the framed error showed 'no human-readable message available'. Set both from one string, and emit the framed error before the stdout summary so the blocked count is stated once. * fix(proxy): flush cloud events from the daemon, not stop The stop process inherits HTTP_PROXY (from 'pmg proxy env'), so its cloud client routed api.safedep.io through the already-stopped proxy and failed with connection refused. Move the flush into the daemon's shutdown, which has no proxy env (it started before env injection) and dials SafeDep directly. - daemon flushes on shutdown via audit.DrainToCloud and records the result in the state file; stop surfaces it (on both success and fail-on-violation paths) since the daemon's own logs aren't visible to stop - coordinate stop's wait with the daemon shutdown budget; on timeout, error out without reading stale state or deleting the file (fail-closed) - persist blocked count before the flush so the gate stays correct if the flush hangs or the daemon is killed mid-flush - remove now-redundant cloud_flush.go * disable auto-sync for proxy cmds * feat(proxy): periodic cloud sync + move proxy env vars to packagemanager - daemon runs a periodic cloud-sync ticker so the shutdown flush stays small; the run total is reported by stop, and shutdown timeouts are coordinated - move EnvVarForProxy from config to packagemanager (it is package-manager knowledge); the shared function now builds the proxy URL and NO_PROXY itself, removing the duplicated construction in the per-command and persistent paths - relocate the #319 yarn and #339 IPv6 regression tests alongside the function - enable cloud sync in the persistent-proxy E2E workflow and fix the stale internal/proxystate path filter * refactor(proxy): rename cloudFlushLockTimeout to cloudFlushLockWait Consistent timeout naming: *LockWait is the lock-acquire bound, *Timeout is the sync-RPC bound. Previously the final-flush pair was cloudFlushLockTimeout vs cloudFlushTimeout — two lookalike names for different operations. * refactor(proxy): extract cloudFlush and trim duplicate shutdown comments The shutdown's final-flush block is now a cloudFlush helper, symmetric with startCloudSyncLoop (one-shot vs loop). Removed the triplicated ticker/lock contention comments, keeping the contract on the function doc and one-line pointers at the call sites. * docs: update persistent proxy cloud sync to daemon-owned model The daemon now owns cloud delivery (periodic sync while serving + final flush on shutdown); stop signals it, waits, and reports the result. Rewrite the Cloud event sync section, fix stop attributions, add the cloud_sync state field, and update the sequence diagram. * docs: move Usage section up below How it works Put the copy-paste recipes near the top so users find them before the internals. * refactor(proxy): address PR review feedback - configurable bind host/port via --host/--port flags + config (listen_host, listen_port), bound directly to config fields per PMG's flag pattern - daemon log path via --log-file and readiness timeout in ProxyDaemonConfig; Daemonize no longer owns path policy (caller validates, fails fast) - gate periodic cloud sync on auto_sync; suppress detached background sync for proxy commands instead of flipping the flag - pmg proxy env --export emits shell-quoted lines for eval (spaces survive) - extract shared flows.BuildCachedMalysisAnalyzer, dropping the analyzer+cache duplication between proxy flow and proxy server - add internal/proxyserver/doc.go documenting the package + boundary vs flows - E2E: assert malicious installs are blocked (drop continue-on-error) - docs: trim Commands/State-file to user contracts; refresh bind address * refactor(proxy): proactive alignment fixes from whole-PR review - gate the shutdown cloud flush on auto_sync too, matching the periodic ticker (auto_sync consistently controls all daemon-driven cloud delivery) - ResolveStatePath takes cacheDir instead of *RuntimeConfig, keeping state.go free of config dependency - drop the empty-host comment in listenAddr; keep the loopback guard so a blank host never silently binds all interfaces * fix: Decouple localdb with malysis analyser construction * fix: Persist global args before proxy server daemon exec * fix: GitHub Action for cloud auto-sync in server mode --------- Co-authored-by: Abhisek Datta <abhisek.datta@gmail.com> |
||
|
Sahil BansalandGitHub
|
d8abfb6c41 |
fix: flatten proxy skip_commands schema and move docs to proxy-mode.md (#241)
- Replace nested policies map with flat skip_commands map in ProxyConfig - Make skip_commands dependent on install_only being enabled - Move proxy configuration docs from proxy.md to proxy-mode.md - Update config template and tests for new schema |
||
|
Sahil BansalandGitHub
|
f1891271c1 |
Add proxy support for pypi package managers (#150)
* initial pypi registry implementation * support proxy mode for pypi package managers * support proxy mode for pypi package managers - 2 * rm default mode as proxy for pip3 * update goproxy version & fix pypi proxy failing on 304 * add PIP_RETRIES=0 env * update pmg e2e & add proxy mode e2e for pypi * rm safedep-test-pkg for pypi proxy e2e |
||
|
Abhisek DattaandGitHub
|
b5696f989f |
chore: Proxy mode without experimental tag (#147)
* chore: Proxy mode without experimental tag * fix: Update proxy mode docs * fix: Code review fixes |
||
|
|
aa5c528a9d |
docs: Update README (#133)
* docs: Update README * docs: Update README * docs: Update README * docs: Update README * docs: Update README * Apply suggestion from @Sahilb315 Co-authored-by: Sahil Bansal <bansalsahil315@gmail.com> Signed-off-by: Abhisek Datta <abhisek.datta@gmail.com> --------- Signed-off-by: Abhisek Datta <abhisek.datta@gmail.com> Co-authored-by: Sahil Bansal <bansalsahil315@gmail.com> |
||
|
Sahil BansalandGitHub
|
ca224f523c |
enable support for proxy mode for npm package managers (#104)
Signed-off-by: Sahil Bansal <bansalsahil315@gmail.com> |
||
|
|
31f23fd065 |
Add support for package executors and support for PTY handling (#100)
* define contract for package executors * introduce npx executor * add npx and pnpx cmd support * fix typo * rm PackageExecutor and depend on PackageManager interface * add support for PTY to handle parent-child process interaction * refactor PTY handling in proxy flow * enforce interactiveSession interface check * close reader explicitly and clean npm version for pkg executors * rm interaction from interceptors * add docs and wait for outputRouter before exit * add support for non interactive TTY for proxy mode * add support for CI env var check for non interactive tty proxy mode * update readme to include npx, pnpx support * Update internal/flows/proxy_flow.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Sahil Bansal <bansalsahil315@gmail.com> * update ptyx lib * fix docs typo --------- Signed-off-by: Sahil Bansal <bansalsahil315@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Abhisek DattaandGitHub
|
a373b5b243 | docs: Add doc for trusted packages and proxy mode (#102) |