* refactor(flows): extract SetupCACertificate for reuse Move the CA load/generate/merge logic out of proxyFlow into an exported flows.SetupCACertificate so the persistent proxy server can reuse it. * feat(proxy): add persistent proxy server with start/stop/env/status Introduces 'pmg proxy' commands backed by internal/proxyserver: a long-lived MITM proxy that intercepts package managers via env vars (no shims). Supports --daemon (Unix), --state, --port; generic 'env' output that skips cert vars when the CA is OS-trusted; opt-in 'stop --fail-on-violation' (fail-closed on crash) with a synchronous cloud event flush; and the malysis analysis cache. * feat(action): add server-mode for persistent proxy When server-mode=true the action starts the proxy daemon and injects proxy env vars into the job instead of installing shims. * test(proxy): add persistent proxy server E2E workflow * docs(readme): document persistent proxy server mode * fix(proxy): create cache dir before writing state file and daemon log On a fresh CI runner the cache directory does not exist yet; os.OpenFile and os.WriteFile do not create parent dirs, so 'pmg proxy start --daemon' failed with 'no such file or directory'. MkdirAll the parent before writing. * docs: add persistent proxy server architecture doc * refactor proxyserver * fix(proxy): always emit cert env vars instead of skipping on OS-trust status npm/pip/yarn/requests trust the MITM CA inconsistently across tools, versions, and configs; many still use bundled CA stores. Always emitting the cert-path env vars is the conservative choice that works regardless, and is harmless for tools that read the OS store (they ignore the vars). Skipping them when a system CA exists would silently break any tool still on a bundled store. * refactor(proxy): drop redundant audit init in daemon; rely on main.go main.go's PersistentPreRun already initializes the audit pipeline for every command (including the daemon's re-exec'd child) and closes it at process exit. Re-initializing in proxyserver.Run created a second auditor and a second cloud-sync WAL connection, orphaning the first. Removing it makes the daemon consistent with the normal proxy flow, which never self-initializes audit. * fix(proxy): bypass proxy env when flushing events to cloud on stop pmg proxy stop inherits HTTP(S)_PROXY (injected by 'pmg proxy env') pointing at the PMG proxy it just shut down. The cloud sync gRPC client honored those vars and routed api.safedep.io through the dead proxy, failing with 'connection refused' so no events were delivered. Clear the proxy env vars before the sync so PMG's own cloud traffic goes direct. * chore(proxy): address review feedback - configurable bind host via proxy.server.listen_host (default loopback) - proxy commands use ui.ErrorExit instead of returning errors to cobra - rename errcode to ProxyPolicyViolation (covers malware + cooldown) - share cloud sync via audit.DrainToCloud (de-dup with cmd/cloud/sync) - centralize proxy CA bundle path in certmanager - docs: persistent proxy cert trust + bind address * fix(proxy): show real message on fail-on-violation error stopExitError set only WithMsg, but ui.ErrorExit renders HumanError, so the framed error showed 'no human-readable message available'. Set both from one string, and emit the framed error before the stdout summary so the blocked count is stated once. * fix(proxy): flush cloud events from the daemon, not stop The stop process inherits HTTP_PROXY (from 'pmg proxy env'), so its cloud client routed api.safedep.io through the already-stopped proxy and failed with connection refused. Move the flush into the daemon's shutdown, which has no proxy env (it started before env injection) and dials SafeDep directly. - daemon flushes on shutdown via audit.DrainToCloud and records the result in the state file; stop surfaces it (on both success and fail-on-violation paths) since the daemon's own logs aren't visible to stop - coordinate stop's wait with the daemon shutdown budget; on timeout, error out without reading stale state or deleting the file (fail-closed) - persist blocked count before the flush so the gate stays correct if the flush hangs or the daemon is killed mid-flush - remove now-redundant cloud_flush.go * disable auto-sync for proxy cmds * feat(proxy): periodic cloud sync + move proxy env vars to packagemanager - daemon runs a periodic cloud-sync ticker so the shutdown flush stays small; the run total is reported by stop, and shutdown timeouts are coordinated - move EnvVarForProxy from config to packagemanager (it is package-manager knowledge); the shared function now builds the proxy URL and NO_PROXY itself, removing the duplicated construction in the per-command and persistent paths - relocate the #319 yarn and #339 IPv6 regression tests alongside the function - enable cloud sync in the persistent-proxy E2E workflow and fix the stale internal/proxystate path filter * refactor(proxy): rename cloudFlushLockTimeout to cloudFlushLockWait Consistent timeout naming: *LockWait is the lock-acquire bound, *Timeout is the sync-RPC bound. Previously the final-flush pair was cloudFlushLockTimeout vs cloudFlushTimeout — two lookalike names for different operations. * refactor(proxy): extract cloudFlush and trim duplicate shutdown comments The shutdown's final-flush block is now a cloudFlush helper, symmetric with startCloudSyncLoop (one-shot vs loop). Removed the triplicated ticker/lock contention comments, keeping the contract on the function doc and one-line pointers at the call sites. * docs: update persistent proxy cloud sync to daemon-owned model The daemon now owns cloud delivery (periodic sync while serving + final flush on shutdown); stop signals it, waits, and reports the result. Rewrite the Cloud event sync section, fix stop attributions, add the cloud_sync state field, and update the sequence diagram. * docs: move Usage section up below How it works Put the copy-paste recipes near the top so users find them before the internals. * refactor(proxy): address PR review feedback - configurable bind host/port via --host/--port flags + config (listen_host, listen_port), bound directly to config fields per PMG's flag pattern - daemon log path via --log-file and readiness timeout in ProxyDaemonConfig; Daemonize no longer owns path policy (caller validates, fails fast) - gate periodic cloud sync on auto_sync; suppress detached background sync for proxy commands instead of flipping the flag - pmg proxy env --export emits shell-quoted lines for eval (spaces survive) - extract shared flows.BuildCachedMalysisAnalyzer, dropping the analyzer+cache duplication between proxy flow and proxy server - add internal/proxyserver/doc.go documenting the package + boundary vs flows - E2E: assert malicious installs are blocked (drop continue-on-error) - docs: trim Commands/State-file to user contracts; refresh bind address * refactor(proxy): proactive alignment fixes from whole-PR review - gate the shutdown cloud flush on auto_sync too, matching the periodic ticker (auto_sync consistently controls all daemon-driven cloud delivery) - ResolveStatePath takes cacheDir instead of *RuntimeConfig, keeping state.go free of config dependency - drop the empty-host comment in listenAddr; keep the loopback guard so a blank host never silently binds all interfaces * fix: Decouple localdb with malysis analyser construction * fix: Persist global args before proxy server daemon exec * fix: GitHub Action for cloud auto-sync in server mode --------- Co-authored-by: Abhisek Datta <abhisek.datta@gmail.com>
8.0 KiB
Persistent Proxy Server
The persistent proxy server runs PMG's MITM proxy as a long-lived process that
intercepts every supported package manager invocation in an environment via
standard proxy environment variables, without shims, aliases, or wrapping each
command with pmg. It is built for non-interactive environments, primarily
CI/CD pipelines (e.g. GitHub Actions), where the environment can be configured
once for the whole job.
It builds on the generic MITM proxy described in proxy.md, reusing
the same interceptor chain, malware analyzer, and certificate manager. The
difference is the lifecycle: instead of PMG starting an ephemeral proxy
around a single subprocess, the proxy is started once, advertises itself to the
other pmg proxy commands, and serves many package manager processes until it
is stopped.
Default proxy mode vs. persistent proxy server
PMG's default proxy mode (see proxy.md) wraps a single command.
pmg npm install starts an ephemeral proxy, runs npm as a child with proxy
env vars injected, then tears the proxy down. The persistent server decouples
these steps.
| Default proxy mode | Persistent proxy server | |
|---|---|---|
| Invocation | pmg npm install (wrapped) |
bare npm install (no wrapper) |
| Proxy lifetime | One subprocess | Until pmg proxy stop |
| Who runs the PM | PMG (as a child) | The user / CI directly |
| Ecosystems served | The one being run | All supported (npm + PyPI) |
| Confirmation on malware | Interactive prompt (TTY) | Auto-block (non-interactive) |
| Reporting | At subprocess exit | At pmg proxy stop |
| Target | Local dev | CI/CD pipelines |
How it works
The diagram below shows the order of events in a CI job. The pmg proxy
commands run in separate workflow steps and coordinate through the running
daemon.
sequenceDiagram
participant CI as CI Job
participant Proxy as Proxy Daemon
participant PM as Package Manager
participant Cloud as SafeDep Cloud
CI->>Proxy: pmg proxy start --daemon
Proxy-->>CI: ready (addr, ca path)
CI->>CI: pmg proxy env (set HTTP_PROXY + CA vars)
PM->>Proxy: package download (via HTTP_PROXY)
Proxy->>Proxy: analyze package
Proxy-->>PM: allow, or 403 block + record event
Proxy->>Cloud: periodic sync of events (while serving)
CI->>Proxy: pmg proxy stop --fail-on-violation
Proxy->>Cloud: final flush of remaining events
Proxy-->>CI: exit non-zero if anything was blocked
Usage
The persistent server targets non-interactive CI/CD. For local development use
the default proxy mode (pmg npm install), which keeps the interactive malware
confirmation prompt. The persistent server auto-blocks without prompting.
GitHub Actions (raw commands):
- run: pmg proxy start --daemon
- run: pmg proxy env >> "$GITHUB_ENV"
- run: npm ci
- run: pmg proxy stop --fail-on-violation
if: always()
GitHub Actions (via the safedep/pmg action server-mode):
- uses: safedep/pmg@v1
with:
server-mode: true
api-key: ${{ secrets.SAFEDEP_API_KEY }}
tenant-id: ${{ secrets.SAFEDEP_TENANT_ID }}
- run: npm ci # intercepted automatically
- name: Enforce PMG policy
if: always()
run: pmg proxy stop --fail-on-violation
In server-mode, the action starts the daemon and injects env vars instead of
installing shims. Because composite actions cannot run an automatic cleanup
step, the final pmg proxy stop --fail-on-violation step is required. It stops
the proxy (the daemon flushes events to the cloud during shutdown) and fails the
job on a block.
Commands
pmg proxy start # start the proxy (foreground, or detached with --daemon)
pmg proxy stop # stop the proxy and report the outcome
pmg proxy env # print env vars that route package managers through it
pmg proxy status # report whether a proxy is running
Run pmg proxy <command> --help for flags. --daemon is Unix only: on
Windows it returns a clear "not supported" error, and the foreground
pmg proxy start still works. To run multiple independent proxies on one host,
give each a distinct --state path and --port.
Bind address
The proxy binds 127.0.0.1 on a random port by default, reachable only from the
host (the right choice for CI and local use). Override with --host/--port, or
the proxy.server.listen_host/listen_port config (flags take precedence).
Bind a non-loopback address (e.g. --host 0.0.0.0) only for a deliberately
hosted deployment: it exposes the MITM proxy to the network, and every client
routed through it has its HTTPS intercepted and must trust the PMG CA.
Certificate trust
The proxy performs TLS MITM, so clients must trust its CA. Trust is delivered
through environment variables, not the OS trust store. pmg proxy env
always emits the cert-path variables pointing at the proxy's CA bundle:
NODE_EXTRA_CA_CERTS, SSL_CERT_FILE, REQUESTS_CA_BUNDLE, PIP_CERT,
YARN_HTTPS_CA_FILE_PATH. Package managers pick these up from the job
environment and trust the proxy's CA, with no OS trust-store install required.
This is deliberate: whether a tool consults the OS trust store varies by tool,
version, and config (npm/Node ignore it by default; modern pip can read it;
requests/certifi ship their own bundle). The cert-path vars work across all
of them, and are harmlessly ignored by tools that do read the OS store.
As a result pmg setup cert install is not needed for the persistent proxy.
If a persisted CA from pmg setup cert install exists the proxy reuses it,
otherwise it generates an ephemeral one. Either way pmg proxy env carries the
trust. OS trust-store install (pmg setup cert install --system) is
intentionally not used: it needs root (breaking container and locked-down
runners), persistently installs a MITM-capable CA into the machine trust store,
and still does not remove the need for the env vars.
Loopback addresses are always excluded from proxying via NO_PROXY
(localhost,127.0.0.1,::1).
Cloud event sync
When SafeDep Cloud is enabled, malware-block events must reach the cloud even on ephemeral CI runners that are destroyed immediately after the job. The daemon owns delivery: it records each blocked package to a durable local event log as it happens, syncs pending events to SafeDep Cloud periodically while serving, and flushes whatever remains on shutdown.
pmg proxy stop reports the recorded result (Synced N event(s) to SafeDep Cloud, or a Cloud sync failed line). A flush failure is surfaced but does not
mask the fail-on-violation exit code.
Fail on violation
By default pmg proxy stop just stops the proxy and exits 0. Failing the CI
job on a policy violation is opt-in via --fail-on-violation.
- It exits non-zero when any package was blocked.
- It fails closed. If the daemon shut down without writing a verifiable
final state (e.g. it crashed),
--fail-on-violationalso fails, because a security gate must not pass on an unverifiable run.
The package manager's own non-zero exit (from the 403 on a blocked download)
is a separate signal. --fail-on-violation gives an authoritative gate from the
proxy regardless of how the package manager reported the failure.
Limitations
- Unix-only daemon.
--daemonis not supported on Windows (foreground mode works). - Non-interactive only. There is no interactive confirmation; flagged packages are always auto-blocked. This is intentional for CI.
- Single proxy per state file. Starting a second proxy that points at the same state file is refused while one is running.
- System-level trust enforcement is out of scope. The server relies on env
var propagation. Enforcing interception for
sudo-scrubbed environments (e.g. viaiptables) and system-wide install (pmg setup install --system) are tracked separately.
References
- proxy.md is the underlying generic MITM proxy server
- config.md is the configuration schema (cloud, proxy, cache dir)
- action.yml is the PMG GitHub Action (
server-mode)