mirror of
https://github.com/safedep/pmg.git
synced 2026-08-03 07:24:09 +02:00
feat(proxy): persistent proxy server mode (#351)
* refactor(flows): extract SetupCACertificate for reuse Move the CA load/generate/merge logic out of proxyFlow into an exported flows.SetupCACertificate so the persistent proxy server can reuse it. * feat(proxy): add persistent proxy server with start/stop/env/status Introduces 'pmg proxy' commands backed by internal/proxyserver: a long-lived MITM proxy that intercepts package managers via env vars (no shims). Supports --daemon (Unix), --state, --port; generic 'env' output that skips cert vars when the CA is OS-trusted; opt-in 'stop --fail-on-violation' (fail-closed on crash) with a synchronous cloud event flush; and the malysis analysis cache. * feat(action): add server-mode for persistent proxy When server-mode=true the action starts the proxy daemon and injects proxy env vars into the job instead of installing shims. * test(proxy): add persistent proxy server E2E workflow * docs(readme): document persistent proxy server mode * fix(proxy): create cache dir before writing state file and daemon log On a fresh CI runner the cache directory does not exist yet; os.OpenFile and os.WriteFile do not create parent dirs, so 'pmg proxy start --daemon' failed with 'no such file or directory'. MkdirAll the parent before writing. * docs: add persistent proxy server architecture doc * refactor proxyserver * fix(proxy): always emit cert env vars instead of skipping on OS-trust status npm/pip/yarn/requests trust the MITM CA inconsistently across tools, versions, and configs; many still use bundled CA stores. Always emitting the cert-path env vars is the conservative choice that works regardless, and is harmless for tools that read the OS store (they ignore the vars). Skipping them when a system CA exists would silently break any tool still on a bundled store. * refactor(proxy): drop redundant audit init in daemon; rely on main.go main.go's PersistentPreRun already initializes the audit pipeline for every command (including the daemon's re-exec'd child) and closes it at process exit. Re-initializing in proxyserver.Run created a second auditor and a second cloud-sync WAL connection, orphaning the first. Removing it makes the daemon consistent with the normal proxy flow, which never self-initializes audit. * fix(proxy): bypass proxy env when flushing events to cloud on stop pmg proxy stop inherits HTTP(S)_PROXY (injected by 'pmg proxy env') pointing at the PMG proxy it just shut down. The cloud sync gRPC client honored those vars and routed api.safedep.io through the dead proxy, failing with 'connection refused' so no events were delivered. Clear the proxy env vars before the sync so PMG's own cloud traffic goes direct. * chore(proxy): address review feedback - configurable bind host via proxy.server.listen_host (default loopback) - proxy commands use ui.ErrorExit instead of returning errors to cobra - rename errcode to ProxyPolicyViolation (covers malware + cooldown) - share cloud sync via audit.DrainToCloud (de-dup with cmd/cloud/sync) - centralize proxy CA bundle path in certmanager - docs: persistent proxy cert trust + bind address * fix(proxy): show real message on fail-on-violation error stopExitError set only WithMsg, but ui.ErrorExit renders HumanError, so the framed error showed 'no human-readable message available'. Set both from one string, and emit the framed error before the stdout summary so the blocked count is stated once. * fix(proxy): flush cloud events from the daemon, not stop The stop process inherits HTTP_PROXY (from 'pmg proxy env'), so its cloud client routed api.safedep.io through the already-stopped proxy and failed with connection refused. Move the flush into the daemon's shutdown, which has no proxy env (it started before env injection) and dials SafeDep directly. - daemon flushes on shutdown via audit.DrainToCloud and records the result in the state file; stop surfaces it (on both success and fail-on-violation paths) since the daemon's own logs aren't visible to stop - coordinate stop's wait with the daemon shutdown budget; on timeout, error out without reading stale state or deleting the file (fail-closed) - persist blocked count before the flush so the gate stays correct if the flush hangs or the daemon is killed mid-flush - remove now-redundant cloud_flush.go * disable auto-sync for proxy cmds * feat(proxy): periodic cloud sync + move proxy env vars to packagemanager - daemon runs a periodic cloud-sync ticker so the shutdown flush stays small; the run total is reported by stop, and shutdown timeouts are coordinated - move EnvVarForProxy from config to packagemanager (it is package-manager knowledge); the shared function now builds the proxy URL and NO_PROXY itself, removing the duplicated construction in the per-command and persistent paths - relocate the #319 yarn and #339 IPv6 regression tests alongside the function - enable cloud sync in the persistent-proxy E2E workflow and fix the stale internal/proxystate path filter * refactor(proxy): rename cloudFlushLockTimeout to cloudFlushLockWait Consistent timeout naming: *LockWait is the lock-acquire bound, *Timeout is the sync-RPC bound. Previously the final-flush pair was cloudFlushLockTimeout vs cloudFlushTimeout — two lookalike names for different operations. * refactor(proxy): extract cloudFlush and trim duplicate shutdown comments The shutdown's final-flush block is now a cloudFlush helper, symmetric with startCloudSyncLoop (one-shot vs loop). Removed the triplicated ticker/lock contention comments, keeping the contract on the function doc and one-line pointers at the call sites. * docs: update persistent proxy cloud sync to daemon-owned model The daemon now owns cloud delivery (periodic sync while serving + final flush on shutdown); stop signals it, waits, and reports the result. Rewrite the Cloud event sync section, fix stop attributions, add the cloud_sync state field, and update the sequence diagram. * docs: move Usage section up below How it works Put the copy-paste recipes near the top so users find them before the internals. * refactor(proxy): address PR review feedback - configurable bind host/port via --host/--port flags + config (listen_host, listen_port), bound directly to config fields per PMG's flag pattern - daemon log path via --log-file and readiness timeout in ProxyDaemonConfig; Daemonize no longer owns path policy (caller validates, fails fast) - gate periodic cloud sync on auto_sync; suppress detached background sync for proxy commands instead of flipping the flag - pmg proxy env --export emits shell-quoted lines for eval (spaces survive) - extract shared flows.BuildCachedMalysisAnalyzer, dropping the analyzer+cache duplication between proxy flow and proxy server - add internal/proxyserver/doc.go documenting the package + boundary vs flows - E2E: assert malicious installs are blocked (drop continue-on-error) - docs: trim Commands/State-file to user contracts; refresh bind address * refactor(proxy): proactive alignment fixes from whole-PR review - gate the shutdown cloud flush on auto_sync too, matching the periodic ticker (auto_sync consistently controls all daemon-driven cloud delivery) - ResolveStatePath takes cacheDir instead of *RuntimeConfig, keeping state.go free of config dependency - drop the empty-host comment in listenAddr; keep the loopback guard so a blank host never silently binds all interfaces * fix: Decouple localdb with malysis analyser construction * fix: Persist global args before proxy server daemon exec * fix: GitHub Action for cloud auto-sync in server mode --------- Co-authored-by: Abhisek Datta <abhisek.datta@gmail.com>
This commit is contained in:
co-authored by
Abhisek Datta
parent
7ee4187d50
commit
c47776db27
@@ -0,0 +1,110 @@
|
||||
# .github/workflows/persistent-proxy-e2e.yml
|
||||
name: Persistent Proxy E2E
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "cmd/proxy/**"
|
||||
- "internal/proxyserver/**"
|
||||
- "internal/flows/cert.go"
|
||||
- "action.yml"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
persistent-proxy-e2e:
|
||||
name: Persistent Proxy E2E
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
SAFEDEP_API_KEY: ${{ secrets.SAFEDEP_CLOUD_API_KEY }}
|
||||
SAFEDEP_TENANT_ID: ${{ secrets.SAFEDEP_CLOUD_TENANT_DOMAIN}}
|
||||
PMG_CLOUD_ENABLED: "true"
|
||||
PMG_CLOUD_ENDPOINT_ID: github-actions/${{ github.repository }}
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
steps:
|
||||
- name: Checkout Source
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Setup Go
|
||||
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: true
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
||||
with:
|
||||
node-version: "24"
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Build PMG
|
||||
run: make
|
||||
|
||||
- name: Add pmg to PATH
|
||||
run: echo "$GITHUB_WORKSPACE/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Start proxy daemon
|
||||
run: pmg proxy start --daemon
|
||||
|
||||
- name: Inject proxy env
|
||||
run: pmg proxy env >> "$GITHUB_ENV"
|
||||
|
||||
- name: Verify proxy env is set
|
||||
run: |
|
||||
echo "HTTP_PROXY=$HTTP_PROXY"
|
||||
test -n "$HTTP_PROXY"
|
||||
|
||||
- name: Benign npm install succeeds
|
||||
run: |
|
||||
mkdir benign-test && cd benign-test
|
||||
npm init -y
|
||||
npm install lodash@4.17.21
|
||||
test -d node_modules/lodash
|
||||
cd .. && rm -rf benign-test
|
||||
|
||||
- name: Malicious npm install is blocked
|
||||
run: |
|
||||
mkdir mal-npm && cd mal-npm
|
||||
npm init -y
|
||||
# The install MUST fail (proxy blocks it). Assert that, instead of
|
||||
# continue-on-error which would hide a regression where it succeeds.
|
||||
if npm --no-cache --prefer-online install safedep-test-pkg@0.1.3; then
|
||||
echo "ERROR: malicious npm install succeeded but should have been blocked"
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: malicious npm install was blocked"
|
||||
cd .. && rm -rf mal-npm
|
||||
|
||||
- name: Malicious pip install is blocked (python3 -m pip)
|
||||
run: |
|
||||
python3 -m venv venv && source venv/bin/activate
|
||||
if python3 -m pip install safedep-test-pkg; then
|
||||
echo "ERROR: malicious pip install succeeded but should have been blocked"
|
||||
deactivate
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: malicious pip install was blocked"
|
||||
deactivate && rm -rf venv
|
||||
|
||||
- name: Stop proxy and verify it fails on the blocks
|
||||
if: always()
|
||||
run: |
|
||||
# We blocked packages above, so --fail-on-violation must exit non-zero.
|
||||
# Assert that (a green test means the gate works), and always run so the
|
||||
# daemon is stopped even if an earlier step failed.
|
||||
if pmg proxy stop --fail-on-violation; then
|
||||
echo "ERROR: stop should have exited non-zero (packages were blocked)"
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: stop correctly failed on the policy violation"
|
||||
@@ -209,12 +209,15 @@ Protect CI workflows with one step. PMG analyzes every `npm install`,
|
||||
`pip install`, etc. in the job.
|
||||
|
||||
```yaml
|
||||
# Consider pinning third-party Actions to a full commit SHA
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
- uses: safedep/pmg@v1
|
||||
- run: npm ci
|
||||
with:
|
||||
server-mode: true
|
||||
|
||||
- run: npm ci # intercepted via HTTP_PROXY automatically
|
||||
|
||||
- name: Enforce PMG policy
|
||||
if: always()
|
||||
run: pmg proxy stop --fail-on-violation # stops the daemon, fails the job on a block
|
||||
```
|
||||
|
||||
By default you get malware blocking and dependency cooldown. Sandbox isolation
|
||||
@@ -262,6 +265,7 @@ PMG builds are reproducible and signed.
|
||||
- [Dependency Cooldown](docs/dependency-cooldown.md)
|
||||
- [Caching](docs/caching.md)
|
||||
- [Proxy Mode Architecture](docs/proxy-mode.md)
|
||||
- [Persistent Proxy Server](docs/persistent-proxy.md)
|
||||
- [Certificate Authority](docs/cert.md)
|
||||
- [Sandboxing](docs/sandbox.md)
|
||||
|
||||
|
||||
+33
-8
@@ -73,6 +73,11 @@ inputs:
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
server-mode:
|
||||
description: Run PMG as a persistent proxy server instead of shims. The proxy is started as a daemon and HTTP_PROXY/cert env vars are exported to the job. Requires a job-end "pmg proxy stop --fail-on-violation" step.
|
||||
required: false
|
||||
default: "false"
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
@@ -235,6 +240,7 @@ runs:
|
||||
IN_VERBOSITY: ${{ inputs.verbosity }}
|
||||
IN_DISABLE_TELEMETRY: ${{ inputs.disable-telemetry }}
|
||||
IN_SKIP_EVENT_LOGGING: ${{ inputs.skip-event-logging }}
|
||||
IN_SERVER_MODE: ${{ inputs.server-mode }}
|
||||
GH_REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
@@ -258,12 +264,18 @@ runs:
|
||||
export_var SAFEDEP_TENANT_ID "$IN_TENANT_ID"
|
||||
export_var PMG_CLOUD_ENABLED "true"
|
||||
|
||||
# PMG's opportunistic background auto-sync is designed for long-lived
|
||||
# workstations. On an ephemeral runner the detached child can be
|
||||
# torn down before it drains the WAL, and the WAL goes with it. Turn
|
||||
# it off and rely on an explicit `pmg cloud sync` at job-end if the
|
||||
# user wants events delivered.
|
||||
export_var PMG_CLOUD_AUTO_SYNC_ENABLED "false"
|
||||
if [ "$IN_SERVER_MODE" = "true" ]; then
|
||||
# In server mode the proxy daemon owns delivery and uses auto_sync
|
||||
# to gate both periodic sync and the shutdown flush.
|
||||
export_var PMG_CLOUD_AUTO_SYNC_ENABLED "true"
|
||||
else
|
||||
# PMG's opportunistic background auto-sync is designed for
|
||||
# long-lived workstations. On an ephemeral runner the detached child
|
||||
# can be torn down before it drains the WAL, and the WAL goes with
|
||||
# it. Turn it off and rely on an explicit `pmg cloud sync` at
|
||||
# job-end if the user wants events delivered.
|
||||
export_var PMG_CLOUD_AUTO_SYNC_ENABLED "false"
|
||||
fi
|
||||
|
||||
if [ -n "$IN_ENDPOINT_ID" ]; then
|
||||
export_var PMG_CLOUD_ENDPOINT_ID "$IN_ENDPOINT_ID"
|
||||
@@ -292,14 +304,27 @@ runs:
|
||||
export_var PMG_DISABLE_TELEMETRY "$IN_DISABLE_TELEMETRY"
|
||||
export_var PMG_SKIP_EVENT_LOGGING "$IN_SKIP_EVENT_LOGGING"
|
||||
|
||||
- name: Run pmg setup install
|
||||
- name: Run pmg setup install (shim mode)
|
||||
if: inputs.server-mode != 'true'
|
||||
shell: bash
|
||||
run: pmg setup install
|
||||
|
||||
- name: Add PMG shims to PATH
|
||||
- name: Add PMG shims to PATH (shim mode)
|
||||
if: inputs.server-mode != 'true'
|
||||
shell: bash
|
||||
run: echo "$HOME/.pmg/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Start PMG proxy server (server mode)
|
||||
if: inputs.server-mode == 'true'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# No cert install needed: `pmg proxy env` exports the CA path via
|
||||
# NODE_EXTRA_CA_CERTS/SSL_CERT_FILE/PIP_CERT, so package managers trust
|
||||
# the proxy without touching the OS trust store.
|
||||
pmg proxy start --daemon
|
||||
pmg proxy env >> "$GITHUB_ENV"
|
||||
|
||||
- name: Verify PMG
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
+7
-44
@@ -1,10 +1,9 @@
|
||||
package cloud
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/dry/usefulerror"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/errcodes"
|
||||
@@ -43,50 +42,14 @@ func runSync(cmd *cobra.Command, args []string) error {
|
||||
WithHelp("Set 'cloud.enabled: true' in PMG config to enable cloud sync"))
|
||||
}
|
||||
|
||||
lock := audit.NewSyncLock(cfg.CloudSyncLockPath())
|
||||
lockCtx, lockCancel := context.WithTimeout(cmd.Context(), manualSyncLockTimeout)
|
||||
defer lockCancel()
|
||||
|
||||
locked, err := lock.TryLockContext(lockCtx, 250*time.Millisecond)
|
||||
synced, err := audit.DrainToCloud(cmd.Context(), cfg, manualSyncLockTimeout, syncTimeout)
|
||||
if err != nil {
|
||||
ui.ErrorExit(usefulerror.NewUsefulError().
|
||||
Wrap(err).
|
||||
WithCode(errcodes.Lifecycle).
|
||||
WithHumanError("Failed to acquire cloud sync lock").
|
||||
WithHelp("Another sync may be in progress; try again shortly"))
|
||||
}
|
||||
if !locked {
|
||||
ui.ErrorExit(usefulerror.NewUsefulError().
|
||||
WithCode(errcodes.Lifecycle).
|
||||
WithHumanError("Another cloud sync is already in progress").
|
||||
WithHelp("Wait for the in-progress sync to finish, then try again"))
|
||||
}
|
||||
defer func() {
|
||||
if err := lock.Unlock(); err != nil {
|
||||
log.Warnf("failed to release cloud sync lock: %v", err)
|
||||
if errors.Is(err, audit.ErrSyncInProgress) {
|
||||
ui.ErrorExit(usefulerror.NewUsefulError().
|
||||
WithCode(errcodes.Lifecycle).
|
||||
WithHumanError("Another cloud sync is already in progress").
|
||||
WithHelp("Wait for the in-progress sync to finish, then try again"))
|
||||
}
|
||||
}()
|
||||
|
||||
ctx, cancel := context.WithTimeout(cmd.Context(), syncTimeout)
|
||||
defer cancel()
|
||||
|
||||
bundle, err := audit.NewSyncClientBundle(cfg)
|
||||
if err != nil {
|
||||
ui.ErrorExit(usefulerror.NewUsefulError().
|
||||
Wrap(err).
|
||||
WithCode(errcodes.Lifecycle).
|
||||
WithHumanError("Failed to initialize cloud sync client").
|
||||
WithHelp("Run 'pmg cloud login' to store credentials, or set SAFEDEP_API_KEY and SAFEDEP_TENANT_ID environment variables"))
|
||||
}
|
||||
defer func() {
|
||||
if err := bundle.Close(); err != nil {
|
||||
log.Warnf("failed to close sync client: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
synced, err := bundle.Sync(ctx)
|
||||
recordLastSyncAttempt(cfg)
|
||||
if err != nil {
|
||||
ui.ErrorExit(usefulerror.NewUsefulError().
|
||||
Wrap(err).
|
||||
WithCode(errcodes.Network).
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/internal/proxyserver"
|
||||
"github.com/safedep/pmg/internal/ui"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var envExportFlag bool
|
||||
|
||||
func newEnvCommand() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "env",
|
||||
Short: "Print proxy environment variables (KEY=VALUE per line)",
|
||||
Long: "Print proxy environment variables as KEY=VALUE lines.\n\n" +
|
||||
"GitHub Actions: pmg proxy env >> \"$GITHUB_ENV\"\n" +
|
||||
"Shell: eval \"$(pmg proxy env --export)\"\n\n" +
|
||||
"Use --export for shell `eval`: it emits quoted `export KEY='VALUE'`\n" +
|
||||
"lines that survive values containing spaces. The\n" +
|
||||
"default raw KEY=VALUE form is for $GITHUB_ENV, which must not be quoted.",
|
||||
RunE: runEnv,
|
||||
}
|
||||
cmd.Flags().BoolVar(&envExportFlag, "export", false, "Emit shell `export KEY='VALUE'` lines for `eval`")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func runEnv(_ *cobra.Command, _ []string) error {
|
||||
cfg := config.Get()
|
||||
statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
|
||||
|
||||
vars, err := proxyserver.EnvVars(statePath)
|
||||
if err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
|
||||
w := bufio.NewWriter(os.Stdout)
|
||||
for _, v := range vars {
|
||||
line := v
|
||||
if envExportFlag {
|
||||
line = exportLine(v)
|
||||
}
|
||||
if _, werr := fmt.Fprintln(w, line); werr != nil {
|
||||
ui.ErrorExit(fmt.Errorf("write env var: %w", werr))
|
||||
}
|
||||
}
|
||||
|
||||
if err := w.Flush(); err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// exportLine turns "KEY=VALUE" into a shell-safe `export KEY='VALUE'`, so values
|
||||
// with spaces (e.g. the macOS "Application Support" path) survive `eval`.
|
||||
func exportLine(kv string) string {
|
||||
k, v, ok := strings.Cut(kv, "=")
|
||||
if !ok {
|
||||
return kv
|
||||
}
|
||||
return fmt.Sprintf("export %s=%s", k, shellSingleQuote(v))
|
||||
}
|
||||
|
||||
// shellSingleQuote wraps s in single quotes, escaping any embedded single quote
|
||||
// as '\” (close, escaped quote, reopen).
|
||||
func shellSingleQuote(s string) string {
|
||||
return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package proxy
|
||||
|
||||
import "github.com/spf13/cobra"
|
||||
|
||||
// stateFlag binds the persistent --state flag shared by all proxy subcommands.
|
||||
var stateFlag string
|
||||
|
||||
func NewProxyCommand() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "proxy",
|
||||
Short: "Manage the persistent PMG proxy server",
|
||||
}
|
||||
cmd.AddCommand(newStartCommand())
|
||||
cmd.AddCommand(newStopCommand())
|
||||
cmd.AddCommand(newEnvCommand())
|
||||
cmd.AddCommand(newStatusCommand())
|
||||
cmd.PersistentFlags().StringVar(&stateFlag, "state", "",
|
||||
"Path to the proxy state file (default: <cache-dir>/proxy-state.json)")
|
||||
return cmd
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/internal/proxyserver"
|
||||
"github.com/safedep/pmg/internal/ui"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var (
|
||||
daemonFlag bool
|
||||
logFileFlag string
|
||||
foregroundInternalFlag bool
|
||||
)
|
||||
|
||||
func newStartCommand() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "start",
|
||||
Short: "Start the persistent PMG proxy server",
|
||||
RunE: runStart,
|
||||
}
|
||||
|
||||
// Bind --host/--port directly onto the config fields with the loaded config
|
||||
// values as defaults, matching PMG's flag pattern (see config/cobra.go): a
|
||||
// supplied flag overwrites the field, otherwise the config value stands.
|
||||
// Precedence: flag > env > config file > default.
|
||||
srv := &config.Get().Config.Proxy.Server
|
||||
|
||||
cmd.Flags().BoolVarP(&daemonFlag, "daemon", "D", false, "Run the proxy as a detached background process")
|
||||
cmd.Flags().StringVar(&srv.ListenHost, "host", srv.ListenHost, "Host to bind")
|
||||
cmd.Flags().IntVar(&srv.ListenPort, "port", srv.ListenPort, "Port to bind (0 = a random free port)")
|
||||
cmd.Flags().StringVar(&logFileFlag, "log-file", "", "File for the daemon's output (default: <cache-dir>/proxy.log)")
|
||||
cmd.Flags().BoolVar(&foregroundInternalFlag, "foreground-internal", false, "Internal: run the foreground server (used by --daemon)")
|
||||
if err := cmd.Flags().MarkHidden("foreground-internal"); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return cmd
|
||||
}
|
||||
|
||||
func runStart(cmd *cobra.Command, _ []string) error {
|
||||
cfg := config.Get()
|
||||
statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
|
||||
host := cfg.Config.Proxy.Server.ListenHost
|
||||
port := cfg.Config.Proxy.Server.ListenPort
|
||||
|
||||
if daemonFlag && !foregroundInternalFlag {
|
||||
if err := startDaemon(cmd, cfg, statePath, host, port); err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
if err := proxyserver.Run(cmd.Context(), cfg, statePath, host, port); err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func startDaemon(cmd *cobra.Command, cfg *config.RuntimeConfig, statePath, host string, port int) error {
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return fmt.Errorf("resolve executable: %w", err)
|
||||
}
|
||||
|
||||
// Daemon log: the --log-file flag if set, else <cache-dir>/proxy.log. The
|
||||
// caller owns this path, so ensure its parent directory exists here.
|
||||
logPath := logFileFlag
|
||||
if logPath == "" {
|
||||
logPath = filepath.Join(cfg.CacheDir(), "proxy.log")
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(logPath), 0o700); err != nil {
|
||||
return fmt.Errorf("create daemon log dir: %w", err)
|
||||
}
|
||||
|
||||
args := daemonArgs(cmd, statePath, host, port)
|
||||
|
||||
daemonCfg := proxyserver.ProxyDaemonConfig{
|
||||
LogPath: logPath,
|
||||
ReadyTimeout: proxyserver.DefaultDaemonReadyTimeout,
|
||||
}
|
||||
state, err := proxyserver.Daemonize(daemonCfg, statePath, exe, args)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, werr := fmt.Fprintf(os.Stdout, "PMG proxy daemon started on %s (pid %d)\n", state.Addr, state.PID)
|
||||
return werr
|
||||
}
|
||||
|
||||
func daemonArgs(cmd *cobra.Command, statePath, host string, port int) []string {
|
||||
args := append([]string{}, config.ChangedConfigFlagArgs(cmd)...)
|
||||
return append(args,
|
||||
"proxy", "start", "--foreground-internal",
|
||||
"--state", statePath,
|
||||
"--host", host,
|
||||
"--port", strconv.Itoa(port),
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/spf13/cobra"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestDaemonArgsPrependsChangedConfigFlags(t *testing.T) {
|
||||
root := &cobra.Command{Use: "pmg"}
|
||||
config.ApplyCobraFlags(root)
|
||||
|
||||
var got []string
|
||||
start := &cobra.Command{
|
||||
Use: "start",
|
||||
Run: func(cmd *cobra.Command, _ []string) {
|
||||
got = daemonArgs(cmd, "/tmp/proxy-state.json", "127.0.0.1", 9000)
|
||||
},
|
||||
}
|
||||
proxyCmd := &cobra.Command{Use: "proxy"}
|
||||
proxyCmd.AddCommand(start)
|
||||
root.AddCommand(proxyCmd)
|
||||
root.SetArgs([]string{
|
||||
"--paranoid",
|
||||
"--skip-dependency-cooldown",
|
||||
"proxy", "start",
|
||||
})
|
||||
|
||||
require.NoError(t, root.Execute())
|
||||
assert.Equal(t, []string{
|
||||
"--paranoid=true",
|
||||
"--skip-dependency-cooldown=true",
|
||||
"proxy", "start", "--foreground-internal",
|
||||
"--state", "/tmp/proxy-state.json",
|
||||
"--host", "127.0.0.1",
|
||||
"--port", "9000",
|
||||
}, got)
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/internal/proxyserver"
|
||||
"github.com/safedep/pmg/internal/ui"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
func newStatusCommand() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "status",
|
||||
Short: "Show the status of the persistent PMG proxy server",
|
||||
RunE: runStatus,
|
||||
}
|
||||
}
|
||||
|
||||
func runStatus(_ *cobra.Command, _ []string) error {
|
||||
cfg := config.Get()
|
||||
statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
|
||||
|
||||
st := proxyserver.GetStatus(statePath)
|
||||
|
||||
var line string
|
||||
switch {
|
||||
case !st.Found:
|
||||
line = "PMG proxy: not running (no state file)\n"
|
||||
case st.Running:
|
||||
line = fmt.Sprintf("PMG proxy: running (pid %d, addr %s, ca %s)\n", st.PID, st.Addr, st.CACert)
|
||||
default:
|
||||
line = fmt.Sprintf("PMG proxy: stopped (stale state for pid %d — run 'pmg proxy stop' to clean up)\n", st.PID)
|
||||
}
|
||||
|
||||
if _, err := fmt.Fprint(os.Stdout, line); err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/safedep/dry/usefulerror"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/errcodes"
|
||||
"github.com/safedep/pmg/internal/proxyserver"
|
||||
"github.com/safedep/pmg/internal/ui"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var failOnViolation bool
|
||||
|
||||
func newStopCommand() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "stop",
|
||||
Short: "Stop the running persistent PMG proxy server",
|
||||
// Failures here are user-facing policy outcomes, not usage errors.
|
||||
SilenceUsage: true,
|
||||
RunE: runStop,
|
||||
}
|
||||
cmd.Flags().BoolVar(&failOnViolation, "fail-on-violation", false,
|
||||
"Exit non-zero if any package was blocked during the proxy session")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func runStop(_ *cobra.Command, _ []string) error {
|
||||
cfg := config.Get()
|
||||
statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
|
||||
|
||||
res, err := proxyserver.Stop(statePath)
|
||||
if err != nil {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
|
||||
// Surface the daemon's cloud flush outcome before the violation gate, so it
|
||||
// shows on both the success and the failure path (the daemon's own logs are
|
||||
// not visible to this process). nil means cloud sync is disabled.
|
||||
if res.CloudSync != nil {
|
||||
line := fmt.Sprintf("Synced %d event(s) to SafeDep Cloud\n", res.CloudSync.Synced)
|
||||
if res.CloudSync.Error != "" {
|
||||
line = fmt.Sprintf("Cloud sync failed: %s\n", res.CloudSync.Error)
|
||||
}
|
||||
if _, werr := fmt.Fprint(os.Stdout, line); werr != nil {
|
||||
ui.ErrorExit(werr)
|
||||
}
|
||||
}
|
||||
|
||||
// On a policy violation the framed error states the blocked count, so exit
|
||||
// here before the plain summary to avoid stating the count twice.
|
||||
if verr := stopExitError(res, failOnViolation); verr != nil {
|
||||
ui.ErrorExit(verr)
|
||||
}
|
||||
|
||||
summary := fmt.Sprintf("PMG proxy stopped — %d package(s) blocked\n", res.BlockedCount)
|
||||
if !res.StateVerified {
|
||||
summary = fmt.Sprintf("PMG proxy (pid %d) stopped (final state unavailable)\n", res.PID)
|
||||
}
|
||||
if _, werr := fmt.Fprint(os.Stdout, summary); werr != nil {
|
||||
ui.ErrorExit(werr)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// stopExitError maps a stop result to the command's exit status. With
|
||||
// --fail-on-violation: any blocked package fails, and an unverifiable final
|
||||
// state (e.g. a crashed proxy) fails closed — a security gate must not pass on
|
||||
// an unverifiable run. Without the flag, stop always succeeds.
|
||||
func stopExitError(res proxyserver.StopResult, failOnViolation bool) error {
|
||||
if !failOnViolation {
|
||||
return nil
|
||||
}
|
||||
|
||||
// HumanError drives the displayed message; Msg drives Error()/logs and the
|
||||
// --verbose tail. Set both from one string so verbose shows the real
|
||||
// message instead of usefulerror's "unknown error" fallback.
|
||||
if !res.StateVerified {
|
||||
msg := "Proxy shut down but the blocked-package count could not be verified"
|
||||
return usefulerror.NewUsefulError().
|
||||
WithCode(errcodes.ProxyPolicyViolation).
|
||||
WithHumanError(msg).
|
||||
WithMsg(msg).
|
||||
WithHelp("The proxy may have crashed; treat this run as failed and re-run")
|
||||
}
|
||||
|
||||
if res.BlockedCount > 0 {
|
||||
msg := fmt.Sprintf("%d package(s) were blocked by the proxy", res.BlockedCount)
|
||||
return usefulerror.NewUsefulError().
|
||||
WithCode(errcodes.ProxyPolicyViolation).
|
||||
WithHumanError(msg).
|
||||
WithMsg(msg).
|
||||
WithHelp("Review the proxy logs for details on blocked packages")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/safedep/pmg/internal/proxyserver"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestStopExitError(t *testing.T) {
|
||||
t.Run("no flag, blocks present -> nil", func(t *testing.T) {
|
||||
err := stopExitError(proxyserver.StopResult{BlockedCount: 3, StateVerified: true}, false)
|
||||
assert.NoError(t, err)
|
||||
})
|
||||
|
||||
t.Run("flag, no blocks -> nil", func(t *testing.T) {
|
||||
err := stopExitError(proxyserver.StopResult{BlockedCount: 0, StateVerified: true}, true)
|
||||
assert.NoError(t, err)
|
||||
})
|
||||
|
||||
t.Run("flag, blocks present -> error", func(t *testing.T) {
|
||||
err := stopExitError(proxyserver.StopResult{BlockedCount: 2, StateVerified: true}, true)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "2 package")
|
||||
})
|
||||
|
||||
t.Run("flag, crash (unverified state) -> fail closed", func(t *testing.T) {
|
||||
err := stopExitError(proxyserver.StopResult{StateVerified: false}, true)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "could not be verified")
|
||||
})
|
||||
|
||||
t.Run("no flag, crash -> nil", func(t *testing.T) {
|
||||
err := stopExitError(proxyserver.StopResult{StateVerified: false}, false)
|
||||
assert.NoError(t, err)
|
||||
})
|
||||
}
|
||||
+2
-2
@@ -8,10 +8,10 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/safedep/dry/localdb"
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/pmg/analyzer/malysiscache"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/internal/localstore"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
@@ -61,7 +61,7 @@ func openCache(ctx context.Context, cfg *config.RuntimeConfig) (cache *malysisca
|
||||
return nil, func() {}, false, statErr
|
||||
}
|
||||
|
||||
mgr := localdb.New(localdb.Config{Dir: cfg.LocalDBDir(), FileName: cfg.LocalDBFileName()})
|
||||
mgr := localstore.NewManager(cfg)
|
||||
store, serr := mgr.Store(ctx, malysiscache.Descriptor())
|
||||
if serr != nil {
|
||||
if cerr := mgr.Close(); cerr != nil {
|
||||
|
||||
@@ -113,6 +113,36 @@ func ApplyCobraFlags(cmd *cobra.Command) {
|
||||
}
|
||||
}
|
||||
|
||||
// ChangedConfigFlagArgs returns the explicitly supplied root config flags in a
|
||||
// form that can be passed to a re-execed PMG process.
|
||||
func ChangedConfigFlagArgs(cmd *cobra.Command) []string {
|
||||
var args []string
|
||||
for _, spec := range configFlagSpecs {
|
||||
flag := cmd.Flags().Lookup(spec.name)
|
||||
if flag == nil || !flag.Changed {
|
||||
continue
|
||||
}
|
||||
|
||||
name := "--" + spec.name
|
||||
if flag.Value.Type() == "bool" {
|
||||
args = append(args, name+"="+flag.Value.String())
|
||||
continue
|
||||
}
|
||||
|
||||
for _, value := range changedFlagValues(flag) {
|
||||
args = append(args, name, value)
|
||||
}
|
||||
}
|
||||
return args
|
||||
}
|
||||
|
||||
func changedFlagValues(flag *pflag.Flag) []string {
|
||||
if value, ok := flag.Value.(pflag.SliceValue); ok {
|
||||
return value.GetSlice()
|
||||
}
|
||||
return []string{flag.Value.String()}
|
||||
}
|
||||
|
||||
// RejectManagedFlagOverrides fails when the active config is a locked global
|
||||
// config and the user explicitly set a flag whose value that config governs.
|
||||
// Operational flags (managed == false) are unaffected, and an unlocked managed
|
||||
|
||||
@@ -83,6 +83,41 @@ func TestRejectManagedFlagOverridesDetectsInheritedFlag(t *testing.T) {
|
||||
assert.Contains(t, err.Error(), "globally managed")
|
||||
}
|
||||
|
||||
func TestChangedConfigFlagArgs(t *testing.T) {
|
||||
withLockedState(t, false)
|
||||
|
||||
root := &cobra.Command{Use: "pmg"}
|
||||
ApplyCobraFlags(root)
|
||||
|
||||
var got []string
|
||||
child := &cobra.Command{
|
||||
Use: "proxy",
|
||||
Run: func(cmd *cobra.Command, _ []string) {
|
||||
got = ChangedConfigFlagArgs(cmd)
|
||||
},
|
||||
}
|
||||
root.AddCommand(child)
|
||||
root.SetArgs([]string{
|
||||
"--paranoid=false",
|
||||
"--transitive-depth", "7",
|
||||
"--sandbox-profile", "strict",
|
||||
"--sandbox-allow", "read=/tmp",
|
||||
"--sandbox-allow", "net-connect=registry.npmjs.org:443",
|
||||
"--skip-dependency-cooldown",
|
||||
"proxy",
|
||||
})
|
||||
|
||||
require.NoError(t, root.Execute())
|
||||
assert.Equal(t, []string{
|
||||
"--transitive-depth", "7",
|
||||
"--paranoid=false",
|
||||
"--sandbox-profile", "strict",
|
||||
"--sandbox-allow", "read=/tmp",
|
||||
"--sandbox-allow", "net-connect=registry.npmjs.org:443",
|
||||
"--skip-dependency-cooldown=true",
|
||||
}, got)
|
||||
}
|
||||
|
||||
// Proves the SSOT table is internally consistent: every spec actually binds a
|
||||
// flag, every registered flag traces back to a spec (no out-of-band flags), and
|
||||
// the managed classification matches intent. Catches accidental managed flips
|
||||
|
||||
@@ -165,6 +165,20 @@ type ProxyConfig struct {
|
||||
Enabled bool `mapstructure:"enabled"`
|
||||
InstallOnly bool `mapstructure:"install_only"`
|
||||
SkipCommands map[string][]string `mapstructure:"skip_commands"`
|
||||
Server ProxyServerConfig `mapstructure:"server"`
|
||||
}
|
||||
|
||||
// ProxyServerConfig configures the persistent proxy server (`pmg proxy start`).
|
||||
type ProxyServerConfig struct {
|
||||
// ListenHost is the host the persistent proxy binds to. Defaults to
|
||||
// 127.0.0.1 (loopback). Set to 0.0.0.0 or a specific interface only for a
|
||||
// deliberately hosted deployment: a non-loopback bind exposes the MITM
|
||||
// proxy to the network. The --host flag overrides this.
|
||||
ListenHost string `mapstructure:"listen_host"`
|
||||
|
||||
// ListenPort is the port the persistent proxy binds to. 0 (default) means a
|
||||
// random free port. The --port flag overrides this.
|
||||
ListenPort int `mapstructure:"listen_port"`
|
||||
}
|
||||
|
||||
// SandboxConfig configures the sandbox system for isolating package manager processes.
|
||||
@@ -481,6 +495,9 @@ func DefaultConfig() RuntimeConfig {
|
||||
Enabled: true,
|
||||
InstallOnly: false,
|
||||
SkipCommands: map[string][]string{},
|
||||
Server: ProxyServerConfig{
|
||||
ListenHost: "127.0.0.1",
|
||||
},
|
||||
},
|
||||
},
|
||||
DryRun: false,
|
||||
|
||||
@@ -51,6 +51,19 @@ proxy:
|
||||
skip_commands:
|
||||
npm: []
|
||||
|
||||
# Persistent proxy server (`pmg proxy start`) settings.
|
||||
server:
|
||||
# Host the persistent proxy binds to. Defaults to 127.0.0.1 (loopback),
|
||||
# which keeps the MITM proxy private to the host (the right choice for CI
|
||||
# and local use). Set to 0.0.0.0 or a specific interface ONLY for a
|
||||
# deliberately hosted deployment: a non-loopback bind exposes the proxy,
|
||||
# and every client must trust the PMG CA. The --host flag overrides this.
|
||||
listen_host: 127.0.0.1
|
||||
|
||||
# Port the persistent proxy binds to. 0 means a random free port. The
|
||||
# --port flag overrides this.
|
||||
listen_port: 0
|
||||
|
||||
# Trusted packages are packages that are trusted by the user and will be ignored by the security guardrails.
|
||||
# This is useful for packages that are known to be safe and are used in the application.
|
||||
# Example:
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
# Persistent Proxy Server
|
||||
|
||||
The persistent proxy server runs PMG's MITM proxy as a long-lived process that
|
||||
intercepts **every** supported package manager invocation in an environment via
|
||||
standard proxy environment variables, without shims, aliases, or wrapping each
|
||||
command with `pmg`. It is built for non-interactive environments, primarily
|
||||
CI/CD pipelines (e.g. GitHub Actions), where the environment can be configured
|
||||
once for the whole job.
|
||||
|
||||
It builds on the generic MITM proxy described in [proxy.md](./proxy.md), reusing
|
||||
the same interceptor chain, malware analyzer, and certificate manager. The
|
||||
difference is the **lifecycle**: instead of PMG starting an ephemeral proxy
|
||||
around a single subprocess, the proxy is started once, advertises itself to the
|
||||
other `pmg proxy` commands, and serves many package manager processes until it
|
||||
is stopped.
|
||||
|
||||
## Default proxy mode vs. persistent proxy server
|
||||
|
||||
PMG's default proxy mode (see [proxy.md](./proxy.md)) wraps a single command.
|
||||
`pmg npm install` starts an ephemeral proxy, runs `npm` as a child with proxy
|
||||
env vars injected, then tears the proxy down. The persistent server decouples
|
||||
these steps.
|
||||
|
||||
| | Default proxy mode | Persistent proxy server |
|
||||
| --- | --- | --- |
|
||||
| Invocation | `pmg npm install` (wrapped) | bare `npm install` (no wrapper) |
|
||||
| Proxy lifetime | One subprocess | Until `pmg proxy stop` |
|
||||
| Who runs the PM | PMG (as a child) | The user / CI directly |
|
||||
| Ecosystems served | The one being run | All supported (npm + PyPI) |
|
||||
| Confirmation on malware | Interactive prompt (TTY) | Auto-block (non-interactive) |
|
||||
| Reporting | At subprocess exit | At `pmg proxy stop` |
|
||||
| Target | Local dev | CI/CD pipelines |
|
||||
|
||||
## How it works
|
||||
|
||||
The diagram below shows the order of events in a CI job. The `pmg proxy`
|
||||
commands run in separate workflow steps and coordinate through the running
|
||||
daemon.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant CI as CI Job
|
||||
participant Proxy as Proxy Daemon
|
||||
participant PM as Package Manager
|
||||
participant Cloud as SafeDep Cloud
|
||||
|
||||
CI->>Proxy: pmg proxy start --daemon
|
||||
Proxy-->>CI: ready (addr, ca path)
|
||||
CI->>CI: pmg proxy env (set HTTP_PROXY + CA vars)
|
||||
PM->>Proxy: package download (via HTTP_PROXY)
|
||||
Proxy->>Proxy: analyze package
|
||||
Proxy-->>PM: allow, or 403 block + record event
|
||||
Proxy->>Cloud: periodic sync of events (while serving)
|
||||
CI->>Proxy: pmg proxy stop --fail-on-violation
|
||||
Proxy->>Cloud: final flush of remaining events
|
||||
Proxy-->>CI: exit non-zero if anything was blocked
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
The persistent server targets non-interactive CI/CD. For local development use
|
||||
the default proxy mode (`pmg npm install`), which keeps the interactive malware
|
||||
confirmation prompt. The persistent server auto-blocks without prompting.
|
||||
|
||||
GitHub Actions (raw commands):
|
||||
|
||||
```yaml
|
||||
- run: pmg proxy start --daemon
|
||||
- run: pmg proxy env >> "$GITHUB_ENV"
|
||||
- run: npm ci
|
||||
- run: pmg proxy stop --fail-on-violation
|
||||
if: always()
|
||||
```
|
||||
|
||||
GitHub Actions (via the [safedep/pmg action](../action.yml) `server-mode`):
|
||||
|
||||
```yaml
|
||||
- uses: safedep/pmg@v1
|
||||
with:
|
||||
server-mode: true
|
||||
api-key: ${{ secrets.SAFEDEP_API_KEY }}
|
||||
tenant-id: ${{ secrets.SAFEDEP_TENANT_ID }}
|
||||
|
||||
- run: npm ci # intercepted automatically
|
||||
|
||||
- name: Enforce PMG policy
|
||||
if: always()
|
||||
run: pmg proxy stop --fail-on-violation
|
||||
```
|
||||
|
||||
In `server-mode`, the action starts the daemon and injects env vars instead of
|
||||
installing shims. Because composite actions cannot run an automatic cleanup
|
||||
step, the final `pmg proxy stop --fail-on-violation` step is required. It stops
|
||||
the proxy (the daemon flushes events to the cloud during shutdown) and fails the
|
||||
job on a block.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
pmg proxy start # start the proxy (foreground, or detached with --daemon)
|
||||
pmg proxy stop # stop the proxy and report the outcome
|
||||
pmg proxy env # print env vars that route package managers through it
|
||||
pmg proxy status # report whether a proxy is running
|
||||
```
|
||||
|
||||
Run `pmg proxy <command> --help` for flags. `--daemon` is **Unix only**: on
|
||||
Windows it returns a clear "not supported" error, and the foreground
|
||||
`pmg proxy start` still works. To run multiple independent proxies on one host,
|
||||
give each a distinct `--state` path and `--port`.
|
||||
|
||||
## Bind address
|
||||
|
||||
The proxy binds `127.0.0.1` on a random port by default, reachable only from the
|
||||
host (the right choice for CI and local use). Override with `--host`/`--port`, or
|
||||
the `proxy.server.listen_host`/`listen_port` config (flags take precedence).
|
||||
|
||||
Bind a non-loopback address (e.g. `--host 0.0.0.0`) **only** for a deliberately
|
||||
hosted deployment: it exposes the MITM proxy to the network, and every client
|
||||
routed through it has its HTTPS intercepted and must trust the PMG CA.
|
||||
|
||||
## Certificate trust
|
||||
|
||||
The proxy performs TLS MITM, so clients must trust its CA. Trust is delivered
|
||||
through **environment variables, not the OS trust store**. `pmg proxy env`
|
||||
always emits the cert-path variables pointing at the proxy's CA bundle:
|
||||
`NODE_EXTRA_CA_CERTS`, `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE`, `PIP_CERT`,
|
||||
`YARN_HTTPS_CA_FILE_PATH`. Package managers pick these up from the job
|
||||
environment and trust the proxy's CA, with no OS trust-store install required.
|
||||
|
||||
This is deliberate: whether a tool consults the OS trust store varies by tool,
|
||||
version, and config (npm/Node ignore it by default; modern pip can read it;
|
||||
`requests`/`certifi` ship their own bundle). The cert-path vars work across all
|
||||
of them, and are harmlessly ignored by tools that do read the OS store.
|
||||
|
||||
As a result `pmg setup cert install` is **not** needed for the persistent proxy.
|
||||
If a persisted CA from `pmg setup cert install` exists the proxy reuses it,
|
||||
otherwise it generates an ephemeral one. Either way `pmg proxy env` carries the
|
||||
trust. OS trust-store install (`pmg setup cert install --system`) is
|
||||
intentionally not used: it needs root (breaking container and locked-down
|
||||
runners), persistently installs a MITM-capable CA into the machine trust store,
|
||||
and still does not remove the need for the env vars.
|
||||
|
||||
Loopback addresses are always excluded from proxying via `NO_PROXY`
|
||||
(`localhost,127.0.0.1,::1`).
|
||||
|
||||
## Cloud event sync
|
||||
|
||||
When SafeDep Cloud is enabled, malware-block events must reach the cloud even on
|
||||
ephemeral CI runners that are destroyed immediately after the job. The daemon
|
||||
owns delivery: it records each blocked package to a durable local event log as
|
||||
it happens, syncs pending events to SafeDep Cloud periodically while serving, and
|
||||
flushes whatever remains on shutdown.
|
||||
|
||||
`pmg proxy stop` reports the recorded result (`Synced N event(s) to SafeDep
|
||||
Cloud`, or a `Cloud sync failed` line). A flush failure is surfaced but does not
|
||||
mask the fail-on-violation exit code.
|
||||
|
||||
## Fail on violation
|
||||
|
||||
By default `pmg proxy stop` just stops the proxy and exits `0`. Failing the CI
|
||||
job on a policy violation is opt-in via `--fail-on-violation`.
|
||||
|
||||
- It exits non-zero when any package was blocked.
|
||||
- It **fails closed**. If the daemon shut down without writing a verifiable
|
||||
final state (e.g. it crashed), `--fail-on-violation` also fails, because a
|
||||
security gate must not pass on an unverifiable run.
|
||||
|
||||
The package manager's own non-zero exit (from the `403` on a blocked download)
|
||||
is a separate signal. `--fail-on-violation` gives an authoritative gate from the
|
||||
proxy regardless of how the package manager reported the failure.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Unix-only daemon.** `--daemon` is not supported on Windows (foreground mode
|
||||
works).
|
||||
- **Non-interactive only.** There is no interactive confirmation; flagged
|
||||
packages are always auto-blocked. This is intentional for CI.
|
||||
- **Single proxy per state file.** Starting a second proxy that points at the
|
||||
same state file is refused while one is running.
|
||||
- **System-level trust enforcement is out of scope.** The server relies on env
|
||||
var propagation. Enforcing interception for `sudo`-scrubbed environments (e.g.
|
||||
via `iptables`) and system-wide install (`pmg setup install --system`) are
|
||||
tracked separately.
|
||||
|
||||
## References
|
||||
|
||||
- [proxy.md](./proxy.md) is the underlying generic MITM proxy server
|
||||
- [config.md](./config.md) is the configuration schema (cloud, proxy, cache dir)
|
||||
- [action.yml](../action.yml) is the PMG GitHub Action (`server-mode`)
|
||||
@@ -68,3 +68,7 @@ Legacy variables `PMG_PROXY_MODE` and `PMG_PROXY_INSTALL_ONLY` (for the old flat
|
||||
| `pip` | ✅ |
|
||||
| `uv` | ✅ |
|
||||
| `poetry` | ✅ |
|
||||
|
||||
## References
|
||||
|
||||
- [Persistent Proxy Mode](./persistent-proxy.md)
|
||||
|
||||
@@ -24,6 +24,11 @@ const (
|
||||
CertTrustStore = "CertTrustStore"
|
||||
UnsupportedPlatform = "UnsupportedPlatform"
|
||||
|
||||
// Proxy error codes. ProxyPolicyViolation is returned when the proxy blocked
|
||||
// one or more packages by policy (malware, dependency cooldown, or a denied
|
||||
// suspicious package) and the run was gated with --fail-on-violation.
|
||||
ProxyPolicyViolation = "ProxyPolicyViolation"
|
||||
|
||||
// Unknown mirrors the default code that dry/usefulerror returns for errors
|
||||
// created without an explicit code, so unset and explicitly-unknown errors
|
||||
// classify identically (e.g. the bug-report hint in ui.ErrorExit).
|
||||
|
||||
@@ -22,6 +22,19 @@ func MarkBackgroundSyncChild() {
|
||||
isBackgroundSyncChild = true
|
||||
}
|
||||
|
||||
// backgroundSyncSuppressed lets a command opt out of the detached auto-sync
|
||||
// spawn for its process. The proxy daemon uses this: it delivers events itself
|
||||
// (periodic sync + shutdown flush), so the detached child would be redundant
|
||||
// and, from `pmg proxy stop`, would route cloud traffic through the now-stopped
|
||||
// proxy.
|
||||
var backgroundSyncSuppressed bool
|
||||
|
||||
// SuppressBackgroundSync disables MaybeSpawnBackgroundSync for the current
|
||||
// process.
|
||||
func SuppressBackgroundSync() {
|
||||
backgroundSyncSuppressed = true
|
||||
}
|
||||
|
||||
// detachedSpawner forks a detached child running `name` with `args`. Pulled
|
||||
// behind a package var so tests can intercept without actually forking the
|
||||
// test binary into the background.
|
||||
@@ -41,7 +54,7 @@ func MaybeSpawnBackgroundSync(cfg *config.RuntimeConfig) {
|
||||
if cfg == nil {
|
||||
return
|
||||
}
|
||||
if isBackgroundSyncChild {
|
||||
if isBackgroundSyncChild || backgroundSyncSuppressed {
|
||||
return
|
||||
}
|
||||
if !cfg.Config.Cloud.Enabled || !cfg.Config.Cloud.AutoSync.Enabled {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/cloud"
|
||||
"github.com/safedep/dry/cloud/endpointsync"
|
||||
@@ -13,6 +14,60 @@ import (
|
||||
appVersion "github.com/safedep/pmg/internal/version"
|
||||
)
|
||||
|
||||
// ErrSyncInProgress is returned by DrainToCloud when another process already
|
||||
// holds the cloud sync lock.
|
||||
var ErrSyncInProgress = errors.New("another cloud sync is already in progress")
|
||||
|
||||
// DrainToCloud acquires the cross-process sync lock and drains pending audit
|
||||
// events from the WAL to SafeDep Cloud, returning the number synced. It records
|
||||
// the attempt (so a failing endpoint does not make every run retry). Callers
|
||||
// are responsible for gating on cloud.enabled and for surfacing errors;
|
||||
// ErrSyncInProgress is returned when the lock is held elsewhere.
|
||||
//
|
||||
// lockTimeout bounds acquiring the shared lock; syncTimeout bounds the drain.
|
||||
func DrainToCloud(ctx context.Context, cfg *config.RuntimeConfig, lockTimeout, syncTimeout time.Duration) (int, error) {
|
||||
lock := NewSyncLock(cfg.CloudSyncLockPath())
|
||||
|
||||
lockCtx, lockCancel := context.WithTimeout(ctx, lockTimeout)
|
||||
defer lockCancel()
|
||||
|
||||
locked, err := lock.TryLockContext(lockCtx, 250*time.Millisecond)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("acquire cloud sync lock: %w", err)
|
||||
}
|
||||
if !locked {
|
||||
return 0, ErrSyncInProgress
|
||||
}
|
||||
defer func() {
|
||||
if uerr := lock.Unlock(); uerr != nil {
|
||||
log.Warnf("failed to release cloud sync lock: %v", uerr)
|
||||
}
|
||||
}()
|
||||
|
||||
bundle, err := NewSyncClientBundle(cfg)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("init cloud sync client: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if cerr := bundle.Close(); cerr != nil {
|
||||
log.Warnf("failed to close cloud sync client: %v", cerr)
|
||||
}
|
||||
}()
|
||||
|
||||
syncCtx, cancel := context.WithTimeout(ctx, syncTimeout)
|
||||
defer cancel()
|
||||
|
||||
synced, syncErr := bundle.Sync(syncCtx)
|
||||
|
||||
// Record the attempt on every outcome so a stuck endpoint does not make the
|
||||
// background auto-sync refire on the next invocation.
|
||||
if werr := WriteLastSyncAttempt(cfg.CloudSyncLastRunPath()); werr != nil {
|
||||
log.Warnf("failed to update cloud sync lastrun: %v", werr)
|
||||
}
|
||||
|
||||
return synced, syncErr
|
||||
}
|
||||
|
||||
// SyncClientBundle holds a SyncClient and its underlying cloud client.
|
||||
// Callers must call Close() when done.
|
||||
type SyncClientBundle struct {
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
package flows
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/safedep/dry/localdb"
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/pmg/analyzer"
|
||||
"github.com/safedep/pmg/analyzer/malysiscache"
|
||||
"github.com/safedep/pmg/config"
|
||||
)
|
||||
|
||||
// BuildMalysisAnalyzer constructs the malysis analyzer with its optional
|
||||
// analyzer-specific persistent cache. The caller owns the shared localdb manager
|
||||
// lifecycle. Cache failures degrade to an uncached analyzer and never abort.
|
||||
func BuildMalysisAnalyzer(ctx context.Context, cfg *config.RuntimeConfig, db localdb.Manager) (analyzer.PackageVersionAnalyzer, error) {
|
||||
return analyzer.NewMalysisAnalyzer(analyzer.MalysisQueryAnalyzerConfig{
|
||||
Cache: buildMalysisCache(ctx, db, cfg.Config.AnalysisCache.Malysis),
|
||||
})
|
||||
}
|
||||
|
||||
func buildMalysisCache(ctx context.Context, db localdb.Manager, cacheCfg config.MalysisCacheConfig) analyzer.MalysisCache {
|
||||
if db == nil || !cacheCfg.Enabled || cacheCfg.TTL <= 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
store, err := db.Store(ctx, malysiscache.Descriptor())
|
||||
if err != nil {
|
||||
log.Warnf("analysis cache unavailable, continuing without it: %v", err)
|
||||
return nil
|
||||
}
|
||||
|
||||
return malysiscache.New(store, cacheCfg)
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
package flows
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/localdb"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
type fakeLocalDBManager struct {
|
||||
storeCalls int
|
||||
storeErr error
|
||||
}
|
||||
|
||||
func (f *fakeLocalDBManager) Store(context.Context, localdb.Descriptor) (*localdb.Store, error) {
|
||||
f.storeCalls++
|
||||
return nil, f.storeErr
|
||||
}
|
||||
|
||||
func (f *fakeLocalDBManager) Close() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestBuildMalysisCacheDisabledDoesNotOpenLocalDB(t *testing.T) {
|
||||
db := &fakeLocalDBManager{}
|
||||
|
||||
cache := buildMalysisCache(context.Background(), db, config.MalysisCacheConfig{
|
||||
Enabled: false,
|
||||
TTL: time.Hour,
|
||||
})
|
||||
|
||||
assert.Nil(t, cache)
|
||||
assert.Equal(t, 0, db.storeCalls)
|
||||
}
|
||||
|
||||
func TestBuildMalysisCacheStoreErrorDegradesToNil(t *testing.T) {
|
||||
db := &fakeLocalDBManager{storeErr: errors.New("db unavailable")}
|
||||
|
||||
cache := buildMalysisCache(context.Background(), db, config.MalysisCacheConfig{
|
||||
Enabled: true,
|
||||
TTL: time.Hour,
|
||||
})
|
||||
|
||||
assert.Nil(t, cache)
|
||||
assert.Equal(t, 1, db.storeCalls)
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package flows
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/pmg/proxy/certmanager"
|
||||
"github.com/safedep/pmg/truststore"
|
||||
)
|
||||
|
||||
// SetupCACertificate loads the persisted CA from configDir or generates an
|
||||
// ephemeral one, merges it with the system CA bundle, and writes the result
|
||||
// to outputPath. Returns the certificate, whether it was ephemeral, and any
|
||||
// error. The caller is responsible for cleaning up outputPath when ephemeral.
|
||||
func SetupCACertificate(configDir, outputPath string) (*certmanager.Certificate, bool, error) {
|
||||
caCert, persisted := loadPersistedCA(configDir)
|
||||
ephemeral := !persisted
|
||||
|
||||
if persisted {
|
||||
log.Debugf("Using persisted CA certificate from %s", configDir)
|
||||
warnIfCANotTrusted()
|
||||
} else {
|
||||
log.Debugf("Generating ephemeral CA certificate for proxy MITM")
|
||||
generated, err := certmanager.GenerateCA(certmanager.DefaultCertManagerConfig())
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("generate CA certificate: %w", err)
|
||||
}
|
||||
caCert = generated
|
||||
}
|
||||
|
||||
merged := certmanager.MergeWithSystemCA(caCert.Certificate)
|
||||
if err := os.MkdirAll(filepath.Dir(outputPath), 0o700); err != nil {
|
||||
return nil, false, fmt.Errorf("create CA certificate directory: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(outputPath, merged, 0o600); err != nil {
|
||||
return nil, false, fmt.Errorf("write CA certificate to %s: %w", outputPath, err)
|
||||
}
|
||||
|
||||
log.Debugf("CA certificate written to %s", outputPath)
|
||||
return caCert, ephemeral, nil
|
||||
}
|
||||
|
||||
func loadPersistedCA(dir string) (*certmanager.Certificate, bool) {
|
||||
caCert, err := certmanager.LoadCA(dir)
|
||||
if err != nil {
|
||||
if !os.IsNotExist(err) {
|
||||
log.Warnf("Failed to load persisted CA, using ephemeral: %v", err)
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
if caCert.IsExpired(time.Hour) {
|
||||
log.Warnf("Persisted CA is expired; using ephemeral. Re-run `pmg setup cert install`")
|
||||
return nil, false
|
||||
}
|
||||
|
||||
return caCert, true
|
||||
}
|
||||
|
||||
func warnIfCANotTrusted() {
|
||||
user, system, err := truststore.Status(certmanager.CACommonName)
|
||||
if err != nil {
|
||||
log.Debugf("Could not determine CA trust status: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if !user && !system {
|
||||
log.Warnf("Persisted CA is not trusted in the OS store; native tools may reject TLS. Run `pmg setup cert install`.")
|
||||
}
|
||||
}
|
||||
+13
-126
@@ -2,26 +2,22 @@ package flows
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/localdb"
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/pmg/analyzer"
|
||||
"github.com/safedep/pmg/analyzer/malysiscache"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/guard"
|
||||
"github.com/safedep/pmg/internal/audit"
|
||||
"github.com/safedep/pmg/internal/localstore"
|
||||
"github.com/safedep/pmg/internal/runner"
|
||||
"github.com/safedep/pmg/internal/ui"
|
||||
"github.com/safedep/pmg/packagemanager"
|
||||
"github.com/safedep/pmg/proxy"
|
||||
"github.com/safedep/pmg/proxy/certmanager"
|
||||
"github.com/safedep/pmg/proxy/interceptors"
|
||||
"github.com/safedep/pmg/truststore"
|
||||
)
|
||||
|
||||
type proxyFlow struct {
|
||||
@@ -137,31 +133,16 @@ func (f *proxyFlow) Run(ctx context.Context, args []string, parsedCmd *packagema
|
||||
return fmt.Errorf("failed to create certificate manager: %w", err)
|
||||
}
|
||||
|
||||
// Optional persistent analysis cache. Disposable: any failure degrades to
|
||||
// running uncached, never blocks the install.
|
||||
var malysisCache analyzer.MalysisCache
|
||||
cacheCfg := cfg.Config.AnalysisCache.Malysis
|
||||
if cacheCfg.Enabled && cacheCfg.TTL > 0 {
|
||||
mgr := localdb.New(localdb.Config{
|
||||
Dir: cfg.LocalDBDir(),
|
||||
FileName: cfg.LocalDBFileName(),
|
||||
})
|
||||
defer func() {
|
||||
if cerr := mgr.Close(); cerr != nil {
|
||||
log.Warnf("failed to close localdb: %v", cerr)
|
||||
}
|
||||
}()
|
||||
|
||||
store, serr := mgr.Store(ctx, malysiscache.Descriptor())
|
||||
if serr != nil {
|
||||
log.Warnf("analysis cache unavailable, continuing without it: %v", serr)
|
||||
} else {
|
||||
malysisCache = malysiscache.New(store, cacheCfg)
|
||||
localDB := localstore.NewManager(cfg)
|
||||
defer func() {
|
||||
if cerr := localDB.Close(); cerr != nil {
|
||||
log.Warnf("failed to close localdb: %v", cerr)
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
// Create analyzer
|
||||
malysisAnalyzer, err := f.createAnalyzer(malysisCache)
|
||||
// Analyzer with an optional persistent cache. Cache failures degrade to
|
||||
// running uncached and never block the install.
|
||||
malysisAnalyzer, err := BuildMalysisAnalyzer(ctx, cfg, localDB)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create analyzer: %w", err)
|
||||
}
|
||||
@@ -231,7 +212,7 @@ func (f *proxyFlow) Run(ctx context.Context, args []string, parsedCmd *packagema
|
||||
PackageManagerName: f.pm.Name(),
|
||||
DryRun: cfg.DryRun,
|
||||
Mode: runner.ExecutionModeAuto,
|
||||
EnvOverrides: f.setupEnvForProxy(proxyAddr, caCertPath),
|
||||
EnvOverrides: packagemanager.EnvVarForProxy(proxyAddr, caCertPath),
|
||||
DirectEnvOverrides: ciEnvOverride(),
|
||||
BeforeDirectRun: func() error {
|
||||
log.Debugf("Executing proxy for non interactive TTY")
|
||||
@@ -329,69 +310,11 @@ func handleExecutionResultError(err error) error {
|
||||
return fmt.Errorf("failed to execute command: %w", err)
|
||||
}
|
||||
|
||||
// setupCACertificate prefers the persisted CA (created by `pmg setup cert install`)
|
||||
// so the proxy signs leaves with the same CA that is in the OS trust store. When no
|
||||
// persisted CA exists it falls back to an ephemeral per-run CA, preserving original
|
||||
// behavior. The temp file always carries the pure CA merged with the system bundle so
|
||||
// env-var trust injection works on every platform.
|
||||
func (f *proxyFlow) setupCACertificate() (*certmanager.Certificate, string, error) {
|
||||
dir := config.Get().ConfigDir()
|
||||
|
||||
caCert, persisted := loadPersistedCA(dir)
|
||||
if persisted {
|
||||
log.Debugf("Using persisted CA certificate from %s", dir)
|
||||
warnIfCANotTrusted()
|
||||
} else {
|
||||
log.Debugf("Generating ephemeral CA certificate for proxy MITM")
|
||||
generated, err := certmanager.GenerateCA(certmanager.DefaultCertManagerConfig())
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("failed to generate CA certificate: %w", err)
|
||||
}
|
||||
caCert = generated
|
||||
}
|
||||
|
||||
mergedPEM := certmanager.MergeWithSystemCA(caCert.Certificate)
|
||||
|
||||
tempDir := os.TempDir()
|
||||
caCertPath := filepath.Join(tempDir, fmt.Sprintf("pmg-ca-cert-%d.pem", os.Getpid()))
|
||||
if err := os.WriteFile(caCertPath, mergedPEM, 0o600); err != nil {
|
||||
return nil, "", fmt.Errorf("failed to write CA certificate to %s: %w", caCertPath, err)
|
||||
}
|
||||
|
||||
log.Debugf("CA certificate written to %s", caCertPath)
|
||||
return caCert, caCertPath, nil
|
||||
}
|
||||
|
||||
// loadPersistedCA returns the on-disk CA when present and not expired.
|
||||
func loadPersistedCA(dir string) (*certmanager.Certificate, bool) {
|
||||
caCert, err := certmanager.LoadCA(dir)
|
||||
if err != nil {
|
||||
if !errors.Is(err, os.ErrNotExist) {
|
||||
log.Warnf("Failed to load persisted CA, using ephemeral: %v", err)
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
if caCert.IsExpired(time.Hour) {
|
||||
log.Warnf("Persisted CA is expired; using ephemeral. Re-run `pmg setup cert install`")
|
||||
return nil, false
|
||||
}
|
||||
|
||||
return caCert, true
|
||||
}
|
||||
|
||||
// warnIfCANotTrusted logs a hint when the persisted CA is not in any OS store,
|
||||
// which matters for native tools (e.g. Go on macOS/Windows) that ignore the
|
||||
// injected env vars. Best-effort; never blocks the run.
|
||||
func warnIfCANotTrusted() {
|
||||
user, system, err := truststore.Status(certmanager.CACommonName)
|
||||
if err != nil {
|
||||
log.Debugf("Could not determine CA trust status: %v", err)
|
||||
return
|
||||
}
|
||||
if !user && !system {
|
||||
log.Warnf("Persisted CA is not trusted in the OS store; native tools may reject TLS. Run `pmg setup cert install`.")
|
||||
}
|
||||
outputPath := certmanager.EphemeralProxyCABundlePath()
|
||||
cert, _, err := SetupCACertificate(dir, outputPath)
|
||||
return cert, outputPath, err
|
||||
}
|
||||
|
||||
// createCertificateManager creates a certificate manager with the given CA certificate
|
||||
@@ -405,12 +328,6 @@ func (f *proxyFlow) createCertificateManager(caCert *certmanager.Certificate) (c
|
||||
return certMgr, nil
|
||||
}
|
||||
|
||||
// createAnalyzer creates the malysis query analyzer
|
||||
func (f *proxyFlow) createAnalyzer(cache analyzer.MalysisCache) (analyzer.PackageVersionAnalyzer, error) {
|
||||
log.Debugf("Creating malysis query analyzer")
|
||||
return analyzer.NewMalysisAnalyzer(analyzer.MalysisQueryAnalyzerConfig{Cache: cache})
|
||||
}
|
||||
|
||||
// createAndStartProxyServer creates and starts the proxy server with the given interceptor
|
||||
func (f *proxyFlow) createAndStartProxyServer(
|
||||
certMgr certmanager.CertificateManager,
|
||||
@@ -446,33 +363,3 @@ func ciEnvOverride() []string {
|
||||
}
|
||||
return []string{"CI=true"}
|
||||
}
|
||||
|
||||
func (f *proxyFlow) setupEnvForProxy(proxyAddr, caCertPath string) []string {
|
||||
proxyURL := fmt.Sprintf("http://%s", proxyAddr)
|
||||
|
||||
// IPv6 loopback uses the bare ::1: the bracketed [::1] is URL syntax that
|
||||
// crashes Python's urllib/httpx (#339). Trade-off: Node's NODE_USE_ENV_PROXY
|
||||
// (undici) only bypasses the bracketed form, so a literal http://[::1] from
|
||||
// Node still gets proxied. localhost/127.0.0.1 cover the common cases; the
|
||||
// IPv6 literal is a rare edge we accept since NO_PROXY can't be set per-client.
|
||||
noProxyList := "localhost,127.0.0.1,::1"
|
||||
|
||||
return []string{
|
||||
"NODE_USE_ENV_PROXY=1",
|
||||
fmt.Sprintf("HTTP_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("HTTPS_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("NO_PROXY=%s", noProxyList),
|
||||
fmt.Sprintf("NODE_EXTRA_CA_CERTS=%s", caCertPath),
|
||||
fmt.Sprintf("YARN_HTTP_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("YARN_HTTPS_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("YARN_HTTPS_CA_FILE_PATH=%s", caCertPath),
|
||||
fmt.Sprintf("http_proxy=%s", proxyURL),
|
||||
fmt.Sprintf("https_proxy=%s", proxyURL),
|
||||
fmt.Sprintf("no_proxy=%s", noProxyList),
|
||||
fmt.Sprintf("SSL_CERT_FILE=%s", caCertPath),
|
||||
fmt.Sprintf("REQUESTS_CA_BUNDLE=%s", caCertPath),
|
||||
fmt.Sprintf("PIP_CERT=%s", caCertPath),
|
||||
fmt.Sprintf("PIP_PROXY=%s", proxyURL),
|
||||
"PIP_RETRIES=0",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ package flows
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/safedep/pmg/proxy/certmanager"
|
||||
@@ -41,3 +42,16 @@ func TestSetupCACertificateWritesMergedTempFile(t *testing.T) {
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, caCert.Certificate, merged[:len(caCert.Certificate)])
|
||||
}
|
||||
|
||||
func TestSetupCACertificateCreatesMissingOutputDir(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
configDir := filepath.Join(root, "config")
|
||||
outputPath := filepath.Join(configDir, "proxy-ca.pem")
|
||||
|
||||
caCert, ephemeral, err := SetupCACertificate(configDir, outputPath)
|
||||
require.NoError(t, err)
|
||||
|
||||
require.NotNil(t, caCert)
|
||||
assert.True(t, ephemeral)
|
||||
assert.FileExists(t, outputPath)
|
||||
}
|
||||
|
||||
@@ -2,61 +2,12 @@ package flows
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func envToMap(env []string) map[string]string {
|
||||
m := make(map[string]string, len(env))
|
||||
for _, e := range env {
|
||||
k, v, ok := strings.Cut(e, "=")
|
||||
if ok {
|
||||
m[k] = v
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// TestSetupEnvForProxyConfiguresYarn proves the root cause of #319: yarn Berry
|
||||
// (yarn 2+) ignores the standard HTTP_PROXY/HTTPS_PROXY env vars and only honors
|
||||
// its own config, which can be set via YARN_* env overrides. Without these,
|
||||
// yarn bypasses the MITM proxy entirely and no packages are analyzed.
|
||||
func TestSetupEnvForProxyConfiguresYarn(t *testing.T) {
|
||||
f := &proxyFlow{}
|
||||
const proxyAddr = "127.0.0.1:54321"
|
||||
const caCertPath = "/tmp/pmg-ca-cert.pem"
|
||||
|
||||
env := envToMap(f.setupEnvForProxy(proxyAddr, caCertPath))
|
||||
|
||||
proxyURL := "http://" + proxyAddr
|
||||
|
||||
assert.Equal(t, proxyURL, env["YARN_HTTP_PROXY"],
|
||||
"yarn ignores HTTP_PROXY; YARN_HTTP_PROXY is required to route yarn through the proxy")
|
||||
assert.Equal(t, proxyURL, env["YARN_HTTPS_PROXY"],
|
||||
"yarn ignores HTTPS_PROXY; YARN_HTTPS_PROXY is required to route yarn through the proxy")
|
||||
assert.Equal(t, caCertPath, env["YARN_HTTPS_CA_FILE_PATH"],
|
||||
"yarn ignores NODE_EXTRA_CA_CERTS; YARN_HTTPS_CA_FILE_PATH is required to trust the MITM CA")
|
||||
}
|
||||
|
||||
// TestSetupEnvForProxyNoProxyIPv6 proves the fix for #339: the IPv6 loopback in
|
||||
// NO_PROXY must be bare (::1), not bracketed ([::1]). Brackets are URL syntax,
|
||||
// not NO_PROXY syntax, and Python's urllib/httpx crashes parsing them with
|
||||
// "Invalid port: ':1]'".
|
||||
func TestSetupEnvForProxyNoProxyIPv6(t *testing.T) {
|
||||
f := &proxyFlow{}
|
||||
env := envToMap(f.setupEnvForProxy("127.0.0.1:54321", "/tmp/pmg-ca-cert.pem"))
|
||||
|
||||
for _, key := range []string{"NO_PROXY", "no_proxy"} {
|
||||
assert.Equal(t, "localhost,127.0.0.1,::1", env[key],
|
||||
"%s must use bare ::1; bracketed [::1] is invalid NO_PROXY syntax and crashes httpx", key)
|
||||
assert.NotContains(t, env[key], "[::1]",
|
||||
"%s must not contain bracketed IPv6 loopback", key)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCIEnvOverride proves the fix for #335: pmg forces CI=true for
|
||||
// non-interactive runs but must not clobber a CI value the user set
|
||||
// explicitly (e.g. CI=false on a build server).
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
package localstore
|
||||
|
||||
import (
|
||||
"github.com/safedep/dry/localdb"
|
||||
"github.com/safedep/pmg/config"
|
||||
)
|
||||
|
||||
func NewManager(cfg *config.RuntimeConfig) localdb.Manager {
|
||||
return localdb.New(localdb.Config{
|
||||
Dir: cfg.LocalDBDir(),
|
||||
FileName: cfg.LocalDBFileName(),
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
//go:build !windows
|
||||
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Daemonize re-execs the current binary with args (which must run the proxy in
|
||||
// foreground mode), detached into its own session (Setsid), with child stdio
|
||||
// redirected to cfg.LogPath. It waits up to cfg.ReadyTimeout for the child to
|
||||
// write the state file and returns the running state. The caller owns the log
|
||||
// path (its parent directory must exist); Daemonize fails if it cannot be
|
||||
// opened.
|
||||
func Daemonize(cfg ProxyDaemonConfig, statePath, exe string, args []string) (State, error) {
|
||||
logFile, err := os.OpenFile(cfg.LogPath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600)
|
||||
if err != nil {
|
||||
return State{}, fmt.Errorf("open daemon log %s: %w", cfg.LogPath, err)
|
||||
}
|
||||
|
||||
defer func() { _ = logFile.Close() }()
|
||||
|
||||
cmd := exec.Command(exe, args...)
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Stdin = nil
|
||||
cmd.Stdout = logFile
|
||||
cmd.Stderr = logFile
|
||||
|
||||
if err := cmd.Start(); err != nil {
|
||||
return State{}, fmt.Errorf("start daemon: %w", err)
|
||||
}
|
||||
if err := cmd.Process.Release(); err != nil {
|
||||
return State{}, fmt.Errorf("release daemon process: %w", err)
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(cfg.ReadyTimeout)
|
||||
for time.Now().Before(deadline) {
|
||||
if state, rerr := readState(statePath); rerr == nil && state.IsRunning() {
|
||||
return state, nil
|
||||
}
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
}
|
||||
|
||||
return State{}, fmt.Errorf("daemon did not become ready within %s; see %s", cfg.ReadyTimeout, cfg.LogPath)
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
//go:build windows
|
||||
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"github.com/safedep/dry/usefulerror"
|
||||
"github.com/safedep/pmg/errcodes"
|
||||
)
|
||||
|
||||
func Daemonize(_ ProxyDaemonConfig, _, _ string, _ []string) (State, error) {
|
||||
return State{}, usefulerror.NewUsefulError().
|
||||
WithCode(errcodes.UnsupportedPlatform).
|
||||
WithMsg("pmg proxy start --daemon is not supported on Windows").
|
||||
WithHelp("Run 'pmg proxy start' in the foreground instead")
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
// Package proxyserver implements the persistent proxy flow: running PMG as a
|
||||
// long-lived MITM proxy server that many package-manager processes route through
|
||||
// via environment variables, for non-interactive CI/CD use.
|
||||
//
|
||||
// It is a flow, not a proxy implementation. The actual MITM proxy lives in the
|
||||
// `proxy` package; proxyserver wires that proxy together with the analyzer,
|
||||
// interceptors, audit pipeline, and cloud sync, and manages the daemon lifecycle
|
||||
// (start/stop/env/status) coordinated through an on-disk state file.
|
||||
//
|
||||
// # Boundary with internal/flows
|
||||
//
|
||||
// internal/flows owns the per-command flow, where PMG wraps a single package
|
||||
// manager invocation (spawns it as a child, injects env, prompts on confirmation,
|
||||
// reports at exit). proxyserver owns the persistent flow, where the proxy
|
||||
// outlives any single invocation and is driven by separate `pmg proxy` commands
|
||||
// across processes. The two share lower-level building blocks (CA setup in
|
||||
// flows.SetupCACertificate, proxy env vars in packagemanager.EnvVarForProxy,
|
||||
// cloud sync in audit.DrainToCloud) but differ in lifecycle and process model,
|
||||
// so they are kept as separate flows rather than one. Some assembly (analyzer +
|
||||
// cache wiring) is intentionally duplicated for the MVP rather than prematurely
|
||||
// unified.
|
||||
package proxyserver
|
||||
@@ -0,0 +1,18 @@
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/safedep/pmg/packagemanager"
|
||||
)
|
||||
|
||||
// EnvVars returns the proxy environment variables (KEY=VALUE lines) for the
|
||||
// running proxy described by the state file at statePath.
|
||||
func EnvVars(statePath string) ([]string, error) {
|
||||
state, err := readState(statePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("proxy not running, start with 'pmg proxy start' first: %w", err)
|
||||
}
|
||||
|
||||
return packagemanager.EnvVarForProxy(state.Addr, state.CACertPath), nil
|
||||
}
|
||||
@@ -0,0 +1,276 @@
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/log"
|
||||
"github.com/safedep/pmg/config"
|
||||
"github.com/safedep/pmg/internal/audit"
|
||||
"github.com/safedep/pmg/internal/flows"
|
||||
"github.com/safedep/pmg/internal/localstore"
|
||||
pmgproxy "github.com/safedep/pmg/proxy"
|
||||
"github.com/safedep/pmg/proxy/certmanager"
|
||||
"github.com/safedep/pmg/proxy/interceptors"
|
||||
)
|
||||
|
||||
const (
|
||||
serverStopTimeout = 5 * time.Second
|
||||
|
||||
// Periodic cloud sync runs while the daemon is alive so most audit events are
|
||||
// delivered during the run and the shutdown flush stays small. A tick that
|
||||
// cannot get the sync lock quickly is skipped (the next tick retries).
|
||||
cloudSyncInterval = 15 * time.Second
|
||||
cloudSyncTickLockWait = 5 * time.Second
|
||||
cloudSyncTickTimeout = 30 * time.Second
|
||||
|
||||
// Final flush at shutdown, when the daemon drains whatever the ticker left.
|
||||
cloudFlushLockWait = 30 * time.Second
|
||||
cloudFlushTimeout = 2 * time.Minute
|
||||
|
||||
// daemonShutdownBudget is the worst-case time the daemon needs to shut down:
|
||||
// drain in-flight requests, wait for an in-flight periodic tick to finish,
|
||||
// then the final flush. `pmg proxy stop` waits at least this long for the
|
||||
// daemon to exit; see stopWaitTimeout.
|
||||
daemonShutdownBudget = serverStopTimeout +
|
||||
cloudSyncTickLockWait + cloudSyncTickTimeout +
|
||||
cloudFlushLockWait + cloudFlushTimeout
|
||||
)
|
||||
|
||||
// DefaultDaemonReadyTimeout is how long the parent waits for the daemon to
|
||||
// become ready before giving up, when ProxyDaemonConfig.ReadyTimeout is unset.
|
||||
const DefaultDaemonReadyTimeout = 10 * time.Second
|
||||
|
||||
// ProxyDaemonConfig carries the daemon-launch parameters the caller decides, so
|
||||
// daemonization stays free of config and path-policy concerns.
|
||||
type ProxyDaemonConfig struct {
|
||||
// LogPath is the file the detached daemon's stdout/stderr is redirected to.
|
||||
// The caller owns this path (its parent directory must exist).
|
||||
LogPath string
|
||||
// ReadyTimeout bounds how long to wait for the daemon to write its state
|
||||
// file and become live.
|
||||
ReadyTimeout time.Duration
|
||||
}
|
||||
|
||||
// Run starts the persistent proxy server in the foreground and blocks until it
|
||||
// receives SIGINT/SIGTERM. It writes the state file on startup, auto-blocks
|
||||
// suspicious packages, and records the final blocked count on shutdown.
|
||||
func Run(ctx context.Context, cfg *config.RuntimeConfig, statePath, host string, port int) error {
|
||||
if existing, err := readState(statePath); err == nil && existing.IsRunning() {
|
||||
return fmt.Errorf("proxy already running (pid %d, addr %s) — run 'pmg proxy stop' first", existing.PID, existing.Addr)
|
||||
}
|
||||
|
||||
caCertPath := certmanager.ProxyCABundlePath(cfg.ConfigDir())
|
||||
caCert, _, err := flows.SetupCACertificate(cfg.ConfigDir(), caCertPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("setup CA certificate: %w", err)
|
||||
}
|
||||
|
||||
certMgr, err := certmanager.NewCertificateManagerWithCA(caCert, certmanager.DefaultCertManagerConfig())
|
||||
if err != nil {
|
||||
return fmt.Errorf("create certificate manager: %w", err)
|
||||
}
|
||||
|
||||
localDB := localstore.NewManager(cfg)
|
||||
defer func() {
|
||||
if cerr := localDB.Close(); cerr != nil {
|
||||
log.Warnf("failed to close localdb: %v", cerr)
|
||||
}
|
||||
}()
|
||||
|
||||
malysisAnalyzer, err := flows.BuildMalysisAnalyzer(ctx, cfg, localDB)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create analyzer: %w", err)
|
||||
}
|
||||
|
||||
cache := interceptors.NewInMemoryAnalysisCache()
|
||||
stats := interceptors.NewAnalysisStatsCollector()
|
||||
confirmationChan := make(chan *interceptors.ConfirmationRequest, 100)
|
||||
go autoBlockConfirmations(confirmationChan)
|
||||
|
||||
factory := interceptors.NewInterceptorFactory(
|
||||
malysisAnalyzer, cache, stats, confirmationChan, interceptors.InterceptorContext{},
|
||||
)
|
||||
|
||||
var interceptorList []pmgproxy.Interceptor
|
||||
for _, eco := range interceptors.SupportedEcosystems() {
|
||||
i, ferr := factory.CreateInterceptor(eco)
|
||||
if ferr != nil {
|
||||
return fmt.Errorf("create interceptor for %s: %w", eco.String(), ferr)
|
||||
}
|
||||
interceptorList = append(interceptorList, i)
|
||||
}
|
||||
interceptorList = append(interceptorList, interceptors.NewAuditLoggerInterceptor())
|
||||
|
||||
proxyConfig := pmgproxy.DefaultProxyConfig()
|
||||
proxyConfig.ListenAddr = listenAddr(host, port)
|
||||
proxyConfig.CertManager = certMgr
|
||||
proxyConfig.Interceptors = interceptorList
|
||||
|
||||
server, err := pmgproxy.NewProxyServer(proxyConfig)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create proxy server: %w", err)
|
||||
}
|
||||
|
||||
if err := server.Start(); err != nil {
|
||||
return fmt.Errorf("start proxy server: %w", err)
|
||||
}
|
||||
|
||||
state := State{
|
||||
PID: os.Getpid(),
|
||||
Addr: server.Address(),
|
||||
CACertPath: caCertPath,
|
||||
}
|
||||
if err := writeState(statePath, state); err != nil {
|
||||
stopCtx, cancel := context.WithTimeout(context.Background(), serverStopTimeout)
|
||||
defer cancel()
|
||||
if serr := server.Stop(stopCtx); serr != nil {
|
||||
log.Warnf("failed to stop proxy after state write failure: %v", serr)
|
||||
}
|
||||
return fmt.Errorf("write proxy state: %w", err)
|
||||
}
|
||||
|
||||
log.Infof("PMG persistent proxy running on %s (pid %d)", state.Addr, state.PID)
|
||||
if _, err := fmt.Fprintf(os.Stderr, "PMG proxy running on %s\nRun: export $(pmg proxy env | xargs) # or: pmg proxy env >> \"$GITHUB_ENV\"\n", state.Addr); err != nil {
|
||||
log.Warnf("failed to write startup message: %v", err)
|
||||
}
|
||||
|
||||
// Periodically flush events to the cloud while serving so the shutdown flush
|
||||
// stays small. See startCloudSyncLoop for the stop-function contract.
|
||||
stopSyncLoop := startCloudSyncLoop(cfg)
|
||||
|
||||
sigCh := make(chan os.Signal, 1)
|
||||
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
|
||||
<-sigCh
|
||||
|
||||
// Drain in-flight requests before closing the confirmation channel, so no
|
||||
// request handler can send on a closed channel (panic) during shutdown.
|
||||
stopCtx, cancel := context.WithTimeout(context.Background(), serverStopTimeout)
|
||||
defer cancel()
|
||||
stopErr := server.Stop(stopCtx)
|
||||
|
||||
close(confirmationChan)
|
||||
|
||||
// Count is read after drain so a package analyzed at shutdown is not missed.
|
||||
// Persist it BEFORE the (possibly slow) cloud flush so the blocked count
|
||||
// survives even if the flush hangs or the daemon is killed mid-flush, which
|
||||
// keeps `stop --fail-on-violation` correct in those cases.
|
||||
state.BlockedCount = stats.GetStats().BlockedCount
|
||||
if werr := writeState(statePath, state); werr != nil {
|
||||
log.Warnf("failed to write final proxy state: %v", werr)
|
||||
}
|
||||
|
||||
// Halt the periodic sync (waits for any in-flight drain) before the final
|
||||
// flush, so the two never hold the sync lock at once.
|
||||
periodicSynced := stopSyncLoop()
|
||||
|
||||
if cs := cloudFlush(cfg, periodicSynced); cs != nil {
|
||||
state.CloudSync = cs
|
||||
if werr := writeState(statePath, state); werr != nil {
|
||||
log.Warnf("failed to write final proxy state: %v", werr)
|
||||
}
|
||||
}
|
||||
|
||||
return stopErr
|
||||
}
|
||||
|
||||
// cloudFlush drains whatever the periodic sync left and returns the outcome
|
||||
// (total delivered, including periodicSynced). Returns nil when automatic cloud
|
||||
// delivery is off (cloud or auto-sync disabled) — same gate as the periodic
|
||||
// ticker, so auto_sync consistently controls all daemon-driven cloud delivery.
|
||||
// The daemon does this itself rather than `pmg proxy stop` because, unlike stop,
|
||||
// it has no proxy env vars and so dials SafeDep directly instead of routing
|
||||
// through the proxy that is now shutting down. Uses a fresh context since the
|
||||
// caller's may already be cancelled at shutdown.
|
||||
func cloudFlush(cfg *config.RuntimeConfig, periodicSynced int) *CloudSyncResult {
|
||||
if !cfg.Config.Cloud.Enabled || !cfg.Config.Cloud.AutoSync.Enabled {
|
||||
return nil
|
||||
}
|
||||
|
||||
synced, err := audit.DrainToCloud(context.Background(), cfg, cloudFlushLockWait, cloudFlushTimeout)
|
||||
res := &CloudSyncResult{Synced: periodicSynced + synced}
|
||||
if err != nil {
|
||||
res.Error = err.Error()
|
||||
log.Warnf("cloud event flush failed: %v", err)
|
||||
} else {
|
||||
log.Infof("Flushed %d events to SafeDep Cloud (%d during the run)", res.Synced, periodicSynced)
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
// startCloudSyncLoop periodically drains pending audit events to SafeDep Cloud
|
||||
// while the daemon runs. It returns a stop function that halts the ticker, waits
|
||||
// for any in-flight drain to finish, and returns the running total of events
|
||||
// delivered. The stop function must be called before the daemon's final flush so
|
||||
// the two never hold the sync lock at once. A no-op when cloud sync or auto-sync
|
||||
// is disabled; the daemon's automatic cloud delivery (periodic ticker and
|
||||
// shutdown flush alike) honors the auto_sync flag.
|
||||
func startCloudSyncLoop(cfg *config.RuntimeConfig) func() int {
|
||||
if !cfg.Config.Cloud.Enabled || !cfg.Config.Cloud.AutoSync.Enabled {
|
||||
return func() int { return 0 }
|
||||
}
|
||||
|
||||
stop := make(chan struct{})
|
||||
done := make(chan struct{})
|
||||
var total int // written only by the goroutine; read after <-done (happens-before)
|
||||
|
||||
go func() {
|
||||
defer close(done)
|
||||
ticker := time.NewTicker(cloudSyncInterval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-stop:
|
||||
return
|
||||
case <-ticker.C:
|
||||
synced, err := audit.DrainToCloud(context.Background(), cfg, cloudSyncTickLockWait, cloudSyncTickTimeout)
|
||||
if err != nil {
|
||||
if errors.Is(err, audit.ErrSyncInProgress) {
|
||||
log.Debugf("periodic cloud sync skipped: another sync in progress")
|
||||
} else {
|
||||
log.Warnf("periodic cloud sync failed: %v", err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
total += synced
|
||||
if synced > 0 {
|
||||
log.Infof("Periodic cloud sync: flushed %d events", synced)
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
return func() int {
|
||||
close(stop)
|
||||
<-done
|
||||
return total
|
||||
}
|
||||
}
|
||||
|
||||
// listenAddr resolves the proxy's bind address from config (host) and the
|
||||
// --port flag. Host defaults to loopback.
|
||||
func listenAddr(host string, port int) string {
|
||||
if host == "" {
|
||||
host = "127.0.0.1"
|
||||
}
|
||||
|
||||
return net.JoinHostPort(host, strconv.Itoa(port))
|
||||
}
|
||||
|
||||
// autoBlockConfirmations drains the confirmation channel and always denies,
|
||||
// appropriate for non-interactive CI/CD environments.
|
||||
func autoBlockConfirmations(ch chan *interceptors.ConfirmationRequest) {
|
||||
for req := range ch {
|
||||
log.Warnf("Persistent proxy: auto-blocking suspicious package %s", req.PackageVersion.GetPackage().GetName())
|
||||
req.ResponseChan <- false
|
||||
close(req.ResponseChan)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
const stateFileName = "proxy-state.json"
|
||||
|
||||
// State is the on-disk record of a running persistent proxy. It is written by
|
||||
// the daemon and read by the stop/env/status commands.
|
||||
type State struct {
|
||||
PID int `json:"pid"`
|
||||
Addr string `json:"addr"`
|
||||
CACertPath string `json:"ca_cert_path"`
|
||||
BlockedCount int `json:"blocked_count"`
|
||||
|
||||
// CloudSync records the daemon's shutdown cloud flush so `pmg proxy stop`
|
||||
// can report the outcome. The daemon's own logs go to proxy.log (and are
|
||||
// suppressed without --debug), so the state file is how the result reaches
|
||||
// the stop process. nil when cloud sync is disabled.
|
||||
CloudSync *CloudSyncResult `json:"cloud_sync,omitempty"`
|
||||
}
|
||||
|
||||
// CloudSyncResult is the outcome of the daemon's shutdown flush to SafeDep Cloud.
|
||||
type CloudSyncResult struct {
|
||||
Synced int `json:"synced"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
func stateFilePath(dir string) string {
|
||||
return filepath.Join(dir, stateFileName)
|
||||
}
|
||||
|
||||
func writeState(path string, s State) error {
|
||||
data, err := json.Marshal(s)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal proxy state: %w", err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
return fmt.Errorf("create proxy state dir: %w", err)
|
||||
}
|
||||
return os.WriteFile(path, data, 0o600)
|
||||
}
|
||||
|
||||
func readState(path string) (State, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return State{}, fmt.Errorf("read proxy state: %w", err)
|
||||
}
|
||||
var s State
|
||||
if err := json.Unmarshal(data, &s); err != nil {
|
||||
return State{}, fmt.Errorf("unmarshal proxy state: %w", err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func removeState(path string) error {
|
||||
return os.Remove(path)
|
||||
}
|
||||
|
||||
// IsRunning reports whether the recorded PID is a live process.
|
||||
func (s State) IsRunning() bool {
|
||||
if s.PID <= 0 {
|
||||
return false
|
||||
}
|
||||
proc, err := os.FindProcess(s.PID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return proc.Signal(syscall.Signal(0)) == nil
|
||||
}
|
||||
|
||||
// ResolveStatePath returns the effective state file path: the flag override
|
||||
// when set, otherwise <cacheDir>/proxy-state.json.
|
||||
func ResolveStatePath(flag, cacheDir string) string {
|
||||
if flag != "" {
|
||||
return flag
|
||||
}
|
||||
return stateFilePath(cacheDir)
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestWriteAndReadState(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := stateFilePath(dir)
|
||||
|
||||
s := State{PID: 12345, Addr: "127.0.0.1:9999", CACertPath: "/tmp/ca.pem"}
|
||||
require.NoError(t, writeState(path, s))
|
||||
|
||||
got, err := readState(path)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, s.PID, got.PID)
|
||||
assert.Equal(t, s.Addr, got.Addr)
|
||||
assert.Equal(t, s.CACertPath, got.CACertPath)
|
||||
}
|
||||
|
||||
func TestWriteStateCreatesMissingDir(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "nested", "dir", "proxy-state.json")
|
||||
require.NoError(t, writeState(path, State{PID: 1, Addr: "127.0.0.1:1"}))
|
||||
|
||||
got, err := readState(path)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 1, got.PID)
|
||||
}
|
||||
|
||||
func TestReadStateMissingFile(t *testing.T) {
|
||||
_, err := readState(filepath.Join(t.TempDir(), "nonexistent.json"))
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestRemoveState(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := stateFilePath(dir)
|
||||
|
||||
require.NoError(t, writeState(path, State{PID: 1, Addr: "127.0.0.1:1"}))
|
||||
require.NoError(t, removeState(path))
|
||||
|
||||
_, err := os.Stat(path)
|
||||
assert.True(t, os.IsNotExist(err))
|
||||
}
|
||||
|
||||
func TestIsRunningCurrentProcess(t *testing.T) {
|
||||
s := State{PID: os.Getpid(), Addr: "127.0.0.1:1"}
|
||||
assert.True(t, s.IsRunning())
|
||||
}
|
||||
|
||||
func TestIsRunningDeadPID(t *testing.T) {
|
||||
s := State{PID: 999999999}
|
||||
assert.False(t, s.IsRunning())
|
||||
}
|
||||
|
||||
func TestStateFilePath(t *testing.T) {
|
||||
assert.Equal(t, "/some/dir/proxy-state.json", stateFilePath("/some/dir"))
|
||||
}
|
||||
|
||||
func TestResolveStatePath(t *testing.T) {
|
||||
t.Run("flag override wins", func(t *testing.T) {
|
||||
assert.Equal(t, "/custom/proxy.json", ResolveStatePath("/custom/proxy.json", "/cache"))
|
||||
})
|
||||
|
||||
t.Run("defaults to cacheDir", func(t *testing.T) {
|
||||
assert.Equal(t, filepath.Join("/cache", "proxy-state.json"), ResolveStatePath("", "/cache"))
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package proxyserver
|
||||
|
||||
// StatusInfo describes the proxy's current state for rendering by the caller.
|
||||
type StatusInfo struct {
|
||||
Found bool
|
||||
Running bool
|
||||
PID int
|
||||
Addr string
|
||||
CACert string
|
||||
}
|
||||
|
||||
// GetStatus reports the proxy status from the state file at statePath.
|
||||
func GetStatus(statePath string) StatusInfo {
|
||||
state, err := readState(statePath)
|
||||
if err != nil {
|
||||
return StatusInfo{Found: false}
|
||||
}
|
||||
|
||||
return StatusInfo{
|
||||
Found: true,
|
||||
Running: state.IsRunning(),
|
||||
PID: state.PID,
|
||||
Addr: state.Addr,
|
||||
CACert: state.CACertPath,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package proxyserver
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/safedep/dry/log"
|
||||
)
|
||||
|
||||
const (
|
||||
stopPollInterval = 200 * time.Millisecond
|
||||
|
||||
// stopWaitTimeout must exceed the daemon's worst-case shutdown (drain + cloud
|
||||
// flush), so we never read stale state or delete the state file while the
|
||||
// daemon is still flushing. Derived from the daemon's budget so the two can't
|
||||
// drift apart.
|
||||
stopWaitTimeout = daemonShutdownBudget + 15*time.Second
|
||||
)
|
||||
|
||||
// StopResult carries the outcome of stopping the proxy so the caller can render
|
||||
// a summary and decide whether to fail (e.g. on policy violations).
|
||||
type StopResult struct {
|
||||
PID int
|
||||
BlockedCount int
|
||||
// StateVerified is false when the final state could not be read after
|
||||
// shutdown (e.g. the proxy crashed), which callers may treat as fail-closed.
|
||||
StateVerified bool
|
||||
// CloudSync is the daemon's shutdown flush outcome (nil when cloud sync is
|
||||
// disabled), surfaced so the caller can report it.
|
||||
CloudSync *CloudSyncResult
|
||||
}
|
||||
|
||||
// Stop signals the running proxy to terminate, waits for it to exit, and
|
||||
// removes the state file. It returns a StopResult describing the run. The
|
||||
// daemon flushes audit events to the cloud itself during shutdown (it, unlike
|
||||
// this process, has no proxy env vars). Operational failures (no proxy running,
|
||||
// signal errors) are returned as errors.
|
||||
func Stop(statePath string) (StopResult, error) {
|
||||
state, err := readState(statePath)
|
||||
if err != nil {
|
||||
return StopResult{}, fmt.Errorf("no proxy state found — is the proxy running? (%w)", err)
|
||||
}
|
||||
|
||||
if !state.IsRunning() {
|
||||
if rerr := removeState(statePath); rerr != nil {
|
||||
log.Warnf("failed to remove proxy state file: %v", rerr)
|
||||
}
|
||||
return StopResult{}, fmt.Errorf("proxy process (pid %d) is not running; state file cleaned up", state.PID)
|
||||
}
|
||||
|
||||
proc, err := os.FindProcess(state.PID)
|
||||
if err != nil {
|
||||
return StopResult{}, fmt.Errorf("find proxy process (pid %d): %w", state.PID, err)
|
||||
}
|
||||
if err := proc.Signal(syscall.SIGTERM); err != nil {
|
||||
return StopResult{}, fmt.Errorf("send SIGTERM to proxy (pid %d): %w", state.PID, err)
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(stopWaitTimeout)
|
||||
exited := false
|
||||
for time.Now().Before(deadline) {
|
||||
if !state.IsRunning() {
|
||||
exited = true
|
||||
break
|
||||
}
|
||||
time.Sleep(stopPollInterval)
|
||||
}
|
||||
|
||||
// If the daemon is still alive it is likely mid-flush. Do not read the state
|
||||
// (it is not final yet) or remove the file (the daemon still owns it).
|
||||
// Return an error so --fail-on-violation fails closed rather than reporting
|
||||
// a stale "0 blocked".
|
||||
if !exited {
|
||||
return StopResult{}, fmt.Errorf("proxy (pid %d) did not shut down within %s; leaving state file in place", state.PID, stopWaitTimeout)
|
||||
}
|
||||
|
||||
final, readErr := readState(statePath)
|
||||
if rerr := removeState(statePath); rerr != nil {
|
||||
log.Warnf("failed to remove proxy state file: %v", rerr)
|
||||
}
|
||||
|
||||
return StopResult{
|
||||
PID: state.PID,
|
||||
BlockedCount: final.BlockedCount,
|
||||
StateVerified: readErr == nil,
|
||||
CloudSync: final.CloudSync,
|
||||
}, nil
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"github.com/safedep/pmg/cmd/executors"
|
||||
landlockCmd "github.com/safedep/pmg/cmd/landlock"
|
||||
"github.com/safedep/pmg/cmd/npm"
|
||||
proxyCmd "github.com/safedep/pmg/cmd/proxy"
|
||||
"github.com/safedep/pmg/cmd/pypi"
|
||||
sandboxCmd "github.com/safedep/pmg/cmd/sandbox"
|
||||
"github.com/safedep/pmg/cmd/setup"
|
||||
@@ -108,6 +109,17 @@ func main() {
|
||||
ui.ErrorExit(err)
|
||||
}
|
||||
|
||||
// The proxy daemon delivers events itself (periodic sync + shutdown
|
||||
// flush), so suppress the detached background auto-sync for proxy
|
||||
// commands. It would otherwise spawn a redundant child that contends
|
||||
// for the sync lock and, from `pmg proxy stop`, routes through the
|
||||
// now-stopped proxy. We suppress the spawn directly rather than
|
||||
// flipping AutoSync.Enabled, because the daemon's periodic sync
|
||||
// honors that flag.
|
||||
if isProxyCommand(cmd) {
|
||||
audit.SuppressBackgroundSync()
|
||||
}
|
||||
|
||||
config.FinalizeDependencyCooldownOverride()
|
||||
|
||||
// Parse and validate --sandbox-allow flags after all flags are resolved
|
||||
@@ -148,6 +160,7 @@ func main() {
|
||||
cmd.AddCommand(pypi.NewUvCommand())
|
||||
cmd.AddCommand(pypi.NewPoetryCommand())
|
||||
cmd.AddCommand(executors.NewPipxCommand())
|
||||
cmd.AddCommand(proxyCmd.NewProxyCommand())
|
||||
cmd.AddCommand(version.NewVersionCommand())
|
||||
cmd.AddCommand(setup.NewSetupCommand())
|
||||
cmd.AddCommand(setup.NewRemoveCommand())
|
||||
@@ -214,3 +227,13 @@ func logDebugContext() {
|
||||
log.Debugf("Dry run: %t, insecure installation: %t, trusted packages: %d",
|
||||
cfg.DryRun, cfg.InsecureInstallation, len(cfg.Config.TrustedPackages))
|
||||
}
|
||||
|
||||
// isProxyCommand reports whether cmd is `pmg proxy` or one of its subcommands.
|
||||
func isProxyCommand(cmd *cobra.Command) bool {
|
||||
for c := cmd; c != nil; c = c.Parent() {
|
||||
if c.Name() == "proxy" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
package packagemanager
|
||||
|
||||
import "fmt"
|
||||
|
||||
// proxyNoProxyList is the NO_PROXY value for proxied package-manager runs.
|
||||
//
|
||||
// The IPv6 loopback uses the bare ::1: the bracketed [::1] is URL syntax that
|
||||
// crashes Python's urllib/httpx (#339). Trade-off: Node's NODE_USE_ENV_PROXY
|
||||
// (undici) only bypasses the bracketed form, so a literal http://[::1] from
|
||||
// Node still gets proxied. localhost/127.0.0.1 cover the common cases; the
|
||||
// IPv6 literal is a rare edge we accept since NO_PROXY can't be set per-client.
|
||||
const proxyNoProxyList = "localhost,127.0.0.1,::1"
|
||||
|
||||
// EnvVarForProxy returns the environment variables (KEY=VALUE lines) that route
|
||||
// the supported package managers through the proxy at proxyAddr and make them
|
||||
// trust its MITM CA at certPath. It encodes per-package-manager quirks: yarn
|
||||
// Berry ignores HTTP_PROXY and needs YARN_* (#319); pip/requests and Node each
|
||||
// read their own CA-bundle var.
|
||||
//
|
||||
// The cert-path variables are always emitted, never skipped based on OS
|
||||
// trust-store status. Whether a tool trusts the OS store varies by tool,
|
||||
// version and config: modern pip (>=24.2) and recent Node (--use-system-ca) can
|
||||
// read it, but older versions, requests/certifi, and default configs still rely
|
||||
// on bundled CA lists. Emitting these vars is the conservative choice that works
|
||||
// across that matrix, and is harmless for tools that do read the OS store (they
|
||||
// ignore the vars). Skipping them when a system CA exists would silently break
|
||||
// any tool still on a bundled store.
|
||||
func EnvVarForProxy(proxyAddr, certPath string) []string {
|
||||
proxyURL := fmt.Sprintf("http://%s", proxyAddr)
|
||||
|
||||
return []string{
|
||||
"PIP_RETRIES=0",
|
||||
"NODE_USE_ENV_PROXY=1",
|
||||
fmt.Sprintf("HTTP_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("HTTPS_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("http_proxy=%s", proxyURL),
|
||||
fmt.Sprintf("https_proxy=%s", proxyURL),
|
||||
fmt.Sprintf("NO_PROXY=%s", proxyNoProxyList),
|
||||
fmt.Sprintf("no_proxy=%s", proxyNoProxyList),
|
||||
fmt.Sprintf("PIP_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("YARN_HTTP_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("YARN_HTTPS_PROXY=%s", proxyURL),
|
||||
fmt.Sprintf("NODE_EXTRA_CA_CERTS=%s", certPath),
|
||||
fmt.Sprintf("SSL_CERT_FILE=%s", certPath),
|
||||
fmt.Sprintf("REQUESTS_CA_BUNDLE=%s", certPath),
|
||||
fmt.Sprintf("PIP_CERT=%s", certPath),
|
||||
fmt.Sprintf("YARN_HTTPS_CA_FILE_PATH=%s", certPath),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
package packagemanager
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func envToMap(env []string) map[string]string {
|
||||
m := make(map[string]string, len(env))
|
||||
for _, e := range env {
|
||||
if k, v, ok := strings.Cut(e, "="); ok {
|
||||
m[k] = v
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// TestEnvVarForProxyConfiguresYarn proves the root cause of #319: yarn Berry
|
||||
// (yarn 2+) ignores the standard HTTP_PROXY/HTTPS_PROXY env vars and only honors
|
||||
// its own config, which can be set via YARN_* env overrides. Without these,
|
||||
// yarn bypasses the MITM proxy entirely and no packages are analyzed.
|
||||
func TestEnvVarForProxyConfiguresYarn(t *testing.T) {
|
||||
const proxyAddr = "127.0.0.1:54321"
|
||||
const caCertPath = "/tmp/pmg-ca-cert.pem"
|
||||
|
||||
env := envToMap(EnvVarForProxy(proxyAddr, caCertPath))
|
||||
proxyURL := "http://" + proxyAddr
|
||||
|
||||
assert.Equal(t, proxyURL, env["YARN_HTTP_PROXY"],
|
||||
"yarn ignores HTTP_PROXY; YARN_HTTP_PROXY is required to route yarn through the proxy")
|
||||
assert.Equal(t, proxyURL, env["YARN_HTTPS_PROXY"],
|
||||
"yarn ignores HTTPS_PROXY; YARN_HTTPS_PROXY is required to route yarn through the proxy")
|
||||
assert.Equal(t, caCertPath, env["YARN_HTTPS_CA_FILE_PATH"],
|
||||
"yarn ignores NODE_EXTRA_CA_CERTS; YARN_HTTPS_CA_FILE_PATH is required to trust the MITM CA")
|
||||
}
|
||||
|
||||
// TestEnvVarForProxyNoProxyIPv6 proves the fix for #339: the IPv6 loopback in
|
||||
// NO_PROXY must be bare (::1), not bracketed ([::1]). Brackets are URL syntax,
|
||||
// not NO_PROXY syntax, and Python's urllib/httpx crashes parsing them with
|
||||
// "Invalid port: ':1]'".
|
||||
func TestEnvVarForProxyNoProxyIPv6(t *testing.T) {
|
||||
env := envToMap(EnvVarForProxy("127.0.0.1:54321", "/tmp/pmg-ca-cert.pem"))
|
||||
|
||||
for _, key := range []string{"NO_PROXY", "no_proxy"} {
|
||||
assert.Equal(t, "localhost,127.0.0.1,::1", env[key],
|
||||
"%s must use bare ::1; bracketed [::1] is invalid NO_PROXY syntax and crashes httpx", key)
|
||||
assert.NotContains(t, env[key], "[::1]",
|
||||
"%s must not contain bracketed IPv6 loopback", key)
|
||||
}
|
||||
}
|
||||
|
||||
// TestEnvVarForProxyAlwaysEmitsCertVars proves the cert-path vars are always
|
||||
// present (never skipped on OS trust-store status), since trust behavior varies
|
||||
// by tool/version/config and many tools still rely on bundled CA stores.
|
||||
func TestEnvVarForProxyAlwaysEmitsCertVars(t *testing.T) {
|
||||
env := envToMap(EnvVarForProxy("127.0.0.1:9000", "/tmp/ca.pem"))
|
||||
|
||||
assert.Equal(t, "http://127.0.0.1:9000", env["HTTP_PROXY"])
|
||||
for _, key := range []string{
|
||||
"NODE_EXTRA_CA_CERTS", "SSL_CERT_FILE", "REQUESTS_CA_BUNDLE",
|
||||
"PIP_CERT", "YARN_HTTPS_CA_FILE_PATH",
|
||||
} {
|
||||
assert.Equal(t, "/tmp/ca.pem", env[key], "%s must point at the CA bundle", key)
|
||||
}
|
||||
}
|
||||
@@ -9,11 +9,29 @@ import (
|
||||
const (
|
||||
caCertFileName = "ca-cert.pem"
|
||||
caKeyFileName = "ca-key.pem"
|
||||
|
||||
// proxyCABundleFileName is the merged CA bundle (PMG CA + system roots) that
|
||||
// proxy clients trust via SSL_CERT_FILE/NODE_EXTRA_CA_CERTS. It is distinct
|
||||
// from caCertFileName, which is just the PMG CA. Owned here so callers don't
|
||||
// invent their own names.
|
||||
proxyCABundleFileName = "proxy-ca.pem"
|
||||
)
|
||||
|
||||
func CACertPath(dir string) string { return filepath.Join(dir, caCertFileName) }
|
||||
func CAKeyPath(dir string) string { return filepath.Join(dir, caKeyFileName) }
|
||||
|
||||
// ProxyCABundlePath returns the stable, machine-local path of the merged CA
|
||||
// bundle. Used by the persistent proxy, whose daemon/env/stop processes all
|
||||
// reference the same file for the proxy's lifetime.
|
||||
func ProxyCABundlePath(dir string) string { return filepath.Join(dir, proxyCABundleFileName) }
|
||||
|
||||
// EphemeralProxyCABundlePath returns a per-process temp path for the merged CA
|
||||
// bundle. Used by the per-command flow, which writes a throwaway bundle and
|
||||
// removes it on exit; the pid keeps concurrent runs from colliding.
|
||||
func EphemeralProxyCABundlePath() string {
|
||||
return filepath.Join(os.TempDir(), fmt.Sprintf("pmg-%d-%s", os.Getpid(), proxyCABundleFileName))
|
||||
}
|
||||
|
||||
// PersistentCACertManagerConfig returns the config for the on-disk,
|
||||
// system-trusted CA. The root is long-lived (10 years) because rotating an
|
||||
// installed, trusted root is expensive; leaf certs remain short (1 day).
|
||||
|
||||
Reference in New Issue
Block a user