feat(proxy): persistent proxy server mode (#351)

* refactor(flows): extract SetupCACertificate for reuse

Move the CA load/generate/merge logic out of proxyFlow into an exported
flows.SetupCACertificate so the persistent proxy server can reuse it.

* feat(proxy): add persistent proxy server with start/stop/env/status

Introduces 'pmg proxy' commands backed by internal/proxyserver: a long-lived
MITM proxy that intercepts package managers via env vars (no shims). Supports
--daemon (Unix), --state, --port; generic 'env' output that skips cert vars
when the CA is OS-trusted; opt-in 'stop --fail-on-violation' (fail-closed on
crash) with a synchronous cloud event flush; and the malysis analysis cache.

* feat(action): add server-mode for persistent proxy

When server-mode=true the action starts the proxy daemon and injects proxy
env vars into the job instead of installing shims.

* test(proxy): add persistent proxy server E2E workflow

* docs(readme): document persistent proxy server mode

* fix(proxy): create cache dir before writing state file and daemon log

On a fresh CI runner the cache directory does not exist yet; os.OpenFile and
os.WriteFile do not create parent dirs, so 'pmg proxy start --daemon' failed
with 'no such file or directory'. MkdirAll the parent before writing.

* docs: add persistent proxy server architecture doc

* refactor proxyserver

* fix(proxy): always emit cert env vars instead of skipping on OS-trust status

npm/pip/yarn/requests trust the MITM CA inconsistently across tools, versions,
and configs; many still use bundled CA stores. Always emitting the cert-path
env vars is the conservative choice that works regardless, and is harmless for
tools that read the OS store (they ignore the vars). Skipping them when a
system CA exists would silently break any tool still on a bundled store.

* refactor(proxy): drop redundant audit init in daemon; rely on main.go

main.go's PersistentPreRun already initializes the audit pipeline for every
command (including the daemon's re-exec'd child) and closes it at process exit.
Re-initializing in proxyserver.Run created a second auditor and a second
cloud-sync WAL connection, orphaning the first. Removing it makes the daemon
consistent with the normal proxy flow, which never self-initializes audit.

* fix(proxy): bypass proxy env when flushing events to cloud on stop

pmg proxy stop inherits HTTP(S)_PROXY (injected by 'pmg proxy env') pointing at
the PMG proxy it just shut down. The cloud sync gRPC client honored those vars
and routed api.safedep.io through the dead proxy, failing with 'connection
refused' so no events were delivered. Clear the proxy env vars before the sync
so PMG's own cloud traffic goes direct.

* chore(proxy): address review feedback

- configurable bind host via proxy.server.listen_host (default loopback)
- proxy commands use ui.ErrorExit instead of returning errors to cobra
- rename errcode to ProxyPolicyViolation (covers malware + cooldown)
- share cloud sync via audit.DrainToCloud (de-dup with cmd/cloud/sync)
- centralize proxy CA bundle path in certmanager
- docs: persistent proxy cert trust + bind address

* fix(proxy): show real message on fail-on-violation error

stopExitError set only WithMsg, but ui.ErrorExit renders HumanError, so the
framed error showed 'no human-readable message available'. Set both from one
string, and emit the framed error before the stdout summary so the blocked
count is stated once.

* fix(proxy): flush cloud events from the daemon, not stop

The stop process inherits HTTP_PROXY (from 'pmg proxy env'), so its cloud
client routed api.safedep.io through the already-stopped proxy and failed with
connection refused. Move the flush into the daemon's shutdown, which has no
proxy env (it started before env injection) and dials SafeDep directly.

- daemon flushes on shutdown via audit.DrainToCloud and records the result in
  the state file; stop surfaces it (on both success and fail-on-violation
  paths) since the daemon's own logs aren't visible to stop
- coordinate stop's wait with the daemon shutdown budget; on timeout, error
  out without reading stale state or deleting the file (fail-closed)
- persist blocked count before the flush so the gate stays correct if the
  flush hangs or the daemon is killed mid-flush
- remove now-redundant cloud_flush.go

* disable auto-sync for proxy cmds

* feat(proxy): periodic cloud sync + move proxy env vars to packagemanager

- daemon runs a periodic cloud-sync ticker so the shutdown flush stays small;
  the run total is reported by stop, and shutdown timeouts are coordinated
- move EnvVarForProxy from config to packagemanager (it is package-manager
  knowledge); the shared function now builds the proxy URL and NO_PROXY itself,
  removing the duplicated construction in the per-command and persistent paths
- relocate the #319 yarn and #339 IPv6 regression tests alongside the function
- enable cloud sync in the persistent-proxy E2E workflow and fix the stale
  internal/proxystate path filter

* refactor(proxy): rename cloudFlushLockTimeout to cloudFlushLockWait

Consistent timeout naming: *LockWait is the lock-acquire bound, *Timeout is
the sync-RPC bound. Previously the final-flush pair was cloudFlushLockTimeout
vs cloudFlushTimeout — two lookalike names for different operations.

* refactor(proxy): extract cloudFlush and trim duplicate shutdown comments

The shutdown's final-flush block is now a cloudFlush helper, symmetric with
startCloudSyncLoop (one-shot vs loop). Removed the triplicated ticker/lock
contention comments, keeping the contract on the function doc and one-line
pointers at the call sites.

* docs: update persistent proxy cloud sync to daemon-owned model

The daemon now owns cloud delivery (periodic sync while serving + final flush
on shutdown); stop signals it, waits, and reports the result. Rewrite the Cloud
event sync section, fix stop attributions, add the cloud_sync state field, and
update the sequence diagram.

* docs: move Usage section up below How it works

Put the copy-paste recipes near the top so users find them before the internals.

* refactor(proxy): address PR review feedback

- configurable bind host/port via --host/--port flags + config (listen_host,
  listen_port), bound directly to config fields per PMG's flag pattern
- daemon log path via --log-file and readiness timeout in ProxyDaemonConfig;
  Daemonize no longer owns path policy (caller validates, fails fast)
- gate periodic cloud sync on auto_sync; suppress detached background sync for
  proxy commands instead of flipping the flag
- pmg proxy env --export emits shell-quoted lines for eval (spaces survive)
- extract shared flows.BuildCachedMalysisAnalyzer, dropping the analyzer+cache
  duplication between proxy flow and proxy server
- add internal/proxyserver/doc.go documenting the package + boundary vs flows
- E2E: assert malicious installs are blocked (drop continue-on-error)
- docs: trim Commands/State-file to user contracts; refresh bind address

* refactor(proxy): proactive alignment fixes from whole-PR review

- gate the shutdown cloud flush on auto_sync too, matching the periodic ticker
  (auto_sync consistently controls all daemon-driven cloud delivery)
- ResolveStatePath takes cacheDir instead of *RuntimeConfig, keeping state.go
  free of config dependency
- drop the empty-host comment in listenAddr; keep the loopback guard so a blank
  host never silently binds all interfaces

* fix: Decouple localdb with malysis analyser construction

* fix: Persist global args before proxy server daemon exec

* fix: GitHub Action for cloud auto-sync in server mode

---------

Co-authored-by: Abhisek Datta <abhisek.datta@gmail.com>

cmd/proxy/env.go

@@ -0,0 +1,74 @@
+package proxy
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/safedep/pmg/config"
+	"github.com/safedep/pmg/internal/proxyserver"
+	"github.com/safedep/pmg/internal/ui"
+	"github.com/spf13/cobra"
+)
+
+var envExportFlag bool
+
+func newEnvCommand() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "env",
+		Short: "Print proxy environment variables (KEY=VALUE per line)",
+		Long: "Print proxy environment variables as KEY=VALUE lines.\n\n" +
+			"GitHub Actions:  pmg proxy env >> \"$GITHUB_ENV\"\n" +
+			"Shell:           eval \"$(pmg proxy env --export)\"\n\n" +
+			"Use --export for shell `eval`: it emits quoted `export KEY='VALUE'`\n" +
+			"lines that survive values containing spaces. The\n" +
+			"default raw KEY=VALUE form is for $GITHUB_ENV, which must not be quoted.",
+		RunE: runEnv,
+	}
+	cmd.Flags().BoolVar(&envExportFlag, "export", false, "Emit shell `export KEY='VALUE'` lines for `eval`")
+	return cmd
+}
+
+func runEnv(_ *cobra.Command, _ []string) error {
+	cfg := config.Get()
+	statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
+
+	vars, err := proxyserver.EnvVars(statePath)
+	if err != nil {
+		ui.ErrorExit(err)
+	}
+
+	w := bufio.NewWriter(os.Stdout)
+	for _, v := range vars {
+		line := v
+		if envExportFlag {
+			line = exportLine(v)
+		}
+		if _, werr := fmt.Fprintln(w, line); werr != nil {
+			ui.ErrorExit(fmt.Errorf("write env var: %w", werr))
+		}
+	}
+
+	if err := w.Flush(); err != nil {
+		ui.ErrorExit(err)
+	}
+
+	return nil
+}
+
+// exportLine turns "KEY=VALUE" into a shell-safe `export KEY='VALUE'`, so values
+// with spaces (e.g. the macOS "Application Support" path) survive `eval`.
+func exportLine(kv string) string {
+	k, v, ok := strings.Cut(kv, "=")
+	if !ok {
+		return kv
+	}
+	return fmt.Sprintf("export %s=%s", k, shellSingleQuote(v))
+}
+
+// shellSingleQuote wraps s in single quotes, escaping any embedded single quote
+// as '\” (close, escaped quote, reopen).
+func shellSingleQuote(s string) string {
+	return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
+}

cmd/proxy/proxy.go

@@ -0,0 +1,20 @@
+package proxy
+
+import "github.com/spf13/cobra"
+
+// stateFlag binds the persistent --state flag shared by all proxy subcommands.
+var stateFlag string
+
+func NewProxyCommand() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "proxy",
+		Short: "Manage the persistent PMG proxy server",
+	}
+	cmd.AddCommand(newStartCommand())
+	cmd.AddCommand(newStopCommand())
+	cmd.AddCommand(newEnvCommand())
+	cmd.AddCommand(newStatusCommand())
+	cmd.PersistentFlags().StringVar(&stateFlag, "state", "",
+		"Path to the proxy state file (default: <cache-dir>/proxy-state.json)")
+	return cmd
+}

cmd/proxy/start.go

@@ -0,0 +1,103 @@
+package proxy
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strconv"
+
+	"github.com/safedep/pmg/config"
+	"github.com/safedep/pmg/internal/proxyserver"
+	"github.com/safedep/pmg/internal/ui"
+	"github.com/spf13/cobra"
+)
+
+var (
+	daemonFlag             bool
+	logFileFlag            string
+	foregroundInternalFlag bool
+)
+
+func newStartCommand() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "start",
+		Short: "Start the persistent PMG proxy server",
+		RunE:  runStart,
+	}
+
+	// Bind --host/--port directly onto the config fields with the loaded config
+	// values as defaults, matching PMG's flag pattern (see config/cobra.go): a
+	// supplied flag overwrites the field, otherwise the config value stands.
+	// Precedence: flag > env > config file > default.
+	srv := &config.Get().Config.Proxy.Server
+
+	cmd.Flags().BoolVarP(&daemonFlag, "daemon", "D", false, "Run the proxy as a detached background process")
+	cmd.Flags().StringVar(&srv.ListenHost, "host", srv.ListenHost, "Host to bind")
+	cmd.Flags().IntVar(&srv.ListenPort, "port", srv.ListenPort, "Port to bind (0 = a random free port)")
+	cmd.Flags().StringVar(&logFileFlag, "log-file", "", "File for the daemon's output (default: <cache-dir>/proxy.log)")
+	cmd.Flags().BoolVar(&foregroundInternalFlag, "foreground-internal", false, "Internal: run the foreground server (used by --daemon)")
+	if err := cmd.Flags().MarkHidden("foreground-internal"); err != nil {
+		panic(err)
+	}
+	return cmd
+}
+
+func runStart(cmd *cobra.Command, _ []string) error {
+	cfg := config.Get()
+	statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
+	host := cfg.Config.Proxy.Server.ListenHost
+	port := cfg.Config.Proxy.Server.ListenPort
+
+	if daemonFlag && !foregroundInternalFlag {
+		if err := startDaemon(cmd, cfg, statePath, host, port); err != nil {
+			ui.ErrorExit(err)
+		}
+		return nil
+	}
+
+	if err := proxyserver.Run(cmd.Context(), cfg, statePath, host, port); err != nil {
+		ui.ErrorExit(err)
+	}
+	return nil
+}
+
+func startDaemon(cmd *cobra.Command, cfg *config.RuntimeConfig, statePath, host string, port int) error {
+	exe, err := os.Executable()
+	if err != nil {
+		return fmt.Errorf("resolve executable: %w", err)
+	}
+
+	// Daemon log: the --log-file flag if set, else <cache-dir>/proxy.log. The
+	// caller owns this path, so ensure its parent directory exists here.
+	logPath := logFileFlag
+	if logPath == "" {
+		logPath = filepath.Join(cfg.CacheDir(), "proxy.log")
+	}
+	if err := os.MkdirAll(filepath.Dir(logPath), 0o700); err != nil {
+		return fmt.Errorf("create daemon log dir: %w", err)
+	}
+
+	args := daemonArgs(cmd, statePath, host, port)
+
+	daemonCfg := proxyserver.ProxyDaemonConfig{
+		LogPath:      logPath,
+		ReadyTimeout: proxyserver.DefaultDaemonReadyTimeout,
+	}
+	state, err := proxyserver.Daemonize(daemonCfg, statePath, exe, args)
+	if err != nil {
+		return err
+	}
+
+	_, werr := fmt.Fprintf(os.Stdout, "PMG proxy daemon started on %s (pid %d)\n", state.Addr, state.PID)
+	return werr
+}
+
+func daemonArgs(cmd *cobra.Command, statePath, host string, port int) []string {
+	args := append([]string{}, config.ChangedConfigFlagArgs(cmd)...)
+	return append(args,
+		"proxy", "start", "--foreground-internal",
+		"--state", statePath,
+		"--host", host,
+		"--port", strconv.Itoa(port),
+	)
+}

cmd/proxy/start_test.go

@@ -0,0 +1,41 @@
+package proxy
+
+import (
+	"testing"
+
+	"github.com/safedep/pmg/config"
+	"github.com/spf13/cobra"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestDaemonArgsPrependsChangedConfigFlags(t *testing.T) {
+	root := &cobra.Command{Use: "pmg"}
+	config.ApplyCobraFlags(root)
+
+	var got []string
+	start := &cobra.Command{
+		Use: "start",
+		Run: func(cmd *cobra.Command, _ []string) {
+			got = daemonArgs(cmd, "/tmp/proxy-state.json", "127.0.0.1", 9000)
+		},
+	}
+	proxyCmd := &cobra.Command{Use: "proxy"}
+	proxyCmd.AddCommand(start)
+	root.AddCommand(proxyCmd)
+	root.SetArgs([]string{
+		"--paranoid",
+		"--skip-dependency-cooldown",
+		"proxy", "start",
+	})
+
+	require.NoError(t, root.Execute())
+	assert.Equal(t, []string{
+		"--paranoid=true",
+		"--skip-dependency-cooldown=true",
+		"proxy", "start", "--foreground-internal",
+		"--state", "/tmp/proxy-state.json",
+		"--host", "127.0.0.1",
+		"--port", "9000",
+	}, got)
+}

cmd/proxy/status.go

@@ -0,0 +1,42 @@
+package proxy
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/safedep/pmg/config"
+	"github.com/safedep/pmg/internal/proxyserver"
+	"github.com/safedep/pmg/internal/ui"
+	"github.com/spf13/cobra"
+)
+
+func newStatusCommand() *cobra.Command {
+	return &cobra.Command{
+		Use:   "status",
+		Short: "Show the status of the persistent PMG proxy server",
+		RunE:  runStatus,
+	}
+}
+
+func runStatus(_ *cobra.Command, _ []string) error {
+	cfg := config.Get()
+	statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
+
+	st := proxyserver.GetStatus(statePath)
+
+	var line string
+	switch {
+	case !st.Found:
+		line = "PMG proxy: not running (no state file)\n"
+	case st.Running:
+		line = fmt.Sprintf("PMG proxy: running (pid %d, addr %s, ca %s)\n", st.PID, st.Addr, st.CACert)
+	default:
+		line = fmt.Sprintf("PMG proxy: stopped (stale state for pid %d — run 'pmg proxy stop' to clean up)\n", st.PID)
+	}
+
+	if _, err := fmt.Fprint(os.Stdout, line); err != nil {
+		ui.ErrorExit(err)
+	}
+
+	return nil
+}

cmd/proxy/stop.go

@@ -0,0 +1,100 @@
+package proxy
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/safedep/dry/usefulerror"
+	"github.com/safedep/pmg/config"
+	"github.com/safedep/pmg/errcodes"
+	"github.com/safedep/pmg/internal/proxyserver"
+	"github.com/safedep/pmg/internal/ui"
+	"github.com/spf13/cobra"
+)
+
+var failOnViolation bool
+
+func newStopCommand() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "stop",
+		Short: "Stop the running persistent PMG proxy server",
+		// Failures here are user-facing policy outcomes, not usage errors.
+		SilenceUsage: true,
+		RunE:         runStop,
+	}
+	cmd.Flags().BoolVar(&failOnViolation, "fail-on-violation", false,
+		"Exit non-zero if any package was blocked during the proxy session")
+	return cmd
+}
+
+func runStop(_ *cobra.Command, _ []string) error {
+	cfg := config.Get()
+	statePath := proxyserver.ResolveStatePath(stateFlag, cfg.CacheDir())
+
+	res, err := proxyserver.Stop(statePath)
+	if err != nil {
+		ui.ErrorExit(err)
+	}
+
+	// Surface the daemon's cloud flush outcome before the violation gate, so it
+	// shows on both the success and the failure path (the daemon's own logs are
+	// not visible to this process). nil means cloud sync is disabled.
+	if res.CloudSync != nil {
+		line := fmt.Sprintf("Synced %d event(s) to SafeDep Cloud\n", res.CloudSync.Synced)
+		if res.CloudSync.Error != "" {
+			line = fmt.Sprintf("Cloud sync failed: %s\n", res.CloudSync.Error)
+		}
+		if _, werr := fmt.Fprint(os.Stdout, line); werr != nil {
+			ui.ErrorExit(werr)
+		}
+	}
+
+	// On a policy violation the framed error states the blocked count, so exit
+	// here before the plain summary to avoid stating the count twice.
+	if verr := stopExitError(res, failOnViolation); verr != nil {
+		ui.ErrorExit(verr)
+	}
+
+	summary := fmt.Sprintf("PMG proxy stopped — %d package(s) blocked\n", res.BlockedCount)
+	if !res.StateVerified {
+		summary = fmt.Sprintf("PMG proxy (pid %d) stopped (final state unavailable)\n", res.PID)
+	}
+	if _, werr := fmt.Fprint(os.Stdout, summary); werr != nil {
+		ui.ErrorExit(werr)
+	}
+
+	return nil
+}
+
+// stopExitError maps a stop result to the command's exit status. With
+// --fail-on-violation: any blocked package fails, and an unverifiable final
+// state (e.g. a crashed proxy) fails closed — a security gate must not pass on
+// an unverifiable run. Without the flag, stop always succeeds.
+func stopExitError(res proxyserver.StopResult, failOnViolation bool) error {
+	if !failOnViolation {
+		return nil
+	}
+
+	// HumanError drives the displayed message; Msg drives Error()/logs and the
+	// --verbose tail. Set both from one string so verbose shows the real
+	// message instead of usefulerror's "unknown error" fallback.
+	if !res.StateVerified {
+		msg := "Proxy shut down but the blocked-package count could not be verified"
+		return usefulerror.NewUsefulError().
+			WithCode(errcodes.ProxyPolicyViolation).
+			WithHumanError(msg).
+			WithMsg(msg).
+			WithHelp("The proxy may have crashed; treat this run as failed and re-run")
+	}
+
+	if res.BlockedCount > 0 {
+		msg := fmt.Sprintf("%d package(s) were blocked by the proxy", res.BlockedCount)
+		return usefulerror.NewUsefulError().
+			WithCode(errcodes.ProxyPolicyViolation).
+			WithHumanError(msg).
+			WithMsg(msg).
+			WithHelp("Review the proxy logs for details on blocked packages")
+	}
+
+	return nil
+}

cmd/proxy/stop_test.go

@@ -0,0 +1,38 @@
+package proxy
+
+import (
+	"testing"
+
+	"github.com/safedep/pmg/internal/proxyserver"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestStopExitError(t *testing.T) {
+	t.Run("no flag, blocks present -> nil", func(t *testing.T) {
+		err := stopExitError(proxyserver.StopResult{BlockedCount: 3, StateVerified: true}, false)
+		assert.NoError(t, err)
+	})
+
+	t.Run("flag, no blocks -> nil", func(t *testing.T) {
+		err := stopExitError(proxyserver.StopResult{BlockedCount: 0, StateVerified: true}, true)
+		assert.NoError(t, err)
+	})
+
+	t.Run("flag, blocks present -> error", func(t *testing.T) {
+		err := stopExitError(proxyserver.StopResult{BlockedCount: 2, StateVerified: true}, true)
+		require.Error(t, err)
+		assert.Contains(t, err.Error(), "2 package")
+	})
+
+	t.Run("flag, crash (unverified state) -> fail closed", func(t *testing.T) {
+		err := stopExitError(proxyserver.StopResult{StateVerified: false}, true)
+		require.Error(t, err)
+		assert.Contains(t, err.Error(), "could not be verified")
+	})
+
+	t.Run("no flag, crash -> nil", func(t *testing.T) {
+		err := stopExitError(proxyserver.StopResult{StateVerified: false}, false)
+		assert.NoError(t, err)
+	})
+}