mirror of
https://github.com/safedep/pmg.git
synced 2026-08-03 07:24:09 +02:00
* refactor: remove guard mode execution paths and guard-only packages Guard (non-proxy) mode is removed; all package-manager commands now always run the proxy flow. Removes the guard engine, the common flow, the extractor package, the npm/pypi dependency resolvers and the PackageResolver plumbing that only guard mode consumed. The guard package retains only PackageManagerGuardInteraction, which the proxy flow and confirmation interceptors reuse for user prompts. Proxy behavior is unchanged. * refactor: remove proxy opt-out surfaces, guard references in config, action and docs Removes Config.ProxyMode, ProxyConfig.Enabled, IsProxyModeEnabled, the proxy_mode legacy fallback, PMG_PROXY_ENABLED handling and the --proxy-mode / --include-dev-dependencies flags. Proxy interception can no longer be disabled. Also removes the proxy-mode input from the GitHub Action, the proxy-mode doctor check and setup info row, updates the E2E workflow to stop passing --proxy-mode=false, and sweeps guard-mode wording from docs and the config template. The legacy proxy_install_only flat key and PMG_PROXY_INSTALL_ONLY env var remain supported. audit.FlowTypeGuard is kept so previously recorded audit events still translate for cloud sync. * feat: fail loudly when a removed proxy opt-out is still configured A leftover proxy.enabled: false / proxy_mode: false config key or PMG_PROXY_ENABLED=false / PMG_PROXY_MODE=false env var previously meant guard mode; silently ignoring it would switch those users to proxy interception without notice. PMG now exits with an actionable error naming the exact source. Precedence mirrors the old resolution order: env (ignored under lockdown) > proxy.enabled > legacy proxy_mode. The pmg config subtree is exempt so the config file can still be fixed with pmg config edit/set. The GitHub Action's proxy-mode input is kept as a tombstone that fails the action when set to false and warns otherwise. * refactor: extract flows.RunProxy and address review findings Collapses the identical parse-then-run body duplicated across the 12 package manager commands into flows.RunProxy. Documents the cache-hit / offline analysis trade-off versus the removed guard manifest path, fixes a stale non-proxy label in the E2E workflow and a stale guard reference in the uvx parser comment. * fix(config): mirror old proxy opt-out precedence exactly PMG_PROXY_MODE only ever took effect through the legacy fallback, which was gated on the presence of a proxy: key in the config file (even a null one). Promoting it to the top env tier caused two inversions: a stale PMG_PROXY_MODE=false hard-failed configs that resolved to proxy mode, and PMG_PROXY_MODE=true silently overrode an explicit proxy.enabled: false file opt-out. The check now resolves in the old order: PMG_PROXY_ENABLED > proxy: section (presence gates the legacy tier) > PMG_PROXY_MODE > flat proxy_mode. parseOptOutBool also accepts numeric values (0 = false) to match viper's WeaklyTypedInput/cast.ToBool coercion, so proxy.enabled: 0 and proxy_mode: 0 are detected as opt-outs. * refactor: move package manager interaction out of guard * refactor: trim package manager interaction * fix(config): normalize config keys viper-style in proxy opt-out check Viper resolved config file keys case-insensitively and expanded dotted keys, so spellings like Proxy:, Enabled:, a literal proxy.enabled key or Proxy_Mode selected guard mode before the removal. The opt-out check now lowercases keys recursively and nests dotted keys before matching, so those existing opt-outs fail loudly instead of being silently ignored. * refactor: remove inert transitive controls, dead parser state and guard audit variant transitive / transitive_depth lost their only consumers with the dependency resolvers; remove the config fields, flags, template and doc entries, and the report/audit plumbing that misreported transitive analysis as enabled. Remove write-only parser state (PackageInstallTarget.Extras, ParsedCommand.ManifestFiles, ShouldExtractFromManifest); IsManifestInstall stays as it feeds sandbox gating via IsInstallationCommand. Remove audit.FlowTypeGuard and its cloud mapping; guard events recorded by pre-removal versions in an unsynced WAL translate to UNSPECIFIED. * fix: address review findings on the opt-out wiring and cleanups Move the removed-opt-out rejection from the CLI PersistentPreRun into proxyFlow.Run: the check now fires exactly for package-manager runs, so non-install commands (pmg setup remove, doctor, config, version) stay usable to fix or remove an opted-out installation, and future commands inherit or avoid the check by construction instead of by exemption list. Also: make the e2e malicious-package assertion actually fail the job when an install is not blocked, route pmg go through flows.RunProxy, and drop the dead extras return from pypiParsePackageInfo (extras are still stripped from package names). * fix(config): make the removed opt-out check faithful to the old resolution The gate that silenced the legacy proxy_mode surfaces matched the raw proxy key case-sensitively in the old code, while values resolved viper-style (case-insensitive, dotted keys); applying each semantic where the old code did fixes both divergences: a case-variant Proxy: section no longer hides a flat proxy_mode: false opt-out, and a dotted proxy.enabled: false overridden by proxy_mode: true no longer errors. Replace the generic key-tree normalization with two targeted lookups (the check only ever resolves proxy.enabled and proxy_mode), which also makes colliding spellings resolve deterministically. Coerce legacy-tier values cast.ToBool-style so PMG_PROXY_MODE=off style opt-outs are detected, log the config read error instead of swallowing it, and shorten the error to a one-line statement with the specific remedy in the help text. Add lockdown coverage (env inert both directions) and a repeated-run determinism test. * fix(config): fall back to defaults for unrecognized proxy opt-out values The old loader swallowed viper errors and ran on defaults, so values like proxy.enabled: yes or PMG_PROXY_ENABLED=banana silently discarded the whole config and defaulted to proxy. Treat them the same way now: unrecognized values mean the default (proxy on) instead of a hard error, and the doc comment no longer claims the old loader failed loudly. Only values that actually meant guard mode fail. Also check the removed opt-out before the CA trust check in pmg go, restoring the old error precedence: a config problem must not steer the user into an unnecessary OS trust store change. * fix(e2e): PMG_PROXY_MODE assertion must match the legacy gate semantics The runner's setup step writes the template config, which has a proxy: section — and with one present the legacy PMG_PROXY_MODE was always inert, so expecting a loud failure there asserts pre-fidelity-fix behavior. Assert both sides instead: inert (command succeeds) with the standard config, loud failure against an empty config dir where the legacy fallback actually applied. * refactor(config): collapse parseOptOutBool to ParseBool over the string form YAML hands us typed values (bool, int), so route them through fmt.Sprintf %v and strconv.ParseBool instead of a per-type switch. Identical behavior for every recognized value; numbers other than 0/1 now read as no opinion instead of cast.ToBool's nonzero-true, which no real config relies on.
312 lines
11 KiB
Go
312 lines
11 KiB
Go
package proxyserver
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"net"
|
|
"os"
|
|
"os/signal"
|
|
"strconv"
|
|
"syscall"
|
|
"time"
|
|
|
|
"github.com/safedep/dry/log"
|
|
"github.com/safedep/pmg/config"
|
|
"github.com/safedep/pmg/internal/audit"
|
|
"github.com/safedep/pmg/internal/flows"
|
|
"github.com/safedep/pmg/internal/localstore"
|
|
"github.com/safedep/pmg/internal/ui"
|
|
pmgproxy "github.com/safedep/pmg/proxy"
|
|
"github.com/safedep/pmg/proxy/certmanager"
|
|
"github.com/safedep/pmg/proxy/interceptors"
|
|
)
|
|
|
|
const (
|
|
serverStopTimeout = 5 * time.Second
|
|
|
|
// Periodic cloud sync runs while the daemon is alive so most audit events are
|
|
// delivered during the run and the shutdown flush stays small. A tick that
|
|
// cannot get the sync lock quickly is skipped (the next tick retries).
|
|
cloudSyncInterval = 15 * time.Second
|
|
cloudSyncTickLockWait = 5 * time.Second
|
|
cloudSyncTickTimeout = 30 * time.Second
|
|
|
|
// Final flush at shutdown, when the daemon drains whatever the ticker left.
|
|
cloudFlushLockWait = 30 * time.Second
|
|
cloudFlushTimeout = 2 * time.Minute
|
|
|
|
// daemonShutdownBudget is the worst-case time the daemon needs to shut down:
|
|
// drain in-flight requests, wait for an in-flight periodic tick to finish,
|
|
// then the final flush. `pmg proxy stop` waits at least this long for the
|
|
// daemon to exit; see stopWaitTimeout.
|
|
daemonShutdownBudget = serverStopTimeout +
|
|
cloudSyncTickLockWait + cloudSyncTickTimeout +
|
|
cloudFlushLockWait + cloudFlushTimeout
|
|
)
|
|
|
|
// DefaultDaemonReadyTimeout is how long the parent waits for the daemon to
|
|
// become ready before giving up, when ProxyDaemonConfig.ReadyTimeout is unset.
|
|
const DefaultDaemonReadyTimeout = 10 * time.Second
|
|
|
|
// ProxyDaemonConfig carries the daemon-launch parameters the caller decides, so
|
|
// daemonization stays free of config and path-policy concerns.
|
|
type ProxyDaemonConfig struct {
|
|
// LogPath is the file the detached daemon's stdout/stderr is redirected to.
|
|
// The caller owns this path (its parent directory must exist).
|
|
LogPath string
|
|
// ReadyTimeout bounds how long to wait for the daemon to write its state
|
|
// file and become live.
|
|
ReadyTimeout time.Duration
|
|
}
|
|
|
|
// Run starts the persistent proxy server in the foreground and blocks until it
|
|
// receives SIGINT/SIGTERM. It writes the state file on startup, auto-blocks
|
|
// suspicious packages, and records the final blocked count on shutdown.
|
|
func Run(ctx context.Context, cfg *config.RuntimeConfig, statePath, host string, port int) error {
|
|
if existing, err := readState(statePath); err == nil && existing.IsRunning() {
|
|
return fmt.Errorf("proxy already running (pid %d, addr %s) — run 'pmg proxy stop' first", existing.PID, existing.Addr)
|
|
}
|
|
|
|
startTime := time.Now()
|
|
|
|
caCertPath := certmanager.ProxyCABundlePath(cfg.ConfigDir())
|
|
caCert, _, err := flows.SetupCACertificate(cfg.ConfigDir(), caCertPath)
|
|
if err != nil {
|
|
return fmt.Errorf("setup CA certificate: %w", err)
|
|
}
|
|
|
|
certMgr, err := certmanager.NewCertificateManagerWithCA(caCert, certmanager.DefaultCertManagerConfig())
|
|
if err != nil {
|
|
return fmt.Errorf("create certificate manager: %w", err)
|
|
}
|
|
|
|
localDB := localstore.NewManager(cfg)
|
|
defer func() {
|
|
if cerr := localDB.Close(); cerr != nil {
|
|
log.Warnf("failed to close localdb: %v", cerr)
|
|
}
|
|
}()
|
|
|
|
malysisAnalyzer, err := flows.BuildMalysisAnalyzer(ctx, cfg, localDB)
|
|
if err != nil {
|
|
return fmt.Errorf("create analyzer: %w", err)
|
|
}
|
|
|
|
cache := interceptors.NewInMemoryAnalysisCache()
|
|
statsCollector := interceptors.NewAnalysisStatsCollector()
|
|
confirmationChan := make(chan *interceptors.ConfirmationRequest, 100)
|
|
go autoBlockConfirmations(confirmationChan)
|
|
|
|
factory := interceptors.NewInterceptorFactory(
|
|
malysisAnalyzer, cache, statsCollector, confirmationChan, interceptors.InterceptorContext{},
|
|
)
|
|
|
|
var interceptorList []pmgproxy.Interceptor
|
|
for _, eco := range interceptors.SupportedEcosystems() {
|
|
i, ferr := factory.CreateInterceptor(eco)
|
|
if ferr != nil {
|
|
return fmt.Errorf("create interceptor for %s: %w", eco.String(), ferr)
|
|
}
|
|
interceptorList = append(interceptorList, i)
|
|
}
|
|
interceptorList = append(interceptorList, interceptors.NewAuditLoggerInterceptor())
|
|
|
|
proxyConfig := pmgproxy.DefaultProxyConfig()
|
|
proxyConfig.ListenAddr = listenAddr(host, port)
|
|
proxyConfig.CertManager = certMgr
|
|
proxyConfig.Interceptors = interceptorList
|
|
presenter := ui.ProxyPresenter{Advisory: config.AdvisoryMessage}
|
|
proxyConfig.BlockMessageRenderer = presenter.BlockMessage
|
|
|
|
server, err := pmgproxy.NewProxyServer(proxyConfig)
|
|
if err != nil {
|
|
return fmt.Errorf("create proxy server: %w", err)
|
|
}
|
|
|
|
if err := server.Start(); err != nil {
|
|
return fmt.Errorf("start proxy server: %w", err)
|
|
}
|
|
|
|
state := State{
|
|
PID: os.Getpid(),
|
|
Addr: server.Address(),
|
|
CACertPath: caCertPath,
|
|
}
|
|
if err := writeState(statePath, state); err != nil {
|
|
stopCtx, cancel := context.WithTimeout(context.Background(), serverStopTimeout)
|
|
defer cancel()
|
|
if serr := server.Stop(stopCtx); serr != nil {
|
|
log.Warnf("failed to stop proxy after state write failure: %v", serr)
|
|
}
|
|
return fmt.Errorf("write proxy state: %w", err)
|
|
}
|
|
|
|
log.Infof("PMG persistent proxy running on %s (pid %d)", state.Addr, state.PID)
|
|
if _, err := fmt.Fprintf(os.Stderr, "PMG proxy running on %s\nRun: export $(pmg proxy env | xargs) # or: pmg proxy env >> \"$GITHUB_ENV\"\n", state.Addr); err != nil {
|
|
log.Warnf("failed to write startup message: %v", err)
|
|
}
|
|
|
|
// Periodically flush events to the cloud while serving so the shutdown flush
|
|
// stays small. See startCloudSyncLoop for the stop-function contract.
|
|
stopSyncLoop := startCloudSyncLoop(cfg)
|
|
|
|
sigCh := make(chan os.Signal, 1)
|
|
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
|
|
<-sigCh
|
|
|
|
// Drain in-flight requests before closing the confirmation channel, so no
|
|
// request handler can send on a closed channel (panic) during shutdown.
|
|
stopCtx, cancel := context.WithTimeout(context.Background(), serverStopTimeout)
|
|
defer cancel()
|
|
stopErr := server.Stop(stopCtx)
|
|
|
|
close(confirmationChan)
|
|
|
|
// Stats are read after drain so a package analyzed at shutdown is not missed.
|
|
// Persist the blocked count BEFORE the (possibly slow) cloud flush so it
|
|
// survives even if the flush hangs or the daemon is killed mid-flush, which
|
|
// keeps `stop --fail-on-violation` correct in those cases.
|
|
stats := statsCollector.GetStats()
|
|
state.BlockedCount = stats.BlockedCount
|
|
if werr := writeState(statePath, state); werr != nil {
|
|
log.Warnf("failed to write final proxy state: %v", werr)
|
|
}
|
|
|
|
// Emit the daemon-lifetime session summary before the final flush so it is
|
|
// delivered alongside the run's other events. Unlike the per-invocation flow,
|
|
// the daemon serves every package manager, so the summary carries no single
|
|
// package manager.
|
|
logSessionSummary(cfg, stats, time.Since(startTime))
|
|
|
|
// Halt the periodic sync (waits for any in-flight drain) before the final
|
|
// flush, so the two never hold the sync lock at once.
|
|
periodicSynced := stopSyncLoop()
|
|
|
|
if cs := cloudFlush(cfg, periodicSynced); cs != nil {
|
|
state.CloudSync = cs
|
|
if werr := writeState(statePath, state); werr != nil {
|
|
log.Warnf("failed to write final proxy state: %v", werr)
|
|
}
|
|
}
|
|
|
|
return stopErr
|
|
}
|
|
|
|
// logSessionSummary emits an aggregate session-complete audit event for the
|
|
// daemon's lifetime, mapping the proxy stats collector onto SessionData. The
|
|
// outcome is blocked when anything was blocked, otherwise success.
|
|
func logSessionSummary(cfg *config.RuntimeConfig, stats interceptors.AnalysisStats, duration time.Duration) {
|
|
outcome := audit.OutcomeSuccess
|
|
if stats.BlockedCount > 0 {
|
|
outcome = audit.OutcomeBlocked
|
|
}
|
|
|
|
audit.LogSessionSummary(audit.SessionData{
|
|
FlowType: audit.FlowTypeProxy,
|
|
Outcome: outcome,
|
|
TotalAnalyzed: uint32(stats.TotalAnalyzed),
|
|
AllowedCount: uint32(stats.AllowedCount),
|
|
BlockedCount: uint32(stats.BlockedCount),
|
|
ConfirmedCount: uint32(stats.ConfirmedCount),
|
|
CooldownBlockedCount: uint32(stats.CooldownBlockedCount),
|
|
Duration: duration,
|
|
SandboxEnabled: cfg.Config.Sandbox.Enabled,
|
|
ParanoidMode: cfg.Config.Paranoid,
|
|
})
|
|
}
|
|
|
|
// cloudFlush drains whatever the periodic sync left and returns the outcome
|
|
// (total delivered, including periodicSynced). Returns nil when automatic cloud
|
|
// delivery is off (cloud or auto-sync disabled) — same gate as the periodic
|
|
// ticker, so auto_sync consistently controls all daemon-driven cloud delivery.
|
|
// The daemon does this itself rather than `pmg proxy stop` because, unlike stop,
|
|
// it has no proxy env vars and so dials SafeDep directly instead of routing
|
|
// through the proxy that is now shutting down. Uses a fresh context since the
|
|
// caller's may already be cancelled at shutdown.
|
|
func cloudFlush(cfg *config.RuntimeConfig, periodicSynced int) *CloudSyncResult {
|
|
if !cfg.Config.Cloud.Enabled || !cfg.Config.Cloud.AutoSync.Enabled {
|
|
return nil
|
|
}
|
|
|
|
synced, err := audit.DrainToCloud(context.Background(), cfg, cloudFlushLockWait, cloudFlushTimeout)
|
|
res := &CloudSyncResult{Synced: periodicSynced + synced}
|
|
if err != nil {
|
|
res.Error = err.Error()
|
|
log.Warnf("cloud event flush failed: %v", err)
|
|
} else {
|
|
log.Infof("Flushed %d events to SafeDep Cloud (%d during the run)", res.Synced, periodicSynced)
|
|
}
|
|
return res
|
|
}
|
|
|
|
// startCloudSyncLoop periodically drains pending audit events to SafeDep Cloud
|
|
// while the daemon runs. It returns a stop function that halts the ticker, waits
|
|
// for any in-flight drain to finish, and returns the running total of events
|
|
// delivered. The stop function must be called before the daemon's final flush so
|
|
// the two never hold the sync lock at once. A no-op when cloud sync or auto-sync
|
|
// is disabled; the daemon's automatic cloud delivery (periodic ticker and
|
|
// shutdown flush alike) honors the auto_sync flag.
|
|
func startCloudSyncLoop(cfg *config.RuntimeConfig) func() int {
|
|
if !cfg.Config.Cloud.Enabled || !cfg.Config.Cloud.AutoSync.Enabled {
|
|
return func() int { return 0 }
|
|
}
|
|
|
|
stop := make(chan struct{})
|
|
done := make(chan struct{})
|
|
var total int // written only by the goroutine; read after <-done (happens-before)
|
|
|
|
go func() {
|
|
defer close(done)
|
|
ticker := time.NewTicker(cloudSyncInterval)
|
|
defer ticker.Stop()
|
|
|
|
for {
|
|
select {
|
|
case <-stop:
|
|
return
|
|
case <-ticker.C:
|
|
synced, err := audit.DrainToCloud(context.Background(), cfg, cloudSyncTickLockWait, cloudSyncTickTimeout)
|
|
if err != nil {
|
|
if errors.Is(err, audit.ErrSyncInProgress) {
|
|
log.Debugf("periodic cloud sync skipped: another sync in progress")
|
|
} else {
|
|
log.Warnf("periodic cloud sync failed: %v", err)
|
|
}
|
|
continue
|
|
}
|
|
total += synced
|
|
if synced > 0 {
|
|
log.Infof("Periodic cloud sync: flushed %d events", synced)
|
|
}
|
|
}
|
|
}
|
|
}()
|
|
|
|
return func() int {
|
|
close(stop)
|
|
<-done
|
|
return total
|
|
}
|
|
}
|
|
|
|
// listenAddr resolves the proxy's bind address from config (host) and the
|
|
// --port flag. Host defaults to loopback.
|
|
func listenAddr(host string, port int) string {
|
|
if host == "" {
|
|
host = "127.0.0.1"
|
|
}
|
|
|
|
return net.JoinHostPort(host, strconv.Itoa(port))
|
|
}
|
|
|
|
// autoBlockConfirmations drains the confirmation channel and always denies,
|
|
// appropriate for non-interactive CI/CD environments.
|
|
func autoBlockConfirmations(ch chan *interceptors.ConfirmationRequest) {
|
|
for req := range ch {
|
|
log.Warnf("Persistent proxy: auto-blocking suspicious package %s", req.PackageVersion.GetPackage().GetName())
|
|
req.ResponseChan <- false
|
|
close(req.ResponseChan)
|
|
}
|
|
}
|