External/pmg
1
0
Fork 0
mirror of https://github.com/safedep/pmg.git synced 2026-08-03 07:24:09 +02:00
Code Issues Packages Projects Releases Wiki Activity

Remove guard mode: proxy interception is now the only flow (#386)

Browse Source 
* refactor: remove guard mode execution paths and guard-only packages

Guard (non-proxy) mode is removed; all package-manager commands now
always run the proxy flow. Removes the guard engine, the common flow,
the extractor package, the npm/pypi dependency resolvers and the
PackageResolver plumbing that only guard mode consumed.

The guard package retains only PackageManagerGuardInteraction, which
the proxy flow and confirmation interceptors reuse for user prompts.
Proxy behavior is unchanged.

* refactor: remove proxy opt-out surfaces, guard references in config, action and docs

Removes Config.ProxyMode, ProxyConfig.Enabled, IsProxyModeEnabled, the
proxy_mode legacy fallback, PMG_PROXY_ENABLED handling and the
--proxy-mode / --include-dev-dependencies flags. Proxy interception can
no longer be disabled.

Also removes the proxy-mode input from the GitHub Action, the
proxy-mode doctor check and setup info row, updates the E2E workflow to
stop passing --proxy-mode=false, and sweeps guard-mode wording from
docs and the config template.

The legacy proxy_install_only flat key and PMG_PROXY_INSTALL_ONLY env
var remain supported. audit.FlowTypeGuard is kept so previously
recorded audit events still translate for cloud sync.

* feat: fail loudly when a removed proxy opt-out is still configured

A leftover proxy.enabled: false / proxy_mode: false config key or
PMG_PROXY_ENABLED=false / PMG_PROXY_MODE=false env var previously meant
guard mode; silently ignoring it would switch those users to proxy
interception without notice. PMG now exits with an actionable error
naming the exact source. Precedence mirrors the old resolution order:
env (ignored under lockdown) > proxy.enabled > legacy proxy_mode.

The pmg config subtree is exempt so the config file can still be fixed
with pmg config edit/set. The GitHub Action's proxy-mode input is kept
as a tombstone that fails the action when set to false and warns
otherwise.

* refactor: extract flows.RunProxy and address review findings

Collapses the identical parse-then-run body duplicated across the 12
package manager commands into flows.RunProxy. Documents the cache-hit /
offline analysis trade-off versus the removed guard manifest path,
fixes a stale non-proxy label in the E2E workflow and a stale guard
reference in the uvx parser comment.

* fix(config): mirror old proxy opt-out precedence exactly

PMG_PROXY_MODE only ever took effect through the legacy fallback, which
was gated on the presence of a proxy: key in the config file (even a
null one). Promoting it to the top env tier caused two inversions: a
stale PMG_PROXY_MODE=false hard-failed configs that resolved to proxy
mode, and PMG_PROXY_MODE=true silently overrode an explicit
proxy.enabled: false file opt-out. The check now resolves in the old
order: PMG_PROXY_ENABLED > proxy: section (presence gates the legacy
tier) > PMG_PROXY_MODE > flat proxy_mode.

parseOptOutBool also accepts numeric values (0 = false) to match
viper's WeaklyTypedInput/cast.ToBool coercion, so proxy.enabled: 0 and
proxy_mode: 0 are detected as opt-outs.

* refactor: move package manager interaction out of guard

* refactor: trim package manager interaction

* fix(config): normalize config keys viper-style in proxy opt-out check

Viper resolved config file keys case-insensitively and expanded dotted
keys, so spellings like Proxy:, Enabled:, a literal proxy.enabled key or
Proxy_Mode selected guard mode before the removal. The opt-out check now
lowercases keys recursively and nests dotted keys before matching, so
those existing opt-outs fail loudly instead of being silently ignored.

* refactor: remove inert transitive controls, dead parser state and guard audit variant

transitive / transitive_depth lost their only consumers with the
dependency resolvers; remove the config fields, flags, template and doc
entries, and the report/audit plumbing that misreported transitive
analysis as enabled.

Remove write-only parser state (PackageInstallTarget.Extras,
ParsedCommand.ManifestFiles, ShouldExtractFromManifest); IsManifestInstall
stays as it feeds sandbox gating via IsInstallationCommand.

Remove audit.FlowTypeGuard and its cloud mapping; guard events recorded
by pre-removal versions in an unsynced WAL translate to UNSPECIFIED.

* fix: address review findings on the opt-out wiring and cleanups

Move the removed-opt-out rejection from the CLI PersistentPreRun into
proxyFlow.Run: the check now fires exactly for package-manager runs, so
non-install commands (pmg setup remove, doctor, config, version) stay
usable to fix or remove an opted-out installation, and future commands
inherit or avoid the check by construction instead of by exemption list.

Also: make the e2e malicious-package assertion actually fail the job
when an install is not blocked, route pmg go through flows.RunProxy,
and drop the dead extras return from pypiParsePackageInfo (extras are
still stripped from package names).

* fix(config): make the removed opt-out check faithful to the old resolution

The gate that silenced the legacy proxy_mode surfaces matched the raw
proxy key case-sensitively in the old code, while values resolved
viper-style (case-insensitive, dotted keys); applying each semantic
where the old code did fixes both divergences: a case-variant Proxy:
section no longer hides a flat proxy_mode: false opt-out, and a dotted
proxy.enabled: false overridden by proxy_mode: true no longer errors.

Replace the generic key-tree normalization with two targeted lookups
(the check only ever resolves proxy.enabled and proxy_mode), which also
makes colliding spellings resolve deterministically. Coerce legacy-tier
values cast.ToBool-style so PMG_PROXY_MODE=off style opt-outs are
detected, log the config read error instead of swallowing it, and
shorten the error to a one-line statement with the specific remedy in
the help text.

Add lockdown coverage (env inert both directions) and a repeated-run
determinism test.

* fix(config): fall back to defaults for unrecognized proxy opt-out values

The old loader swallowed viper errors and ran on defaults, so values
like proxy.enabled: yes or PMG_PROXY_ENABLED=banana silently discarded
the whole config and defaulted to proxy. Treat them the same way now:
unrecognized values mean the default (proxy on) instead of a hard
error, and the doc comment no longer claims the old loader failed
loudly. Only values that actually meant guard mode fail.

Also check the removed opt-out before the CA trust check in pmg go,
restoring the old error precedence: a config problem must not steer
the user into an unnecessary OS trust store change.

* fix(e2e): PMG_PROXY_MODE assertion must match the legacy gate semantics

The runner's setup step writes the template config, which has a proxy:
section — and with one present the legacy PMG_PROXY_MODE was always
inert, so expecting a loud failure there asserts pre-fidelity-fix
behavior. Assert both sides instead: inert (command succeeds) with the
standard config, loud failure against an empty config dir where the
legacy fallback actually applied.

* refactor(config): collapse parseOptOutBool to ParseBool over the string form

YAML hands us typed values (bool, int), so route them through
fmt.Sprintf %v and strconv.ParseBool instead of a per-type switch.
Identical behavior for every recognized value; numbers other than 0/1
now read as no opinion instead of cast.ToBool's nonzero-true, which no
real config relies on.
This commit is contained in:
Sahil Bansal
2026-07-22 15:19:19 +05:30
committed by GitHub
parent 3031665ac7
commit 94781d6bda
80 changed files with 669 additions and 3036 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packagemanager/dependency_resolver.go
-229
View File
@@ -1,229 +0,0 @@
package packagemanager
import (
"context"
"fmt"
"sync"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/safedep/dry/log"
"github.com/safedep/dry/packageregistry"
)
// Contract for a function that implements ecosystem specific version
// resolver from a version range specification.
type versionSpecResolverFn func(packageName, version string) string
type dependencyResolverFn func(packageName, version string) (*packageregistry.PackageDependencyList, error)
type packageIdentifierFn func(pkg *packagev1.PackageVersion) string
type dependencyResolverConfig struct {
IncludeDevDependencies bool
IncludeTransitiveDependencies bool
TransitiveDepth int
FailFast bool
MaxConcurrency int
}
type dependencyResolver struct {
client packageregistry.Client
config dependencyResolverConfig
mutex sync.Mutex
versionSpecResolver versionSpecResolverFn
packageDependencyResolver dependencyResolverFn
packageIdentifierFn packageIdentifierFn
resultSet map[string]bool
}
func newDependencyResolver(client packageregistry.Client, config dependencyResolverConfig,
versionSpecResolver versionSpecResolverFn, packageDependencyResolver dependencyResolverFn, packageKeyFn packageIdentifierFn) *dependencyResolver {
if config.MaxConcurrency <= 0 {
config.MaxConcurrency = 10
}
if versionSpecResolver == nil {
// Default version spec resolver
versionSpecResolver = func(packageName, version string) string {
return version
}
}
return &dependencyResolver{
client: client,
config: config,
versionSpecResolver: versionSpecResolver,
packageDependencyResolver: packageDependencyResolver,
packageIdentifierFn: packageKeyFn,
resultSet: make(map[string]bool),
}
}
func (r *dependencyResolver) resolveDependencies(ctx context.Context,
packageVersion *packagev1.PackageVersion) ([]*packagev1.PackageVersion, error) {
pd, err := r.client.PackageDiscovery()
if err != nil {
return nil, fmt.Errorf("failed to get package discovery: %w", err)
}
// Track visited packages to avoid cycles
visitedPackages := make(map[string]bool)
// Result collection
dependencies := make([]*packagev1.PackageVersion, 0)
r.resultSet = make(map[string]bool) // Reset
// Start concurrent resolution
err = r.resolvePackageDependenciesConcurrent(ctx, pd, packageVersion, 0, visitedPackages, &dependencies)
if err != nil {
return nil, fmt.Errorf("failed to resolve dependencies: %w", err)
}
return dependencies, nil
}
// resolvePackageDependenciesConcurrent resolves dependencies for a package version concurrently
func (r *dependencyResolver) resolvePackageDependenciesConcurrent(
ctx context.Context,
pd packageregistry.PackageDiscovery,
packageVersion *packagev1.PackageVersion,
depth int,
visitedPackages map[string]bool,
result *[]*packagev1.PackageVersion) error {
// Check for context cancellation
select {
case <-ctx.Done():
return ctx.Err()
default:
}
ff := func(err error) error {
if r.config.FailFast {
return err
}
log.Warnf("error resolving package dependencies: %s", err)
return nil
}
// Check depth limit
if depth > r.config.TransitiveDepth {
return ff(fmt.Errorf("exceeded maximum transitive depth of %d", r.config.TransitiveDepth))
}
var packageKey string
var packageKeyFn packageIdentifierFn
// Skip if already visited
if r.packageIdentifierFn != nil {
packageKeyFn = r.packageIdentifierFn
} else {
packageKeyFn = createPackageKey
}
packageKey = packageKeyFn(packageVersion)
shouldProcess := false
r.synchronize(func() {
if !visitedPackages[packageKey] {
visitedPackages[packageKey] = true
shouldProcess = true
}
})
// If another goroutine is already processing this package, skip
if !shouldProcess {
return nil
}
log.Debugf("resolving dependencies for %s@%s", packageVersion.Package.Name, packageVersion.Version)
// Get dependencies for the current package
var dependencyList *packageregistry.PackageDependencyList
var err error
if r.packageDependencyResolver != nil {
dependencyList, err = r.packageDependencyResolver(packageVersion.Package.Name, packageVersion.Version)
} else {
dependencyList, err = pd.GetPackageDependencies(packageVersion.Package.Name, packageVersion.Version)
}
if err != nil {
return ff(fmt.Errorf("failed to get package dependencies: %w", err))
}
// Collect all dependencies (and optionally dev dependencies)
dependencies := dependencyList.Dependencies
if r.config.IncludeDevDependencies {
dependencies = append(dependencies, dependencyList.DevDependencies...)
}
// Create package version objects for all dependencies and clean versions
resolvedDependencies := make([]*packagev1.PackageVersion, 0, len(dependencies))
for _, dependency := range dependencies {
resolvedDependencies = append(resolvedDependencies, &packagev1.PackageVersion{
Package: &packagev1.Package{
Ecosystem: packageVersion.GetPackage().GetEcosystem(),
Name: dependency.Name,
},
Version: r.versionSpecResolver(dependency.Name, dependency.VersionSpec),
})
}
// Add resolved dependencies to the result
r.synchronize(func() {
for _, dependency := range resolvedDependencies {
dependencyKey := packageKeyFn(dependency)
if !r.resultSet[dependencyKey] {
r.resultSet[dependencyKey] = true
*result = append(*result, dependency)
}
}
})
// Process transitive dependencies if enabled and depth limit not reached
if r.config.IncludeTransitiveDependencies && depth < r.config.TransitiveDepth && len(resolvedDependencies) > 0 {
// Create worker pool using semaphore pattern
semaphore := make(chan struct{}, r.config.MaxConcurrency)
errCh := make(chan error, len(resolvedDependencies))
var wg sync.WaitGroup
for _, dependency := range resolvedDependencies {
wg.Add(1)
go func(dep *packagev1.PackageVersion) {
defer wg.Done()
semaphore <- struct{}{}
defer func() { <-semaphore }()
err := r.resolvePackageDependenciesConcurrent(ctx, pd, dep, depth+1, visitedPackages, result)
if err != nil {
errCh <- err
}
}(dependency)
}
// Wait for all goroutines to finish
wg.Wait()
close(errCh)
// Check for errors
for err := range errCh {
return ff(fmt.Errorf("failed to resolve transitive dependency: %w", err))
}
}
return nil
}
func createPackageKey(pkg *packagev1.PackageVersion) string {
return fmt.Sprintf("%s@%s", pkg.Package.Name, pkg.Version)
}
func (r *dependencyResolver) synchronize(fn func()) {
r.mutex.Lock()
defer r.mutex.Unlock()
fn()
}
packagemanager/errors.go
-17
View File
@@ -6,29 +6,12 @@ import (
)
var (
ErrPackageNotFound = usefulerror.NewUsefulError().
WithCode(errcodes.NotFound).
WithHumanError("The requested package could not be found.").
WithHelp("Please check the package name and try again.")
ErrFailedToFetchPackage = usefulerror.NewUsefulError().
WithCode(errcodes.Network).
WithHumanError("Failed to retrieve the requested package.").
WithHelp("Check your network connection and try again.").
WithMsg("failed to fetch package")
ErrFailedToResolveVersion = usefulerror.NewUsefulError().
WithCode(errcodes.Network).
WithHumanError("Failed to resolve the requested package version.").
WithHelp("Check your network connection and try again.").
WithMsg("failed to resolve package version")
ErrFailedToResolveDependencies = usefulerror.NewUsefulError().
WithCode(errcodes.DependencyResolutionFailed).
WithHumanError("Failed to resolve dependencies.").
WithHelp("Check your network connection and try again.").
WithMsg("failed to resolve dependencies")
ErrFailedToParsePackage = usefulerror.NewUsefulError().
WithCode(errcodes.PackageParseFailed).
WithHumanError("The package data could not be processed.").
packagemanager/golang.go
-2
View File
@@ -76,12 +76,10 @@ func (g *goPackageManager) ParseCommand(args []string) (*ParsedCommand, error) {
switch modCmd {
case "tidy":
parsed.IsManifestInstall = true
parsed.ManifestFiles = []string{"go.mod"}
case "download":
parsed.InstallTargets = goRemoteModuleTargets(modRest)
if len(parsed.InstallTargets) == 0 {
parsed.IsManifestInstall = true
parsed.ManifestFiles = []string{"go.mod"}
}
}
}
packagemanager/golang_test.go
+11 -15
View File
@@ -16,12 +16,11 @@ func TestGoPackageManagerParseCommand(t *testing.T) {
}
cases := []struct {
name string
args []string
nonDownload bool
manifestInstall bool
targets []target
wantManifestFiles []string
name string
args []string
nonDownload bool
manifestInstall bool
targets []target
}{
{
name: "go version is non-download",
@@ -88,16 +87,14 @@ func TestGoPackageManagerParseCommand(t *testing.T) {
targets: []target{{name: "example.com/m", version: "v2.0.0", explicit: true}},
},
{
name: "go mod tidy is manifest install",
args: []string{"go", "mod", "tidy"},
manifestInstall: true,
wantManifestFiles: []string{"go.mod"},
name: "go mod tidy is manifest install",
args: []string{"go", "mod", "tidy"},
manifestInstall: true,
},
{
name: "go mod download without args is manifest install",
args: []string{"go", "mod", "download"},
manifestInstall: true,
wantManifestFiles: []string{"go.mod"},
name: "go mod download without args is manifest install",
args: []string{"go", "mod", "download"},
manifestInstall: true,
},
{
name: "go mod download with module",
@@ -123,7 +120,6 @@ func TestGoPackageManagerParseCommand(t *testing.T) {
assert.Equal(t, tc.nonDownload, parsed.IsKnownNonDownloadCommand)
assert.Equal(t, tc.manifestInstall, parsed.IsManifestInstall)
assert.Equal(t, tc.wantManifestFiles, parsed.ManifestFiles)
require.Len(t, parsed.InstallTargets, len(tc.targets))
for i, want := range tc.targets {
packagemanager/noop_resolver.go
-25
View File
@@ -1,25 +0,0 @@
package packagemanager
import (
"context"
"fmt"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
)
// noopPackageResolver satisfies PackageResolver for flows that never resolve
// dependencies up front, such as the proxy flow where every download is
// intercepted and analyzed on the wire.
type noopPackageResolver struct{}
func NewNoopPackageResolver() PackageResolver {
return noopPackageResolver{}
}
func (noopPackageResolver) ResolveLatestVersion(context.Context, *packagev1.Package) (*packagev1.PackageVersion, error) {
return nil, fmt.Errorf("package resolution is not supported by the noop resolver")
}
func (noopPackageResolver) ResolveDependencies(context.Context, *packagev1.PackageVersion) ([]*packagev1.PackageVersion, error) {
return nil, fmt.Errorf("dependency resolution is not supported by the noop resolver")
}
packagemanager/npm.go
-2
View File
@@ -102,7 +102,6 @@ func (npm *npmPackageManager) ParseCommand(args []string) (*ParsedCommand, error
Command: command,
InstallTargets: []*PackageInstallTarget{},
IsManifestInstall: true,
ManifestFiles: []string{},
}, nil
}
@@ -205,7 +204,6 @@ func (npm *npmPackageManager) ParseCommand(args []string) (*ParsedCommand, error
Command: command,
InstallTargets: installTargets,
IsManifestInstall: isManifestInstall,
ManifestFiles: []string{},
}, nil
}
packagemanager/npm_resolver.go
-89
View File
@@ -1,89 +0,0 @@
package packagemanager
import (
"context"
"fmt"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/safedep/dry/log"
"github.com/safedep/dry/packageregistry"
)
type NpmDependencyResolverConfig struct {
IncludeDevDependencies bool
IncludeTransitiveDependencies bool
TransitiveDepth int
// FailFast will stop resolving dependencies after the first error
FailFast bool
// MaxConcurrency limits the number of concurrent goroutines used for dependency resolution
MaxConcurrency int
}
func NewDefaultNpmDependencyResolverConfig() NpmDependencyResolverConfig {
return NpmDependencyResolverConfig{
IncludeDevDependencies: false,
IncludeTransitiveDependencies: true,
TransitiveDepth: 5,
FailFast: false,
MaxConcurrency: 10,
}
}
type npmDependencyResolver struct {
registry packageregistry.Client
config NpmDependencyResolverConfig
}
var _ PackageResolver = &npmDependencyResolver{}
func NewNpmDependencyResolver(config NpmDependencyResolverConfig) (*npmDependencyResolver, error) {
client, err := packageregistry.NewNpmAdapter()
if err != nil {
return nil, fmt.Errorf("failed to create npm adapter: %w", err)
}
return &npmDependencyResolver{
registry: client,
config: config,
}, nil
}
func (r *npmDependencyResolver) ResolveLatestVersion(ctx context.Context,
pkg *packagev1.Package) (*packagev1.PackageVersion, error) {
pd, err := r.registry.PackageDiscovery()
if err != nil {
return nil, fmt.Errorf("failed to get package discovery: %w", err)
}
pkgInfo, err := pd.GetPackage(pkg.Name)
if err != nil {
return nil, ErrFailedToFetchPackage.Wrap(err)
}
log.Debugf("Resolved npm/%s to latest version %s", pkg.Name, pkgInfo.LatestVersion)
return &packagev1.PackageVersion{
Package: pkg,
Version: pkgInfo.LatestVersion,
}, nil
}
func (r *npmDependencyResolver) ResolveDependencies(ctx context.Context,
packageVersion *packagev1.PackageVersion) ([]*packagev1.PackageVersion, error) {
npmVersionSpecResolverFn := func(packageName, version string) string {
return npmCleanVersion(version)
}
resolver := newDependencyResolver(r.registry, dependencyResolverConfig{
IncludeDevDependencies: r.config.IncludeDevDependencies,
IncludeTransitiveDependencies: r.config.IncludeTransitiveDependencies,
TransitiveDepth: r.config.TransitiveDepth,
FailFast: r.config.FailFast,
MaxConcurrency: r.config.MaxConcurrency,
}, npmVersionSpecResolverFn, nil, nil)
return resolver.resolveDependencies(ctx, packageVersion)
}
packagemanager/npm_resolver_test.go
-167
View File
@@ -1,167 +0,0 @@
package packagemanager
import (
"context"
"testing"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/safedep/dry/semver"
"github.com/stretchr/testify/require"
)
func TestNpmDependencyResolver_ResolveLatestVersion(t *testing.T) {
cases := []struct {
name string
pkg *packagev1.Package
assertFn func(t *testing.T, pv *packagev1.PackageVersion, err error)
}{
{
name: "should resolve latest version for a package",
pkg: &packagev1.Package{
Name: "react",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
assertFn: func(t *testing.T, pv *packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.True(t, semver.IsAhead("19.0.0", pv.Version))
},
},
{
name: "should return an error if the package is not found",
pkg: &packagev1.Package{
Name: "nonexistent",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
assertFn: func(t *testing.T, pv *packagev1.PackageVersion, err error) {
require.Error(t, err)
require.Nil(t, pv)
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
resolver, err := NewNpmDependencyResolver(NewDefaultNpmDependencyResolverConfig())
require.NoError(t, err)
pv, err := resolver.ResolveLatestVersion(context.Background(), tc.pkg)
tc.assertFn(t, pv, err)
})
}
}
func TestNpmDependencyResolver_ResolveDependencies(t *testing.T) {
cases := []struct {
name string
pkg *packagev1.PackageVersion
includeTransitiveDependencies bool
transitiveDepth int
failFast bool
assertFn func(t *testing.T, dependencies []*packagev1.PackageVersion, err error)
}{
{
name: "should resolve dependencies for a package when transitive dependencies are not included",
pkg: &packagev1.PackageVersion{
Package: &packagev1.Package{
Name: "react",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
Version: "18.2.0",
},
includeTransitiveDependencies: false,
transitiveDepth: 1,
assertFn: func(t *testing.T, dependencies []*packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.Equal(t, 1, len(dependencies))
require.Equal(t, "loose-envify", dependencies[0].Package.Name)
require.Equal(t, "1.1.0", dependencies[0].Version)
},
},
{
name: "should resolve dependencies for a package up to a given depth",
pkg: &packagev1.PackageVersion{
Package: &packagev1.Package{
Name: "react",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
Version: "18.2.0",
},
includeTransitiveDependencies: true,
transitiveDepth: 2,
assertFn: func(t *testing.T, dependencies []*packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.Equal(t, 2, len(dependencies))
packageNames := []string{}
for _, dep := range dependencies {
packageNames = append(packageNames, dep.Package.Name)
}
require.ElementsMatch(t, []string{
"loose-envify",
"js-tokens",
}, packageNames)
},
},
{
name: "should resolve all dependencies for a package when transitive dependencies are included",
pkg: &packagev1.PackageVersion{
Package: &packagev1.Package{
Name: "express",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
Version: "4.18.2",
},
includeTransitiveDependencies: true,
transitiveDepth: 5,
assertFn: func(t *testing.T, dependencies []*packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.Greater(t, len(dependencies), 5, "Express should have more than 5 dependencies")
},
},
{
name: "should not fail when package is not found without fail fast",
pkg: &packagev1.PackageVersion{
Package: &packagev1.Package{
Name: "nonexistent",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
Version: "1.0.0",
},
assertFn: func(t *testing.T, dependencies []*packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.Empty(t, dependencies)
},
},
{
name: "should fail when package is not found with fail fast",
pkg: &packagev1.PackageVersion{
Package: &packagev1.Package{
Name: "nonexistent",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_NPM,
},
Version: "1.0.0",
},
failFast: true,
assertFn: func(t *testing.T, dependencies []*packagev1.PackageVersion, err error) {
require.Error(t, err)
require.Nil(t, dependencies)
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
config := NewDefaultNpmDependencyResolverConfig()
config.IncludeTransitiveDependencies = tc.includeTransitiveDependencies
config.TransitiveDepth = tc.transitiveDepth
config.FailFast = tc.failFast
resolver, err := NewNpmDependencyResolver(config)
require.NoError(t, err)
dependencies, err := resolver.ResolveDependencies(context.Background(), tc.pkg)
tc.assertFn(t, dependencies, err)
})
}
}
packagemanager/packagemanager.go
+25 -24
View File
@@ -1,11 +1,13 @@
package packagemanager
import (
"context"
"io"
"os"
"slices"
"strings"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/safedep/pmg/analyzer"
)
type Command struct {
@@ -16,11 +18,6 @@ type Command struct {
type PackageInstallTarget struct {
PackageVersion *packagev1.PackageVersion
// Extras specifies additional features to be installed with a Python package
// Example: "django[mysql,redis]" has Extras as ["mysql", "redis"]
// Currently only specific to Python packages
Extras []string
// IsExplicitVersion indicates the user provided an explicit version constraint
// (e.g. ==1.2.3) as opposed to the version being auto-resolved by the resolver.
IsExplicitVersion bool
@@ -41,10 +38,6 @@ type ParsedCommand struct {
// (e.g., npm install, pip install -r requirements.txt)
IsManifestInstall bool
// ManifestFiles contains the list of manifest files to install from
// (e.g., ["requirements.txt"] for pip install -r requirements.txt)
ManifestFiles []string
// IsKnownNonDownloadCommand is true for commands that are known to not download packages
// (e.g., npm ls, pip list, yarn why). Used by the proxy to decide whether to skip
// interception when proxy.install_only is enabled. Unknown commands default to false so
@@ -53,7 +46,6 @@ type ParsedCommand struct {
}
// IsInstallationCommand returns true if command installs packages (explicit targets or from manifest).
// This is used by guard mode where we need to know which packages are being installed.
func (pc *ParsedCommand) IsInstallationCommand() bool {
return pc.HasInstallTarget() || pc.HasManifestInstall()
}
@@ -73,10 +65,6 @@ func (pc *ParsedCommand) HasManifestInstall() bool {
return pc.IsManifestInstall
}
func (pc *ParsedCommand) ShouldExtractFromManifest() bool {
return pc.IsManifestInstall && !pc.HasInstallTarget()
}
// IsFirstNonFlagArgInList checks if the first non-flag argument in args is in the given list.
// Only the first non-flag arg (the subcommand) is checked to avoid false positives when package
// names or script arguments happen to match a known command.
@@ -103,14 +91,27 @@ type PackageManager interface {
Ecosystem() packagev1.Ecosystem
}
// PackageResolver is the contract for resolving package info
type PackageResolver interface {
// ResolveLatestVersion resolves the latest version for a given package
ResolveLatestVersion(context.Context, *packagev1.Package) (*packagev1.PackageVersion, error)
// PackageManagerInteraction carries the confirmation prompt callback and input
// routing used by proxy-mode malware confirmations.
type PackageManagerInteraction struct {
// GetConfirmationOnMalware is called to get the confirmation of the user on the malware packages
GetConfirmationOnMalware func(malwarePackages []*analyzer.PackageVersionAnalysisResult) (bool, error)
// ResolveDependencies resolves the dependencies for a given package version
// It returns a flattened list of all the dependencies based on implementation
// specific config. The version resolution is based on minimum version selection
// for a given version range.
ResolveDependencies(context.Context, *packagev1.PackageVersion) ([]*packagev1.PackageVersion, error)
// inputReader is the reader to use for user input during confirmations.
// If nil, os.Stdin is used. This is set via SetInput to allow PTY input routing.
inputReader io.Reader
}
// SetInput sets the input reader for user confirmations.
// This allows the PTY switchboard to route input to the prompt during confirmations.
func (i *PackageManagerInteraction) SetInput(r io.Reader) {
i.inputReader = r
}
// Reader returns the configured input reader, or os.Stdin if none is set.
func (i *PackageManagerInteraction) Reader() io.Reader {
if i.inputReader != nil {
return i.inputReader
}
return os.Stdin
}
packagemanager/pypi.go
+13 -38
View File
@@ -172,14 +172,10 @@ func (p *pipCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
// Determine if this is a manifest install
isManifestInstall := len(requirementFiles) > 0
// Combine all manifest files
var allManifestFiles []string
allManifestFiles = append(allManifestFiles, requirementFiles...)
// Process packages
var installTargets []*PackageInstallTarget
for _, pkg := range packages {
packageName, version, extras, err := pypiParsePackageInfo(pkg)
packageName, version, err := pypiParsePackageInfo(pkg)
if err != nil {
return nil, ErrFailedToParsePackage.Wrap(err)
}
@@ -199,7 +195,6 @@ func (p *pipCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
},
Version: version,
},
Extras: extras,
IsExplicitVersion: isExplicit,
})
}
@@ -208,7 +203,6 @@ func (p *pipCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
Command: command,
InstallTargets: installTargets,
IsManifestInstall: isManifestInstall,
ManifestFiles: allManifestFiles,
}, nil
}
@@ -239,19 +233,15 @@ func (u *uvCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
Command: command,
InstallTargets: nil,
IsManifestInstall: true,
ManifestFiles: []string{"uv.lock"},
}, nil
}
// Handles pip sync command (installs from requirements.txt style files)
if len(args) >= 3 && args[0] == "pip" && args[1] == "sync" {
manifestFile := args[2]
return &ParsedCommand{
Command: command,
InstallTargets: nil,
IsManifestInstall: true,
ManifestFiles: []string{manifestFile},
}, nil
}
@@ -298,7 +288,7 @@ func (u *uvCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
var installTargets []*PackageInstallTarget
for _, pkg := range packages {
packageName, version, extras, err := pypiParsePackageInfo(pkg)
packageName, version, err := pypiParsePackageInfo(pkg)
if err != nil {
return nil, ErrFailedToParsePackage.Wrap(err)
}
@@ -318,7 +308,6 @@ func (u *uvCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
},
Version: version,
},
Extras: extras,
IsExplicitVersion: isExplicit,
})
}
@@ -327,7 +316,6 @@ func (u *uvCommandParser) ParseCommand(args []string) (*ParsedCommand, error) {
Command: command,
InstallTargets: installTargets,
IsManifestInstall: isManifestInstall,
ManifestFiles: manifestFiles,
}, nil
}
@@ -357,7 +345,6 @@ func (p *poetryCommandParser) ParseCommand(args []string) (*ParsedCommand, error
Command: command,
IsManifestInstall: true,
InstallTargets: nil,
ManifestFiles: []string{"poetry.lock"},
}, nil
}
@@ -397,7 +384,7 @@ func (p *poetryCommandParser) ParseCommand(args []string) (*ParsedCommand, error
return nil, ErrFailedToParsePackage.Wrap(err)
}
packageName, version, extras, err := pypiParsePackageInfo(convertedPkg)
packageName, version, err := pypiParsePackageInfo(convertedPkg)
if err != nil {
return nil, ErrFailedToParsePackage.Wrap(err)
}
@@ -417,7 +404,6 @@ func (p *poetryCommandParser) ParseCommand(args []string) (*ParsedCommand, error
},
Version: version,
},
Extras: extras,
IsExplicitVersion: isExplicit,
})
}
@@ -426,38 +412,27 @@ func (p *poetryCommandParser) ParseCommand(args []string) (*ParsedCommand, error
Command: command,
InstallTargets: installTargets,
IsManifestInstall: false,
ManifestFiles: nil,
}, nil
}
// pypiParsePackageInfo parses a python package installation specification, separating the package name,
// version constraints, and any extras (additional features) to be installed.
// Example: "django[mysql,redis]>=3.0" returns ("django", ">=3.0", ["mysql", "redis"], nil)
func pypiParsePackageInfo(input string) (packageName, version string, extras []string, err error) {
// pypiParsePackageInfo parses a python package installation specification,
// separating the package name from version constraints and stripping extras.
// Example: "django[mysql,redis]>=3.0" returns ("django", ">=3.0", nil)
func pypiParsePackageInfo(input string) (packageName, version string, err error) {
if input == "" {
return "", "", nil, fmt.Errorf("package info cannot be empty")
return "", "", fmt.Errorf("package info cannot be empty")
}
input = strings.TrimSpace(input)
// First extract any extras if present
// Strip any extras (e.g. "[all]") so they never leak into the package name
openBracket := strings.Index(input, "[")
closeBracket := strings.Index(input, "]")
if openBracket != -1 && closeBracket != -1 && openBracket < closeBracket {
extrasStr := strings.TrimSpace(input[openBracket+1 : closeBracket])
if extrasStr != "" {
// Split extras by comma and trim each extra
for _, extra := range strings.Split(extrasStr, ",") {
if trimmedExtra := strings.TrimSpace(extra); trimmedExtra != "" {
extras = append(extras, trimmedExtra)
}
}
}
// Remove the extra part from input for further processing
input = input[:openBracket] + input[closeBracket+1:]
} else if (openBracket != -1 && closeBracket == -1) || (openBracket == -1 && closeBracket != -1) {
return "", "", nil, fmt.Errorf("mismatched brackets in input '%s'", input)
return "", "", fmt.Errorf("mismatched brackets in input '%s'", input)
}
// Python package version specifiers are typically separated by one of:
@@ -475,17 +450,17 @@ func pypiParsePackageInfo(input string) (packageName, version string, extras []s
if index == -1 {
// No operator found, whole input is package name, no version
return strings.TrimSpace(input), "", extras, nil
return strings.TrimSpace(input), "", nil
}
packageName = strings.TrimSpace(input[:index])
version = strings.TrimSpace(input[index:])
if packageName == "" {
return "", "", nil, fmt.Errorf("invalid package name in input '%s'", input)
return "", "", fmt.Errorf("invalid package name in input '%s'", input)
}
return packageName, version, extras, nil
return packageName, version, nil
}
// pypiConvertPoetryVersionConstraints converts Poetry's caret (^) and tilde (~) version constraints
packagemanager/pypi_executor.go
+1 -2
View File
@@ -193,7 +193,7 @@ func (p *pypiPackageExecutor) buildInstallTargets(command Command, packages []st
var installTargets []*PackageInstallTarget
for _, pkg := range packages {
packageName, version, extras, err := pypiParsePackageInfo(pkg)
packageName, version, err := pypiParsePackageInfo(pkg)
if err != nil {
return nil, ErrFailedToParsePackage.Wrap(err)
}
@@ -212,7 +212,6 @@ func (p *pypiPackageExecutor) buildInstallTargets(command Command, packages []st
},
Version: version,
},
Extras: extras,
IsExplicitVersion: isExplicit,
})
}
packagemanager/pypi_resolver.go
-254
View File
@@ -1,256 +1,13 @@
package packagemanager
import (
"context"
"encoding/json"
"fmt"
"net/http"
"regexp"
"slices"
"strings"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/Masterminds/semver"
"github.com/safedep/dry/log"
"github.com/safedep/dry/packageregistry"
)
type PyPiDependencyResolverConfig struct {
IncludeDevDependencies bool
IncludeTransitiveDependencies bool
TransitiveDepth int
// FailFast will stop resolving dependencies after the first error
FailFast bool
// MaxConcurrency limits the number of concurrent goroutines used for dependency resolution
MaxConcurrency int
PackageInstallTargets []*PackageInstallTarget
}
func NewDefaultPypiDependencyResolverConfig() PyPiDependencyResolverConfig {
return PyPiDependencyResolverConfig{
IncludeDevDependencies: false,
IncludeTransitiveDependencies: true,
TransitiveDepth: 5,
FailFast: false,
MaxConcurrency: 10,
PackageInstallTargets: []*PackageInstallTarget{},
}
}
type pypiDependencyResolver struct {
registry packageregistry.Client
config PyPiDependencyResolverConfig
}
var _ PackageResolver = &pypiDependencyResolver{}
func NewPypiDependencyResolver(config PyPiDependencyResolverConfig) (*pypiDependencyResolver, error) {
client, err := packageregistry.NewPypiAdapter()
if err != nil {
return nil, fmt.Errorf("failed to create pypi adapter: %w", err)
}
return &pypiDependencyResolver{
config: config,
registry: client,
}, nil
}
func (p *pypiDependencyResolver) ResolveDependencies(ctx context.Context, pkg *packagev1.PackageVersion) ([]*packagev1.PackageVersion, error) {
pypiVersionSpecResolverFn := func(packageName, version string) string {
ver, err := pypiGetMatchingVersion(packageName, version)
if err != nil {
log.Debugf("error getting matching version for %s@%s", packageName, version)
return ""
}
return ver
}
pypiDependencyResolverFn := func(packageName, version string) (*packageregistry.PackageDependencyList, error) {
resolvedDependencies, err := getPypiPackageDependencies(packageName, version, p.config.PackageInstallTargets)
if err != nil {
return nil, err
}
dependencies := make([]packageregistry.PackageDependencyInfo, 0)
for _, dep := range resolvedDependencies {
dependencies = append(dependencies, packageregistry.PackageDependencyInfo{
Name: dep.PackageNameExtra,
VersionSpec: dep.VersionSpec,
})
}
return &packageregistry.PackageDependencyList{
Dependencies: dependencies,
}, nil
}
// Python treats package names with '-' and '_' as equivalent (e.g., 'my-package' and 'my_package' refer to the same package)
packageKeyFn := func(pkg *packagev1.PackageVersion) string {
normalizedName := normalizePackageName(pkg.Package.Name)
return fmt.Sprintf("%s@%s", normalizedName, pkg.Version)
}
resolver := newDependencyResolver(p.registry, dependencyResolverConfig{
IncludeDevDependencies: p.config.IncludeDevDependencies,
IncludeTransitiveDependencies: p.config.IncludeTransitiveDependencies,
TransitiveDepth: p.config.TransitiveDepth,
FailFast: p.config.FailFast,
MaxConcurrency: p.config.MaxConcurrency,
}, pypiVersionSpecResolverFn, pypiDependencyResolverFn, packageKeyFn)
return resolver.resolveDependencies(ctx, pkg)
}
func (p *pypiDependencyResolver) ResolveLatestVersion(ctx context.Context, pkg *packagev1.Package) (*packagev1.PackageVersion, error) {
pd, err := p.registry.PackageDiscovery()
if err != nil {
return nil, fmt.Errorf("failed to get package discovery: %w", err)
}
pkgInfo, err := pd.GetPackage(pkg.Name)
if err != nil {
return nil, fmt.Errorf("failed to get package: %w", err)
}
log.Debugf("Resolved pypi/%s to latest version %s", pkg.Name, pkgInfo.LatestVersion)
return &packagev1.PackageVersion{
Package: pkg,
Version: pkgInfo.LatestVersion,
}, nil
}
type pypiPackage struct {
Info pypiPackageInfo `json:"info"`
Releases map[string]any `json:"releases"`
}
type PyPIDependencySpec struct {
// PackageNameExtra is the package name including any direct extras in brackets
// Example: "uvicorn[standard]"
PackageNameExtra string
// VersionSpec is the version constraint for the package
// Example: ">=0.12.0", "==1.0.0", ">=2.0,<3.0"
VersionSpec string
// Extra is the conditional extra marker that defines when this dependency applies
// Example: "all" from "; extra == \"all\""
Extra string
}
type pypiPackageInfo struct {
Name string `json:"name"`
Description string `json:"summary"`
LatestVersion string `json:"version"`
PackageURL string `json:"package_url"`
Author string `json:"author"`
AuthorEmail string `json:"author_email"`
Maintainer string `json:"maintainer"`
MaintainerEmail string `json:"maintainer_email"`
RequiresDist []string `json:"requires_dist"`
}
func getPypiPackageDependencies(packageName, version string, packageTargets []*PackageInstallTarget) ([]PyPIDependencySpec, error) {
url := fmt.Sprintf("https://pypi.org/pypi/%s/%s/json", packageName, version)
res, err := http.Get(url)
if err != nil {
return nil, ErrFailedToFetchPackage.Wrap(err)
}
if res.StatusCode == 404 {
return nil, ErrPackageNotFound.Wrap(err)
}
if res.StatusCode != 200 {
return nil, ErrFailedToFetchPackage.Wrap(err)
}
defer func() {
if err := res.Body.Close(); err != nil {
log.Warnf("failed to close PyPI response body: %v", err)
}
}()
var pypipkg pypiPackage
err = json.NewDecoder(res.Body).Decode(&pypipkg)
if err != nil {
return nil, ErrFailedToParsePackage.Wrap(err)
}
// Find if this package has any specified extras in the install targets
var requestedExtras []string
for _, target := range packageTargets {
if target.PackageVersion.Package.Name == packageName {
requestedExtras = target.Extras
break
}
}
pkgDeps := make([]PyPIDependencySpec, 0, len(pypipkg.Info.RequiresDist))
for _, dep := range pypipkg.Info.RequiresDist {
name, version, extra := pypiParseDependency(dep)
// Include dependencies if they either:
// 1. Have no extras (base dependencies)
// 2. Have an extra that matches one of our requested extras
if extra == "" || (len(requestedExtras) > 0 && slices.Contains(requestedExtras, extra)) {
pkgDeps = append(pkgDeps, PyPIDependencySpec{
PackageNameExtra: name,
VersionSpec: version,
Extra: extra,
})
}
}
return pkgDeps, nil
}
// pypiParseDependency parses a PyPI dependency specification, handling both package extras
// and conditional dependencies. Keeps extras as part of the package name.
// Example: "uvicorn[standard]>=0.12.0; extra == \"all\"" returns ("uvicorn[standard]", ">=0.12.0", "all")
func pypiParseDependency(input string) (string, string, string) {
// Split line by ';' to separate version and markers
parts := strings.SplitN(input, ";", 2)
mainPart := strings.TrimSpace(parts[0])
// Regex to match the first occurrence of version operators
// Using lookahead to ensure we match standalone operators
operatorRegex := regexp.MustCompile(`(==|>=|<=|!=|>|<|~=)(?:\d|$)`)
match := operatorRegex.FindStringIndex(mainPart)
var name, version string
if match != nil {
// Everything before the operator is the name
name = strings.TrimSpace(mainPart[:match[0]])
// Remove trailing parentheses from name if present
name = strings.TrimRight(name, " (")
// Everything from the operator onwards is the version spec
version = strings.TrimSpace(mainPart[match[0]:])
// Remove parentheses from version spec if present
version = strings.Trim(version, "()")
} else {
// No version operator found
name = mainPart
version = ""
}
// Extract extra marker if present
var extra string
if len(parts) == 2 {
extraRe := regexp.MustCompile(`extra\s*==\s*["']([^"']+)["']`)
if match := extraRe.FindStringSubmatch(parts[1]); len(match) == 2 {
extra = match[1]
}
}
return name, version, extra
}
func pypiGetMatchingVersion(packageName, versionConstraint string) (string, error) {
// Already a exact version
if after, ok := strings.CutPrefix(versionConstraint, "=="); ok {
@@ -338,14 +95,3 @@ func findBestMatchingVersion(releases []packageregistry.PackageVersionInfo, cons
}
return bestMatch, nil
}
func normalizePackageName(name string) string {
// Convert to lowercase
name = strings.ToLower(name)
// Replace any sequence of [-_.] with a single hyphen
re := regexp.MustCompile(`[-_.]+`)
name = re.ReplaceAllString(name, "-")
return name
}
packagemanager/pypi_resolver_test.go
-147
View File
@@ -1,56 +1,13 @@
package packagemanager
import (
"context"
"fmt"
"testing"
packagev1 "buf.build/gen/go/safedep/api/protocolbuffers/go/safedep/messages/package/v1"
"github.com/safedep/dry/semver"
"github.com/stretchr/testify/require"
)
func TestPypiDependencyResolver_ResolveLatestVersion(t *testing.T) {
cases := []struct {
name string
pkg *packagev1.Package
assertFn func(t *testing.T, pv *packagev1.PackageVersion, err error)
}{
{
name: "should resolve latest version for a package",
pkg: &packagev1.Package{
Name: "requests",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_PYPI,
},
assertFn: func(t *testing.T, pv *packagev1.PackageVersion, err error) {
require.NoError(t, err)
require.True(t, semver.IsAhead("2.30.0", pv.Version))
},
},
{
name: "should return an error if the package is not found",
pkg: &packagev1.Package{
Name: "nonexistent-package-12345",
Ecosystem: packagev1.Ecosystem_ECOSYSTEM_PYPI,
},
assertFn: func(t *testing.T, pv *packagev1.PackageVersion, err error) {
require.Error(t, err)
require.Nil(t, pv)
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
resolver, err := NewPypiDependencyResolver(NewDefaultPypiDependencyResolverConfig())
require.NoError(t, err)
pv, err := resolver.ResolveLatestVersion(context.Background(), tc.pkg)
tc.assertFn(t, pv, err)
})
}
}
func TestPipGetLatestMatchingVersion(t *testing.T) {
cases := []struct {
name string
@@ -96,107 +53,3 @@ func TestPipGetLatestMatchingVersion(t *testing.T) {
})
}
}
func TestPypiParseDependency(t *testing.T) {
tests := []struct {
name string
input string
wantName string
wantVersion string
wantExtra string
}{
{
name: "simple package without version",
input: "requests",
wantName: "requests",
wantVersion: "",
wantExtra: "",
},
{
name: "package with exact version",
input: "requests==2.28.1",
wantName: "requests",
wantVersion: "==2.28.1",
wantExtra: "",
},
{
name: "package with greater than version",
input: "django>=4.2.0",
wantName: "django",
wantVersion: ">=4.2.0",
wantExtra: "",
},
{
name: "package with less than version",
input: "pylint<3.0.0",
wantName: "pylint",
wantVersion: "<3.0.0",
wantExtra: "",
},
{
name: "package with not equal version",
input: "pytest!=3.0.0",
wantName: "pytest",
wantVersion: "!=3.0.0",
wantExtra: "",
},
{
name: "package with compatible release version",
input: "sphinx~=4.0.0",
wantName: "sphinx",
wantVersion: "~=4.0.0",
wantExtra: "",
},
{
name: "package with extra",
input: "requests;extra=='security'",
wantName: "requests",
wantVersion: "",
wantExtra: "security",
},
{
name: "package with version and extra",
input: "requests>=2.28.1;extra=='security'",
wantName: "requests",
wantVersion: ">=2.28.1",
wantExtra: "security",
},
{
name: "package with single quotes in extra",
input: "django>=4.2.0;extra=='testing'",
wantName: "django",
wantVersion: ">=4.2.0",
wantExtra: "testing",
},
{
name: "package with double quotes in extra",
input: "django>=4.2.0;extra==\"testing\"",
wantName: "django",
wantVersion: ">=4.2.0",
wantExtra: "testing",
},
{
name: "package with multiple version constraints",
input: "requests>=2.28.1,<3.0.0",
wantName: "requests",
wantVersion: ">=2.28.1,<3.0.0",
wantExtra: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
gotName, gotVersion, gotExtra := pypiParseDependency(tt.input)
if gotName != tt.wantName {
t.Errorf("pypiParseDependency() gotName = %v, want %v", gotName, tt.wantName)
}
if gotVersion != tt.wantVersion {
t.Errorf("pypiParseDependency() gotVersion = %v, want %v", gotVersion, tt.wantVersion)
}
if gotExtra != tt.wantExtra {
t.Errorf("pypiParseDependency() gotExtra = %v, want %v", gotExtra, tt.wantExtra)
}
})
}
}
packagemanager/pypi_test.go
+1 -51
View File
@@ -13,7 +13,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input string
pkgName string
version string
extras []string
wantErr bool
}{
{
@@ -21,7 +20,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "fastapi",
pkgName: "fastapi",
version: "",
extras: nil,
wantErr: false,
},
{
@@ -29,7 +27,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "fastapi[all]==0.115.7",
pkgName: "fastapi",
version: "==0.115.7",
extras: []string{"all"},
wantErr: false,
},
{
@@ -37,7 +34,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "requests>=2.0,<3.0",
pkgName: "requests",
version: ">=2.0,<3.0",
extras: nil,
wantErr: false,
},
{
@@ -52,7 +48,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "django~=3.1.0",
pkgName: "django",
version: "~=3.1.0",
extras: nil,
wantErr: false,
},
{
@@ -60,7 +55,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "numpy[]>1.20.0",
pkgName: "numpy",
version: ">1.20.0",
extras: nil,
wantErr: false,
},
{
@@ -68,7 +62,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "pandas<2.0.0",
pkgName: "pandas",
version: "<2.0.0",
extras: nil,
wantErr: false,
},
{
@@ -76,7 +69,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "",
pkgName: "",
version: "",
extras: nil,
wantErr: true,
},
{
@@ -84,7 +76,6 @@ func TestPipParsePackageInfo(t *testing.T) {
input: "==1.0.0",
pkgName: "",
version: "",
extras: nil,
wantErr: true,
},
{
@@ -92,21 +83,19 @@ func TestPipParsePackageInfo(t *testing.T) {
input: " requests == 2.0.0 ",
pkgName: "requests",
version: "== 2.0.0",
extras: nil,
wantErr: false,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
pkgName, version, extras, err := pypiParsePackageInfo(tc.input)
pkgName, version, err := pypiParsePackageInfo(tc.input)
if tc.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, tc.pkgName, pkgName)
assert.Equal(t, tc.version, version)
assert.Equal(t, tc.extras, extras)
}
})
}
@@ -120,70 +109,60 @@ func TestPipParseCommand(t *testing.T) {
name string
args []string
expectedManifest bool
expectedFiles []string
expectedTargets int
}{
{
name: "pip install with -r flag",
args: []string{"install", "-r", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip install with -r flag with different filename",
args: []string{"install", "-r", "requirements-dev.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements-dev.txt"},
expectedTargets: 0,
},
{
name: "pip install with --requirement flag",
args: []string{"install", "--requirement", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip install with combined -r flag",
args: []string{"install", "-rrequirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip install without args",
args: []string{"install"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 0,
},
{
name: "pip install with explicit package",
args: []string{"install", "django"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 1,
},
{
name: "pip install with mixed args",
args: []string{"install", "django", "-r", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 1,
},
{
name: "pip install with multiple -r flags",
args: []string{"install", "-r", "requirements.txt", "-r", "dev-requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt", "dev-requirements.txt"},
expectedTargets: 0,
},
{
name: "non-install command",
args: []string{"list"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 0,
},
}
@@ -194,14 +173,10 @@ func TestPipParseCommand(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, tc.expectedManifest, parsed.IsManifestInstall, "IsManifestInstall mismatch")
assert.Equal(t, tc.expectedFiles, parsed.ManifestFiles, "ManifestFiles mismatch")
assert.Equal(t, tc.expectedTargets, len(parsed.InstallTargets), "InstallTargets count mismatch")
// Test helper methods
assert.Equal(t, tc.expectedManifest, parsed.HasManifestInstall(), "HasManifestInstall mismatch")
expectedShouldExtract := tc.expectedManifest && tc.expectedTargets == 0
assert.Equal(t, expectedShouldExtract, parsed.ShouldExtractFromManifest(), "ShouldExtractFromManifest mismatch")
})
}
}
@@ -214,70 +189,60 @@ func TestPip3ParseCommand(t *testing.T) {
name string
args []string
expectedManifest bool
expectedFiles []string
expectedTargets int
}{
{
name: "pip3 install with -r flag",
args: []string{"install", "-r", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip3 install with -r flag with different filename",
args: []string{"install", "-r", "requirements-dev.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements-dev.txt"},
expectedTargets: 0,
},
{
name: "pip3 install with --requirement flag",
args: []string{"install", "--requirement", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip3 install with combined -r flag",
args: []string{"install", "-rrequirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
},
{
name: "pip3 install without args",
args: []string{"install"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 0,
},
{
name: "pip3 install with explicit package",
args: []string{"install", "django"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 1,
},
{
name: "pip3 install with mixed args",
args: []string{"install", "django", "-r", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 1,
},
{
name: "pip3 install with multiple -r flags",
args: []string{"install", "-r", "requirements.txt", "-r", "dev-requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt", "dev-requirements.txt"},
expectedTargets: 0,
},
{
name: "non-install command",
args: []string{"list"},
expectedManifest: false,
expectedFiles: nil,
expectedTargets: 0,
},
}
@@ -288,13 +253,9 @@ func TestPip3ParseCommand(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, tc.expectedManifest, parsed.IsManifestInstall, "IsManifestInstall mismatch")
assert.Equal(t, tc.expectedFiles, parsed.ManifestFiles, "ManifestFiles mismatch")
assert.Equal(t, tc.expectedTargets, len(parsed.InstallTargets), "InstallTargets count mismatch")
assert.Equal(t, tc.expectedManifest, parsed.HasManifestInstall(), "HasManifestInstall mismatch")
expectedShouldExtract := tc.expectedManifest && tc.expectedTargets == 0
assert.Equal(t, expectedShouldExtract, parsed.ShouldExtractFromManifest(), "ShouldExtractFromManifest mismatch")
})
}
}
@@ -695,7 +656,6 @@ func TestUvParseCommand(t *testing.T) {
name string
args []string
expectedManifest bool
expectedFiles []string
expectedTargets int
expectedPackages []string
wantErr bool
@@ -704,7 +664,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv add simple package",
args: []string{"add", "flask"},
expectedManifest: false,
expectedFiles: []string{""},
expectedTargets: 1,
expectedPackages: []string{"flask"},
wantErr: false,
@@ -713,7 +672,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv add multiple packages",
args: []string{"add", "flask", "requests"},
expectedManifest: false,
expectedFiles: []string{""},
expectedTargets: 2,
expectedPackages: []string{
"flask",
@@ -725,7 +683,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv pip install simple package",
args: []string{"pip", "install", "fastapi"},
expectedManifest: false,
expectedFiles: []string{""},
expectedTargets: 2,
expectedPackages: []string{"fastapi"},
wantErr: false,
@@ -734,7 +691,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv pip install multiple packages",
args: []string{"pip", "install", "flask", "requests"},
expectedManifest: false,
expectedFiles: []string{""},
expectedTargets: 2,
expectedPackages: []string{
"flask",
@@ -746,7 +702,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv pip install from manifest file",
args: []string{"pip", "install", "-r", "requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt"},
expectedTargets: 0,
expectedPackages: []string{},
wantErr: false,
@@ -755,7 +710,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv pip install from multiple manifest files",
args: []string{"pip", "install", "-r", "requirements.txt", "-r", "dev-requirements.txt"},
expectedManifest: true,
expectedFiles: []string{"requirements.txt", "dev-requirements.txt"},
expectedTargets: 0,
expectedPackages: []string{},
wantErr: false,
@@ -764,7 +718,6 @@ func TestUvParseCommand(t *testing.T) {
name: "uv sync",
args: []string{"sync"},
expectedManifest: true,
expectedFiles: []string{"uv.lock"},
expectedTargets: 0,
expectedPackages: []string{},
wantErr: false,
@@ -782,9 +735,6 @@ func TestUvParseCommand(t *testing.T) {
assert.Equal(t, tc.expectedManifest, result.HasManifestInstall(), "HasManifestInstall mismatch")
expectedShouldExtract := tc.expectedManifest && tc.expectedTargets == 0
assert.Equal(t, expectedShouldExtract, result.ShouldExtractFromManifest(), "ShouldExtractFromManifest mismatch")
assert.Equal(t, len(tc.expectedPackages), len(result.InstallTargets), "Number of install targets mismatch")
for i, expectedPkg := range tc.expectedPackages {
packagemanager/pypi_uvx_executor.go
+2 -2
View File
@@ -46,8 +46,8 @@ func (p *pypiPackageExecutor) parseUvxCommand(command Command, args []string) (*
// the pipx/pip/uv executors. uv adds flags frequently; failing closed on an
// unrecognized flag would break otherwise-valid uvx invocations after a uv
// upgrade. The residual gap — a future value-taking flag consuming the tool
// positional and yielding no audit target — only affects non-proxy guard
// mode; the default proxy flow still intercepts every registry download.
// positional and yielding no audit target — is contained because the proxy
// flow still intercepts every registry download.
flagSet.ParseErrorsAllowlist.UnknownFlags = true
flagSet.SetOutput(io.Discard)