117 lines
3.8 KiB
Go
Raw Permalink Normal View History

package disk
import (
"fmt"
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
"io/fs"
"os"
"path/filepath"
"strings"
2021-12-06 11:38:22 +01:00
"github.com/pkg/errors"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
fileutil "github.com/projectdiscovery/utils/file"
urlutil "github.com/projectdiscovery/utils/url"
)
// ResolvePath resolves the path to an absolute one in various ways.
//
// It checks if the filename is an absolute path, looks in the current directory
// or checking the nuclei templates directory. If a second path is given,
// it also tries to find paths relative to that second path.
func (c *DiskCatalog) ResolvePath(templateName, second string) (string, error) {
if filepath.IsAbs(templateName) {
return templateName, nil
}
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
if c.templatesFS != nil {
if potentialPath, err := c.tryResolve(templateName); err != errNoValidCombination {
return potentialPath, nil
}
}
if second != "" {
secondBasePath := filepath.Join(filepath.Dir(second), templateName)
2021-12-06 11:38:22 +01:00
if potentialPath, err := c.tryResolve(secondBasePath); err != errNoValidCombination {
return potentialPath, nil
}
}
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
if c.templatesFS == nil {
curDirectory, err := os.Getwd()
if err != nil {
return "", err
}
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
templatePath := filepath.Join(curDirectory, templateName)
if potentialPath, err := c.tryResolve(templatePath); err != errNoValidCombination {
return potentialPath, nil
}
}
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
templatePath := filepath.Join(config.DefaultConfig.GetTemplateDir(), templateName)
if potentialPath, err := c.tryResolve(templatePath); err != errNoValidCombination {
return potentialPath, nil
}
return "", fmt.Errorf("no such path found: %s", templateName)
}
2021-12-06 11:38:22 +01:00
var errNoValidCombination = errors.New("no valid combination found")
// tryResolve attempts to load locate the target by iterating across all the folders tree
func (c *DiskCatalog) tryResolve(fullPath string) (string, error) {
Add more support for `fs.FS` in template parsing (#5421) * misc update * chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252) Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker (#4316) Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix README_CN.md typos (#4369) * version update * Add more support for `fs.FS` in the disk catalog This adds more support for `fs.FS` in the disk catalog. This fixes some places where direct `os` file-related calls were being made to use the catalog interface instead. Note that the JavaScript compiler *still* does not work in any context where the `pkg/js/libs/fs` package is used. In particular, the `ReadFilesFromDir` function is hard-coded to use the `os` package and not respect the catalog. * Remove some testing artifacts * Wrap up * Unwind other changes * Add a LoadHelperFileFunction to Options * Use a direct func * Tweak validation * Use a function type --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com> Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-20 17:11:22 -04:00
if c.templatesFS == nil {
if fileutil.FileOrFolderExists(fullPath) {
return fullPath, nil
}
} else {
if _, err := fs.Stat(c.templatesFS, fullPath); err == nil {
return fullPath, nil
}
2021-12-06 11:38:22 +01:00
}
return "", errNoValidCombination
}
// BackwardsCompatiblePaths returns new paths for all old/legacy template paths
// Note: this is a temporary function and will be removed in the future release
func BackwardsCompatiblePaths(templateDir string, oldPath string) string {
// TODO: remove this function in the future release
// 1. all http related paths are now moved at path /http
// 2. network related CVES are now moved at path /network/cves
newPathCallback := func(path string) string {
// trim prefix slash if any
path = strings.TrimPrefix(path, "/")
// try to resolve path at /http subdirectory
if fileutil.FileOrFolderExists(filepath.Join(templateDir, "http", path)) {
return filepath.Join(templateDir, "http", path)
// try to resolve path at /network/cves subdirectory
} else if strings.HasPrefix(path, "cves") && fileutil.FileOrFolderExists(filepath.Join(templateDir, "network", "cves", path)) {
return filepath.Join(templateDir, "network", "cves", path)
}
// most likely the path is not found
return filepath.Join(templateDir, path)
}
switch {
case fileutil.FileOrFolderExists(oldPath):
// new path specified skip processing
return oldPath
case filepath.IsAbs(oldPath):
tmp := strings.TrimPrefix(oldPath, templateDir)
if tmp == oldPath {
// user provided absolute path which is not in template directory
// skip processing
return oldPath
}
// trim the template directory from the path
return newPathCallback(tmp)
case strings.Contains(oldPath, urlutil.SchemeSeparator):
Spelling (#4008) * spelling: addresses Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: asynchronous Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: basic Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: brute force Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: constant Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: disables Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: engine Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: every time Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: execution Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: false positives Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: from Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: further Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: github Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: gitlab Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: highlight Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: hygiene Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: ignore Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: input Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: item Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: itself Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: latestxxx Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: navigation Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: negative Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: nonexistent Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: occurred Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: override Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: overrides Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: payload Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: performed Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: respective Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: retrieve Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: scanlist Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: separated Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: separator Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: severity Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: source Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: strategy Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: string Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: templates Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: terminal Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: timeout Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: trailing slash Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: trailing Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> * spelling: websocket Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com> --------- Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-08-01 14:33:43 -04:00
// scheme separator is used to identify the path as url
// TBD: add support for url directories ??
return oldPath
case strings.Contains(oldPath, "*"):
// this is most likely a glob path skip processing
return oldPath
default:
// this is most likely a relative path
return newPathCallback(oldPath)
}
}