External/nuclei
1
0
Fork 0
mirror of https://github.com/projectdiscovery/nuclei.git synced 2025-12-17 22:25:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,731 Commits 49 Branches 135 Tags
Commit Graph

98 Commits

Author SHA1 Message Date
Sandeep Singh
b4644af80a
Lint + test fixes after utils dep update (#6393) 
* fix: remove undefined errorutil.ShowStackTrace

* feat: add make lint support and integrate with test

* refactor: migrate errorutil to errkit across codebase

- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces

* fix: improve DNS test reliability using Google DNS

- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged

* fixing logic

* removing unwanted branches in makefile

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
 2025-08-20 05:28:23 +05:30
Ice3man
1b6ae44bb7 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into loading-performance-improvements-v2 2025-08-06 01:57:41 +05:30
Ice3man
5ba21e272a feat: loading templates performance improvements 2025-08-02 15:58:18 +05:30
jishudashen
0337b33490 chore: fix inconsistent function name in comment 
Signed-off-by: jishudashen <jishudashen@foxmail.com>
 2025-07-21 14:13:22 +08:00
HD Moore
5b89811b90
Support concurrent Nuclei engines in the same process (#6322) 
* support for concurrent nuclei engines

* clarify LfaAllowed race

* remove unused mutex

* update LfaAllowed logic to prevent races until it can be reworked for per-execution ID

* Update pkg/templates/parser.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* debug tests

* debug gh action

* fixig gh template test

* using atomic

* using synclockmap

* restore tests concurrency

* lint

* wiring executionId in js fs

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
 2025-07-19 00:10:58 +05:30
HD Moore
f26996cb89
Remove singletons from Nuclei engine (continuation of #6210) (#6296) 
* introducing execution id

* wip

* .

* adding separate execution context id

* lint

* vet

* fixing pg dialers

* test ignore

* fixing loader FD limit

* test

* fd fix

* wip: remove CloseProcesses() from dev merge

* wip: fix merge issue

* protocolstate: stop memguarding on last dialer delete

* avoid data race in dialers.RawHTTPClient

* use shared logger and avoid race conditions

* use shared logger and avoid race conditions

* go mod

* patch executionId into compiled template cache

* clean up comment in Parse

* go mod update

* bump echarts

* address merge issues

* fix use of gologger

* switch cmd/nuclei to options.Logger

* address merge issues with go.mod

* go vet: address copy of lock with new Copy function

* fixing tests

* disable speed control

* fix nil ExecuterOptions

* removing deprecated code

* fixing result print

* default logger

* cli default logger

* filter warning from results

* fix performance test

* hardcoding path

* disable upload

* refactor(runner): uses `Warning` instead of `Print` for `pdcpUploadErrMsg`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Revert "disable upload"

This reverts commit 114fbe6663361bf41cf8b2645fd2d57083d53682.

* Revert "hardcoding path"

This reverts commit cf12ca800e0a0e974bd9fd4826a24e51547f7c00.

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <25837540+dwisiswant0@users.noreply.github.com>
 2025-07-10 01:17:26 +05:30
Dwi Siswanto
87ed0b2bb9
build: bump all direct modules (#6290) 
* chore: fix non-constant fmt string in call

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: bump all direct modules

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(hosterrorscache): update import path

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(charts): break changes

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: pinned `github.com/zmap/zcrypto` to v0.0.0-20240512203510-0fef58d9a9db

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: golangci-lint auto fixes

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: migrate `github.com/xanzy/go-gitlab` => `gitlab.com/gitlab-org/api/client-go`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(json): update build constraints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: dont panicking on close err

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
 2025-07-01 00:40:44 +07:00
threehonor
d1b1c23e4e chore: fix some function names in comment 
Signed-off-by: threehonor <pengqi@email.cn>
 2025-03-26 11:03:43 +08:00
Dwi Siswanto
622c5503fa
perf(*): replace encoding/json w/ sonic or go-json (fallback) (#6019) 
* perf(*): replace `encoding/json` w/ sonic

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(utils): add `json` pkg (sonic wrapper)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(*): use `sonic` wrapper instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(*): replace `sonic.ConfigStd` -> `json` (wrapper)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(model): adjust expected marshal'd JSON

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(json): dynamic backend; `sonic` -> `go-json` (fallback)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(json): merge config - as its not usable

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(json): rm go version constraints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: go mod tidy

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
 2025-02-11 03:01:37 +05:30
Sandeep Singh
53748c47d8
Misc sdk changes (#6018) 
* feat: misc sdk changes to parser

* misc

* feat: fixed failing tests

* fix lint error + update yamldoc-go

* return 0 exit code if integration test re-run passes

* exclude tech / wordpress template from test

---------

Co-authored-by: Ice3man <nizamulrana@gmail.com>
 2025-01-31 18:53:55 +05:30
Dwi Siswanto
052fd8b79a
feat(hosterrorscache): add Remove and MarkFailedOrRemove methods (#5984) 
* feat(hosterrorscache): add `Remove` and `MarkFailedOrRemove` methods

and also deprecating `MarkFailed`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(*): unwraps `hosterrorscache\.MarkFailed` invocation

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(hosterrorscache): add sync in `Check` and `MarkFailedOrRemove` methods

* test(hosterrorscache): add concurrent test for `Check` method

* refactor(hosterrorscache): do NOT change `MarkFailed` behavior

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(*): use `MarkFailedOrRemove` explicitly

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
 2025-01-31 15:46:57 +05:30
Dwi Siswanto
265051fdf3
feat(templates): rm string conversion (#6016) 
Signed-off-by: Dwi Siswanto <git@dw1.io>
 2025-01-31 01:22:25 +05:30
Dogan Can Bakir
358249bdb4
fix recursive struct validation during JSON marshaling (#5883) 2024-12-19 20:36:21 +05:30
Tarun Koyalwar
16735f5243
fix template signing singnature issue (#5869) 
* fix incorrect .gitignore

* template signer utility tool

* use yaml marhsal & unmarshal for normalization

* normalize before verification
 2024-12-02 14:31:46 +05:30
Dwi Siswanto
3a07fa9c22
feat: add -enable-global-matchers flag (#5857) 
* feat: add `-enable-global-matchers` flag

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(templates): use embedded `types.Options` in `Template`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(lib): add `EnableGlobalMatchersTemplates` SDK opt

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
 2024-11-27 14:37:59 +05:30
sandeep
a09fcc196b auto gen docs update 2024-11-21 18:16:12 +05:30
Dogan Can Bakir
63687c2ce0
disable self-contained and file protocol templates as default (#5825) 
* disable self-contained and file protocol templates as default

* make excluding default

* add config funcs

* fix wrn display

* fix integration tests

* enable self-contained templates when code templates are enabled

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2024-11-19 22:00:28 +05:30
ghost
159a8a53cf Auto Generate Syntax Docs + JSONSchema [Mon Oct 14 15:26:08 UTC 2024] 🤖 2024-10-14 15:26:08 +00:00
ghost
53f56e179d Auto Generate Syntax Docs + JSONSchema [Mon Oct 14 13:56:50 UTC 2024] 🤖 2024-10-14 13:56:50 +00:00
Dwi Siswanto
cc5c5509dc
feat: global matchers (#5701) 
* feat: global matchers

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Ice3man543 <ice3man543@users.noreply.github.com>

* feat(globalmatchers): make `Callback` as type

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: update `passive` term to `(matchers-)static`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): add `origin-template-*` event

also use `Set` method instead of `maps.Clone`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: update `matchers-static` term to `global-matchers`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): clone event before `operator.Execute`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(tmplexec): don't store `matched` on `global-matchers` templ

This will end up generating 2 events from the same
`scan.ScanContext` if one of the templates has
`global-matchers` enabled. This way, non-
`global-matchers` templates can enter the
`writeFailureCallback` func to log failure output.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): initializes `requests` on `New`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): add `hasStorage` method

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(templates): rename global matchers checks method

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(loader): handle nil `templates.Template` pointer

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Ice3man543 <ice3man543@users.noreply.github.com>
 2024-10-14 19:25:46 +05:30
ghost
690089e1ce Auto Generate Syntax Docs + JSONSchema [Wed Oct 9 21:36:38 UTC 2024] 🤖 2024-10-09 21:36:38 +00:00
Tarun Koyalwar
1f945d6d50
consider protocolType in max host error (#5668) 
* consider protocolType in max host error

* add mutex when updating internal-event
 2024-09-28 18:50:35 +05:30
Ice3man
6a561c6470
feat: clone event in clustering to generate correct failure events (#5653) 2024-09-24 18:43:35 +05:30
ghost
1610d96bc2 Auto Generate Syntax Docs + JSONSchema [Thu Sep 19 13:34:08 UTC 2024] 🤖 2024-09-19 13:34:08 +00:00
Ramana Reddy
3d2f31a56f
fix missing template_url for pd signed templates when executed from custom path (#5644) 2024-09-19 18:58:20 +05:30
ghost
cb05f55251 Auto Generate Syntax Docs + JSONSchema [Mon Sep 2 10:01:02 UTC 2024] 🤖 2024-09-02 10:01:02 +00:00
Mzack9999
0da993afe6
Merge commit from fork 
* fix template signature verification

* fix signature pattern check

* add tests

* remove signature count constraint, check for lines len

* Add more tests

* Centralize signature extraction logic in signer package

* Move signature handling in Sign function to beginning

* Remove comment

* Revert `NewTemplateSigVerifier`

* update tests

* use ExtractSignatureAndContent func

* Allow signing code templates

* Remove unused const

---------

Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
Co-authored-by: Guy Goldenberg <guy.goldenberg@wiz.io>
 2024-08-19 18:02:54 +05:30
ghost
d20ec34f63 Auto Generate Syntax Docs + JSONSchema [Fri Aug 16 12:41:50 UTC 2024] 🤖 2024-08-16 12:41:50 +00:00
Tarun Koyalwar
2df1b2e88e
file proto missing vars in flow & multi-protocol (#5480) 
* fix missing template context in file proto

* fix file protocol missing vars

* fix test

* skip example advanced test
 2024-08-04 18:14:08 +05:30
Mzack9999
bc229a46ca
Merge pull request #5331 from projectdiscovery/use_containsall 
use `stringsutil.ContainsAll`
 2024-07-15 13:21:03 +02:00
GitHub Action
49d8579662 Auto Generate Syntax Docs + JSONSchema [Mon Jul 15 10:28:25 UTC 2024] 🤖 2024-07-15 10:28:25 +00:00
Dogan Can Bakir
f080d614c3
introduce timeouts config in types.Options (#5228) 
* introduce timeout variants

* update instances and add codeexectimeout

* fix test

* default to 10s

* minor

* make timeouts pluggable and rename

* remove residual code

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
 2024-07-15 15:57:15 +05:30
mzack
ead444b88b Merge branch 'dev' into use_containsall 2024-07-12 13:05:14 +02:00
Kristinn Vikar Jónsson
381ebba6a2
Clustering performance improvements (#5319) 
* Clustering performance improvements

* IsClusterable filters out beforehand, update test to mirror that

* inverse IsClusterable
This makes much more sense

* HashMap based clustering

* furthur improvements to clustering

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
 2024-06-27 13:14:43 +05:30
Doğan Can Bakır
c7006a9168 use stringsutil.ContainsAll 2024-06-25 12:26:18 +03:00
Mzack9999
1c51a6bef6
improving workflows (#5318) 
* improving workflows

* fixing count
 2024-06-22 22:42:00 +05:30
Ramana Reddy
0eddafe2a9
Fix tempalte-id validation (#5261) 
* Fix tempalte-id validation

* Add checks to mandatory fields

* misc update
 2024-06-16 04:27:31 +05:30
GitHub Action
1c355c54ad Auto Generate Syntax Docs + JSONSchema [Mon Jun 10 23:14:52 UTC 2024] 🤖 2024-06-10 23:14:52 +00:00
Tarun Koyalwar
23bd0336fb
multiple bug fixes + performance improvements (#5148) 
* prototype errkit

* complete errkit implementation

* add cause to all timeouts

* fix request timeout annotation @timeout

* increase responseHeaderTimeout to 8 for stability

* rawhttp error related improvements

* feat: add port status caching

* add port status caching to http

* migrate to new utils/errkit

* remote dialinterface + error cause

* debug dir support using .gitignore debug-*

* make nuclei easy to debug

* debug dir update .gitignore

* temp change (to revert)

* Revert "temp change (to revert)"

This reverts commit d3131f777713b9f80e2275142e80f36340a76d36.

* use available context instead of new one

* bump fastdialer

* fix hosterrorscache + misc improvements

* add 'address' field in error log

* fix js vague errors + pgwrap driver

* fix max host error + misc updates

* update tests as per changes

* fix request annotation context

* remove closed dialer reference

* fix sdk panic issue

* bump retryablehttp-go,utils,fastdialer

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
 2024-05-25 00:29:04 +05:30
Tarun Koyalwar
d6424ea5d0
pdcp result upload: bug fix + (optional) scan name support using -sname flag (#5155) 
* add default template severity and error when validating

* ignore workflows when validating severity

* add scan name support in pdcp result upload

* scan upload: fix missing name query param

* make profile-loader integration tests generic

* add scan-id validation

* ignore invalid scan id's
 2024-05-11 00:44:14 +05:30
Raúl Sampedro
673404a80d
Add list template tags command (#4798) 
* add list template tags command

* update readme

* misc changes to implementation

* misc

* misc update

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2024-05-05 00:41:39 +05:30
Ice3man
0b82e8b7aa
feat: added support for context cancellation to engine (#5096) 
* feat: added support for context cancellation to engine

* misc

* feat: added contexts everywhere

* misc

* misc

* use granular http timeouts and increase http timeout to 30s using multiplier

* track response header timeout in mhe

* update responseHeaderTimeout to 5sec

* skip failing windows test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
 2024-04-25 15:37:56 +05:30
GitHub Action
f2c0b4b443 Auto Generate Syntax Docs + JSONSchema [Mon Apr 15 14:44:52 UTC 2024] 🤖 2024-04-15 14:44:52 +00:00
Ramana Reddy
8c27ca2591
fix(schema): generation of missing JSON schema definitions (#4995) 
* fix(schema): generation of missing JSON schema definitions

* make headers and data to accept multi-type inputs

* misc update
 2024-04-08 03:29:42 +05:30
Tarun Koyalwar
3907e20bde
fix multiple panics & missing matcher-status in flow templates (#4978) 
* validate and fix empty internal-event

* fix on error with interactsh req

* disable clustering in flow & multiproto

* fix empty/missing matcher-status result

* fix cluster unit test

* fix no results found unit test
 2024-04-03 17:19:06 +05:30
GitHub Action
39ee0b7c39 Auto Generate Syntax Docs + JSONSchema [Mon Apr 1 13:49:30 UTC 2024] 🤖 2024-04-01 13:49:30 +00:00
Tarun Koyalwar
255032f4f2
pre-condition in code , fuzz and other misc updates (#4966) 
* fuzz: rename 'filters' -> 'pre-condition'

* code proto: pre-condition + integration test

* feat: dsl document generator

* update dsl page header

* fix lint error

* add js defined helper funcs in docs

* remove panic recovery unless its for third party(go-rod,goja)

* handle dynamic values flattening edgecase in flow+multiprotocol

* fix order of kv in form-data (failing test)

* fix template loading counters

* Revert "handle dynamic values flattening edgecase in flow+multiprotocol"

This reverts commit 58fdd4faf7df5d654b46a9585011f614d5c98aa4.

* fix flow iteration using 'iterate'
 2024-04-01 19:18:21 +05:30
Tarun Koyalwar
25e7799c09
req_url_pattern for vuln_hash calculation + unit test (#4964) 2024-03-30 23:50:31 +05:30
GitHub Action
5ce912e316 Auto Generate Syntax Docs + JSONSchema [Fri Mar 29 08:03:07 UTC 2024] 🤖 2024-03-29 08:03:07 +00:00
Tarun Koyalwar
e88889b263
add -dast flag and multiple bug fixes for dast templates (#4941) 
* add default get method

* remove residual payload logic from old implementation

* fuzz: clone current state of component

* fuzz: bug fix stacking of payloads in multiple mode

* improve stdout template loading stats

* stdout: force display warnings if no templates are loaded

* update flags in README.md

* quote non-ascii chars in extractor output

* aws request signature can only be used in signed & verified tmpls

* deprecate request signature

* remove logic related to deprecated fuzzing input

* update test to use ordered params

* fix interactsh-url lazy eval: #4946

* output: skip unnecessary updates when unescaping

* updates as per requested changes
 2024-03-29 13:31:30 +05:30