78 Commits

Author SHA1 Message Date
Mzack9999
089e2a4ee0 centralizing ratelimiter logic 2025-09-12 17:46:42 +02:00
Tarun Koyalwar
19247ae74b
Path-Based Fuzzing SQL fix (#6400)
* setup claude

* migrate to using errkit

* fix unused imports + lint errors

* update settings.json

* fix url encoding issue

* fix lint error

* fix the path fuzzing component

* fix lint error
2025-08-25 13:36:58 +05:30
Sandeep Singh
b4644af80a
Lint + test fixes after utils dep update (#6393)
* fix: remove undefined errorutil.ShowStackTrace

* feat: add make lint support and integrate with test

* refactor: migrate errorutil to errkit across codebase

- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces

* fix: improve DNS test reliability using Google DNS

- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged

* fixing logic

* removing unwanted branches in makefile

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-08-20 05:28:23 +05:30
HD Moore
5b89811b90
Support concurrent Nuclei engines in the same process (#6322)
* support for concurrent nuclei engines

* clarify LfaAllowed race

* remove unused mutex

* update LfaAllowed logic to prevent races until it can be reworked for per-execution ID

* Update pkg/templates/parser.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* debug tests

* debug gh action

* fixig gh template test

* using atomic

* using synclockmap

* restore tests concurrency

* lint

* wiring executionId in js fs

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-07-19 00:10:58 +05:30
HD Moore
f26996cb89
Remove singletons from Nuclei engine (continuation of #6210) (#6296)
* introducing execution id

* wip

* .

* adding separate execution context id

* lint

* vet

* fixing pg dialers

* test ignore

* fixing loader FD limit

* test

* fd fix

* wip: remove CloseProcesses() from dev merge

* wip: fix merge issue

* protocolstate: stop memguarding on last dialer delete

* avoid data race in dialers.RawHTTPClient

* use shared logger and avoid race conditions

* use shared logger and avoid race conditions

* go mod

* patch executionId into compiled template cache

* clean up comment in Parse

* go mod update

* bump echarts

* address merge issues

* fix use of gologger

* switch cmd/nuclei to options.Logger

* address merge issues with go.mod

* go vet: address copy of lock with new Copy function

* fixing tests

* disable speed control

* fix nil ExecuterOptions

* removing deprecated code

* fixing result print

* default logger

* cli default logger

* filter warning from results

* fix performance test

* hardcoding path

* disable upload

* refactor(runner): uses `Warning` instead of `Print` for `pdcpUploadErrMsg`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Revert "disable upload"

This reverts commit 114fbe6663361bf41cf8b2645fd2d57083d53682.

* Revert "hardcoding path"

This reverts commit cf12ca800e0a0e974bd9fd4826a24e51547f7c00.

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <25837540+dwisiswant0@users.noreply.github.com>
2025-07-10 01:17:26 +05:30
Jose De La O Hernandez
285c5e1442
fixing panic caused by uninitialized colorizer (#6315) 2025-07-09 04:34:05 +05:30
Dwi Siswanto
7e2ec686ae
fix(lib): scans didn't stop on ctx cancellation (#6310)
* fix(lib): scans didn't stop on ctx cancellation

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Update lib/sdk_test.go

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(lib): wait resources to be released b4 return

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-07-09 01:04:16 +07:00
Eric Gruber
b95b04fc4d
feat: add EnableMatcherStatus function to configure matcher status in NucleiEngine (#6191) 2025-06-17 05:08:01 +05:30
Dogan Can Bakir
357fe9efa7
add resume option to sdk (#6083) 2025-03-07 16:53:55 +05:30
meme-lord
08c46ffaeb
Added WithResponseReadSize function to allow SDK users to modify max response read opt (#5961)
* added WithResponseReadSize function to allow SDK users to modify max response read opt

* Update lib/config.go

improved comment, changed casing of param name, added negative input check

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fixing rabbitai commit >:(

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-01-14 15:35:12 +05:30
chuu
258f38f72d
fix(sdk): NoHostErrors not set (#5783) 2025-01-02 17:48:36 +05:30
Mzack9999
1e87ca82c8
fix missing browser init (#5896)
* fix missing browser init

* .

* using lazy init

* updating test with new web ui

* go mod

* sandbox test

* non fatal error
2024-12-17 15:38:42 +05:30
Shubham Rasal
be1f634eae
Add Alive Proxy into Options (#5903)
* Move proxy variable from global to options

- Provides ability to pass diff proxy in single nuclei instance using sdk

* add type check (resolve comments)
2024-12-13 04:23:27 +05:30
Dwi Siswanto
3a07fa9c22
feat: add -enable-global-matchers flag (#5857)
* feat: add `-enable-global-matchers` flag

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(templates): use embedded `types.Options` in `Template`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(lib): add `EnableGlobalMatchersTemplates` SDK opt

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-11-27 14:37:59 +05:30
Dogan Can Bakir
63687c2ce0
disable self-contained and file protocol templates as default (#5825)
* disable self-contained and file protocol templates as default

* make excluding default

* add config funcs

* fix wrn display

* fix integration tests

* enable self-contained templates when code templates are enabled

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-11-19 22:00:28 +05:30
Dwi Siswanto
3e4ec90cea
ci: refactor workflows (#5818)
* ci: refactor workflows

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: structured build outputs

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: use `go-ci`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(make): misused var for `vet` cmd

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-11-14 19:19:49 +07:00
Dogan Can Bakir
7547f1a5c3
Merge pull request #5678 from alban-stourbe-wmx/feature/sdk-add-vars
Add vars with SDK
2024-11-08 15:11:58 +03:00
chuu
8a4be44631 perf(sdk): sdk functions to access workflow store 2024-10-24 14:34:50 +08:00
Dogan Can Bakir
92213e1335
Merge pull request #5733 from projectdiscovery/fix_template_loading_logic
fix template loading logic
2024-10-15 13:52:47 +03:00
chuu
aab2cadb64
fix: input helper in executor options (#5712) 2024-10-14 19:22:52 +05:30
Doğan Can Bakır
4ef058758c fix template loading logic 2024-10-14 15:53:53 +03:00
alban.stourbe stourbe
ed3e06a701 Add checkerr on runtimeVars & Add a new TestSDK 2024-10-01 09:29:43 +02:00
alban.stourbe stourbe
d0463e8ca3 Add vars with SDK 2024-09-30 16:52:54 +02:00
Dogan Can Bakir
a45e4bbd19
move code around (#5626) 2024-09-12 17:05:32 +05:30
Doğan Can Bakır
50023428d0 fix lint 2024-08-19 23:09:07 +03:00
Tarun Koyalwar
1c76398aea
lint error fixes (#5531)
* lint error fixes

* chore: satisfy non-constant format str in call lint (govet)

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <git@dw1.io>
2024-08-16 20:31:23 +05:30
alban-stourbe-wmx
0787ff29e0
Add Workflows SDK scan (#5409)
* Add Workflows SDK scan

* minor

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2024-08-03 23:13:31 +05:30
Dogan Can Bakir
1fa0e2b473
disable http probe when passive mode is enabled (#5418) 2024-07-22 18:25:02 +05:30
Dogan Can Bakir
6c32eb7d1e
add sdk option to disable update check (#5346) 2024-07-15 18:51:16 +05:30
Douglas Danger Manley
e61ca0c42d
Use fs.FS when explicitly given (#5312)
* Use the `templateFS` if it's there when calling `OpenFile`

* Add a new constructor

* More refactoring

* Both of my use cases are working

* Fix for legacy assumptions

* minor update: remove gologger debug stmts

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-06-23 01:12:01 +05:30
Mzack9999
1e85e70d99 removed as recommended 2024-06-13 17:24:07 +02:00
Mzack9999
52975373ff Merge branch 'dev' into feat-4808-planner 2024-06-13 17:19:43 +02:00
Douglas Danger Manley
8011012c42
Fix ExecuteCallbackWithCtx to use the context that was provided (#5236)
* Fix `ExecuteCallbackWithCtx` to use the context that was provided

This updates `ExecuteCallbackWithCtx` to use the context that was
provided.

* remove more hardcoded context

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-05-30 16:04:15 +05:30
mzack
3307517a6c conflict 2024-05-25 02:52:55 +02:00
mzack
46e2a54bfe Merge branch 'dev' into feat-4808-planner 2024-05-25 02:45:54 +02:00
Tarun Koyalwar
23bd0336fb
multiple bug fixes + performance improvements (#5148)
* prototype errkit

* complete errkit implementation

* add cause to all timeouts

* fix request timeout annotation @timeout

* increase responseHeaderTimeout to 8 for stability

* rawhttp error related improvements

* feat: add port status caching

* add port status caching to http

* migrate to new utils/errkit

* remote dialinterface + error cause

* debug dir support using .gitignore debug-*

* make nuclei easy to debug

* debug dir update .gitignore

* temp change (to revert)

* Revert "temp change (to revert)"

This reverts commit d3131f777713b9f80e2275142e80f36340a76d36.

* use available context instead of new one

* bump fastdialer

* fix hosterrorscache + misc improvements

* add 'address' field in error log

* fix js vague errors + pgwrap driver

* fix max host error + misc updates

* update tests as per changes

* fix request annotation context

* remove closed dialer reference

* fix sdk panic issue

* bump retryablehttp-go,utils,fastdialer

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-05-25 00:29:04 +05:30
Mzack9999
4fc16e36e1 Merge branch 'dev' into feat-4808-planner 2024-05-23 09:08:53 +02:00
Ice3man
4170e1cbb8
more goroutine leak fixes to nuclei (#5188)
* more goroutine leak fixes to nuclei

* run only dns templates for test

* updated httpx to dev

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-05-20 00:48:40 +05:30
Mzack9999
b820fb9464 fixing shared cleanup 2024-05-15 21:43:43 +02:00
Mzack9999
47ca8fe842 fix non gc-able dialer
closes #5165
2024-05-15 21:25:40 +02:00
Tarun Koyalwar
098392b604
add context support in sdk (#5154) 2024-05-07 01:31:33 +05:30
Tarun Koyalwar
3e54ca54b0
feat: fix utils and add goroutine leak unit tests (#5112)
* feat: fixed leak

* add go leak unit test in sdk

* added goleak unit tests

* bugfix: add random user agents to fuzzing requests

* misc

* misc

* fix lint + use utils pr + misc

* fix ratelimit memleak in sdk

* close protocolstate shared resources in nuclei sdk/lib

* add missing close references

* ignore read/write loop of intransit connections

* close unnecessary idle conns

* add ignore method

* using fixed utils

* dep update

---------

Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-05-01 00:28:11 +05:30
Anna Smith
e96fdf2e40
chore: fix function names in comment (#5110) 2024-04-27 18:40:33 +05:30
Ice3man
0b82e8b7aa
feat: added support for context cancellation to engine (#5096)
* feat: added support for context cancellation to engine

* misc

* feat: added contexts everywhere

* misc

* misc

* use granular http timeouts and increase http timeout to 30s using multiplier

* track response header timeout in mhe

* update responseHeaderTimeout to 5sec

* skip failing windows test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-04-25 15:37:56 +05:30
scottdharvey
4b9c3b8d16
Feat 5059 (#5060)
* change catalog

* add usesuppliedcatalog option

* add catalog nil check

* Update config.go
2024-04-23 19:11:12 +05:30
lvyaoting
84582183dc
chore: fix function names in comment (#5008)
Signed-off-by: lvyaoting <lvyaoting@outlook.com>
2024-04-16 16:57:51 +05:30
mzack
582a85d9c0 mimic follow behavior 2024-04-09 18:31:22 +02:00
mzack
99a018d12a adding speed change example 2024-04-04 20:19:46 +02:00
Mzack9999
620287f76b deprecating rlm 2024-04-03 19:28:39 +02:00
Tarun Koyalwar
e88889b263
add -dast flag and multiple bug fixes for dast templates (#4941)
* add default get method

* remove residual payload logic from old implementation

* fuzz: clone current state of component

* fuzz: bug fix stacking of payloads in multiple mode

* improve stdout template loading stats

* stdout: force display warnings if no templates are loaded

* update flags in README.md

* quote non-ascii chars in extractor output

* aws request signature can only be used in signed & verified tmpls

* deprecate request signature

* remove logic related to deprecated fuzzing input

* update test to use ordered params

* fix interactsh-url lazy eval: #4946

* output: skip unnecessary updates when unescaping

* updates as per requested changes
2024-03-29 13:31:30 +05:30