HD Moore
5b89811b90
Support concurrent Nuclei engines in the same process ( #6322 )
...
* support for concurrent nuclei engines
* clarify LfaAllowed race
* remove unused mutex
* update LfaAllowed logic to prevent races until it can be reworked for per-execution ID
* Update pkg/templates/parser.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* debug tests
* debug gh action
* fixig gh template test
* using atomic
* using synclockmap
* restore tests concurrency
* lint
* wiring executionId in js fs
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-07-19 00:10:58 +05:30
gopherorg
1079498182
refactor: use maps.Copy for cleaner map handling ( #6283 )
...
Signed-off-by: gopherorg <gopherworld@icloud.com>
2025-07-12 02:50:47 +05:30
HD Moore
f26996cb89
Remove singletons from Nuclei engine (continuation of #6210 ) ( #6296 )
...
* introducing execution id
* wip
* .
* adding separate execution context id
* lint
* vet
* fixing pg dialers
* test ignore
* fixing loader FD limit
* test
* fd fix
* wip: remove CloseProcesses() from dev merge
* wip: fix merge issue
* protocolstate: stop memguarding on last dialer delete
* avoid data race in dialers.RawHTTPClient
* use shared logger and avoid race conditions
* use shared logger and avoid race conditions
* go mod
* patch executionId into compiled template cache
* clean up comment in Parse
* go mod update
* bump echarts
* address merge issues
* fix use of gologger
* switch cmd/nuclei to options.Logger
* address merge issues with go.mod
* go vet: address copy of lock with new Copy function
* fixing tests
* disable speed control
* fix nil ExecuterOptions
* removing deprecated code
* fixing result print
* default logger
* cli default logger
* filter warning from results
* fix performance test
* hardcoding path
* disable upload
* refactor(runner): uses `Warning` instead of `Print` for `pdcpUploadErrMsg`
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Revert "disable upload"
This reverts commit 114fbe6663361bf41cf8b2645fd2d57083d53682.
* Revert "hardcoding path"
This reverts commit cf12ca800e0a0e974bd9fd4826a24e51547f7c00.
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <25837540+dwisiswant0@users.noreply.github.com>
2025-07-10 01:17:26 +05:30
Dwi Siswanto
87ed0b2bb9
build: bump all direct modules ( #6290 )
...
* chore: fix non-constant fmt string in call
Signed-off-by: Dwi Siswanto <git@dw1.io>
* build: bump all direct modules
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(hosterrorscache): update import path
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(charts): break changes
Signed-off-by: Dwi Siswanto <git@dw1.io>
* build: pinned `github.com/zmap/zcrypto` to v0.0.0-20240512203510-0fef58d9a9db
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore: golangci-lint auto fixes
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore: satisfy lints
Signed-off-by: Dwi Siswanto <git@dw1.io>
* build: migrate `github.com/xanzy/go-gitlab` => `gitlab.com/gitlab-org/api/client-go`
Signed-off-by: Dwi Siswanto <git@dw1.io>
* feat(json): update build constraints
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore: dont panicking on close err
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-07-01 00:40:44 +07:00
Shubham Rasal
f89a6d33e9
Use proxy for dns and ssl templates ( #6255 )
...
* Use proxy for dns and ssl templates
- while using template execute level function we need to override custom dialer
* rename overridedialer to customdialer
* Add proxy into hash
- proxy client is shared between non proxy requests
* add dialer into request object
- use request.dialer instead of global variable
* resolve comments
* rename dialer
2025-06-16 22:24:52 +05:30
Dwi Siswanto
622c5503fa
perf(*): replace encoding/json w/ sonic or go-json (fallback) ( #6019 )
...
* perf(*): replace `encoding/json` w/ sonic
Signed-off-by: Dwi Siswanto <git@dw1.io>
* feat(utils): add `json` pkg (sonic wrapper)
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(*): use `sonic` wrapper instead
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(*): replace `sonic.ConfigStd` -> `json` (wrapper)
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test(model): adjust expected marshal'd JSON
Signed-off-by: Dwi Siswanto <git@dw1.io>
* feat(json): dynamic backend; `sonic` -> `go-json` (fallback)
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(json): merge config - as its not usable
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(json): rm go version constraints
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore: go mod tidy
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-02-11 03:01:37 +05:30
Dwi Siswanto
58ae87cb05
test(dns): update input, requires, and enable recursion ( #6014 )
...
* test(dns): update input, requires, and enable recursion
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Update go.mod
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2025-01-30 14:44:29 +05:30
Dwi Siswanto
2c832f5590
refactor(vardump): use godump lib ( #5676 )
...
* refactor(vardump): use `godump` lib
also increate limit char to `255`.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* feat(vardump): add global var `Limit`
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(protocols): rm newline
Signed-off-by: Dwi Siswanto <git@dw1.io>
* feat(types): add `VarDumpLimit` option
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test(vardump): add test cases
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore: tidy up mod
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-10-14 19:31:36 +05:30
Ramana Reddy
3d2f31a56f
fix missing template_url for pd signed templates when executed from custom path ( #5644 )
2024-09-19 18:58:20 +05:30
Dogan Can Bakir
6b71af448a
Fixed issue with -ms option to scan non accessible host ( #5576 )
...
* fail if OnResult callback is not called
* generate error message from error logs
* try..parse..
* fix lint
* add error message to last matcher event
* fix network protocol error logging
* log returned log from ExecuteWithResults
* add back specific logging
* clean up the msg
* minor
* init integration test for -ms
* add tests for http,network,js,ws protocols
* fix lint
* fix network test
* return err for dns protocol
* add integration test for dns protocol
2024-08-28 16:27:43 +05:30
Tarun Koyalwar
1c76398aea
lint error fixes ( #5531 )
...
* lint error fixes
* chore: satisfy non-constant format str in call lint (govet)
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <git@dw1.io>
2024-08-16 20:31:23 +05:30
Mohammed Diaa
ff23949bb0
Apply input transformation to multi-protocol templates ( #5426 )
...
* Apply input transformation to multi-protocol template execution
* Remove ad hoc input transoformation from DNS protocol
* Add SSL protocol input transformer
* Remove ad hoc input transoformation from SSL protocol
* Remove unused function extractDomain from the DNS protocol engine
* transform in flow as well
* bug fix + update test
* bug fix multi proto
:
* bug fix multi proto input
* bug fixes in input transform
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-08-01 20:43:47 +05:30
Kristinn Vikar Jónsson
381ebba6a2
Clustering performance improvements ( #5319 )
...
* Clustering performance improvements
* IsClusterable filters out beforehand, update test to mirror that
* inverse IsClusterable
This makes much more sense
* HashMap based clustering
* furthur improvements to clustering
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-06-27 13:14:43 +05:30
Tarun Koyalwar
3e54ca54b0
feat: fix utils and add goroutine leak unit tests ( #5112 )
...
* feat: fixed leak
* add go leak unit test in sdk
* added goleak unit tests
* bugfix: add random user agents to fuzzing requests
* misc
* misc
* fix lint + use utils pr + misc
* fix ratelimit memleak in sdk
* close protocolstate shared resources in nuclei sdk/lib
* add missing close references
* ignore read/write loop of intransit connections
* close unnecessary idle conns
* add ignore method
* using fixed utils
* dep update
---------
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-05-01 00:28:11 +05:30
Ice3man
0b82e8b7aa
feat: added support for context cancellation to engine ( #5096 )
...
* feat: added support for context cancellation to engine
* misc
* feat: added contexts everywhere
* misc
* misc
* use granular http timeouts and increase http timeout to 30s using multiplier
* track response header timeout in mhe
* update responseHeaderTimeout to 5sec
* skip failing windows test
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-04-25 15:37:56 +05:30
Dogan Can Bakir
cfe6f5da58
fix tests ( #5092 )
2024-04-24 18:49:39 +05:30
Mzack9999
d576db3893
Merge pull request #5035 from projectdiscovery/bugfix-4947-internal-resolvers
...
Fixing internal resolver override
2024-04-15 17:02:09 +01:00
mzack
38e185c410
simpler logic
2024-04-12 00:32:06 +02:00
mzack
7b71886309
Fixing internal resolver override
2024-04-11 19:10:31 +02:00
mzack
2ed33e4723
adding dns srv type
2024-04-11 18:57:50 +02:00
mzack
7e363984b2
Merge branch 'dev' into feat-3072-init-adaptive-speed
2024-04-09 15:19:51 +02:00
Ramana Reddy
8c27ca2591
fix(schema): generation of missing JSON schema definitions ( #4995 )
...
* fix(schema): generation of missing JSON schema definitions
* make headers and data to accept multi-type inputs
* misc update
2024-04-08 03:29:42 +05:30
mzack
af7450737a
making payload concurrency dynamic via direct int change
2024-04-03 23:06:08 +02:00
Mzack9999
a140a4194e
boh - placing resize in wrapped method
2024-04-03 19:40:09 +02:00
Mzack9999
a8d1393e96
init- using resizable components
2024-04-03 17:50:57 +02:00
kchason
fb3c3d828d
Initial switch of libraries
2024-03-25 15:52:20 -04:00
Tarun Koyalwar
ead58f4ab9
implicit thread count when not specified in payloads + threads support in dns,network ( #4715 )
...
* default threads + add threads support in dns payloads
* add threads support in network protocol
* add optional callback to override threadSetter
* fix broken fuzz integration tests
2024-02-02 02:05:30 +05:30
Tarun Koyalwar
c7c35ffb94
fix multiple mem leaks + optimizations ( #4630 )
...
* fix mem leak
* bump version tag
* http: add global resp body read limit of 4MB
* skip creating templateCtx in normal templates
* fix mem leak via retryablehttp , fastdialer
* go mod tidy
* remove unused var
* dep update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-01-18 05:53:42 +05:30
Dogan Can Bakir
ce5df9cc02
introduce scan context ( #4373 )
...
* introduce scan context
* minor
* add joined errors to resultevents
* change `executor` funcs' signature
* fix tests
* join errors in `LogError` func
* change func signature
* add guard
2023-11-28 00:24:45 +05:30
Dogan Can Bakir
7c2db9c394
introduce template-encoded field ( #4315 )
...
* introduce `template-encoded` field
* remove IsCustomTemplate func
* refactor and move encoding to `MakeResultEventItem` func
* encode template in case of no results were found
* commit to last commit
* don't encode templates when`-ms` is used
2023-11-11 04:42:27 +05:30
Dogan Can Bakir
c79d2f05c4
fix trailing dot ( #4295 )
...
* fix trailing dot
* remove trailing dot from `domain`
* remove trailing dots from answer
* remove dots
* fix integration test
2023-11-01 16:51:22 +05:30
Tarun Koyalwar
dc44105baf
nuclei v3 : misc updates ( #4247 )
...
* use parsed options while signing
* update project layout to v3
* fix .gitignore
* remove example template
* misc updates
* bump tlsx version
* hide template sig warning with env
* js: retain value while using log
* fix nil pointer derefernce
* misc doc update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-10-17 17:44:13 +05:30
