27 Commits

Author SHA1 Message Date
Tarun Koyalwar
19247ae74b
Path-Based Fuzzing SQL fix (#6400)
* setup claude

* migrate to using errkit

* fix unused imports + lint errors

* update settings.json

* fix url encoding issue

* fix lint error

* fix the path fuzzing component

* fix lint error
2025-08-25 13:36:58 +05:30
Sandeep Singh
b4644af80a
Lint + test fixes after utils dep update (#6393)
* fix: remove undefined errorutil.ShowStackTrace

* feat: add make lint support and integrate with test

* refactor: migrate errorutil to errkit across codebase

- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces

* fix: improve DNS test reliability using Google DNS

- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged

* fixing logic

* removing unwanted branches in makefile

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-08-20 05:28:23 +05:30
Dwi Siswanto
9fcacd0f86
ci(tests): migrate to golangci-lint v2 (#6380)
* chore: satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* ci(tests): migrate to golangci-lint v2

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-08-16 13:20:09 +07:00
ysokolovsky
d569cfe864
fix(headless): merge extra headers (#6376)
* headless: fix extra headers overwrite

* headless: set Accept-Language when no custom headers
2025-08-16 04:48:34 +05:30
HD Moore
f26996cb89
Remove singletons from Nuclei engine (continuation of #6210) (#6296)
* introducing execution id

* wip

* .

* adding separate execution context id

* lint

* vet

* fixing pg dialers

* test ignore

* fixing loader FD limit

* test

* fd fix

* wip: remove CloseProcesses() from dev merge

* wip: fix merge issue

* protocolstate: stop memguarding on last dialer delete

* avoid data race in dialers.RawHTTPClient

* use shared logger and avoid race conditions

* use shared logger and avoid race conditions

* go mod

* patch executionId into compiled template cache

* clean up comment in Parse

* go mod update

* bump echarts

* address merge issues

* fix use of gologger

* switch cmd/nuclei to options.Logger

* address merge issues with go.mod

* go vet: address copy of lock with new Copy function

* fixing tests

* disable speed control

* fix nil ExecuterOptions

* removing deprecated code

* fixing result print

* default logger

* cli default logger

* filter warning from results

* fix performance test

* hardcoding path

* disable upload

* refactor(runner): uses `Warning` instead of `Print` for `pdcpUploadErrMsg`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Revert "disable upload"

This reverts commit 114fbe6663361bf41cf8b2645fd2d57083d53682.

* Revert "hardcoding path"

This reverts commit cf12ca800e0a0e974bd9fd4826a24e51547f7c00.

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <25837540+dwisiswant0@users.noreply.github.com>
2025-07-10 01:17:26 +05:30
Dwi Siswanto
87ed0b2bb9
build: bump all direct modules (#6290)
* chore: fix non-constant fmt string in call

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: bump all direct modules

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(hosterrorscache): update import path

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(charts): break changes

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: pinned `github.com/zmap/zcrypto` to v0.0.0-20240512203510-0fef58d9a9db

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: golangci-lint auto fixes

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* build: migrate `github.com/xanzy/go-gitlab` => `gitlab.com/gitlab-org/api/client-go`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(json): update build constraints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: dont panicking on close err

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-07-01 00:40:44 +07:00
Dwi Siswanto
695a7520b9
fix(headless): incorrect last navigated URL (#6278)
* chore(headless): uses `maps.Copy`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): implements update last navigated URL

for `ActionNavigate`, `WaitPageLifecycleEvent`, and
`WaitStable` based on latest navigation URL.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Update pkg/protocols/headless/engine/page.go

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-06-24 05:32:18 +05:30
Chris Grieger
bc551fc3f1 fix: improve headless engine startup and shutdown
Fixes #6221

Instead of enumerating all chrome processes to determine
which ones need to be killed on shutdown, use the launcher.Kill()
method to terminate the process that was launched for this
browser instance.
2025-05-14 16:14:21 +02:00
Dwi Siswanto
622c5503fa
perf(*): replace encoding/json w/ sonic or go-json (fallback) (#6019)
* perf(*): replace `encoding/json` w/ sonic

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(utils): add `json` pkg (sonic wrapper)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(*): use `sonic` wrapper instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(*): replace `sonic.ConfigStd` -> `json` (wrapper)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(model): adjust expected marshal'd JSON

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(json): dynamic backend; `sonic` -> `go-json` (fallback)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(json): merge config - as its not usable

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(json): rm go version constraints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: go mod tidy

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-02-11 03:01:37 +05:30
Dwi Siswanto
d2636b9ca2
feat(headless): eval DSL exprs in args (#6017)
* refactor(headless): mv `input` -> `ctx` field name

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): eval DSL exprs in args

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(headless): rm duplicate imports

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): rm duplicate dumped req vars

* refactor(headless): unify `getTimeParameter` retrieval

Now, `getTimeParameter` tries to get the parameter
as an integer, then as a `time.Duration`, and
finally falls back to the default value
(multiplied by the unit).

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): adjust default timeout value to 5s

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): use `getTimeParameter`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(headless): add nolint directive - `replaceWithValues`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): revert parameter automerge & adds `inputURL` field

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): add headless-dsl integration test

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-02-10 20:16:35 +05:30
Mzack9999
1e87ca82c8
fix missing browser init (#5896)
* fix missing browser init

* .

* using lazy init

* updating test with new web ui

* go mod

* sandbox test

* non fatal error
2024-12-17 15:38:42 +05:30
Shubham Rasal
be1f634eae
Add Alive Proxy into Options (#5903)
* Move proxy variable from global to options

- Provides ability to pass diff proxy in single nuclei instance using sdk

* add type check (resolve comments)
2024-12-13 04:23:27 +05:30
Dwi Siswanto
2c832f5590
refactor(vardump): use godump lib (#5676)
* refactor(vardump): use `godump` lib

also increate limit char to `255`.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(vardump): add global var `Limit`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(protocols): rm newline

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(types): add `VarDumpLimit` option

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(vardump): add test cases

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: tidy up mod

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-10-14 19:31:36 +05:30
Dwi Siswanto
4cd065df5f
feat(headless): supporting standard lifecycle events (#5632)
* refactor(headless): use `WaitStable` for `waitload` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `getNavigationFunc`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitDOM` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitFMP` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitFCP` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitIdle` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): `ActionWaitLoad` waits for `proto.PageLifecycleEventNameLoad`

also rename `Page.WaitLoad` to `Page.WaitStable` method.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitStable` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): supporting `duration` arg for `WaitStable` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: ignore `*.png`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): update `TestActionScreenshot*`

call `ActionWaitFMP` instead of `WaitLoad` before take screenshot

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): chained with `Timeout` when `WaitStable`

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-09-19 19:01:12 +05:30
Dwi Siswanto
841d8913e5
feat(headless): add ActionWaitDialog type (#5545)
* feat(headless): add `dialog` action type

also implement it

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): add `ActionData` for action output datas

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): rm `value` arg for `*Page.HandleDialog`

also:
* expose `err` from \*proto.PageHandleJavaScriptDialog`
* conditional ActionData assignment based on

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): rename to `ActionWaitDialog`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): fix mismatch assertion of `src` output of `ActionGetResource`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): add TestActionWaitDialog test case

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `GetActionDataWithDefault` generic func

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): implement `GetActionDataWithDefault`

to `header` & `status_code`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): use `mapsutil.Map` instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Revert "feat(headless): add `GetActionDataWithDefault` generic func"

This reverts commit fa12e0d6a221c8a7bf62200f69814ee27681f08f.

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-09-02 15:29:52 +05:30
Dwi Siswanto
e0b2542868
feat: conditionally panic-recover (#5553)
* feat: conditionally panic-recover

As discussed with @Mzack9999, we should avoid
overusing panic-recover. We need to review the RCA
first to determine whether this is an exceptional
situation or if it's a higher-level function meant
to recover from a panic. This approach will help
us establish a robust error-handling strategy.

The implementation of panic-recover should be
conditional and NOT applied when running in a CI
environment AND IS temporary. Once we've caught
all errors and made the necessary corrections, we
can remove the deferred recover function.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump `go-ci` to v1.0.2

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(make): add `-race` to `GOFLAGS` in `test`

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-08-28 17:57:45 +05:30
Tarun Koyalwar
23bd0336fb
multiple bug fixes + performance improvements (#5148)
* prototype errkit

* complete errkit implementation

* add cause to all timeouts

* fix request timeout annotation @timeout

* increase responseHeaderTimeout to 8 for stability

* rawhttp error related improvements

* feat: add port status caching

* add port status caching to http

* migrate to new utils/errkit

* remote dialinterface + error cause

* debug dir support using .gitignore debug-*

* make nuclei easy to debug

* debug dir update .gitignore

* temp change (to revert)

* Revert "temp change (to revert)"

This reverts commit d3131f777713b9f80e2275142e80f36340a76d36.

* use available context instead of new one

* bump fastdialer

* fix hosterrorscache + misc improvements

* add 'address' field in error log

* fix js vague errors + pgwrap driver

* fix max host error + misc updates

* update tests as per changes

* fix request annotation context

* remove closed dialer reference

* fix sdk panic issue

* bump retryablehttp-go,utils,fastdialer

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-05-25 00:29:04 +05:30
Ice3man
0b82e8b7aa
feat: added support for context cancellation to engine (#5096)
* feat: added support for context cancellation to engine

* misc

* feat: added contexts everywhere

* misc

* misc

* use granular http timeouts and increase http timeout to 30s using multiplier

* track response header timeout in mhe

* update responseHeaderTimeout to 5sec

* skip failing windows test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-04-25 15:37:56 +05:30
mzack
7e363984b2 Merge branch 'dev' into feat-3072-init-adaptive-speed 2024-04-09 15:19:51 +02:00
Ramana Reddy
8c27ca2591
fix(schema): generation of missing JSON schema definitions (#4995)
* fix(schema): generation of missing JSON schema definitions

* make headers and data to accept multi-type inputs

* misc update
2024-04-08 03:29:42 +05:30
Mzack9999
a8d1393e96 init- using resizable components 2024-04-03 17:50:57 +02:00
kchason
fb3c3d828d
Initial switch of libraries 2024-03-25 15:52:20 -04:00
Tarun Koyalwar
d064801270
fallthrough extracted values in order (#4574)
* fallthrough extracted values in order

* fix unit test & update .gitignore

* remove temp file

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-01-05 05:25:31 +05:30
Tarun Koyalwar
f663d1c9cf
deprecate(remove): file write in extractor using to (#4565)
* fix race-condition & oow in extracted file output

* add mutex for file.Write + set finalizer for os.File

* fix integration test

* disable extractor save to file in lib mode(configurable)

* use sync.Once for init

* disable out of bound image write in headless

* misc updates

* fix headless screenshot test

* fix extractor save to file integration test

* remove 'to' feature in extractors
2024-01-05 03:23:08 +05:30
Tarun Koyalwar
47e75038f0
headless: fix panic + refactor waitevent action (#4465)
* fix waitEvent action

* avoid future panics

* integration test + bug fix

* headless: add max-duration support in waitevent

* fix comment + max-duration input
2023-12-06 19:08:26 +05:30
Dogan Can Bakir
a9efb75d59
introduce disable-cookie (#4292)
* introduce disable-cookie

* remove debug statement

* fix headless template

* increase `-interactions-poll-duration` value to 5

* docs update

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-11-18 13:02:10 +05:30
Tarun Koyalwar
dc44105baf
nuclei v3 : misc updates (#4247)
* use parsed options while signing

* update project layout to v3

* fix .gitignore

* remove example template

* misc updates

* bump tlsx version

* hide template sig warning with env

* js: retain value while using log

* fix nil pointer derefernce

* misc doc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-10-17 17:44:13 +05:30