179 Commits

Author SHA1 Message Date
Dogan Can Bakir
d77eab6e87
Ensure Headless protocol accesses flow protocol variables (#6003) 2025-01-31 13:29:11 +05:30
Roy Reznik
dac38732dc
stop using deprecated mholt/archiver (#5951)
* stop using deprecated mholt/archiver

* Fix CR

* chore: go mod tidy

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Dwi Siswanto <git@dw1.io>
2025-01-31 13:27:16 +05:30
Dwi Siswanto
58ae87cb05
test(dns): update input, requires, and enable recursion (#6014)
* test(dns): update input, requires, and enable recursion

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Update go.mod

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2025-01-30 14:44:29 +05:30
Douglas Danger Manley
e4a0b8ec60
Do not clobber the "tcp" dialer for MySQL (#5681)
This introduces a "nucleitcp" protocol that Nuclei will use when
making MySQL connections as part of its templates.

Previously, this would register (and de-register!) a custom "tcp"
dialer, and that applied globally, so any piece of software that
used a MySQL database and included nuclei in SDK mode would have
its database connections ripped out from under it due to the dialer
hijacking.

By using "nucleitcp" as the protocol, we are free to do whatever
we want with the dialer and not impact any other packages.

Within our `BuildDSN` function, we quietly replace the protocol to
"nucleitcp" if it was "tcp", so nuclei developers don't have to do
anything special to use this functionality; it will always do it.
2025-01-15 00:29:30 +05:30
Ice3man
2ac02f43b0
feat: added improvements to sdk for offlinehttp (#5982) 2025-01-14 01:21:27 +05:30
Dogan Can Bakir
525d2caf66
fix unresolved interactsh-url for raw http templates (#5938) 2024-12-24 20:27:13 +05:30
Dwi Siswanto
f21a82aac3
fix(httpclientpool): rebuild malformed Location URL (#5902)
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
2024-12-19 20:31:41 +05:30
Mzack9999
1e87ca82c8
fix missing browser init (#5896)
* fix missing browser init

* .

* using lazy init

* updating test with new web ui

* go mod

* sandbox test

* non fatal error
2024-12-17 15:38:42 +05:30
Shubham Rasal
be1f634eae
Add Alive Proxy into Options (#5903)
* Move proxy variable from global to options

- Provides ability to pass diff proxy in single nuclei instance using sdk

* add type check (resolve comments)
2024-12-13 04:23:27 +05:30
lvyaoting
a6b5b1dd49 chore: fix some function name in comment
Signed-off-by: lvyaoting <lvyaoting@outlook.com>
2024-12-05 19:34:33 +08:00
Dogan Can Bakir
1f985459b8
add max-os-threads flag (#5622)
* add `max-os-threads` flag

* minor

* use global pkg
2024-11-27 20:06:26 +05:30
Ice3man
b046f7686f
feat: Added time based delay analyzer to fuzzing implementation (#5781)
* feat: added fuzzing output enhancements

* changes as requested

* misc

* feat: added dfp flag to display fuzz points + misc additions

* feat: added support for fuzzing nested path segments

* feat: added parts to fuzzing requests

* feat: added tracking for parameter occurence frequency in fuzzing

* added cli flag for fuzz frequency

* fixed broken tests

* fixed path based sqli integration test

* feat: added configurable fuzzing aggression level for payloads

* fixed failing test

* feat: added analyzers implementation for fuzzing

* feat: misc changes to analyzer

* feat: misc additions of units + tests fix

* misc changes to implementation
2024-11-19 11:51:32 +05:30
Dwi Siswanto
13af7ccd49
fix: data race at protocolstate, contextargs & some outdated test cases (#5820)
* test(flow): update outdated test cases

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(multiproto): update outdated test cases

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: fixed failing tests

* fixed data race

* fixed memgaurdian race conditiong

* test(customtemplates): use test repo

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(customtemplates): add more `{Clone,Pull}Options`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(customtemplates): validate `{Clone,Pull}Options`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* bugfix: fixed failing integration tests for flow and multi

* chore: either 1 or 2 results in interactsh

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
2024-11-19 11:08:25 +05:30
David Maciejak
2039b5c43e
Fix error message in request.go (#5768)
I have that error displayed which sounds grammatically wrong:

Could not execute request for http://x.x.x.x/: could get html element: error occurred executing action: [:RUNTIME] could not navigate to url http://x.x.x.x/login <- navigation failed: net::ERR_ADDRESS_UNREACHABLE
2024-10-25 23:38:19 +05:30
Tarun Koyalwar
40a08a4db3
Code protocol bug fixes and improvements (#5767)
* bump gozero + improve debug mode (code protocol)

* run failed integration again assuming they are flaky

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-10-25 23:35:03 +05:30
Ramana Reddy
7e4b4a8c55
fix: interactsh-url placeholder replacement in variables for network template (#5677) 2024-10-18 20:44:40 +05:30
Ramana Reddy
98948d0266
support stop-at-first-match for network templates (#5554) 2024-10-14 20:54:58 +05:30
Dwi Siswanto
2c832f5590
refactor(vardump): use godump lib (#5676)
* refactor(vardump): use `godump` lib

also increate limit char to `255`.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(vardump): add global var `Limit`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(protocols): rm newline

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(types): add `VarDumpLimit` option

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(vardump): add test cases

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: tidy up mod

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-10-14 19:31:36 +05:30
Dwi Siswanto
cc5c5509dc
feat: global matchers (#5701)
* feat: global matchers

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Ice3man543 <ice3man543@users.noreply.github.com>

* feat(globalmatchers): make `Callback` as type

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: update `passive` term to `(matchers-)static`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): add `origin-template-*` event

also use `Set` method instead of `maps.Clone`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: update `matchers-static` term to `global-matchers`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): clone event before `operator.Execute`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(tmplexec): don't store `matched` on `global-matchers` templ

This will end up generating 2 events from the same
`scan.ScanContext` if one of the templates has
`global-matchers` enabled. This way, non-
`global-matchers` templates can enter the
`writeFailureCallback` func to log failure output.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): initializes `requests` on `New`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(globalmatchers): add `hasStorage` method

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(templates): rename global matchers checks method

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(loader): handle nil `templates.Template` pointer

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Ice3man543 <ice3man543@users.noreply.github.com>
2024-10-14 19:25:46 +05:30
Danny Shemesh
888a732fbc
Unlock memguard global change mutex only when locked (#5714) 2024-10-14 14:18:59 +05:30
Dogan Can Bakir
f0624820d3
update ssl part definitions (#5710) 2024-10-10 03:04:23 +05:30
Tarun Koyalwar
1f945d6d50
consider protocolType in max host error (#5668)
* consider protocolType in max host error

* add mutex when updating internal-event
2024-09-28 18:50:35 +05:30
Dwi Siswanto
c9f67897c4
fix(http): prevent addCNameIfAvailable from using closed Dialer (#5665)
added a check in `addCNameIfAvailable` to ensure
the `Dialer` isnot NIL before attempting to fetch
DNS data.

this prevents potential panics (ex. SIGSEGV) when
the `Dialer` is closed due to an interruption.

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-09-25 22:00:39 +05:30
Dwi Siswanto
9983d7415c
refactor(runner): adjust max-host-error if gt concurrency (#5633)
* refactor(common): use `ParseRequestURI` instead when `NormalizeCacheValue`

also it exports the method

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(runner): adjust `max-host-error` if gt `concurrency`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix lint

* chore(runner): expose adjusted `max-host-error` value

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
2024-09-23 15:57:30 +05:30
Douglas Danger Manley
694835c459
Add more support for fs.FS in template parsing (#5421)
* misc update

* chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#4252)

Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/gin-gonic/gin/releases)
- [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gin-gonic/gin/compare/v1.9.0...v1.9.1)

---
updated-dependencies:
- dependency-name: github.com/gin-gonic/gin
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/docker/docker (#4316)

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.5+incompatible to 24.0.7+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v24.0.5...v24.0.7)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix README_CN.md typos (#4369)

* version update

* Add more support for `fs.FS` in the disk catalog

This adds more support for `fs.FS` in the disk catalog.  This
fixes some places where direct `os` file-related calls were being
made to use the catalog interface instead.

Note that the JavaScript compiler *still* does not work in any
context where the `pkg/js/libs/fs` package is used.  In particular,
the `ReadFilesFromDir` function is hard-coded to use the `os`
package and not respect the catalog.

* Remove some testing artifacts

* Wrap up

* Unwind other changes

* Add a LoadHelperFileFunction to Options

* Use a direct func

* Tweak validation

* Use a function type

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-09-21 02:41:22 +05:30
Dwi Siswanto
4cd065df5f
feat(headless): supporting standard lifecycle events (#5632)
* refactor(headless): use `WaitStable` for `waitload` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `getNavigationFunc`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitDOM` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitFMP` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitFCP` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitIdle` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): `ActionWaitLoad` waits for `proto.PageLifecycleEventNameLoad`

also rename `Page.WaitLoad` to `Page.WaitStable` method.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `WaitStable` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): supporting `duration` arg for `WaitStable` action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: ignore `*.png`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): update `TestActionScreenshot*`

call `ActionWaitFMP` instead of `WaitLoad` before take screenshot

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): chained with `Timeout` when `WaitStable`

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-09-19 19:01:12 +05:30
Ramana Reddy
3d2f31a56f
fix missing template_url for pd signed templates when executed from custom path (#5644) 2024-09-19 18:58:20 +05:30
Dwi Siswanto
841d8913e5
feat(headless): add ActionWaitDialog type (#5545)
* feat(headless): add `dialog` action type

also implement it

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): add `ActionData` for action output datas

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): rm `value` arg for `*Page.HandleDialog`

also:
* expose `err` from \*proto.PageHandleJavaScriptDialog`
* conditional ActionData assignment based on

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): rename to `ActionWaitDialog`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): fix mismatch assertion of `src` output of `ActionGetResource`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(headless): add TestActionWaitDialog test case

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): add `GetActionDataWithDefault` generic func

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(headless): implement `GetActionDataWithDefault`

to `header` & `status_code`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(headless): use `mapsutil.Map` instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Revert "feat(headless): add `GetActionDataWithDefault` generic func"

This reverts commit fa12e0d6a221c8a7bf62200f69814ee27681f08f.

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-09-02 15:29:52 +05:30
Dwi Siswanto
e0b2542868
feat: conditionally panic-recover (#5553)
* feat: conditionally panic-recover

As discussed with @Mzack9999, we should avoid
overusing panic-recover. We need to review the RCA
first to determine whether this is an exceptional
situation or if it's a higher-level function meant
to recover from a panic. This approach will help
us establish a robust error-handling strategy.

The implementation of panic-recover should be
conditional and NOT applied when running in a CI
environment AND IS temporary. Once we've caught
all errors and made the necessary corrections, we
can remove the deferred recover function.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump `go-ci` to v1.0.2

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(make): add `-race` to `GOFLAGS` in `test`

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-08-28 17:57:45 +05:30
Dogan Can Bakir
6b71af448a
Fixed issue with -ms option to scan non accessible host (#5576)
* fail if OnResult callback is not called

* generate error message from error logs

* try..parse..

* fix lint

* add error message to last matcher event

* fix network protocol error logging

* log returned log from ExecuteWithResults

* add back specific logging

* clean up the msg

* minor

* init integration test for -ms

* add tests for http,network,js,ws protocols

* fix lint

* fix network test

* return err for dns protocol

* add integration test for dns protocol
2024-08-28 16:27:43 +05:30
mzack9999
5e102b782b fixing race + nil crash 2024-08-21 16:09:47 +02:00
Doğan Can Bakır
d1f4c98cd7 Revert "remove redundant code"
This reverts commit 35a0d673ad8e12b11e90e8e0090feb26ea042b46.
2024-08-21 15:03:41 +03:00
Doğan Can Bakır
35a0d673ad remove redundant code 2024-08-21 11:36:33 +03:00
Doğan Can Bakır
46782ff90c use sync.Once 2024-08-21 11:26:17 +03:00
Doğan Can Bakır
3064788d35 fix race condition 2024-08-19 23:02:27 +03:00
Tarun Koyalwar
1c76398aea
lint error fixes (#5531)
* lint error fixes

* chore: satisfy non-constant format str in call lint (govet)

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Dwi Siswanto <git@dw1.io>
2024-08-16 20:31:23 +05:30
Ramana Reddy
f29b94521e
fix unresolved variables in dast templates (#5443)
* fix unresolved variables in dast templates

* dedupe interactsh urls

* misc update
2024-08-16 18:19:44 +05:30
Dwi Siswanto
1af29f97a9
feat(http): add skip-secret-file field (#5522)
* feat(http): add `BypassSecretFile` field

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(http): conditionally apply auth strategies

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(http): rename `BypassSecretFile` field to `SkipSecretFile`

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-08-16 18:10:48 +05:30
Ramana Reddy
2609d2d135
feat: add support for multiple auth strategies per target from secrets file (#5500) 2024-08-16 11:59:15 +05:30
Tarun Koyalwar
2df1b2e88e
file proto missing vars in flow & multi-protocol (#5480)
* fix missing template context in file proto

* fix file protocol missing vars

* fix test

* skip example advanced test
2024-08-04 18:14:08 +05:30
Mohammed Diaa
ff23949bb0
Apply input transformation to multi-protocol templates (#5426)
* Apply input transformation to multi-protocol template execution

* Remove ad hoc input transoformation from DNS protocol

* Add SSL protocol input transformer

* Remove ad hoc input transoformation from SSL protocol

* Remove unused function extractDomain from the DNS protocol engine

* transform in flow as well

* bug fix + update test

* bug fix multi proto
:

* bug fix multi proto input

* bug fixes in input transform

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-08-01 20:43:47 +05:30
Tarun Koyalwar
2418319df4
js: generate matcher-status event (#5450)
* js: generate matcher-status event

* isPortOpen: use fastdialer instance

* update sdk unit test

* add docs :)
2024-07-27 02:46:34 +05:30
Dwi Siswanto
6d325a4ebe
feat(http): assign customHeaders to the map directly (#5445)
also add skip expr if header key is "Host"

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-07-26 22:24:35 +07:00
Ramana Reddy
33dbb51505
fix unresolved interactsh-url variable with fuzzing (#5289)
* fix unresolved interactsh variable with fuzzing

* fix variables override with fuzzing
2024-07-26 00:01:05 +05:30
Ramana Reddy
3e1cd27943
Fix stop-at-first-match in headless mode fuzzing (#5330) 2024-07-15 16:57:12 +05:30
Mzack9999
bc229a46ca
Merge pull request #5331 from projectdiscovery/use_containsall
use `stringsutil.ContainsAll`
2024-07-15 13:21:03 +02:00
Dogan Can Bakir
f080d614c3
introduce timeouts config in types.Options (#5228)
* introduce timeout variants

* update instances and add codeexectimeout

* fix test

* default to 10s

* minor

* make timeouts pluggable and rename

* remove residual code

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-07-15 15:57:15 +05:30
mzack9999
3c2af0e47c fixing many data races 2024-07-12 17:23:44 +02:00
mzack
ead444b88b Merge branch 'dev' into use_containsall 2024-07-12 13:05:14 +02:00
Tarun Koyalwar
c9a9bd3bfc
include cname in http output if available (#5389) 2024-07-10 20:43:22 +05:30