106 Commits

Author SHA1 Message Date
Mzack9999
3dca03163c
Automatic target merge in network templates (#3904)
* skip visited actual addressess

* removed test

* adding disable clustering support
2023-07-06 21:33:52 +05:30
Ramana Reddy
6707bc777a
fix showing multiple failure matches per template on -ms set (#3770)
* fix showing multiple failure matchers per template
add integration test

* exclude AS134029 from unit test

* Add flag for match status per request

* chore(deps): bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#3777)

Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3778)

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.83.0 to 0.84.0.
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.83.0...v0.84.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/spf13/cast from 1.5.0 to 1.5.1 in /v2 (#3780)

Bumps [github.com/spf13/cast](https://github.com/spf13/cast) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/spf13/cast/releases)
- [Commits](https://github.com/spf13/cast/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cast
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* enable no-httpx when passive scan is launched (#3789)

* chore(deps): bump github.com/projectdiscovery/fastdialer from 0.0.26 to 0.0.28 in /v2 (#3779)

* chore(deps): bump github.com/projectdiscovery/fastdialer in /v2

Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.0.26 to 0.0.28.
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.0.26...v0.0.28)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump retryabledns to 0.28

* Update the retryabledns

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>

* deprecatedProtocolNameTemplates concurrent map writes (#3785)

* deprecatedProtocolNameTemplates

* use syncLock

* fix lint error

* change version in deprecated warning msg

* comment asnmap expand unit test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

* chore(deps): bump golang from 1.20.4-alpine to 1.20.5-alpine (#3809)

Bumps golang from 1.20.4-alpine to 1.20.5-alpine.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/go-playground/validator/v10 in /v2 (#3810)

Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.11.2 to 10.14.1.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.11.2...v10.14.1)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/rawhttp in /v2 (#3811)

Bumps [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) from 0.1.11 to 0.1.13.
- [Release notes](https://github.com/projectdiscovery/rawhttp/releases)
- [Commits](https://github.com/projectdiscovery/rawhttp/compare/v0.1.11...v0.1.13)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/rawhttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 in /v2 (#3812)

Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.6.1 to 5.7.0.
- [Release notes](https://github.com/go-git/go-git/releases)
- [Commits](https://github.com/go-git/go-git/compare/v5.6.1...v5.7.0)

---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/hmap in /v2 (#3781)

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.11 to 0.0.13.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.11...v0.0.13)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Using safe dereferencing

* adding comment

* fixing and condition

* fixing test id

* adding integration test

* update goflags dependency

* update goflags dependency

* bump goflags v0.1.9 => v0.1.10

* handle failure matcher flags logic at executor itself

* add integration test to matcher status per request

* Adding random tls impersonate (#3844)

* adding random tls impersonate

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>

* Use templateman enhance api to populate CVE info (#3788)

* use templateman enhance api to populate cve info

* rename cve-annotate => tmc
add additional flags to format, lint and enhance template using templateman apis

* minior changes

* remove duplicate code

* misc update

* Add validate and error log option

* print if updated

* print format and enhance only if updated

* make max-request optional

* fix reference unmarshal error

* fix removing self-contained tag

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

* fix matcher status with network protocol

* fix test

* remove -msr flag

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: 三米前有蕉皮 <kali-team@qq.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-06-30 23:32:00 +05:30
Thomas Hendrickson
b6002a5e89 set read and write deadline 2023-06-19 09:53:24 -04:00
Mzack9999
168d9e2c21
Extending multistep support (#3546)
* Extending multistep support

* duping buffer

* adding python pack/unpack

* fixing deps

* bump dsl version

* fixing go sum

* adding inline matchers/extractors

* pointing to latest dsl

* fixing looping

* refining inline matchers/extractors

* bumping dsl

* bump dsl

* show conn read errors

* removing redundant logic

superseded by js layer

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-06-01 06:19:19 +05:30
Mzack9999
0d2d510689
Adding support for constants (#3692)
* adding support for constants

* fixing typo

* adding integration test

* fixing lint issues

* fixing template syntax
2023-05-25 22:02:35 +05:30
Shubham Rasal
449afc0c5c
Issue 3564 var override (#3599)
* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-02 23:49:56 +05:30
Shubham Rasal
45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Mzack9999
7556416e5b
adding interactsh support to sni (#3276) 2023-02-07 14:02:10 +05:30
Mzack9999
093d691c16
Issue 3033 deny list (#3037)
* fixing file deny list + refactoring

* err variable renaming

* removing redundant function

* removing unused code

* adding check on empty operator

* updating tests
2022-12-21 02:29:28 +05:30
Mzack9999
1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
Ice3man
b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
Mzack9999
781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
Ice3man
0be596efb4
Added variable debug support with debug mode (#2442)
* Added variable debug support with debug mode

* Added changes as per review comments

* Fixed debug request condition
2022-08-25 15:37:03 +05:30
Ice3man
aea79638c6 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into variables-introduction 2022-04-06 19:04:27 +05:30
Ice3man
d0d65f8d6b Added integration tests for variables + misc changes 2022-04-02 02:14:00 +05:30
Sami
301307bb77
Issue 1705 save responses on disk (#1727)
* save response on disk

* lint error check

* store raw request/response

* lint error fix

* file path

* mock test fix

* readme update

* .txt extension

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-04-02 00:59:02 +05:30
Ice3man
a0ece302d1 Moved variables to template level + misc 2022-03-31 00:54:35 +05:30
Ice3man
d09e71accf Added variables to http and network 2022-03-30 20:35:46 +05:30
Ice3man
4a9000d22e Fixed network response error + timeout handling 2022-03-01 16:24:02 +05:30
Ice3man
2ae66ff530 Fixed unncessary hostname addition in payloads and output in network 2022-02-25 22:53:51 +05:30
Ice3man
24326389a4
Added interactsh additional variables support (#1468)
* Added interactsh additional variables support

* Added support for interact server rotation

* Misc fixes to interact related modules

* Fixed a crash with interactsh

* Perform deduplication of output extract data before printing
2022-01-13 13:22:43 +05:30
Sajad Parra
0edb4274b1 add matchedTemplates to support template wise stop at first match 2021-12-22 21:42:21 +05:30
Sajad Parra
c0f9c1da70 interactsh stopAtFirstMatch intergration test 2021-12-21 15:24:16 +05:30
Sajad Parra
911045ae9a add stop at first match for interactsh matchers 2021-12-21 14:20:03 +05:30
Sajad Parra
10e5595980 Merge branch 'dev' into dsl-variable-support 2021-11-30 18:41:18 +05:30
Sandeep Singh
be66166b9d
Merge pull request #1282 from projectdiscovery/network-vars
Added additional variables for network + simplified logic
2021-11-30 12:10:26 +05:30
Sajad Parra
d97aee01c3 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dsl-variable-support 2021-11-29 13:36:09 +05:30
Sajad Parra
5deb454a81 merge Hostname variable to payloads in network request 2021-11-29 13:26:03 +05:30
Sandeep Singh
670f0d6775
Merge branch 'dev' into network-vars 2021-11-28 04:42:08 +05:30
forgedhallpass
ec6889931d refactor: linter driven fixes
* x = x + ""  => x += ""
* pre-allocating slice with known size
* added t.Helper() methods in test helpers
* complex if-else conditions replaced by switches
* errors should be checked using error.Is() instead of ==
* function parameter should start with lower case letter
* removed unnecessary type definition
* variable/label naming convention: camelCase instead of snake_case
2021-11-25 17:57:22 +02:00
Sajad Parra
7e9272776d add variable support to dsl, remove dynamicValues from request struct 2021-11-25 16:19:42 +05:30
Ice3man543
6c819d7917 Added additional variables for network + simplified logic 2021-11-23 10:44:31 +05:30
Ice3man
1581c96e4e
Added matched-status flag + template-path and url to output (#1272)
* Added matched-status flag + template-path and url to output
2021-11-22 17:53:25 +05:30
LuitelSamikshya
64a93a4570
enum changes for network type (#1233)
* enum changes for network type
2021-11-18 19:18:47 +05:30
Ice3man
50a816fce9
Fixed a crash with uninitialized interactsh client (#1251)
* Fixed a crash with uninitialized interactsh client
2021-11-16 20:02:39 +05:30
Sandeep Singh
f78ff42dbe
Merge branch 'dev' into more-protocols 2021-11-09 18:27:13 +05:30
Sandeep Singh
952a821c6c
Merge pull request #1208 from projectdiscovery/colored_hexdump
Enhance hexdump content highlighting
2021-11-08 18:30:00 +05:30
Ice3man543
213853c45d Merge from dev 2021-11-08 15:40:18 +05:30
Ice3man543
8ad3ebcd05 Made code changes as per review comments 2021-11-05 03:01:41 +05:30
Sajad Parra
41393fc09d unique payload per interactsh placeholder #1068 2021-11-04 17:13:47 +05:30
Ice3man543
645ae30a47 Moved to an enum for TemplateType in protocols 2021-11-03 19:53:45 +05:30
forgedhallpass
8f6280dc9d refactor: In case of binary data, show a hexadecimal view as well #1080
* small enhancements with regards to dumping responses
2021-11-01 20:45:54 +02:00
forgedhallpass
b91166a034 Merge remote-tracking branch 'origin/dev' into colored_hexdump 2021-11-01 11:57:45 +02:00
forgedhallpass
38f2cf245e feat: In case of binary data, show a hexadecimal view as well #1080
* added Compact hex view to the output if the -vv flag is provided
2021-10-30 16:41:10 +03:00
Alexey Zhuchkov
463c1c0142 Use template path in output request 2021-10-30 14:28:30 +03:00
forgedhallpass
04e3c0165a feat: In case of binary data, show a hexadecimal view as well #1080 2021-10-30 13:17:47 +03:00
Ice3man
aa8b15c1f4
Merge pull request #1111 from projectdiscovery/issue-829-network-read-all
Adding support to read tcp data stream till the end
2021-10-25 17:28:56 +05:30
mzack
ae99a88f52 timer.after => timer 2021-10-19 19:39:33 +02:00
Ice3man543
55eab08faa Slight change to formatting 2021-10-19 21:33:17 +05:30
mzack
0ba3b19f1f fixing merge issues 2021-10-19 11:31:36 +02:00