177 Commits

Author SHA1 Message Date
Mzack9999
fa199ed3b3
Improving clientpool with client certificates (#3851)
* Improving clientpool with client certificates

* adding test case

* Revert "Merge branch 'dev' into issue-3800-client-cert"

This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.

* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""

This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.

* go fmt

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 17:45:12 +05:30
Ramana Reddy
cddae989f3
Add template option to disable merging target url path with raw request path (#3799)
* add template option to disable merging target url path with raw request path

* rename disable-merge-path -> disable-path-automerge
add integration test
2023-06-19 20:22:17 +05:30
Mzack9999
200faa107e adding integration test 2023-06-13 17:27:26 +02:00
Mzack9999
18e2d2cb24 fixing test id 2023-06-13 17:24:31 +02:00
Mzack9999
a7fb15d0bd
Adding support for code templates (#2930)
* Adding support for code templates

* adding support for python, powershell and echo (test)

* removing debug code

* introducing command + trivial trust store mechanism

* updating tests

* adding basic tests

* removing deprecated oracle

* mod tidy

* adding signature proto with debug prints

* removing debug code

* fixing test

* fixing param order

* improving test conditional build

* disable file+offlinehttp+code with cloud

* adding env vars

* removing debug code

* reorganizing test folders

* adding code template test prototype with dummy priv/pub keys

* bump go to 1.20

* fixing go version

* fixing lint errors

* adding fatal on pub-key test failure

* switching to ecdsa asn1

* removing unused signature

* fixing signature

* adding more tests

* extending core with engine args + powershell win test

* adding unsigned code test

* skip template signing in particular test case

* improving test coverage

* refactoring key names + adding already signed algo

* removing debug code

* fixing syntax

* fixing lint issues

* removing test template

* fixing dns tests path

* output fmt

* adding interact

* fixing lint issues

* adding -sign cli helper

* fixing nil pointer + parse inline keys

* making rsa default

* adding code prot. ref

* moving file to correct loc

* moving test

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

* add retry to py-interactsh integration test

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
2023-06-09 20:54:24 +05:30
Tarun Koyalwar
e1d3f474a4
support for dynamic variables in template context (multi protocol execution) (#3672)
* multi proto request genesis

* adds template context dynamic vars

* feat: proto level resp variables

* remove proto prefix hacky logic

* implement template ctx args

* remove old var name logic

* improve AddTemplateVars func

* add multi proto comments+docs

* vardump with sorted keys

* fix race condition in ctx args

* default initialize ctx args

* use generic map

* index variables with multiple values

* fix nil cookies

* use synclock map

* fix build failure

* fix lint error

* resolve merge conflicts

* multi proto: add unit+ integration tests

* fix unit tests

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

---------

Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
2023-06-09 19:52:56 +05:30
Dogan Can Bakir
a4ca2021cd
Add headless header and status matchers (#3794)
* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo
2023-06-09 15:03:03 +05:30
Shubham Rasal
a34b94e62f
Issue 3339 headless fuzz (#3790)
* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-06-09 05:50:44 +05:30
Mzack9999
0d2d510689
Adding support for constants (#3692)
* adding support for constants

* fixing typo

* adding integration test

* fixing lint issues

* fixing template syntax
2023-05-25 22:02:35 +05:30
Shubham Rasal
9c2fa8f9c4
Add payload in dns protocol (#3632)
* add execute function in dns

* Add payload in dns protocol

* Add integration test to cover dns payload

- also check command line overriding a payload variable

* Update matchedAt and remove trailing dot

* Consider payload data for request count

- Update verbose output to print question
- Update dns requests Requests function to consider payload data

* update gitignore

* bump nuclei version to v2.9.4-dev

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-11 03:26:29 +05:30
Shubham Rasal
449afc0c5c
Issue 3564 var override (#3599)
* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-02 23:49:56 +05:30
Shubham Rasal
f640187709
Expose DNS fields for matchers and extractors (#3613)
* Extend dns extractor to dns answer records

* add test template

* Ignore error for dns variables are not found

* Add all the records of answer section

* Fixed the wrong typecasting
2023-05-02 17:13:11 +05:30
Shubham Rasal
6ebb8e98f4
Fix wrong template loading in dev branch (#3629)
* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded
2023-05-02 15:12:55 +05:30
Tarun Koyalwar
7f5e4e2336
aws signer: fix missing x-content-sha256 header (#3601)
* fix missing x-content-sha256 header

* fix variable priority in self-contained templates

* remove debug statement

* adds generic raw request parser for self-contained req

* more integration tests

* bug fix: 10x faster race requests

* fix failing integration test
2023-05-01 12:15:35 +05:30
Tarun Koyalwar
4e6ef4490e
duplicated params in self contained requests (#3608)
* fix duplicated params in self-contained+ export extracted values to file

* add integration tests + fix percentage overflow in pb

* fix integration test template id

* integration test: validate if file exists
2023-04-26 12:35:07 +05:30
Mzack9999
6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
Shubham Rasal
45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Ramana Reddy
c9634fae72
Issue 3350 matcher condition or not work (#3397)
* fix or condition match even interactsh includes as matcher-part (#3350)

* add integration test

* add new template to integration test

* matcher-condtion: test case for both conditions

* fix lint errors

* upgrade dependencies

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-15 20:45:44 +05:30
Shubham Rasal
572c8eb780
Issue 2987 fuzz options (#3355)
* Add override fuzzing type and mode flags

* Update english readme

* Fix failing tests

* Add the integration tests

- validate the command line overriding type and mode for fuzzing
2023-03-06 16:56:38 +05:30
Tarun Koyalwar
d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
xm1k3
f26e01551e
resolve() helper function (#3321)
* started the implementation of resolve helper function

* fixes go mod and sum

* fixes and use makeDslWithOptionalArgsFunction signature

* added tests

* added more dnsTypes based on dnsx docs

* used dns client pool

* dsl functions

* fixes on dnsclientpool, added init()

* go mod tidy

* go mod tidy

* dsl signature makeMultiSignatureDslFunction

* error on len

* managed mx

* fix on mx record

* dns types managed with FirstNonZero func

* error handling

* utils to stable version

* version bump

* fixing var name

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-03-04 04:21:24 +05:30
Mzack9999
d80cbef51d
Merge pull request #3333 from CodFrm/main
fix some json deserialization issues
2023-02-28 09:02:04 +01:00
Tarun Koyalwar
21b03a2e8a
bug fix in url path and adds integration tests (#3331)
* fix unsafe edgecases+ adds integration test

* bug fixes and more url testcases

* upgrade cfssl

* fix template id in integration test
2023-02-20 22:26:04 +05:30
王一之
10693efa05 add integration tests 2023-02-17 14:23:28 +08:00
xm1k3
cc2f796d2f
Helper function for IP Formats (#3286)
* implemented ip_format helper function

* added tests on ip_format() helper

* fixes on logic

* fixes related to mapcidr docs

* better error value

* fixes + unit test to check index 11

* added call for integration tests

* fixes on dsl-functions number

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-17 01:47:40 +05:30
Tarun Koyalwar
c49383de98 detect data race panics in integration_tests 2023-02-10 19:28:07 +05:30
Tarun Koyalwar
d18fa6f6b2 fix data race in race requests 2023-02-06 16:18:30 +05:30
Tarun Koyalwar
628628893c
fix make http request inconsistencies (#3243)
* fix make http request inconsistencies

* remove parameters from http vars

* fix trailingslash unit test

* naming conventions: best practices

* fix publish docs action

* remove branch ref from push
2023-02-01 17:23:28 +05:30
xm1k3
a81c754db5
support env var from report yaml (#3188)
* added support yaml for report options

* better to use .HasPrefix()

* wip: working on unmarshal YAML optimization

* managed yaml tag + nil pointers + unit test

* implemented tests

* removed unused code from reporting + code refactoring

* WIP: code refactoring and tests

* check on env var

* more test coverage and added callback func

* docs + renaming func

* moved callback logic + removed yaml validation

* used yaml decoder

* struct typo

* refactoring walk method with generic signature

* removed yamlwrapper refs, used yaml2 + docs

implemented test to check also fields without yaml tag

* used DecodeAndValidate()

* removed double import reference

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-01 02:38:17 +05:30
Tarun Koyalwar
0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Mzack9999
d956275e98
Adding content length edge cases (#3147)
* adding content length edge cases

* fixing CL behavior

* suppressing -1 error

* fixing path
2023-01-07 18:06:44 +05:30
Mzack9999
a96f764959
extending request/response hijacking with native calls (#3091)
* extending request/response hijacking with native calls

* fixing tests
2023-01-05 17:26:18 +05:30
Tarun Koyalwar
e66ed30cec
fix missing trailing slash (#3127)
* raw: fix missing trailing slash

* adds rawpath integration test

* rename trailing slash test
2023-01-03 23:45:34 +05:30
Mzack9999
a4ce231983
Replacing unstable interactsh with scanme.sh (#3107) 2022-12-30 20:13:07 +05:30
Taufik Mulyana
7093180b6d
Add support to query DNS TLSA record (#3054)
* Add support to query DNS TLSA record

* fix build test

* fix ci-lint

* set expected to 0

* test domain update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 21:12:02 +05:30
xm1k3
bbb561b097
CLI variables are not accessible in SSL Protocol (#3069)
* added vars payload also in ssl

* fix on ssl.go, moved function on payloadValues creation

* added integration test

* rebase + minor changes

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-12-21 02:02:18 +05:30
forgedhallpass
0295ca19bc
Add split DSL function (#2838)
* Add support for showing overloaded DSL method signatures

* Add `split` DSL function #2837

* fixing lint warnings

* replacing faulty regex with strings methods

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-11-14 06:08:12 +05:30
Mzack9999
1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
Ice3man
b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
Ice3man
d956f08cb9
Added attack-type option to override template attack-type (#2724)
* Added attack-type option to override template attack-type

* Added docs + integration tests
2022-10-19 03:51:45 +05:30
Mzack9999
9493dfdb20
Adding automatic request condition detection (#2707)
* Adding automatic request condition detection

* adding missing checks on part

* test update as per latest change

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-15 15:19:04 +05:30
Mzack9999
781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
Mzack9999
18f14b631c
Adding same host redirect support (#2655)
* simplifying test syntax

* adding same host redirect + refactoring redirect handling

* adding missing file

* adding support for template syntax

* adding integration test

* updating options

* fixing issue on same host redirect
2022-09-29 04:11:28 +05:30
Myung-jong Kim
9eea441b0e
Add sort(list), sort(string), uniq(list), uniq(string) helper functions (#2372)
* Add feature in join() to sort a single string slice

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>

* Add sort helper function and related tests

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>

* Add uniq helper function and related tests

Signed-off-by: mjkim610 <mjkim610@gmail.com>

Signed-off-by: Myung-jong Kim <mjkim610@gmail.com>
Signed-off-by: mjkim610 <mjkim610@gmail.com>
2022-09-27 02:59:13 +05:30
Sami
f3eb0daa39
additional dsl functions (#2550)
* additional dsl functions

* avoid conversion at each iteration
2022-09-08 14:25:34 +05:30
Ice3man
e193e7c87e
Added tlsx integration to nuclei (#2522)
* Added tlsx integration to nuclei

* tls tests fix

* Added helper functions + upgrade tlsx to fix

* go mod update

* workflow fix to race test on windows

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-01 23:56:55 +05:30
James Turner
42a0732d68
Add sha512 support (#2517) 2022-08-31 12:36:02 +05:30
Ice3man
e7cffad312
Fixed request annotation based timeout bugs + tests + misc (#2476) 2022-08-23 12:45:55 +05:30
Sajad
c4ba2b4edb
Additional helpers (#2359)
* add starts_with, ends_with helper functions

* add test cases, update dsl signatures

* change split_starts_with to line_starts_with
2022-08-06 23:16:03 +05:30
Mzack9999
57b2ec7ade
Adding offline http tests (#2276) 2022-07-12 02:58:36 +05:30