125 Commits

Author SHA1 Message Date
Tarun Koyalwar
8125b6805c resolve merge conflicts with dev 2023-08-04 20:21:22 +05:30
Josh Soref
4c1c5301b9
Spelling (#4008)
* spelling: addresses

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: asynchronous

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: basic

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: brute force

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: constant

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: disables

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: engine

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: every time

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: execution

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: false positives

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: from

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: further

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: github

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: gitlab

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: highlight

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: hygiene

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: ignore

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: input

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: item

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: itself

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: latestxxx

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: navigation

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: negative

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: nonexistent

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: occurred

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: override

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: overrides

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: payload

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: performed

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: respective

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: retrieve

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: scanlist

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separated

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separator

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: severity

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: source

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: strategy

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: string

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: templates

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: terminal

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: timeout

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing slash

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: websocket

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

---------

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-08-02 00:03:43 +05:30
Tarun Koyalwar
beb1bf6d2c
headless: automerge and other improvements (#3958)
* headless: automerge and other improvements

* fix typo in function signature
2023-07-28 19:28:20 +05:30
Tarun Koyalwar
d51e058791
add ErrNoMoreRequests for generator (#3918)
* add ErrNoMoreRequests for generator

* fix gh repo name convention

* fix dirname in unit test
2023-07-13 00:51:06 +05:30
Tarun Koyalwar
38129bac18
preserve order of query parameters (#3887)
* preserve order of parameters

* rawhttp version bump

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-07-03 12:43:24 +05:30
Tarun Koyalwar
bdf77005d6 resolve merge conflicts 2023-06-27 20:21:14 +05:30
Ramana Reddy
cddae989f3
Add template option to disable merging target url path with raw request path (#3799)
* add template option to disable merging target url path with raw request path

* rename disable-merge-path -> disable-path-automerge
add integration test
2023-06-19 20:22:17 +05:30
Tarun Koyalwar
e1d3f474a4
support for dynamic variables in template context (multi protocol execution) (#3672)
* multi proto request genesis

* adds template context dynamic vars

* feat: proto level resp variables

* remove proto prefix hacky logic

* implement template ctx args

* remove old var name logic

* improve AddTemplateVars func

* add multi proto comments+docs

* vardump with sorted keys

* fix race condition in ctx args

* default initialize ctx args

* use generic map

* index variables with multiple values

* fix nil cookies

* use synclock map

* fix build failure

* fix lint error

* resolve merge conflicts

* multi proto: add unit+ integration tests

* fix unit tests

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

---------

Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
2023-06-09 19:52:56 +05:30
Mzack9999
0d2d510689
Adding support for constants (#3692)
* adding support for constants

* fixing typo

* adding integration test

* fixing lint issues

* fixing template syntax
2023-05-25 22:02:35 +05:30
Shubham Rasal
449afc0c5c
Issue 3564 var override (#3599)
* Check if the variables are override by other means

- you can override the template variable value using command line flags

* Update lazy eval logic

- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
  protocol variable(global)

* add integration tests

* Add test to check the dsl function working in variable

* gather all generate variables logic in utils

* go mod update

* Refactor the generate variables function

* go mod update+ fix typo

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-05-02 23:49:56 +05:30
Tarun Koyalwar
7f5e4e2336
aws signer: fix missing x-content-sha256 header (#3601)
* fix missing x-content-sha256 header

* fix variable priority in self-contained templates

* remove debug statement

* adds generic raw request parser for self-contained req

* more integration tests

* bug fix: 10x faster race requests

* fix failing integration test
2023-05-01 12:15:35 +05:30
Tarun Koyalwar
4e6ef4490e
duplicated params in self contained requests (#3608)
* fix duplicated params in self-contained+ export extracted values to file

* add integration tests + fix percentage overflow in pb

* fix integration test template id

* integration test: validate if file exists
2023-04-26 12:35:07 +05:30
Shubham Rasal
d0f22f8b73
Issue 3488 http race (#3533)
* debug

* Add body after the request creation

* fix race_count template hangs

* remove printf

* update if condition
2023-04-19 01:57:53 +05:30
Shubham Rasal
45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Tarun Koyalwar
d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
Tarun Koyalwar
21b03a2e8a
bug fix in url path and adds integration tests (#3331)
* fix unsafe edgecases+ adds integration test

* bug fixes and more url testcases

* upgrade cfssl

* fix template id in integration test
2023-02-20 22:26:04 +05:30
Tarun Koyalwar
e622b989fe
fix url re-encoding issues (#3294)
* fix double url encoding in urls

* remove extra slash

* url encode matchedURL
2023-02-10 18:28:28 +05:30
Mzack9999
7556416e5b
adding interactsh support to sni (#3276) 2023-02-07 14:02:10 +05:30
Tarun Koyalwar
628628893c
fix make http request inconsistencies (#3243)
* fix make http request inconsistencies

* remove parameters from http vars

* fix trailingslash unit test

* naming conventions: best practices

* fix publish docs action

* remove branch ref from push
2023-02-01 17:23:28 +05:30
Tarun Koyalwar
0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Tarun Koyalwar
edcab07fec
fix aws signer missing template variables (#3206)
* aws sign: fix missing variables

* signer: add aws defaults

* aws signer default values
2023-01-24 20:50:20 +05:30
Tarun Koyalwar
4aa2002e72
urlencode key characters only (#3150)
* only encode key characters

* improve test cases
2023-01-05 16:41:59 +05:30
Tarun Koyalwar
a5b39dcaa5
automerge url parameters from input and templates (#3010)
* fix automerge url parameters

* fix url encoding & refactor raw request

* handle trailing slash edgecases

* minor code refactoring
2022-12-13 12:09:31 +05:30
Mzack9999
96c1dd3720
Adding custom ip to protocol generated variables (#3011)
* lint errors

* Extending context args support

* Ip => ip
2022-12-10 00:17:03 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
Ice3man
b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
mzack
70cecf83fb Adding custom cancel function 2022-10-10 08:10:07 +02:00
Mzack9999
30054d1fb6
Adding advanced template filtering (#2374)
* Adding advanced template filtering

* fixing bug in slice

* refactoring tests

* adding test cases

* increasing error verbosity

* fixing quoted fields with spaces

* adding more test cases

* fixing merge error

* fixing lint errors

* switching to []string

* updating tag filter tests

* updating functional tests

* fixing functional test cases

* updating syntax
2022-08-25 16:52:08 +05:30
Ice3man
0be596efb4
Added variable debug support with debug mode (#2442)
* Added variable debug support with debug mode

* Added changes as per review comments

* Fixed debug request condition
2022-08-25 15:37:03 +05:30
Ice3man
e7cffad312
Fixed request annotation based timeout bugs + tests + misc (#2476) 2022-08-23 12:45:55 +05:30
Ice3man
2873e6ebc8
Added timeout context cancellation to http requests (#2319) 2022-07-21 21:29:34 +05:30
Ice3man
8040b66370
Added http request timeout support with annotations (#2233)
* Added http request timeout support with annotations

* Added nolint statements for lostcontext

* misc

* misc
2022-06-27 18:36:46 +05:30
Mzack9999
02eaf91e6a
Adding variables support for headless templates (#2064) 2022-05-27 21:31:56 +05:30
Sajad
4f834f1f33
store vars in values instead of payloads to fix #1882 for self contained http templates (#1924)
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-05-23 15:12:58 +05:30
Mzack9999
39c7317ec3
Adding SNI override via request annotations (#1970)
* Adding SNI override via request annotations

* adding cli flag priority
2022-05-12 16:43:56 +05:30
Sami
6ca4374f91
sonar category: String literals should not be duplicated (#1944)
* sonar category: String literals should not be duplicated

* lint error fix

* better naming conventions for constants

* improved naming conventions and methods
2022-05-12 15:40:14 +05:30
Mzack9999
777b75d305 fixing internal bug for unsafe oob 2022-04-20 17:11:14 +02:00
Sandeep Singh
b26ebcfa60
Merge pull request #1805 from projectdiscovery/issue-1289-multiple-host
HTTP Requests Annotation Prototype
2022-04-16 17:23:31 +05:30
Mzack9999
2f9af8cc71 adding support for digest authentication 2022-04-05 11:43:56 +02:00
Mzack9999
7b032b1733 annotation prototype 2022-04-04 09:32:41 +02:00
Ice3man
eaa9db19c0 Misc changes to meta and dynamic http values 2022-03-29 20:36:26 +05:30
Ice3man
bea8955dd6 Evaluate payload helpers before matching 2022-03-29 17:28:29 +05:30
Ice3man
693796789b fix: strip default http/https ports from Host header 2022-01-18 04:13:59 +05:30
mzack
33f6f510b1 Extending http variables list with dns generated variables 2022-01-14 12:00:59 +01:00
Mzack9999
c26a1ac21c
Improving payloads support in AWS self-contained requests (#1443)
* Improving payloads support in AWS self-contained requests

* removing internal only values from output

* handling dynamic values in url
2022-01-09 18:09:50 +05:30
Sajad Parra
0edb4274b1 add matchedTemplates to support template wise stop at first match 2021-12-22 21:42:21 +05:30
Sajad Parra
c0f9c1da70 interactsh stopAtFirstMatch intergration test 2021-12-21 15:24:16 +05:30
Sajad Parra
911045ae9a add stop at first match for interactsh matchers 2021-12-21 14:20:03 +05:30
mzack
e59da29371 improving error/args handling 2021-12-18 20:06:51 +01:00
mzack
714f0c82a9 adding missing return error 2021-12-16 23:41:18 +01:00