452 Commits

Author SHA1 Message Date
Ramana Reddy
c9634fae72
Issue 3350 matcher condition or not work (#3397)
* fix or condition match even interactsh includes as matcher-part (#3350)

* add integration test

* add new template to integration test

* matcher-condtion: test case for both conditions

* fix lint errors

* upgrade dependencies

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-15 20:45:44 +05:30
Mzack9999
0bf8fc027d
Fixing nil pointer reference + use map helper (#3421)
* Fixing nil pointer reference + use map helper

* bump tlsx version to v1.0.6

* increase interactsh polling in integration_test

* fix nil pointer dereference in integration_test

* fix lint error

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 21:27:48 +05:30
Austin Traver
0d90a555f6
adds -track-error option to add custom errors to max-host-error watchlist (#3399)
* Allow user to specify for "context deadline exceeded" errors to count toward the max host error count

* Convert flag to a string slice `--track-error`

* Minimize diff

* Add documentation for `-track-error`

* adds unit test & minor improvements

* update flag description

---------

Co-authored-by: Austin Traver <austin_traver@intuit.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 13:59:42 +05:30
Shubham Rasal
572c8eb780
Issue 2987 fuzz options (#3355)
* Add override fuzzing type and mode flags

* Update english readme

* Fix failing tests

* Add the integration tests

- validate the command line overriding type and mode for fuzzing
2023-03-06 16:56:38 +05:30
sandeep
c91588830e win os test fix 2023-03-06 01:57:55 +05:30
Tarun Koyalwar
d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
xm1k3
bab15e122b http.DefaultClient replaced with retryablehttp 2023-03-02 21:21:04 +01:00
xm1k3
886fdcf0a9 started move to retryablehttp 2023-03-02 14:54:01 +01:00
Mzack9999
d80cbef51d
Merge pull request #3333 from CodFrm/main
fix some json deserialization issues
2023-02-28 09:02:04 +01:00
Mzack9999
62af038617
Add template sign/verify functionality (#3029)
* add template sign/verify functionality

* fixing syntax
2023-02-26 03:54:46 +05:30
Tarun Koyalwar
21b03a2e8a
bug fix in url path and adds integration tests (#3331)
* fix unsafe edgecases+ adds integration test

* bug fixes and more url testcases

* upgrade cfssl

* fix template id in integration test
2023-02-20 22:26:04 +05:30
王一之
10693efa05 add integration tests 2023-02-17 14:23:28 +08:00
xm1k3
cc2f796d2f
Helper function for IP Formats (#3286)
* implemented ip_format helper function

* added tests on ip_format() helper

* fixes on logic

* fixes related to mapcidr docs

* better error value

* fixes + unit test to check index 11

* added call for integration tests

* fixes on dsl-functions number

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-17 01:47:40 +05:30
Mzack9999
d608ffaeb2
clear after stop (#3312)
* clear after stop

* fixing data races

* adding atomic cache

* fixing lint errors

* fixing imports
2023-02-13 16:46:41 +05:30
Mzack9999
1fbd2c069a
adding json schema for extractor (#3240)
* adding json schema for extractor

* adding argument validation
2023-02-07 14:01:34 +05:30
Tarun Koyalwar
628628893c
fix make http request inconsistencies (#3243)
* fix make http request inconsistencies

* remove parameters from http vars

* fix trailingslash unit test

* naming conventions: best practices

* fix publish docs action

* remove branch ref from push
2023-02-01 17:23:28 +05:30
Tarun Koyalwar
0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Mzack9999
6c56a20544
Adding support for nmhe (#3219)
* adding support for nmhe

* updating docs
2023-01-22 15:08:50 +05:30
Jaideep Khandelwal
a1642be911
Enable/Disable a reporting source (#3183)
* Enable/Disable a reporting source

* misc options update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-15 22:21:16 +05:30
xm1k3
63f1b9c2e5
Add timestamp for debug logs (#3110)
* replaced and tested logs on debug

* timestamp on debug logs

* removed debug.timestamp and used instead new implementation on gologger

* renamed global function to SetTimestamp(bool)

* used logger

* used DefaultLogger

* updated gologger ref

github.com/projectdiscovery/gologger@1c357ca

* bumping gologger

* resolved problem related to failing test

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-01-10 21:01:16 +05:30
Mzack9999
d956275e98
Adding content length edge cases (#3147)
* adding content length edge cases

* fixing CL behavior

* suppressing -1 error

* fixing path
2023-01-07 18:06:44 +05:30
Mzack9999
8beb6b06f9
Extending advanced filtering (#3146)
* adding more metadata to advanced filtering

* adding functional test cases

* converting metadata to lowercase

* misc update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-05 21:59:58 +05:30
Mzack9999
a96f764959
extending request/response hijacking with native calls (#3091)
* extending request/response hijacking with native calls

* fixing tests
2023-01-05 17:26:18 +05:30
Sandeep Singh
c273cbc8cb
Merge pull request #3134 from projectdiscovery/issue-3131-unused-code
Removing unused code in integration tests
2023-01-04 11:59:12 +05:30
Tarun Koyalwar
e66ed30cec
fix missing trailing slash (#3127)
* raw: fix missing trailing slash

* adds rawpath integration test

* rename trailing slash test
2023-01-03 23:45:34 +05:30
Mzack9999
463f4dd8c8 removing unused code in integration tests 2023-01-03 08:24:33 +01:00
Taufik Mulyana
7093180b6d
Add support to query DNS TLSA record (#3054)
* Add support to query DNS TLSA record

* fix build test

* fix ci-lint

* set expected to 0

* test domain update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 21:12:02 +05:30
Mzack9999
31ee08ecf0
Initialize flags with nil slice (#3089) 2022-12-28 02:29:51 +05:30
Tarun Koyalwar
ff17d12ced
adds scan strategy (#3075) 2022-12-28 02:18:00 +05:30
Mzack9999
34976029d3
removing most go routine leaks (#3073)
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-24 19:22:14 +05:30
Sandeep Singh
96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
xm1k3
bbb561b097
CLI variables are not accessible in SSL Protocol (#3069)
* added vars payload also in ssl

* fix on ssl.go, moved function on payloadValues creation

* added integration test

* rebase + minor changes

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-12-21 02:02:18 +05:30
Ice3man
3904d541be
Added http probing to clustering + add disable-clustering flag (#3019)
* Added http probing to clustering + add disable-clustering flag

* misc update

* Commented out failing test

* Fixed lint error

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-11 18:06:21 +05:30
Ice3man
b95501e641 Misc 2022-12-07 22:28:45 +05:30
Ice3man
ccfa249f14
Ignore .git and .github during checksum generate (#2990) 2022-12-07 14:38:29 +05:30
Tarun Koyalwar
2874a9dff7
fix -iv & minor bugs (#2951)
* fix -iv & minor bugs

* fix ipversion defaults

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-12-05 18:22:04 +05:30
xm1k3
628b96f768
added force http2 option (#2919)
* added force http2 option

* implemented http2 with transport method

* fix and added forcehttp on clientpool

* updated readme with new flag

* option update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-04 22:32:01 +05:30
Ice3man
514c6e2d1e
Added timestamp optional flag + user-agent to probing (#2962)
* Added timestamp optional flag + user-agent to probing

* fix typo

* misc update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-04 22:16:55 +05:30
Sandeep Singh
6ae9eee8d3
README + misc updates (#2961)
* readme update

* version + banner update

* misc option update

* go fmt'ed code

* misc update
2022-12-04 20:51:33 +05:30
Shubham Rasal
d5a09e733a
Issue 2772 s3 provider support (#2825)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Remove github and aws update variables from flag

* Rename CustomTemplateProvider to Provider

* Update integration and function command in makefile

* Update github test case, accept token

* readme update

* go mod tidy

* Update build-test.yml

* handle empty dir in s3

* Add requested changes

- download/update s3 and github only when `-ut` is passed
- only print the missing env variable for s3
- add the custom templates path in
  ~/.config/nuclei/.template-config.json

* print custom paths only if exists in config file

* misc update

* tag update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-12-02 03:57:00 +05:30
Víctor
9c17284616
Display tpl contents (#2906)
* New "td" flag, displays the highlighted template contents
New verboseTemplate method to avoid duplicate code
Grouped (and sorted) template list per directory

* Updated README about the td flag

* Going back to the previous template list format

The new one can't be pipelined

* Implicit template list on template display
Respect --no-color option to disable colors when -td is used

* misc option update

Co-authored-by: Víctor Zamanillo <victor.zamanillo@cifraeducacion.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-29 17:48:44 +05:30
Mzack9999
440bfe030a Merge branch 'dev' into issue-2832-race-condition-test 2022-11-24 22:10:42 +01:00
Thibault Soubiran
67fda109b2
Fix usage typos (#2889) 2022-11-24 21:18:02 +05:30
xm1k3
2d76498692 fixes 2022-11-23 14:30:12 +01:00
xm1k3
3c978e8a46 removed some targets 2022-11-23 10:51:04 +01:00
xm1k3
632c68a996 added multiple targets on gh action 2022-11-23 10:37:30 +01:00
Ice3man
e7fb40a413 Added sandboxing for payload files and requests 2022-11-16 13:49:24 +05:30
Shubham Rasal
6b142d794a
Issue 2254 uncover integration (#2786)
* nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/

- `nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/`

* Add automatic template execution using metadata

- Query uncover after the template is loaded.
- Add the received hosts to the input provider from uncover
- Make NormalizeStoreInputValue() function public to add hosts from the
  runner after uncover hosts received.

* run go mod tidy

* Remove unnecessary comments

* Resolve the requested changes

- move uncover code to protocols/common/uncover package
- Use uncover delay to create uncover rate limiter
- Use single ratelimiter object and remove not required ratelimiters
- Create Set() method for input provider interface
- Rename normalizeStoreInputValue to Set() method

* Solved the uncover running twice.

- flag StringSliceVarP adds the default value twice in the variable
- Check if provider keys exists or not
- Add uncover help block to english readme.md

* Add uncover field functionality

- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf host
- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf ip:port

* Update error messages and solve nuclei hang for wrong uncover engine

- Get uncover engine values from uncover package

* Resolve merge conflicts

* misc option update

* Update logging for templates

- remove duplicate env log printing
- Log message for template queries

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-16 11:12:39 +05:30
forgedhallpass
0c588cf9af
Fixed typo in the -stop-at-first-match instead of path (#2850) 2022-11-15 15:47:09 +05:30
forgedhallpass
0295ca19bc
Add split DSL function (#2838)
* Add support for showing overloaded DSL method signatures

* Add `split` DSL function #2837

* fixing lint warnings

* replacing faulty regex with strings methods

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-11-14 06:08:12 +05:30