679 Commits

Author SHA1 Message Date
Tarun Koyalwar
f8c5a45966
add mkdir support in headless screenshot (#3457)
* add mkdir support in headless screenshot

* use filepath to join paths

* print info when screenshot is saved

* change version to v2.9.1-dev

* minor fixings on windows path

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-03-24 00:44:32 +05:30
Mzack9999
c182434130
json templates support (load with flags, run & validate ) (#3424)
* extending template identification logic

* removing test code

* local debug

* json template loading support using flags

* blacklist meta json files

* minor changes

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-16 13:33:59 +05:30
Mzack9999
0bf8fc027d
Fixing nil pointer reference + use map helper (#3421)
* Fixing nil pointer reference + use map helper

* bump tlsx version to v1.0.6

* increase interactsh polling in integration_test

* fix nil pointer dereference in integration_test

* fix lint error

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 21:27:48 +05:30
Austin Traver
0d90a555f6
adds -track-error option to add custom errors to max-host-error watchlist (#3399)
* Allow user to specify for "context deadline exceeded" errors to count toward the max host error count

* Convert flag to a string slice `--track-error`

* Minimize diff

* Add documentation for `-track-error`

* adds unit test & minor improvements

* update flag description

---------

Co-authored-by: Austin Traver <austin_traver@intuit.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-14 13:59:42 +05:30
sullo
62bc659914
Change "Operative System" to "Operating System" (#3400) 2023-03-09 12:36:55 +05:30
Tarun Koyalwar
d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
xm1k3
bab15e122b http.DefaultClient replaced with retryablehttp 2023-03-02 21:21:04 +01:00
xm1k3
886fdcf0a9 started move to retryablehttp 2023-03-02 14:54:01 +01:00
xm1k3
ffd758dcb1 added InitNucleiVersion 2023-03-02 09:37:42 +01:00
xm1k3
5959daa58f removed nucleiVersion var as unused 2023-03-01 18:05:56 +01:00
Mzack9999
d57aec5ec7 converting reporting client to interface 2023-02-07 09:45:49 +01:00
xm1k3
1e5358b1fa
Improve passive templates error handling (#3098)
* fixes on passive templates

* Auto Generate Syntax Docs + JSONSchema [Thu Dec 29 08:47:22 UTC 2022] 🤖

* removed empty line

* warning management

When passive flag is provided we ignore all templates which are not compatible, without posting misleading errors

* removing redundant code

* skip offline errors with err var

* remove check on debug flag + used errors.Is() to check errors

important note for future refactoring: use errorsutil.Is() instead of errors.Is()

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-07 05:10:38 +05:30
Ice3man
197d055c5e
Use httpx as library for http probing (#3261)
* Use httpx as library for http probing

* Changed HTTP method to HEAD for probing
2023-02-06 23:34:33 +05:30
xm1k3
a81c754db5
support env var from report yaml (#3188)
* added support yaml for report options

* better to use .HasPrefix()

* wip: working on unmarshal YAML optimization

* managed yaml tag + nil pointers + unit test

* implemented tests

* removed unused code from reporting + code refactoring

* WIP: code refactoring and tests

* check on env var

* more test coverage and added callback func

* docs + renaming func

* moved callback logic + removed yaml validation

* used yaml decoder

* struct typo

* refactoring walk method with generic signature

* removed yamlwrapper refs, used yaml2 + docs

implemented test to check also fields without yaml tag

* used DecodeAndValidate()

* removed double import reference

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-01 02:38:17 +05:30
Tarun Koyalwar
68d1b2f3f3
fix nuclei panic with ratelimit v0.0.5 (#3257)
* fix ratelimit panic

* fix race conditions in ratelimit

* bump ratelimit to v0.0.6
2023-01-31 21:27:13 +05:30
Mzack9999
6c56a20544
Adding support for nmhe (#3219)
* adding support for nmhe

* updating docs
2023-01-22 15:08:50 +05:30
Tarun Koyalwar
6ebf5a789e
fix host spray race condition (#3213)
* core: bug fixes

* best practices: uniform comments
2023-01-20 23:49:04 +05:30
Ice3man
dbb4de028e
Added clustering support for DNS protocol templates (#3204) 2023-01-17 13:01:20 +05:30
Ice3man
7200e83d47 Fixed panic with non-existent target and no cloud flag typo 2023-01-16 19:06:04 +05:30
Jaideep Khandelwal
a1642be911
Enable/Disable a reporting source (#3183)
* Enable/Disable a reporting source

* misc options update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-15 22:21:16 +05:30
Ice3man
67c094444e
Added cloud scan progress tracking using stats (#3180)
* Added cloud scan progress tracking using stats

* Changed log messsage

* Fixed linting error

* Fixed bug in progress calculation logic

* Changed requests to input with cloud flag

* Changed progress name + removed redundant fields
2023-01-13 13:41:05 +05:30
Siddharth Shashikar
924da4197b
Use target filepath instead of target dir path to upload targets (#3182) 2023-01-12 01:34:58 +05:30
Shubham Rasal
25fcae1493
add reporting source in nuclei-cloud (#3151)
* add reporting source in nuclei-cloud

- `nuclei -cloud -rc reporting-config.yaml`

* update error message

* add severity options for jira,(used for cloud only)
2023-01-10 22:49:01 +05:30
Jaideep Khandelwal
eabd4954cf
Fix panic when when nuclei server is shutdown (#3139) 2023-01-03 23:56:33 +05:30
Ice3man
ded218a88a Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into cloud-bug-fixes 2022-12-28 20:43:12 +05:30
Ice3man
2bf8ff5987 Misc 2022-12-28 20:43:08 +05:30
Mzack9999
260dd1a2c4
Disable include directive preprocessing by default (#3045)
* adding strict syntax check

* returning error on disabled preprocessing

* adding check on matchers
2022-12-28 19:16:34 +05:30
Tarun Koyalwar
ff17d12ced
adds scan strategy (#3075) 2022-12-28 02:18:00 +05:30
Mzack9999
34976029d3
removing most go routine leaks (#3073)
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-24 19:22:14 +05:30
Ice3man
f3d5a15cbe Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into cloud-bug-fixes 2022-12-23 15:34:08 +05:30
Ice3man
3411293e6a Cloud related REST API bug fixes 2022-12-23 11:08:38 +05:30
Sandeep Singh
96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
Ice3man
76e9b34b81 Added omitempty for filtering request 2022-12-20 21:48:12 +05:30
Ice3man
dc792a4036
Added cloud tags etc based filtering (#3070) 2022-12-20 15:06:37 +05:30
Ice3man
7b69b36a4f Review changes 2022-12-20 00:31:13 +05:30
Ice3man
8eda1a2458 Fixed list scan endpoint 2022-12-18 19:16:39 +05:30
Ice3man
51a26ca2c1 Added tablewriter for tabular data writing by default 2022-12-17 23:02:04 +05:30
Ice3man
bfa0bd7bee Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into cloud-templates-targets-sync 2022-12-17 22:52:39 +05:30
sandeep
3fdf3be29a duplicate options + typo updates 2022-12-17 02:51:47 +05:30
Ice3man
3409f9fca3 Added changes as per code review 2022-12-16 23:10:43 +05:30
xm1k3
0e3be82c72
Resuming rewriting the output file contents, so previous data removed (#2890)
* used OpenFile instead of Create()

* reverted to original mode

* fixes and resume flag added

* fix on noTimestapt var

* fix on flag

* better code refactoring

* fix on debug error

* code refactoring on file management
2022-12-15 21:11:23 +05:30
Ice3man
7179beab1c Added target count list 2022-12-13 13:39:10 +05:30
Ice3man
a6c88817fa Added get-templates-targets endpoint + JSON + validation 2022-12-12 23:39:21 +05:30
Ice3man
42bfa08f85 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into cloud-templates-targets-sync 2022-12-12 22:43:55 +05:30
Ice3man
2a1721a162 Use integer IDs for scan tasks 2022-12-12 20:31:15 +05:30
Jaideep Khandelwal
19b56570b5
retry with 504 status code (timeouts) (#2852)
* Incase of 504/timeouts, do retry

* Update the version and pass the custom policy from outside
2022-12-12 16:43:21 +05:30
Siddharth Shashikar
1032189b3a
Add JSON output support for list-scan option (#2876)
* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-12 16:41:28 +05:30
sandeep
82c1e6fef3 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into cloud-templates-targets-sync 2022-12-12 14:14:21 +05:30
Ice3man
d3f47180a4 Misc 2022-12-10 22:21:32 +05:30
Mzack9999
96c1dd3720
Adding custom ip to protocol generated variables (#3011)
* lint errors

* Extending context args support

* Ip => ip
2022-12-10 00:17:03 +05:30