265 Commits

Author SHA1 Message Date
Ice3man
ecc1964ad9 Added optional doNotCache to protocols.ExecuterOptions 2023-02-13 18:00:25 +05:30
Mzack9999
454a883241
Using retryablehttp for templates loading (#3291)
* Using retryablehttp for templates loading

* Update compile_test.go
2023-02-09 19:15:44 +05:30
Tarun Koyalwar
a62e57f6c0
fix publish docs (#3296)
* add branch ref while checkout

* Auto Generate Syntax Docs + JSONSchema [Thu Feb  9 13:36:50 UTC 2023] 🤖

---------

Co-authored-by: GitHub Action <action@github.com>
2023-02-09 19:14:43 +05:30
xm1k3
1e5358b1fa
Improve passive templates error handling (#3098)
* fixes on passive templates

* Auto Generate Syntax Docs + JSONSchema [Thu Dec 29 08:47:22 UTC 2022] 🤖

* removed empty line

* warning management

When passive flag is provided we ignore all templates which are not compatible, without posting misleading errors

* removing redundant code

* skip offline errors with err var

* remove check on debug flag + used errors.Is() to check errors

important note for future refactoring: use errorsutil.Is() instead of errors.Is()

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-02-07 05:10:38 +05:30
GitHub Action
afe2114d58 Auto Generate Syntax Docs + JSONSchema [Mon Feb 6 23:25:05 UTC 2023] 🤖 2023-02-06 23:25:05 +00:00
Ice3man
78c4b9b7d2
Added clustering support for TLS templates (#3209)
* Added clustering support for DNS protocol templates

* Added clustering support for TLS templates

* Fixed randomly populated info block in ssl templates

* Moved to a switch-case + added tests for clustering
2023-01-17 18:20:05 +05:30
Ice3man
dbb4de028e
Added clustering support for DNS protocol templates (#3204) 2023-01-17 13:01:20 +05:30
Ice3man
67c094444e
Added cloud scan progress tracking using stats (#3180)
* Added cloud scan progress tracking using stats

* Changed log messsage

* Fixed linting error

* Fixed bug in progress calculation logic

* Changed requests to input with cloud flag

* Changed progress name + removed redundant fields
2023-01-13 13:41:05 +05:30
shubhamrasal
b03a90df50 fix template path bug, pass the filepath in executeroptions 2023-01-03 12:17:56 +05:30
Mzack9999
2c3453500c
Adding empty comparison for scan strategy (#3100)
* Adding empty edge case to scan strategy

* Auto Generate Syntax Docs + JSONSchema [Thu Dec 29 10:10:55 UTC 2022] 🤖

* adding test file

* removing test file

Co-authored-by: GitHub Action <action@github.com>
2022-12-29 20:12:52 +05:30
Shubham Rasal
91f6fdfb36
Merge pull request #3031 from projectdiscovery/parse_from_reader
Add ParseTemplateFromReader function
2022-12-27 20:47:49 +05:30
Tarun Koyalwar
aee0870617
scanallip handle edge cases (#3080)
* bug fix:remove port during dns resolution

* scanallip fix edge cases

* add scanallips testcases

* workflow fix

* removing pull cmd

* Auto Generate Syntax Docs + JSONSchema [Sat Dec 24 13:29:21 UTC 2022] 🤖

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
2022-12-24 19:03:23 +05:30
shubhamrasal
406d0c1873 Rename function from ParseFromReader to ParseTemplateFromReader 2022-12-22 11:04:16 +05:30
Sandeep Singh
96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
shubhamrasal
b51aac290e Add tests folder 2022-12-19 19:15:04 +05:30
shubhamrasal
cff0d88742 Add tests for parse function
- update the ioutils to io.ReadAll
- remove extra line
2022-12-19 18:51:20 +05:30
shubhamrasal
de3dda0030 Update the Parse function to use ParseFromReader 2022-12-15 17:41:07 +05:30
shubhamrasal
198d84ec25 Add ParseFromReader function 2022-12-13 12:57:47 +05:30
Ice3man
3904d541be
Added http probing to clustering + add disable-clustering flag (#3019)
* Added http probing to clustering + add disable-clustering flag

* misc update

* Commented out failing test

* Fixed lint error

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-11 18:06:21 +05:30
Víctor
b77787e9bd
OfflineHTTPProtocol mapping (#2993)
* OfflineHTTPProtocol mapping, fixes #2988

* Reject not mapped protocols

Co-authored-by: Víctor Zamanillo <victor.zamanillo@cifraeducacion.com>
2022-12-11 17:34:25 +05:30
GitHub Action
bf1fb7bcb7 Auto Generate Syntax Docs + JSONSchema [Sun Dec 4 17:58:29 UTC 2022] 🤖 2022-12-04 17:58:29 +00:00
Sandeep Singh
6ae9eee8d3
README + misc updates (#2961)
* readme update

* version + banner update

* misc option update

* go fmt'ed code

* misc update
2022-12-04 20:51:33 +05:30
Mzack9999
1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
GitHub Action
411e39c40b Auto Generate Syntax Docs + JSONSchema [Tue Nov 1 15:01:38 UTC 2022] 🤖 2022-11-01 15:01:38 +00:00
Ice3man
363ffb75db
Added probing for URL + input based on protocol (#2614)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Added probing for URL + input based on protocol

* Remove debug comments

* Fixed typo

* Fixed failing tests

* Fixed workflow matcher condition + tests

* Fixed workflow item name

* Switch to if-else

* Fixed review comment strict

* Increase bulk size

* Added default port for SSL protocol + misc changes

* Fixed failing tests

* Fixed misc changes to executer

* Fixed failing self-contained and offlinehttp tests

* Fixed atomic increment operation

* misc update

* Fixed failing builds

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-20 17:23:00 +05:30
Mzack9999
781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
GitHub Action
935dd68355 Auto Generate Syntax Docs + JSONSchema [Wed Sep 28 22:44:26 UTC 2022] 🤖 2022-09-28 22:44:26 +00:00
GitHub Action
3ad1fec1d2 Auto Generate Syntax Docs + JSONSchema [Tue Sep 20 21:37:56 UTC 2022] 🤖 2022-09-20 21:37:56 +00:00
Ice3man
03c7e89d9e
Added workflow names based condition (#2594)
* Added workflow names based condition

* Added conditional filtering to workflow executor

* Replaced names with single name stringslice

* Fixed workflow matcher condition + tests

* Fixed workflow item name
2022-09-19 16:49:30 +05:30
Ice3man
7e209dad01
Added multi-protocol requests support for templates (#2611) 2022-09-16 23:10:39 +05:30
Ice3man
8892250583 Updated goflags to latest + misc 2022-08-27 19:35:17 +05:30
Ice3man
67d5769cd9
Added initial catalog interface implementation (#2318)
* Added initial catalog interface implementation

* Added OpenFile to Catalog + disk catalog implementation

* Fixed merge issues

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
Mike Rheinheimer
9efba05e0c
expose hosterrorscache.Cache as an interface (#2291)
* expose hosterrorscache as an interface, change signature to capture the error reason

* use the hosterrorscache.CacheInterface as struct field so users of Nuclei embedded can provide their own cache implementation

Co-authored-by: Mike Rheinheimer <mrheinheimer@atlassian.com>
2022-07-19 02:05:53 +05:30
Mzack9999
3c945f6ae9
Adding stricter check on offline templates list (#2213) 2022-07-11 22:38:07 +05:30
Ice3man
7875b06fc8
Added exclude-matchers support for template & matchers (#2218)
* Added exclude-matchers support for template & matchers

* Fixed panics due to typo

* Added support for only template ID + misc cleanup
2022-06-24 23:09:27 +05:30
Sajad
b715a601e8
Adding nil operator check on clustering (#2173)
* add nil operator check

* move nil pointer check
2022-06-16 14:41:05 +05:30
tanimdiucse123
e575f53be7
Empty string test can be improved (#2115)
It is not recommended to use len for empty string test.

A string can be tested for its emptiness either by treating it as a slice and calculating the length of the slice, or by treating it as a string and directly comparing the value. While both produce identical code when compiled, it makes more sense to treat a string as itself, than a slice, for the sake of comparison of values.

Examples

Bad practice

len(s) == 0

Recommended

s == ""

The recommended practice is considered more idiomatic in Go.

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-06-11 14:11:43 +05:30
GitHub Action
737132bc88 Auto Generate Syntax Docs + JSONSchema [Mon Apr 25 08:40:12 UTC 2022] 🤖 2022-04-25 08:40:12 +00:00
Sandeep Singh
d3b4a86d2a
Merge pull request #1785 from projectdiscovery/variables-introduction
Added variables for nuclei + refactor variable handling
2022-04-21 17:59:01 +05:30
GitHub Action
a734b9d6c1 Auto Generate Syntax Docs + JSONSchema [Thu Apr 21 11:19:39 UTC 2022] 🤖 2022-04-21 11:19:39 +00:00
GitHub Action
ef18886d39 Auto Generate Syntax Docs + JSONSchema [Fri Apr 15 13:31:14 UTC 2022] 🤖 2022-04-15 13:31:14 +00:00
Ice3man
aea79638c6 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into variables-introduction 2022-04-06 19:04:27 +05:30
Ice3man
a0ece302d1 Moved variables to template level + misc 2022-03-31 00:54:35 +05:30
Sandeep Singh
6ede70116d
Merge pull request #1762 from projectdiscovery/clustering-improvements
Added comments to clustering + misc improvements
2022-03-30 12:54:33 +05:30
GitHub Action
924b615a6c Auto Generate Syntax Docs + JSONSchema [Tue Mar 29 12:22:43 UTC 2022] 🤖 2022-03-29 12:22:43 +00:00
GitHub Action
f4e6e892e0 Auto Generate Syntax Docs + JSONSchema [Mon Mar 21 10:22:10 UTC 2022] 🤖 2022-03-21 10:22:10 +00:00
Ice3man
32bc6c1718 Misc changes, moving things around as per review 2022-03-14 12:32:05 +05:30
Ice3man
9da8c0593b Added comments to clustering + misc improvements 2022-03-11 22:08:30 +05:30
GitHub Action
165c9e7013 Auto Generate Syntax Docs + JSONSchema [Tue Mar 1 17:21:24 UTC 2022] 🤖 2022-03-01 17:21:24 +00:00