1863 Commits

Author SHA1 Message Date
GitHub Action
afe2114d58 Auto Generate Syntax Docs + JSONSchema [Mon Feb 6 23:25:05 UTC 2023] 🤖 2023-02-06 23:25:05 +00:00
Tarun Koyalwar
d18fa6f6b2 fix data race in race requests 2023-02-06 16:18:30 +05:30
Tarun Koyalwar
32a6adb82e
fix data race in payload (#3265) 2023-02-05 00:00:01 +05:30
Tarun Koyalwar
628628893c
fix make http request inconsistencies (#3243)
* fix make http request inconsistencies

* remove parameters from http vars

* fix trailingslash unit test

* naming conventions: best practices

* fix publish docs action

* remove branch ref from push
2023-02-01 17:23:28 +05:30
Sandeep Singh
2a24025397
go mod + version update (#3238) 2023-01-24 22:58:34 +05:30
Tarun Koyalwar
0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Tarun Koyalwar
edcab07fec
fix aws signer missing template variables (#3206)
* aws sign: fix missing variables

* signer: add aws defaults

* aws signer default values
2023-01-24 20:50:20 +05:30
Mzack9999
6c56a20544
Adding support for nmhe (#3219)
* adding support for nmhe

* updating docs
2023-01-22 15:08:50 +05:30
Tarun Koyalwar
6ebf5a789e
fix host spray race condition (#3213)
* core: bug fixes

* best practices: uniform comments
2023-01-20 23:49:04 +05:30
Ice3man
78c4b9b7d2
Added clustering support for TLS templates (#3209)
* Added clustering support for DNS protocol templates

* Added clustering support for TLS templates

* Fixed randomly populated info block in ssl templates

* Moved to a switch-case + added tests for clustering
2023-01-17 18:20:05 +05:30
Ice3man
dbb4de028e
Added clustering support for DNS protocol templates (#3204) 2023-01-17 13:01:20 +05:30
sandeep
08e89cc541 release workflow update 2023-01-15 23:06:22 +05:30
Jaideep Khandelwal
a1642be911
Enable/Disable a reporting source (#3183)
* Enable/Disable a reporting source

* misc options update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-15 22:21:16 +05:30
xm1k3
84b0581306
Markdown Reporting Index File (#3129)
* implemented index.md creation on markdown export

* fixes on markdown generation

added also hyperlink to markdown created

* code optimization and refactoring

* index header refactor
2023-01-15 22:00:34 +05:30
Dwi Siswanto
94ec553234
Add generate_jwt & json_{minify,prettify} helper functions (#3160)
* feat(dsl): add generate_jwt helper func

* feat(dsl): add json_{minify,prettify} & quote_escape

* update(dsl): change type of data var to map[string]interface{}

* docs(dsl): list valid algos for generate_jwt

* test(dsl): add test case for json_{minify,prettify} & quote_escape

* update(dsl): refactor generate_jwt

* fix(lint): use time.Until instead of t.Sub(time.Now()) (gosimple)

* revert(dsl): remove quote_escape func

* ability to fuzz jwt noNe algorithm

* fix lint error

* jwt dsl minor improvement

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-01-15 21:58:51 +05:30
Ice3man
67c094444e
Added cloud scan progress tracking using stats (#3180)
* Added cloud scan progress tracking using stats

* Changed log messsage

* Fixed linting error

* Fixed bug in progress calculation logic

* Changed requests to input with cloud flag

* Changed progress name + removed redundant fields
2023-01-13 13:41:05 +05:30
Mzack9999
e4402e7449
lowering hmap storage requirement via omitempty (#3111) 2023-01-12 20:01:45 +05:30
Tarun Koyalwar
e899afafdf
skip scanallip if input is ip (#3186)
* skip scanallips for ip input

* uncover,url parsing bug fix

* minor changes:best practices
2023-01-11 22:50:57 +05:30
Shubham Rasal
25fcae1493
add reporting source in nuclei-cloud (#3151)
* add reporting source in nuclei-cloud

- `nuclei -cloud -rc reporting-config.yaml`

* update error message

* add severity options for jira,(used for cloud only)
2023-01-10 22:49:01 +05:30
sandeep
1c046c387b version update 2023-01-05 22:07:02 +05:30
Mzack9999
8beb6b06f9
Extending advanced filtering (#3146)
* adding more metadata to advanced filtering

* adding functional test cases

* converting metadata to lowercase

* misc update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-01-05 21:59:58 +05:30
Tarun Koyalwar
f646e00c3d
fix unsafe raw request matchedUrl (#3155)
* fix unsafe raw request matchedurl

* quote metadata in CLI output
2023-01-05 21:02:36 +05:30
Mzack9999
a96f764959
extending request/response hijacking with native calls (#3091)
* extending request/response hijacking with native calls

* fixing tests
2023-01-05 17:26:18 +05:30
Tarun Koyalwar
4aa2002e72
urlencode key characters only (#3150)
* only encode key characters

* improve test cases
2023-01-05 16:41:59 +05:30
Sandeep Singh
2d7948af55
Fixing host skipping error (#3143)
* removed error resulting into excessive error count

* banner update
2023-01-04 00:43:18 +05:30
Tarun Koyalwar
e66ed30cec
fix missing trailing slash (#3127)
* raw: fix missing trailing slash

* adds rawpath integration test

* rename trailing slash test
2023-01-03 23:45:34 +05:30
shubhamrasal
b03a90df50 fix template path bug, pass the filepath in executeroptions 2023-01-03 12:17:56 +05:30
Sandeep Singh
e66821b49f
Added more error + display skipped host on default run for more visibility. (#3123) 2023-01-02 19:00:10 +05:30
Sandeep Singh
212d0e5cfc
Adding more error to ignore with hosts (#3121) 2023-01-02 17:09:39 +05:30
sandeep
b6ec247701 version update 2023-01-02 13:59:51 +05:30
xm1k3
34120fbecc
#3046 persistent failed item status and #2065 failed items reporting error once (#3047)
* added logs for debug

* fixes

* removed logs

* using cache item

* implemented multiple tests

* fixed some unit tests

* implemented test for skipping

* added multiple tests together

* added mark failed

* fix on tests

* better test implementation + concurrent

* fix: fixes on concurrent tests

* removed parallel and 1 unit test

DOCS: by default the command go test runs in parallel tests for different packages, and default is the number of CPUs available (see go help build)

* fixes on go routine

* increasing parallelism of once.Do

* bumping go to 1.19 for atomic types support

* removing redundant check + fixing test concurrency on create

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-01-02 13:52:06 +05:30
Mzack9999
2c3453500c
Adding empty comparison for scan strategy (#3100)
* Adding empty edge case to scan strategy

* Auto Generate Syntax Docs + JSONSchema [Thu Dec 29 10:10:55 UTC 2022] 🤖

* adding test file

* removing test file

Co-authored-by: GitHub Action <action@github.com>
2022-12-29 20:12:52 +05:30
Mzack9999
6cdff62381
removing unwanted print debug (#2995) 2022-12-28 21:19:06 +05:30
Taufik Mulyana
7093180b6d
Add support to query DNS TLSA record (#3054)
* Add support to query DNS TLSA record

* fix build test

* fix ci-lint

* set expected to 0

* test domain update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 21:12:02 +05:30
Mzack9999
260dd1a2c4
Disable include directive preprocessing by default (#3045)
* adding strict syntax check

* returning error on disabled preprocessing

* adding check on matchers
2022-12-28 19:16:34 +05:30
Tarun Koyalwar
ff17d12ced
adds scan strategy (#3075) 2022-12-28 02:18:00 +05:30
Tarun Koyalwar
af2aaacbc9
fix url panic (#3092) 2022-12-27 22:41:33 +05:30
Shubham Rasal
91f6fdfb36
Merge pull request #3031 from projectdiscovery/parse_from_reader
Add ParseTemplateFromReader function
2022-12-27 20:47:49 +05:30
shubhamrasal
1df878fdd2 add template path in json output 2022-12-27 20:02:48 +05:30
Mzack9999
34976029d3
removing most go routine leaks (#3073)
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-24 19:22:14 +05:30
Tarun Koyalwar
aee0870617
scanallip handle edge cases (#3080)
* bug fix:remove port during dns resolution

* scanallip fix edge cases

* add scanallips testcases

* workflow fix

* removing pull cmd

* Auto Generate Syntax Docs + JSONSchema [Sat Dec 24 13:29:21 UTC 2022] 🤖

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
2022-12-24 19:03:23 +05:30
shubhamrasal
406d0c1873 Rename function from ParseFromReader to ParseTemplateFromReader 2022-12-22 11:04:16 +05:30
Sandeep Singh
96646c8f53
cloud templates targets sync (#2959)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Added initial data_source sync to cloud

* Misc

* Add pagination to scan output and scan list (#2858)

* Add pagination to scan output and scan list

* Use time based parameters instead of page numbers

* Fix linting errors

* Do not check limits at client, check at server

* Remove unused constant

* Misc update

* Removed unnecessary flags

* Misc

* Misc

* Misc endpoint additions

* Added more routes

* Typo fix

* Misc fixes

* Misc

* Misc fixes to cloud target logic + use int for IDs

* Misc

* Misc fixes

* Misc

* Misc fixes

* readme update

* Add JSON output support for list-scan option (#2876)

* Add JSON output support for list-scan option

* Fix typo in cloud JSON output description

* Following changes

- Update status(finished, running) to be lower-case by default
- Convert status to upper-case in DisplayScanList()

* Update status to be lower-case by default

* Remove additional json flag, instead use existing

* Merge conflict

* Accomodate comment changes and restructure code

Co-authored-by: Jaideep K <jaideep@one2n.in>

* Use integer IDs for scan tasks

* Added get-templates-targets endpoint + JSON + validation

* Added target count list

* misc option / description updates

* Added changes as per code review

* duplicate options + typo updates

* Added tablewriter for tabular data writing by default

* Fixed list scan endpoint

* Review changes

* workflow fix

* Added cloud tags etc based filtering (#3070)

* Added omitempty for filtering request

* go mod tidy

* misc format update

Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Jaideep Khandelwal <jdk2588@gmail.com>
Co-authored-by: Siddharth Shashikar <60960197+shashikarsiddharth@users.noreply.github.com>
Co-authored-by: Jaideep K <jaideep@one2n.in>
2022-12-21 22:48:43 +05:30
Mzack9999
093d691c16
Issue 3033 deny list (#3037)
* fixing file deny list + refactoring

* err variable renaming

* removing redundant function

* removing unused code

* adding check on empty operator

* updating tests
2022-12-21 02:29:28 +05:30
xm1k3
bbb561b097
CLI variables are not accessible in SSL Protocol (#3069)
* added vars payload also in ssl

* fix on ssl.go, moved function on payloadValues creation

* added integration test

* rebase + minor changes

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-12-21 02:02:18 +05:30
xm1k3
aaae68ee50
The debug flags do not work correctly with headless templates (#3030)
* added check on BaseURL

* added debug flag instead on info

* added also debug response flag

* code refactoring
2022-12-20 17:30:34 +05:30
Mzack9999
2445662fb1
Fix memory leak on hmap with LRU (#3052)
* fix memory leak on hmap with LRU

* fixing lint issues
2022-12-20 15:37:48 +05:30
Tarun Koyalwar
63150a8ee4
add stop-at-first-match for headless protocol (#3034)
* headless: adds stop-at-first-match

* GH workflow fix

* misc fix

* headless use spm option

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-19 19:52:17 +05:30
shubhamrasal
b51aac290e Add tests folder 2022-12-19 19:15:04 +05:30
shubhamrasal
cff0d88742 Add tests for parse function
- update the ioutils to io.ReadAll
- remove extra line
2022-12-19 18:51:20 +05:30