45 Commits

Author SHA1 Message Date
Tarun Koyalwar
ab0f0ea92b
Printing bug fixes (#4238)
* fix printing issue of unsigned workflows

* add NUCLEI_LOG_ALL env variable

* enable tlsx -dns and update print msg

* misc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-10-16 14:34:52 +05:30
Tarun Koyalwar
c35162c8ef
nuclei v3 bug fixes (#4176)
* store and generate signer keys

* fix trailing newline in code_response

* fix formatting and update error string

* fix integration test

* fix rsaSigned code integration test

* bug fixes , docs and more

* bump go -> 1.21

* use 'response' as default part in code templates

* disable sourcemaps for all js runtimes

* disable eval function

* rewrite file validation in sandbox mode

* sandbox file read improvements + minor refactor

* refactor sign and verify logic

* fix panic and missing id in code protocol

* disable re-signing code protocol templates

* fix code resigning in tests

* allow -lfa in test for signing templates

* start index from 1 in flow and multiproto

* remove testfiles

* add python in integration test

* update code protocol docs

* add python engine in template

* rework template signer

* fix integration test and more

* reworked template signer

* fix lint error

* display signature stats

* update docs

* add user fragment to signature

* use md5 to generate fragment

* update docs with code re-sign

* misc updates

* public crt update

* remove workflow info statement

* fix printing issues

* refactor preprocessor logic

* remove debug statement

* fix failing example test

* go mod tidy

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-10-13 13:17:27 +05:30
Tarun Koyalwar
bdf77005d6 resolve merge conflicts 2023-06-27 20:21:14 +05:30
Mzack9999
fa199ed3b3
Improving clientpool with client certificates (#3851)
* Improving clientpool with client certificates

* adding test case

* Revert "Merge branch 'dev' into issue-3800-client-cert"

This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.

* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""

This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.

* go fmt

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 17:45:12 +05:30
Mzack9999
a7fb15d0bd
Adding support for code templates (#2930)
* Adding support for code templates

* adding support for python, powershell and echo (test)

* removing debug code

* introducing command + trivial trust store mechanism

* updating tests

* adding basic tests

* removing deprecated oracle

* mod tidy

* adding signature proto with debug prints

* removing debug code

* fixing test

* fixing param order

* improving test conditional build

* disable file+offlinehttp+code with cloud

* adding env vars

* removing debug code

* reorganizing test folders

* adding code template test prototype with dummy priv/pub keys

* bump go to 1.20

* fixing go version

* fixing lint errors

* adding fatal on pub-key test failure

* switching to ecdsa asn1

* removing unused signature

* fixing signature

* adding more tests

* extending core with engine args + powershell win test

* adding unsigned code test

* skip template signing in particular test case

* improving test coverage

* refactoring key names + adding already signed algo

* removing debug code

* fixing syntax

* fixing lint issues

* removing test template

* fixing dns tests path

* output fmt

* adding interact

* fixing lint issues

* adding -sign cli helper

* fixing nil pointer + parse inline keys

* making rsa default

* adding code prot. ref

* moving file to correct loc

* moving test

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

* add retry to py-interactsh integration test

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
2023-06-09 20:54:24 +05:30
Shubham Rasal
6ebb8e98f4
Fix wrong template loading in dev branch (#3629)
* Templates wrong loading

* Add tests to cover following scenarios

- check optional fields only if template loaded
- it should return warning only if template is loaded
2023-05-02 15:12:55 +05:30
Dogan Can Bakir
4e0ccb38be
Check severity att while validating (#3540)
* Make severity attribute required

* Update test err msg

* minor

* Do not strict check serverity

* Fix failing test

* Don't print warning in workflow loader

- workflow loader that contains tags load all the template and parse it
- i.e it iw printing warning recursively, ignore as the templates
  already getting valiated

* Fix error typo

* Resolve comments

- split the function into two diff

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
2023-04-27 15:27:30 +05:30
Mzack9999
c182434130
json templates support (load with flags, run & validate ) (#3424)
* extending template identification logic

* removing test code

* local debug

* json template loading support using flags

* blacklist meta json files

* minor changes

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-16 13:33:59 +05:30
Mzack9999
62af038617
Add template sign/verify functionality (#3029)
* add template sign/verify functionality

* fixing syntax
2023-02-26 03:54:46 +05:30
Mzack9999
a19385376c
Extending YAML to support include preprocessing (#1767)
* Add support for include directive

* adding yamlc support

* mod tidy

* removing yamlc

* moving code around
2022-12-14 01:05:14 +05:30
Mzack9999
30054d1fb6
Adding advanced template filtering (#2374)
* Adding advanced template filtering

* fixing bug in slice

* refactoring tests

* adding test cases

* increasing error verbosity

* fixing quoted fields with spaces

* adding more test cases

* fixing merge error

* fixing lint errors

* switching to []string

* updating tag filter tests

* updating functional tests

* fixing functional test cases

* updating syntax
2022-08-25 16:52:08 +05:30
Ice3man
67d5769cd9
Added initial catalog interface implementation (#2318)
* Added initial catalog interface implementation

* Added OpenFile to Catalog + disk catalog implementation

* Fixed merge issues

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
Sajad
005b92217f
return on parse template error unconditionally (#2327) 2022-07-24 14:56:06 +05:30
invist
db727db006
Optionally disable templates syntax strict check (#2266)
* nuclei::templates|define strict option (default)

* renaming flag and internal variable

* misc flag update

Co-authored-by: c-f <you@example.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-07-13 17:00:11 +05:30
Sajad Parra
c6de2ca406 rename few vars, extract duplicate functions to utils 2022-01-24 16:48:12 +05:30
Sajad Parra
d1303797c0 * Add support to run remote template
* Add remote-template-domain config only flag to specify allowed domain list to load remote templates from
2022-01-12 18:33:17 +05:30
Mzack9999
39519c01a6
Adding support for template id based execution (#1448) 2022-01-07 17:30:20 +05:30
Mzack9999
3c88afac0c
Fixing payloads path during validation (#1320)
* Fixing payloads path during validation

* Added GH Action for public template parsing / validation

* tracking payload errors as syntax warnings

* improving path parsing + introducing hard failure for runtime errors on validation

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-12-05 19:41:14 +05:30
Ice3man543
47949c0b52 Use separate type enum for protocol types 2021-11-03 17:18:35 +05:30
Ice3man543
6541b04f4c Added new type and exclude-type flag 2021-10-25 23:24:42 +05:30
Ice3man543
ef3397b84d Mark error for template syntax warning 2021-10-25 17:42:01 +05:30
Alexey Zhuchkov
ff7a5997a2 Return multiple errors in template validation 2021-10-20 23:24:11 +03:00
Alexey Zhuchkov
f1cd0a5d28 Update template id regexp 2021-10-20 23:07:24 +03:00
Alexey Zhuchkov
9e0144b6ae Add template-id validation 2021-10-20 01:31:38 +03:00
forgedhallpass
0cabce518f Merge branch 'dev' into code_smells 2021-09-02 12:30:43 +03:00
sandeep
605393d1a8 misc CLI UI update 2021-09-01 02:01:55 +05:30
Ice3man543
19770d186f Changed code as per review comments 2021-08-31 21:39:20 +05:30
Ice3man543
44700e6c60 Show errors when validate flag is passed for syntax 2021-08-31 19:53:53 +05:30
Ice3man543
96adaf5257 Misc 2021-08-31 19:28:20 +05:30
Ice3man543
c3e32f5e15 Added new stats counters + validation fixes 2021-08-31 19:27:26 +05:30
forgedhallpass
f36ed8df64 Removed unused parameter 2021-08-31 13:21:15 +03:00
forgedhallpass
40d88d2304 In-lined error checks, reduced scope of error variables, introduced new error variables instead of re-using them 2021-08-31 12:55:52 +03:00
Ice3man543
78554252ed Misc changes to workflow parts 2021-08-28 00:27:37 +05:30
Ice3man543
f5ea35d45c Don't match for metadata on workflow 2021-08-28 00:19:05 +05:30
Ice3man543
08fdc3bdcc Added cache package + misc changes to loaders 2021-08-28 00:15:28 +05:30
forgedhallpass
36b1c08edc Optimize template validation 2021-08-27 17:06:06 +03:00
forgedhallpass
6bfd6bd28f Adjusting the template/workflow loading/filtering logic. 2021-08-19 15:17:25 +03:00
Ice3man543
66af0297c0 Exported ParseTemplate + use it everywhere for parsing 2021-08-19 02:10:36 +05:30
Ice3man543
01b902f119 Cleanup some loader parts + moved workflow loading separate 2021-08-19 01:58:54 +05:30
Ice3man543
297ce0fc83 Changed error message + return template and don't skip 2021-08-18 21:58:20 +05:30
Ice3man543
46347c599d Added error logging on unknown fields in verbose mode 2021-08-18 21:50:42 +05:30
forgedhallpass
138745a6b1 RES-84 # Improve Nuclei CLI interface (WIP)
* Pleasing the linter
2021-08-03 15:39:15 +03:00
forgedhallpass
2f162e859e RES-84 # Improve Nuclei CLI interface (WIP)
* removed the generic isEmpty implementation
2021-08-03 14:51:34 +03:00
forgedhallpass
5d89bb8056 RES-84 # Improve Nuclei CLI interface
* fixed issues reported by the linter
2021-07-19 21:04:08 +03:00
forgedhallpass
5a495e1e99 RES-84 # Improve Nuclei CLI interface (WIP)
* Integration of the previous logic to 2.4.0
* Unit and ITs passing
* refactored the template matching logic
2021-07-15 13:41:41 +03:00