51 Commits

Author SHA1 Message Date
Sandeep Singh
6ae9eee8d3
README + misc updates (#2961)
* readme update

* version + banner update

* misc option update

* go fmt'ed code

* misc update
2022-12-04 20:51:33 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
M. Ángel Jimeno
62a4e0aa52
Return wrapped errors for DSL compilation problems (#2492)
This allows the DSL help information to be printed when in debug mode.

Fixes #2481
2022-08-29 13:41:32 +05:30
Ice3man
8f313629b8
Memory usage optimizations (#2350)
* Replaced strings.Replaced with fasttemplate reducing allocations

Custom template parsing logic was replaced with fasttemplate package for reducing
allocations in the replacer.Replace hotpath leading to allocation reduction which
accounted for 30% of total nuclei allocations.

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer
BenchmarkReplacer-8               837232              1422 ns/op            2112 B/op         31 allocs/op
BenchmarkReplacerNew-8           3672765               320.3 ns/op            48 B/op          4 allocs/op

* Fixed tests failing

* Use pre-compiled map of DSL expressions

* Reworked expression parsing logic to reduce memory allocations

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions
BenchmarkEvaluate-8        31560             37769 ns/op           31731 B/op        265 allocs/op
BenchmarkEvaluateNew-8       109144              9621 ns/op            6253 B/op        116 allocs/op
2022-08-23 13:16:41 +05:30
Mzack9999
3838d06aeb
Adding prototype for unexpected fields validation in matchers (#2171)
* Adding unexpected fields validation

* using expected fields

* adding missing expected field

* using reflect
2022-06-30 16:50:54 +05:30
Ice3man
50d21c0464
Fixed a issue with random invalid matches in DSL (#2195) 2022-06-21 21:58:43 +05:30
Sajad
7170cc2828
dsl matcher separate ignorable(No parameter) errors from others (#2127) 2022-06-07 18:23:07 +05:30
Sajad
b79817e0a9
change dsl evaluate warning messages to error (#2096)
* change dsl evaluate warning messages to error

* add template-id to dsl match error logs
2022-06-03 13:41:36 +05:30
Ice3man
7d83d3f8c9
Added line number for file results + stats fixes (#1495)
* Added line number for file results + stats fixes

* Misc

* Improved file result line calculation as per review

* Added new match-all attribute for file template matcher line count
2022-02-10 15:59:05 +05:30
Ice3man
6e71313d2b Misc changes to description 2022-01-17 15:35:27 +05:30
Sajad
5200bcd94f
Whois Protocol Support (using rdap) (#1354)
* init rdap

* add an integration test, option to supply RDAP server to execute the request on

* add rdap protocolMappings

* add debug info, add IP, ASN query type support

* rename rdap to whois, Host to Query in template

* rename pending rdap to whois

* remove port from whois varaiables

* set Host variable even if input is not a parsable url
2021-12-16 17:08:02 +05:30
forgedhallpass
2d5784d992 Merge branch 'dev' into dsl_signatures 2021-12-07 18:22:10 +02:00
forgedhallpass
d05c4dc122 refactor: renamed misleading receiver names 2021-12-07 18:01:34 +02:00
forgedhallpass
79aed22d46 feat: Improve DSL function UX #1295
Added support for letting people know if:
* the DSL expression does not return a boolean value
* an invalid custom function signature was provided and then display all available function signatures
* an invalid function was provided and then display the correct signature

Unified the DSL function names to use snake case. The old signatures are also kept for backward compatibility.
2021-12-07 17:34:36 +02:00
Sajad Parra
7a96a7c931 Merge branch 'dev' into dsl-variable-support 2021-12-02 16:50:50 +05:30
Sajad Parra
5b99921d75 add unit and integration tests for dsl variable #555 2021-11-30 20:20:43 +05:30
Ice3man543
eb7c704d36 Merging from dev 2021-11-30 16:23:39 +05:30
Sajad Parra
d97aee01c3 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dsl-variable-support 2021-11-29 13:36:09 +05:30
forgedhallpass
c61ec5f673 feat: Improve DSL function UX #1295 2021-11-26 17:14:25 +02:00
forgedhallpass
7e22d70ded refactor/documentation: typos and grammatical errors 2021-11-25 18:54:16 +02:00
forgedhallpass
3fd1f57b96 refactor: godoc and comment uniformization
Adding space after // and before the godoc/comment
2021-11-25 17:03:56 +02:00
forgedhallpass
c8d009654c fix: Receiver mismatch warnings 2021-11-25 16:57:43 +02:00
Ice3man543
caaa5c6594 Automatically generate docs for enum values 2021-11-25 19:34:01 +05:30
Sajad Parra
7e9272776d add variable support to dsl, remove dynamicValues from request struct 2021-11-25 16:19:42 +05:30
Ice3man543
bdf1c2010b Fixed jsonschema for matchertype missing String() 2021-11-20 19:50:18 +05:30
Ice3man543
66dacccfb4 Fixed matcher type integer issue 2021-11-20 19:47:05 +05:30
LuitelSamikshya
4b8ec29d8d
struct to enums changes for Matchers (#1246)
* struct to enums changes for Matchers
2021-11-19 16:24:09 +05:30
Ice3man
eaa0d37065
feat #1092: Validate binary values + precompile them as well (#1213)
* Added validation for binary values + precompile them

* Changed name of the binary matcher field

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-11-17 02:04:27 +05:30
Ice3man
50a816fce9
Fixed a crash with uninitialized interactsh client (#1251)
* Fixed a crash with uninitialized interactsh client
2021-11-16 20:02:39 +05:30
Ice3man543
90a0502b60 Misc fixes to default part 2021-11-11 17:30:25 +05:30
Alexey Zhuchkov
3a47413cd4 Add case-insensitive flag to matchers and extractors 2021-10-29 19:08:23 +03:00
forgedhallpass
404f8ebb27 Fixes:Binary matcher always returns true if the input cannot be decoded as hex #1085
* changed the Log to Warn instead of Error, so it wouldn't show up in the output without the verbose flag
2021-10-05 16:49:55 +03:00
forgedhallpass
4d34513881 Fix for Binary matcher always returns true if the input cannot be decoded as hex #1085 2021-10-05 16:44:18 +03:00
forgedhallpass
dccd47ed19 [feature] Add coloring to debug information #999 [WIP]
Return the matched content in case of multiple regex/binary matchers (+test)
2021-09-30 20:36:39 +03:00
forgedhallpass
4be6b3cc96 [feature] Add coloring to debug information #999 [WIP]
TODO:
* if there are multiple matchers, make sure the response is only displayed once, with all the matching values colored
* remove code duplication from the request.go files
2021-09-29 19:43:46 +03:00
sandeep
aff4197ea9 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into matchers-enhancement#555 2021-09-21 16:36:58 +05:30
forgedhallpass
9b45c31db8 Merge remote-tracking branch 'origin/dev' into code_smells 2021-09-16 15:49:51 +03:00
Sajad Parra
f1ad6f6bc5 add payload, variable, helper support to matchers 2021-09-15 18:02:22 +05:30
Ice3man543
ad9755dd83 Misc work on jsonschema validations 2021-09-14 17:34:10 +05:30
forgedhallpass
0ce33927c7 Typo fixes. 2021-09-07 17:31:46 +03:00
sandeep
cb5c0cde1d misc docs update 2021-09-01 15:08:46 +05:30
Ice3man543
6a673053c2 Added jsonschema generation for yaml syntax 2021-08-23 23:50:45 +05:30
Ice3man543
d04511494d Added new YAML based doc to structures 2021-07-27 16:03:56 +05:30
Ice3man543
8512b684c5 Lint errors fix 2021-02-26 13:13:11 +05:30
Ice3man543
8a7cabb88d Added hex encoding support in matchers 2021-02-24 11:23:22 +05:30
Ice3man543
60789f4ba2 More refactoring of nuclei packages 2020-12-24 20:47:41 +05:30
Ice3man543
5153647e0f Misc work on extractors + compat 2020-12-24 12:13:18 +05:30
Ice3man543
8a64578890 Work on operators package and generic protocol agnostic matching capabilities 2020-12-24 01:41:32 +05:30
Ice3man543
2317e1ba1b Data modelling, work on executor started 2020-12-23 16:16:16 +05:30
Ice3man543
ed84bb187b Added per protocol responseToDSL function + misc cleanup with operators 2020-12-21 15:51:43 +05:30