60 Commits

Author SHA1 Message Date
Tarun Koyalwar
eec907a370 resolve merge conflicts 2023-09-13 20:28:48 +05:30
DoI
f520d7e843
XPath matcher support (#4087)
* Added xpath response matching support

* Add validation for user-supplied XPath

* xpath matcher comment fix

* Added XPath matched documentation

* minor changes: remove warnings

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-08-25 22:41:51 +05:30
Tarun Koyalwar
8125b6805c resolve merge conflicts with dev 2023-08-04 20:21:22 +05:30
Tarun Koyalwar
6bdef68734
ignore version parsing error (#3984)
* ignore version parsing error

* hide no parameter error

* integration test+ DEBUG.md

* typo fix in DEBUG.md

* go mod tidy

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-07-28 21:04:02 +05:30
Tarun Koyalwar
bdf77005d6 resolve merge conflicts 2023-06-27 20:21:14 +05:30
Mzack9999
a4ef3ea3a9 fixing and condition 2023-06-13 17:24:23 +02:00
Tarun Koyalwar
e1d3f474a4
support for dynamic variables in template context (multi protocol execution) (#3672)
* multi proto request genesis

* adds template context dynamic vars

* feat: proto level resp variables

* remove proto prefix hacky logic

* implement template ctx args

* remove old var name logic

* improve AddTemplateVars func

* add multi proto comments+docs

* vardump with sorted keys

* fix race condition in ctx args

* default initialize ctx args

* use generic map

* index variables with multiple values

* fix nil cookies

* use synclock map

* fix build failure

* fix lint error

* resolve merge conflicts

* multi proto: add unit+ integration tests

* fix unit tests

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

---------

Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
2023-06-09 19:52:56 +05:30
王一之
27fefe59d3 fix json deserialization issues 2023-02-27 14:29:49 +08:00
王一之
85090b7531 fix some json deserialization issues 2023-02-17 14:21:25 +08:00
Sandeep Singh
6ae9eee8d3
README + misc updates (#2961)
* readme update

* version + banner update

* misc option update

* go fmt'ed code

* misc update
2022-12-04 20:51:33 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
M. Ángel Jimeno
62a4e0aa52
Return wrapped errors for DSL compilation problems (#2492)
This allows the DSL help information to be printed when in debug mode.

Fixes #2481
2022-08-29 13:41:32 +05:30
Ice3man
8f313629b8
Memory usage optimizations (#2350)
* Replaced strings.Replaced with fasttemplate reducing allocations

Custom template parsing logic was replaced with fasttemplate package for reducing
allocations in the replacer.Replace hotpath leading to allocation reduction which
accounted for 30% of total nuclei allocations.

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer
BenchmarkReplacer-8               837232              1422 ns/op            2112 B/op         31 allocs/op
BenchmarkReplacerNew-8           3672765               320.3 ns/op            48 B/op          4 allocs/op

* Fixed tests failing

* Use pre-compiled map of DSL expressions

* Reworked expression parsing logic to reduce memory allocations

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions
BenchmarkEvaluate-8        31560             37769 ns/op           31731 B/op        265 allocs/op
BenchmarkEvaluateNew-8       109144              9621 ns/op            6253 B/op        116 allocs/op
2022-08-23 13:16:41 +05:30
Mzack9999
3838d06aeb
Adding prototype for unexpected fields validation in matchers (#2171)
* Adding unexpected fields validation

* using expected fields

* adding missing expected field

* using reflect
2022-06-30 16:50:54 +05:30
Ice3man
50d21c0464
Fixed a issue with random invalid matches in DSL (#2195) 2022-06-21 21:58:43 +05:30
Sajad
7170cc2828
dsl matcher separate ignorable(No parameter) errors from others (#2127) 2022-06-07 18:23:07 +05:30
Sajad
b79817e0a9
change dsl evaluate warning messages to error (#2096)
* change dsl evaluate warning messages to error

* add template-id to dsl match error logs
2022-06-03 13:41:36 +05:30
Ice3man
7d83d3f8c9
Added line number for file results + stats fixes (#1495)
* Added line number for file results + stats fixes

* Misc

* Improved file result line calculation as per review

* Added new match-all attribute for file template matcher line count
2022-02-10 15:59:05 +05:30
Ice3man
6e71313d2b Misc changes to description 2022-01-17 15:35:27 +05:30
Sajad
5200bcd94f
Whois Protocol Support (using rdap) (#1354)
* init rdap

* add an integration test, option to supply RDAP server to execute the request on

* add rdap protocolMappings

* add debug info, add IP, ASN query type support

* rename rdap to whois, Host to Query in template

* rename pending rdap to whois

* remove port from whois varaiables

* set Host variable even if input is not a parsable url
2021-12-16 17:08:02 +05:30
forgedhallpass
2d5784d992 Merge branch 'dev' into dsl_signatures 2021-12-07 18:22:10 +02:00
forgedhallpass
d05c4dc122 refactor: renamed misleading receiver names 2021-12-07 18:01:34 +02:00
forgedhallpass
79aed22d46 feat: Improve DSL function UX #1295
Added support for letting people know if:
* the DSL expression does not return a boolean value
* an invalid custom function signature was provided and then display all available function signatures
* an invalid function was provided and then display the correct signature

Unified the DSL function names to use snake case. The old signatures are also kept for backward compatibility.
2021-12-07 17:34:36 +02:00
Sajad Parra
7a96a7c931 Merge branch 'dev' into dsl-variable-support 2021-12-02 16:50:50 +05:30
Sajad Parra
5b99921d75 add unit and integration tests for dsl variable #555 2021-11-30 20:20:43 +05:30
Ice3man543
eb7c704d36 Merging from dev 2021-11-30 16:23:39 +05:30
Sajad Parra
d97aee01c3 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into dsl-variable-support 2021-11-29 13:36:09 +05:30
forgedhallpass
c61ec5f673 feat: Improve DSL function UX #1295 2021-11-26 17:14:25 +02:00
forgedhallpass
7e22d70ded refactor/documentation: typos and grammatical errors 2021-11-25 18:54:16 +02:00
forgedhallpass
3fd1f57b96 refactor: godoc and comment uniformization
Adding space after // and before the godoc/comment
2021-11-25 17:03:56 +02:00
forgedhallpass
c8d009654c fix: Receiver mismatch warnings 2021-11-25 16:57:43 +02:00
Ice3man543
caaa5c6594 Automatically generate docs for enum values 2021-11-25 19:34:01 +05:30
Sajad Parra
7e9272776d add variable support to dsl, remove dynamicValues from request struct 2021-11-25 16:19:42 +05:30
Ice3man543
bdf1c2010b Fixed jsonschema for matchertype missing String() 2021-11-20 19:50:18 +05:30
Ice3man543
66dacccfb4 Fixed matcher type integer issue 2021-11-20 19:47:05 +05:30
LuitelSamikshya
4b8ec29d8d
struct to enums changes for Matchers (#1246)
* struct to enums changes for Matchers
2021-11-19 16:24:09 +05:30
Ice3man
eaa0d37065
feat #1092: Validate binary values + precompile them as well (#1213)
* Added validation for binary values + precompile them

* Changed name of the binary matcher field

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2021-11-17 02:04:27 +05:30
Ice3man
50a816fce9
Fixed a crash with uninitialized interactsh client (#1251)
* Fixed a crash with uninitialized interactsh client
2021-11-16 20:02:39 +05:30
Ice3man543
90a0502b60 Misc fixes to default part 2021-11-11 17:30:25 +05:30
Alexey Zhuchkov
3a47413cd4 Add case-insensitive flag to matchers and extractors 2021-10-29 19:08:23 +03:00
forgedhallpass
404f8ebb27 Fixes:Binary matcher always returns true if the input cannot be decoded as hex #1085
* changed the Log to Warn instead of Error, so it wouldn't show up in the output without the verbose flag
2021-10-05 16:49:55 +03:00
forgedhallpass
4d34513881 Fix for Binary matcher always returns true if the input cannot be decoded as hex #1085 2021-10-05 16:44:18 +03:00
forgedhallpass
dccd47ed19 [feature] Add coloring to debug information #999 [WIP]
Return the matched content in case of multiple regex/binary matchers (+test)
2021-09-30 20:36:39 +03:00
forgedhallpass
4be6b3cc96 [feature] Add coloring to debug information #999 [WIP]
TODO:
* if there are multiple matchers, make sure the response is only displayed once, with all the matching values colored
* remove code duplication from the request.go files
2021-09-29 19:43:46 +03:00
sandeep
aff4197ea9 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into matchers-enhancement#555 2021-09-21 16:36:58 +05:30
forgedhallpass
9b45c31db8 Merge remote-tracking branch 'origin/dev' into code_smells 2021-09-16 15:49:51 +03:00
Sajad Parra
f1ad6f6bc5 add payload, variable, helper support to matchers 2021-09-15 18:02:22 +05:30
Ice3man543
ad9755dd83 Misc work on jsonschema validations 2021-09-14 17:34:10 +05:30
forgedhallpass
0ce33927c7 Typo fixes. 2021-09-07 17:31:46 +03:00
sandeep
cb5c0cde1d misc docs update 2021-09-01 15:08:46 +05:30