41 Commits

Author SHA1 Message Date
Mzack9999
18f14b631c
Adding same host redirect support (#2655)
* simplifying test syntax

* adding same host redirect + refactoring redirect handling

* adding missing file

* adding support for template syntax

* adding integration test

* updating options

* fixing issue on same host redirect
2022-09-29 04:11:28 +05:30
aprp
57b0d78dcf
add unit tests for extract.go (#2635) 2022-09-28 01:24:00 +05:30
M. Ángel Jimeno
62a4e0aa52
Return wrapped errors for DSL compilation problems (#2492)
This allows the DSL help information to be printed when in debug mode.

Fixes #2481
2022-08-29 13:41:32 +05:30
Ice3man
8f313629b8
Memory usage optimizations (#2350)
* Replaced strings.Replaced with fasttemplate reducing allocations

Custom template parsing logic was replaced with fasttemplate package for reducing
allocations in the replacer.Replace hotpath leading to allocation reduction which
accounted for 30% of total nuclei allocations.

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer
BenchmarkReplacer-8               837232              1422 ns/op            2112 B/op         31 allocs/op
BenchmarkReplacerNew-8           3672765               320.3 ns/op            48 B/op          4 allocs/op

* Fixed tests failing

* Use pre-compiled map of DSL expressions

* Reworked expression parsing logic to reduce memory allocations

$ go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions
BenchmarkEvaluate-8        31560             37769 ns/op           31731 B/op        265 allocs/op
BenchmarkEvaluateNew-8       109144              9621 ns/op            6253 B/op        116 allocs/op
2022-08-23 13:16:41 +05:30
Dani Goland
8670c8b20d
Modified "xpath" extractor to support XML XPath in addition to HTML XPath (#2471)
* Modified "xpath" extractor to support XML XPath in addition to HTML XPath

* Updated function docs
2022-08-22 15:27:32 +05:30
Mzack9999
72c5c399ec adding support for dsl extractors 2022-04-20 11:32:13 +02:00
Ice3man543
eb7c704d36 Merging from dev 2021-11-30 16:23:39 +05:30
forgedhallpass
47340f06b0 refactor: uniformly sorted imports 2021-11-25 17:09:20 +02:00
forgedhallpass
3fd1f57b96 refactor: godoc and comment uniformization
Adding space after // and before the godoc/comment
2021-11-25 17:03:56 +02:00
Ice3man543
caaa5c6594 Automatically generate docs for enum values 2021-11-25 19:34:01 +05:30
LuitelSamikshya
2856e7e247
Extractor + Headless Actions structures to enums (#1217)
* enum support for extractor
2021-11-19 01:41:10 +05:30
Ice3man543
90a0502b60 Misc fixes to default part 2021-11-11 17:30:25 +05:30
Alexey Zhuchkov
3a47413cd4 Add case-insensitive flag to matchers and extractors 2021-10-29 19:08:23 +03:00
forgedhallpass
0ce33927c7 Typo fixes. 2021-09-07 17:31:46 +03:00
forgedhallpass
0cabce518f Merge branch 'dev' into code_smells 2021-09-02 12:30:43 +03:00
sandeep
cb5c0cde1d misc docs update 2021-09-01 15:08:46 +05:30
forgedhallpass
40d88d2304 In-lined error checks, reduced scope of error variables, introduced new error variables instead of re-using them 2021-08-31 12:55:52 +03:00
forgedhallpass
9375094a48 Minor typo fixes 2021-08-27 17:06:33 +03:00
Ice3man543
6a673053c2 Added jsonschema generation for yaml syntax 2021-08-23 23:50:45 +05:30
Ice3man543
2bf3652593 Merge branch 'yamldoc' of https://github.com/projectdiscovery/nuclei into yamldoc 2021-08-20 15:15:09 +05:30
Ice3man543
4cc770f102 Merged dev 2021-08-19 16:51:02 +05:30
mzack
a038889c21 Merge branch 'dev' into yamldoc 2021-08-10 08:04:50 +02:00
mzack
fbe9d3d0a4 Merge branch 'dev' into xpath-extractors 2021-08-09 13:04:57 +02:00
Ice3man543
8449a5e73f Misc changes to docs 2021-08-05 00:54:34 +05:30
Ice3man543
01dcb01867 Added more docs examples + Misc 2021-08-04 14:20:48 +05:30
Ice3man543
c743d5d299 Merged from dev 2021-08-03 20:22:16 +05:30
Ice3man543
688d9e466b Fixed lint error 2021-08-02 21:46:29 +05:30
Ice3man543
11e5520166 Added xpath based extractor support 2021-08-02 21:43:50 +05:30
seb
aae26f37a6 add JSONScalarToString and more tests for JSON extractor 2021-08-02 13:19:39 +02:00
seb
d2e7cca17b fix golint 2021-08-01 14:42:04 +02:00
seb
cc12babf58 Update extractors.go 2021-08-01 12:38:35 +02:00
seb
c76504f0d6 use import () in extractors.go 2021-08-01 12:38:13 +02:00
seb
3df9be0fa7 Update extractors.go 2021-08-01 00:02:43 +02:00
seb
166344d793 add and fix tests for json 2021-07-31 23:48:14 +02:00
seb
3529cfa1d6 json extractor support 2021-07-31 22:49:23 +02:00
Ice3man543
d04511494d Added new YAML based doc to structures 2021-07-27 16:03:56 +05:30
Ice3man543
2ea6e7699a Added tests for DNS package 2021-02-03 02:09:45 +05:30
Ice3man543
50eafb29d1 Bugfix: kval extractor not working 2021-01-12 11:21:32 +05:30
Ice3man543
60789f4ba2 More refactoring of nuclei packages 2020-12-24 20:47:41 +05:30
Ice3man543
5153647e0f Misc work on extractors + compat 2020-12-24 12:13:18 +05:30
Ice3man543
d631074e35 Separating matchers, extractors and requests as protocols and operators 2020-12-21 14:31:32 +05:30