217 Commits

Author SHA1 Message Date
Mzack9999
6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
Ramana Reddy
6ffdfcf19c
fix rate-limit on query fuzzing (#3458) (#3532) 2023-04-12 23:55:21 +05:30
Shubham Rasal
45cc676f96
Evaluate payload variables (#3503)
* Evaluate payload variables

* Add variables evaluation

* Extend variables test

- to check evaluation of global variables in variables
- to check evaluation of golbal variables in payload

* Add default and cli variables to websocket, whois and dns proto

- use url.Parse with urlutil.Parse
2023-04-12 01:50:58 +05:30
Mzack9999
4c0d988a67 reworking interact mutex mechanism 2023-03-17 14:41:16 +01:00
Tarun Koyalwar
c3771e874d
fix data race in internal resultevent (#3432) 2023-03-16 23:20:38 +05:30
Ramana Reddy
c9634fae72
Issue 3350 matcher condition or not work (#3397)
* fix or condition match even interactsh includes as matcher-part (#3350)

* add integration test

* add new template to integration test

* matcher-condtion: test case for both conditions

* fix lint errors

* upgrade dependencies

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-03-15 20:45:44 +05:30
Tarun Koyalwar
d9e953acfa
fix file input in custom vars for self contained http template (#3385)
* fix file input in variables(-V)

* fix lint error

* fix nuclei-ignore file failures
2023-03-04 04:57:27 +05:30
Tarun Koyalwar
21b03a2e8a
bug fix in url path and adds integration tests (#3331)
* fix unsafe edgecases+ adds integration test

* bug fixes and more url testcases

* upgrade cfssl

* fix template id in integration test
2023-02-20 22:26:04 +05:30
Mzack9999
d608ffaeb2
clear after stop (#3312)
* clear after stop

* fixing data races

* adding atomic cache

* fixing lint errors

* fixing imports
2023-02-13 16:46:41 +05:30
Tarun Koyalwar
e622b989fe
fix url re-encoding issues (#3294)
* fix double url encoding in urls

* remove extra slash

* url encode matchedURL
2023-02-10 18:28:28 +05:30
Tarun Koyalwar
0b2a3e296a
fix url encoding issues and inconsistencies (#3211)
* fix url encoding issues

* complete requested changes and improvements

* fix missing issue-tracker-config.yaml

* fuzz: deepcopy and use urlutil.Params
2023-01-24 22:04:52 +05:30
Tarun Koyalwar
edcab07fec
fix aws signer missing template variables (#3206)
* aws sign: fix missing variables

* signer: add aws defaults

* aws signer default values
2023-01-24 20:50:20 +05:30
Tarun Koyalwar
f646e00c3d
fix unsafe raw request matchedUrl (#3155)
* fix unsafe raw request matchedurl

* quote metadata in CLI output
2023-01-05 21:02:36 +05:30
Mzack9999
96c1dd3720
Adding custom ip to protocol generated variables (#3011)
* lint errors

* Extending context args support

* Ip => ip
2022-12-10 00:17:03 +05:30
Tarun Koyalwar
3b31799847
Issue 2840 aws signature (#2924)
* docker go version update

* docker fix

* version update

* update chinese readme and typo fixes. (#2862)

* fix aws request signer

* fix reader by upgrading retryablehttp-go

* go mod tidy

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Xc1Ym <xuedongyuming2233@gmail.com>
2022-12-03 07:10:57 +05:30
Mzack9999
1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
Ice3man
b9472cf7e1
Added fuzzing support for query params + var dump feature (#2679)
* Added fuzzing support for query params + var dump feature

* Added query-fuzz integration test

* Fixed payloads + added keys-regex fuzz parameter

* Fixed interactsh not working + misc

* Fixed evaluation + added global variables/dsl support to payloads

* Misc fixes related to variables evaluations

* Added http variables support to fuzz

* misc

* Misc

* Added testing playground + misc renaming

* Added support for path and raw request to fuzzing

* Fixed fuzz integration test

* Fixed variable unresolved issue

* Add multiple parameter support with same name

* Added parameter value as 'value' dsl variable for parts

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-11-01 20:28:50 +05:30
Mzack9999
cc0c20053a
Improving unsafe uri path (#2722)
* Improving unsafe uri path

* fixing raw path output
2022-10-27 20:09:38 +02:00
Mzack9999
9493dfdb20
Adding automatic request condition detection (#2707)
* Adding automatic request condition detection

* adding missing checks on part

* test update as per latest change

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-10-15 15:19:04 +05:30
mzack
70cecf83fb Adding custom cancel function 2022-10-10 08:10:07 +02:00
Ice3man
9944f5e94e
Added response truncation support with flags (#2688)
* Added response truncation support with flags

* Fixed failing tests for no size
2022-10-07 20:10:00 +05:30
Mzack9999
781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
Mzack9999
99c14f4c9c
implementation of rate limiter with bucket refill and unrestricted token burst (#2536)
* implementation of rate limiter with interval burst

* fixing import path

* fixing syntax

* adding tests

* fixing lint errors

* adding support for context

* moving rate limiter earlier to avoid hitting timeout
2022-09-19 17:09:28 +05:30
Mzack9999
30054d1fb6
Adding advanced template filtering (#2374)
* Adding advanced template filtering

* fixing bug in slice

* refactoring tests

* adding test cases

* increasing error verbosity

* fixing quoted fields with spaces

* adding more test cases

* fixing merge error

* fixing lint errors

* switching to []string

* updating tag filter tests

* updating functional tests

* fixing functional test cases

* updating syntax
2022-08-25 16:52:08 +05:30
51pwn
606c361b2a
Add substr and aes_cbc DSL functions (#2361)
* 1、add DSL substr for #2304 By @hktalent
substr('xxtestxxx',2)。   testxxx
substr('xxtestxxx',2,-2)  testx
substr('xxtestxxx',2,6)   test

2、add DSL aes_cbc for #2243 By @hktalent
aes_cbc("key111key111key111key111", "dataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxxdataxxxxxx")

3、fixed An error occurs when running nuclei with multiple instances #2301 By @hktalent

* refactoring helpers

* removing unwanted mutex

* commenting out test

* removing aes_cbc test due to random iv

Co-authored-by: 51pwn <51pwn@51pwn.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-25 15:50:08 +05:30
Sami
4da4ca5a16
missing ip in json (#2310)
* missing ip in json

* using GetDNSData in place of GetDialedIP

* updated go mod

* bumping rawhttp test version

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-07-26 17:08:53 +05:30
Ice3man
2873e6ebc8
Added timeout context cancellation to http requests (#2319) 2022-07-21 21:29:34 +05:30
Mike Rheinheimer
9efba05e0c
expose hosterrorscache.Cache as an interface (#2291)
* expose hosterrorscache as an interface, change signature to capture the error reason

* use the hosterrorscache.CacheInterface as struct field so users of Nuclei embedded can provide their own cache implementation

Co-authored-by: Mike Rheinheimer <mrheinheimer@atlassian.com>
2022-07-19 02:05:53 +05:30
Ice3man
07d5beb73a
Fixed race condition with raw http options (#2306) 2022-07-19 01:08:30 +05:30
Ice3man
5b3c2861c2
Added interact-url placeholder support to variables in http requests (#2237)
* Added interact-url placeholder support to variables in http requests

* Fixed variable errors

* Fixed issue with interactsh in req
2022-07-11 22:18:13 +05:30
Ice3man
f3de611b49
Added enhancements for http variables support (#2223) 2022-06-28 20:20:18 +05:30
Sajad
644c951449
use formedURL instead reqURL in http request dump message (#2174) 2022-06-16 17:24:21 +05:30
Sajad
e7591ec8b3
use request numbering as per template definition in req-condition (#2135)
* use original request number instead of current iteration in request-condition

* add previousEvent tracking back for request condition

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-11 14:29:05 +05:30
Ice3man
be5f1a7623
Added redirected matched URL + stop-at-first-match for redirect chains (#2050)
* Added redirected matched URL + stop-at-first-match for redirect chains

* Pleasing go-linter
2022-05-30 15:19:09 +05:30
Ice3man
f53f360c74
Fixed websocket status-101 response reading for http (#2081) 2022-05-27 22:36:42 +05:30
Mzack9999
16a05d0aa2
Adding CLI SNI support to unsafe http (#2077)
* Adding CLI SNI support to unsafe http

* adding http unsafe sni test
2022-05-27 21:53:07 +05:30
Ice3man
80f3cec293
Disable printing variables in http protocol output (#2037) 2022-05-23 20:50:51 +05:30
Sajad Parra
14f11d24c2 replace rawhttp dev with fix branch, don't drain resp body if pipeline is on 2022-05-04 19:52:35 +05:30
Sandeep Singh
d3b4a86d2a
Merge pull request #1785 from projectdiscovery/variables-introduction
Added variables for nuclei + refactor variable handling
2022-04-21 17:59:01 +05:30
Mzack9999
777b75d305 fixing internal bug for unsafe oob 2022-04-20 17:11:14 +02:00
LuitelSamikshya
3c4250cc81 truncated http response check 2022-04-12 13:15:01 -05:00
Ice3man
aea79638c6 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into variables-introduction 2022-04-06 19:04:27 +05:30
Sami
301307bb77
Issue 1705 save responses on disk (#1727)
* save response on disk

* lint error check

* store raw request/response

* lint error fix

* file path

* mock test fix

* readme update

* .txt extension

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-04-02 00:59:02 +05:30
Ice3man
a0ece302d1 Moved variables to template level + misc 2022-03-31 00:54:35 +05:30
Ice3man
d09e71accf Added variables to http and network 2022-03-30 20:35:46 +05:30
Ice3man
eaa9db19c0 Misc changes to meta and dynamic http values 2022-03-29 20:36:26 +05:30
Sandeep Singh
80e969f917
Merge pull request #1688 from projectdiscovery/issue-1214-truncated-resp
Adding truncation info on debug
2022-03-09 00:36:50 +05:30
mzack
b55dac6ae2 Adding truncation info on debug 2022-03-07 13:32:17 +01:00
mzack
6cb522c9b2 Merge branch 'dev' into issue-1594-regexp 2022-02-28 11:42:46 +01:00