63 Commits

Author SHA1 Message Date
Tarun Koyalwar
f7fe99f806
add flow support in template (i.e javascript scripting) (#4015)
* add flow logic

* progress

* working POC

* fix string slice normalization issue in variables

* update

* fix nil panic

* remove poll()

* load file with sandbox and more

* fix failing integration tests

* JS: log: print in vardump format

* fix missing id in protocols

* fix proto prefix in template context

* flow: add unit tests

* conditional flow support using flow

* fix proto callbacks + more unit tests

* adds integration test

* conditional flow: check if req has any matchers

* fix lint error

* deprecate iterate-all+ missing multi-proto implementation

* fix ip input in raw request

* JS: feat dedupe object+ more builtin funcs

* feat: hide protocol result using hide

* feat: async execution

* complete async execution support

* fix condition-flow without any matchers

* refactor: template executer package (tmplexec)

* flow executor working

* fix data race in templateCtx

* templateCtx redesign

* fix failing unit test

* add multiprotocol support to deprecated syntax

* fix race condition in utils & tlsx

* add documentation in flow package

* remove regions.txt file

* fix minor issue with self contained templates

* fix typos of copilot

* dep + misc update

* fix reqID: use req.Type instead of template.Type

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-08-31 18:03:01 +05:30
Tarun Koyalwar
bdf77005d6 resolve merge conflicts 2023-06-27 20:21:14 +05:30
Mzack9999
a4ef3ea3a9 fixing and condition 2023-06-13 17:24:23 +02:00
Tarun Koyalwar
e1d3f474a4
support for dynamic variables in template context (multi protocol execution) (#3672)
* multi proto request genesis

* adds template context dynamic vars

* feat: proto level resp variables

* remove proto prefix hacky logic

* implement template ctx args

* remove old var name logic

* improve AddTemplateVars func

* add multi proto comments+docs

* vardump with sorted keys

* fix race condition in ctx args

* default initialize ctx args

* use generic map

* index variables with multiple values

* fix nil cookies

* use synclock map

* fix build failure

* fix lint error

* resolve merge conflicts

* multi proto: add unit+ integration tests

* fix unit tests

* Issue 3339 headless fuzz (#3790)

* Basic headless fuzzing

* Remove debug statements

* Add integration tests

* Update template

* Fix recognize payload value in matcher

* Update tempalte

* use req.SetURL()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>

* Auto Generate Syntax Docs + JSONSchema [Fri Jun  9 00:23:32 UTC 2023] 🤖

* Add headless header and status matchers (#3794)

* add headless header and status matchers

* rename headers as header

* add integration test for header+status

* fix typo

---------

Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
2023-06-09 19:52:56 +05:30
Keith Chason
4d6080f3bc
"Executer" to "Executor" (#3760)
* Fix spelling of "executer" to "executor"

* minor change: use defer file.Close()

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-06-01 02:28:10 +05:30
Mzack9999
093d691c16
Issue 3033 deny list (#3037)
* fixing file deny list + refactoring

* err variable renaming

* removing redundant function

* removing unused code

* adding check on empty operator

* updating tests
2022-12-21 02:29:28 +05:30
Mzack9999
1fbbca66f9
Adding support to scan all v4/v6 IPs (#2709)
* Adding support to scan all v4/v6 IPs

* adding tests

* metainput prototype

* using new signature

* fixing nil pointer

* adding request context with metadata

* removing log instruction

* fixing merge conflicts

* adding clone helpers

* attempting to fix ipv6 square parenthesis wrap

* fixing dialed ip info

* fixing syntax

* fixing output ip selection

* adding integration tests

* disabling test due to gh ipv6 issue

* using ipv4 only due to GH limited networking

* extending metainput marshaling

* fixing hmap key

* adding test for httpx integration

* fixing lint error

* reworking marshaling/id-calculation

* adding ip version validation

* improving handling non url targets

* fixing condition check
2022-11-09 18:48:56 +05:30
vrenzolaverace
2aaf2a2158
Use utils helpers libraries (#2809) (#2810)
* Use utils helpers libraries (#2809)

* Use utils helpers libraries (#2809)
2022-11-07 01:54:23 +05:30
mzack
fe798c07eb fixing function syntax 2022-10-25 21:01:42 +02:00
Mzack9999
781e4e6105
Shared Execution Context Prototype (#2576)
* renaming var

* Introducing shared execution context prototype

* fixing field name

* adding shared values propagation

* adding shared context lock

* add slice values normalization

* adding integration tests

* adding metadata support for dns

* adding multi-protocol context sharing test

* removing debug test files

* moving contextargs around

* adding comments

* refactoring code

- getter/setter for complex types
- using pointers to avoid heap allocations
2022-10-03 15:42:20 +05:30
Ice3man
2b631e8e95
Add optional line number for file templates by default (#1966)
* Add optional line number for file templates by default

* updating docs

* misc flag update

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-06-02 17:06:50 +05:30
Mzack9999
dd3b0a3cfc
Increasing buffer size on AND condition (#2030)
* Increasing buffer size on AND condition

* adding integration tests

* fixing tests data path
2022-05-30 13:45:28 +05:30
mzack
5cd25bd069 more checks + test cases fix 2022-03-22 14:18:01 +01:00
mzack
e7d260ea4d Fixing stats counter 2022-03-22 10:52:57 +01:00
mzack
cf9c844c62 adding line number for extracted results 2022-03-04 09:28:58 +01:00
mzack
4bcc84df9d using int argument 2022-03-03 10:26:43 +01:00
mzack
ea9f9c9c9a fixing logic condition + removing redundant code 2022-03-03 07:18:35 +01:00
mzack
e9d4b4bc52 adding archive analysis support 2022-03-01 18:59:33 +01:00
mzack
c89b9ef820 method renaming 2022-03-01 06:37:59 +01:00
mzack
06ab0dfa99 making code more uniform with chosen architecture 2022-02-26 09:06:43 +01:00
mzack
a51d307967 . 2022-02-26 08:02:16 +01:00
mzack
73d1247b71 fixing typo 2022-02-26 00:54:11 +01:00
mzack
a4b24ea802 adding outputunique logic handling 2022-02-25 12:51:28 +01:00
mzack
1a42780370 improving dump logic 2022-02-25 08:59:10 +01:00
mzack
b3e2456c80 removing unused stuff 2022-02-25 02:13:57 +01:00
mzack
baabe412dd using custom implementation to avoid multiple file read 2022-02-25 02:12:54 +01:00
mzack
763ec611ab . 2022-02-25 01:49:14 +01:00
mzack
748617a9f3 first part 2022-02-25 00:55:05 +01:00
mzack
cf1526b7ed adding result dedupe 2022-02-24 23:54:45 +01:00
mzack
6af462c215 . 2022-02-24 23:35:51 +01:00
mzack
91ad446212 . 2022-02-24 23:18:37 +01:00
mzack
11286210e5 using helper dedupeint 2022-02-24 22:50:41 +01:00
mzack
2409d7a713 dropping multiple scan 2022-02-24 22:41:33 +01:00
mzack
1551feda5a temporary line calculation with multiple file read
todo: replace with one pass scan via io.reader
2022-02-23 23:32:25 +01:00
mzack
6746071979 Refactoring file templates to handle large files in chunks + removing deprecated io methods 2022-02-23 13:54:46 +01:00
Ice3man
7d83d3f8c9
Added line number for file results + stats fixes (#1495)
* Added line number for file results + stats fixes

* Misc

* Improved file result line calculation as per review

* Added new match-all attribute for file template matcher line count
2022-02-10 15:59:05 +05:30
Sandeep Singh
f78ff42dbe
Merge branch 'dev' into more-protocols 2021-11-09 18:27:13 +05:30
Ice3man543
8ad3ebcd05 Made code changes as per review comments 2021-11-05 03:01:41 +05:30
Ice3man543
645ae30a47 Moved to an enum for TemplateType in protocols 2021-11-03 19:53:45 +05:30
forgedhallpass
8f6280dc9d refactor: In case of binary data, show a hexadecimal view as well #1080
* small enhancements with regards to dumping responses
2021-11-01 20:45:54 +02:00
forgedhallpass
b91166a034 Merge remote-tracking branch 'origin/dev' into colored_hexdump 2021-11-01 11:57:45 +02:00
Alexey Zhuchkov
463c1c0142 Use template path in output request 2021-10-30 14:28:30 +03:00
forgedhallpass
04e3c0165a feat: In case of binary data, show a hexadecimal view as well #1080 2021-10-30 13:17:47 +03:00
forgedhallpass
435ec5cd5d Do not show AND matcher information in the command line output if debug is not enabled #1081 2021-10-12 20:06:55 +03:00
forgedhallpass
08cd7a4ba7 [feature] Add coloring to debug information #999
* extracted common logic and made sure that all requests implement the same interface
2021-10-01 16:52:38 +03:00
forgedhallpass
898c9431b5 [feature] Add coloring to debug information #999
* corrected/renamed receivers from one character names to human-readable format
2021-10-01 14:30:04 +03:00
forgedhallpass
a00083e221 [feature] Add coloring to debug information #999
* remove some of the code duplication with a different approach
2021-10-01 14:24:45 +03:00
forgedhallpass
aa78b58866 [feature] Add coloring to debug information #999 [WIP]
* if there are multiple matchers, the response is only displayed once, with all the matching values colored
2021-09-30 21:12:44 +03:00
forgedhallpass
4be6b3cc96 [feature] Add coloring to debug information #999 [WIP]
TODO:
* if there are multiple matchers, make sure the response is only displayed once, with all the matching values colored
* remove code duplication from the request.go files
2021-09-29 19:43:46 +03:00
forgedhallpass
448388a825 Added TODOs next to un-used parameters 2021-08-23 16:21:18 +03:00