* rebase js-layer PR from @ice3man543
* package restructuring
* working
* fix duplicated event & matcher status
* fix lint error
* fix response field
* add new functions
* multiple minor improvements
* fix incorrect stats in js protocol
* sort output metadata in cli
* remove temp files
* remove dead code
* add unit and integration test
* fix lint error
* add jsdoclint using llm
* fix error in test
* add js lint using llm
* generate docs of libs
* llm lint
* remove duplicated docs
* update generated docs
* update prompt in doclint
* update docs
* temp disable version check test
* fix unit test and add retry
* fix panic in it
* update and move jsdocs
* updated jsdocs
* update docs
* update container platform in test
* dir restructure and adding docs
* add api_reference and remove markdown docs
* fix imports
* add javascript design and contribution docs
* add js protocol documentation
* update integration test and docs
* update doc ext mdx->md
* minor update to docs
* new integration test and more
* move go libs and add docs
* gen new net docs and more
* final docs update
* add new devtool
* use fastdialer
* fix build fail
* use fastdialer + network sandbox support
* add reserved keyword 'Port'
* update Port to new syntax
* misc update
* always enable templatectx in js protocol
* move docs to 'js-proto-docs' repo
* remove scrapefuncs binary
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add weekly performance test with 1k targets
* enabling gh action on PR for test
* testing big list
* updating action
* lowering targets to 250
* lowering targets amount
* workflow update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add flow logic
* progress
* working POC
* fix string slice normalization issue in variables
* update
* fix nil panic
* remove poll()
* load file with sandbox and more
* fix failing integration tests
* JS: log: print in vardump format
* fix missing id in protocols
* fix proto prefix in template context
* flow: add unit tests
* conditional flow support using flow
* fix proto callbacks + more unit tests
* adds integration test
* conditional flow: check if req has any matchers
* fix lint error
* deprecate iterate-all+ missing multi-proto implementation
* fix ip input in raw request
* JS: feat dedupe object+ more builtin funcs
* feat: hide protocol result using hide
* feat: async execution
* complete async execution support
* fix condition-flow without any matchers
* refactor: template executer package (tmplexec)
* flow executor working
* fix data race in templateCtx
* templateCtx redesign
* fix failing unit test
* add multiprotocol support to deprecated syntax
* fix race condition in utils & tlsx
* add documentation in flow package
* remove regions.txt file
* fix minor issue with self contained templates
* fix typos of copilot
* dep + misc update
* fix reqID: use req.Type instead of template.Type
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* use -lfa and -lna in headless
* fix lna in headless
* misc update
* fix nil pointer dereference in test
* fix lint & unit test
* use urlutil
* headless protocol scheme improvements
* add unit and integration tests
* run unit test from binary
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* add headless options flag
* disable some tests for windows
* disable interactsh tests on darwin
* disable network/hex.yaml on windows
* make DisableOn func
* bump fastdialer
* make -ztls flag as no-op and deprecate
* bump fastdialer version from main branch
* dep update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* Add command docs and CLI hook
* Add configurable exclusion from reports
* Register the CLI argument with exporter configuration
* Switch to inverted logic with JSONRequest flag
* Switch variable name for the -include-rr/-irr flag
* Remove flags from README
* Update call for -irr and -or
* convert -irr to no-op
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Improving clientpool with client certificates
* adding test case
* Revert "Merge branch 'dev' into issue-3800-client-cert"
This reverts commit 7f057d742f4b9bda8e83b2052e29617b86b6776d, reversing
changes made to 7297cebcf8bb0f88961b644fc2ac7c040df8ffd9.
* Revert "Revert "Merge branch 'dev' into issue-3800-client-cert""
This reverts commit 2053a248a0cdc2002e0b4b4faa3472cf11c29760.
* go fmt
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* add execute function in dns
* Add payload in dns protocol
* Add integration test to cover dns payload
- also check command line overriding a payload variable
* Update matchedAt and remove trailing dot
* Consider payload data for request count
- Update verbose output to print question
- Update dns requests Requests function to consider payload data
* update gitignore
* bump nuclei version to v2.9.4-dev
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Add utility to write max-requests to templates
* fix lint error
* fix max-request update edge case
* fix convert max-request: 1 => max-request: 1
* WIP, most of the code is commented
* Refactor the find and replace logic
* Skip if template has the max-requests, do not overwrite
- return errors
- add warnings
* Fix the wrong index calculation
- Refactor the getInfoBlock function to not compile regex everytime
* Update -tc flag to filter fields within the classification section (#3606)
* Add fields from Classification section in a template to the -tc flag expression evaluation
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Add tests for filtering Classification section using -tc flag
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Fix hyphenated Metadata keys beings added to parameters
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Add tests to the fix for hyphenated fields encountered in Metadata section
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
---------
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Check severity att while validating (#3540)
* Make severity attribute required
* Update test err msg
* minor
* Do not strict check serverity
* Fix failing test
* Don't print warning in workflow loader
- workflow loader that contains tags load all the template and parse it
- i.e it iw printing warning recursively, ignore as the templates
already getting valiated
* Fix error typo
* Resolve comments
- split the function into two diff
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
* tlsx dep update (#3620)
* updated interactsh version (#3621)
* updated interactsh version
* workflow update
* aws signer: fix missing x-content-sha256 header (#3601)
* fix missing x-content-sha256 header
* fix variable priority in self-contained templates
* remove debug statement
* adds generic raw request parser for self-contained req
* more integration tests
* bug fix: 10x faster race requests
* fix failing integration test
* chore(deps): bump github.com/xanzy/go-gitlab in /v2 (#3624)
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.82.0 to 0.83.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.82.0...v0.83.0)
---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump github.com/miekg/dns from 1.1.53 to 1.1.54 in /v2 (#3625)
Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.53 to 1.1.54.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.53...v1.1.54)
---
updated-dependencies:
- dependency-name: github.com/miekg/dns
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /v2 (#3626)
Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.61 to 1.11.64.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.61...feature/s3/manager/v1.11.64)
---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Fix check for OS made in MustDisableSandbox() (#3631)
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Fix wrong template loading in dev branch (#3629)
* Templates wrong loading
* Add tests to cover following scenarios
- check optional fields only if template loaded
- it should return warning only if template is loaded
* enable color in windows (#3634)
* enable color in windows
* fixed win workflow
* typo update
* tlsx dep update (#3633)
* tlsx dep update
* upgrde httpx => 1.3.0
* Fix check for OS made in MustDisableSandbox() (#3631)
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
* Fix wrong template loading in dev branch (#3629)
* Templates wrong loading
* Add tests to cover following scenarios
- check optional fields only if template loaded
- it should return warning only if template is loaded
* enable color in windows (#3634)
* enable color in windows
* fixed win workflow
* typo update
---------
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
Co-authored-by: Suraj Kamath <kamathsuraj95@gmail.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
* Expose DNS fields for matchers and extractors (#3613)
* Extend dns extractor to dns answer records
* add test template
* Ignore error for dns variables are not found
* Add all the records of answer section
* Fixed the wrong typecasting
* Issue 3564 var override (#3599)
* Check if the variables are override by other means
- you can override the template variable value using command line flags
* Update lazy eval logic
- previously, we were checking any function/expression in variable
- now, update the logic, lazy eval only if variable contains any
protocol variable(global)
* add integration tests
* Add test to check the dsl function working in variable
* gather all generate variables logic in utils
* go mod update
* Refactor the generate variables function
* go mod update+ fix typo
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* update rod to v0.112.9 #3552 (#3637)
* update rod to v0.112.9
* removed unused reflection
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* change max-requests label to max-request
---------
Signed-off-by: iamargus95 <kamathsuraj95@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
Co-authored-by: Suraj Kamath <kamathsuraj95@gmail.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shubham Rasal <shubham@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: lu4nx <lx@shellcodes.org>
