* Enhance matcher compilation with caching for regex and DSL expressions to improve performance. Update template parsing to conditionally retain raw templates based on size constraints.
* Implement caching for regex and DSL expressions in extractors and matchers to enhance performance. Introduce a buffer pool in raw requests to reduce memory allocations. Update template cache management for improved efficiency.
* feat: improve concurrency to be bound
* refactor: replace fmt.Sprintf with fmt.Fprintf for improved performance in header handling
* feat: add regex matching tests and benchmarks for performance evaluation
* feat: add prefix check in regex extraction to optimize matching process
* feat: implement regex caching mechanism to enhance performance in extractors and matchers, along with tests and benchmarks for validation
* feat: add unit tests for template execution in the core engine, enhancing test coverage and reliability
* feat: enhance error handling in template execution and improve regex caching logic for better performance
* Implement caching for regex and DSL expressions in the cache package, replacing previous sync.Map usage. Add unit tests for cache functionality, including eviction by capacity and retrieval of cached items. Update extractors and matchers to utilize the new cache system for improved performance and memory efficiency.
* Add tests for SetCapacities in cache package to ensure cache behavior on capacity changes
- Implemented TestSetCapacities_NoRebuildOnZero to verify that setting capacities to zero does not clear existing caches.
- Added TestSetCapacities_BeforeFirstUse to confirm that initial cache settings are respected and not overridden by subsequent capacity changes.
* Refactor matchers and update load test generator to use io package
- Removed maxRegexScanBytes constant from match.go.
- Replaced ioutil with io package in load_test.go for NopCloser usage.
- Restored TestValidate_AllowsInlineMultiline in load_test.go to ensure inline validation functionality.
* Add cancellation support in template execution and enhance test coverage
- Updated executeTemplateWithTargets to respect context cancellation.
- Introduced fakeTargetProvider and slowExecuter for testing.
- Added Test_executeTemplateWithTargets_RespectsCancellation to validate cancellation behavior during template execution.
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* feat: move fuzz package to root directory
* feat: added support for input providers like openapi,postman,etc
* feat: integration of new fuzzing logic in engine
* bugfix: use and instead of or
* fixed lint errors
* go mod tidy
* add new reqresp type + bump utils
* custom http request parser
* use new struct type RequestResponse
* introduce unified input/target provider
* abstract input formats via new inputprovider
* completed input provider refactor
* remove duplicated code
* add sdk method to load targets
* rename component url->path
* add new yaml format + remove duplicated code
* use gopkg.in/yaml.v3 for parsing
* update .gitignore
* refactor/move + docs fuzzing in http protocol
* fuzz: header + query integration test using fuzzplayground
* fix integration test runner in windows
* feat add support for filter in http fuzz
* rewrite header/query integration test with filter
* add replace regex rule
* support kv fuzzing + misc updates
* add path fuzzing example + misc improvements
* fix matchedURL + skip httpx on multi formats
* cookie fuzz integration test
* add json body + params body tests
* feat add multipart/form-data fuzzing support
* add all fuzz body integration test
* misc bug fixes + minor refactor
* add multipart form + body form unit tests
* only run fuzzing templates if -fuzz flag is given
* refactor/move fuzz playground server to pkg
* fix integration test + refactor
* add auth types and strategies
* add file auth provider
* start implementing auth logic in http
* add logic in http protocol
* static auth implemented for http
* default :80,:443 normalization
* feat: dynamic auth init
* feat: dynamic auth using templates
* validate targets count in openapi+swagger
* inputformats: add support to accept variables
* fix workflow integration test
* update lazy cred fetch logic
* fix unit test
* drop postman support
* domain related normalization
* update secrets.yaml file format + misc updates
* add auth prefetch option
* remove old secret files
* add fuzzing+auth related sdk options
* fix/support multiple mode in kv header fuzzing
* rename 'headers' -> 'header' in fuzzing rules
* fix deadlock due to merge conflict resolution
* misc update
* add bool type in parsed value
* add openapi validation+override+ new flags
* misc updates
* remove optional path parameters when unavailable
* fix swagger.yaml file
* misc updates
* update print msg
* multiple openapi validation enchancements + appMode
* add optional params in required_openapi_vars.yaml file
* improve warning/verbose msgs in format
* fix skip-format-validation not working
* use 'params/parameter' instead of 'variable' in openapi
* add retry support for falky tests
* fix nuclei loading ignored templates (#4849)
* fix tag include logic
* fix unit test
* remove quoting in extractor output
* remove quote in debug code command
* feat: issue tracker URLs in JSON + misc fixes (#4855)
* feat: issue tracker URLs in JSON + misc fixes
* misc changes
* feat: status update support for issues
* feat: report metadata generation hook support
* feat: added CLI summary of tickets created
* misc changes
* introduce `disable-unsigned-templates` flag (#4820)
* introduce `disable-unsigned-templates` flag
* minor
* skip instead of exit
* remove duplicate imports
* use stats package + misc enhancements
* force display warning + adjust skipped stats in unsigned count
* include unsigned skipped templates without -dut flag
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Purge cache on global callback set (#4840)
* purge cache on global callback set
* lint
* purging cache
* purge cache in runner after loading templates
* include internal cache from parsers + add global cache register/purge via config
* remove disable cache purge option
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* misc update
* add application/octet-stream support
* openapi: support path specific params
* misc option + readme update
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* handle 1 more edgecase
* add integration test for this edgecase
* fix multi-http-var-sharing with integration test
* add -payload-concurrency (-pc) flag
* fix missing internal:true login in multiprotocol engine
* fix/handle absolute invalid url parsing
* support -pc & -jc in go sdk
* fix missing variables in code protocol operators
* add payload count parallelhttp check
* fix mem leak
* bump version tag
* http: add global resp body read limit of 4MB
* skip creating templateCtx in normal templates
* fix mem leak via retryablehttp , fastdialer
* go mod tidy
* remove unused var
* dep update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* log warnings + use scanContext in flow
* refactor flow to use scanContext + log all events
* feat: internal matcher
* fix integration test
* bug fix extractor: merge dynamic values, fix missing extractors in file
* flow: fix 'No Results Found' if last statement output is false
* fix unit test
* fix race-condition & oow in extracted file output
* add mutex for file.Write + set finalizer for os.File
* fix integration test
* disable extractor save to file in lib mode(configurable)
* use sync.Once for init
* disable out of bound image write in headless
* misc updates
* fix headless screenshot test
* fix extractor save to file integration test
* remove 'to' feature in extractors
