49 Commits

Author SHA1 Message Date
Dwi Siswanto
1e49cd9800
ci: don't clean modules cache (#5519)
* chore(make): add `verify` command

Signed-off-by: Dwi Siswanto <git@dw1.io>

* ci: remove hygiene (`clean -modcache`) with verify instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-08-15 18:45:42 +05:30
Tarun Koyalwar
2df1b2e88e
file proto missing vars in flow & multi-protocol (#5480)
* fix missing template context in file proto

* fix file protocol missing vars

* fix test

* skip example advanced test
2024-08-04 18:14:08 +05:30
Dwi Siswanto
ddcc9211be
ci: use composite actions (#5483)
Signed-off-by: Dwi Siswanto <git@dw1.io>
2024-08-03 23:44:37 +05:30
Ramana Reddy
61e9be530f
Fix: skip-variables-check option in self-contained templates (#5053)
* fix: skip-variables-check option in self-contained templates

* Update build workflow envs
2024-04-23 16:04:32 +05:30
mzack
96d7d02701 adding speed example 2024-04-04 20:22:28 +02:00
Tarun Koyalwar
dc44105baf
nuclei v3 : misc updates (#4247)
* use parsed options while signing

* update project layout to v3

* fix .gitignore

* remove example template

* misc updates

* bump tlsx version

* hide template sig warning with env

* js: retain value while using log

* fix nil pointer derefernce

* misc doc update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-10-17 17:44:13 +05:30
Tarun Koyalwar
c35162c8ef
nuclei v3 bug fixes (#4176)
* store and generate signer keys

* fix trailing newline in code_response

* fix formatting and update error string

* fix integration test

* fix rsaSigned code integration test

* bug fixes , docs and more

* bump go -> 1.21

* use 'response' as default part in code templates

* disable sourcemaps for all js runtimes

* disable eval function

* rewrite file validation in sandbox mode

* sandbox file read improvements + minor refactor

* refactor sign and verify logic

* fix panic and missing id in code protocol

* disable re-signing code protocol templates

* fix code resigning in tests

* allow -lfa in test for signing templates

* start index from 1 in flow and multiproto

* remove testfiles

* add python in integration test

* update code protocol docs

* add python engine in template

* rework template signer

* fix integration test and more

* reworked template signer

* fix lint error

* display signature stats

* update docs

* add user fragment to signature

* use md5 to generate fragment

* update docs with code re-sign

* misc updates

* public crt update

* remove workflow info statement

* fix printing issues

* refactor preprocessor logic

* remove debug statement

* fix failing example test

* go mod tidy

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2023-10-13 13:17:27 +05:30
Tarun Koyalwar
2d317884b5
SDK: abstracted and minimal nuclei v3 sdk (#4104)
* new sdk progress

* nuclei v3 new sdk/library

* fix TestActionGetResource broken link

* fix clistats + clustering and more

* fix lint error

* fix missing ticker

* update advanced library usage example

* fix integration tests

* misc update

* add utm_source and fix lint error

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-09-02 14:34:05 +05:30
Josh Soref
4c1c5301b9
Spelling (#4008)
* spelling: addresses

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: asynchronous

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: basic

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: brute force

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: constant

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: disables

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: engine

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: every time

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: execution

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: false positives

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: from

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: further

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: github

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: gitlab

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: highlight

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: hygiene

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: ignore

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: input

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: item

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: itself

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: latestxxx

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: navigation

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: negative

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: nonexistent

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: occurred

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: override

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: overrides

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: payload

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: performed

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: respective

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: retrieve

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: scanlist

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separated

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: separator

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: severity

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: source

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: strategy

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: string

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: templates

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: terminal

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: timeout

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing slash

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: trailing

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

* spelling: websocket

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>

---------

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-08-02 00:03:43 +05:30
Mzack9999
c9d0942bc1
Extend headless contextargs (#3850)
* extend headless contextargs

* using darwin-latest

* grouping page options

* temp commenting code out

* fixing test

* adding more checks

* more checks

* fixing first navigation metadata

* adding integration test

* proto update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-26 22:55:51 +05:30
Mzack9999
dfd4d5b855
Adding interact keepalive to reduce server-side id pruning (#3680)
* adding interact keepalive + improving init logic

* dep update

* go version update

* readme update

* version bump

* fixing invalid format

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-05-21 01:56:13 +05:30
Sandeep Singh
7ac9f05e53
updated interactsh version (#3621)
* updated interactsh version

* workflow update
2023-04-29 14:55:58 +05:30
Mzack9999
6f4b1ae48a
Replacing ccache with generic gcache (#3523)
* Replacing ccache with generic gcache

* fixing lint issues

* removing unecessary hashing + using errorutils

* making test more tolerant

* removing dead code + refactor

* removing redundant code

* removing race

* maint

* moving code

* adding more iterations

* note + typo

* temporary fixing stop-at-first-match with interact

* wrapping internal map with mux

* sort before running integration test

* fix deadlock in requestShouldStopAtFirstMatch

* add timeout to integration_test workflow

* attempting to remove outer lock

* adds interactsh protocol tests in integration_test

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2023-04-16 23:19:35 +05:30
dependabot[bot]
a2e4b4197d
chore(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 05:57:09 +00:00
Mzack9999
b3b2ec9baa
Merge pull request #3351 from projectdiscovery/issue-3081-retry-gh
Add retry script for build, test and integration commands
2023-03-06 09:53:11 +01:00
sandeep
0f80575c44 Update version number and add paths to pull request. 2023-03-06 01:03:30 +05:30
Tarun Koyalwar
3e53087617
fix missing port in matched ssl templates (#3380)
* add openssl support + fix missing port

* fix failing tests

* go mod update

* workflow update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-03-04 04:47:41 +05:30
shubhamrasal
d32c9c7f56 Revert "Resolve the requested changes"
This reverts commit 4ad64eb3575aa5b3d390b31d4a1bcd49630da697.
2023-03-01 19:06:30 +05:30
shubhamrasal
4ad64eb357 Resolve the requested changes
- Add the logic to retry the test case that is failed.
- Accpet the number of how many times we want to retry
- Run the last retry in debug
2023-02-24 13:42:19 +05:30
shubhamrasal
ed46e9560e Revert "Add retry script for build, test and integration commands"
This reverts commit 63747a5c541ef9c32370feb8212317c152979b3b.
2023-02-24 06:15:16 +05:30
shubhamrasal
311ea5bd01 Update integration tests wd 2023-02-21 20:00:48 +05:30
shubhamrasal
63747a5c54 Add retry script for build, test and integration commands 2023-02-21 19:50:52 +05:30
Mzack9999
0b3992cdb8
moving examples to go file (#3187) 2023-01-12 14:57:32 +05:30
xm1k3
34120fbecc
#3046 persistent failed item status and #2065 failed items reporting error once (#3047)
* added logs for debug

* fixes

* removed logs

* using cache item

* implemented multiple tests

* fixed some unit tests

* implemented test for skipping

* added multiple tests together

* added mark failed

* fix on tests

* better test implementation + concurrent

* fix: fixes on concurrent tests

* removed parallel and 1 unit test

DOCS: by default the command go test runs in parallel tests for different packages, and default is the number of CPUs available (see go help build)

* fixes on go routine

* increasing parallelism of once.Do

* bumping go to 1.19 for atomic types support

* removing redundant check + fixing test concurrency on create

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-01-02 13:52:06 +05:30
sandeep
2230e0be49 misc workflow update 2022-12-05 23:26:25 +05:30
Sandeep Singh
5c8e715fc6
workflows improvements (#2963) 2022-12-05 18:22:55 +05:30
Shubham Rasal
d5a09e733a
Issue 2772 s3 provider support (#2825)
* Add s3 bucket template provider

- Refactor the custom github template code
- add interface for template provider

* Validate if aws creds are passed if bucket flag

- refactor s3 provider struct to take client
- add function which returns the aws s3 client
- update error messages

* Add aws s3 bucket flags documentation in README.md

- Rename the github_test.go to customTemplate_test.go

* go mod update

* Move template provider code to pkg/external/customtemplates dir

* Remove github and aws update variables from flag

* Rename CustomTemplateProvider to Provider

* Update integration and function command in makefile

* Update github test case, accept token

* readme update

* go mod tidy

* Update build-test.yml

* handle empty dir in s3

* Add requested changes

- download/update s3 and github only when `-ut` is passed
- only print the missing env variable for s3
- add the custom templates path in
  ~/.config/nuclei/.template-config.json

* print custom paths only if exists in config file

* misc update

* tag update

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-12-02 03:57:00 +05:30
Sandeep Singh
2b7b7649a8
build workflow update (#2911) 2022-11-29 04:20:20 +05:30
Mzack9999
2586d9b4a7 removing subset 2022-11-24 22:12:44 +01:00
Mzack9999
440bfe030a Merge branch 'dev' into issue-2832-race-condition-test 2022-11-24 22:10:42 +01:00
xm1k3
6d76fcd4e7 fix oh wrong path 2022-11-23 14:40:00 +01:00
Sandeep Singh
ff9596f618
workflow updates (#2883)
* workflow updates

* added go mod download in build test
2022-11-23 17:13:52 +05:30
xm1k3
632c68a996 added multiple targets on gh action 2022-11-23 10:37:30 +01:00
xm1k3
ef6b7d9dc9 added tags to race condition 2022-11-23 10:11:46 +01:00
Shubham Rasal
721c4964d7
Issue 2613 custom template GitHub (#2630)
* Add custom template download/update support from github

- Accept the -gtr flag to accept the list of custom template
  repos(public/private)
- Accept the -gt flag for github token. It internally sets os.Env
  variable
- Update the flags from
   - -update to -nuclei-update for nuclei self update
   - -ut to -tup for template-update
   - -ud to -tud for custom template location
- Add github.go file which has code related to download and update
  custom templates repos.

* Reslove golint and test case error

* Take default template from community directory

- No need to give explicit community directory path.
- Update the integration test to support the change in path

* Update functional test script update template flag

* Update the path from community to nuclei-template

- Revert the code changes that were made to add community directory

* remove the comment

* Update the interactsh server url for testing

* Update race condition command

* update race condition cmd to download the templates

* Debug integration test failure

* update integration test to update templates

* Refactor downloadCustomTemplate function.

- Remove the log prining instead send the message.

* Add test case for custom template repo download

* move the download repo for loop into diff function

* refactor updateTemplate function.

* Create struct for github repos.

- Create customtemplate struct for repo.
- Add functions to customtemplate

* update readme.md file

* Refactor the downloadCustomTemplate function

- create const variables for github & community as template type
- Update gologger to INF
- Validate templateUpdate to accept only github & community value.
- Validate tempalteUpdate require githubTemplateRepo

* Resolve requested changes

* go mod update

* misc option update

* test update

* Revert back update-template flag to boolean.

- to update community templates
  `nuclei -ut`
- to update custom templates
  `nuclei -ut -gtr ehsandeep/mobile-nuclei-templates`

* Update readme to update flag documentation

* Update go.mod

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-03 20:27:18 +05:30
Ice3man
e193e7c87e
Added tlsx integration to nuclei (#2522)
* Added tlsx integration to nuclei

* tls tests fix

* Added helper functions + upgrade tlsx to fix

* go mod update

* workflow fix to race test on windows

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-01 23:56:55 +05:30
Sandeep Singh
1bf885e97b
Increases race condition coverage (#2293) 2022-07-19 01:10:10 +05:30
forgedhallpass
c6eaff10f9 chore: updated the GitHub actions to use go 1.18 2022-04-18 15:43:37 +03:00
dependabot[bot]
34e8125844
chore(deps): bump actions/setup-go from 2 to 3 (#1827)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2 to 3.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-11 14:03:19 +05:30
LuitelSamikshya
bfb19fc74e added github token 2022-03-21 15:42:19 -05:00
dependabot[bot]
b531b47668
chore(deps): bump actions/checkout from 2 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-07 05:04:20 +00:00
Mzack9999
860a40717c
Removing outdated macos-11.0 from gh action (#1390) 2021-12-18 18:29:13 +05:30
LuitelSamikshya
3f86bc7d62
Support major os (#1347)
* workflow check

* changes in run.sh file for windows support
2021-12-16 13:47:29 +05:30
forgedhallpass
974cbfb35e
feat: Re-run failed integration tests in debug mode (#1367)
* If the tests are executed through GitHub actions and there are failed integration tests, they will be re-executed with verbose output to help figuring out the underlying issues.
* Added some grouping to make reading the logs easier

Ticket: #1365
2021-12-14 18:13:53 +02:00
Mzack9999
4c565679b5
Adding race condition test (#1357) 2021-12-10 21:42:46 +05:30
sandeep
16d7dbeb66 Misc changes 2021-09-01 15:40:53 +05:30
sandeep
f3fced5bad Update build-test.yml 2021-08-21 23:37:04 +05:30
sandeep
4e857cb951 Separating build and lint tests 2021-06-16 16:07:19 +05:30
sandeep
ff4a855bc8 Workflow improvements 2021-06-15 21:31:56 +05:30