20 Commits

Author SHA1 Message Date
Tarun Koyalwar
bf08913cd0
update logic + config management refactor (#3567)
* adds template manager

* refactor: checkpoint

* centrailized config & template download logic

* refactor removed unused code

* use global template directory

* update related bug fixes

* bug fix create cfg dir if missing

* fix lint error

* bug fix skip writing template dir in callback

* misc update

* remove unused code

* use strings.equalfold for comparison

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-19 21:58:48 +05:30
Keith Chason
4d96025bec
JSON Export Handling Updates (#3466)
* Switch -json to -jsonl

* Add JSON output file

* Update docs for EN and ID

* Fix linting issue with error wrap

* Add -j flag

* Fix call for short flag

* Correct typo "Ciper" to "Cipher" (#3468)

* migrate dsl helper functions to dsl repo (#3461)

* migrate dsl pkg code to dsl repo

* fix lint error

* upgrade dsl dependency

* upgrade deps

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>

* chore(deps): bump github.com/projectdiscovery/httpx in /v2 (#3469)

Bumps [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) from 1.2.7 to 1.2.9.
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Changelog](https://github.com/projectdiscovery/httpx/blob/main/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.2.7...v1.2.9)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/httpx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/weppos/publicsuffix-go in /v2 (#3472)

Bumps [github.com/weppos/publicsuffix-go](https://github.com/weppos/publicsuffix-go) from 0.20.0 to 0.30.0.
- [Release notes](https://github.com/weppos/publicsuffix-go/releases)
- [Changelog](https://github.com/weppos/publicsuffix-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/weppos/publicsuffix-go/compare/v0.20.0...v0.30.0)

---
updated-dependencies:
- dependency-name: github.com/weppos/publicsuffix-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/wappalyzergo in /v2 (#3473)

Bumps [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) from 0.0.81 to 0.0.88.
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.0.81...v0.0.88)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/projectdiscovery/hmap in /v2 (#3470)

Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.10 to 0.0.11.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.10...v0.0.11)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* debug catalog path

* use paths instead of filepath for aws path

* deps update (#3477)

* deps update

* fixing gologger via callback

* Moved `json-export` flag to the other exporters

* Switch "json[-_]exporter to jsonexporter"

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Ramana Reddy <90540245+RamanaReddy0M@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: shubhamrasal <shubhamdharmarasal@gmail.com>
2023-03-31 15:29:29 +05:30
Ice3man
67c094444e
Added cloud scan progress tracking using stats (#3180)
* Added cloud scan progress tracking using stats

* Changed log messsage

* Fixed linting error

* Fixed bug in progress calculation logic

* Changed requests to input with cloud flag

* Changed progress name + removed redundant fields
2023-01-13 13:41:05 +05:30
dependabot[bot]
e0dfc476c3
chore(deps): bump github.com/projectdiscovery/ratelimit from 0.0.1 to 0.0.2 in /v2 (#2915)
* chore(deps): bump github.com/projectdiscovery/ratelimit in /v2

Bumps [github.com/projectdiscovery/ratelimit](https://github.com/projectdiscovery/ratelimit) from 0.0.1 to 0.0.2.
- [Release notes](https://github.com/projectdiscovery/ratelimit/releases)
- [Commits](https://github.com/projectdiscovery/ratelimit/compare/v0.0.1...v0.0.2)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/ratelimit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* fixing int type

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-11-28 12:49:30 +05:30
Shubham Rasal
6b142d794a
Issue 2254 uncover integration (#2786)
* nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/

- `nuclei -uq 'vuln:CVE-2021-26855' -t cves/2021/`

* Add automatic template execution using metadata

- Query uncover after the template is loaded.
- Add the received hosts to the input provider from uncover
- Make NormalizeStoreInputValue() function public to add hosts from the
  runner after uncover hosts received.

* run go mod tidy

* Remove unnecessary comments

* Resolve the requested changes

- move uncover code to protocols/common/uncover package
- Use uncover delay to create uncover rate limiter
- Use single ratelimiter object and remove not required ratelimiters
- Create Set() method for input provider interface
- Rename normalizeStoreInputValue to Set() method

* Solved the uncover running twice.

- flag StringSliceVarP adds the default value twice in the variable
- Check if provider keys exists or not
- Add uncover help block to english readme.md

* Add uncover field functionality

- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf host
- ./nuclei -uq 'vuln:CVE-2021-26855' -t dns -duc -uf ip:port

* Update error messages and solve nuclei hang for wrong uncover engine

- Get uncover engine values from uncover package

* Resolve merge conflicts

* misc option update

* Update logging for templates

- remove duplicate env log printing
- Log message for template queries

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-16 11:12:39 +05:30
Shubham Rasal
721c4964d7
Issue 2613 custom template GitHub (#2630)
* Add custom template download/update support from github

- Accept the -gtr flag to accept the list of custom template
  repos(public/private)
- Accept the -gt flag for github token. It internally sets os.Env
  variable
- Update the flags from
   - -update to -nuclei-update for nuclei self update
   - -ut to -tup for template-update
   - -ud to -tud for custom template location
- Add github.go file which has code related to download and update
  custom templates repos.

* Reslove golint and test case error

* Take default template from community directory

- No need to give explicit community directory path.
- Update the integration test to support the change in path

* Update functional test script update template flag

* Update the path from community to nuclei-template

- Revert the code changes that were made to add community directory

* remove the comment

* Update the interactsh server url for testing

* Update race condition command

* update race condition cmd to download the templates

* Debug integration test failure

* update integration test to update templates

* Refactor downloadCustomTemplate function.

- Remove the log prining instead send the message.

* Add test case for custom template repo download

* move the download repo for loop into diff function

* refactor updateTemplate function.

* Create struct for github repos.

- Create customtemplate struct for repo.
- Add functions to customtemplate

* update readme.md file

* Refactor the downloadCustomTemplate function

- create const variables for github & community as template type
- Update gologger to INF
- Validate templateUpdate to accept only github & community value.
- Validate tempalteUpdate require githubTemplateRepo

* Resolve requested changes

* go mod update

* misc option update

* test update

* Revert back update-template flag to boolean.

- to update community templates
  `nuclei -ut`
- to update custom templates
  `nuclei -ut -gtr ehsandeep/mobile-nuclei-templates`

* Update readme to update flag documentation

* Update go.mod

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-03 20:27:18 +05:30
LuitelSamikshya
944d24a252 mics changes 2022-10-13 09:30:25 -05:00
LuitelSamikshya
cb0da81a14 ratelimit library 2022-10-12 22:04:37 -05:00
Ice3man
09ceb29ba3 Fixed build error on 32bit arch 2022-10-08 01:55:18 +05:30
Mzack9999
99c14f4c9c
implementation of rate limiter with bucket refill and unrestricted token burst (#2536)
* implementation of rate limiter with interval burst

* fixing import path

* fixing syntax

* adding tests

* fixing lint errors

* adding support for context

* moving rate limiter earlier to avoid hitting timeout
2022-09-19 17:09:28 +05:30
Ice3man
67d5769cd9
Added initial catalog interface implementation (#2318)
* Added initial catalog interface implementation

* Added OpenFile to Catalog + disk catalog implementation

* Fixed merge issues

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-08-10 11:05:58 -07:00
sandeep
c815f53e67 interactsh test domain update 2022-08-04 19:37:33 +05:30
Sami
301307bb77
Issue 1705 save responses on disk (#1727)
* save response on disk

* lint error check

* store raw request/response

* lint error fix

* file path

* mock test fix

* readme update

* .txt extension

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
2022-04-02 00:59:02 +05:30
Sandeep Singh
ed309e446a
Interactsh bugfix release (#1368)
* dev version update

* Adding race condition test (#1357)

* Removing linux/arm - Missing chrome package (#1355)

* Trim paths info from released binary (#1356)

* chore(deps): bump golang from 1.17.4-alpine to 1.17.5-alpine (#1363)

Bumps golang from 1.17.4-alpine to 1.17.5-alpine.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: updating default interactsh server to use

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 18:46:41 +05:30
forgedhallpass
7e22d70ded refactor/documentation: typos and grammatical errors 2021-11-25 18:54:16 +02:00
forgedhallpass
3fd1f57b96 refactor: godoc and comment uniformization
Adding space after // and before the godoc/comment
2021-11-25 17:03:56 +02:00
Ice3man
1581c96e4e
Added matched-status flag + template-path and url to output (#1272)
* Added matched-status flag + template-path and url to output
2021-11-22 17:53:25 +05:30
Ice3man543
c1a35b3ff9 Merge branch 'dev' of https://github.com/projectdiscovery/nuclei into more-protocols 2021-11-11 17:21:25 +05:30
Ice3man543
213853c45d Merge from dev 2021-11-08 15:40:18 +05:30
Ice3man543
8ad3ebcd05 Made code changes as per review comments 2021-11-05 03:01:41 +05:30