- `body` string (containing all request bodies if any)
- `matcher_type` slice of string
- `extractor_type` slice of string
- `description` string
Also, every key-value pair from the template metadata section is accessible. All fields can be combined with logical operators (`||` and `&&`) and used with DSL helper functions.
Similarly, all filters are supported in workflows as well.
In Workflows, Nuclei filters are applied on templates or sub-templates running via workflows, not on the workflows itself.
</Note>
### Public Templates
Nuclei has built-in support for automatic template download/update from [**nuclei templates**](https://github.com/projectdiscovery/nuclei-templates) project which provides [community-contributed](https://github.com/projectdiscovery/nuclei-templates#-community) list of ready-to-use templates that is constantly updated.
Nuclei checks for new community template releases upon each execution and automatically downloads the latest version when available. optionally, this feature can be disabled using the `-duc` cli flag or the configuration file.
### Custom Templates
Users can create custom templates on a personal public / private GitHub / AWS Bucket that they wish to run / update while using nuclei from any environment without manually downloading the GitHub repository everywhere.
To use this feature, users need to set the following environment variables:
-sj, -stats-json write statistics data to an output file in JSONL(ines) format
-si, -stats-interval int number of seconds to wait between showing a statistics update (default 5)
-m, -metrics expose nuclei metrics on a port
-mp, -metrics-port int port to expose nuclei metrics on (default 9092)
```
### Rate **Limits**
Nuclei have multiple rate limit controls for multiple factors, including a number of templates to execute in parallel, a number of hosts to be scanned in parallel for each template, and the global number of request / per second you wanted to make/limit using nuclei, here is an example of each flag with description.
| rate-limit | Control the total number of request to send per seconds |
| bulk-size | Control the number of hosts to process in parallel for each template |
| c | Control the number of templates to process in parallel |
Feel free to play with these flags to tune your nuclei scan speed and accuracy.
<Tip>
`rate-limit` flag takes precedence over the other two flags, the number of
requests/seconds can't go beyond the value defined for `rate-limit` flag
regardless the value of `c` and `bulk-size` flag.
</Tip>
### Traffic **Tagging**
Many BugBounty platform/programs requires you to identify the HTTP traffic you make, this can be achieved by setting custom header using config file at `$HOME/.config/nuclei/config.yaml` or CLI flag `-H / header`
Nuclei supports a variety of methods for excluding / blocking templates from execution. By default, **nuclei** excludes the tags/templates listed below from execution to avoid unexpected fuzz based scans and some that are not supposed to run for mass scan, and these can be easily overwritten with nuclei configuration file / flags.
**exclude-templates** flag is used to exclude single or multiple templates and directory, multiple `-exclude-templates` flag can be used to provide multiple values.
2. Exclude Tags (`-exclude-tags/etags`)
**exclude-tags** flag is used to exclude templates based in defined tags, single or multiple can be used to exclude templates.
Nuclei supports integration with [uncover module](https://github.com/projectdiscovery/uncover) that supports services like Shodan, Censys, Hunter, Zoomeye, many more to execute Nuclei on these databases.
-uf, -uncover-field string uncover fields to return (ip,port,host) (default "ip:port")
-ul, -uncover-limit int uncover results to return (default 100)
-ucd, -uncover-delay int delay between uncover query requests in seconds (0 to disable) (default 1)
```
You need to set the API key of the engine you are using as an environment variable in your shell.
```
export SHODAN_API_KEY=xxx
export CENSYS_API_ID=xxx
export CENSYS_API_SECRET=xxx
export FOFA_EMAIL=xxx
export FOFA_KEY=xxx
export QUAKE_TOKEN=xxx
export HUNTER_API_KEY=xxx
export ZOOMEYE_API_KEY=xxx
```
Required API keys can be obtained by signing up on following platform [Shodan](https://account.shodan.io/register), [Censys](https://censys.io/register), [Fofa](https://fofa.info/toLogin), [Quake](https://quake.360.net/quake/#/index), [Hunter](https://user.skyeye.qianxin.com/user/register?next=https%3A//hunter.qianxin.com/api/uLogin&fromLogin=1) and [ZoomEye](https://www.zoomeye.org/login) .
Example of template execution using a search query.
It can also read queries from templates metadata and execute template against hosts returned by uncover for that query.
Example of template execution using template-defined search queries.
Template snippet of [CVE-2021-26855](https://github.com/projectdiscovery/nuclei-templates/blob/master/cves/2021/CVE-2021-26855.yaml)
```yaml
metadata:
shodan-query: 'vuln:CVE-2021-26855'
```
```console
nuclei -t cves/2021/CVE-2021-26855.yaml -uncover
nuclei -tags cve -uncover
```
We can update the nuclei configuration file to include these tags for all scans.
## **Large scans** using Nuclei
Nuclei fully utilises resources to optimise scanning speed. However, when scanning **thousands**, if not **millions, of targets**, scanning using default parameter values is bound to cause some performance issues ex: low RPS, Slow Scans, Process Killed, High RAM consumption, etc. this is due to limited resources and network I/O. Hence following parameters need to be tuned based on system configuration and targets.
For enterprises dealing with large-scale scanning, optimizing Nuclei can be a burdensome task, especially when scans change frequently. That's where [Nuclei Enterprise](https://nuclei.sh/enterprise) comes in. With its managed offering and dedicated support, Nuclei Enterprise minimizes the burden of optimizing Nuclei on large scans, making it an ideal choice for enterprise-level scanning needs.
- When targets < 1000, `template-spray` should be used. This strategy is slightly faster than `host-spray` but uses more RAM and does not optimally reuse connections.
- When targets > 1000, `host-spray` should be used. This strategy uses less RAM than `template-spray` and reuses HTTP connections along with some minor improvements and these are crucial when mass scanning.
Whatever the `scan-strategy` is `-concurrency` and `-bulk-size` are crucial for tuning any type of scan. While tuning these parameters following points should be noted.
> Since release of [v2.3.2](https://blog.projectdiscovery.io/nuclei-v2-3-0-release/) nuclei uses [goflags](https://github.com/projectdiscovery/goflags) for clean CLI experience and long/short formatted flags.
> [goflags](https://github.com/projectdiscovery/goflags) comes with auto-generated config file support that coverts all available CLI flags into config file, basically you can define all CLI flags into config file to avoid repetitive CLI flags that loads as default for every scan of nuclei.
>
> Default path of nuclei config file is `$HOME/.config/nuclei/config.yaml`, uncomment and configure the flags you wish to run as default.
Here is an example config file:
```yaml
# Headers to include with all HTTP request
header:
- 'X-BugBounty-Hacker: h1/geekboy'
# Directory based template execution
templates:
- cves/
- vulnerabilities/
- misconfiguration/
# Tags based template execution
tags: exposures,cve
# Template Filters
tags: exposures,cve
author: geeknik,pikpikcu,dhiyaneshdk
severity: critical,high,medium
# Template Allowlist
include-tags: dos,fuzz # Tag based inclusion (allows overwriting nuclei-ignore list)
include-templates: # Template based inclusion (allows overwriting nuclei-ignore list)
- vulnerabilities/xxx
- misconfiguration/xxxx
# Template Denylist
exclude-tags: info # Tag based exclusion
exclude-templates: # Template based exclusion
- vulnerabilities/xxx
- misconfiguration/xxxx
# Rate Limit configuration
rate-limit: 500
bulk-size: 50
concurrency: 50
```
Once configured, **config file be used as default**, additionally custom config file can be also provided using `-config` flag.
<Note>
**Running nuclei with custom config file**
```
nuclei -config project.yaml -list urls.txt
```
</Note>
## Nuclei **Reporting**
Nuclei comes with reporting module support with the release of [v2.3.0](https://nuclei.projectdiscovery.io/releases/nuclei-changelog/#nuclei-v230-10-march-2021) supporting GitHub, GitLab, and Jira integration, this allows nuclei engine to create automatic tickets on the supported platform based on found results.
`-rc, -report-config` flag can be used to provide a config file to read configuration details of the platform to integrate. Here is an [example config file](https://github.com/projectdiscovery/nuclei/blob/main/cmd/nuclei/issue-tracker-config.yaml) for all supported platforms.
To store results in Elasticsearch, create a config file with the following content and replace the appropriate values:
```yaml
# elasticsearch contains configuration options for elasticsearch exporter
elasticsearch:
# IP for elasticsearch instance
ip: 127.0.0.1
# Port is the port of elasticsearch instance
port: 9200
# IndexName is the name of the elasticsearch index
index-name: nuclei
```
To forward results to Splunk HEC, create a config file with the following content and replace the appropriate values:
```yaml
# splunkhec contains configuration options for splunkhec exporter
splunkhec:
# Hostname for splunkhec instance
host: '$hec_host'
# Port is the port of splunkhec instance
port: 8088
# IndexName is the name of the splunkhec index
index-name: nuclei
# SSL enables ssl for splunkhec connection
ssl: true
# SSLVerification disables SSL verification for splunkhec
ssl-verification: true
# HEC Token for the splunkhec instance
token: '$hec_token'
```
To forward results to Jira, create a config file with the following content and replace the appropriate values:
The Jira reporting options allows for custom fields, as well as using variables from the Nuclei templates in those custom fields.
The supported variables currently are: `$CVSSMetrics`, `$CVEID`, `$CWEID`, `$Host`, `$Severity`, `$CVSSScore`, `$Name`
In addition, Jira is strict when it comes to custom field entry. If the field is a dropdown, Jira accepts only the case sensitive specific string and the API call is slightly different. To support this, there are three types of customfields.
- `name` is the dropdown value
- `id` is the ID value of the dropdown
- `freeform` is if the customfield the entry of any value
To avoid duplication, the JQL query run can be slightly modified by the config file.
The `CLOSED_STATUS` can be changed in the Jira template file using the `status-not` variable.
`summary ~ TEMPLATE_NAME AND summary ~ HOSTNAME AND status != CLOSED_STATUS`
```yaml
jira:
# cloud is the boolean which tells if Jira instance is running in the cloud or on-prem version is used
cloud: true
# update-existing is the boolean which tells if the existing, opened issue should be updated or new one should be created
update-existing: false
# URL is the jira application url
url: https://localhost/jira
# account-id is the account-id of the Jira user or username in case of on-prem Jira
account-id: test-account-id
# email is the email of the user for Jira instance
email: test@test.com
# token is the token for Jira instance or password in case of on-prem Jira
token: test-token
#project-name is the name of the project.
project-name: test-project-name
#issue-type is the name of the created issue type (case sensitive)
issue-type: Bug
# SeverityAsLabel (optional) sends the severity as the label of the created issue
# User custom fields for Jira Cloud instead
severity-as-label: true
# Whatever your final status is that you want to use as a closed ticket - Closed, Done, Remediated, etc
# When checking for duplicates, the JQL query will filter out status's that match this.
# If it finds a match _and_ the ticket does have this status, a new one will be created.
status-not: Closed
# Customfield supports name, id and freeform. name and id are to be used when the custom field is a dropdown.
# freeform can be used if the custom field is just a text entry
# Variables can be used to pull various pieces of data from the finding itself.
Similarly, other platforms can be configured. Reporting module also supports basic filtering and duplicate checks to avoid duplicate ticket creation.
```yaml
allow-list:
severity: high,critical
```
This will ensure to only creating tickets for issues identified with **high** and **critical** severity; similarly, `deny-list` can be used to exclude issues with a specific severity.
If you are running periodic scans on the same assets, you might want to consider `-rdb, -report-db` flag that creates a local copy of the valid findings in the given directory utilized by reporting module to compare and **create tickets for unique issues only**.
Nuclei supports markdown export of valid findings with `-me, -markdown-export` flag, this flag takes directory as input to store markdown formatted reports.
Including request/response in the markdown report is optional, and included when `-irr, -include-rr` flag is used along with `-me`.
2. By uploading a SARIF file to [Github Actions](https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)
Nuclei expose running scan metrics on a local port `9092` when `-metrics` flag is used and can be accessed at **localhost:9092/metrics**, default port to expose scan information is configurable using `-metrics-port` flag.
Here is an example to query `metrics` while running nuclei as following `nuclei -t cves/ -l urls.txt -metrics`
```bash
curl -s localhost:9092/metrics | jq .
```
```json
{
"duration": "0:00:03",
"errors": "2",
"hosts": "1",
"matched": "0",
"percent": "99",
"requests": "350",
"rps": "132",
"startedAt": "2021-03-27T18:02:18.886745+05:30",
"templates": "256",
"total": "352"
}
```
## Passive Scan
Nuclei engine supports passive mode scanning for HTTP based template utilizing file support, with this support we can run HTTP based templates against locally stored HTTP response data collected from any other tool.
```sh
nuclei -passive -target http_data
```
<Note>Passive mode support is limited for templates having `{{BasedURL}}` or `{{BasedURL/}}` as base path.</Note>
