Add a New Scanner¶
The process of adding a new scanner is relatively simple. The steps are outlined below.
Create a tool definition file¶
This step isn’t strictly necessary, but if you want the pipeline to know how to install/uninstall the tool your scanner uses, this is where that is defined. Tool definition files live in the pipeline/tools
directory.
pipeline/
...
├── recon-pipeline.py
└── tools
├── amass.yaml
├── aquatone.yaml
...
Tool Definition Required Fields¶
Create a .yaml
file with the following fields.
Field Name | Type | Description | Required |
---|---|---|---|
commands |
Array of strings | Which commands to run to install the tool | True |
dependencies |
Array of strings | Each dependency must be defined in a separate definition file, as they’ll be installed before the current defintion’s tool | False |
environ |
Dictionary | Use this if you need to pass information via the environment to your tool (amass.yaml has an example) | False |
shell |
Boolean | true means each command in commands will be run via
/bin/sh -c (see Popen’s shell
argument for more details) |
False |
Useful yaml Helpers¶
pipeline.tools.loader
defines a few helpful functions to assist with dynamically creating values in yaml files as well as linking user-defined configuration values.
Dynamically creating strings and filesystem paths are handled by the following two functions.
!join
- join items in an array with a space character!join_path
- join items in an array with a/
character
In order to get values out of pipeline.recon.config.py
, you’ll need to use one of the yaml helpers listed below.
!get_default
- get a value from thepipeline.recon.config.defaults
dictionary!get_tool_path
- get a path value from thepipeline.tools.tools
dictionary
Simple Example Tool Definition¶
The example below needs go to be installed prior to being installed itself. It then grabs the path to the go
binary from pipeline.tools.tools
by using !get_tool_path
. After that, it creates a command using !join
that will look like /usr/local/go/bin/go get github.com/tomnomnom/waybackurls
. This command will be run by the install waybackurls
command (or install all
).
dependencies: [go]
go: &gobin !get_tool_path "{go[path]}"
commands:
- !join [*gobin, get github.com/tomnomnom/waybackurls]
If you’re looking for a more complex example, check out searchsploit.yaml
.
Write Your Scanner Class¶
You can find an abundance of information on how to write your scanner class starting with Part II of the blog posts tied to recon-pipeline’s creation. Because scanner classes are covered in so much detail there, we’ll only briefly summarize the steps here:
- Select
luigi.Task
orluigi.ExternalTask
as your base class. Task allows more flexibility while ExternalTask is great for simple scans. - Implement the
requires
,output
, and eitherrun
(Task) orprogram_args
(ExternalTask) methods
Add Your Scan to a Wrapper (optional)¶
If you want to run your new scan as part of an existing pipeline, open up pipeline.recon.wrappers
and edit one of the existing wrappers (or add your own) to include your new scan. You should be able to import your new scan, and then add a yield MyNewScan(**args)
in order to add it to the pipeline. The only gotcha here is that depending on what arguments your scan takes, you may need to strategically place your scan within the wrapper in order to ensure it doesn’t get any arguments that it doesn’t expect.