readme.md in aws_recon-0.3.5 vs readme.md in aws_recon-0.4.0
- old
+ new
@@ -3,21 +3,21 @@
# AWS Recon
A multi-threaded AWS inventory collection tool.
-The [creators](https://darkbit.io) of this tool have a recurring need to be able to efficiently collect a large amount of AWS resource attributes and metadata to help clients understand their cloud security posture.
+This tool was created to facilitate efficient collection of a large amount of AWS resource attributes and metadata. It aims to collect nearly everything that is relevant to the security configuration and posture of an AWS environment.
-Existing tools (e.g. [AWS Config](https://aws.amazon.com/config)) that do some form of resource collection lack the coverage and specificity we needed. We also needed a tool that produced consistent output that was easily consumed by other tools/systems.
+Existing tools (e.g. [AWS Config](https://aws.amazon.com/config)) that do some form of resource collection lack the coverage and specificity to accurately measure security posture (e.g. detailed attribute data and full policy documents).
Enter AWS Recon, multi-threaded AWS inventory collection tool written in plain Ruby. Though Python tends to dominate the AWS tooling landscape, the [Ruby SDK](https://aws.amazon.com/sdk-for-ruby/) has a few convenient advantages over the [other](https://aws.amazon.com/sdk-for-node-js/) [AWS](https://aws.amazon.com/sdk-for-python/) [SDKs](https://aws.amazon.com/sdk-for-go/) we tested. Specifically, easy handling of automatic retries, paging of large responses, and - with some help - threading huge numbers of requests.
## Project Goals
- More complete resource coverage than available tools (especially for ECS & EKS)
- More granular resource detail, including nested related resources in the output
-- Flexible output (console, JSON lines, plain JSON, file, standard out)
+- Flexible output (console, JSON lines, plain JSON, file, S3 bucket, and standard out)
- Efficient (multi-threaded, rate limited, automatic retries, and automatic result paging)
- Easy to maintain and extend
## Setup
@@ -52,17 +52,17 @@
To run locally, first install the gem:
```
$ gem install aws_recon
-Fetching aws_recon-0.3.0.gem
+Fetching aws_recon-0.4.0.gem
Fetching aws-sdk-3.0.1.gem
Fetching parallel-1.20.1.gem
...
Successfully installed aws-sdk-3.0.1
Successfully installed parallel-1.20.1
-Successfully installed aws_recon-0.3.0
+Successfully installed aws_recon-0.4.0
```
Or add it to your Gemfile using `bundle`:
```
@@ -70,11 +70,11 @@
Fetching gem metadata from https://rubygems.org/
Resolving dependencies...
...
Using aws-sdk 3.0.1
Using parallel-1.20.1
-Using aws_recon 0.3.0
+Using aws_recon 0.4.0
```
## Usage
AWS Recon will leverage any AWS credentials (see [requirements](#requirements)) currently available to the environment it runs in. If you are collecting from multiple accounts, you may want to leverage something like [aws-vault](https://github.com/99designs/aws-vault) to manage different credentials.
@@ -156,17 +156,41 @@
```
#### Example command line options
```
+# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2
+
$ AWS_PROFILE=<profile> aws_recon -s S3,EC2 -r global,us-east-1,us-east-2
```
```
+# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2
+
$ AWS_PROFILE=<profile> aws_recon --services S3,EC2 --regions global,us-east-1,us-east-2
```
+```
+# save output to S3 bucket
+
+$ AWS_PROFILE=<profile> aws_recon \
+ --services S3,EC2 \
+ --regions global,us-east-1,us-east-2 \
+ --verbose \
+ --s3-bucket my-recon-bucket
+```
+
+```
+# save output to S3 bucket with a home region other than us-east-1
+
+$ AWS_PROFILE=<profile> aws_recon \
+ --services S3,EC2 \
+ --regions global,us-east-1,us-east-2 \
+ --verbose \
+ --s3-bucket my-recon-bucket:us-west-2
+```
+
Example [OpenCSPM](https://github.com/OpenCSPM/opencspm) formatted (NDJSON) output.
```
$ AWS_PROFILE=<profile> aws_recon -j \
-s S3,EC2 \
@@ -223,17 +247,18 @@
Most users will want to limit collection to relevant services and regions. Running without any exclusions will attempt to collect all resources from all regions enabled for the account.
```
$ aws_recon -h
-AWS Recon - AWS Inventory Collector (0.3.0)
+AWS Recon - AWS Inventory Collector (0.4.0)
Usage: aws_recon [options]
-r, --regions [REGIONS] Regions to scan, separated by comma (default: all)
-n, --not-regions [REGIONS] Regions to skip, separated by comma (default: none)
-s, --services [SERVICES] Services to scan, separated by comma (default: all)
-x, --not-services [SERVICES] Services to skip, separated by comma (default: none)
-c, --config [CONFIG] Specify config file for services & regions (e.g. config.yaml)
+ -b, --s3-bucket [BUCKET:REGION] Write output file to S3 bucket (default: '')
-o, --output [OUTPUT] Specify output file (default: output.json)
-f, --format [FORMAT] Specify output format (default: aws)
-t, --threads [THREADS] Specify max threads (default: 8, max: 128)
-u, --user-data Collect EC2 instance user data (default: false)
-z, --skip-slow Skip slow operations (default: false)