readme.md in aws_recon-0.3.5 vs readme.md in aws_recon-0.4.0

- old
+ new

@@ -3,21 +3,21 @@ # AWS Recon A multi-threaded AWS inventory collection tool. -The [creators](https://darkbit.io) of this tool have a recurring need to be able to efficiently collect a large amount of AWS resource attributes and metadata to help clients understand their cloud security posture. +This tool was created to facilitate efficient collection of a large amount of AWS resource attributes and metadata. It aims to collect nearly everything that is relevant to the security configuration and posture of an AWS environment. -Existing tools (e.g. [AWS Config](https://aws.amazon.com/config)) that do some form of resource collection lack the coverage and specificity we needed. We also needed a tool that produced consistent output that was easily consumed by other tools/systems. +Existing tools (e.g. [AWS Config](https://aws.amazon.com/config)) that do some form of resource collection lack the coverage and specificity to accurately measure security posture (e.g. detailed attribute data and full policy documents). Enter AWS Recon, multi-threaded AWS inventory collection tool written in plain Ruby. Though Python tends to dominate the AWS tooling landscape, the [Ruby SDK](https://aws.amazon.com/sdk-for-ruby/) has a few convenient advantages over the [other](https://aws.amazon.com/sdk-for-node-js/) [AWS](https://aws.amazon.com/sdk-for-python/) [SDKs](https://aws.amazon.com/sdk-for-go/) we tested. Specifically, easy handling of automatic retries, paging of large responses, and - with some help - threading huge numbers of requests. ## Project Goals - More complete resource coverage than available tools (especially for ECS & EKS) - More granular resource detail, including nested related resources in the output -- Flexible output (console, JSON lines, plain JSON, file, standard out) +- Flexible output (console, JSON lines, plain JSON, file, S3 bucket, and standard out) - Efficient (multi-threaded, rate limited, automatic retries, and automatic result paging) - Easy to maintain and extend ## Setup @@ -52,17 +52,17 @@ To run locally, first install the gem: ``` $ gem install aws_recon -Fetching aws_recon-0.3.0.gem +Fetching aws_recon-0.4.0.gem Fetching aws-sdk-3.0.1.gem Fetching parallel-1.20.1.gem ... Successfully installed aws-sdk-3.0.1 Successfully installed parallel-1.20.1 -Successfully installed aws_recon-0.3.0 +Successfully installed aws_recon-0.4.0 ``` Or add it to your Gemfile using `bundle`: ``` @@ -70,11 +70,11 @@ Fetching gem metadata from https://rubygems.org/ Resolving dependencies... ... Using aws-sdk 3.0.1 Using parallel-1.20.1 -Using aws_recon 0.3.0 +Using aws_recon 0.4.0 ``` ## Usage AWS Recon will leverage any AWS credentials (see [requirements](#requirements)) currently available to the environment it runs in. If you are collecting from multiple accounts, you may want to leverage something like [aws-vault](https://github.com/99designs/aws-vault) to manage different credentials. @@ -156,17 +156,41 @@ ``` #### Example command line options ``` +# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2 + $ AWS_PROFILE=<profile> aws_recon -s S3,EC2 -r global,us-east-1,us-east-2 ``` ``` +# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2 + $ AWS_PROFILE=<profile> aws_recon --services S3,EC2 --regions global,us-east-1,us-east-2 ``` +``` +# save output to S3 bucket + +$ AWS_PROFILE=<profile> aws_recon \ + --services S3,EC2 \ + --regions global,us-east-1,us-east-2 \ + --verbose \ + --s3-bucket my-recon-bucket +``` + +``` +# save output to S3 bucket with a home region other than us-east-1 + +$ AWS_PROFILE=<profile> aws_recon \ + --services S3,EC2 \ + --regions global,us-east-1,us-east-2 \ + --verbose \ + --s3-bucket my-recon-bucket:us-west-2 +``` + Example [OpenCSPM](https://github.com/OpenCSPM/opencspm) formatted (NDJSON) output. ``` $ AWS_PROFILE=<profile> aws_recon -j \ -s S3,EC2 \ @@ -223,17 +247,18 @@ Most users will want to limit collection to relevant services and regions. Running without any exclusions will attempt to collect all resources from all regions enabled for the account. ``` $ aws_recon -h -AWS Recon - AWS Inventory Collector (0.3.0) +AWS Recon - AWS Inventory Collector (0.4.0) Usage: aws_recon [options] -r, --regions [REGIONS] Regions to scan, separated by comma (default: all) -n, --not-regions [REGIONS] Regions to skip, separated by comma (default: none) -s, --services [SERVICES] Services to scan, separated by comma (default: all) -x, --not-services [SERVICES] Services to skip, separated by comma (default: none) -c, --config [CONFIG] Specify config file for services & regions (e.g. config.yaml) + -b, --s3-bucket [BUCKET:REGION] Write output file to S3 bucket (default: '') -o, --output [OUTPUT] Specify output file (default: output.json) -f, --format [FORMAT] Specify output format (default: aws) -t, --threads [THREADS] Specify max threads (default: 8, max: 128) -u, --user-data Collect EC2 instance user data (default: false) -z, --skip-slow Skip slow operations (default: false)