Assisted Log Enabler for AWS - Find AWS resources that are not logging, and turn them on.
Assisted Log Enabler for AWS is for customers who do not have logging turned on for various services, and lack knowledge of best practices and/or how to turn them on.
With Assisted Log Enabler for AWS, logging is turned on automatically for the various AWS Services for a customer:
Link to related AWS Open Source Blog Post: Introducing Assisted Log Enabler for AWS
Logging information is important for troubleshooting issues and analyzing performance, and when Amazon Web Services (AWS) customers do not have logging turned on, the ability to assist them becomes limited, to the point that performing analysis may be impossible. In some cases, customers may not have the technical expertise needed to set up logging properly for the various AWS services.
Assisted Log Enabler for AWS is designed to ease the customer burden of learning how to turn on logs in the middle of a security incident. Assisted Log Enabler for AWS performs the work of creating an Amazon Simple Storage Service (S3) bucket, checking the services to see if logging is turned on, and activating logging when it's found to be off.
When this work is performed, the customer can be assured that logging within their AWS environment is active to facilitate the investigation of future (and possibly ongoing) security incidents.
The following is a simple diagram on how Assisted Log Enabler for AWS works in a single account, in order to turn on logging for customers.
The following is a simple diagram on how Assisted Log Enabler for AWS works with turning on Amazon S3 Server Access Logging in a single account:
The following is a simple diagram on how Assisted Log Enabler for AWS works with turning on Elastic Load Balancing Access Logging in a single account:
The following permissions are needed within AWS IAM for Assisted Log Enabler for AWS to run. Please see each section for a breakdown per AWS Service and functionality:
# All permissions used within Assisted Log Enabler for AWS:
"ec2:DescribeVpcs",
"ec2:DescribeFlowLogs",
"ec2:CreateFlowLogs",
"ec2:CreateTags",
"logs:CreateLogDelivery",
"s3:GetBucketPolicy",
"s3:PutBucketPolicy",
"s3:PutLifecycleConfiguration"
"s3:PutObject",
"s3:GetObject",
"s3:CreateBucket",
"cloudtrail:StartLogging",
"cloudtrail:CreateTrail",
"cloudtrail:DescribeTrails",
"eks:UpdateClusterConfig",
"eks:ListClusters",
"route53resolver:ListResolverQueryLogConfigAssociations",
"route53resolver:CreateResolverQueryLogConfig",
"route53resolver:AssociateResolverQueryLogConfig",
"route53resolver:TagResource",
"iam:CreateServiceLinkRole", # This is used to create the AWSServiceRoleForRoute53 Resolver, which is used for creating the Amazon Route 53 Query Logging Configurations.
"route53resolver:ListResolverQueryLogConfigs",
"route53resolver:ListTagsForResource",
"route53resolver:DisassociateResolverQueryLogConfig",
"route53resolver:DeleteResolverQueryLogConfig"
"s3:PutBucketLogging",
"s3:GetBucketLogging",
"s3:ListBucket",
"s3:ListAllMyBuckets",
"s3:GetBucketLocation",
"s3:GetBucketAcl",
"s3:PutBucketAcl",
"s3:PutBucketPublicAccessBlock",
"s3:PutBucketLifecycleConfiguration",
"guardduty:ListDetectors",
"guardduty:TagResource",
"guardduty:GetDetector",
"guardduty:CreateDetector",
"guardduty:UpdateDetector",
"guardduty:ListPublishingDestinations",
"guardduty:CreatePublishingDestination",
"guardduty:DescribePublishingDestination",
"wafv2:ListWebACLs",
"wafv2:ListLoggingConfigurations",
"wafv2:PutLoggingConfiguration"
# For adding AWS CloudTrail logs:
"s3:GetBucketPolicy",
"s3:PutBucketPolicy",
"s3:PutLifecycleConfiguration"
"s3:PutObject",
"s3:CreateBucket",
"cloudtrail:StartLogging",
"cloudtrail:CreateTrail",
"cloudtrail:DescribeTrails"
# For adding Amazon VPC Flow Logs:
"s3:GetBucketPolicy",
"s3:PutBucketPolicy",
"s3:PutLifecycleConfiguration"
"s3:PutObject",
"s3:CreateBucket",
"ec2:DescribeVpcs",
"ec2:DescribeFlowLogs",
"ec2:CreateFlowLogs",
"ec2:CreateTags"
# For adding Amazon EKS logs:
"eks:UpdateClusterConfig",
"eks:ListClusters",
"logs:CreateLogDelivery"
# For adding Amazon Route 53 Resolver Query Logs:
"s3:GetBucketPolicy",
"s3:PutBucketPolicy",
"s3:PutLifecycleConfiguration"
"s3:PutObject",
"s3:CreateBucket",
"ec2:DescribeVpcs",
"route53resolver:ListResolverQueryLogConfigAssociations",
"route53resolver:CreateResolverQueryLogConfig",
"route53resolver:AssociateResolverQueryLogConfig",
"route53resolver:TagResource",
"iam:CreateServiceLinkRole" # This is used to create the AWSServiceRoleForRoute53 Resolver, which is used for creating the Amazon Route 53 Query Logging Configurations.
# For adding Amazon S3 Server Access Logs:
"s3:PutBucketLogging",
"s3:GetBucketLogging",
"s3:ListBucket",
"s3:ListAllMyBuckets",
"s3:GetBucketLocation",
"s3:GetBucketAcl",
"s3:PutBucketAcl",
"s3:PutBucketPublicAccessBlock",
"s3:PutBucketLifecycleConfiguration"
# NEW! For adding Elastic Load Balancing Access Logs:
"elb:DescribeLoadBalancers",
"elb:DescribeLoadBalancerAttributes",
"elb:ModifyLoadBalancerAttributes",
"elbv2:DescribeLoadBalancers",
"elbv2:DescribeLoadBalancerAttributes",
"elbv2:ModifyLoadBalancerAttributes",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:ModifyLoadBalancerAttributes"
# For enabling GuardDuty and export findings:
"guardduty:ListDetectors",
"guardduty:GetDetector",
"guardduty:TagResource",
"guardduty:CreateDetector",
"guardduty:UpdateDetector",
"guardduty:ListPublishingDestinations",
"guardduty:CreatePublishingDestination",
"guardduty:DescribePublishingDestination",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:GetBucketLocation"
# For adding WAFv2 logging:
"wafv2:ListWebACLs",
"wafv2:ListLoggingConfigurations",
"wafv2:PutLoggingConfiguration"
# For cleanup of Amazon Route 53 Resolver Query Logs created by Assisted Log Enabler for AWS:
"route53resolver:ListResolverQueryLogConfigs",
"route53resolver:ListTagsForResource",
"route53resolver:ListResolverQueryLogConfigAssociations",
"route53resolver:DisassociateResolverQueryLogConfig",
"route53resolver:DeleteResolverQueryLogConfig"
Additionally, if running from within a AWS Lambda function, the function will need the AWSLambdaBasicExecutionRole in order to run successfully. Please refer to the following link for more details: https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html
The following are the details of what happens within the Assisted Log Enabler for AWS workflow:
The code in its current form can be ran inside the following:
python3 assisted_log_enabler.py
█████ ███████ ███████ ██ ███████ ████████ ███████ ██████
██ ██ ██ ██ ██ ██ ██ ██ ██ ██
███████ ███████ ███████ ██ ███████ ██ █████ ██ ██
██ ██ ██ ██ ██ ██ ██ ██ ██ ██
██ ██ ███████ ███████ ██ ███████ ██ ███████ ██████
██ ██████ ██████
██ ██ ██ ██
██ ██ ██ ██ ███
██ ██ ██ ██ ██
███████ ██████ ██████
███████ ███ ██ █████ ██████ ██ ███████ ██████
██ ████ ██ ██ ██ ██ ██ ██ ██ ██ ██
█████ ██ ██ ██ ███████ ██████ ██ █████ ██████
██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██
███████ ██ ████ ██ ██ ██████ ███████ ███████ ██ ██
Joshua "DozerCat" McKiddy - Customer Incident Response Team (CIRT) - AWS
Cydney "StudyCat" Stude - Customer Incident Response Team (AWS) - Twitter: @cydneystude
Rogerio Kasa - Security Solutions Architect (AWS)
Andrew Yankowsky - Professional Services (AWS)
Twitter: @jdubm31
Type -h for help.
No valid option selected. Please run with -h to display valid options.
python3 assisted_log_enabler.py -h
usage: assisted_log_enabler.py [-h] [--mode MODE] [--bucket BUCKET]
[--include_accounts ACCOUNT_NUMBERS]
[--exclude_accounts ACCOUNT_NUMBERS] [--all]
[--eks] [--vpcflow] [--r53querylogs] [--s3logs]
[--lblogs] [--cloudtrail] [--guardduty]
[--wafv2] [--single_r53querylogs]
[--single_cloudtrail] [--single_vpcflow]
[--single_all] [--single_s3logs]
[--single_lblogs] [--single_guardduty]
[--single_wafv2] [--single_account]
[--multi_account]
Assisted Log Enabler - Find resources that are not logging, and turn them on.
optional arguments:
-h, --help show this help message and exit
--mode MODE Choose the mode that you want to run Assisted Log
Enabler in. Available modes: single_account,
multi_account, cleanup, dryrun. WARNING: For
multi_account, You must have the associated
CloudFormation template deployed as a StackSet. See
the README file for more details.
--bucket BUCKET Specify the name of a pre-existing S3 bucket that you
want Assisted Log Enabler to store logs in. Otherwise,
a new S3 bucket will be created (default). Only used
for Amazon VPC Flow Logs, Amazon Route 53 Resolver
Query Logs, AWS CloudTrail logs, and Amazon GuardDuty.
WARNING: This will replace the bucket policy.
--include_accounts ACCOUNT_NUMBERS
Specify a comma separated list of AWS account numbers
to INCLUDE for multi_account mode.
--exclude_accounts ACCOUNT_NUMBERS
Specify a comma separated list of AWS account numbers
to EXCLUDE for multi_account mode.
Single & Multi Account Options:
Use these flags to choose which services you want to turn logging on for.
--all Turns on all of the log types within the Assisted Log
Enabler for AWS (does not include GuardDuty).
--eks Turns on Amazon EKS audit & authenticator logs.
--vpcflow Turns on Amazon VPC Flow Logs.
--r53querylogs Turns on Amazon Route 53 Resolver Query Logs.
--s3logs Turns on Amazon Bucket Logs.
--lblogs Turns on Amazon Load Balancer Logs.
--cloudtrail Turns on AWS CloudTrail. Only available in Single
Account version.
--guardduty Turns on Amazon GuardDuty and exports findings to an
S3 bucket. Will used specified bucket. WARNING: This
creates a KMS Key to export findings.
--wafv2 Turns on AWS WAFv2 Logs.
Cleanup Options:
Use these flags to choose which resources you want to turn logging off
for.
--single_r53querylogs
Removes Amazon Route 53 Resolver Query Log resources
created by Assisted Log Enabler for AWS.
--single_cloudtrail Removes AWS CloudTrail trails created by Assisted Log
Enabler for AWS.
--single_vpcflow Removes Amazon VPC Flow Log resources created by
Assisted Log Enabler for AWS.
--single_all Turns off all of the log types within the Assisted Log
Enabler for AWS.
--single_s3logs Removes Amazon Bucket Log resources created by
Assisted Log Enabler for AWS.
--single_lblogs Removes Amazon Load Balancer Log resources created by
Assisted Log Enabler for AWS.
--single_guardduty Removes Amazon GuardDuty detectors created by Assisted
Log Enabler for AWS.
--single_wafv2 Removes AWS WAFv2 Logging Configurations created by
Assisted Log Enabler for AWS.
Dry Run Options:
Use these flags to run Assisted Log Enabler for AWS in Dry Run mode.
--single_account Runs Assisted Log Enabler for AWS in Dry Run mode for
a single AWS account.
--multi_account Runs Assisted Log Enabler for AWS in Dry Run mode for
a multi-account AWS environment, using AWS
Organizations.
git clone https://github.com/awslabs/assisted-log-enabler-for-aws.git
cd assisted-log-enabler-for-aws
# For all services:
python3 assisted_log_enabler.py --mode single_account --all
# For Amazon EKS:
python3 assisted_log_enabler.py --mode single_account --eks
# For Amazon VPC Flow Logs:
python3 assisted_log_enabler.py --mode single_account --vpcflow
# For Amazon Route 53 Resolver Query Logs:
python3 assisted_log_enabler.py --mode single_account --r53querylogs
# For AWS CloudTrail:
python3 assisted_log_enabler.py --mode single_account --cloudtrail
# For Amazon S3 Server Access Logs:
python3 assisted_log_enabler.py --mode single_account --s3logs
# NEW! For Elastic Load Balancing Access Logs:
python3 assisted_log_enabler.py --mode single_account --lblogs
# NEW! For GuardDuty:
python3 assisted_log_enabler.py --mode single_account --guardduty
# NEW! For WAFv2 Logs:
python3 assisted_log_enabler.py --mode single_account --wafv2
git clone https://github.com/awslabs/assisted-log-enabler-for-aws.git
cd assisted-log-enabler-for-aws
# For all services:
python3 assisted_log_enabler.py --mode multi_account --all
# For Amazon EKS:
python3 assisted_log_enabler.py --mode multi_account --eks
# For Amazon VPC Flow Logs:
python3 assisted_log_enabler.py --mode multi_account --vpcflow
# For Amazon Route 53 Resolver Query Logs:
python3 assisted_log_enabler.py --mode multi_account --r53querylogs
For Amazon S3 Server Access Logs:
python3 assisted_log_enabler.py --mode multi_account --s3logs
# NEW! For Elastic Load Balancing Access Logs:
python3 assisted_log_enabler.py --mode multi_account --lblogs
# NEW! For GuardDuty:
python3 assisted_log_enabler.py --mode multi_account --guardduty
# NEW! For WAFv2 Logs:
python3 assisted_log_enabler.py --mode multi_account --wafv2
To run Assisted Log Enabler for AWS on GovCloud, make the following adjustments to the code:
region_list
in with only GovCloud regions (i.e. region_list = ['us-gov-east-1', 'us-gov-west-1']
) in all files in the subfunctions directory.arn:aws
to arn:aws-us-gov
)wafv2_logs
) to a GovCloud region, for example:s3.create_bucket(
Bucket=bucket_name,
CreateBucketConfiguration={
"LocationConstraint": "us-gov-east-1"
}
)
A log file containing the detailed output of actions will be placed in the root directory of the Assisted Log Enabler for AWS tool. The format of the file will be ALE_timestamp_here.log
Sample output within the log file:
2021-02-23 05:31:54,207 - INFO - Creating a list of VPCs without Flow Logs on in region us-west-2.
2021-02-23 05:31:54,208 - INFO - DescribeVpcs API Call
2021-02-23 05:31:54,679 - INFO - List of VPCs found within account 111122223333, region us-west-2:
2021-02-23 05:31:54,679 - INFO - DescribeFlowLogs API Call
2021-02-23 05:31:54,849 - INFO - List of VPCs found within account 111122223333, region us-west-2 WITHOUT VPC Flow Logs:
2021-02-23 05:31:54,849 - INFO - Activating logs for VPCs that do not have them turned on.
2021-02-23 05:31:54,849 - INFO - If all VPCs have Flow Logs turned on, you will get an MissingParameter error. That is normal.
2021-02-23 05:31:54,849 - INFO - CreateFlowLogs API Call
2021-02-23 05:31:54,944 - ERROR - An error occurred (MissingParameter) when calling the CreateFlowLogs operation: The request must include the ResourceIds parameter. Add the required parameter and retry the request.
2021-02-23 05:31:54,946 - INFO - Checking to see if CloudTrail is on, and will activate if needed.
2021-02-23 05:31:54,946 - INFO - DescribeTrails API Call
2021-02-23 05:31:54,983 - INFO - There is a CloudTrail trail active. No action needed.
2021-02-23 05:31:54,984 - INFO - Turning on audit and authenticator logging for EKS clusters in region af-south-1.
Dry Run modes for single and multi-account are both available. These modes allow you to check for resources in your environment that do not have logging turned on, but does not activate the logging for said resources.
To run Assisted Log Enabler for AWS in Dry Run mode, you can use the commands below:
# Single Account Dry Run
python3 assisted_log_enabler.py --mode dryrun --single_account
# Multi-Account Dry Run
python3 assisted_log_enabler.py --mode dryrun --multi_account
Once the logs have been enabled, you can safely remove any of the downloaded files from AWS CloudShell.
For any AWS IAM Roles that are created, either manually or using AWS CloudFormation StackSets, those can be safely deleted upon enablement of logs through the Assisted Log Enabler for AWS.
A cleanup mode is available within the Assisted Log Enabler for AWS (currently only for single account). Collected logs within Amazon S3 will NOT be removed, however, logging resources can be removed by following the below commands:
# To remove Amazon Route 53 Resolver Query Log resources created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_r53querylogs
# To remove Amazon VPC Flow Log resources created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_vpcflow
# To remove AWS CloudTrail trails created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_cloudtrail
# To remove Amazon S3 Server Access logging created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_s3logs
# NEW! To remove Elastic Load Balancing Access logging created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_lblogs
# NEW! To remove GuardDuty detectors created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_guardduty
# NEW! To remove WAFv2 logging created by Assisted Log Enabler for AWS (single account):
python3 assisted_log_enabler.py --mode cleanup --single_wafv2
All resources created fall into the customer side of the Shared Responsibility Model.
For AWS customers, please refer to the following link for more information about the Shared Responsibility Model: Link
For analyzing logs created by Assisted Log Enabler for AWS, consider taking a look at the AWS Security Analytics Bootstrap, a tool that provides an Amazon Athena analysis environment that's quick to deploy, ready to use, and easy to maintain. Link to GitHub repository.
For an point-and-quick solution to analyze Amazon VPC Flow Logs, check out this AWS blog post for instructions on how to deploy an Amazon Athena analysis environment that's compatible with your Amazon VPC Flow Logs, and provides several sample queries that can allow you to perform an investigation quickly without worrying about the format of the Amazon VPC Flow Logs.
For answers to cost-related questions involved with this solution, refer to the following links:
Please use the Issues section to submit any feedback, such as features or recommendations, as well as any bugs that are encountered.
See CONTRIBUTING for more information.
This project is licensed under the Apache-2.0 License.