This use-case was developed for a customer in the automotive industry operating a large set of accounts. Their problems were:
To address this we created the Automated Incident Response and Forensics framework. The framework aims to facilitate automated steps for incident response and forensics based on the AWS Incident Response White Paper.
The goal is to provide a set of processes enabled by Lambda functions as to:
Note that this framework does not intend to generate artifacts which can be considered as electronic evidence, submissible in court.
Automated Incident Response and Forensics follows a standard digital forensic process (or phases) consisting of:
Investigations can be performed on static data (e.g. acquired memory or disk images) as well as dynamic, “live” but segregated systems.
By using this environment, a Security Operations Centre team can improve their security incident response process through:
Additionally, customers will avoid using persistent infrastructure and pay for resources when they need them.
The environment will consist of 2 main accounts – a Security and a Forensics accounts. The reason for having 2 accounts is to separate them from any other customer accounts to reduce blast radius in case of a failed forensic analysis, ensure the isolation and protection of the integrity of the artifacts being analyzed, and keeping the investigation confidential. Separate accounts also avoid situations where the threat actors might have used all the resources immediately available to your compromised AWS account by hitting service quotas and so preventing you from instantiating an Amazon EC2 instance to perform investigations. Also, having separate Security and Forensic account allows for creating separate roles – a Responder for acquiring evidence and an Investigator for analyzing it. Each of the roles would have access to their separate accounts.
The Security account is where the 2 main Step Functions are created for memory and disk image acquisition. Once running, those reach into the Member account (an account with the EC2 instances involved in an incident) and trigger a set of Lambda functions that will gather either a memory or disk dump. Those artifacts are then stored in the Forensics account.
A Forensics account will hold the artifacts gathered by the Step Functions in the “Analysis artifacts” S3 bucket. The Forensics account will also have a EC2 Image Builder pipeline that builds an AMI image of a Forensics instance. Currently it’s based on SANS SIFT Workstation (https://www.sans.org/tools/sift-workstation/). The build process uses the Maintenance VPC which has connectivity to the Internet. This can be later used for spinning up EC2 instance for analysis of the gathered artifacts in the Analysis VPC. The Analysis VPC does not have Internet connectivity. By default, the AWS ProServe creates 3 analysis subnets. More subnets can be created (up to 200 which is the quota for number of subnets in VPC) but the VPC endpoints need to have those subnets added for SSM Sessions Manager to work in them.
The CloudFormation scripts are marked 1 to 8, with the first word of the script name indicating in which account the script needs to be deployed. Note that the order of launching the CFN templates is important.
The incident response framework can be triggered by creating a Tag with key SecurityIncidentStatus
and value Analyze
for a given EC2 instance. This will trigger the member Lambda function that will automatically start isolation and memory/disk acquisition. It will also re-tag the asset at the end (or on failure) with Contain
. This triggers the containment which fully isolates the instance with a no INBOUND/OUTBOUND security group and with an IAM role that disallows all access.
When an EC2 instance is compromised or suspect to be compromised, a tag must be attached to it with as key SecurityIncidentStatus with as value either "Analyze" or "Contain" (note that this is case sensitive).
This can be done from the console:
Or use the AWS CLI aws ec2 create-tags --resources <instance-id> --tags Key=SecurityIncidentStatus,Value=Contain
The tag change event will be forwarded to the Lambda function which will evaluate the tag value If the tag value matches the analyze or containment value it will proceed with executing the containment actions:
The flow can also be triggered directly from the security account directly through the stepfunction:
In the pop-up form, fill out the required details. The “Name” field of the execution if optional but helps to identify previous executions (as seen on screen above in point 4.)
The “Input” field, although described as optional, is in fact required by the Memory and Disk Acquisition state machines. It provides required parameters for the underlying scripts to run successfully. The format used is JSON.
The following parameters are used:
An example execution JSON:
{
"CaseId": "INC456789",
"InstanceId": "i-3f182ce62a7962a53",
"Account": "234567890123",
"Region": "us-east-2",
"RetainArtefacts": "true"
}
After the necessary parameters have been provided, press the “Start execution” button on the bottom right.
As the state machine execution progresses, you can check the details of each step. Click on a completed step (marked in green) and then click the link to see the CloudWatch Logs. This functionality is useful in case of a need to troubleshoot.
After a successful execution of the state machine, the artifacts can be found in the Forensics Account in the “
If you want to create a custom action so that you can use the dropdown box from Security Hub, you can deploy the CFN template found under “modules/Security Hub Custom Actions“. Subsequently you will need to modify the ”IRAutomation“ role in each of the member accounts to allow the Lambda function that executes the action to assume the ”IRAutomation“ role.
To do this, go to each member account, find the “IRAutiomation” IAM role: [Image: image.png]And subsequently add “arn:aws:iam::9999999:role/SecurityHubCustomActionLambdaRole” (replace 9999999 with your security account id) so that the policy looks like this:
{
"Version":"2012-10-17",
"Statement":[
{
"Effect":"Allow",
"Principal":{
"AWS":[
"arn:aws:iam::9999999:role/MemoryAutomationLambdaRole",
"arn:aws:iam::9999999:role/SnapshotAutomationLambdaRole",
"arn:aws:iam::9999999:role/SecurityHubCustomActionLambdaRole"
]
},
"Action":"sts:AssumeRole"
}
]
}
Once you have done that you can trigger responses on EC2 events by using the actions dropdown in SecurityHub as shown above.