Cloud computing has transformed the enterprise IT landscape through unprecedented scalability and flexibility, cost efficiencies, and access to a vast array of cutting-edge technologies. But it also presents new challenges to incident response because of the complex and dynamic nature of cloud-based workloads.
Systems are highly distributed and services are continually launching and closing down, making it difficult for forensics teams to capture the evidence they need to deal with an attack. This continually changing patchwork of resources also complicates matters for containment and can significantly increase the time it takes to get an attack under control.
Forensics-ready design and one-click containment are two cyber-resilience practices that set out to address these problems by providing better visibility into an attack and automated procedures to speed up the response.
In this post, we briefly explain the two concepts and run through in detail the typical measures you should expect to include in your forensics-readiness and rapid-containment strategies.
Forensics-ready design largely refers to the tools and processes an organization puts in place so that, in the event of an attack on their systems, they're able to quickly access the digital evidence they need for forensic investigation. But it also covers:
Logs are usually the first port of call in any cyber incident investigation, as they provide a high-level overview of the attacker's activity. They are relatively easy to interpret and provide an important timeline of events, such as login and authorization attempts, API calls, and file access.
As part of your design, you should centralize log collection so that investigation teams have a clear, unified view of activity, thereby avoiding the burden of piecing together information from siloed log sources.
Snapshots are important not only for recovery but also for investigation of an attack. They complement logs by providing a record of the system state at specific moments in time before, during, and after an attack. These yield information such as changes to configuration files, modifications to data, and the point at which the system state was clean and free of any potential persistence.
Memory dumps give you access to highly transient artifacts that are never recorded on logs or in storage. For example, they capture details such as running processes, encryption keys, and shellcode; they are particularly useful for identifying and investigating fileless malware.
However, bear in mind you need a different approach to access memory dumps in the public cloud. This is because cloud service providers (CSPs) don’t allow direct access to their physical hardware. All the same, each vendor does provide customers with a number of options. These come in the form of tooling or sets of procedures, which allow you to capture the memory dumps you need.
Alternatively, you can use third-party tools. These can perform selective memory dumps triggered by suspicious activity and may also provide support for containerized environments.
You should take care to protect the security and integrity of all your forensic evidence. But pay particular attention to your memory dumps, as they invariably contain highly sensitive information.
To build a fully comprehensive picture of the attack, you should also perform detailed PCAP analysis, as this provides invaluable network-derived information such as:
Resource tagging provides the context you need to quickly and efficiently understand the scope of the attack and target containment measures accordingly. Moreover, you can use tags in automated workflows for selective quarantine of resources.
Robust tagging practices should aid the investigation process by providing you with information such as:
It should also go without saying that you should have policies in place to enforce resource tagging.
Investigation playbooks are key to effective incident response, as they serve as a reference guide to the steps you'll need to take to investigate an attack. For example, how to quickly and efficiently determine:
In much the same way crime-scene investigators follow strict protocols for recording criminal evidence, you'll need a set of procedures for handling digital evidence to ensure its integrity and authenticity in the event an incident progresses to legal proceedings.
So you'll need to develop a documentation process for methodically collecting and labelling evidence, such as logs, actions taken, who performed them, and who collected and accessed your evidence.
You'll also need tools to manage and protect your artifacts. For example, by using an implementation of immutable file technology, such as Amazon S3 Object Lock, to tamper-proof your logs. Furthermore, you should encrypt logs and other evidence if they're likely to contain sensitive information.
Tabletop and game-day exercises are a series of dry runs designed to validate the effectiveness of your incident response processes. Tabletop exercises are typically discussion oriented. By contrast, game days are more operational in nature and involve simulated incidents, which serve as an opportunity to test automated containment measures work as they should do.
One-click containment is an automation strategy that incident response teams use to halt the spread of an attack as quickly as possible—with the view to minimize damage to their systems and help maintain business continuity.
It comprises a series of structured containment workflows, which they can trigger at the click of a button, saving precious time they'd otherwise have to spend on manual processes. Security solutions such as security orchestration, automation, and response (SOAR) and endpoint detection and response (EDR) support out-of-the-box workflows for the most common scenarios. But you'll likely need to complement these with custom scripts for your own unique IT setting.
The following are typical examples of the procedures your one-click containment workflows would need to perform.
In order to completely lock out a compromised identity, your one-click containment mechanism will need to automate a variety of manual steps you'd usually have to perform in the identity and access management (IAM) portal of your CSP.
These will include:
You'll also have to quickly block traffic into and out of your virtual machines to prevent further damage to your cloud environment.
You'll therefore need to programmatically create a series of steps that reconfigure your virtual firewall to deny all traffic. In the case of AWS, your workflow will need to change your security group settings. The concept is basically the same in Microsoft Azure and Google Cloud Platform but using network security groups (NSG) and VPC firewall rules respectively.
Similarly, you should have a one-click routine in place to swiftly revoke compromised API keys so an attacker can no longer access associated resources. And you'll also need to immediately quarantine any devices affected by the attack, thereby necessitating an automated method of disconnecting them from the rest of your network.
You could simply block the attacker with a blanket ban on public access to the data at risk. But you should ideally use a more targeted approach to help maintain business continuity. For example, by automating the process you'd use to blacklist untrusted IP addresses—via access policies available to storage services such as Amazon S3 and Azure Blob Storage.
In addition to breaches, you'll need to quickly deal with any denial-of-service attack to prevent your servers from being brought down by bogus incoming service requests.
All three leading CSPs offer some form of blackhole routing functionality—a process whereby you redirect an attacker to a null0 interface that discards the traffic it receives. However, the technical implementation can be quite complex, involving a number of different vendor services.
Alternatively, you could integrate third-party DDoS mitigation technology, which is available through numerous cloud security solution providers.
Virtually every organization is now highly dependent on information technology. So, when a cyberattack strikes, it's a race against time to get it under control and bring systems back online as soon as possible.
The complexity of cloud-based deployments will only serve to compound the problem—unless you already have tooling and procedures to efficiently investigate and contain threats and minimize the potential for further damage.
But it's also a race against time to get your forensic-ready and one-click containment strategies in place. Because the clock is ticking and an attack can happen at any time.
Don’t wait for an incident to expose the gaps. RapidScale’s experts help you design forensic-ready architectures and one-click containment strategies that keep your business moving—no panic, no guesswork, just uncompromising outcomes. Let’s strengthen your cloud defenses before the next attack strikes. Send our team a message today.