5 cloud incident response priorities for the first hour
In recent months, almost every incident my team has handled has started with a compromised identity.
In recent months, almost every incident my team has handled has started with a compromised identity. The attacker did not need an exploit. They had a key, a token, or a role that already belonged in the environment and used the permissions that came with it.
The challenge is understanding how far that access extends. A seemingly limited identity can often read the credentials of a much more powerful one. If you disable the first account you find and stop there, the attacker may still have another way in.
This article explains where compromised credentials come from, how a “read-only” identity can lead to administrative access, and the five priorities I focus on in the first hour of a cloud incident. The guiding principle is to cut off the attacker’s access while keeping the business running. Shutting down production is a last resort.
The incidents I investigate repeatedly lead back to four sources of credentials. Each needs attention because addressing one leaves the other three unresolved.
A CI/CD pipeline needs permissions to build, test, and deploy software. If a workflow runs untrusted code in a job with access to deployment tokens, that code can steal them. The code could come from a pull request submitted through a fork or a compromised third-party action.
The Trivy supply chain compromise illustrates the risk. According to Aqua Security, attackers exploited a misconfiguration in Trivy’s GitHub Actions environment in late February 2026 and stole a privileged token. Credentials were rotated on March 1, but some were missed. On March 19, the attacker used the remaining credentials to publish a malicious Trivy release and redirect dozens of trivy-action tags. Rotating most of the credentials left the attacker with access.
Infostealer malware collects saved passwords and session tokens from an infected machine. Wiping the laptop does not invalidate tokens the attacker has already copied elsewhere.
In its June 2024 report on attacks against Snowflake customer environments, Mandiant traced the incidents it investigated to stolen customer credentials. Some had been taken by infostealers years earlier and remained valid, and the affected accounts lacked multifactor authentication. Mandiant and Snowflake notified approximately 165 organizations of potential exposure.
A single commit can expose a key. In one case we handled, a key remained in a public repository for five months before anyone noticed. It could read every storage bucket in the account.
The length of the exposure matters, but the key’s reach matters even more. Finding the key is the start of the investigation. The next step is to establish everything it could access.
Applications running on EC2, in containers, or in Lambda get their cloud permissions from an attached IAM role. An attacker who compromises the application can use that role from the affected machine or take its credentials and use them elsewhere.
This route is easy to overlook because it does not begin with a leaked key or a stolen laptop. The attacker compromises the workload and gains access to its role. GuardDuty even has a finding for EC2 role credentials used from outside AWS. As soon as you suspect a workload has been compromised, treat its role as compromised too.
Consider a reporting identity that everyone describes as read-only. Its policy also allows it to read a secret containing credentials for a backup job that runs as an administrator. The attacker now has a route to administrative access, and that route can appear unremarkable until it is used.
The “read-only” label describes the intended access; the actual policy determines what is possible. AWS’s own ReadOnlyAccess managed policy includes s3:Get* and lambda:Get*, allowing it to download S3 objects and read Lambda environment variables. These are places where Terraform state, .env files, and database passwords stored “for now” tend to end up. The policy does not include secretsmanager:GetSecretValue, and reading a secret’s metadata is different from reading its value.
In one case we investigated, the attacker moved from the initial identity, through a stored secret, to administrative access in about three minutes. Eleven hours later, 2,800 GPU instances were running across 13 regions, and the attacker was checking the account’s cost dashboard.
That path existed before the attacker arrived. Reviewing the initial identity’s permissions in isolation would have missed it.
These are the five priorities I use to guide the response. Although they are numbered, several need to begin in parallel. Naming an incident lead, preserving evidence and calling for help should start alongside containment, without waiting for the identity investigation to finish.
Move from the least disruptive action to the most disruptive. In most cloud incidents, you can cut off the attacker’s access without taking production down. Shutting systems down is a last resort when other containment options have failed.
Identify the user or role behind the suspicious activity and block its access. Deactivate access keys rather than deleting them. A deactivated key stops working while its ID remains available for subsequent CloudTrail searches. For roles, attach a deny policy or revoke the role’s active sessions. Rotating a credential does not end sessions that have already been issued, so both actions are necessary.
Then trace the identity’s access. Establish which secrets it could read, which credentials those secrets contained, and which roles it could assume. Treat every credential it could reach as exposed, even without evidence of use, and repeat the investigation for each identity you find.
Check for access the attacker created, including new IAM users, access keys, roles and identity providers, as well as changes to trust policies. Attackers use these to regain access after the original entry point has been blocked.
Revoking and rotating credentials also has an operational cost. If production depends on a role you revoke, it loses access along with the attacker. Rotating a database password that live applications use creates an outage of its own. Make these decisions deliberately with the incident lead, and deploy replacement credentials to systems the attacker does not control.
Shutting down a machine destroys evidence held in memory, including running processes, open connections, and credentials. It also interrupts whatever the machine was serving. Work through the following actions in order and stop when the threat is contained:
Verify that isolation has worked. AWS’s containment guidance notes that changing an instance’s security groups does not terminate existing tracked connections. Only new traffic is blocked, so a configuration change alone does not prove containment.
CISA’s ransomware guidance follows the same order. Isolate first, and power devices down only if they cannot be disconnected, accepting the loss of evidence in memory. Make that decision with your responders.
Continue addressing credentials in parallel. Isolating a laptop does not invalidate tokens an attacker has already copied from it.
Name an incident lead within the first five minutes. Agree in advance which actions that person can approve independently, including taking a production service offline if necessary. Containment should not depend on reaching an executive who is unavailable.
The lead also owns decisions about business impact, including which services are affected, what can be taken offline and for how long. Engineers investigate different parts of the incident and provide the facts the lead needs to make decisions. Record who is responsible for each task so nothing falls between teams.
If the attacker makes contact through a ransom note in a bucket or an email to an employee, nobody should reply or discuss payment. Preserve the message and send it to the incident lead. The first hour should focus on containment, evidence and establishing the scope of the incident.
Begin collecting audit records for the initial identity and every connected identity as soon as the response starts. Establish what they accessed, which actions succeeded, where requests originated, and when they occurred. Collect records from every region, including those your organization does not use, as attackers often use these regions.
Check early for signs that logging was disabled. Search for StopLogging, DeleteTrail, and UpdateTrail calls, as well as deleted GuardDuty detectors. Copy logs to an account or bucket the compromised identity cannot access.
Be precise about what the logs can establish. Coverage depends on configuration and retention. CloudTrail does not log data events by default, including S3 object reads. Without those records, you may be unable to determine which objects were downloaded. Missing records do not establish that no data was accessed, so keep confirmed findings separate from gaps in visibility.
Maintain a timestamped record of the response itself, documenting each action and the reason for it. Preserve suspicious files and system state before cleanup where possible, without delaying containment. This record supports the investigation and subsequent reporting to leadership, insurers, and regulators.
Call your incident response partner as soon as there is a credible sign of compromise, ideally within the first 15 minutes. Share what you know, what remains uncertain, and what you have already done, without waiting for confirmation.
Open a case with your cloud provider through the security escalation route available under your support plan. State “suspected compromise” in the first line and list the affected accounts and resources. Confirm who owns the investigation and what support is included. AWS has a dedicated Security Incident Response case process for customers using that service. A full forensic investigation should not be assumed to be included in every support plan.
Involve your legal team within the same hour. Some notification deadlines, such as GDPR’s 72 hours, begin when you become aware of a breach, rather than when the investigation ends.
Keep these contact numbers somewhere accessible without the compromised account. The scope will evolve as evidence emerges. Involving responders early gives you support with the decisions that cannot wait for a complete picture.
Contain the compromised identities and trace their access to other credentials. Isolate affected systems, keeping them running where possible. Name an incident lead, begin preserving logs, and contact your incident response partner. Start these activities in parallel, without waiting for a complete picture of the incident.
Usually not. An attacker may have retrieved other credentials, assumed another role, created new users or keys, or retained a session that requires separate revocation. Investigate everything the compromised account could access, along with any new access created since the compromise began.
Yes, if its permissions allow it to read credentials belonging to an administrator. AWS’s ReadOnlyAccess managed policy can download S3 objects and read Lambda environment variables, which often contain keys and passwords. It cannot read Secrets Manager secret values, so a “read-only” role that can do so has a custom policy that warrants close review.
Shutdown should not be the first step. Revoke the server’s credentials, remove it from its load balancer and Auto Scaling group, and isolate its network while keeping it running. This preserves evidence in memory while other instances continue serving traffic. Shut it down only if isolation is impossible and damage is still spreading.
Collect cloud audit logs, such as AWS CloudTrail records, for the compromised identity and every connected identity across all regions. Include sign-ins, API calls, role assumptions and resource access, along with available endpoint and application logs. Check for signs that logging was disabled and document any gaps in the available records.
In recent months, almost every incident my team has handled has started with a compromised identity.
.png)
.png)
