While talking to many of our customers, prospects, and partners who have established cloud strategies, a common pattern is evolving. The majority of them have an issue where an IAM Access Key is compromised, resulting in their cloud infrastructure getting misused for Crypto Mining activities. That’s scary, considering that you end up paying exponential cloud costs for the period until the breach gets detected.
What is Crypto Mining? Why is it a threat?
“Cryptocurrency mining (crypto mining) uses the processing power of computers to solve complex mathematical problems and verify digital transactions, and the miners get rewarded with a small amount of cyber currency.”
Source : cyber.gov.au
Illegal Cryptomining = Cryptojacking
Cryptojacking is a crime and also a straightforward way for hackers to make money in the public cloud; because all they have to do is find a way into a customers cloud, spin up a lot of high capacity CPU rich servers for mining, and keep making money until the hapless customer detects the breach (mostly after they get an alarming bill from the cloud provider).
“Many recent surveys point out that 99% of misconfigurations in the cloud go unnoticed.“
These commonly prevalent misconfigurations are also why the public cloud is a gold mine for Crypto miners. Utilizing those misconfigurations, miners have unlimited CPU and computing power at their disposal.
Anatomy of a cloud credential breach
We recently interviewed a customer who went through a Service Account Key breach in Google Cloud, resulting in Crypto Mining activities.
We will break down this incident into multiple steps to understand better how attackers can leverage the leaked credentials and, most importantly, how organizations can be better prepared to tackle this scenario (when it happens)
The Incident
Cloud Provider
An employee accidentally saved a GCP Service Account Username/Key into his personal GIT Repo.
Shortly thereafter, they got an email from Google stating that a service account compromise had been detected. It gave the following details in the email ( a lifesaver service, we must say – because it dramatically reduces the detection time).
The email clearly highlighted the service account in question and the URL of the public Github repo. It also recommended two steps, including decommissioning the service account, which is a must-do.
But REMEMBER, just like the good guys(cloud service providers), the BAD guys are also continuously scanning GIT repos for any code commits that expose an IAM Access Key. The moment you make a mistake, be assured that someone will definitely exploit it in seconds.
Investigation
The customer contacted the employee in question and the personal GITHub Repo got quarantined. Further investigation in the Git Repo revealed no immediate Indicators Of Compromise (IOC). Thankfully one of the cloud administrators started looking at the GCP Projects that the Service Account had access to. Lo-behold, there were many high-capacity servers launched in every region in that project.
Left unchecked, these VMs could have led to tens of thousands of dollars in cloud costs.
This is what the miners target; the probability that most organizations would not have the means or mechanisms to detect and react to these intrusions.
It is money left on the table.
If this happens to me/my-organization, what should we do?
All Cloud Providers recommend two immediate steps that organizations should do following an incident like this (refer to pic above).
Below are the two steps
- Revoke all (or listed) credentials for compromised Service Accounts.
- Delete all unauthorized VMs or resources
Step 1
Step #1 is a safety measure to minimize further damage to your cloud infrastructure. Revoking all the credentials prevents the hacker/bot from spinning up unauthorized resources or inflicting further damage.
Step #2 is DIFFICULT – Why, you ask ?
There are multiple factors. The dynamicity of the Public Cloud makes it very difficult to have a baselined and hardened environment.
There are multiple regions associated with every cloud provider. A leaked service account may have permissions to create infrastructure/resources in ALL of those regions. In the web console view of a cloud provider, there is always a regional view of your resources. Hence you would have to switch between regions to identify if there are new compute resources or billable resources spun up in any region.
What if that compromised service account had cross-account permissions? A nightmare scenario if you are part of the cloud administration team. It is not easy when you have hundreds or thousands of accounts/projects.
How do we triage and contain risk?
First, ensure you REVOKE all (or listed) credentials for the compromised Service Account.
Next, instead of panicking, use the Cloud Service Providers IAM Logs to understand the activities performed by the compromised service account.
If you are using GCP (as in this example), check the Stackdriver IAM Audit logs. Filter by the Service Account in question and look for all recent activities. The subsequent filtered list will show newly launched EC2 instances (if any) or other resource creation/deletion/modification activities by that Service Account.
If you identify unauthorized launches, shut them down immediately and inform the cloud provider by raising a ticket. The cloud provider will typically waive any costs incurred via intrusions like these if they are disclosed in a timely basis.
Can it get complicated?
Absolutely – if you don’t have centralized logging for all of your cloud accounts. It then becomes a nightmare for the cloud administrator to wade through all the log files of different cloud accounts and detect compromises.
In the case of AWS, it can get quite messy since attackers typically use Role Chaining to move through your cloud laterally. Detecting and triaging role chaining activities in your cloud accounts and narrowing down to the root event is complex and time consuming.
In this article, we cited the example of “cryptojacking” and how Crypto miners leverage common cloud misconfigurations and misuse public cloud compute resources.
However, a service account breach is in no way limited to Cryptojacking activities. Once an intruder is in – they can inflict catastrophic damages to your cloud infrastructure. They’ll hunt for your crown jewel data, and exfiltrate it or do something as bad as bringing down your running production servers if that Service Account have been granted Admin privileges.
Protecting your cloud from Cryptojacking
We recommend the following steps to protect your cloud from cryptojacking activities.
- Secure your cloud credentials.
- Train your employees on following best practices in the cloud.
- Have a documented incident response mechanism in place to swiftly react to accidental credentials exposures by employees.
- Follow the principle of least privilege while creating human/non-human roles and identities in the cloud. This is your topmost threat vector in the cloud.
- In the above example - if the service account didn't have permission to create Virtual Machines, the breach in itself is low risk - because the exploitability factor is very low.
- Constantly review and rightsize the permissions associated with your cloud identities. If there are unused permissions, remove them.
- Centralize your IAM Logs. Your administrators need to have a unified view across events happening in your cloud portfolio (single or multi-cloud).
- The ability to audit and do forensics on time across all cloud identities is critical. Streamline and store all your logs to a common platform to make this easy.
How can C3M help mitigate your cloud risks?
There are four key areas where C3M can help
Visibility –
- Our dashboards give you visibility into every user/role/service-account/group that exists across your cloud and the compliance status.
- An IAM Activity Dashboard gives you a birds-eye view of all the sensitive events that happen across your cloud infrastructure.
- Interactive visualizations with the ability to ask questions like
- Who can assume certain roles in my account?
- Can I reach a cloud entity via a certain role?
- What roles have access to certain cloud applications? and many more...
Security Governance
- Real-time detection and alerting for all cloud IAM Activities.
- An Event Log screen to view, audit, report, and do forensics on all identities across ALL Clouds for a given time range
- Monitor and Govern Privileged Users/Service Accounts
- Anomaly Detection for Cloud Identities including detection of Crypto Mining activities
- Risk Score for All Identities based on CVSS
Reporting
- Ability to generate and download various activity and IAM reports
Automation
- Ability to configure real-time remediation via pre-defined or custom playbooks.
The public cloud is designed to be inherently secure. Just configure it securely. Have questions around how to configure your cloud correctly? – Please reach out to info@c3m.io and one of our Cloud Security SMEs will conduct a FREE 30 min session with your team. No commitment required.
