'CloudImposer' Flaw in Google Cloud Affected Millions of Servers

Google has patched a flaw in its Google Cloud Platform (GCP) that attackers could have exploited to execute a supply chain attack on millions of customer cloud servers, simply by deploying a single malicious code package.

Researchers from Tenable discovered the remote code execution (RCE) vulnerability, dubbed "CloudImposer," that attackers could have used to hijack an internal software dependency affecting GCP services, they revealed in analysis published Sept. 16.

Specifically, the flaw was found in GCP's Cloud Composer service for orchestrating software pipelines, but it also affected the Google services App Engine and Cloud Function. The flaw created a scenario called a dependency confusion, a technique discovered several years ago but widely misunderstood even by cloud platform providers, according to Tenable.

A dependency confusion attack, first discovered by security researcher Alex Birsan in 2021, starts when an attacker creates a malicious software package, gives it the same name as a legitimate internal package, and publishes it to a public repository.

"When a developer's system or build process mistakenly pulls the malicious package instead of the intended internal one, the attacker gains access to the system," Tenable senior security researcher Liv Matan explained in the analysis. "This attack exploits the trust developers place in package management systems and can lead to unauthorized code execution or data breaches."

He added: "There’s a surprising and concerning lack of awareness about it and about how to prevent [dependency confusion], even among leading tech vendors like Google. And unfortunately, this type of dependency can be exploited to execute supply chain attacks in the cloud that "are exponentially more harmful than on-premises."

"For example, one malicious package in a cloud service can be deployed to — and harm — millions of users," Matan observed. In essence, then, one single faulty command in GCP could potentially have created a ripple affect across myriad cloud deployments, giving attackers access to customers' enterprise cloud environments.

Tenable's findings were first presented in a session by Matan at Black Hat USA in August called "The GCP Jenga Tower: Hacking Millions of Google's Servers With a Single Package (and More)," — one a Dark Reading expert advised not to miss at the conference. However, he published his full analysis on Tenable's blog only this week.

Risky Documentation Leads to Flaw

The first sign of the flaw was Google documentation regarding GCP and the Python Software Foundation that introduced the possibility of dependency confusion in cloud deployments, according to Tenable. The researchers dug further and found that Google itself applied the same risky implementation advice to GCP, introducing the flaw.

Specifically, Google advised users who want to use private Python packages in the GCP services App Engine, Cloud Function and Cloud Composer services to use what's called the "--extra-index-url" argument.

"This argument looks for the public registry (PyPI) in addition to the specified private registry from which the application or user intends to install the private dependency," Matan explained. "This behavior opens the door for attackers to carry out a dependency confusion attack."

The researchers inferred that there are "numerous GCP customers" who followed Google's risky guidance, as well as ultimately discovered that Google itself took its own advice when installing private packages in their own internal services.

Specifically, Tenable researchers found that Google used the risky --extra-index-url argument to install a private code package missing from the public registry in a way "that allows attackers to upload a malicious package to the public registry, and take over the pipeline," Matan wrote.

Google Fix & Other Mitigations

The researchers responsibly disclosed both the documentation and the CloudImposer RCE vulnerability to Google, which promptly responded and took action, according to Tenable. Specifically, Google fixed the vulnerable script in Google Cloud Composer that was utilizing the --extra-index-url argument when installing a private package from a private registry.

The company also inspected the checksum of vulnerable package instances and notified Tenable that, as far as Google knows, there is no evidence that the CloudImposer was ever exploited, Matan noted.

Google also acknowledged that while the exploit code that Tenable developed ran in Google's internal servers, it's likely that it would not have run in customers' environments because it wouldn't pass the integration tests.

Further, the company fixed the risky documentation, now recommending that GCP customers use the --index-url argument instead of the --extra-index-url argument, and the tech giant has adopted Tenable's suggestion to recommend that GCP customers use the GCP Artifact Registry's virtual repository to safely control the Python package manager search order, Matan noted.

GCP customers should analyze their environments for their package installation process to prevent breaches, specifically searching for the use of the --extra-index-url argument in Python to ensure they are not vulnerable to a dependency confusion attack.

Matan concluded: "A combination of responsible security practices by both cloud providers and cloud customers can mitigate many risks associated with cloud supply chain attacks."