Widely used Trivy scanner compromised in ongoing supply-chain attack - Ars Technica

Overview

Widely used Trivy scanner compromised in ongoing supply-chain attackvar abtest_2146546 = new ABTest(2146546, 'click');

Admins: Sorry to say, but it’s likely a rotate-your-secrets kind of weekend.

Details

Hackers have compromised virtually all versions of Aqua Security’s widely used Trivy vulnerability scanner in an ongoing supply chain attack that could have wide-ranging consequences for developers and the organizations that use them.

Trivy maintainer Itay Shakury confirmed the compromise on Friday, following rumors and a thread, since deleted by the attackers, discussing the incident. The attack began in the early hours of Thursday. When it was done, the threat actor had used stolen credentials to force-push all but one of the trivy-action tags and seven setup-trivy tags to use malicious dependencies.

A forced push is a git command that overrides a default safety mechanism that protects against overwriting existing commits. Trivy is a vulnerability scanner that developers use to detect vulnerabilities and inadvertently hardcoded authentication secrets in pipelines for developing and deploying software updates. The scanner has 33,200 stars on Git Hub, a high rating that indicates it’s used widely.

“If you suspect you were running a compromised version, treat all pipeline secrets as compromised and rotate immediately,” Shakury wrote.

Security firms Socket and Wiz said that the malware, triggered in 75 compromised trivy-action tags, causes custom malware to thoroughly scour development pipelines, including developer machines, for Git Hub tokens, cloud credentials, SSH keys, Kubernetes tokens, and whatever other secrets may live there. Once found, the malware encrypts the data and sends it to an attacker-controlled server.

The end result, Socket said, is that any CI/CD pipeline using software that references compromised version tags executes code as soon as the Trivy scan is run. Spoofed version tags include the widely used @0.34.2, @0.33, and @0.18.0. Version @0.35.0 appears to be the only one unaffected.

“When the malicious binary is executed it starts both the legitimate trivy service and the malicious code in parallel,” Wiz researchers wrote. “In our initial analysis the malicious code exfiltrates secrets with a primary and backup mechanism. If it detects it is on a developer machine it additionally writes a base 64 encoded python dropper for persistence.”

The malicious process gathers environmental variables, sweeps the system for any credentials stored in the filesystem and enumerates the network interface. It then compresses and encrypts the data and attempts to exfiltrate it via post request to https://scan.aquasecurtiy[.]org. If that fails (returns a non-2xx status code), the malware then attempts to use a stolen GITHUB_TOKEN to create a repo tpcp-docs and post the data there.

The malicious process gathers environmental variables, sweeps the system for any credentials stored in the filesystem and enumerates the network interface. It then compresses and encrypts the data and attempts to exfiltrate it via post request to https://scan.aquasecurtiy[.]org. If that fails (returns a non-2xx status code), the malware then attempts to use a stolen GITHUB_TOKEN to create a repo tpcp-docs and post the data there.

Although the mass compromise began Thursday, it stems from a separate compromise last month of the Aqua Trivy VS Code extension for the Trivy scanner, Shakury said. In the incident, the attackers compromised a credential with write access to the Trivy Git Hub account. Shakury said maintainers rotated tokens and other secrets in response, but the process wasn’t fully “atomic,” meaning it didn’t thoroughly remove credential artifacts such as API keys, certificates, and passwords to ensure they couldn’t be used maliciously.

“This [failure] allowed the threat actor to perform authenticated operations, including force-updating tags, without needing to exploit Git Hub itself,” Socket researchers wrote. “While the exact credential used in this phase has not been publicly specified, the root cause is now understood to be residual access from the earlier credential compromise.”

The compromise technique is a departure from typical supply-chain attacks, which use stolen credentials to poison a repository with malicious code, usually by pushing a new commit. The newer method allowed the compromise to fly under the radar of many common defenses.

After getting access to Trivy’s credentials, the attacker compromised the aquasecurity/trivy-action Git Hub action but not by pushing to a branch or creating a new release, which would appear in the commit history and trigger notifications. Instead, the attacker force-pushed 75 existing version tags to point to new malicious commits. The technique involved multiple layers of deception that merit close examination.

After getting access to Trivy’s credentials, the attacker compromised the aquasecurity/trivy-action Git Hub action but not by pushing to a branch or creating a new release, which would appear in the commit history and trigger notifications. Instead, the attacker force-pushed 75 existing version tags to point to new malicious commits. The technique involved multiple layers of deception that merit close examination.

Git tags are pointers that refer to the SHA fingerprint of a commit. Git Hub Actions that reference such tags resolve them to the specified commit. By force-updating the tags from the legitimate commits to attacker-written malicious ones, any workflow that referenced them automatically began pulling the latter ones.

Socket said the precise process used by the threat attacker, which self-identified as Team PCP, was:

Started from the master HEAD tree (57a 97c 7e), the current file tree containing all latest code

Swapped entrypoint.sh with the infostealer payload, leaving everything else from master intact

Looked up the original commit that the tag previously pointed to (e.g., the PR #481 merge for tag 0.33.0)

Cloned that commit’s metadata, spanning author name, email, committer, both timestamps, and the full commit message including PR number and “Fixes” references

Set the parent to 57a 97c 7e (master HEAD) rather than the original parent

The threat actor, self-identifying as Team PCP, made imposter commits that were pushed to actions/checkout (while spoofing user rauchg) and to aquasecurity/trivy (while spoofing user Dmitriy Lewen. At 17:43:37 UTC, the Trivy repository’s tag was pushed, triggering a release. This resulted in a malicious checkout that fetched credential stealer code from a typosquatted domain (scan.aquasecurtiy[.]org, resolving to 45.148.10.212), and backdoored binaries being published to Git Hub Releases, Docker Hub, GHCR, and ECR. The maintainers have since removed these malicious artifacts. The attacker also compromised the service account and then abused their access push malicious workflows to traceeshark and trivy-action and steal additional credentials from Aqua (including GPG keys and credentials for Docker Hub, Twitter, and Slack). These secrets were exfiltrated to a Cloudflare Tunnel C2 (plug-tab-protective-relay.trycloudflare.com). Furthermore, 75 out of 76 trivy-action tags were force-pushed to malicious versions, and 7 tags were force-pushed as well.

The threat actor, self-identifying as Team PCP, made imposter commits that were pushed to actions/checkout (while spoofing user rauchg) and to aquasecurity/trivy (while spoofing user Dmitriy Lewen. At 17:43:37 UTC, the Trivy repository’s tag was pushed, triggering a release. This resulted in a malicious checkout that fetched credential stealer code from a typosquatted domain (scan.aquasecurtiy[.]org, resolving to 45.148.10.212), and backdoored binaries being published to Git Hub Releases, Docker Hub, GHCR, and ECR. The maintainers have since removed these malicious artifacts.

The attacker also compromised the service account and then abused their access push malicious workflows to traceeshark and trivy-action and steal additional credentials from Aqua (including GPG keys and credentials for Docker Hub, Twitter, and Slack). These secrets were exfiltrated to a Cloudflare Tunnel C2 (plug-tab-protective-relay.trycloudflare.com). Furthermore, 75 out of 76 trivy-action tags were force-pushed to malicious versions, and 7 tags were force-pushed as well.

So far, there are no known reports of breaches that have hit developers or organizations that used the compromised Trivy scanner. Given the app’s popularity, the thoroughness of the info-stealer, and the stealth of the operation, the potential fallout could be severe. All Trivy users should read the Socket and Wiz blog posts and follow the described defense steps.

1.          TCL’s German QLED ban puts pressure on TV brands to be more honest about QDs
   

2.          Google details new 24-hour process to sideload unverified Android apps
   

3.          After three months, Samsung is ending sales of the $2,899 Galaxy Z Tri Fold
   

4.          Coal plant forced to stay open due to emergency order isn't even running
   

5.          Rocket Report: Canada makes a major move, US Space Force says actually, let's be hasty
   

Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.

Key Takeaways

• Widely used Trivy scanner compromised in ongoing supply-chain attackvar abtest_2146546 = new ABTest(2146546, 'click');

• Admins: Sorry to say, but it’s likely a rotate-your-secrets kind of weekend

• Hackers have compromised virtually all versions of Aqua Security’s widely used Trivy vulnerability scanner in an ongoing supply chain attack that could have wide-ranging consequences for developers and the organizations that use them

• Trivy maintainer Itay Shakury confirmed the compromise on Friday, following rumors and a thread, since deleted by the attackers, discussing the incident

• A forced push is a git command that overrides a default safety mechanism that protects against overwriting existing commits