Backdoor found in widely used Linux utility breaks encrypted SSH connections

Researchers have found a malicious backdoor in a compression tool that made its way into widely used Linux distributions, including those from Red Hat and Debian.

The compression utility, known as xz Utils, introduced the malicious code in versions 5.6.0 and 5.6.1, according to Andres Freund, the developer who discovered it. There are no known reports of those versions being incorporated into any production releases for major Linux distributions, but both Red Hat and Debian reported that recently published beta releases used at least one of the backdoored versions—specifically, in Fedora Rawhide and Debian testing, unstable and experimental distributions. A stable release of Arch Linux is also affected. That distribution, however, isn't used in production systems.

Because the backdoor was discovered before the malicious versions of xz Utils were added to production versions of Linux, “it's not really affecting anyone in the real world,” Will Dormann, a senior vulnerability analyst at security firm Analygence, said in an online interview. “BUT that's only because it was discovered early due to bad actor sloppiness. Had it not been discovered, it would have been catastrophic to the world.”

Several people, including two Ars readers, reported that the multiple apps included in the HomeBrew package manager for macOS rely on the backdoored 5.6.1 version of xz Utils. HomeBrew has now rolled back the utility to version 5.4.6. Maintainers have more details available here.

Breaking SSH authentication

The first signs of the backdoor were introduced in a February 23 update that added obfuscated code, officials from Red Hat said in an email. An update the following day included a malicious install script that injected itself into functions used by sshd, the binary file that makes SSH work. The malicious code has resided only in the archived releases—known as tarballs—which are released upstream. So-called GIT code available in repositories aren’t affected, although they do contain second-stage artifacts allowing the injection during the build time. In the event the obfuscated code introduced on February 23 is present, the artifacts in the GIT version allow the backdoor to operate.

The malicious changes were submitted by JiaT75, one of the two main xz Utils developers with years of contributions to the project.

“Given the activity over several weeks, the committer is either directly involved or there was some quite severe compromise of their system,” Freund wrote. “Unfortunately the latter looks like the less likely explanation, given they communicated on various lists about the ‘fixes’” provided in recent updates. Those updates and fixes can be found here, here, here, and here.

On Thursday, someone using the developer's name took to a developer site for Ubuntu to ask that the backdoored version 5.6.1 be incorporated into production versions because it fixed bugs that caused a tool known as Valgrind to malfunction.

“This could break build scripts and test pipelines that expect specific output from Valgrind in order to pass,” the person warned, from an account that was created the same day.

One of maintainers for Fedora said Friday that the same developer approached them in recent weeks to ask that Fedora 40, a beta release, incorporate one of the backdoored utility versions.

“We even worked with him to fix the valgrind issue (which it turns out now was caused by the backdoor he had added),” the Ubuntu maintainer said. "He has been part of the xz project for two years, adding all sorts of binary test files, and with this level of sophistication, we would be suspicious of even older versions of xz until proven otherwise."

Maintainers for xz Utils didn’t immediately respond to emails asking questions.

The malicious versions, researchers said, intentionally interfere with authentication performed by SSH, a commonly used protocol for connecting remotely to systems. SSH provides robust encryption to ensure that only authorized parties connect to a remote system. The backdoor is designed to allow a malicious actor to break the authentication and, from there, gain unauthorized access to the entire system. The backdoor works by injecting code during a key phase of the login process.

“I have not yet analyzed precisely what is being checked for in the injected code, to allow unauthorized access,” Freund wrote. “Since this is running in a pre-authentication context, it seems likely to allow some form of access or other form of remote code execution.”

In some cases, the backdoor has been unable to work as intended. The build environment on Fedora 40, for example, contains incompatibilities that prevent the injection from correctly occurring. Fedora 40 has now reverted to the 5.4.x versions of xz Utils.

Xz Utils is available for most if not all Linux distributions, but not all of them include it by default. Anyone using Linux should check with their distributor immediately to determine if their system is affected. Freund provided a script for detecting if an SSH system is vulnerable.