The Linux operating system, which helps run millions of servers worldwide is revered for its robustness and open-source nature but has recently faced a critical security challenge when a concealed backdoor was discovered within the xz compression utility, a tool widely employed across numerous Linux distributions. This revelation has prompted strong reactions from developers and a significant examination of the security measures and review processes within the Linux community.
The Discovery
The discovery of the xz backdoor was made by a Microsoft software engineer whose keen observations and tech intuition led to its unveiling.
While engaged in micro-benchmarking to optimize performance, he observed an unusual lag in Secure Shell Daemon (sshd) processes, consuming a significant amount of CPU despite their failure due to incorrect usernames. Delving deeper, he profiled sshd and found an inexplicable CPU time linked to the liblzma library, with performance profiling tools unable to pinpoint any specific symbol causing the lag.
This anomaly in system behavior paired with a previous odd valgrind complaint during postgres testing following an update raised red flags. This combination of minute observations, which may have gone unnoticed in isolation, painted a bigger picture that something was amiss. In this case, the engineer’s acumen earned him accolades akin to being dubbed the ‘Silverback Gorilla of nerds,’ standing as a final boss in the realm of internet security.
The exploit cleverly disguised as a two-year maintainership, allowed malicious actors to manipulate the Secure Shell Daemon (sshd), potentially granting them admin-level access to a myriad of systems. The security breach has sparked a wildfire in the Linux ecosystem. The incident involves a backdoor in the xz utility, typically used for file compression, which had been introduced subtly over two years of consistent contributions by a maintainer, Jia Tan. The backdoor’s discovery raises alarms about the vulnerability of open-source software and the sophisticated methods used to infiltrate such systems.
The malicious code, which was cleverly disguised and merged into the xz codebase through an apparently benign commit, only activated under specific conditions during the Debian and Red Hat Package Manager (RPM) builds, particularly modifying the SSH daemon (sshd) in memory. This triggered questions about the depth and breadth of the intrusion, leading the community to scrutinize Jia Tan’s 449 commits for other potential threats.
“This Linux hack is like the equivalent of slipping a hole into a Windows update… Microsoft doesn’t catch it, and it gets redistributed worldwide… then a hacker can access any Windows machine they want.”
Nikos Dimou (Cybersecurity expert)
The open-source community initially welcomed Tan’s contributions, which included creating tests and fixing issues like the Landlock sandboxing syntax error. However, the latter fix turned out to be part of the backdoor mechanism, and the pull request’s acceptance without thorough scrutiny has raised concerns about the review processes for open-source contributions.
One of the many open questions involves the decision to use indirect functions (ifunc) within the payload. This technique is speculated to be less detectable at runtime compared to Procedure Linkage Table (PLT) hooking but has raised eyebrows for its potential to cause more significant disruption.
92.4% of the world’s top 1 million servers run on Linux
Gitnux.org
The payload’s exact actions remain partially shrouded in mystery, particularly its pace of execution. The injected sshd code was set to run slower than 600 milliseconds, which may have been a deliberate move to evade detection by not causing noticeable delays.
Further inquiries pertain to the RSA_public_decrypt function’s unusual slowness and whether the payload’s activities extend beyond sshd into other programs. The community is left with a daunting task: to uncover the full extent of the backdoor’s capabilities and the true intention behind this sophisticated infiltration.
Cybersecurity Expert Reaction
Nikos Dimou, a Greek cybersecurity expert, sheds light on a significant breach in the Linux ecosystem involving the widely used compression tool, xz. According to Dimou, the vulnerability lay hidden within the code pushed to GitHub. While the code did not trigger automatic checks, it was incorporated into the build process, meaning the final products distributed online contained the malicious code. This backdoor could have potentially opened up millions of systems to hackers, allowing them to gain admin access to millions of Linux servers.
Dimou likens the breach to a hypothetical scenario where a Windows update contains a flaw that Microsoft fails to detect, which is then redistributed globally, granting hackers entry and access to any Windows system they target. However, the actual situation pertains to Linux servers and has been described by Dimou as extremely grave.
The discovery of the breach caused an uproar in the open-source community, leading to the temporary suspension of xz’s GitHub repository to investigate the issue. Major players in the industry, such as Red Hat and Canonical, have responded by releasing CVEs (Common Vulnerabilities and Exposures) to address the security flaw. Dimou’s insights highlight the critical nature of vigilance and timely detection in the sphere of cybersecurity.
Plotted Malice
Other cybersecurity experts have zeroed in on a sophisticated backdoor planted in the xz compression tool. This backdoor has turned heads not only for its complexity but for the methodical approach taken by the alleged perpetrator, Jia Tan, who maintained a two-year façade of innocuous contributions.
Security analysts like Alden have pointed out a peculiar pattern in Tan’s commit history. A cluster of offending commits seems to deviate from his usual activity schedule, suggesting that the malefactor might have been working in a different timezone, possibly hinting at the true origin of the threat.
Ryan Dahl, a key figure in the open-source community, expressed his unease, questioning whether future open-source software contributions might necessitate background checks. He also raised concerns about the role of autoconf in potentially compounding these security issues within a Rust code base.
In hacker forums, conversations have escalated. One user, rwjm, recounts the backdoor author’s engagement with the Fedora project to address a “valgrind issue,” which now appears to have been a ruse connected to the backdoor itself. This individual’s involvement with xz, characterized by intricate binary test files, has led many to question the security of even older versions of the tool.
Another community member, move-on-by, reflects on a recent SECURITY.md file addition by the author to the xz-java project, now seen in a more sinister light. This move could be interpreted as a calculated strategy to tweak exploits without drawing attention.
This breach serves as a sobering reminder of the constant vigilance required in open-source software maintenance. The incident highlights the need for more rigorous security practices and reviews, especially for software that forms the backbone of countless systems worldwide.
Author Profile
- Ex-community moderator of the Banano memecoin. I have since been involved with numerous cryptocurrencies, NFT projects and DeFi organizations. I write about crypto mainly.
Latest entries
- December 9, 2024Stock MarketMaster the Time Value of Money Financial Concept
- November 18, 2024Stock MarketFinancial Ratios Guide to Measuring Business Performance
- November 11, 2024NewsWireLabour’s UK Budget: A Fiscal Smirk of Contempt for Working People
- October 25, 2024CryptoThe DAO Governance Battle Between Corporations & Blockchain Rebels