John Mark Walker recently published a post on how the Linux distribution has lost prominence in the DevOps ecosystem (right around the time we started using the term “DevOps”) and how it might be poised for a comeback. His take is centered around the idea of the “download your dependencies on the fly as you build and deploy software” being a risky proposition, supply-chain-securitally-speaking.
I can’t disagree with that. I’ve always felt that modern language ecosystems are bonkers in that regard. Yes, it’s great that there are better dependency relationships than in C/C++ where the relationships might as well be “LOL, GFY”. But the idea that you’d just download some libraries on the fly and hope it all works as expected? Well, we’ve learned a few times how that can go sideways.
So, yeah. I generally agree with what John Mark says here. Heck, I’ve even given a talk on several occasions with a premise of “operating systems are boring.” So can the Linux distribution become un-boring again and help us fix our woeful supply chains?
Why distros aren’t the answer
The fundamental problem that Linux distributions face is sometimes called “too fast, too slow.” Distributions prioritize stability and coherence to some degree, even the cutting edge distributions. Enterprise distributions — or more accurately, their customers — place a particular emphasis on long-term stability. But development tools and programming languages change quickly.
Some applications need the latest version of libraries. Others update slowly and need old versions. It’s hard to meet both of those needs at the same time. Various attempts have been made, like Fedora’s Modularity, but there’s no great answer. The modern language ecosystems exist, in part, to sidestep this problem.
Another problem is that not all distribution package maintainers are technically solid or even ethical. The vast majority are, of course, but there are exceptions. As a package maintainer myself, I’m not adding a ton of value from a supply chain security standpoint. I take the upstream sources, wrap them in an RPM spec file, and submit them to the build system.
Distributions also create a disconnect between developers and end users. This separation has some value, but it also creates additional points for slow bug fixes, malware injection, and social engineering attacks.
Why distros can be the answer
While it’s true that not all distribution package maintainers are capable of fixing issues in the packages they maintain, some can. And even when a maintainer can’t fix an issue, they can get help from other maintainers in the distribution’s community. Issues that upstream chooses not to fix can be fixed in the distribution’s build.
Distributions aren’t just RPMs or debs or whatever anymore. Contained application delivery methods like Flatpak, Snap, and toolbx give distributions the ability to provide curated environments in a way that improves parallel installability. Of course, upstreams can produce their own containers, but this gives end users the ability to pick the right option for their needs.
In conclusion
I’m not sure I see distributions making the comeback that John Mark hopes for. Enterprise distros move too slowly, by and large, to address the needs of language ecosystems. And there’s not a ton of value in distributions blindly packaging language libraries.
John Mark suggested that organizations would want to “outsource risk mitigation to a curated distribution as much as possible.” I don’t disagree. But community distributions can’t take on the risk that companies want to outsource.
That said, the problem won’t fix itself, so let’s work toward something. If that ends up being Linux distributions, then great. If not, distributions will still have an important role to play.