CentOS is NOT dead. Please Stop Saying It Is (at least until you read this)
Note: I work for Red Hat, but I work on OpenShift stuff and am NOT a part of RHEL or CentOS in a professional capacity. I am a community member, and have been for almost 10 years now, much longer than I’ve worked at Red Hat. I have run CentOS in personal production for years and rely on it. Everything I am about to say is my opinion alone, and is not an official statement from Red Hat in any way. As employees we are allowed (and encouraged!) to participate in the community, and that is what I am doing with this blog post. This post will not make public anything internal.
Since the news broke Tuesday about the changing role of CentOS (aka the death of CentOS) my inbox and chat have exploded with questions. Until now I’ve said very little because I was still processing it myself. I make great effort to think rationally and logically, eschewing the type of emotional thinking and knee jerking that pervades much of our society today. I think I’m ready to talk.
My initial reaction to the news was anger, and a feeling that Red Hat was doing something evil to the community, which is a distinctly un-Red Hat thing to do. Many others inside and outside of Red Hat had the same reaction as well, and we had some hard conversations and debates about what was going on. I grieved the loss of CentOS, which seemed pretty much dead given what was said. It seemed obvious (via Occam’s Razor) that CentOS had cannibalized RHEL sales for the last time and was being put out to die. Statements like:
If you are using CentOS Linux 8 in a production environment, and are
concerned that CentOS Stream will not meet your needs, we encourage you
to contact Red Hat about options.
That line sure seemed like horrific marketing speak for “call our sales people and open your wallet if you use CentOS in prod.” (cue evil mustache-stroking capitalist villain).
Thankfully, my (and many other’s) first reaction was not correct. I’m not saying the original announcement was wrong, but given that the vast majority of people interpreted it the same way that I did, I think it’s fair to say it was incredibly misleading and used phrasing and descriptions that have multiple meanings which can be easily taken in a way different than the writer intended.
So what is the truth? What’s actually going on with CentOS? In this post I will strive to answer that question. I’ll also tell you where I think you should go if you really just want “the old CentOS.” (That said, you really should re-evaluate whether the old CentOS actually met your needs, or if it was just the closest thing at the time like it was for me, which is why you adopted it in the first place).
A saying I love is, “don’t piss on my leg and tell me it’s raining.” I will not do that here. This is my real, unfiltered opinion, void of marketing speak and PR. If Red Hat fired me tomorrow I’d feel the same way.
As this post will undoubtedly go very long, I’m going to break this up into some sections. The first section will attempt a TL:DR; that will state some facts and opinions without providing any argument. If you want to see my arguments (which I hope you do), then please read the relevant section of the post. If you prefer a FAQ format, please let me know. I’m considering making a “FAQ” version of this post but I don’t want to unless it would be helpful (I’m donating my Saturday morning already writing this post. I do it because I care deeply about this community and seeing it fly apart over misunderstanding is a tragedy).
This post is the post that I wanted to read on Tuesday as a community member. I was wondering what is going on, what’s in the future, and if I’m going to have a horrible migration in a few months. News of this magnitude needs long form explanation, and that is my goal here.
Main points (each have a matching section below):
- Red Hat really botched the messaging behind this
- CentOS is NOT dead. In fact it’s just now starting to reach it’s full potential!
- CentOS will no longer be downstream of RHEL as it was previously. CentOS will now be upstream of the next RHEL minor release. This is NOT a “RHEL Beta” and it doesn’t mean that CentOS is the same as Fedora now (that’s laughable). Fedora is the upstream of the next RHEL major release (which if you’ve spent any time in this eco-system you know that is a significant difference :-P)
- CentOS will no longer be old, crusty, and barely alive, trailing RHEL by months at times
- CentOS will now play a valuable role in the ecosystem, which means more Red Hat resources will be allocated to it
- What does a healthy community look like? please read that section.
Some questions you may have:
- Is everything roses then? Is there any downside to this?
- Where do I migrate to if I’m not convinced? Keep an eye on the Rocky Linux project. More info below
- Did IBM do this?
- Why such a short window for CentOS 8?
Note: Apologies for the lack of anchor links. Medium doesn’t seem to be able to handle links to anchors in the same document (such as html and markdown support), so I had to remove them. Each item above corresponds to a section below. I’ll try to figure out how to add them back in.
Red Hat really botched the messaging behind this
A big part of the reason we are having this conversation right now is the way the news was messaged. To be frank, Red Hat did a very poor job of this and (in what I believe was an attempt to be straight forward) unintentionally misled people about the severity of the changes (hint: it’s not actually that big of a deal in my opinion, explained more below).
This was not done intentionally. It is also water under the bridge. It has been acknowledged and at this point I agree with others that we need to move forward. I will not dwell on it anymore.
CentOS is NOT dead. In fact it’s just now starting to reach it’s full potential!
If you have been using CentOS as long as I have, you know it can take a while for RHEL updates to make it to CentOS. Most of us have always seen this as “a cost of free” and do not complain.
Nevertheless at times it can be painful, especially when a new major release is coming that we are all excited for. The RHEL beta is great but it sometimes looks very different than the end release as a result of the way changes flow.
Both of these problems are now much improved thanks to this change! CentOS has long been at risk of languishing on the vine, especially because it doesn’t provide a whole lot tangible/measurable/observable value to it’s upstream (I often make the case that CentOS does provide a lot of value, such as familiarizing people with the eco-system and “Red Hat Way,” but I concede those are nearly impossible to measure). That isn’t an issue anymore (see below for more on this).
CentOS will no longer be downstream of RHEL as it was previously
This is important to understand if you are concerned about stability (which most of us CentOS users are, hence why we use CentOS).
In the spectrum of changes/speed, it used to look something like this:
Now it looks like this:
If it seems like RHEL and CentOS just traded places, that’s because that’s essentially what happened!
If you are someone who is thinking, “CentOS is now just the RHEL beta,” please ask yourself, did you use to consider RHEL to be the CentOS beta? If not, you shouldn’t be thinking that now about CentOS. The only thing that has changed here is the order. CentOS is now upstream of RHEL instead of downstream. The way software makes it in is the same. It just hits CentOS first instead of RHEL first.
Let’s talk about the updates that CentOS gets.
If you have some time, I would highly recommend reading the blog post “How RHEL is Made.” It is immensely helpful when trying to understand how CentOS relates to RHEL now.
Previously, when updates were slated for RHEL they would go straight there. If you think of RHEL as a “solid, stable, enterprise distro” well, so does Red Hat. I like to think Red Hat knows a thing or two about running a stable Linux distro, especially considering the support resources they put behind it with guarantees. The value of the brand is built on stability and support puts Red Hat’s money where their mouth is, so to speak.
Instead of updates hiding out for a quarter and then suddenly showing up publicly, now they will be done in the open. I think this is great! Having opaque RHEL releases drop where you don’t know what’s coming until it’s here has been an unfortunate way of running things. That’s over now!
This is important too: Before stuff goes to CentOS Stream it has passed RHEL QA and CI. It has also typically been in Fedora for months or years at this point, and is battle tested.
This software would have gone straight to RHEL before. It is NOT unstable if it would have gone straight to RHEL! This is why I say CentOS Stream is not a RHEL beta! It is where production RHEL used to be, and nobody considered production RHEL to be “a beta.”
If you would have trusted RHEL previously, you should trust CentOS Stream. Bugs can and do happen. They currently happen in RHEL, CentOS, and everywhere else because that’s the nature of software. Usually they are minor by the time they make it to CentOS/RHEL and then they get fixed promptly. That will continue to be true for CentOS.
Lastly, let’s look at what the CentOS blog says about the stability goals for CentOS:
CentOS Stream intends to be as stable as RHEL
If you want to learn more about exactly how the upstream flows and where Fedora, CentOS, and RHEL all fit in with each other, there’s a very helpful graphic on the CentOS blog, along with a detailed explanation.
CentOS will no longer be old, crusty, and barely alive, trailing RHEL by months at times
Somewhat continued from the previous point, if RHEL is the gold standard of stability (which many would suggest it is) then why would CentOS Stream, a distro effectively taking its place in the line-up, be less stable?
You could make an argument that RHEL is become “more stable” and CentOS is becoming “less stable” by trading places, sure. But I have never, ever heard complaints that RHEL is “unstable” and not worthy of running in production!
One of my complaints about CentOS historically is that it is so far behind RHEL, which itself can be pretty stale. This is becoming less and less an issue as I migrate my workloads to containers, but it’s still a consideration. In this respect CentOS is getting an upgrade!
CentOS will now play a valuable role in the ecosystem, which means more Red Hat resources will be allocated to it
Let’s talk about the elephant in the room: Red Hat is a for-profit company that needs to make money to stay in business. While Red Hat does many, many things that are charitable, at the end of the day it has be profitable or the company goes away, taking all the valuable community contributions (that most of us get and use for free) away along with it. Forks would likely happen, but the expertise and labor required for such a project is immense. I would be dubious about a claim that the open source world could replace the thousands upon thousands of expert man hours invested each month by Red Hat.
The future of CentOS, and RHEL are more certain this way. If you think you don’t care about RHEL, remember that without RHEL there is no CentOS.
It’s no secret that CentOS competes with RHEL. I’ve personally heard CTOs tell Red Hat salespeople, “why should I buy RHEL when I can use CentOS for free?” I die inside when I hear that. It is a fair and good question, but asking it tends to fire up a salesperson and gives them direct financial reasons to hate on CentOS.
This fixes that problem somewhat by aligning incentives better. CentOS is no longer viewed as a direct equivalent to RHEL. This addresses a very real problem that Red Hat (the company) has faced. This change (hopefully) turns CentOS into an asset rather than a liability, which is very good for the long term success of it all. It does so in a way that still maintains a lot of value in CentOS both for Red Hat and for the community.
If you benefit from Fedora, CentOS, or any other projects, you should care very much about the success of RHEL. Without it, you wouldn’t have those things. If you can afford to pay for RHEL, please consider doing so. Red Hat support is amazing (I encounter them all the time) and the first time you need them you’ve more than paid for the sub. There are plenty of benefits that come with a sub besides just a license. The training that Red Hat offers is incredible and I can’t recommend it highly enough. If you want to learn more about the value of a Red Hat subscription feel free to reach out to Red Hat (I’m positive a sales person would be happy to explain it to you :-)
What does a healthy community look like?
A healthy community is not just a few producing and the many consuming and never giving back. If you aren’t giving back in some way, consider that you might be a taker. In open source, especially when you are starting, it’s ok to be a taker! But ideally as you gain experience and expertise, you would be helping others with that. At the very least, if you’re a taker and simultaneously trashing Red Hat for murdering CentOS (a distro you feel entitled to) then maybe you should reconsider what you’re doing.
If you enthusiastically jumped to help out with one of the many projects springing up to replace “old CentOS,” (such as Rocky Linux) it’s worth asking yourself, “Did you ever offer to help with CentOS before? If not, why not?” If you are mad about the new direction of CentOS but you never helped out before, you might should direct some of your anger at yourself. It is supposed to be a community project, not just a Red Hat one. Red Hat has sure done an awful lot (and given it away for free-as-in-beer and free-as-in-speech). What have you done?
Is everything roses then? Is there any downside to this?
No, I don’t think it’s all roses by any stretch. If Red Hat were to get lax with quality as a result of having CentOS as a buffer now, then the stability of CentOS could be at stake. This is a legitimate concern.
I’m not terribly worried however because (especially given this change) the CentOS brand is important to Red Hat. If CentOS looks bad, so does RHEL. Also many RHEL customers also run CentOS, and while it’s conceivable that Red Hat could go evil and use that to push customers to RHEL, this would no doubt backfire.
Also I look to Fedora, which has pretty good QA and CI automation in place. Fedora gets things way, way sooner than CentOS and RHEL will, and Fedora is remarkably stable. I’m not worried personally about instability in CentOS Stream.
But if I were worried about it, Rocky Linux (or something else not yet emerged) will be there as a fully compatible alternative to RHEL. Not only will this give users options, but competition is healthy and leads to quality.
If Red Hat decided to stop publishing their sources publicly (which they would be in their rights to do) then that would kill a project like Rocky, but given they have published those sources for decades I am not worried about that. I’ll throw my arms up in outrage if that time ever comes.
Where do I migrate to if I’m not convinced?
If you aren’t convinced or if you still really just want old CentOS, I encourage you to contact Red Hat about options. Just kidding! Sorry I couldn’t resist that joke.
I’m watching Rocky Linux closely myself. While I plan to use CentOS for the vast majority of my needs, Rocky Linux may have a place in my life as well, as an example powering my home router. Generally speaking, I want my router to be as boring as absolute possible. That said even that may not stay true forever, if for example CentOS gets good WireGuard support.
Lastly, but certainly not least, Red Hat has talked about upcoming low/no-cost RHEL options. Keep an eye out for those! I have no idea the details, but if you currently use CentOS for personal use, I am optimistic that there may be a way to get RHEL for free coming soon. Again, this is just my speculation (I have zero knowledge of this beyond what has been shared publicly), but I’m personally excited.
Did IBM do this?
No. IBM did not do this. The CentOS governing board, some of which work for Red Hat, did this.
Why such a short window for CentOS 8?
I agree, this part is bad and it shouldn’t be. In my opinion, (which isn’t worth much), CentOS 8 should have been supported at least as long as CentOS 7, if not to meet the original expectation.
Please, remember that Red Hat has done a lot for all of us in the community. While Red Hat is a for-profit company, they make a lot of decisions that are self-harming in the interest of benefiting others (examples below). I can’t think of any company that does more give-back than Red Hat.
What other company would have acquired and subsequently fully open sourced Ansible? Who would have invested engineering time and effort into making one of their most important products (OpenShift) available and usable by open source users (I’m referring to OKD?
Who else would voluntarily publish their sources to the public even when companies like Oracle and Amazon take them and build competing products with them? Red Hat does not have to do that, but they choose to because they are good.
If there is anybody that deserves benefit of the doubt and love and understanding, it is Red Hat in my opinion. I will continue to give that benefit of the doubt, and I think you should too.