Redhat has really been grating my nerves as of late. Undocumented features that are advertised but unusable, recommended updates that are thoroughly broken, failure to update packages with high-impact problems, and a poor software packaging and distribution policy have me testing other distributions with a goal of taking my business elsewhere.
This heightened level of aggravation with Redhat started with their split from the community version. It should be noted that I think this was a smart business decision for Redhat. There was no way they could continue to have a single distribution that was all things to all people. However (and some of this is coincidental), after that happened, I started having very large problems with the commercial version of Redhat Linux.
I wanted to standardize on the commercial version of Redhat for all of the machines in my department that were running Linux. I also wanted that done so that we wouldn’t go down the road of maintaining more than one distribution as more and more production services were deployed on Linux.
Another part of my goal was to reduce the number of packages we build from source. Building packages from source means that every time the application is updated, we have to build it from source again, and hope it all works. One or two packages isn’t so bad, but we build our SSH server, SSL, print server, database server, web server, PHP, and plenty of other stuff from source. This adds up to a lot of overhead. Some of this is, I believe, historical in nature, so I decided to analyze the actual need for this, and I found some success in one service in particular: CUPS.
The CUPS Print Server Debacle
CUPS is a print server package. The first problem I had with the Redhat version was that the version in the package is almost totally unrelated to the capabilities of the package. What exactly, then, is the purpose of having a version number on the package? That was an annoyance — luckily, the package, as it turns out, came with a copy of the ‘ps2ps’ command that worked as it was supposed to, which is to say that it properly stripped out PJL code from PS files printed from Windows machines. Great news, since we could then retire an aging Perl script we used to do this. Great — we were ready to go into testing.
We (staff) all tested printing against this machine for a couple of weeks. The week before we were to move it into production, we ran ‘up2date’ on the machine, and ‘ps2ps’ broke again. Wonderful. As if that weren’t enough, there’s another problem on that new server that could result in an unintentional DoS. The installed (and running by default) authd daemon is configured with the key “instances” set to “UNLIMITED”. As a result, every time one of our guys who was running a fairly strict firewall tried to connect to the box via SSH, the authd server would spawn hundreds of processes trying to connect back to his machine. This caused the load to immediately spike, and the entire machine becomes completely unusable. Thanks, Redhat.
Lacadazical
In addition, other machines that were to move into production had also been updated, only to find that the aacraid driver was broken, and they couldn’t boot! The report in bugzilla for the RAID driver resulted in Redhat saying they would release a fix for this in the next kernel package, which would be released the next time a security vulnerability was reported and fixed — they would release them in the same package. I couldn’t believe this, especially since I knew that the biggest server retailers used hardware that required that driver. As a result of this, my machines remained running with a locally-exploitable kernel vulnerability for the sake of a RAID driver. Thanks, Redhat.
Up2Date Rollbacks, or not
This makes for a nice segue into the undocumented-but-advertised features area. After running up2date on these machines and noting their extreme brokenness, I very much wanted to do a rollback on certain packages. I had enabled rollbacks in my up2date configuration, only to find when I actually needed to do one that the process is completely undocumented by Redhat. It would seem antithetical to the cause of releasing a product that is, if nothing else, stable and predictable, to release a package with features enabled that are not only undocumented, but to the best sources my searches could find, are not recommended for production use by Redhat. Unfortunately, I didn’t learn of the issues with rollbacks until I needed to use the feature, at which time I found that the best solution was to scrap rollbacks, save my disk space, and just reinstall the package in question from scratch again. Rollbacks are not a part of a well-balanced administrative policy, as implemented by Redhat.
The most recent update also broke the quota program. The feature that was broken is one that had been noted as being broken several months before the updated release. This was a recommended update from Redhat, and it’s not like just a command line option is broken — the program, as a whole, is unusable. This was in Redhat’s bugzilla in May 2004. How it wound up in an update in December of the same year is beyond me. How it hasn’t been updated since that time is also beyond me. Thanks, Redhat.
New Face, Old Packages, Same Price
Finally, due to Redhat being slow as molasses to implement new, and highly recommended features of software packages, I am forced to completely discount them as a solution for some of the services I’d like to deploy without building from source. My prime example is OpenLDAP. I’ve been trying for way too long to migrate the department from NIS to LDAP for authentication. NIS won’t go away completely, because it’s useful for some things, but we like LDAP for several reasons, and for some things it’s far better than NIS, so we’re moving in this direction.
The current version of OpenLDAP is 2.2.23. The version the Redhat package is based on is 2.0.x. That version of OpenLDAP, if I’m not mistaken, didn’t even fully support LDAPv2, let alone version 3, and used a backend that the OpenLDAP developers have long ago shunned. For some unknown reason that would seem like a whole lot of work, Redhat appears to be applying patches to this old, decrepid version of OpenLDAP and distributing it, presumably with the expectation that someone, somewhere will use it. As far as I know, it is that oldest version of OpenLDAP distributed by default with any Linux distribution. If I use Redhat for this service, I would be forced to build not only OpenLDAP, but the updated version of the Berkley DB backend that is (for the past ~2 years) the standard OpenLDAP backend recommended by the development team. I’d be doing this by myself, and applying updates by myself, in spite of paying Redhat a yearly fee for updates. Thanks, Redhat.
The Point
The point is, what exactly is my motivation for staying with Redhat? I can’t seem to find one. I still use Fedora Core on things that don’t matter much (when I can get it to work — that distro has plenty of issues of its own), but outside of that, I’m evaluating other options. I’m finding some really nice tools. I’m finding far more up to date software packages. I’m finding much more admin-friendly tools and environments. And I’m finding that Redhat, comparatively speaking, is not a very big deal in the server space. Maybe they started charging for the wrong distribution.