I’ve been meaning to write this up for some time - but just haven’t gotten there. Today seems to be the day. So I’ll assume you’re ready for a break from my steady post of things that I’m eating to take a small journey down memory lane.
Some disclosures:
- I love Linux and Open Source.
- I am not a purist! I dual boot XP (need me some games) and carry an iPhone (though it annoys me). I even have an Xbox360 (see note about games).
- Although I use a lot of open source, and have contributed code back (not enough mind you) - I also pay for software and don’t think it is morally wrong to have closed source software.
- I don’t think Linux is for everyone (yet). I got my mom a netbook and it runs XP (i didn’t even make it dual boot). I’m not above exposing people to Linux - but I’d prefer they get to it when it solves their problems.
Awkward Upbringing
I got my start in computers in a weird way. Mostly because although my mom had the foresight to know that they were the future - neither of my parents used them (by that I mean my dad never really used a computer and my mom has slowly gotten the hang of it in the last few years). So I started with a TI-99/4a.
That isn’t that strange. A lot of people back then started with them. It is the next part that put me on the path to unusual. Namely, my second computer was an Atari 1040ST. This revolutionary device had a mouse and a graphical interface while Micrsoft was still grappling with DOS. I had two versions of it. That is the computer I actually learned to program on. I can still remember trying to save up money to buy a C compiler for it. I just stuck with GFA Basic - which was pretty full featured considering it had a GUI builder included.
This computer basically lasted me until I went to college 1993. As a result, I had very limited exposure to Microsoft, DOS, or Windows. I did use it a little - mostly because I had a friend who worked in a computer shop and he introduced me to Sierra Games and Commander Keen. We would go to my father’s office at night and play games on the secretary’s computer.
When it came time to go to college - I headed off to Texas with a lot of stuff - but no computer. Hard to believe now.
HP-UX
Trinity University had a lot of computing resources available. The most important thing was getting a crash course in the Internet in the Mac lab. At the time - that mostly meant news groups, email, and Bolo (Yes while normal people were knee deep in Doom - I was in a computer lab playing a game about tanks and turrets.)
I got an account on the VAX mainframe as soon as I landed. I quickly set out to learn how to make it do stuff. I can still remember trying to get the library to track down books on the tools for the mainframe. I suppose that should have been my first sign that I was pursuing a dead technology.
In my second year, Trinity got a grant to do Virtual Reality research for the military. They put in a huge lab of HP-UX boxes. I ended up declaring as a Computer Science major just to get access to them.
I started taking Computer Science classes and learning the actually theory of programming. (Unfortunately, learning the practice of programming wouldn’t happen until much much later). This is when I started learning Unix. Don’t ask me why but in the back of my head I almost always type
ps -ef
instead of
ps auxw
Since the ef version is what HP UX expected to see what processes are running.
I was just a user on the HP-UX systems. I could do stuff - but there were a lot of things off limits. Plus - I had a disk quota. (Which if memory serves - they setup quota but they didn’t restrict chown - so I spent some time downloading large files and chmod 777/chown mail - so they wouldn’t count against me).
It was during this period that I got to see the beginning of the web. The first time I saw it - I didn’t understand why it was any better than gopher. But they added images and things go better. I would love to say that I used it for scientific research - but in truth - I used it the same way we do now. Meaning I surfed the web during class when the lecture was boring.
I still remember seeing this guy celebrating his operating system.
I didn’t really get it at the time. I figured it was a class project or something. Certainly not worth anything. I just filed it away in my head and continued on.
Outside Academia
My life changed in my senior year of college. I came back from study abroad and got introduced to a life long friend. I ended up joining a small company that he had started. The goal was to build software. Right out of the gate he told me he was using something called - Linux - and asked if I wanted root access.
How do I explain what that meant at the time? Before I was just a user. A lowly peon who could only kill his own processes. I couldn’t install things in /bin. I couldn’t use ports below 1024. There is a much much longer list - but you get the idea. Most importantly - and this is an idea that insinuates itself into anyone who spends time with *nix - as root you are a god because you have the power to destroy anything in the system. When I say anything - I mean it. You can even destroy the system itself. A little rm -fR in the wrong place and you can wipe the drive and *nix will happily let you do it because you are root! Root is not meant for the unwashed masses. It is meant for people with experience and expertise who know what the right thing to do is and can be trusted to do it.
As you can tell I was pretty excited. It was an incredibly liberating idea to have such complete power. Of course now I’ve had root on Linux boxes (lots and lots and lots of them) for over 13 years - so some of the excitement has died down.
RedHat - The Salad Days
Although Richard had used Slackware, we quickly shifted to RedHat. At the time it just seemed like the clear leader. It had a geek spirit but it seemed to be very commercial friendly.
I think I started with RedHat 3.0. I had to look that up and guess. Mostly because before that Linux used something called a.out format instead of ELF binaries. I never really knew the difference - only that I started with ELF and found the explanations confusing.
I used RedHat for a long time. I can remember telling my brother back in Indiana to go to Barnes and Noble and buy an install disk and try it out. (Yes there was a time when Linux was sold in book stores - wacky).
I put up with changes that happen often in the community. For example, I was writing a lot of C/C++ at the time and a new libc came out and we had to spend a bunch of time updating everything and keeping the code working on the boxes we hadn’t upgraded.
It was a good time. This was early in the revolution. So early in fact that people couldn’t agree on how to pronounce it. Thankfully I was always on the right side of the argument. Though for many years I looked at anyone who said Line-ix with suspicion.
Our little band of Linux users even made in roads with the local Unix grey beards. We were a sort of novelty at the Unix user group in town. Almost everyone else worked for a government agency or big company and used Sun or some other commercial Unix. They were the sort of seasoned Unix people that you still read about.
We went for the technical discussion (and free pizza). When Solaris x86 was launched they convinced us to try it out. After a week of trying to get it to install, we got it up and running. And then learned that it didn’t ship with anything. So we ended up putting GNU tools on it.
YellowDog - Huh?
I had to inject this because I had almost forgotten about this little side jaunt. This whole time I was running a PowerPC mac. I would switch to x86 on the desktop for some time. I was constantly trying to get Linux onto my Mac. I had a variety of failed attempts. Later when I switched to a PowerBook I was able to install YellowDog Linux. That was basically RedHat recompiled for the PowerPC. This was before OSX.
Actually when they originally decided to buy NextStep instead of BeOS - I was kind of upset. I ended up leaving the Apple fold when OSX came in. I think I said something along the lines of -
“I already have a Unix -it’s called Linux”
I know this is hard to believe if you’re a die hard mac person - but the Reality Distortion Field eventually wears off. I just got tired of having to fit in to Steve’s world.
From this you can see that my conversion to Linux which had started some time ago was taking hold.
RedHat - The Betrayal
RedHat had all potential in the world to be great. But in a number of ways it just didn’t live up to that potential.
There were three different things that ended up pushing me away from them.
First, we used to sell a specially licensed version of RedHat that shipped with a legally licensed version of SSL. This was before RSA patent wore off. It started out as a small product. But we grew and so did sales of the product. Here’s the rub. We had to buy physical boxes for the SSL OS. I tried to get RedHat to just sell me license keys. But they were to busy to sort that out. So I had a corner of my office dedicated to stacks and stacks of boxes of unopened SSL disks.
Second, RPM hell. I started out as a staunch opponent of packages. I thought that things should be compiled to your specification. As a matter of fact, I believed this so completely - that if you were an early customer of my hosting company you got a stock RedHat server with a custom compiled version of Apache. Yup - we compiled it every time. I am stuck trying to think of an analogy that could express how stupid this was. Especially because we just straight up compiled it. Which means it sort of broke the package system.
This is the sort of dumb idea that seems like sound technical thinking at the time. Let me just say this. I’m sure there is some case where compiling from source all the time is a great idea (Gentoo is built on that idea). It is certainly a maintainable practice when you have a couple of boxes to admin. But when you start talking about 1,000s of boxes it’s crazy. (Actually, I’ve become such a believer in packages that I go out of my way to use software and versions that are already packaged).
I converted over to using RPM - and once I saw how great it could be at scale - I loved it. There were a couple of things that bugged me. Often times, no rpm existed for the software I wanted. Not a big deal - I ended up learning RPM and built a lot of them. Including an tweaked version of Apache so we didn’t have to compile it. Fine - so there are packages you have to build yourself. I learned to live with that.
Then the dreaded RPM Hell. There are two different flavors of this. One is the case where it just can’t figure out all the dependencies or get things you need. This could turn a simple install into a research project. (Please note I haven’t used RedHat in a long time - maybe they fixed this.) The other is that the process of installing, uninstalling, upgrading, and such especially with your own packages can result in a completely broken RPM database. You haven’t experience true torture until your package system itself starts working against you.
Finally, RedHat did something that changed how I looked at them. The started an Enterprise edition. I understand why they did it - they needed to generate revenue. They were also doing a good thing by raising the awareness of Linux as a viable Linux alternative. All that is true - but they way they did it made it very clear that they wanted certain kinds of customers - and I wasn’t one of them. To be fair, I think they eventually tried to address their geekier fans with RawHide and then Fedora, but by that time I’d been introduced to something else.
Enter the Debian
About this time I was working at a hosting company called ServerBeach. The main target was a $99/month server. Needless to say, we weren’t going to be using RedHat Enterprise for our internal servers - it was just too expensive.
That’s when they introduced me to Debian. Truth be told I don’t know if it was Henry Pugsley or Chris Blumentritt that converted me - but once I got exposed to it I was hooked.
I’ll admit I’d heard of Debian before this time. It was a distro for the faithful. It was designed by geeks for geeks and was notorious for its purity and difficulty. Since I had to give up RedHat anyway I figured I’d give it a whirl. After some time spent with apt (which never had any of the problems that I’d had with RPM) and I learned that if software exists there is a deb for it. (Corollary - if no deb exists then the software isn’t any good) I was in heaven.
I even got used to the idea that our servers ran stable and my workstation ran testing. We’d occasionally move packages from testing over to stable because I needed something - but all in all halcyon days.
At the time, Debian felt like coming home. The more I looked around the more it seemed like people who were into the weird stuff with Linux that I was - used Debian. I also didn’t realize it but I wasn’t the only one making this switch. The community was growing - thanks RedHat!
The Dark Side of Debian
I had a great time with Debian - until I hit its dark side. This comes in two parts. The first is their deep and abiding love for democracy. I mean that in the most serious way. Debian wants it to be a completely community effort. As a distro it has continually stood up for true open source - go so far as to fight Mozilla over icons and even throw out documentation or firmware that wasn’t open. This cuts both ways. On the good side, they are a force for openness on every front. They are able to mobilize a huge army to that end. The bad side is that something I don’t care about the principle - I just want to use my video card (I’m looking at you Nvidia). So basically I’m happy they fight the fight but my standards are lower - which in some ways keeps me an outside in the community.
Which leads to the second thing. When I started I was playing with potato (all debian releases are named after ToyStory characers). Then out came Woody. Debian has a very strong belief that stable should be stable. They didn’t care how old it was as long as it was stable. In the beginning that wasn’t a big deal. But then they didn’t release for three years. Look for a commercial software shop - sending out a new version of the OS once every couple of years is actually pretty brisk. But that’s now how Linux works. Distros are collections of a lot of different software projects all updating and releasing all the time. Three years in the open source world is decades.
In response to this, I switched from testing to unstable. I ended up having more than a few fights as I tried to get unstable installed on production web servers so I could have modern libraries. Stable just got crustier and crustier - and I became more and more disenchanted.
It’s Breezy and So Am I
I wish I knew how I found it or why I switched. Now all I remember is that it was called Breezy Badger - and I figure I’m pretty Breezy myself - so maybe I’d try it. ( Actually, I was corrected by my wife. My first version of Ubuntu was Warty Warthog. I just couldn’t remove the fart joke.)
As far as I can remember - I had gotten really frustrated with Debian. Then I find this distro that had Debian packages (woot!), they were designing it for the desktop (which is great since I’ve been using a linux desktop for years), they had a benevolent dictator in the form of Mark Shuttleworth (which meant there was someone who could make sure things moved along), they didn’t mind dealing with proprietary hardware, and they promised a release cycle of every six months. So basically they gave me everything I loved about Debian - but fixed all the things I didn’t like about Debian.
Slowly but surely, Ubuntu has taken over everything I run. I use it on the desktop and a heck of a lot of servers. It doesn’t have the commercial driver support that RedHat enjoys in the datacenter - but I’ve found my ways around that.
The Return of the Zombies
During my time in the Ubuntu camp, I’ve seen the rise of the Apple Zombies again. One by one, I’ve seen my Linux friends switch over to OSX. They always say the same thing
“It just works! I got tired of tinkering with Linux.”
They have that look in their eye. They quickly launch into the sales pitch on how awesome it is. Little do they know that I’ve been on their side before - long before. I remember how comfortable the fuzzy hand cuffs were. I’m not anxious to go back. I don’t miss the ridiculous secrecy or arbitrary decisions. Now it appears that much like the classic zombie films - I’m hold up in a house fighting them off with a variety of make shift weaponry. I can’t be the only person on the planet who uses Linux as my main desktop, but some days it certainly feels like it.
Bumpy but Back On Track
Truth be told, things have been bumpy lately. I couldn’t find the great article I read that showed how the number of issues with each release of Ubuntu has gotten worse (Update a commenter found it for me - Ubuntu Release Quality. Post Hardy has been a rough time - with a lot of experimentation and a lot of explosions. Part of me was beginning to think that I was just being pig headed. Maybe I should join the zombie army and switch over. They have lots of shiny things. All the cool kids are doing it - etc.
In the last few weeks, I’ve spent a lot of time designing the infrastructure for my next project. I was trying to design something that would be flexible and easy to maintain. Something that would use cheap commodity hardware but be able to easily scale into the cloud.
I’ve been knee deep in Jaunty, JeOS, LVM, KVM, and puppet. I’ve loving it. Jaunty has cleaned up the experience a lot. Although I was wavering before, my faith has largely been restored. Mainly because I’m having fun with Linux again. There are few things more exciting than to see it dance - and dance it does.
A long long time ago, I saw Tim O’Reilly speak at OSCon. He talked about how when you choose a technology you also choose the community that goes with it. I guess that is the thing that keeps me embedded in the Linux world. Linux people are the ones who say - “That just came out - can I put Linux on it - No? Let’s get to work!”. They’re the ones who fight for our rights to be able to watch a DVD on our computers. They’re the ones who have come to join Linus and Stallman and others at their invitation to make Stone Soup. Nothing that eventually turned into something. Their the ones that are pushing the world to be a better place for everyone. Even if you don’t use Open Source stuff - know that the stuff you buy is better because it is competing with it.
It’s a Linux/Open Source/GNU World
I’ve been using Linux for a long long time. Looking back now, I realize just how primitive it was. That’s ok - it has grown up a lot since then.
Even The Economist (one of my favorite magazines) has declared Open Source the winner Every year some says - “This is the year of the Linux desktop.” I’m not even sure what that means any more. Or if it even matters. Looking around my house I see that it is a Linux world.
My servers in the datacenter - Linux (one has sarge which I’m ashamed to admit - but I swear I’m moving off of it)
Sure my file server runs Linux. That’s not weird.
Mythbox - Linux
Yes, my laptop too - Linux.
Yes my desktop - Linux.
My router - DD-WRT - that’s Linux.
My iphone - jailbroken running GNU software in debs
My kindle - Linux.
My Roku - Linux
My Popcorn Hour Linux
My search engine - Linux
Maybe we should all broaden our minds and realize that Lucas was actually talking about Linux/Open Source/GNU -
“It’s an energy field created by all living things. It surrounds us, penetrates us, and binds the galaxy together.”
Yes - I just said it - Linux is the force :P I suppose this is part of that shift that has been happening for some time. Sure you can be a specialist in computer science, but the real shift is that you can’t avoid technology. Whatever you do - technology is involved. And that’s the part I love. People may choose Apple for their desktop or god forbid Microsoft for their server - but when they just consume the output - they’re going to be getting it from a Linux system because “it just works”.
One last thing - this quote is often spoken of in the Linux community
First they ignore us,
then they laugh at us,
then they fight us,
then we win.
I think you know which part of the process I think we’re in.
