Frustrating

I’ve been running my own servers for years and I do it for a couple of reasons … first, it is fun setting them up, and second it allows me to install and use any sorts of programs I’d like without having to talk a system administrator into it.

Of course, there is the downside … I have to be the system administrator. I’m also lazy … and because I’ve been running either Linux, I’ve also become quite complacent. If I used a Microsoft product for my server, I would be awake all night. But I haven’t had problems for years, so I’ve become complacent. Enough so that I haven’t been monitoring my box or monitoring the security leaks and whatnot.

Consequently, there was this bug in a library/program called OpenSSH that Apache (my web server) uses. The bug was exploited by the slapper.worm and really became a slap in the face of us smug Linux users (see this article).

So a couple of months ago a hacker broke into my system and installed a number of “backdoors” and whatnot. This was before public knowledge of the bug, so I assumed that a bad password from a user was to blame, and so I plug up the holes that I found. It wasn’t until the slapper.worm infected my system did I patch my system … of course, the hackers had been using my box to attack other boxes for weeks.

When these sorts of things happen, the best thing to do is reinstall the operating system … but that is a lot of work, and if anything goes wrong, your web site, your email, your friend’s web sites, etc. are all down. So, I got my backup server outfitted with more drive space and memory and transferred the data over to it. I then began the arduous project of getting the new system working as well as the old. And this is time-consuming.

So I decided that while I was working on it, I would keep my old system working. Every now and then I would hop on the machine to see what was going on. Hmmm… yup, they uploaded their guns and are shooting another system. So I killed the processes and deleted the guns.

Hmm… I changed the password, so maybe they uploaded their public key credentials … better delete those directories just to make sure. Oh great, they’ve installed a new syslog to take all system information and send it someplace else…. the bastards.

And so the battle would rage on a regular basis. But my old system had so many compromises by these hackers that in the process of cleaning things up, I broke the house of cards, and the system died.

Of course, it died before my new system was fully functional, so I spent the last couple of days struggling to adapt my old configuration files to the new operating system.

But I’m bleary-eyed and tired, but the system is up and while there are rough-edges that I’ll have polish over time, it is mostly functional. Will I still be lazy? Probably. But I’m not complacent anymore, and I need to start paying attention.