Tue - Jan 24, 2006 : 09:59 pm
Baptism By Fire...
All I gotta say is, I'm glad I had a lot of clothes on this morning. Yes, it was friggin' cold outside, but that was only a small nuisance. The real reason I'm glad I had many layers on was due to the large amount of crap I shot in my drawers when I checked the main server log files this morning.
Around 9:30, the floor admin came to me saying he thought we had been hacked. I didn't think too much of it until I went to the log files on our main server (which hosts our db, ftp and http) only to find that the mysql log was *completely* empty, along with the mail logs and the boot logs. Now, I'm not sure if you know much about servers, but.... log files don't just magically erase themselves.
It was at this point I was pretty certain something very bad was happening. People couldn't connect to our sites, get their email, or pretty much do anything productive which had to do with their daily tasks, so... we thought we were screwed.
Anyway... To make a long story short, I did the quickest and most intense search for rootkit discovery programs in my life, and quickly found out that our server hadn't been hacked... It was just set up by kindergarteners. I swear, the people who set this server up must have been drunk. All of those log files had been 0 bytes for months (we later were to find out), and that just isn't right. I'm not exactly sure what happened, but there's got to be a reason those logs are all empty.
Anywho.... Other than the fun day at work (now I've got to work on getting the server up to par *in addition* to what I was supposed to get done this week), and finding out that we have *no* backups at all for our server (which is absolutely unbelievable seeing that this company I'm with is 100% dependent on the successful operation of this server for its pay right now), and my being sick with a bad cold / sore throat.... yeah... I'd say I had a great day. :)
At least we *didn't* get hacked. That would have been even more fun. This week is gonna rock. :-|
Talk to ya'll tomorrow.
Oh... and just in case you're a hacker-cracker-sicko, I'm gonna get everything backed up as early as tomorrow morning, so you've got less than 12 hours to get everything rooted. :)