Perfect_timing

title: Perfect timing date: 2004-09-02 06:59:49

So back in February (Damn, I've got to get all these old journal entries into WordPress) (although that isn't one I've ported...) I wrote about network problems.

We've got three Linksys Etherfast 4124 switches cascaded together, and an errant 8-port switch elsewhere in the office managed to freeze up all three of them. To track down the source of the problem, I had to do a Binary Level One Elimination Search (rip out half of the cables, see if the problem went away; if so, plug in half the remaining cables and see if it came back; rinse and repeat).

Naturally, all this was complicated by the fact that we're using dumb switches -- no management ability, no console ability, so no way to figure out what might be going on. (The fact these things were freezing up in the first place is yet another complicating factor, but that's another story.)

I did a little reading, asked a few questions (which you folks were kind enough to answer), and put in a request for Catalyst switches. The budget is currently before The Board.

Welp, yesterday it happened again. There was a brief thunderstorm, and at least one power surge knocked a bunch of the smaller (8-port) switches we have deployed at the network edge (ie, desks) for a loop. I was in the middle of trying to figure out the cause of one (small, just a few computers that couldn't connect) network outage when a telephone repair guy showed up on an unrelated service call. When I finished showing him what needed to be done, the whole network was frozen.

The BLOES revealed the problem to be the switch I'd been looking at before, which I suppose I should've suspected. A power cycle seemed to fix the small switches, and was the only thing that unfroze the big switches. Afterward, I went around and checked all the small switches we have, power-cycled the zombies, and made sure everyone was okay. Then I told my boss what had happened and why. I suspect I'll get the switches, plus some UPSs and better surge protectors.

The larger problem is that we don't have nearly the network drops we need. These small switches are everywhere, because the channels in the floor for network cables are nearly full. We're looking for more space -- a move is scheduled by next June, when our current lease expires -- and I've let my boss know that we couldn't possibly have too many network drops. 3 ethernet + 1 phone jack every ten feet would be great, and I think they'd all be used within six months.

In other news, the move went well; tunneling MySQL over SSH worked perfectly. Hurray! Also, I'm in the process of setting up a Debian server for a friend of mine. He's a web designer who's thinking about running his own server. He's also a Mac guy who doesn't have the funds to get an Xserve, so I'm giving him an old computer to learn Linux on: how to install stuff, how to run a nameserver, build a firewall, send mail, yadda yadda. I've come across some good tutorials on setting up a server, but I'm also looking for something that'll tell him why, not just how. If anyone has any suggestions, please let me know.