Carousel is a lie!

So back in February (Damn, I've got to get all these old journal entries into WordPress) (although that isn't one I've ported...) I wrote about network problems.

We've got three Linksys Etherfast 4124 switches cascaded together, and an errant 8-port switch elsewhere in the office managed to freeze up all three of them. To track down the source of the problem, I had to do a Binary Level One Elimination Search (rip out half of the cables, see if the problem went away; if so, plug in half the remaining cables and see if it came back; rinse and repeat).

Naturally, all this was complicated by the fact that we're using dumb switches -- no management ability, no console ability, so no way to figure out what might be going on. (The fact these things were freezing up in the first place is yet another complicating factor, but that's another story.)

I did a little reading, asked a few questions (which you folks were kind enough to answer), and put in a request for Catalyst switches. The budget is currently before The Board.

Welp, yesterday it happened again. There was a brief thunderstorm, and at least one power surge knocked a bunch of the smaller (8-port) switches we have deployed at the network edge (ie, desks) for a loop. I was in the middle of trying to figure out the cause of one (small, just a few computers that couldn't connect) network outage when a telephone repair guy showed up on an unrelated service call. When I finished showing him what needed to be done, the whole network was frozen.

The BLOES revealed the problem to be the switch I'd been looking at before, which I suppose I should've suspected. A power cycle seemed to fix the small switches, and was the only thing that unfroze the big switches. Afterward, I went around and checked all the small switches we have, power-cycled the zombies, and made sure everyone was okay. Then I told my boss what had happened and why. I suspect I'll get the switches, plus some UPSs and better surge protectors.

The larger problem is that we don't have nearly the network drops we need. These small switches are everywhere, because the channels in the floor for network cables are nearly full. We're looking for more space -- a move is scheduled by next June, when our current lease expires -- and I've let my boss know that we couldn't possibly have too many network drops. 3 ethernet + 1 phone jack every ten feet would be great, and I think they'd all be used within six months.

In other news, the move went well; tunneling MySQL over SSH worked perfectly. Hurray! Also, I'm in the process of setting up a Debian server for a friend of mine. He's a web designer who's thinking about running his own server. He's also a Mac guy who doesn't have the funds to get an Xserve, so I'm giving him an old computer to learn Linux on: how to install stuff, how to run a nameserver, build a firewall, send mail, yadda yadda. I've come across some good tutorials on setting up a server, but I'm also looking for something that'll tell him why, not just how. If anyone has any suggestions, please let me know.

A while back I set up greylisting on Postfix for my home server. It works well, but I have the same concerns now that I did then. The script (smtpd-policy.pl from the examples section of Postfix' source) feels like a bit of a crock; yes, it's just the example script, but I don't like the Berkeley DB files, and comments in the code like "DO NOT create the greylist database in a file system that can run out of space" make me nervous. It hasn't been a problem -- in, oh, six months of running the file is only up to about 5.5 MB. But still: there's no provision for removing old entries, which means an awful soul-searching battle with the database if you ever need to trim it.

I had a brief look at the script tonight, hoping to find a way to maybe hack in MySQL support, but decided to check with Saint Google first. Sure enough, there's gps, the Greylist Policy Service for Postfix. Uses C++ for speed and MySQL/PostgreSQL for the backend, which is nice. I should be able to hack up a migration script for the old entries (just as soon as I hack up a migration script for all the old journal entries...), and all should be good.

One thing I'm noticing with greylisting, though, is just how many attempts are being made from multiple IP addresses within a short time; one attempt, today, had attempts from four different IP addresses within five minutes, all from the same made-up email address. The original Perl script has the advantage that I can change it easily -- I know Perl, and I'd be pretty much starting from scratch with C++ -- and maybe add the ability to track this sort of thing. It'd be nice to be able to tarpit attempts to do this, say on the third attempt.

Tarpitting...another problem with Linux. The TARPIT module for netfilter has yet to be updated to work with the 2.6 kernel, and I really don't want to switch back to 2.4 just for this. LaBrea is nice, and I'm running a lashed-together natd configuration on my FreeBSD firewall box in conjunction with LaBrea running on my desktop on a second interface. It works, but it doesn't work in the case of a Linux webserver running on its own, outside the main firewall. I'm even less a kernel hacker than I am a C++ programmer, and figuring out the compiling problems and changed skbuff route structures (say) is beyond me. It's things like this that make me want to move to OpenBSD. Yeah, rebuilding a server and learning a new firewall language is a pain in the ass, but at least it's one I can handle.

So I run Ansel as a gallery program for my website. I recommend it: it's simple, does what it needs to, and gets out of the way. (I notice, BTW, that 2.0 has just been released. I'll have to have a look.)

I've customized it a fair bit for my site -- little things like the ability to edit a caption, or to go backward in an album as well as forward. It's nothing big so far, but I've been thinking of releasing the changes as a fork of Ansel. (I did send the author the first batch of changes I made, but they didn't make it to his version [which is fair and his choice].)

I've been working a bit, every now and then, on the ability to make big sweeping changes at once: edit a bunch of photos, move 'em around in an album, that sort of thing. But there's another issue holding me back: getting pictures into Ansel in the first place.

You can upload a picture to Ansel, as you'd expect. But having a digital camera has really increased the volume of pictures my wife and I take, and adding pictures one at a time is a major pain in the ass. It's one thing to do it for the first 50 pictures, but after that it's insane. And adding 50 pictures in one upload is doubly crazy, at least with my ignorance of PHP. So what to do?

We just bought an iBook, and iPhoto is on there, of course. One of the impressive things is that it found the digital camera and grabbed the pictures from it without problems. By contrast, gphoto took some messing -- to be fair, the camera we bought wasn't supported by gphoto when we bought it, but was added shortly thereafter (phew!). Red Hat or gtkam or some combination of the two will occasionally get cranky: drivers don't get loaded, or the pictures won't all be downloaded, or things will just crash. So when iPhoto Just Worked (TM), I assumed that was it for gphoto.

But it wasn't: the way it manages photos pisses off my wife, and she prefers gphoto and Linux (!). So we're back to our original problem: how to add a bunch of photos to the gallery in one go?

I got to thinking about this the other day, and realized the problem was a bit more basic than that; really, it was: how to add a bunch of photos to a database in one go? I started thinking about duplicating iPhoto's functionality with MySQL, then forgot about it. Instead, I decided that what I really wanted was something like an SQL plugin for gtkam. You'd specify a database, connection details, and what table to put it in. Exif info could go in, along with thumbnails. Then, for full geek points, you tunnel the connection over SSL to your server. Huh? Huh? Yeah!

I had a quick look through Google to see if anyone else was doing this...no. Then I braced myself, held my nose and waded into the awful, slow-as-fuck, confusing and generally bad Sourceforge mailing list archive for gphoto. (I've tried, without success, to subscribe to the gphoto mailing list three times. Don't get me started.) Again, nada.

I don't know enough about how libmysqlclient.so (ha!) to know if, say, just using gphoto from the command line to, say, dump to standard out might work; it seems unlikely.

Partly I'm putting this here as a reminder to myself. Partly I'm putting this here in the hopes that someone will call bullshit if it's needed. But I'm also putting it here in the hopes that some actual programmer, of which I am not, will take up the idea. It'd be cool. I'd totally link to your site.

A quick Google turns up SSH over DNS. But after reading this, I'm wondering if anyone has tried RSS over DNS. Bueller? Bueller? (I hesitate to link to this posting, because this guy seems to know what he's doing, and I'm pretty sure I don't. But since it's 8pm on a Friday and I've had my snifter of port, I'm going to risk looking like an ignorant ass.)

Pro: You've got an infrastructure that has caching built in. You control the TTL. There's the TXT record, which has already been hacked to do awful, terrible things. You could compress your feed ('cos everyone knows that bzip2 is a magic bullet), or split it into different host names like the DNSTorrent guy.

Con: Short record limit. Me talking out my ass. Millions of geeks gnashing their teeth at the downfall of the intarweb. (It looks like there's been lots of thought [or at least some] about using BitTorrent, with a general consensus of "no, it won't work".)

Incidentally, the government of Alberta has a crapload of RSS feeds available. Who knew?

• http://xoomer.virgilio.it/flavio.stanchina/debian/fglrx-installer.html
• http://fabrice.bellard.free.fr/qemu/

My wife and I are thinking of moving back downtown soon, so that means housecleaning in preparation. I've been biting the bullet and putting together an email to the local LUG mailing list offering up 7 (!) computers in various states. There's a lot of things started that never got finished.

Cie (named after Cie Baxter, of course) was my first server. It's got a 200MHz Pentium, 48MB of RAM and a 2GB IDE drive. Friends of mine had upgraded, and they asked me if I wanted their old computer. Hey, who's gonna turn down a free computer?

My first computer job was at a small ISP. I quickly wrangled a static IP (still allocated!) from the sysadmin and snaked an ethernet cable over the ceiling tiles from my desk to the server room. It was meant to be an IPv6 tunnel broker, but that quickly fell by the wayside; I got the basic routing sorted out, but then I lost interest when it came time to figure out an authentication scheme. I'm like that: lots of ideas, little follow-through. I've been lucky to stay interested in computers as long as I have.

What it did become was the web, mail and DNS server for my domain, my wife's, and a few friends of mine (though that last one I'm only doing DNS and secondary MX). It ran Slackware 7.0, straight from the CD set I bought at Chapters back when they still had books (and Linux CDs, apparently).

I remember convincing the sysadmin at work that I could handle securing BIND 9 (quietly convinced the entire time I was going to get r00ted within a week), and telling the owner of the company (who is even more flighty than I am) that an IPv6 tunnel broker would provide lots of value to our customers, and amazing the friends who had donated it in the first place that an old computer that would barely run Windows 98 could be of any use to anyone.

Incidentally, I was always a bit amused by the fact that I had an AOpen sticker on my server; I learned to loathe AOpen modems while at Dowco, and I swore I'd never buy anything with their name on it. But hey, since Cie was free, I guess it doesn't count...

I left Dowco but kept Cie, and it sat here by my desk at home, using up my ridiculously expensive static IP address for a long time. It was fine. And then, a while back, I tried to set up a pretty heavy PHP-based CMS for a friend of mine. Cie choked, right away: it took 20 seconds to render a page. That was no good at all.

I asked around, and the consensus was it was the RAM, idiot. That and the noisy PS fan convinced me it was time to upgrade.

My boss was selling his old computer -- a Compaq desktop machine with a 500MHz P3, 64MB of RAM and a SCSI tape drive. I bought a big-ass hard drive (and made sure it was going to be quiet -- best investment ever), put the tape drive in my fileserver, bought a 256MB stick of RAM and sat down to build a new server. I went with Slackware 9, spent some time locking down my firewall and /etc/fstab, got the latest kernel and OpenSSH sources, and installed, compiled, swore, reinstalled, formatted, and reinstalled again. Thus was born Thornhill (named after Lisa Thornhill, of course).

It's only fitting that I always meant to use Cie as a honeypot, but never got around to it.

One of the pieces of equipment at work is an oscilloscope that runs W2K. "WTF?" I hear you say. "Saint Aardvark, WTFSOF?" But it's true. Don't know why, but it's true.

(We have another oscilloscope that came with an unregistered copy of XP, but that's another story.)

Being the aspiring good sysadmin I aspire to be, I bought some cheap cable routers -- you know, the generic DLinkSys jobbies with a built-in firewall. I hooked it up, told people not to hook it up to the network without it, and forgot about it... ...until today when I was working on our firewall and noticed it was blocking broadcasts to 172.16.0.255, UDP port 137.

That's not a netblock we use, so I was a bit surprised. Good ol' tcpdump showed it was anouncing itself as the local master for workgroup INFINEON. Oh shit, it's the oscilloscope. I checked out the lab and, sure enough, the firewall was being used as a quick-n-dirty switch on the firewalled side, and the oscilloscope was plugged in. Fuck!

To make matters worse, a little bit later someone comes up to me and asks if there's anything "funny" with the network. (I love that question. It's so...definite.) Checked it out, and his laptop has grabbed an IP address from the (fortunately, by-now-disconnected) DHCP server that comes with the router. Double fuck!

I ran off to London Drugs to get a switch, and was lucky enough to find a 16-port Linksys. (SMCs are for shit. SMC? Quality? It is to laugh. Linksys switches are giving me trouble too, but at least it's less trouble.) Set up, and everything is working for now. So here's my mistakes:

1. Not making it perfectly clear how to hook up the router correctly, and not making it impossible (or at least painful) to hook it up any other way.
2. Not making it obvious -- written warnings, flashing neon, whatever-- that the router was not a switch.
3. Not having something, somewhere, to at the very least watch for weird IP addresses and report them, or (better yet) to watch for rogue DHCP servers and report them, or (best of all) to watch for and shoot down with lasers any rogue DHCP servers.

There is, of course, the mistake of not having managed switches that would mitigate all of these mistakes, but with luck we'll be getting those shortly.

Goddamn, but this is one god-awful case. Any time I'm asked why we use Shuttles, I'm gonna give 'em an earful.

1. The electrical outlet in the back of the case is not part of the power supply; instead, it's connected by a three-inch cable to the actual power supply. There are three bare pieces of metal coming out the back of the outlet, which have three leads with those metal sleeves attached. Think old AT power supply switches.
2. The power supply is held in place by screws that go into the ventilation holes in the power supply case. There's a small chance they might actually be dual-purpose by design, but mostly it looks like they're using very short screws in order to avoid hitting anything inside.
3. The disk basket (what the hell is the right term? the metal bit that holds the CD, hard drive and a floppy) can't be removed (as it can in Shuttles), so installing the motherboard or hard drive is a huge pain in the ass.
4. The disk basket has a bracket that hangs underneath that is meant to hold the hard drive. There is a flange at one end where it slides into place, and at the other there is ONE screw holding it in place. Needless to say, that does not inspire confidence.
5. The disk basket looks like the space for the floppy can hold a hard drive (like Shuttles). That's a lie. Because of the screws holding the front cover in place, it's about a quarter-inch too narrow for a hard drive.
6. The clearance on either side of the hard drive bracket is very narrow, so it's difficult to guess which end is meant to go in first. I tried one side only to find that the electrical connector wouldn't allow the case to close.
7. The disk basket is just a little off-center. (It was the office Apple guy who noticed this.) That means the hard drive must go in so the electrical connector goes on the wider side, right? Right -- but it still doesn't let the case close nicely. It'll do it, but it causes cringing when you think of the electrical connector being squished up like that.
8. Getting the IDE cable to go to the hard drive is a huge pain in the ass. The orientation of the motherboard (VIA 1000, I believe) put the first IDE connector on one side of the case, and the disk basket -- naturally -- put the hard drive connector at the other end. It'll go, but it causes cringing when you think of how much you had to crimp the IDE cable to get it to work.

I may put in more later as therapy brings back more memories. But for the love of everything you hold holy, stay away from this case. That's the Checkercube case, available at http://www.checkercube.com; it sucks ass through a straw. You got that, Google?

Okay, this is a tangled tale...

I've got an old install of FreeBSD I decided to bring up to date. Being the funky sort, I decided to do a portupgrade and keep the OS the same. (Security fixins [fixins! yeah!] have been applied, so I'm not too worried about doing make world.) And of course, desktop of the elder gods (...) Gnome was installed -- 2.4. Whee! Dive right in, right? That's why the gods gave us port-upgrade! Wrong. Got this error:

/usr/bin/ld: warning: libintl.so.4, needed by /usr/X11R6/lib/libgconf-2.so, may conflict with libintl.so.6
/usr/bin/ld: warning: libgmodule-2.0.so.200, needed by /usr/X11R6/lib/libgconf-2.so, may conflict with libgmodule-2.0.so.400
/usr/bin/ld: warning: libgobject-2.0.so.200, needed by /usr/X11R6/lib/libgconf-2.so, may conflict with libgobject-2.0.so.400
/usr/bin/ld: warning: libgthread-2.0.so.200, needed by /usr/X11R6/lib/libgconf-2.so, may conflict with libgthread-2.0.so.400
/usr/bin/ld: warning: libglib-2.0.so.200, needed by /usr/X11R6/lib/libgconf-2.so, may conflict with libglib-2.0.so.400
../../libgnomevfs/.libs/libgnomevfs-2.so: undefined reference to `bonobo_poa_get_threaded'
gmake[3]: *** [test-vfolder] Error 1
gmake[3]: Leaving directory `/usr/ports/devel/gnomevfs2/work/gnome-vfs-2.6.1.1/modules/vfolder'
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory `/usr/ports/devel/gnomevfs2/work/gnome-vfs-2.6.1.1/modules'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/usr/ports/devel/gnomevfs2/work/gnome-vfs-2.6.1.1'
gmake: *** [all] Error 2 *** Error code 2 Stop in /usr/ports/devel/gnomevfs2.

which lead me to this page, which (sadly) wasn't much help. I figured out that the problem was conflicting versions of glib, but WTF to do about that? After much searching, I began to remember faintly something about a FreeBSD Gnome upgrade page. Sure enough, this was the one I was thinking about. And look at this:

It is not possible to upgrade from GNOME 2.4 to GNOME 2.6 by simply running portupgrade(1). There are new dependencies, and ports will build out-of-order, eventually causing the build to fail. Additionally, GTK+-2 cannot install when there are input methods installed which were linked against older GTK+-2 versions. To work around these problems, and to provide an update mechanism as simple as portupgrade(1), the FreeBSD GNOME team has produced a comprehensive upgrade script. The script can be downloaded from: http://www.FreeBSD.org/gnome/gnome_upgrade.sh Simply download that script, and save it to disk.

Sigh. So I read the bit about how running the script after doing The Thing You Shouldn't Do would probably not cause problems, and decided to plunge ahead. But things ended badly when it came time to upgrade Scrollkeeper: [configure does some stuff...] checking for DocBook XML DTD... configure: error: not found. Make sure you have the DocBook DTD installed and ensure that it is registered in /usr/local/share/xml/catalog.

Wha'? Being me, I ignored the bit about the file it was looking for and dove right into the configure script. By the time I came out the other side, I figured out that it was unable to find /usr/local/share/xml/catalog. Then I re-read the error message. Well, fuck. After some digging around, I found out that this file was provided by sdocbook-xml on the old system -- version 4.1.2.5. The version I was trying to upgrade to ws 4.1.2.5_2. Shouldn't be that big a difference...but it is. The Makefile for the older port has this at the end: post-install: ${MKCATALOG} -q -c ${CATALOG} install sdocbook catalog But the newer port's Makefile has this: post-install: ${XMLCATMGR} -sc ${CATALOG_PORTS_SGML} add CATALOG ${SDOCBOOKDIR}/catalog ${XMLCATMGR} -c ${CATALOG_PORTS_XML} add nextCatalog ${SDOCBOOKDIR}/catalog.xml which, natch, doesn't make the file that scrollkeeper's configure script is looking for. Freshports.org has this note on its sdocbook-xml page:

Switch to using xmlcatmgr from mkcatalog. # Maintainers, please let me know if I break something in your port. Submitted by: hrs

After that is listed a crapload of PRs that the change fixes, or at least addresses. What to do? The thing that every sysadmin does sooner or later: cheat. # ln -s /usr/local/share/xml/catalog.xml /usr/local/share/xml/catalog And bugger me senseless with an iPod if it doesn't work. Who said cheaters never prosper?

Good idea:

The sad thing I've noticed is that some people new to the world of lawyer blogging have never heard of blogs like BeSpacific.com. Instead, their news aggregators may be filled with new blogs, which is not a bad thing in itself, but lack blogs that I consider to be bedrock legal blogs. That bothers me. Maybe it shows that I'm getting older. But Bob is on to something important. So, I'm announcing a new feature of this blog where I'll highlight the core legal blogs that meet my definition of excellence.

Terrible title:

I'm tentatively calling this feature "Essential Blawgs."

My advice: stay away from the fish.