The Life of a Sysadmin

Carousel is a lie!

Entries from October 2004.

DSL's down, and I don't care
2004-10-03 08:29:27

So as of November, we've got a new place to live now, right in downtown Vancouver. It's back in the West End; I lived there for seven years, and with my (now) wife for two of those. We moved out to the sticks for cheaper rent and a bigger place, but realized we missed downtown: most of our friends are here, and there was really nothing much where we were living. It was (is) a nice place, but a bit of a black hole as far as things to do go. We were lucky, and found a nice place; the building manager took a shine to us, I guess, and offered us the place as we were looking at it. It costs more but it's about the same size as our current place, and we're happy to be moving back. 'Course, this does bring up the question of Internet access. I'm hosting five websites on Thornhill, the Linux server, and doing DNS for another domain that belongs to a friend of mine. My ISP is not the greatest value; a static IP is currently costing me an extra $80 per month (and the TOS still prohibit servers, although they've yet to enforce it), and I just can't justify that with the extra rent we'll be paying. That means going to DHCP, dynamic DNS, and I don't know what-all. There are other options, of course. Shaw is the local cable company, and I could always go to ADSL. God knows there's lots of choices there; I used to work for one of them. However, my experience there has made me extremely wary of ADSL in Vancouver. We resold ADSL service from a company that I don't want to name; let's just say that if you think of the c in E=mc2, you'll think of their name. When we started I was quite impressed: static IP address (usually a 10.something, but public ones were available if you asked) and servers were okay. But then it turned into this absolute nightmare:

Christ almighty, it was such a giant clusterfuck. I began to sing between support calls, to the tune of "Jimmy Crack Corn":

A customer called in to 'fess Our techn'cal service was the best So why was he in such distress? The database dropped his MAC address. Sing it! DSL's down, and I don't care DSL's down, and I don't care DSL's down, and I don't care -- It's Lightspeed's fault again!

Eventually it did begin to work, and from what I understand it does quite well today. But still...shudder. Never again. They resold their services to a bunch of different local ISPs, most of which I recognised (can't remember who they are now, worse luck), that I'm deathly afraid of getting stuck in the giant sucking wound that is Lightspeed Internet. There's Telus, of course, but they're liable to be even worse; they've been going through hellacious layoffs in the last year, and horror stories abound about the wretched customer service these days. In fact, they're even being investigated by the CRTC because of the number of complaints. Besides, Telus' service was spotty to begin with; their DHCP servers would go down frequently, and take people's connection with it. (Ha. It got to the point where the only selling point I could repeat in good conscience was the fact that our tech support people were easier to reach than Telus'.) So...unless I can find an ISP with a penchant for handing out cheap static IP addresses and being generous with traffic, I'll do dynamic DNS. Some day I'll colocate, or get a virtual server. Until then, I'll settle for cheap.

No tags
Big Hair Books
2004-10-06 19:11:26

Network problems again last week. Cheap switches will be the death of me, I swear, unless cable management gets me first. (Actually, it was both this time...cable looped back on itself + cheap switch == lots of embarassing explanations.)

But there are bright spots in this morass -- 48 of them, to be precise, in the form of 2 x HP 2626 Procurve Managed Switches. SSH login, VLANs up the wazoo, and much muchness. The only thing I'm not sure about is whether or not it does port mirroring (which I can live without, but it'd be nice). (UPDATE: Yes it does. Weeoo!) If these work out, then I think it'll be 2 x 2650s to replace the DLink unmanaged ones that keep crashing. The Ciscos seem nice and all, but the cost...oh my. And the respondents to the recent Ask Slashdot seemed to like HP a lot. Plus, we used to use 'em at my old job, and everyone was pretty happy. We'll see how it goes.

Just bought Neal Stephenson's The System Of The World at Big Hair Bookstore. Twenty-two pages and I love it already. God, the man can write.

Tags: books, hardware.
Oh my
2004-10-07 19:51:01

It's Udo. He's got The Groove Cave and a Boney M page. Be sure to check out his room. Oh yes.

No tags
bcwireless.net
2004-10-10 19:34:11

I've been meaning to do some reading at bcwireless.net for a while now; instead of packing for the move, I'm reading up at last. It seems pretty damned cool, especially the idea of links between different free wireless networks. For some reason, the idea of rebuilding FIDOnet or the Internet just seems really cool to me. I'm going to be moving downtown in a few weeks. I've got a wireless access point I inherited from a friend of mine; I think I'll point it out the window and join the network once we're settled in. On another note, I finally got Snort and ACID set up on my web server. It's interesting to see what it catches, like formmail access and special CyberKit pings and whatnot. Nothing drop-dead scary yet, which is good.

No tags
No cascading netgroups please
2004-10-11 13:56:00

Top Tip: Red Hat and NIS groups A while back, we ran into problems with netgroups and FreeBSD. I've lost the links, but it turns out that NIS groups can be a total of 1024 characters, not including whitespace. Lemme tell you, it doesn't take many entries like: (foo.example.com,,) to fill up that limit, and it's pretty stupid. The solution, such as it is, is to create container netgroups like this: master.netgroup @subgroup1, @subgroup2 @subgroup1 (foo.example.com,,) ... It's a crock, but at least it's a solution for FreeBSD. Well, last week it caused problems. We've got a RedHat machine, and guess what? Yep, doesn't recursively expand the netgroups: if you tell it to export to master.netgroup, it'll say it's doing it, but won't actually do it. It'll happily export to subgroup1 if you list them explicitly; it will not expand master.netgroup into subgroup1 and subgroup2. Bollocks. Bollocks, I say.

2 comments. No tags
Now that's what I call quite good!
2004-10-17 21:30:32

Just going over the alerts from ACID and Snort tonight while listening to The Housemartins, which really is the perfect accompaniment (sp?). Sure, I could have a life, but what fun would that be to write about? Interesting to see how many things Snort twigs on, like all the stop-doing-that ICMP messages that come back at 3 in the morning. After a bit of digging, I noticed that they were almost all triggered by an initial UDP packet to port 53 of some host -- which in turn is caused by the web stats program trying to figure out what country everyone's coming from. Not sure if Webalizer (which rox, btw) is being too aggressive in its timing or what; I've got it set up to do 35 concurrent queries, which now that I think of it could probably be scaled back a bit...what else has my server got to do at 3am? Next step is to try and come up with a rule to catch WordPress comment spam; my wife's blog has been hit by gambling site spammers a couple times already this month. The pattern may allow me to watch for it -- a quick POST, followed by a GET two to three seconds later, with the User Agent set to look like IE 4.0 on Windows 98 -- but the question is how to get Snort to watch for a two-part signature like that. Actually, the real question is how to build automatic weapons fire into Snort's flexible response options, but that's another point. Mmm, The Housemartins. I'd forgotten how good they were. Drop down, baby, drop down dead tonight...

No tags
One wire == One wire. More than one wire == rat's nest.
2004-10-22 06:09:00

Q: Why did the Romans lose their empire? A: Poor cable management. I swear to God, if I get an ulcer and heliobacteria are not responsible, it'll be from cable management. All this week I've been trying to install a new managed switch so that we can keep test equipment from, say, making FreeBSD cry by claiming the broadcast MAC address as their own (whee!). And all this week the job has been utter, thieving hell. It's a lab, of course, so that means test equipment and its attendant cables: USB, power, network, RS-232, phone (two kinds), RF, and telegraph (I swear I've found the last gutta-percha telegraph line in existance). They have mated and nested and raised more cables and set them free to find their own fortune. It's a mess of Darwinian proportions. Slowly, very slowly, I've been separating them and attempting to organize them. It's difficult, but I have many sins to atone for. Finally, I'd reached the point today where I could consider hooking up test equipment to the managed switch: separate VLAN, traffic logging, port mirroring in case of trouble...oh, it was gonna be sweet. And then my cables didn't work. I'd crimped five 18-foot cables, tested them with the brand-new cable tester, then carefully stepped behind the rack, threaded them underneath, across a floor, up a table leg, then through a channel underneath the table using old telephone cables tied together in a double sheet bend that I'd carefully practiced following instructions brought to me via Google. I'd poked 'em through holes in the table, brought up the port in the switch, then plugged 'em in only to find that they didn't fucking work. I checked the ports again: good. I checked 'em with the cable tester, putting the detachable bit at one end and running to the other end to press "AutoTest": good. I checked 'em with the iBook: bad. Tried 'em again on the workstations: bad. I tried another cable, purchased long ago, in the same ports that were giving me trouble now: good, and the iBook began again to work. I got mad. I unthreaded the cables from the table and table leg, unhooked 'em from the switch, then ran to my desk. I loaded up the web page of one of our many suppliers, and searched for "Ethernet Cables", "10baseT", "Category 5", "Network Cables", "Wires", "Network", "Ethernet", then found them by browsing to "Computers : Accessories : Connections : Switch/Hub : Wired : Misc : Uncategorized". I shook the cables angrily at the screen, shouting, "Do you see how much those cables cost? It's still worth it to me, you worthless pieces of shit!" Then I saw that they were selling 10 15-foot cables for $22.95. Then I ordered 5 packs. It's said about Linux and circumcisions, but it's really true about network cables: crimping them yourself only saves you money if your time is worth nothing.

No tags
Hash ne mix pas avec Cygwin
2004-10-23 19:55:06

Top Tip #1: You can set up SSH under Cygwin so that you can SSH into your W2K box and make it useful. But when you want to allow people with domain accounts to do this, you need to add the appropriate entries yourself into /etc/passwd. Here's how to do it: mkpasswd -d | perl -ne'@line = split /:/, $_; @line[3]=545; print join ":", @line;" >> /etc/passwd As part of a much larger problem, I had to get one of these SSH-enabled 2K machines to rejoin its domain. The SID had changed, so that meant I had to recreate the password file entries. Not being one to dive in where a more careful approach might do just as much harm, I ran the line above with a subtle variation: mkpasswd -d -u foo| perl -ne'@line = split /:/, $_; @line[3]=545; print join ":", @line;" >> /etc/passwd This got the info for my account alone. I then commented out the original entry for foo with a hash, then tried SSHing in: ssh bar -l foo Password: //bar/foo: Permission denied WTF? I uncommented the old entry and tried again. This time it worked: mounting my home directory worked a treat. This was not good. Going back to the old domain was not the best of options -- certainly not one that could last very long -- and this was supposed to be a routine prisoner transfer anyway. What the hell was going on? I tried rebooting. I tried rejoining the new domain again. I tried restarting the SSH service. I tried tweaking the SIDs for the Administrator and ssh privilege-separation entries in the password file. No luck. I got desperate enough to turn on Samba debugging, and that gave me a clue about what might be happening. I compared the output in Samba's logfiles for two machines: the one I was migrating and another that still worked. When it came time to try and mount my home directory on the machine, the working one was trying it using my credentials, and the non-working one was trying it using the credentials of the guest account. Since we don't allow guest access to home shares, this was a problem. But why the hell was the machine losing my identity along the way? I decided, for no good reason at all, to see if I could mount my home directory by hand using Windows' net use command. I went up to the / directory and thought about typing: net use /user:domainfoo foo which wouldn't have worked anyway, but I was (as mentioned) desperate. I decided to see what was there, first, and where in God's name I might actually mount this thing. And I saw it: # ls -l / drwx------ 16 #foo Users 544 8 Oct 14:15 bar I'm sorry, who owns that directory? I deleted the line in /etc/passwd that began with "#foo", and tried SSHing in again: ssh bar -l foo Password: Success! You are logged into this server! ...which I'd never been happier to see. So as far as I can tell: Top Tip #2: Using a hash to comment out a line in /etc/passwd in Cygwin doesn't really work. Thank you, and good night. On a lighter note, this post was originally written outside Waterfront Theatre in Vancouver's beautiful faux marketplace, Granville Island Public Market ("GIPM: Authentic(tm), but still with parking!") while waiting to see Neal Stephenson, along with two other writers who I'm sure deserve more from me than being lumped in with the rest of the non-Stephenson world. I could not get wireless access at GIPM on this iMac. There is no justice in this world. But at least I was first in line.

No tags
Stupid, stupid Purolator
2004-10-28 08:46:01

Want to track a Purolator shipment? Getting that stupid error message about how you can only use Internet Explorer? Fuck that noise! Put this bookmark into Mozilla or Firefox: http://shipnow.purolator.com/shiponline/track/moredetailsFramesetWeb.asp?pin=%s Edit the properties of the bookmark so that the keyword is something like "purolator". Then, when you get a shipment ID -- ABC123, for example -- just type purolator ABC123 into your location bar. Stupid, stupid IE only sites.

1 comments. No tags

RSS Feed