The Life of a Sysadmin

Carousel is a lie!

Entries from September 2005.

Trackback: A Tragedy in Three Acts
2005-09-01 19:17:41

From Ant's Eye View:

You should assume that systems on the public network will be abused. This is a lesson as old as the Internet, but every programmer seems to have to learn it for him/herself: if you make a system available to the public, people will abuse it.

Worth reading.

No tags
APIC error on CPU0: 40(40)
2005-09-01 19:19:22

Getting tons of error messages on a machine here at work like this:

APIC error on CPU0: 40(40)

A quick Google turns up this message suggesting a problem with SiS chipsets and APIC. Sure enough, this machine does use the SiS chipset...I've come to be very suspicious of SiS chipsets: lots of trouble, and not worth the money saved compared to (say) a mobo with an Intel chipset.

No tags
I'm sorry...
2005-09-01 19:20:11

...but when you use the word "solution", do you mean:

I'd include links, but I think it would kill me.

No tags
Hah
2005-09-01 20:03:00

From the OpenSSH FAQ:

Why should it be used? [snip] No need to retrain normal users.

So I've got a low humour threshold. Sue me.

No tags
NWR04B: Flash, telnetd, serial port...plus Ted Leo
2005-09-04 12:50:10

Things are coming along on this router, and I've managed to make some progress on a couple of fronts. First off, I've managed to get access to the flash memory on this thing. It's a little embarrassing, because I went through a lot of code in the mtd section of the kernel before realizing that I had simply not included the driver in the configuration file. Managed to learn a bit about how it fits together, though, so it wasn't wasted effort. I've been able to get the contents of the flash out -- at least, the bit that's covered by the memory map that Codeman put together in his driver, which is about 1MB of the 2MB on board. Still, that appears to include the bootloader menu on this thing, which is good; with luck I'll be able to figure out the checksum for that, and maybe upload armboot or something. Of course, I could always just overwrite the flash directly...but I'm a little scared of that. We'll see. The other thing that this'll lead to, of course, is including a filesystem in the flash memory itself. Right now I'm mounting everything by NFS, which is very flexible but not terribly self-contained. With something like JFFS2 and a separate partition for the kernel, I should be able to have something pretty skookum. I ran into some weirdness with Minicom and the serial port: at random times, for reasons I couldn't figure out, the display from the router would get all scrambled. Letters and newlines would be dropped, or transposed, or just garbled out of recognition entirely. I tried everything I could think of: power-cycling the router, letting it cool (it doesn't take long for it to heat up, and things tend to go south pretty quickly when it does...must do something about that), swapping cables, swapping serial ports, exiting minicom, trying other serial port terminal programs (and let me tell you, there aren't many for Linux). Eventually I gave up and ran:

cat /dev/ttyS0 & cat > /dev/ttyS0

which worked perfectly: I could watch it boot, run commands, all that stuff. I could even see that the shell was using colours for ls, which made me wonder if maybe that was a problem. Finally, though, it came time to try uploading another kernel image. I tried fooling around with sb, but while I could get it to upload to the router w/o problems, it was difficult to get the timing right when it ended, and the image didn't seem to load properly. All right, I thought, I'll use Minicom just for uploading. But check it out: when I ran Minicom again, it was perfect -- no display problems at all. Still don't know what changed, but I'm glad it's working again. This led me to try getting the telnet daemon from BusyBox working...if I can't use a serial port, why not just use the network? But getting it going is going to take some work. With uClibc, there is neither a fork() nor a daemon() routine, both of which are used by telnetd. Instead, you get vfork, which lets a child run but blocks the parent until the child calls either exit() or exec. So, as uCdot points out, the trick is to do exec() the same program, but with a command-line option that tells the application that it's a child, and should be treated accordingly. Good trick. By the time I realized that, though, it was midnight, and I figured I'd be too tired to do it coherently. And then I got the flash memory working, so I was distracted. Coming soon, though... On another note: on Friday my wife and I went with the famous Victor Scott to see Ted Leo and the Pharmacists. Holy fuck can that man play. And his drummer! His drummer has the beard I want plus all the drumming chops in the entire world. The last drummer I saw who was anywhere close to him played for Wilco; before that, Lotion. Absolutely fucking amazing, and a must-see if you ever get the chance. Hon. mention to opening band The Parallels, for whom I can't find a website. Great 60s-mod outfits and music, and a fun show.

No tags
FYT
2005-09-10 06:25:38

FYT #1: New Firefox 1.5 Beta. It's great: wicked fast, and they've added drag-n-drop tabs. Slashdot comment pages render in a heartbeat. But it's pissing me off right now for two reasons. First, the Profile Manager only seems to come up if no other Firefox window is running. If there is another window running, it comes up with that profile no matter what arguments you pass (-P, -ProfileManager, -P Profile Name, ). (When I was first writing this entry, I tried that last one just to make sure. When the current profile came up yet again I closed it -- but closed the browser window that had this entry, too. I'm writing this in vi in an xterm now.)

This is irritating because I have two profiles: Default and Wide Open. Default is where I spend nearly all my time; Java, JavaScript, pop-up windows and flash are turned off; AdBlock shoots to kill; animations go once and then stop; I'm asked about cookies. I hate dancing baloney. Wide Open is where I go if I need to visit my bank's website (it's not that wide open, of course), or if there's something that won't work in my Default profile that I'm convinced is worth the effort (which doesn't happen often). Keeping two profiles is much easier than toggling all that nonsense each time.

Second, a lot of the extensions I love aren't yet ready for 1.5 (or at least, don't say they're ready...I seem to remember when the upgrade to 1.0 happened that you could edit some of the extensions directly and just lie about what version was required). Adblock is running -- if it wasn't for that, I don't think I'd be using the new version at all. But Session Saver, Sage and Mozex aren't, and I've come to rely on them. We'll have to see.

FYT #2: I went into work this morning to reboot a couple of servers. I'd let everyone know about it, and got up with my wife at 4.45am. But when I got to the building, the card that let me in the front door would not make the elevators go -- they just sat in the lobby waiting for, I don't know, drugs or Jesus. (Double punishment!) I'd used the card before to make the elevators go, so WTF? (Stairwells are not an option; you can't get into your floor [or any other] using your key or any access card.)

After failing to find a security guard anywhere, I called tenant services for the building. They said that the elevators might be turned off, but they couldn't be sure; I could get a better answer calling back during the week. (Fair enough, since our building's managed by a company that owns buildings all across Canada.) Oh, and security starts at 8am. Fuck. I'll have to reschedule for during the week, but after making sure that I can get in at 6am. Double fuck!

FYT #3: Why am I rebooting servers? Good question: they're running FreeBSD, after all, so it's not like it should need to happen all that often. The answer is: because amd sucks ass through straws. Not only does amd:

  1. create a mess of symlinks (people who complain about SysV init symlinks messes need to look at amd: /home/foo symlinked to /net/machine/home/foo symlinked to /.amd_mnt/machine/host/home/foo, the only place the directory is actually mounted) (interesting: quick Google for sysv init symlink turns up this post by my namesake)

  2. interact badly with FreeBSD symlink caching (okay, FreeBSD's fault maybe)

but it will also get wedged sometimes, requiring a reboot -- and don't talk to me about the -r option for amd, because that simply doesn't work.

F'r instance: a while back one guy at worked moved from FreeBSD to Linux. I took the opportunity to give him a bigger hard drive; he'd had a second one, mounted at /home/foo/scratch, because he'd run out of room on the first. Unfortunately, one of the servers in question had /home/foo/scratch mounted at the time through amd -- and when his machine came back online w/no scratch directory, amd/NFS refused to umount it and refused to mount his home directory, because the bogus /home/foo/scratch was blocking it. That's what this morning's reboot was meant to get around. Okay, again, not all amd's fault -- NFS and me, not in that order -- but still.

I mentioned two servers, though, so what about the second? Aha, that's the symlink caching thing. We get around this by running a newer version of amd than is supplied w/FreeBSD; it doesn't have quite so many problems. But I'd missed the second server, and it didn't have the pointer to the newer version of amd. Again, my fault -- I should've caught this a long time ago -- but dangit, it shouldn't be necessary to do this just to restart amd. (I'm setting up cfengine to catch this sort of thing. cfengine rox.)

Minor update re: earlier problems with Vinum and a Maxtor IDE card: I picked up a new RocketRaid 454 that was reputed to work much better, plus had four controllers rather than two. Cheap, too -- $135. Long story short is that it still caused problems, I think; the machine seized up again in the middle of backups, apropos of nothing and with no message or panic. (Took a while for this to happen, though, so it was an improvement. I think I should've taken to heart the warning I got a while back that Vinum was not the most stable of code.

Tags: amd, freebsd.
New machine, or, How does one crack Solaris 2.8 again?
2005-09-10 17:20:43

I went out to the swap meet today, but I couldn't get excited about anything there. I'd gone out hoping to find a Sun workstation or some such, and nothing. Closest I got to interesting was the guy offering a stripped bare P2 laptop -- no battery, no hard drive, no CD, no floppy, and a busted screen -- for $100. Oh wait, not interesting -- crack-induced. That's what I meant. (Yeah yeah yeah parts nothing.) So I took off to go see Cal, since I was (sort of) in the neighbourhood. I was a lot happier when I found the upstairs where they'd hid the good stuff -- including a [SparcStation LX][3] and a Sun External Hard Drive, model 411 (4.5GB Seagate Barracuda, baby!). Even managed to find a keyboard and a cable and a 25-to-9-pin serial adapter. I was briefly tempted to buy an SGI keyboard, which looked like it'd been made to beat Communists to death, but resisted. Stuffed it in my backpack and brought it home...man, those things are pigs. I hooked up a serial port, tried the magic keycodes, and nothing. After wiggling the keyboard cable a bit I could get the keyboard to beep at power-on (speaker in a keyboard? no wonder Sun's got such a good rep...:-), but nothing else -- not even the caps lock light. Well, what about networking? Yep, it's working, and appears to be convinced that it's kootenay.cs.ubc.ca judging by the ypbind requests. A quick nmap and rpcinfo -p confirmed that all it had open was portmapper and ypbind. Allegedly, the POST has failed if the caps lock key doesn't flash -- but surely it wouldn't be running RPC services if that were true. ..which is going to make cracking it over the crossover cable a little difficult. It's not exporting any directories. I figure I can spoof the domain, but then what? In other news: Firefox moved the tab select key from Ctrl to Alt (ie, Alt-3 selects the 3rd tab). I thought this feature was completely gone a few versions back, and was quite sad. Finally figuring it out makes me happy.

[3]:

3 comments. No tags
Sigh II
2005-09-20 17:56:40

Seeking individual with access to quality lists of opt-in email addresses to mail to. Offer is to be geared to US addresses, Mortgage Refinance.

Why is it that only spammers advertise on Craigslist?

No tags
First post in a while...
2005-09-29 17:01:56

...and it's another Wine problem. Coworker says he's having trouble, as of a couple days ago, running Wine on machine Foo, a FreeBSD machine that lots of people use. He's running into these errors:

err:ntdll:RtlpWaitForCriticalSection Critical section 0x65430070 wait timed out, retrying (60 sec) fs=008f                     
err:seh:EXC_DefaultHandling Unhandled exception code c0000194 flags 0 addr 0x280abf94                                          

...which is pretty strange, and doubly so since he's only having this problem on Foo -- machines Bar and Baz are working just fine. I spent entirely too much time fucking around with --debugmsg +all, truss(1) and ktrace before I finally thought to check:

$ ps auxw | grep wine                                                                                                          
coworker        81344  2.0  0.0     0    0  p3- Z    Tue11PM   0:00.00 (wine)                                                  
coworker        81341  0.0  0.2 11120 1452  p3- I    Tue11PM   0:00.34 (wine)                                                  
coworker        81343  0.0  0.0  3100  324  ??  Is   Tue11PM   0:00.12 wineserver                                              

Sigh. Kill off 81341, the rest fell too, and it all worked again.

No tags

RSS Feed