Weekend_update

title: Weekend Update date: 2006-02-24 20:37:13

So we bought the townhome...something I keep forgetting that folks don't know about, since my normally on-the-ball wife has not yet written about this. (We have a good division of labour: I write about computers, and she writes about everything else.) So far our biggest screw-up has been asking for a possession date of April 1st, when we have to be out of our apartment the day before. Oops. Oh well, we'll make it work.

I'll be storing Thornhill at a friend's place (thanks, John!) for a few days around the move. I've got, what, 8 domains on it at the moment for friends and family. Amazing what you can get a poor, underworked Sempron to do these days. :-)

Still working on getting Xen working, but it's going slowly with all the house stuff. I gotta say, I'm pretty impressed; it's very, very neat to just fire up a new machine and have at it.

Tags:

Far too much Windows

Saturday after Patch Tuesday, and I spent far too much time today dealing with it. KB 911564 (aka Vulnerability in Windows Media Player Plug-in with Non-Microsoft Internet Browsers Could Allow Remote Code Execution) simply would not work, remotely nor interactively nor interactively through the Windows Update website. In the end, we had to go around booting machines into fucking safe mode (thank you, the posters of this thread, for the tip) in order to get the damned things to apply.

Sysinternal's handle showed that WinLogon.exe, for some reason, had C:\Program Files\Windows Media Player open, on one machine we checked that was having problems. No idea why, but it's about the only thing we could find that might be causing problems.

However, the news wasn't entirely bad...Windflower, the Perl-based rewrite of Ivy, actually patched a few machines today over an SSH session. Version 0.2 is available here. Hurray!

Tags: windows

Holy crap

Our offer for this townhome has been accepted. We have until the end of the week to lose our nerve. If we don't, we move in April 1st.

Holy crap.

Tags:

Saint Aardvark's Axiom of Self-Righteous Anger

A user at work wanted to move from a desktop machine to a laptop. The Windows profile moved over just fine, so all that was left to do was copy over his outlook.pst. Only it turns out his desktop's hard drive has been quietly failing for a while, and there's some corruption right in his 1.2GB Outlook file. Well, fuck.

The Inbox Recover Tool is meant to help with this sort of thing. It took me a while to find a mention of that, longer to realize that it was actually called scanpst.exe, and even longer to decide that the Windows search tool wasn't going to find C:\Program Files\Common Files\MAPI\1033 -- a fact that is fucking buried in Microsoft's Office support section. (Why 1033? Something to do with Unicode and US English character sets.) Of course, it didn't work.

So okay, what about getting Outlook to export to another file? Good idea! Only it fails about 700MB through, and there's no indication what worked and what didn't -- so no chance for the user to decide if that's enough or not.

So what about exporting a subset of the folders, seeing what fails, and then repeating the process without the failing folder? A little tedious, sure, but it'll work, right? Wrong: you can export one folder, or you can export one folder and its subfolders, but you cannot export more than one folder at one time. Jesus fucking Christ!

Workaround for that was to copy folders (one at a fucking time, natch) to another folder (call it Backup) and try exporting that -- and then see what fails, yadda yadda. But natch, that doesn't work either. You have to watch closely to see what folders are being exported, and anyway a folder may be displayed as being exported more than once, so you still don't know whether a given folder may have worked.

Plus, there was the failing hard drive (remember that?); I suspect that it this new backup folder was just getting thrown on the same crappy chunk of hard drive, making the export of the Backup folder fail in interestingly inconsistent ways. And of course, the whole process takes fifteen minutes to fail, during which time I can't do anything else and neither can the user.

And in the middle of my frustration and rage, an even greater rage welled up in me when I realized that Outlook had totally ruined this guy's email.

Think about it! Here's all this plain text email -- even attachments are encoded in ASCII -- and it has been completely fucking borked by being irretrievably (well, in this case anyway) converted to some proprietary binary format that is completely opaque to me, without at least the saving grace of having good tools for its manipulation available. Redundancy, ease of recovery and ease of manipulation has been thrown away for the sake of (let's be generous here) speed and functionality (indexing, correlation, etc). It's completely ridiculous.

This led to the formation of Saint Aardvark's Axiom of Information Utility:

Any sufficiently important information must be indistinguishable from plain text.

Plain text is redundant, easily (though not necessarily speedily) recognized by the human brain, and has many automated tools to deal with it (think of Unix). All these things make it very, very recoverable. If the information is that important, you need to be able to get at it even if there's a hardware failure. Binary formats throw that away, and that is simply wrong.

But what's a self-important axiom without an equally self-important corrollary?

Any gains in the functionality or speed of information access must be obtained from derived versions of the original information, leaving the original in its plain text form.

I'm perfectly willing to give Outlook the benefit of the doubt in this case; having used a PDA for all of two weeks, I feel uniquely qualified to recognize the utility of having cross-referenced contacts, to-do lists, email, and so on. But this must not come at the expense of recovery!

Think of source code. It's possible to hack on a binary with a hex editor or a disassembler. You can even fix bugs or change the way a program works in this way. But you would never maintain a program in this way: it's hard to understand, it's easy to make a mistake, and it's hard to (say) port to a new language or hardware platform. That's what source code is for: it's easy to understand (assuming you're a programmer), and even if some of it gets garbled it's easy to recover. Plus, you can use tools like indent to change how it looks, or grep to pick out interesting bits, or tags to cross-reference function calls with their definitions.

Of course, you wouldn't try to run source code -- that's what a compiler is for. You gain speed by transforming the source code while still leaving that source code intact: nothing is lost in the process. And that's what Outlook should have done: compiled the plain text email into whatever database (I'm assuming) format Outlook likes, that allows Outlook to do Outlook stuff quickly, while still leaving the original source code -- the email -- intact.

Of course, you don't have to imagine recompiling Outlook's PST file each time; this'd be an incremental thing. And really, it shouldn't be that much different from what it does now...same speed, just a little more disk space taken up. And if the PST file gets borked, no matter -- the recovery tool is nothing more than a compiler that regenerates it from the original email.

As much as I'm picking on Outlook though, this isn't Outlook's problem alone. I've written before about how PHPWiki obscures the information it stores in MySQL. And I did a similar thing to myself years ago by compressing email, since I was running out of disk space. Somewhere along the way the files got corrupted, and I can't get that email back because gzip barfs on it.

And of course, this is just my opinion, formed in the heat of anger. It's almost certainly not a new idea, and might even be wrong. I'd love to hear some feedback on this.

Tags: rant windows outlook

One week 'til I'm 34!

It feels like I've been slacking with my entries lately, so it's time to do some catchup.

First, the NWR04B: I've not been very active on this lately, but there has been a little progress. When last I wrote I was trying to figure out why the kernel was hanging at rtnl_lock, when I used the ADM5120 driver for the switch. It turned out that I was calling register_netdev, which in turn calls rtnl_lock, from within another routine that calls register_netdev itself. That's a problem right there. I fixed this (it was due to some blind cut-n-paste from the old driver), and now it's getting further: it initializes eth0 through eth6...though still doesn't actually send or receive traffic, near as I can tell. I need to spend some time sprinkling more printks throughout the code to figure out where it's failing.

Next, I'm doing some work on Thornhill, my web server. Amanda has been installed; I want to back up stuff a little more intelligently than I'm doing now (tar up everything and dump it on my desktop, which gets backed up by Amanda running on my desktop). Running into a few firewall problems, but nothing unexpected or too difficult.

I'm also trying out Xen again, with an eye to upgrading Thornhill. A while back Alioth answered some questions I had about Xen and servers, and it seemed worth trying. So I've got VMWare Player running on the fastest machine I have (Hunsacker, a 2.4GHz P4 MythTV backend) while I practice getting things right. I've put Gentoo both in dom0 and a guest domain (FristDomain (I kill myself)), and I'm populating FristDomain with the usual LAMP environment. This is all pretty preliminary; I'm pretty much just trying to get familiar with how it all fits together.

I'm considering moving to NetBSD for dom0...stateful IPv6 filtering (though Linux has that now), pf, and just the chance to try something new. For the web server OS, though, I think I'll stick with Linux, and probably with Gentoo. I want something easily upgradeable, and for that it's Gentoo or Debian. I think Gentoo will be a little more up-to-date than Debian, and I want to give portage a try...Hunsacker runs Gentoo, but I rarely touch it.

At work, we had a problem last week with the Subversion repository when, against my advice, someone acting under their manager's direction tried checking in the contents of a SuSE DVD. They weren't trying to check in the ISO itself, at least, but rather, all the contents: whole lotta binary RPMs, mostly. This borked the repository, probably because of a default 2GB limit for Apache. The user saw this error:

svn: MERGE request failed on '/svn'
svn:  Revision file lacks trailing newline

So did everyone else who tried to work with the repository afterward.

I tried svnadmin recover like the good book says, but ze goggles, zey did nossing! Well, crap. We were running hotbackup.py every night, and a quick look showed that last night's copy had everything up to revision 1538 -- 14 revisions ago. (It was revision 1553 that failed.) So I could try moving that in place and losing a bunch of work, or look for something else.

In the end, I was able to get things working by taking a copy of the hotbackup, dumping everything since then, and then applying that dump to the backup. To wit:

$ cp /path/to/hotbackup /path/to/recovered_repository
$ svnadmin dump /path/to/repository --revision 1539:1552 --incremental > dumpfile
$ svnadmin load /path/to/recovered_repository < dumpfile
$ svnadmin verify /path/to/recovered_repository
$ mv /path/to/recovered_repository /path/to/repository

I may up the limit for Apache, but I'm very much inclined not to do so. I really don't think we'll need to check in 2GB at one time, and I still think checking in a DVD is a stupid thing to do.

Tags: nwr04b

How_to_extract_audio_from_a_movie_with_mplayer

title: How to extract audio from a movie with mplayer date: 2006-02-03 07:49:42

Just a little tip for Google:

mplayer -ao pcm /path/to/movie

will produce a WAV file called audiodump.wav.

Tags:

Zaurus:_evolution Sync.pl

title: Zaurus: evolution-sync.pl date: 2006-01-28 16:26:11

Dear god, this took me a long time to download. I managed to finally track it down here, but I'm putting up a copy here as a backup.

Tags:

Zaurus:_first_impressions

title: Zaurus: First Impressions date: 2006-01-22 09:19:45

The Sharp Zaurus has arrived, and so far I'm pretty happy with it. It's a handheld, it runs Linux, it's nice. Handwriting recognition is pretty good with this -- better than I remember a Handspring (last handheld I tried, maybe two years ago) being. The keyboard is decent, but it doesn't take long before I can feel my thumbs start to seize up, and anyway there's not always a way to do something without tapping somewhere on the screen.

The software, however...hm. It came with the stock Sharp distro and apps, so I tried using that at first. They were pretty good -- calendar worked, to-do list worked -- but things were all separate from each other. A todo item might have a due date, but it wouldn't be added to the calendar. A calendar item can't include a link to contact information. All this means a lot of re-entering stuff, which is a pain in the ass.

But hey, beauty of open source, right? So I tried Opie, the QT-based OpenZaurus distro. I had some problems installing because I missed part of the instructions (when it says rename the file initrd.bin, don't rename it initrd), but once it came up things seemed good. You still can't have a todo item show up in your calendar, but there's a decent workaround: an app called "Today" that shows you what's coming up in your calendar and your todo list. Perfect!

Except for suspend: it doesn't always come back from the dead, at which point you need to flip a tiny switch on the back of the thing and reboot. (This is a known problem, and a kernel upgrade is supposed to make it better.)

So fine, beauty of open source, right? I tried GPE, the GTK-based OpenZaurus distro, and am even more annoyed with it. Suspend seems to work better, but the handwriting recognition software is different from stock/Opie, so I have to relearn strokes. Plus, there's no separate area for capital (as opposed to lower-case) letters, which I've come to find pretty damned handy. That's enough to push me back to Opie, and try the kernel upgrade.

Tags:

Updates, Patches and Upgrades.

Been a lot happening here that I haven't written down...time to correct that.

First off, work is BUSY. We have ten -- no, wait, twelve -- -- people starting this month. About 8 have started already, so that leaves four. Fortunately, one of them is a new sysadmin who will be helping me out. Thank whoever for small mercies.

The sheer number of people has been part of the reason I've been so busy; another has been the Windows patches this month. Three goddamn times I've been in this month patching machines: once with the unofficial WMF fix, once with the out-of-band official WMF fix, and once with the two regular patch Tuesday patches. I am sick and tired of Windows problems.

However, I have managed to cobble together Windflower, a small-and-so-far-stupid Perl wrapper around the Microsoft Security Baseline Analyzer. So far it will run MSBA on the target computer and come up with a list of fixes it would like to see applied. It'll run over SSH, which is a blessing; I envision this as a way of automagically applying Windows patches remotely without getting a copy of SUS and IIS. It's called Windflower because it's heavily influenced/inspired by Daisy and Ivy, two programs released by Virginia Tech. (It was originally gonna be called Sunflower, but it turns out VT has already released a program with that name...I had no idea 'til now.)

Why not stick with Ivy (which worked better for me than Daisy)? Ivy's great, but it needs a GUI and its UI is irritating (keeps stealing focus, new logs overwrite old logs, etc). I've long wanted something that can work over SSH, and this looks like it should be able to. Plus, Ivy was written in Winbatch, which I don't know and don't have a compiler for. Windflower is written in Perl, which I do know pretty well.

Version 0.1, in all its completely unfinished glory, is available here. GPL'd for open-source goodness!

There's also Amanda, which has been giving me grief. First the estimates were taking hours to finish, which meant that even if backups started at 9pm they wouldn't finish 'til noon the next day. This was fixed by upgrading to 2.4.5, which uses calcsize, quicker-but-slightly-more-inaccurate estimator of the Elder Gods.

Then I ran into another problem: estimate requests, including all the exclusions for each directory, were taking up more than 32KB -- so they were split up into more than one packet by the requesting process. Unfortunately, the receiving process still ignores all but the first packet. Patches, as they say, are welcomed; in the meantime, the workaround is to make the packets smaller. The easiest way to do that is to have one big list of exclusions, rather than specifying each item in that list for each backup. The problem with that is that leads to problems where you (say) want to exclude certain stuff for everyone, plus allow people to specify their own list of exclusions: only the first list gets accepted. My own special workaround, hereby released under the GPL, is:

for i in `ypcat passwd.byuid | awk -F":" '{print $1}'` ; do
    cat /path/to/onebiglist >> $i/.exclude_from_backup
done

Arghhh.

One thing that has helped with work is Time Management for System Administrators, by Tom Limoncelli. I just got this last week, but it's already helped a lot. The sample chapter gives a good overview of The Cycle, the system that TL advocates. The book irritates me in a couple places -- the odd buzzword, and an illustrative anecdote about a friend who was late reviewing a chapter that, frankly, makes TL sound like a bit of an ass. But these are pretty minor complaints, and I recommend getting it.

One of the things he recommends is either a PDA or a PAA (personal analog assistant, aka DayTimer(tm)). I decided to hunt around Ebay for a PDA, thinking I would pick up a used Handspring or some such; instead, I got a Sharp Zaurus SL-5500. Woohoo! Should be arriving next Wednesday.

Finally, I managed to spend a couple hours last night hacking on the NWR04B. I got the driver for the ADM5120 switch compiled; however, it hung when it came time to initialize the switch. A liberal sprinkling of printks showed that the kernel was hung in register_netdev at the call to rtnl_lock. Just for fun, I tried taking that out, and the initialization continued...though other networking drivers complained about RTNL_ASSERT failing, and the ethernet interface didn't actually work, since it couldn't mount its home directory via NFS. Still, progress of a sort.

Tags: windows

Ew! 2

In Visual Studio, a solution is the basic unit of project management.

-- "Working with Microsoft Visual Studio 2005", by Craig Skibo, Marc Young and Brian Johnson

Tags: rant

Ew!

The Atmosphere Player for Acrobat and Adobe Reader is designed to enable use of Atmosphere environments within a PDF document enabling the user the ability to experience a rich variety of interactive content, including three-dimensional objects, directional sound, streaming audio and video, SWF animations, and physical behaviors.

Tags: rant

WMF notes

In here attempting to patch the 35 or so Windows machines that we've got at work. So far, it looks like I should be able to do this remotely using SSH and Cygwin. That depends, of course, on having very fucking silent ways of running everything. So far this has worked for me, on XP/SP2 and 2K/SP4:

1. regsvr32: None of the bits I've seen from SANS mentions it, but there is a silent option. Do it like so: regsvr32 /s /u %windir%\wystem32\shimgvw.dll However: I cannot get %windir% to work with Cygwin. According to this it should work as %WINDIR%, but it doesn't for me. Two things do seem to work: either change directory to /cygdrive/c (Cygwin-specific location of the C: drive) and use an absolute path (winnt\\system32\\shimgvw.dll), or run CMD to get a DOS/Windows shell and use %windir%.

2. The Unofficial Patch: Use the options: /VERYSILENT /SUPPRESSMSGBOXES However, it has problems if you try applying it on a machine that already has had the patch -- remotely, execution will just hang. If you run it locally w/o those options, you'll get a message saying it's already been applied; I guess that case is not handled well when run silently. Oh, and when the patch is applied silently, it'll reboot the machine immediately and without warning.

Thanks to Cygwin, I've got SSH running on most Windows machines here; I should be able to come up with some way of doing this all in one step. I'll post whatever I can figure out.

Update: Yep, a simple batch file does the trick:

regsvr32 /s /u %windir%\system32\shimgvw.dll c:\cygwin\home\Administrator\wmffix_hexblog13.exe /VERYSILENT /SUPPRESSMSGBOXES

chmod 755 both the batch file and the fix, and away we go. The machine passes the test made by the guy who wrote the unofficial patch, which is as close as I think I can come to being sure that it all works. Further Update: Four hours later, done...but I've finally got SSH set up on the few machines I had left, so that's what took up most of the time.

Tags: windows

NWR04B: Turning to the switch

I'm not making much headway erasing flash, so I'm taking a bit of a break from that by turning my attention to the network interfaces.

There are five ethernet interfaces on the NWR04B, plus the wireless card. Only one of the ethernet interfaces comes up enabled in Linux, so I've been trying to track down how it's all initialized and how to change it. The problem is, once again, that I really don't know what I'm doing, and I'm having to teach myself as I go along.

For example: the driver and the datasheet for the CPU talk about the CPU port for the switch. What the hell is that? Originally I thought that might be a special ethernet interface -- you know, like the one that's enabled in Linux. But this info says no, it's essentially a logical interface that may or may not be connected to a particular ethernet interface.

Yeah, the info is for the ADM5120 switch (which in turn is based on a MIPS chip) "a/o/t the ADM5106 (which is ARM-based) I'm working with, but I think the principal should be the same. It certainly seems to match the sort of stuff I'm seeing in the driver code. I'm hopeful, too, that the configuration tool for the 5120 will be, broadly speaking, applicable to the 5106...they talk about ioctls in the driver being used for this sort of thing; not sure if they're in the 5106 driver (I suspect not), but the source code available for 5120-based routers might have enough info to let me cut-and-paste^Wport something over.

Update: So the config tool for the 5120 uses a couple ioctls, SIOCSMATRIX and SIOCGMATRIX, to control which interfaces are on which VLAN. SIOCSMATRIX is defined in the config tool's code as SIOCDEVPRIVATE, and that has to be defined by the driver. It wasn't there in the source code, but a quick search for SIOCSMATRIX turned up a few diffs against the main kernel tree for the 5120. Looks like the switch driver may be a bit more full-featured than what I've currently got for the 5106, and I think it may be more or less a simple cut-n-paste to get it working for the NWR04B. Here's hoping.

Tags: nwr04b

Stupid_hardware_changes

title: Stupid hardware changes date: 2006-01-01 22:36:23

I got these 1U units 'cos I was running out of room, and I figured I'd test these out. Natch, I'm running into problems with them.

First off, I've got one that has panicked on me a few times in the last week. There's never any indication in the logs of what's happened, of course. I thought it was heat, but it recurred after moving it around. Fortunately I had a monitor hooked up, and I saw the panic message. Nothing much that I can make sense of, except that it seems to be in the middle of dealing with an interrupt when it happens -- part of the message had scrolled off the screen. I've turned on console logging and hooked up a serial cable to another machine, so now I wait.

I'm unsure if the second problem is on the same machine or its twin, but it's strange. When SSHing in, or using rsync to transfer big files between the two, I suddenly found the session gone and this error on the screen:

Disconnecting:  Corrupt MAC on input

A bunch of Googling turns up this thread on Google and this patch from Andrew Morton's kernel, which apparently just missed going into 2.6.11. I'll have to give that a try and see how it does. (I feel oddly proud of using a patch from Morton -- something about being that much closer to celebrity. Sometimes I have absolutely no clue how my mind works.)

How things have changed, BTW...I had to check to see what the latest kernel was. Also, when I was trying to get these machines working, I had difficulty with the stock 2.6.8 kernel in Debian -- wouldn't recognize the hard drive. My response? Grab the 2.6.11 stock kernel from Debian, rather than compile my own. Strange days, indeed.

One difference between the two: ACPI was enabled in the BIOS of one, and not the other. Weirdly, HT depends on ACPI...who knew? Also, the IRQ list is slightly differnet without them; you don't get IRQ set to Cascade if HT/ACPI is disabled.

Tags:

Craig Murray's Letters

From CraigMurray.org.uk, via Ken MacLeod, come these letters. From his website:

The British Foreign Office is now seeking to block publication of Craig Murray's forthcoming book, which documents his time as Ambassador to Uzbekistan. The Foreign Office has demanded that Craig Murray remove all references to two especially damning British government documents, indicating that our government was knowingly receiving information extracted by the Uzbeks through torture, and return every copy that he has in his possession. Craig Murray is refusing to do this. Instead, the documents are today being published simultaneously on blogs all around the world. The first document contains the text of several telegrams that Craig Murray sent back to London from 2002 to 2004, warning that the information being passed on by the Uzbek security services was torture-tainted, and challenging MI6 claims that the information was nonetheless "useful". The second document is the text of a legal opinion from the Foreign Office's Michael Wood, arguing that the use by intelligence services of information extracted through torture does not constitute a violation of the UN Convention Against Torture. Craig Murray says: In March 2003 I was summoned back to London from Tashkent specifically for a meeting at which I was told to stop protesting. I was told specifically that it was perfectly legal for us to obtain and to use intelligence from the Uzbek torture chambers. After this meeting Sir Michael Wood, the Foreign and Commonwealth Office's legal adviser, wrote to confirm this position. This minute from Michael Wood is perhaps the most important document that has become public about extraordinary rendition. It is irrefutable evidence of the government's use of torture material, and that I was attempting to stop it. It is no wonder that the government is trying to suppress this.

These files are also available here.

Letter #1 Confidential FM Tashkent (Ambassador Craig Murray) TO FCO, Cabinet Office, DFID, MODUK, OSCE Posts, Security Council Posts 16 September 02 SUBJECT: US/Uzbekistan: Promoting Terrorism SUMMARY US plays down human rights situation in Uzbekistan. A dangerous policy: increasing repression combined with poverty will promote Islamic terrorism. Support to Karimov regime a bankrupt and cynical policy. DETAIL The Economist of 7 September states: "Uzbekistan, in particular, has jailed many thousands of moderate Islamists, an excellent way of converting their families and friends to extremism." The Economist also spoke of "the growing despotism of Mr Karimov" and judged that "the past year has seen a further deterioration of an already grim human rights record". I agree. Between 7,000 and 10,000 political and religious prisoners are currently detained, many after trials before kangaroo courts with no representation. Terrible torture is commonplace: the EU is currently considering a demarche over the terrible case of two Muslims tortured to death in jail apparently with boiling water. Two leading dissidents, Elena Urlaeva and Larissa Vdovna, were two weeks ago committed to a lunatic asylum, where they are being drugged, for demonstrating on human rights. Opposition political parties remain banned. There is no doubt that September 11 gave the pretext to crack down still harder on dissent under the guise of counter-terrorism. Yet on 8 September the US State Department certified that Uzbekistan was improving in both human rights and democracy, thus fulfilling a constitutional requirement and allowing the continuing disbursement of $140 million of US aid to Uzbekistan this year. Human Rights Watch immediately published a commendably sober and balanced rebuttal of the State Department claim. Again we are back in the area of the US accepting sham reform [a reference to my previous telegram on the economy]. In August media censorship was abolished, and theoretically there are independent media outlets, but in practice there is absolutely no criticism of President Karimov or the central government in any Uzbek media. State Department call this self-censorship: I am not sure that is a fair way to describe an unwillingness to experience the brutal methods of the security services. Similarly, following US pressure when Karimov visited Washington, a human rights NGO has been permitted to register. This is an advance, but they have little impact given that no media are prepared to cover any of their activities or carry any of their statements. The final improvement State quote is that in one case of murder of a prisoner the police involved have been prosecuted. That is an improvement, but again related to the Karimov visit and does not appear to presage a general change of policy. On the latest cases of torture deaths the Uzbeks have given the OSCE an incredible explanation, given the nature of the injuries, that the victims died in a fight between prisoners. But allowing a single NGO, a token prosecution of police officers and a fake press freedom cannot possibly outweigh the huge scale of detentions, the torture and the secret executions. President Karimov has admitted to 100 executions a year but human rights groups believe there are more. Added to this, all opposition parties remain banned (the President got a 98% vote) and the Internet is strictly controlled. All Internet providers must go through a single government server and access is barred to many sites including all dissident and opposition sites and much international media (including, ironically, waronterrorism.com). This is in essence still a totalitarian state: there is far less freedom than still prevails, for example, in Mugabe's Zimbabwe. A Movement for Democratic Change or any judicial independence would be impossible here. Karimov is a dictator who is committed to neither political nor economic reform. The purpose of his regime is not the development of his country but the diversion of economic rent to his oligarchic supporters through government controls. As a senior Uzbek academic told me privately, there is more repression here now than in Brezhnev's time. The US are trying to prop up Karimov economically and to justify this support they need to claim that a process of economic and political reform is underway. That they do so claim is either cynicism or self-delusion. This policy is doomed to failure. Karimov is driving this resource-rich country towards economic ruin like an Abacha. And the policy of increasing repression aimed indiscriminately at pious Muslims, combined with a deepening poverty, is the most certain way to ensure continuing support for the Islamic Movement of Uzbekistan. They have certainly been decimated and disorganised in Afghanistan, and Karimov's repression may keep the lid on for years - but pressure is building and could ultimately explode. I quite understand the interest of the US in strategic airbases and why they back Karimov, but I believe US policy is misconceived. In the short term it may help fight terrorism but in the medium term it will promote it, as the Economist points out. And it can never be right to lower our standards on human rights. There is a complex situation in Central Asia and it is wrong to look at it only through a prism picked up on September 12. Worst of all is what appears to be the philosophy underlying the current US view of Uzbekistan: that September 11 divided the World into two camps in the "War against Terrorism" and that Karimov is on "our" side. If Karimov is on "our" side, then this war cannot be simply between the forces of good and evil. It must be about more complex things, like securing the long-term US military presence in Uzbekistan. I silently wept at the 11 September commemoration here. The right words on New York have all been said. But last week was also another anniversary - the US-led overthrow of Salvador Allende in Chile. The subsequent dictatorship killed, dare I say it, rather more people than died on September 11. Should we not remember then also, and learn from that too? I fear that we are heading down the same path of US-sponsored dictatorship here. It is ironic that the beneficiary is perhaps the most unreformed of the World's old communist leaders. We need to think much more deeply about Central Asia. It is easy to place Uzbekistan in the "too difficult" tray and let the US run with it, but I think they are running in the wrong direction. We should tell them of the dangers we see. Our policy is theoretically one of engagement, but in practice this has not meant much. Engagement makes sense, but it must mean grappling with the problems, not mute collaboration. We need to start actively to state a distinctive position on democracy and human rights, and press for a realistic view to be taken in the IMF. We should continue to resist pressures to start a bilateral DFID programme, unless channelled non-governmentally, and not restore ECGD cover despite the constant lobbying. We should not invite Karimov to the UK. We should step up our public diplomacy effort, stressing democratic values, including more resources from the British Council. We should increase support to human rights activists, and strive for contact with non-official Islamic groups. Above all we need to care about the 22 million Uzbek people, suffering from poverty and lack of freedom. They are not

just pawns in the new Great Game. MURRAY

Letter #2 Confidential Fm Tashkent (Ambassador Craig Murray) To FCO 18 March 2003 SUBJECT: US FOREIGN POLICY SUMMARY 1. As seen from Tashkent, US policy is not much focussed on democracy or freedom. It is about oil, gas and hegemony. In Uzbekistan the US pursues those ends through supporting a ruthless dictatorship. We must not close our eyes to uncomfortable truth. DETAIL 2. Last year the US gave half a billion dollars in aid to Uzbekistan, about a quarter of it military aid. Bush and Powell repeatedly hail Karimov as a friend and ally. Yet this regime has at least seven thousand prisoners of conscience; it is a one party state without freedom of speech, without freedom of media, without freedom of movement, without freedom of assembly, without freedom of religion. It practices, systematically, the most hideous tortures on thousands. Most of the population live in conditions precisely analogous with medieval serfdom. 3. Uzbekistan's geo-strategic position is crucial. It has half the population of the whole of Central Asia. It alone borders all the other states in a region which is important to future Western oil and gas supplies. It is the regional military power. That is why the US is here, and here to stay. Contractors at the US military bases are extending the design life of the buildings from ten to twenty five years. 4. Democracy and human rights are, despite their protestations to the contrary, in practice a long way down the US agenda here. Aid this year will be slightly less, but there is no intention to introduce any meaningful conditionality. Nobody can believe this level of aid - more than US aid to all of West Africa - is related to comparative developmental need as opposed to political support for Karimov. While the US makes token and low-level references to human rights to appease domestic opinion, they view Karimov's vicious regime as a bastion against fundamentalism. He - and they - are in fact creating fundamentalism. When the US gives this much support to a regime that tortures people to death for having a beard or praying five times a day, is it any surprise that Muslims come to hate the West? 5. I was stunned to hear that the US had pressured the EU to withdraw a motion on Human Rights in Uzbekistan which the EU was tabling at the UN Commission for Human Rights in Geneva. I was most unhappy to find that we are helping the US in what I can only call this cover-up. I am saddened when the US constantly quote fake improvements in human rights in Uzbekistan, such as the abolition of censorship and Internet freedom, which quite simply have not happened (I see these are quoted in the draft EBRD strategy for Uzbekistan, again I understand at American urging). 6. From Tashkent it is difficult to agree that we and the US are activated by shared values. Here we have a brutal US sponsored dictatorship reminiscent of Central and South American policy under previous US Republican administrations. I watched George Bush talk today of Iraq and "dismantling the apparatus of terror… removing the torture chambers and the rape rooms". Yet when it comes to the Karimov regime, systematic torture and rape appear to be treated as peccadilloes, not to affect the relationship and to be downplayed in international fora. Double standards? Yes. 7. I hope that once the present crisis is over we will make plain to the US, at senior

level, our serious concern over their policy in Uzbekistan. MURRAY

[Transcript of facsimile sent 25 March 2003 from the Foreign Office] From: Michael Wood, Legal Advisor Date: 13 March 2003 CC: PS/PUS; Matthew Kidd, WLD Linda Duffield UZBEKISTAN: INTELLIGENCE POSSIBLY OBTAINED UNDER TORTURE 1. Your record of our meeting with HMA Tashkent recorded that Craig had said that his understanding was that it was also an offence under the UN Convention on Torture to receive or possess information under torture. I said that I did not believe that this was the case, but undertook to re-read the Convention. 2. I have done so. There is nothing in the Convention to this effect. The nearest thing is article 15 which provides: "Each State Party shall ensure that any statement which is established to have been made as a result of torture shall not be invoked as evidence in any proceedings, except against a person accused of torture as evidence that the statement was made." 3. This does not create any offence. I would expect that under UK law any statement established to have been made as a result of torture would not be

admissible as evidence. [signed] M C Wood Legal Adviser

Letter #3 CONFIDENTIAL FM TASHKENT (Ambassador Craig Murray) TO IMMEDIATE FCO TELNO 63 OF 220939 JULY 04 INFO IMMEDIATE DFID, ISLAMIC POSTS, MOD, OSCE POSTS UKDEL EBRD LONDON, UKMIS GENEVA, UKMIS MEW YORK SUBJECT: RECEIPT OF INTELLIGENCE OBTAINED UNDER TORTURE SUMMARY 1. We receive intelligence obtained under torture from the Uzbek intelligence services, via the US. We should stop. It is bad information anyway. Tortured dupes are forced to sign up to confessions showing what the Uzbek government wants the US and UK to believe, that they and we are fighting the same war against terror. 2. I gather a recent London interdepartmental meeting considered the question and decided to continue to receive the material. This is morally, legally and practically wrong. It exposes as hypocritical our post Abu Ghraib pronouncements and fatally undermines our moral standing. It obviates my efforts to get the Uzbek government to stop torture they are fully aware our intelligence community laps up the results. 3. We should cease all co-operation with the Uzbek Security Services they are beyond the pale. We indeed need to establish an SIS presence here, but not as in a friendly state. DETAIL 4. In the period December 2002 to March 2003 I raised several times the issue of intelligence material from the Uzbek security services which was obtained under torture and passed to us via the CIA. I queried the legality, efficacy and morality of the practice. 5. I was summoned to the UK for a meeting on 8 March 2003. Michael Wood gave his legal opinion that it was not illegal to obtain and to use intelligence acquired by torture. He said the only legal limitation on its use was that it could not be used in legal proceedings, under Article 15 of the UN Convention on Torture. 6. On behalf of the intelligence services, Matthew Kydd said that they found some of the material very useful indeed with a direct bearing on the war on terror. Linda Duffield said that she had been asked to assure me that my qualms of conscience were respected and understood. 7. Sir Michael Jay's circular of 26 May stated that there was a reporting obligation on us to report torture by allies (and I have been instructed to refer to Uzbekistan as such in the context of the war on terror). You, Sir, have made a number of striking, and I believe heartfelt, condemnations of torture in the last few weeks. I had in the light of this decided to return to this question and to highlight an apparent contradiction in our policy. I had intimated as much to the Head of Eastern Department. 8. I was therefore somewhat surprised to hear that without informing me of the meeting, or since informing me of the result of the meeting, a meeting was convened in the FCO at the level of Heads of Department and above, precisely to consider the question of the receipt of Uzbek intelligence material obtained under torture. As the office knew, I was in London at the time and perfectly able to attend the meeting. I still have only gleaned that it happened. 9. I understand that the meeting decided to continue to obtain the Uzbek torture material. I understand that the principal argument deployed was that the intelligence material disguises the precise source, ie it does not ordinarily reveal the name of the individual who is tortured. Indeed this is true - the material is marked with a euphemism such as "From detainee debriefing." The argument runs that if the individual is not named, we cannot prove that he was tortured. 10. I will not attempt to hide my utter contempt for such casuistry, nor my shame that I work in and organisation where colleagues would resort to it to justify torture. I have dealt with hundreds of individual cases of political or religious prisoners in Uzbekistan, and I have met with very few where torture, as defined in the UN convention, was not employed. When my then DHM raised the question with the CIA head of station 15 months ago, he readily acknowledged torture was deployed in obtaining intelligence. I do not think there is any doubt as to the fact 11. The torture record of the Uzbek security services could hardly be more widely known. Plainly there are, at the very least, reasonable grounds for believing the material is obtained under torture. There is helpful guidance at Article 3 of the UN Convention; "The competent authorities shall take into account all relevant considerations including, where applicable, the existence in the state concerned of a consistent pattern of gross, flagrant or mass violations of human rights." While this article forbids extradition or deportation to Uzbekistan, it is the right test for the present question also. 12. On the usefulness of the material obtained, this is irrelevant. Article 2 of the Convention, to which we are a party, could not be plainer: "No exceptional circumstances whatsoever, whether a state of war or a threat of war, internal political instability or any other public emergency, may be invoked as a justification of torture." 13. Nonetheless, I repeat that this material is useless - we are selling our souls for dross. It is in fact positively harmful. It is designed to give the message the Uzbeks want the West to hear. It exaggerates the role, size, organisation and activity of the IMU and its links with Al Qaida. The aim is to convince the West that the Uzbeks are a vital cog against a common foe, that they should keep the assistance, especially military assistance, coming, and that they should mute the international criticism on human rights and economic reform. 14. I was taken aback when Matthew Kydd said this stuff was valuable. Sixteen months ago it was difficult to argue with SIS in the area of intelligence assessment. But post Butler we know, not only that they can get it wrong on even the most vital and high profile issues, but that they have a particular yen for highly coloured material which exaggerates the threat. That is precisely what the Uzbeks give them. Furthermore MI6 have no operative within a thousand miles of me and certainly no expertise that can come close to my own in making this assessment. 15. At the Khuderbegainov trial I met an old man from Andizhan. Two of his children had been tortured in front of him until he signed a confession on the family's links with Bin Laden. Tears were streaming down his face. I have no doubt they had as much connection with Bin Laden as I do. This is the standard of the Uzbek intelligence services. 16. I have been considering Michael Wood's legal view, which he kindly gave in writing. I cannot understand why Michael concentrated only on Article 15 of the Convention. This certainly bans the use of material obtained under torture as evidence in proceedings, but it does not state that this is the sole exclusion of the use of such material. 17. The relevant article seems to me Article 4, which talks of complicity in torture. Knowingly to receive its results appears to be at least arguable as complicity. It does not appear that being in a different country to the actual torture would preclude complicity. I talked this over in a hypothetical sense with my old friend Prof Francois Hampson, I believe an acknowledged World authority on the Convention, who said that the complicity argument and the spirit of the Convention would be likely to be winning points. I should be grateful to hear Michael's views on this. 18. It seems to me that there are degrees of complicity and guilt, but being at one or two removes does not make us blameless. There are other factors. Plainly it was a breach of Article 3 of the Convention for the coalition to deport detainees back here from Baghram, but it has been done. That seems plainly complicit. 19. This is a difficult and dangerous part of the World. Dire and increasing poverty and harsh repression are undoubtedly turning young people here towards radical Islam. The Uzbek government are thus creating this threat, and perceived US support for Karimov strengthens anti-Western feeling. SIS ought to establish a presence here, but not as partners of the Uzbek Security Services, whose sheer brutality puts them beyond the pale. MURRAY

Tags: politics

HP's restore disk

+1 to HP for using Linux on their diagnostic CD. -1 to HP for suggesting that if poor performance is experienced, you should consider "upgrading the graphics solution". I swear to God, I'm starting to twitch whenever I hear that fucking word.

Tags: rant

NWR04B: Another year older, and what have I done?

Gaw'bless you, Matt Johnson.

A year ago today I mentioned, almost in passing, that I had picked up a cheap wireless router and hoped to get Linux running on it shortly. Since then, I've learned an incredible amount about electronics, reverse-engineering, assembly language, compilers, the Linux kernel, and programming as I moved further up the abstraction ladder. I'm still no expert at any of this, but it astounds me how far I've managed to get along.

Currently I'm stuck at getting flash memory to work -- specifically, being able to erase and then program a chunk of flash memory. The trouble is that the magic numbers that the Linux drivers and the datasheet say are needed don't seem to be working. Previously, I was having the same sort of problem getting the kernel to detect the flash in the first place; the trick was figuring out that GPIO was involved in all this. But I'm doing that same trick now, and it's still not working. As always, I'm not sure what I'm doing wrong.

Still, though, I think I'm going to keep poking at it -- for a while, anyway. My interest is beginning to wane a bit (I flit a lot; a year is a long time to me), plus I got a kid on the way (ack!). I may move on to trying to make all the ethernet interfaces work, not to mention the wireless card, as a way of taking a bit of a break. And of course, I'm still aiming at making the world's first Beowulf cluster of wireless routers.

On another note: today's entry is brought to you by the fine, fine folks at the Free Software Foundation, to whom I've just paid my membership dues for another year. I owe these people a huge amount: not only do I get to use a staggering amount of world-class software, written by their members and with their support, for free (I'm writing this on Emacs right now), not only have I been able to earn a fucking living from what I've taught myself using GPL'd and BSD'd software, RMS has also given us the language to, I dunno, frame the whole question of why this is important: by starting the FSF, by naming the Free Software movement, by giving us the GPL. There are those who disagree, while still cherishing the freedom the FSF seeks -- but I think you'd be hard-pressed to deny the power that one pissed-off geek gained when he got pissed off about some closed-source printer drivers.

(Yes, that may be a big myth -- but that is not the same as being a lie, and the providing^Hsynthesis of motivating myths is important too.)

From their website:

Please support the work of the FSF bymaking a donation,joiningas anassociate member,ordering books and merchandise, or signing your organization up as acorporate patron.

Hate RMS? Fine by me. Give to others:

• The FreeBSD Foundation
• The OpenBSD folks (Don't use SSH? Liar.)
• NetBSD
• KDE
• Gnome
• Mozilla
• Gentoo
• Software in the Public Interest (Debian, among others)
• Slackware

Do it. We owe them.

Tags: nwr04b freeasinfreedom emacs wontyoupleaselendahand

Amanda, restore disks

At work we use Amanda for backups, and it's pretty good -- but for various reasons we don't use the amanda server/client on every single machine. For these exceptions, we point amanda at the host it's running on, where we have copies of the important stuff kept by rsync. This usually works pretty well, and it also fits in well with our other backup mechanism: the copy of yesterday. This is a copy of home directoiries and some other things, updated with rsync every morning at 3am. This gives people an easy way to get something they had yesterday, which means less trips to the backup tapes.

We also do a couple sets of backups with Amanda: daily, where we let Amanda juggle full vs. incremental in the usual way, and weekly/monthly, where we tell Amanda to just do full backups. For those, we just point Amanda at the copy of yesterday, rather than grab full backups over the network.

I've run into problems over the last few weeks, though, where weekly backups have failed for a few home directories -- the fullest ones, natch. It's taken me a while to figure out what's going on, but I think I've got a handle on it finally.

Full weekly/monthly backups take a while to do -- typically two full days, because of non-automated tape changers. While this happens, I let regular backups pile up on the holding disk (close to half a terabyte available), then flush them when the weeklies are done. Here's the error that amstatus shows:

wait for dumping driver: (aborted:nak error:  amandad busy)

Thanks to this post (Nabble? Never heard of 'em...) I finally clued in to the obvious: Amanda sometimes asks the local host for backups twice -- once as part of a daily backup, and once as part of a weekly backup. If this is right (why haven't I come across this more often?) it's going to cause pain. We don't have a tape changer, so backups just plain take a long time; there's no one here at 3am to switch a tape. I'm uncomfortable with the idea of turning off regular backups for two days a week. I really don't want to have to come in on weekends to switch tapes. Hm.

Maybe I'll look at just letting weeklies dump to disk over the weekend, then flush 'em during the week. That might work pretty well, actually.

In other news, got a bunch of HP workstations in from CDW, and I'm quite happy with them. At last count, the company has 879 people starting in January (no, not really) and the idea of setting up that many Shuttles (my usual workstation of choice), manually installing XP (no, no automated install yet) just filled me w/dread. The HPs are nice, very well put together (they're built like fucking tanks and weigh just as much), and they come with XP Pro installed. But hey, manager wanted 160GB drives and these came with 80GB drives. What to do?

Turns out you can take out the old, put in the new (bigger) drive, and just use the restore disk. Boo hiss restore disks with no full copy of the OS, but damn it's nice: very few questions, and when you're done you're ready to go. And by "ready to go" I mean of course "ready to turn off all the crap, turn on other crap and install even more crap". I've either got to swallow my pride and get an AD controller in here (Noooooooo!) or else figure out some other way of automating all this.

Tags: backups

NWR04B: Rebooting, set\_vpp

Up way too late for this sort of thing, but I can't sleep.

Managed to get rebooting working. I'd had a jerry-rigged workaround: use devmem2 to read the magic location, 0x88000004. The weird thing was, that was already in the kernel, at include/asm-armnommu/mach-cx84200/system.h:

extern inline void arch_reset(char mode)
{
    /* REVISIT --gmcnutt */
#define CX84200_RESET                                  0x88000004
    int data;
        data = *(__u8 *)(CX84200_RESET);
}

So what the hell? Just for fun, I tried sticking this in at the end of the function:

    printk (KERN_EMERG "Did that work? data = 0x%08x\n", data);

And sure enough, bam! There I am rebooting. I'm guessing the compiler was optimizing away the read, since it was never used or returned...but that seems like an obvious thing to overlook. Hm. Can't deny that it's working, though. Interestingly, a simple:

       return data;

does not seem to work. The plot thickens.

Also managed to find another chip driver that has to twiddle GPIO in order to write to flash, and it looks like there's a standard place to put this: the set_vpp member (part?) of the map_info structure that is deep, deep at the heart of the MTD driver system. Along with the usual stuff you might expect to find there -- how to read 8 bytes, how to write 16 bytes, and so on -- there is this bit that the Dilnet PC board uses to twiddle GPIO in what looks like a most familiar way. I may manage to soothe baby Linus before long.

Update: Ashtead provided the answer: declare data as volatile, so GCC doesn't optimize away the read.

Tags: nwr04b

NWR04B: Version 0.3 released, or, \"The madness continues\"

Welp, six weeks after upgrading to 2.4.31-uc0, four months after the first release, and nearly a year after getting the goddamned thing in the first place, I'm finally releasing version 0.3 of Linux for the NWR04B. Share and enjoy!

The big change from 0.1 (version 0.2 was never released to my adoring public) is that I've upgraded to the 2.4.31 uClinux kernel, copying over the necessary bits from Codeman's original kernel. This was mainly done in hopes of getting access to the onboard flash memory through the more up-to-date MTD code tree. After a lot of work disassembling the factory firmware for this thing, I finally figured out that one of the GPIO lines? leads? values? needs to be twiddled in order to write to flash successfully. Thus, the onboard flash is being recognized as an AMD-compatable device, which should allow me to erase it and write my own FS there.

Note that I say should. Right now the necessary twiddling is only done at detection time in the kernel, and the code to do so has been rammed in with a crowbar; there is absolutely no grace to this at all. (cfi_probe_chip() in uClinux-2.4.31-uc0/drivers/mtd/chips/cfi_probe.c makes Baby Linus cry.) And when I try changing the necessary bits using devmem2 (included in the tarballs, which I forgot to document) and then erase, the kernel panics. But hey! It's all progress, far as I'm concerned. :-)

Tags: nwr04b