New disk

At $other_university today adding a new hard drive to our server here: 300GB, instead of 30GB. The users will be very happy. And what with the snow coming down, I'll be very happy if transit keeps running 'til I'm all done.

And now a story about how sometimes it's not all Sympa's fault…

As part of the premptive strike against the mail server's impending failure, I upgraded Sympa (shudder) on $big_server using pkg-src. After copying the list config files over, and pointing it at a separate database, I tried telling it to update its list of subscribers. When I compared a few lists to the already-existing ones, though, they were short random numbers of subscribers. One that I used as a test case, for example, was short about 100 subscribers. That was a concern to me.

The lists that were short all seemed to be ones that grabbed information from the LDAP server, so I tried looking at the queries that Sympa made. The query itself was pretty simple:

(&(objectclass=inetLocalMailRecipient)(gidNumber=10000))

with Sympa being told that mailLocalAddress was the important bit. Should be simple to compare the results from this server and the one where things work…But I lost a good hour of my life, and possibly a couple years of life expectancy, when I became convinced that, somehow, replication to that server was failing, and big chunks of information (like mailLocalAddress) were being lost. Finally, though, I figured out that I'd stupidly been querying the two servers using different credentials. Unfortunately, I figured that out on the way home Friday night.

(Obviously I'd forgotten Aeileen Frisch's rules about system administration:

1. It's a permission problem.
2. If it's not a permission problem, it's a DNS problem.)

Monday, though, was a fresh start and a whole new day. I started by looking again at the queries Sympa made. On $old_server, we'd get about 210 results, with mailLocalAddress in each one of 'em. But on $big_server, we'd get about 210 results, with mailLocalAddress missing in about 100 of them. No wonder Sympa was short.

I double-checked by specifically requesting mailLocalAddress on the problematic server, and it was returned. But $big_server didn't volunteer that information.

I did some digging, and it seems this may be a bug in Sun Directory Server: it should be returning mailLocalAddress as one of the attributes. However, it does not do so for all entries, even when the querying user should have permission to see them. However, I'm unable to see the PR that the thread mentions, since we're not paying for support. (Thank you, Sun.)

Digging into Sympa, though, I found out that this was not the entire reason for the failure. Sympa uses Perl's Net::LDAP module to do its queries. It turns out that Net::LDAP wants a list when you're asking for particular attributes. But in List.pm's _include_users_ldap function, the search is created like so:

$fetch = $ldaph->search ( base => "$ldap_suffix",
                          filter =>; "$ldap_filter",
                          attrs => "$ldap_attrs",
                          scope =>; "$param->{'scope'}");

Changing one line:

$ rcsdiff -r1.1 List.pm
===================================================================
RCS file: RCS/List.pm,v
retrieving revision 1.1
diff -r1.1 List.pm
8646c8646,8647
<                                     attrs => "$ldap_attrs",
---
>                                     # attrs => "$ldap_attrs",
>                                     attrs =>  ["$ldap_attrs"],

meant that, instead of asking for the default attributes (which $big_server was calculating incorrectly), it was asking for mailLocalAddress and succeeding.

And now you know the rest of the story.