Posts tagged “cacti”

December 20, 2012 Post-Maintenance Fallout
Two things bit me after doing big patching yesterday.

First, Cacti's logs suddenly exploded with a crapton of errors like this:
```
12/20/2012 03:41:41 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!,
Error:'1146', SQL:"SELECT 1 AS id, ph.name, ph.file, ph.function FROM
plugin_hooks AS ph LEFT JOIN plugin_config AS ...
```
and on it went. The problem: Cacti got upgraded, but I forgot to run the upgrade step.

Second, LDAP Replication stopped working. The single master (multi-master replication is for people who don't get enough pain in their lives already) suddenly stopped, with terribly uninformative log messages like:
```
NSMMReplicationPlugin - Replication agreement for agmt="cn=eg-02" (eg-02:636) could not be updated. For replication to take place, please enable the suffix and restart the server
```
Forcing initialization didn't work, and neither did recreating the agreement; that got me this error:
```
agmtlist_add_callback: Can't start agreement "cn=eg-02,cn=replica,cn=dc\example\2c dc\3dcom,cn=mapping tree,cn=config"
```
But that log message did hold the key. As described here, 389/CentOS/Fedora DS/RHDS switched to a new DN format. And near as I can figure, either some upgrade step didn't work or it simply wasn't there in the first place.

The solution: Shut down the server. Edit dse.ldif and change
```
cn=eg-02,cn=replica,cn=dc\example\2c dc\3dcom,cn=mapping tree,cn=config
```
to:
```
cn=eg-02,cn=replica,cn=dc\example\2cdc\3dcom,cn=mapping tree,cn=config
```
- Notice the space that just went away. NOTICE IT.
- Now restart the server.
- I also deleted the replication agreements and recreated them; not sure if that was strictly necessary, but there you go.
UPDATE: Nope, the problem recurred, leading to this amusing return from the Nagios plugin:
```
UNKNOWN - WTF is return code 'ERROR'???
```
In unrelated news, I have now switched to keeping account information in flat files distributed by rcp. Replication agreements are for the fucking birds.

SECOND UPDATE: A second re-initialization of the client fixed the problem. In still yet unrelated news, I've submitted a patch to the Linux folks to eliminate UIDs entirely.