Working with Rocks

So we want a cluster at $WORK. I don't know a lot about this, so I figure that something like Rocks or OSCAR is the way to go. OSCAR didn't look like it had been worked on in a while, so Rocks it is. I downloaded the CDs and got ready to install on a handful of old machines.

(Incidentally, I was a bit off-base on OSCAR. It is being worked on, but the last "production-ready" release was version 5.0, in 2006. The newest release is 6.0.5, but as the documentation says:

Note that the OSCAR-6.0.x version is not necessarily suitable for production. OSCAR-6.0.x is actually very similar to KDE-4.0.x: this version is not necessarily "designed" for the users who need all the capabilities traditionally shipped with OSCAR, but this is a good new framework to include and develop new capabilities and move forward. If you are looking for all the capabilities normally supported by OSCAR, we advice you to wait for a later release of OSCAR-6.1.

So yeah, right now it's Rocks.)

Rocks promises to be easy: it installs a frontend, then that frontend installs all your compute nodes. You install different rolls: collections of packages. Everything is easy. Whee!

Only it's not that way, at least not consistently.

• I'm reinstalling this time because I neglected to install the Torque roll last time. In theory you can install a roll to an already-existing frontend; I couldn't get it to work.

• A lot of stuff -- no, that's not true. Some stuff in Rocks is just not documented very well, and it's the little but important and therefore irritating-when-it's-missing stuff. For example: want the internal compute nodes to be LDAP clients, rather than syncing /etc/passwd all around? That means modifying /var/411/Files.mk to include things like /etc/nsswitch and /etc/ldap.conf. That's documented in the 4.x series, but it has been left out of the 5.x series. I can't tell why; it's still in use.

• When you boot from the Rocks install CD, you're directed to type "Build" (to build a new frontend) or "Rescue" (to go into rescue mode). What it doesn't tell you is that if you don't type in something quickly enough, it's going to boot into a regular-looking-but-actually-non-functional CentOS install and after a few minutes will crap out, complaining that it can't find certain files -- instead of either booting from the hard drive or waiting for you to type something. You have to reboot again in order to get another chance.

Right now I'm reinstalling the front end for the THIRD TIME in two days. For some reason, the installation is crapping out and refusing to store the static IP address for the outward-facing interface of the front end. Reinstalling means sitting in the server room feeding CDs (no network installation without an already-existing front end) into old servers (which have no DVD drives) for an hour, then waiting another half hour to see what's gone wrong this time.

Sigh.