Thing I should have already known

You can configure OpenSSH's ~/.ssh/authorized_keys file to restrict the commands that key is allowed to run via SSH...thus, say, restricting a particular key to running rsync or dump. You can also restrict it to connections only from certain hosts; as the manual points out, this means that "name servers and/or routers would have to be compromised in addition to just the key."