I've tripped over this error a few times; time to write it down.
A few times now, I've run serverspec-init, added a couple tests,
then had the first Rake fail like so:
Circular dependency detected: TOP => default => spec => spec:all =>
spec:default => spec:all
Tasks: TOP => default => spec => spec:all => spec:default
(See full trace by running task with --trace)
Circular dependency detected:
Turns out that this is a known problem in Serverspec, but it's not exactly a bug. The problem appears to be that some part of the Vagrantfile I'm using is named "default". The original reporter said it was the hostname, but I'm not sure I have that in mine. In any case, this causes problems with the Rakefile: the target is default, but that also matches the hostname, and so it's circular and Baby Jesus cries.
(Side rant: I really wish the Serverspec project would use a proper bug tracker, rather than just having everything in pull requests. Grrr.)
One way around this is to change the Rakefile itself. Open it up and look for this part:
namespace :spec do
targets = []
Dir.glob('./spec/*').each do |dir|
next unless File.directory?(dir)
targets << File.basename(dir)
end
task :all => targets
task :default => :all
Comment out that last line, task :default => :all:
namespace :spec do
targets = []
Dir.glob('./spec/*').each do |dir|
next unless File.directory?(dir)
targets << File.basename(dir)
end
task :all => targets
# task :default => :all
Problem solved (though probably in a fairly hacky way...)
I've just upgraded to the latest version of Vagrant, which includes a plugin that lets you use Cfengine as a provisioner. It doesn't seem to be documented right now, so here's my first stab at laying out the options. Apologies for the rough notes.
ampolicyhub: From the source: "Policy hubs need to do additional things before they're ready to accept agents. Force that run now..." Runs "cf-agent -KI -f /var/cfengine/masterfiles/failsafe.cf [classes]", then "cf-agent -KI [classes] [extraagentargs]
extraagentargs: Just what it says.
classes: Define extra classes; appends "-D [class]" args to cf-agent. Multiple classes must be separated by spaces. (or is this a ruby array?)
debrepofile, deprepoline: Specify a deb repo line, to be placed in debrepofile, before running "apt-get install [packagename]". debrepo_file will be clobbered.
filespath: Copy localpath to /var/cfengine using install_files method defined in cfengine/provisioner.rb. Example: you do a git checkout of your repo and want it copied to the machine.
forcebootstrap: Not sure; checked by cfengine/cap/linuxcfengineneeds_bootstrap.rb, but does not appear to do anything. FIXME: See where this module is called from.
install: "force" option seems to be the only poss. value, but not clear what it does. Doesn't seem to be mentioned anywhere else but in provisioner.rb.
mode: Poss values are:
policyserveraddress: Just what it says.
repogpgkey_url: Just what it says.
runfile: Single run if set? Uploads to VM and runs "cf-agent -KI -f [file] [classes] [extraagent_args]".
uploadpath: Where to copy runfile. Default is /tmp/vagrant-cfengine-file.
yumrepofile: Default is /etc/yum.repos.d/cfengine-community.repo. Probably clobbered.
yumrepourl: Default is http://cfengine.com/pub/yum/.
package_name: For use by yum or apt. Default is cfengine-community.
At $WORK I need to package stuff up. Most of the time, I've worked with RPMs. Those are easy; it took, I dunno, an hour? to come up with my first spec file, and it's been pretty simple after that -- even with oddball software that uses "make configure" to mean "make build" and "make" to actually install it. But deb files...man, these are hard. There is a lot more policy built into making a deb file than there ever was in an RPM, and you overlook/override/ignore it at your peril. I want to do the right thing -- which means, since I'm stubborn and have thought idly about becoming a Debian developer someday, I bang my head against the deb files until it works. Except, that as Jordan Sissel has so rightly pointed out, sometimes I just don't have time.
So Jordan Sissel, being Jordan Sissel, has put together fpm, the Effing Package Management. And it's awesome: take your source files, point fpm at them, and you get rpms AND debs. But I want more: the pbuilder system is truly awesome in its you're-gonna-compile-on-an-empty-chroot-or-you're-borked brutish stubbornness, and I want that for fpm. So I'm using vagrant for this. My approach is not as nice as debuild or pbuilder, but it is so far working for me.
It's actually pretty trivial; I'm mainly putting it up here so that I remember it. Hopefully it's useful to someone else, too.
The heart is a dirt-simple shell script; here's what I've got for vmd:
#!/bin/bash
# Bail on error; want this to be as hands-off as possible
set -e
# Add build-essential
sudo apt-get update
sudo apt-get install -y build-essential
# Install fpm. FIXME: That check is borken; not sure about using vagrant_ruby to install it.
which fpm || sudo /opt/vagrant_ruby/bin/gem install fpm
# Where to look for stuff
ORIGIN=/vagrant
INSTALLROOT=/opt
BUILDDIR=/tmp/vmd
TARBALL=vmd-1.9.1.bin.LINUXAMD64.opengl.tar.gz
BUILDDEPS=
# FIXME: gotta put in all the "--depends" by hand.
DEPS="--depends libgl1-mesa-dev --depends libglu1-mesa --depends libxinerama1 --depends libxi6"
[ -d $BUILDDIR ] || mkdir $BUILDDIR
cd $BUILDDIR
tar xvzf ${ORIGIN}/$TARBALL
cd vmd-1.9.1
export VMDINSTALLBINDIR=${INSTALLROOT}/bin
export VMDINSTALLLIBRARYDIR=${INSTALLROOT}/vmd
./configure
sudo make -C src install
# And here's the fpm magic. FIXME: Note the stupid assumptions about opt.
/opt/vagrant_ruby/bin/fpm -s dir -t deb -n vmd -v 1.9.1 -f $DEPS -p ${ORIGIN}/vmd-VERSION_ARCH.deb -x opt/vagrant_ruby -x opt/VBoxGuestAdditions* -C / opt
Drop this in a directory along with the vmd tarball:
-rwxr-xr-x 1 hugh hugh 1153 Sep 12 14:40 build_vmd.sh
-rw-r--r-- 1 hugh hugh 22916955 Sep 12 10:05 vmd-1.9.1.bin.LINUXAMD64.opengl.tar.gz
Vagrant up, then build:
$ vagrant init precise64 # That's Ubuntu 12.04, yo.
$ vagrant up
$ vagrant ssh -- /vagrant/build_vmd.sh
If all goes well, you now have a deb in that directory:
$ ls -l *deb
-rw-rw-r-- 1 hugh hugh 22396738 Sep 12 14:41 vmd-1.9.1_amd64.deb
Like I say, it's dirt-simple -- but it does make stuff a lot easier. Thanks, Mitchell and Jordan!
I wanted to test a new version of Wordpress on $WORK's website, and ran into an interesting set of problems. I figured it would be worth setting them down here.
First: I set up a Vagrant box, got it to forward port 8000 to port 80 on the VM, told Cfengine to set it up as a webserver, then copied over the files and database. Turned out I'd forgotten a few things, like installation of modreverseproxy. I really need to make that into an RPM, especially since it should be pretty trivial, but for now I settled for documenting and scripting my instructions. There were a few other things like that; it's always a good exercise to do this and see what you've left out. Eventually I got it down to a Makefile that I could run on the box itself:
OLD=wordpress-3.4.2
NEW=wordpress-3.5.2
CF=/var/cfengine/bin/cf-agent -f /vagrant/cfengine/masterfiles/promises.cf -KI
go: /var/cfengine/bin /etc/firstrun /www/www.example.com-wordpress /usr/bin/mysql /var/lib/mysql/chibiwp /etc/httpd/modules/mod_proxy_html.so
sudo $(CF)
/var/cfengine/bin:
sudo rpm -ivh cfengine-community-3.3.0-1.x86_64.rpm
/etc/firstrun:
sudo $(CF) -Dinstall_now_please
sudo touch /etc/firstrun
/www/www.example.com-wordpress:
sudo tar -C /www -xvzf /vagrant/wordpress.tgz
/var/lib/mysql/example-wordpress:
mysql -u root -e"create database example-wordpress; grant all on example-wordpress.* to 'wordpress'@'localhost' identified by 's33kr1t'; flush privileges; use example-wordpress; source /vagrant/example-wordpresswp.sql;"
/usr/bin/mysql:
sudo /var/cfengine/bin/cf-agent -f /vagrant/cfengine/masterfiles/promises.cf -KI -Dinstall_now_please
/etc/httpd/modules/mod_proxy_html.so:
tar -C /tmp -xvjf /vagrant/mod_proxy_html.tar.bz2
sudo bash -c 'cd /tmp/mod_proxy_html ; /usr/sbin/apxs -I /usr/include/libxml2 -I . -c -i mod_proxy_html.c'
disable_plugins:
mysql -B -u root chibiwp -e "select option_value from wp_options where option_name='active_plugins';" | sed -e's/^/update wp_options set option_value=QQQ/;s/$/QQQ where option_name="active_plugins";/;' | tail -1 | sed -e"s/QQQ/'/g" > /tmp/restore
mysql -u root chibiwp -e'update wp_options set option_value="a:0:{}" where option_name="active_plugins";'
enable_plugins:
mysql -u root chibiwp < /tmp/restore
unpack_wp:
sudo tar -C /www/www.example.com-wordpress -xvzf /vagrant/wordpress-3.5.2.tar.gz
sudo mv /www/www.example.com-wordpress/wordpress $(NEW)
-sudo rm -r $(OLD)/wp-includes
-sudo rm -r $(OLD)/wp-admin
-sudo mv $(NEW)/wp-includes $(OLD)
-sudo mv $(NEW)/wp-admin $(OLD)
sudo find $(NEW) -type f -maxdepth 1 -exec cp -v {} $(OLD) \;
force_upgrade:
wget "http://localhost/wp-admin/upgrade.php?step=1&backto=%2Fwp-admin%2F"
upgrade: disable_plugins unpack_wp force_upgrade enable_plugins
However, when I browsed to localhost:8000 it tried to redirect me to the actual URL for the work website (http://work.example.com), rather than simply showing me the page and serving it all locally. Turns out this is a known problem, and the solution is to use one of Wordpress' many ways to set the site URL. The original poster used the RELOCATE method, but I had better luck setting the URL manually:
define('WP_HOME','http://localhost:8000');
define('WP_SITEURL','http://localhost:8000');
I can do this manually, but it's better to get Cfengine to do this. First, we have an agent bundle to edit the file:
bundle agent configure_wp_for_vagrant_testing {
files:
vagrantup_com::
"/var/www/wordpress/wp-config.php"
edit_line => vagrant_testing_wpconfig;
}
We specify the lines to add. Rather than install the lines in two passes, which is non-convergent, we add just one line that happens to have an embedded newline:
bundle edit_line vagrant_testing_wpconfig {
insert_lines:
"define('WP_HOME','http://localhost:8000');
define('WP_SITEURL','http://localhost:8000');" location => wp_config_thatsallfolks;
}
(I found that on the Cfengine mailing list, but I've lost the link.) And finally, we specify the location. This depends on having the default comment in wp-config that indicates the end of user-settable vars, but it seems a safe bet:
body location wp_config_thatsallfolks {
select_line_matching => "^/\* That's all, stop editing. Happy blogging. \*\/.*$";
before_after => "before";
}
Second, the production webserver actually hosts a bunch of different sites, and we have separate config files for each of them. Since I was getting Cf3 to configure the VM just as if it was production, the VM got all these config files too. Turned out that browsing to http://localhost:8000 gave me what Apache thought was the default site -- which is the VirtualHost config listed first, which in our case was not our main site. I got around that by renaming our main site's config file to 000-www.example.com.conf (a trick I stole from Debian). Now I could see our main website at http://localhost:8000.
Third, testing: normally I rely on Nagios to do this sort of thing, but it's kind of hard to point it at a VM that might be only around for a few minutes. I could add tests to Cfengine, and that's probably a good idea; however, right now I wanted to try out serverspec, a Ruby-based test suite that lets you verify server attributes.
The serverspec docs say they can run tests on a Vagrant machine, and that all you have to do is tell it so when running "serverspec init." However, I had problems with this; it asked me for a VM name, and I didn't have one...there was only one machine set up, and it didn't seem to like "default". I didn't spend a lot of time on this, but instead went to running the serverspec tests on the Vagrant box itself. That brought its own problems, sinc installing gems in CentOS 5 via the default Ruby (1.8.5) causes buffer overflows. A better person would build a newer RPM, rather than complain about non-schedule repos. However, this Gist does the trick rather nicely (though I also removed the stock Ruby and didn't bother installing Chef).
Okay, so: running "serverspec init" on the Vagrant box created a nice set of default tests for a website. I modified the test for the website config file to look for the right config file and server name:
describe file('/etc/httpd/conf.d/000-www.example.com.conf') do
it { should be_file }
it { should contain "ServerName www.example.com }
end
Over the last two days, in a frenzy of activity, I got some awesome done at work: using git and Vagrant, I finally got Cfengine to install packages in Ubuntu without asking me any goram questions. There were two bits involved:
Telling Apt to use old config files. This prevents it from asking for confirmation when it comes across your already-installed-with-Cfengine config file. Cfengine doesn't do things in a particular order, and in any case I do package installation once an hour -- so I might well have an NTP config file show up before the NTP package itself.
Preseeding, which I've known about for a while but have not had a chance to get right. My summer student came up with a script to do this, and I hope to be able to release it.
Now: Fully automated package installation FTMFW.
And did you know that Emacs can check your laptop battery status? I didn't.