Posts tagged “python”

October 05, 2016 CAN I ROCK BUSINESS WITH YOU?
Today's title from the subject line of some spam I just got. ("a spam"? "a spammy email"? just "spam"?)
- Mystery flu-like illness continues, or at least its fallout; I've had lower back pain for the last ~ 4 weeks. Doctor says removing spine is "not an option" but I've done some Googling and
- $WORK continues apace. After taking a week of Python training, we're using Go for a new tool we're building. Haven't got a good sense for what it's like just yet, but so far I don't seem to be making a mess of things.
- Tried out drone.io at $WORK yesterday and holy god, is it good. Auth with our internal Github, then activate repos, and boom! it runs tests on every new commit on any branch, watches for PRs, the whole nine yards. When I think of the amount of work we had to do to get Jenkins to do this, it's insane. Plus the whole run-as-a-Docker-container, fire-up-sibling-docker-containers-for-tests thing is very, very impressive.
- Sportsball has started up again with a vengeance: practices on Monday and Wednesday, games on Fridays and Saturdays. Somebody stop this merry-go-round!
- I've registered for LISA 16, woot! This will be my fifth -- wait, sixth? -- LISA, ten years after my first time attending. Not sure who's gonna be the theme band this year -- I've done New Pornographers, Josh Rouse, Soul Coughing and Sloan. And since he's co-chair this year, it seems like a good time to pull out that picture of Matt Simmons (@standaloneSA) as a PHP dev:
August 21, 2013 Python, Virtualenv and Tkinter
A coworker was running into problems installing Matplotlib under virtualenv; while running "pip install", he saw this error:
```
/usr/lib64/gcc/x86_64-suse-linux/4.5/../../../../x86_64-suse-linux/bin/ld:
cannot find -ltcl
```
A complication was that he was using Enthough Python; when he used that directly, rather than under virtualenv, it worked just fine.

In the end it turned out to be a long-standing bug in virtualenv. This StackExchange question produced the solution, which is to add these lines to the activate script in the virtualenv:
```
ENTHOUGHT=/path/to/enthought
TK_LIBRARY=${ENTHOUGHT}/lib/python2.7/lib-tk:${ENTHOUGHT}lib/python2.7/site-packages/PIL
TKPATH=${ENTHOUGHT}/lib/python2.7/lib-tk:${ENTHOUGHT}/lib/python2.7/lib-tk
TCL_LIBRARY=${ENTHOUGHT}/lib/tcl8.5/
export TCL_LIBRARY TK_LIBRARY TKKPATH
```
Re-source the activate script, and all is well.
October 09, 2012 Pythonbrew
So it turns out there's a PythonBrew, inspired by PerlBrew. Sweet!
August 27, 2012 Running pip in your home directory
For future reference:
- use pip
- use the "--user" flag"
- Background: [PEP 370], "Per user site-packages directory"
June 06, 2011 How to print character arrays in Python
I've come across this problem a number of times, so here's a reminder. When printing in Python, occasionally I'll end up with something like this:
```
>>> print "foo=%s" % row["Name"]
foo=array('c', 'bar')
```
The solution is to use the .tostring() method:
```
>>> print "foo=%s" % row["Name"].tostring()
foo=bar
```
Details here.
May 26, 2010 Config file parsing
I've been setting up some new VMs for a separate project at work. I've realized that this is painful for two reasons: Bacula and Nagios.

Both are important...can't have a service without monitoring, and can't have a machine without backups. But each of these are configured by vast files; Bacula's is monolithic (the director's, anyhow, which is where you add new jobs) and Nagios' is legion. And they're hard to configure automagically with sed/awk/perl or cfengine; their stanzas span lines, and whitespace is important.

I've recently added a short script to my Nagios config; it regenerates a file that monitors all the Bacula jobs and makes sure they happen often enough. This is good...and I want more.

I found pynag, a Python module to parse and configure Nagios files. This is a start. I've had problems getting its head around my config files, because it didn't understand recursion in hostgroups (which I think is a recent feature of Nagios) or a hostname equal to "*". I've got the first working, and I'm banging my head against the second. The three books I got recently on Python should help (wow, IronPython looks nice).

There are a lot of example scripts with pynag. None do exactly what I want, but it looks like it should be possible to generate Nagios config files from some kind of list of hosts and services. This would be a big improvement.

But then there's Augeas, which does bi-directional parsing of config files. Have a look at the walk-through...it's pretty astounding. I realized that I've been looking for something like this for a long time: an easier way of managing all sorts of config files. Cfengine (v2 to be sure) just isn't cutting it anymore for me.

Now, the problem with Augeas for my present task is that there isn't anything in the current tree that does what I want, either. There is a commit for parsing nagios.cfg -- not sure if it's been released, or if it will parse everything in a Nagios config_dir. There's nothing for Bacula, either. This will mean a lot more work to get my ideal configuration management tool.

(On a side note, my wife said something to me the other day that was quite striking: I need tasks that can be divvied up into 45-minute chunks. That's how much free time I've got in the morning, bus rides to and from work, and the evening. Commute + kids != long blocks of free time.)

And I've got a congenital weakness for grand overarching syntheses of all existing knowledge...or at least big tasks like managing config files. So I'm trying to be aware of my brain.

...and there's son #2 waking up. Time to post.
May 05, 2010 Time well spent
Just spent two hours staring at this Python regex:
```
method_re = re.compile('(?P<urlcmd>\s*url\s.*)|(?P<nfscmd>\s*nfs\s.*)')
```
and trying to figure out why it wouldn't match this line:
```
install url --url http://cobbler.example.org/cobbler/ks_mirror/CentOS-5.4/os/x86_64
```
After much research, I can report two things:
- In python, the match() method of a regular expression object matches from the beginning of the line. If you think in Perl, like I do, it's like if they prepended a ^ to the beginning of the regex.
- (?P<urlcmd> puts the match in the variable urlcmd. That wasn't a problem, I just think it's neat (though obscure).
- A kickstart file, which the line is taken from, is meant to have the install line be separate from the method (in this case, url). I did not know that.
And now to read more of Dr. Zhivago.

Update: by way of explanation, I was trying to use koan and Cobbler to install a new qemu instance like so:
```
/usr/bin/koan --nogfx --server=cobbler.example.com --virt --system=virtserver-01.example.com
```
and was getting two errors:
```
warning: kickstart found but no install_tree found
Cannot find install source in kickstart file, aborting.
```
The problem turned out to be that my kickstart file had this line:
```
install url --url http://cobbler.example.org/cobbler/ks_mirror/CentOS-5.4/os/x86_64
```
which should have been two lines:
```
install
url --url http://cobbler.example.org/cobbler/ks_mirror/CentOS-5.4/os/x86_64
```
April 18, 2010 Sunday blues
I'm at work today. There was a scheduled power outage in the building that holds our server room. It was set for 7am to 11am; I got in at 6am 'cos I'm a keener and wanted to make sure I had lots of time to swap to the backup website.

Power came back on at 10:30 or so. 10:45 I wandered over to the building to see if that was it; didn't want to rush anyone, but thought it'd be worth asking. There was no one there. Sweet, thinks I, let's head down and turn on some machines. (Most are Sun machines and therefore work just fine remotely; some are older IBM machines, where I haven't figured out how to do IPMI over the network, and some are Dell machines where, whee! who knows how it'll work today?)

Turns out A/C's still off. Call the university folks; turns out someone's still working on it. But they're off for lunch, so no idea just yet how long that'll be. I'll be calling back in an hour to see if we have any idea. If they're working on it 'cos something went wrong, well, that's life. If they're working on it 'cos they had scheduled something, then I'm irritated I wasn't told beforehand. Oh well, we'll see what happens. (Update: Our rooftop A/C failed to come on after the power came back on. A full investigation, with twelve helicopters full of determined journalist-engineers, will be launched tomorrow. THIS GOES ALL THE WAY TO THE TOP, PEOPLE.)

In other news, it wasn't a complete waste of time today:
- I found the Python LDAP module and used it to add a bunch of AutoFS entries to the tree. The simplebrowser.py script included in the examples is quite nice. (Though it does make me wonder why I haven't started using one of the billion-and-three LDAP browser tools instead of a) complaining about phpLDAPadmin or b) trying to remember the syntax for ldapsearch).
- I got to wear my safety shoes.
- Downloading Rocks to try installing it on three old servers haning around.
- Discovered that a workstation's hard drive is failing; fortunately it's not needed right away.
- And, erm, that's it.
Loadsoflinks:
- OpenParliament.ca has opened, and it's 'way cool.
- Via my boss comes this detailed legal analysis of the recent US court ruling against gene patents; interesting stuff
- Shades of O_PONIES (via Bob Plankers)
- Alberta privacy commisioner OKs U of A move to Gmail; must read up on this. I heard about this at LISA '09 from a U of A sysadmin
- A down-to-earth, remarkably non-panicky talk (MP3, overview) on Advanced Persistent Threats. (The Loyal Opposition: here, here and especially here)
- How a Sandwich Makes You Its Bitch in 11 Easy Steps
- Gource and Logstalgia
And just one more thing: my younger son turns two on Tuesday. We had an early party:
October 15, 2009 Grabbing Confluence markup from an XML export
As part of a slow migration from Confluence to FosWiki, I had to grab the Confluence markup from an XML dump. I found a Python script to do this, but I think the XML format must have changed in the meantime; the script was unable to grab the body content.

I've lashed together a version available here that works with Confluence 2.10 and Python 2.5 . Now to convert the pages to Foswiki markup...

Ack! Just discovered this updated script in the comments section. Looks like that one grabs a lot more than this does (labels, attachments). Oh well, I needed the practice with Python.