Blog Without An Important Name

I’ve been setting up a Puppet system at work so we can easily set up virtual servers, and also so that all the configuration is in one place.

Yesterday, someone suggested to me how I can have our existing Nagios system monitor the state of the puppet configuration. This allows me to be notified if there is something causing puppet to fail on any of the monitored nodes.

There is a Ruby script out there that is a Nagios plugin, however it requires extra Ruby libraries, and I don’t know how to handle them nicely on a Debian system. I do, however, know how to handle Perl libs. So I wrote a Perl plugin that does the same task. It’s got a lot of hard-coded paths and times and such, you’ll want to make completely sure that they work in your configuration. It’s also not well documented, but it is quite basic: check_puppet.pl

The main tricky dependency it has is on Nagios::Plugin, but it’s in CPAN, so some dh-make-perl should get you a Debian package for it easily.

(Note: files updated for Ubuntu 8.04)
(Another note: this is all unnecessary now, as Hercules has released Linux drivers for their devices. Just install, and away it goes.)

A little while back, I got a Hercules DJ Control MP3, which is an entry-level DJ console that connects to the computer with USB. Unlike most of these kinds of things, this isn’t a MIDI device, so making it work requires a bit of hacking.

Mixxx supports the Herc DJ Control Mk1 and Mk2 just fine, by talking to them directly over USB, unfortunately the Control MP3 requires some magic for this to work that noone has figured out yet. However, the kernel can talk to it using the HID system, which usually is used for keyboards, mice, and other random controllers. The problem with this is that the LEDs don’t work. For some reason, the Linux kernel (as of 2.6) ignores any LEDs that the device says it has if it doesn’t know what they are. With a bit of kernel hacking, this behaviour can be changed, and then Mixxx will use it just fine. If you don’t do this, the controls work, but the LEDs don’t. And we all want blinky LEDs, don’t we? (Also, I used some of the spare LEDs on the controller and made them into a VU meter, which works surprisingly well.)

So to start with: making the kernel work right. I’ve made a patch against the Ubuntu 2.6.22 2.6.24 kernel, here. Or, if you are using Ubuntu 32-bit with this kernel, you can just use these two modules: hid.ko and usbhid.ko. (Files for Ubuntu 7.10 are also available). With the Herc unplugged, do:

sudo rmmod usbhid; sudo rmmod hid; sudo insmod hid.ko; sudo insmod usbhid.ko

(it pays to do them all on one line, as USB keyboards and mice may stop working while the module is unloaded). If these load properly, then you’re half way there.

The next step is to make Mixxx work with this. By default, it only supports the Mk1 and Mk2, as it uses the direct USB way of talking to them. To make it use HID, you need to recompile it, using the current SVN version. How to do this is documented on the Mixxx website, but the compile command needs an extra option, thus:

scons djconsole_legacy=1

The binary that this spits out should handle the LEDs just fine.

This is all very hacky, but hopefully it won’t be in the future. I’m going to see if I can get that patch put in the kernel proper, which will solve the most annoying part of the problem. It would also be good to get Mixxx supporting both legacy mode and the newer one at the same time.

However, we’ve heard from Hercules, and they say that some time in the first part of this year, they’re going to try to get some Linux drivers out. Hopefully they’re open source, and turn the Herc devices into MIDI devices, which would make life a whole lot easier for support.

A few days back I posted this, where I mentioned the lack of wireless access at Christchurch airport. Today, I got a reply saying that they’ll be doing something about that. I’m pleasantly surprised that a company like that (I always think of airports as kinda staid and conservative, that may not actually be the truth) actually monitors ‘blogs, and that they’d take the time to reply.

On the note of wireless, wireless at the conference seems to be a bit intermittent, but I’m not sure if it’s the actual network itself, or my card. I’m suspecting my card more and more. Periodically it seems to just drop out, and to get the drivers to activate it again requires a reboot. It’s a Belkin 54g card, but with a Broadcom chipset and using the bcm43xx drivers. If it’s a real issue, I can try ndiswrapper, but I’d like to not do that if I can get away with it (there is some hope here that someone reading this at the conference will be willing to find me and help me fix it

All my transport for Linux.conf.au 2007 is booked, and there’s quite a comfortable holiday surrounding it all. I have yet to pay for the conference itself, but that’s because they’re having technical problems it seems. (Update: turns out the payment went through fine)

My itinerary is:

• 11/01: Christchurch -> Melbourne
• 14/01: Melbourne -> Sydney, Conference checkin
• 20/01: Conference checkout
• 24/01: Sydney -> Canberra
• 27/01: Canberra -> Brisbane
• 31/01: Brisbane -> Christchurch

Involved in this, are four plane flights with three different airlines, and a train ride from Sydney to Canberra. It should be fun.

There is a lot going on at the conference. I haven’t yet gone through that properly to see what I want to go to. If I’m lucky, I’ll manage to get a laptop for the duration, which will mean I can go to the tutorials (this one on GTK could be interesting, an alternative to using SWT in Java).

Aside from that, there’s a fair number of family and friends to visit in all the places. It seems that half the people I know have ended up in Sydney, and a sizable chunk of the rest scattered around a bit further away.

My credit card is just about smoking from all the money passing through it though…

When I was home over the weekend, I was working on helping my parents sort out the photos they took while they were in Europe. Doing this, I noticed that the date on the camera was out by a month. Being a stickler for correct metadata, I tried to figure out a way to fix this. This is the command line that I came up with. I’ve broken it into multiple lines here for clarity.

for f in *.jpg ; do
    NEWDATE=`exiftool -CreateDate $f |
        perl -pe 's/.*: 2006:([^:]+):/sprintf "2006:%02d:", ($1+1)/e'`;
    echo $f: $NEWDATE;
    exiftool -CreateDate="$NEWDATE" -DateTimeOriginal="$NEWDATE" $f;
done

This will modify the month field in the EXIF data of each file by increasing it by 1. It doesn’t handle wrap around or anything, I didn’t need to do that. The fanciest it gets is making sure there’s the correct number of zeros padding the number.

You don’t want to run this twice.

I spent some time over the weekend getting my parent’s new Epson CX5900 printer/scanner to work. It seems to be either so new or so uncommon that Ubuntu didn’t really know what to do with it. So here I’ll document what I had to do (and then I’ll go file some bug reports so the right people get to see it )

All this was done on Edgy, I upgraded them to that just before trying, in the hope that it would have drivers for it already. no such luck.

This is split into two parts because the functions are in no way related to each other.

The Printer
When you attempt to set the printer up with the Ubuntu printer interface, it detects it correctly, but has no driver for it. Instead, it suggest you use the CX5100 driver. This doesn’t work, instead it spits out blank pages. [Bug report]

After spending a while poking around, I eventually created a PPD file for the CX5800, and loaded that in through the CUPS web interface. It’s quite possible that simply selecting the CX5800 driver would work too. Once I got it going, I didn’t try to change it. Here’s what I did in case it’s necessary. Update: apparently generating the PPD is quite unnecessary, and just picking the CX5800 will do the job just the same. That’s not surprising at all.

Create the PPD file:
cups-genppd.5.0 -v 'Epson Stylus CX5800' -p .
Then add printer in the web interface, and use this driver. Note that I had added the package foomatic-filters-ppds before doing this. It may or may not be necessary.

After this, it seems to work fine. I still don’t know how to make it pick colour or grayscale, a quick glance at the properties box didn’t show anything obvious, but it may be there and I didn’t notice.

The Scanner
Making the scanner work was a bit trickier, but actually took me less time. Credit where it’s due, this thread on the Ubuntu forums was a big help.

First, make sure all the extra sane stuff is installed (libsane, libsane-extras and sane-utils).

Running sane-find-scanner spits out a fair amount of text, but also includes this line, which is key:
found USB scanner (vendor=0x04b8 [EPSON], product=0x082e [USB2.0 MFP]) at libusb:001:003

This gives the USB details of the scanner. Then add them into udev. I put them in /etc/udev/rules.d/45-libsane.rules, but in hindsight, it would be best to put them into a custom rules file so you don’t get messages about upgrades wanting to replace changed files.

This is the line to add:

# Epson CX-5900
SYSFS{idVendor}=="04b8", SYSFS{idProduct}=="082e", MODE="664", GROUP="scanner"

[Bug report]

Note that you can’t add them into /etc/udev/libsane-extras.rules because it’s a symlink, and so the changing file doesn’t get picked up by udev (it’s documented in the readme, so I guess it’s a bug with libsane-extras). It would work after a reboot, but that’s just annoying.[Bug report]

You also need to tell sane about the scanner. This is done simply by adding:
usb 0x04b8 0x082e to the file /etc/sane.d/epson.conf. [Bug report]

To test it, unplug the printer’s USB connection and put it back in, to allow the new rule to be used. Now a ’scanimage -L’ should cause the scanner part to move a bit, and it should report that it’s found something. It seems to detect it as a CX6000 or something.

I’ve finally gotten around to putting all my kallisti.net.nz stuff onto a real server. Well, I guess it’s not a real server, as it doesn’t actually exist anywhere. But it’s a real virtual server, as opposed to the quaint Pentium Celeron 466 that was doing the job.

It’s now hosted by TekTonic, who do a pretty good deal for a fairly minimal unmanaged virtual server. It’s the first one of those listed that I’ve got. The CPU is more than enough for what I’m going to be doing, and the bandwidth is many times better than the 256k/128k that the old server was on. The only catch really is the disk, I have not too far off 10G of stuff to put on it. Of course, I guess I could delete some email or something…

I’m hoping that someone builds a system that lets me use Amazon S3 to store the photos in gallery, that’ll make life a lot easier, as then I won’t have to worry about space on it at all.

Anyway, the migration is still in progress. Jabber.kallisti.net.nz was moved first, as that was pretty easy, and I now have www.kallisti.net.nz on there, except gallery. Now it’s just the other websites that are hosted there, and finally the mail. Mail is going to be the most tricky, as new mail will always be coming in, and it has a fairly complex setup. A curious side-effect of it being split up like this, is that if one webmail user sends an email to another webmail user, it will travel to the US and back up to four times, as it’s stored in NZ, the webmail runs in the US, and then that’s likely to be viewed in NZ again. However, this will become a little more sane soon I hope.

The eventual upshot of this is that for the first time in years, I won’t have to have a computer running in my bedroom 24/7

A few days back I upgraded my machine to the newest Ubuntu release. As is often the case with upgrades like this, there were a few issues. The main one is a conflict between my M2N4-SLI motherboard and the 2.6.17 kernel. I get the impression that Asus has left quite a few bugs in the firmware, and if you boot straight up, it gets so far and then just locks up. My first solution was booting the kernel with the noapic parameter, and that works. However, I was also reminded of this comment that suggest using the enable_8254_timer parameter. I’ll try that next time I reboot.

One of the other problems I have is that Wyrd is completely, totally broken. A pity, because it’s really a nice tool. I suppose I could go back to using straight remind, and scripts that email me reminders. But I like wyrd.

The final thing that’s been annoying is that beagle (the search indexing program) is really prone to sucking up all CPU when it thinks you’re not looking. Originally, it used a lot of CPU indexing some files (including ones that it didn’t understand). I could live with that, it only had to do it once, however now when the computer has been idle for a while it starts up, not doing anything as far as I can tell. At least, nothing is being logged. This wouldn’t be a huge problem, but my computer’s fan only turns on when it’s under load. So if it’s on overnight doing something like downloading, it’ll be completely silent – until beagled starts up, and the computer makes a constant whine.

Oh, something else, but this is not at all a Ubuntu problem, it’s an example of why closed-source software is a pain to work with. What is it with Sun being so backwards about 64-bit machines? The third most requested enhancement is for a 64-bit version of webstart and the browser plugin. Apparently, 64-bit machines are all server machines (despite the fact that Sun sells 64-bit SPARC workstations), and so have no use for plugins and such. So they haven’t gotten around to releasing them, and it looks like they’re not going to be in Java 1.6 either. I’m fairly sure it couldn’t require much more than a recompile to make them work. Hopefully when Java is open sourced, the community will come out and produce these.

In the meantime, I’m living without a Java plugin (I don’t use applets heaps anyway), and OpenJNLP for webstart. It’s old, appears to be unmaintained, but it works. I’m thinking about writing a ‘loading’ screen for it though, currently it doesn’t have one.

Anyway, about Edgy: other than the problems, I haven’t noticed huge differences. Things are a bit newer, a bit nicer. KMail seems a bit less crashy. OpenOffice is finally proper 64-bit, not that I ever use it anyway. Oh, and the spellchecking in Firefox is good, too. I wouldn’t rush to upgrade if I didn’t care about using the most current stuff.

Note: most of this is rambling, go to the bottom for the useful technical bit.

I’m doing some work with large amounts of data (about 100 million rows), and today I tried loading it all into MySQL on my home machine, so that I could run some collaborative filtering tools over it easily. However, when I set this machine up, I did a dumb thing and gave / 4Gb and /home the rest of the space. Thinking of course, that / wouldn’t need much space. It just holds programs and stuff.

I’d forgotten that it’s also where MySQL puts its database files. It also turns out that when MySQL runs out of of disk space, it gets really unhappy and kinda belligerent. You can’t even get it working well enough to do a ‘drop database’ on the over-sized culprit until you free up some space somewhere else.

So, after a bit of playing I got that sorted, and decided to move /var somewhere else. This is a scary proposition, because it means resizing /home (with my 200Gb+ of accumulated cruft). But that’s OK, with careful use of resize_reiserfs and lvreduce (in that order! no exceptions!), I had 10Gb to put a new /var into. This is where it gets tricky. I created the new /var, copied the old stuff over, set the fstab to mount it, etc. I then rebooted to have everything clean up…and mysql wouldn’t start. I eventually tracked that down to a lack of networking.

Here start the useful technical bits, if you’re googling this or something:

However, networking either refused to start, or wouldn’t bring up lo, complaining about a lack of the /var/run/network/ifstate file. Well, true, it wasn’t there, and creating it made things come OK, until the next reboot. On bootup, I was also seeing messages like ’special device /var/run does not exist’. After a bit of poking around the internet, I discovered that /var/run (and /var/lock) have a special filesystem mounted on them before /var is mounted. The upshot of this is that if you don’t have /var/run and /var/lock living on the /var that lies underneath of the mounted /var, things go haywire. Basically, if you don’t have these directories that are invisible 99% of the time your system is running, things break. That is something that really needs to go into a really early startup script.

A nice trick for creating them on a running system (found here) is this command:

mount --bind / /mnt && mkdir /mnt/var/run /mnt/var/lock && umount /mnt

I hope this stops someone else spending an unnecessary couple of hours trying to figure out why networking is failing mysteriously.

I’ve just put out an 0.20 release of eMusic/J. This fixes an issue that appears only on some JVMs (in this case, it seems to be Blackdown) where System.getenv(String) isn’t supported. However, it doesn’t fail gracefully, no…it throws an Error (not just an Exception even).

See these Sun bug reports for details. I think that in Java 1.5 they put it back in, and GCJ seems to handle it OK too, which is why I hadn’t noticed it in my testing. Good thing too, I think removing it to be a very short-sighted decision. It’s more ‘pure’, but so much less practical for real world things that I can’t believe they actually took it out at all.

Users of eMusic/J 0.19 don’t need to upgrade (and won’t be prompted to), nothing else has changed. This is just to help new users.