Error: I'm afraid this is the first I've heard of a "robots.txt" flavoured Blosxom. Try dropping the "/+robots.txt" bit from the end of the URL.
Dell OMSA
Following on from my IPMI explorations, here's the next chapter in my getting-down-and-dirty-with-dell-hardware-on-linux adventures. This time I'm setting up Dell's OpenManage Server Administrator software, primarily in order to explore being able to configure bios settings from within the OS. As before, I'm running CentOS 5, but OMSA supports any of RHEL4, RHEL5, SLES9, and SLES10, and various versions of Fedora Core and OpenSUSE.
Here's what I did to get up and running:
# Configure the Dell OMSA repository
wget -O bootstrap.sh http://linux.dell.com/repo/hardware/latest/bootstrap.cgi
# Review the script to make sure you trust it, and then run it
sh bootstrap.sh
# OR, for CentOS5/RHEL5 x86_64 you can just install the following:
rpm -Uvh http://linux.dell.com/repo/hardware/latest/platform_independent/rh50_64/prereq/\
dell-omsa-repository-2-5.noarch.rpm
# Install base version of OMSA, without gui (install srvadmin-all for more)
yum install srvadmin-base
# One of daemons requires /usr/bin/lockfile, so make sure you've got procmail installed
yum install procmail
# If you're running an x86_64 OS, there are a couple of additional 32-bit
# libraries you need that aren't dependencies in the RPMs
yum install compat-libstdc++-33-3.2.3-61.i386 pam.i386
# Start OMSA daemons
for i in instsvcdrv dataeng dsm_om_shrsvc; do service $i start; done
# Finally, you can update your path by doing logout/login, or just run:
. /etc/profile.d/srvadmin-path.sh
Now to check whether you're actually functional you can try a few of the following (as root):
omconfig about
omreport about
omreport system -?
omreport chassis -?
omreport is the OMSA CLI reporting/query tool, and omconfig is the
equivalent update tool. The main documentation for the current version of
OMSA is here.
I found the CLI User's Guide
the most useful.
Here's a sampler of interesting things to try:
# Report system overview
omreport chassis
# Report system summary info (OS, CPUs, memory, PCIe slots, DRAC cards, NICs)
omreport system summary
# Report bios settings
omreport chassis biossetup
# Fan info
omreport chassis fans
# Temperature info
omreport chassis temps
# CPU info
omreport chassis processors
# Memory and memory slot info
omreport chassis memory
# Power supply info
omreport chassis pwrsupplies
# Detailed PCIe slot info
omreport chassis slots
# DRAC card info
omreport chassis remoteaccess
omconfig allows setting object attributes using a key=value syntax, which
can get reasonably complex. See the CLI User's Guide above for details, but
here are some examples of messing with various bios settings:
# See available attributes and settings
omconfig chassis biossetup -?
# Turn the AC Power Recovery setting to On
omconfig chassis biossetup attribute=acpwrrecovery setting=on
# Change the serial communications setting (on with serial redirection via)
omconfig chassis biossetup attribute=serialcom setting=com1
omconfig chassis biossetup attribute=serialcom setting=com2
# Change the external serial connector
omconfig chassis biossetup attribute=extserial setting=com1
omconfig chassis biossetup attribute=extserial setting=rad
# Change the Console Redirect After Boot (crab) setting
omconfig chassis biossetup attribute=crab setting=enabled
omconfig chassis biossetup attribute=crab setting=disabled
# Change NIC settings (turn on PXE on NIC1)
omconfig chassis biossetup attribute=nic1 setting=enabledwithpxe
Finally, there are some interesting formatting options available to both omreport, for use in scripting e.g.
# Custom delimiter format (default semicolon)
omreport chassis -fmt cdv
# XML format
omreport chassis -fmt xml
# To change the default cdv delimiter
omconfig preferences cdvformat -?
omconfig preferences cdvformat delimiter=pipe
IPMI on CentOS/RHEL
Spent a few days deep in the bowels of a couple of datacentres last week, and realised I didn't know enough about Dell's DRAC base management controllers to use them properly. In particular, I didn't know how to mess with the drac settings from within the OS. So spent some of today researching that.
Turns out there are a couple of routes to do this. You can use the Dell
native tools (e.g. racadm) included in Dell's
OMSA product, or you can use
vendor-neutral IPMI,
which is well-supported by Dell DRACs. I went with the latter as it's
more cross-platform, and the tools come native with CentOS, instead of
having to setup Dell's OMSA repositories. The Dell-native tools may give
you more functionality, but for what I wanted to do IPMI seems to work
just fine.
So installation is just:
yum install OpenIPMI OpenIPMI-tools
chkconfig ipmi on
service ipmi start
and then from the local machine you can use ipmitool to access and
manipulate all kinds of useful stuff:
# IPMI commands
ipmitool help
man ipmitool
# To check firmware version
ipmitool mc info
# To reset the management controller
ipmitool mc reset [ warm | cold ]
# Show field-replaceable-unit details
ipmitool fru print
# Show sensor output
ipmitool sdr list
ipmitool sdr type list
ipmitool sdr type Temperature
ipmitool sdr type Fan
ipmitool sdr type 'Power Supply'
# Chassis commands
ipmitool chassis status
ipmitool chassis identify [<interval>] # turn on front panel identify light (default 15s)
ipmitool [chassis] power soft # initiate a soft-shutdown via acpi
ipmitool [chassis] power cycle # issue a hard power off, wait 1s, power on
ipmitool [chassis] power off # issue a hard power off
ipmitool [chassis] power on # issue a hard power on
ipmitool [chassis] power reset # issue a hard reset
# Modify boot device for next reboot
ipmitool chassis bootdev pxe
ipmitool chassis bootdev cdrom
ipmitool chassis bootdev bios
# Logging
ipmitool sel info
ipmitool sel list
ipmitool sel elist # extended list (see manpage)
ipmitool sel clear
For remote access, you need to setup user and network settings, either at boot time
on the DRAC card itself, or from the OS via ipmitool:
# Display/reset password for default root user (userid '2')
ipmitool user list 1
ipmitool user set password 2 <new_password>
# Display/configure lan settings
ipmitool lan print 1
ipmitool lan set 1 ipsrc [ static | dhcp ]
ipmitool lan set 1 ipaddr 192.168.1.101
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 192.168.1.254
Once this is configured you should be able to connect using the 'lan' interface to ipmitool, like this:
ipmitool -I lan -U root -H 192.168.1.101 chassis status
which will prompt you for your ipmi root password, or you can do the following:
echo <new_password> > ~/.racpasswd
chmod 600 ~/.racpasswd
and then use that password file instead of manually entering it each time:
ipmitool -I lan -U root -f ~/.racpasswd -H 192.168.1.101 chassis status
I'm using an 'ipmi' alias that looks like this:
alias ipmi='ipmitool -I lan -U root -f ~/.racpasswd -H'
# which then allows you to do the much shorter:
ipmi 192.168.1.101 chassis status
# OR
ipmi <hostname> chassis status
Finally, if you configure serial console redirection in the bios as follows:
Serial Communication -> Serial Communication: On with Console Redirection via COM2
Serial Communication -> External Serial Connector: COM2
Serial Communication -> Redirection After Boot: Disabled
then you can setup standard serial access in grub.conf and inittab on com2/ttyS1
and get serial console access via IPMI serial-over-lan using the 'lanplus' interface:
ipmitool -I lanplus -U root -f ~/.racpasswd -H 192.168.1.101 sol activate
which I typically use via a shell function:
# ipmi serial-over-lan function
isol() {
if [ -n "$1" ]; then
ipmitool -I lanplus -U root -f ~/.racpasswd -H $1 sol activate
else
echo "usage: sol <sol_ip>"
fi
}
# used like:
isol 192.168.1.101
isol <hostname>
Further reading:
Mocking RPMs on CentOS
Mock is a Fedora project that allows you to build RPM packages within a chroot environment, allowing you to build packages for other systems than the one you're running on (e.g. building CentOS 4 32-bit RPMs on a CentOS 5 64-bit host), and ensuring that all the required build dependencies are specified correctly in the RPM spec file.
It's also pretty under-documented, so these are my notes on things I've figured out over the last week setting up a decent mock environment on CentOS 5.
First, I'm using mock 1.0.2 from the EPEL repository, rather than older 0.6.13 available from CentOS Extras. There are apparently backward-compatibility problems with versions of mock > 0.6, but as I'm mostly building C5 packages I decided to go with the newer version. So installation is just:
# Install mock and python-ctypes packages (the latter for better setarch support)
$ sudo yum --enablerepo=epel install mock python-ctypes
# Add yourself to the 'mock' group that will have now been created
$ sudo usermod -G mock gavin
The mock package creates an /etc/mock directory with configs for various OS
versions (mostly Fedoras). The first thing you want to tweak there is the
site-defaults.cfg file which sets up various defaults for all your builds. Mine now
looks like this:
# /etc/mock/site-defaults.cfg
# Set this to true if you've installed python-ctypes
config_opts['internal_setarch'] = True
# Turn off ccache since it was causing errors I haven't bothered debugging
config_opts['plugin_conf']['ccache_enable'] = False
# (Optional) Fake the build hostname to report
config_opts['use_host_resolv'] = False
config_opts['files']['etc/hosts'] = """
127.0.0.1 buildbox.openfusion.com.au nox.openfusion.com.au localhost
"""
config_opts['files']['etc/resolv.conf'] = """
nameserver 127.0.0.1
"""
# Setup various rpm macros to use
config_opts['macros']['%packager'] = 'Gavin Carr <gavin@openfusion.com.au>'
config_opts['macros']['%debug_package'] = '%{nil}'
You can use the epel-5-{i386,x86_64}.cfg configs as-is if you like; I copied them
to centos-5-{i386,x86_64}.cfg versions and removed the epel 'extras', 'testing',
and 'local' repositories from the yum.conf section, since I typically want to build
using only 'core' and 'update' packages.
You can then run a test by doing:
# e.g. initialise a centos-5-i386 chroot environment
$ CONFIG=centos-5-i386
$ mock -r $CONFIG --init
which will setup an initial chroot environment using the given config. If that seemed to work (you weren't inundated with error messages), you can try a build:
# Rebuild the given source RPM within the chroot environment
# usage: mock -r <mock_config> --rebuild /path/to/SRPM e.g.
$ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/clix-0.3.4-1.of.src.rpm
If the build succeeds, it drops your packages into the /var/lib/mock/$CONFIG/result
directory:
$ ls -1 /var/lib/mock/$CONFIG/result
build.log
clix-0.3.4-1.of.noarch.rpm
clix-0.3.4-1.of.src.rpm
root.log
state.log
If it fails, you can check mock output, the *.log files above for more info, and/or
rerun mock with the -v flag for more verbose messaging.
A couple of final notes:
the chroot environments are cached, but rebuilding them and checking for updates can be pretty network intensive, so you might want to consider setting up a local repository to pull from. mrepo (available from rpmforge) is pretty good for that.
there don't seem to be any hooks in mock to allow you to sign packages you've
built, so if you do want signed packages you need to sign them afterwards via a
rpm --resign $RPMS.
Backup Regimes with Brackup
After using brackup for a while you find
you have a big list of backups sitting on your server, and start to think
about cleaning up some of the older ones. The standard brackup tool for this
is brackup-target, and the prune and gc (garbage collection)
subcommands.
Typical usage is something like this:
# List the backups for a particular target on the server e.g.
TARGET=myserver_images
brackup-target $TARGET list-backups
Backup File Backup Date Size (B)
----------- ----------- --------
images-1262106544 Thu 31 Dec 2009 03:32:49 1263128
images-1260632447 Sun 13 Dec 2009 08:19:13 1168281
images-1250042378 Wed 25 Nov 2009 06:25:06 977464
images-1239323644 Mon 09 Nov 2009 00:30:34 846523
images-1239577352 Thu 29 Oct 2009 13:03:02 846523
...
# Decide how many backups you want to keep, and prune (delete) the rest
brackup-target --keep-backups 15 $TARGET prune
# Prune just removes the brackup files on the server, so now you need to
# run a garbage collect to delete any 'chunks' that are now orphaned
brackup-target --interactive $TARGET gc
This simple scheme - "keep the last N backups" - works pretty nicely for backups you do relatively infrequently. If you do more frequent backups, however, you might find yourself wanting to be able to implement more sophisticated retention policies. Traditional backup regimes often involve policies like this:
It's not necessarily obvious how to do something like this with brackup, but it's actually pretty straightforward. The trick is to define multiple 'sources' in your brackup.conf, one for each backup 'level' you want to use. For instance, to implement the regime above, you might define the following:
# Daily backups
[SOURCE:images]
path = /data/images
...
# Weekly backups
[SOURCE:images-weekly]
path = /data/images
...
# Monthly backups
[SOURCE:images-monthly]
path = /data/images
...
You'd then use the images-monthly source once a month, the images-weekly
source once a week, and the images source the rest of the time. Your list
of backups would then look something like this:
Backup File Backup Date Size (B)
----------- ----------- --------
images-1234567899 Sat 05 Dec 2009 03:32:49 1263128
images-1234567898 Fri 04 Dec 2009 03:19:13 1168281
images-1234567897 Thu 03 Dec 2009 03:19:13 1168281
images-1234567896 Wed 02 Dec 2009 03:19:13 1168281
images-monthly-1234567895 Tue 01 Dec 2009 03:19:13 1168281
images-1234567894 Mon 30 Nov 2009 03:19:13 1168281
images-weekly-1234567893 Sun 29 Nov 2009 03:19:13 1168281
images-1234567892 Sat 28 Nov 2009 03:25:06 977464
...
And when you prune, you want to specify a --source argument, and specify
separate --keep-backups settings for each level e.g. for the above:
# Keep 2 weeks worth of daily backups
brackup-target --source images --keep-backups 12 $TARGET prune
# Keep 8 weeks worth of weekly backups
brackup-target --source images-weekly --keep-backups 8 $TARGET prune
# Keep all monthly backups, so we don't prune them at all
# And then garbage collect as normal
brackup-target --interactive $TARGET gc
Anycast DNS
(Okay, brand new year - must be time to get back on the blogging wagon ...)
Linux Journal recently had a really good article by Philip Martin on Anycast DNS. It's well worth a read - I just want to point it out and record a cutdown version of how I've been setting it up recently.
As the super-quick intro, anycast is the idea of providing a network service at multiple points in a network, and then routing requests to the 'nearest' service provider for any particular client. There's a one-to-many relationship between an ip address and the hosts that are providing services on that address.
In the LJ article above, this means you provide a service on a /32 host address, and then use a(n) (interior) dynamic routing protocol to advertise that address to your internal routers. If you're a non-cisco linux shop, that means using quagga/ospf.
The classic anycast service is dns, since it's stateless and benefits from the high availability and low latency benefits of a distributed anycast service.
So here's my quick-and-dirty notes on setting up an anycast dns server on
CentOS/RHEL using dnsmasq for dns, and quagga zebra/ospfd for the routing.
First, setup your anycast ip address (e.g. 192.168.255.1/32) on a random
virtual loopback interface e.g. lo:0. On CentOS/RHEL, this means you want
to setup a /etc/sysconfig/network-scripts/ifcfg-lo:0 file containing:
DEVICE=lo:0 IPADDR=192.168.255.1 NETMASK=255.255.255.255 ONBOOT=yes
Setup your dns server to listen to (at least) your anycast dns interface.
With dnsmasq, I use an /etc/dnsmasq.conf config like:
interface=lo:0 domain=example.com local=/example.com/ resolv.conf=/etc/resolv.conf.upstream expand-hosts domain-needed bogus-priv
Use quagga's zebra/ospfd to advertise this host address to your internal
routers. I use a completely vanilla zebra.conf, and an /etc/quagga/ospfd.conf
config like:
hostname myhost password mypassword log syslog ! router ospf ! Local segments (adjust for your network config and ospf areas) network 192.168.1.0/24 area 0 ! Anycast address redistribution redistribute connected metric-type 1 distribute-list ANYCAST out connected ! access-list ANYCAST permit 192.168.255.1/32
That's it. Now (as root) start everything up:
ifup lo:0
for s in dnsmasq zebra ospfd; do
service $s start
chkconfig $s on
done
tail -50f /var/log/messages
And then check on your router that the anycast dns address is getting advertised
and picked up by your router. If you're using cisco, you're probably know how to
do that; if you're using linux and quagga, the useful vtysh commands are:
show ip ospf interface <interface>
show ip ospf neighbor
show ip ospf database
show ip ospf route
show ip route
Skype 2.1 on CentOS 5
The new skype 2.1 beta (woohoo - Linux users are now only 2.0 versions behind Windows, way to go Skype!) doesn't come with a CentOS rpm, unlike earlier versions. And the Fedora packages that are available are for FC9 and FC10, which are too recent to work on a stock RHEL/CentOS 5 system.
So here's how I got skype working nicely on CentOS 5.3, using the static binary tarball.
Note that while it appears skype has finally been ported to 64-bit architectures, the
only current 64-bit builds are for Ubuntu 8.10+, so installing on a 64-bit CentOS
box requires 32-bit libraries to be installed (sigh). Otherwise you get the error:
skype: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory.
# the available generic skype binaries are 32-bit, so if you're running a 64-bit
# system you need to make sure you have various 32-bit libraries installed
yum install glib2.i386 qt4.i386 zlib.i386 alsa-lib.i386 libX11.i386 \
libXv.i386 libXScrnSaver.i386
# installing to /opt (tweak to taste)
cd /tmp
wget http://www.skype.com/go/getskype-linux-beta-static
cd /opt
tar jxvf /tmp/skype_static-2.1.0.47.tar.bz2
ln -s skype_static-2.1.0.47 skype
# Setup some symlinks (the first is required for sounds to work, the second is optional)
ln -s /opt/skype /usr/share/skype
ln -s /opt/skype/skype /usr/bin/skype
You don't seem to need pulseaudio installed (at least with the static binary - I assume it's
linked in statically already).
Tangentially, if you have any video problems with your webcam, you might want to check out
the updated video drivers available in the
kmod-video4linux package from the shiny new
ELRepo.org. I'm using their updated uvcvideo module with a Logitech
QuickCam Pro 9000 and Genius Slim 1322AF, and both are working well.
Yum Download SRPMs
Found a nice post today on how to use yum to download source RPMs, rather than having to do a manual search on the relevant mirror.
#pvmove Disk Migrations
Lots of people make use of linux's lvm (Logical Volume Manager) for
providing services such as disk volume resizing and snapshotting under linux.
But few people seem to know about the little pvmove utility, which offers a
very powerful facility for migrating data between disk volumes on the fly.
Let's say, for example, that you have a disk volume you need to rebuild for some reason. Perhaps you want to change the raid type you're using on it; perhaps you want to rebuild it using larger disks. Whatever the reason, you need to migrate all your data to another temporary disk volume so you can rebuild your initial one.
The standard way of doing this is probably to just create a new filesystem on
your new disk volume, and then copy or rsync all the data across. But how
do you verify that you have all the data at the end of the copy, and that
nothing has changed on your original disk after the copy started? If you did
a second rsync and nothing new was copied across, and the disk usage totals
exactly match, and you remember to unmount the original disk immediately,
you might have an exact copy. But if your original disk data is changing at
all, getting a good copy of a large disk volume can actually be pretty tricky.
The elegant lvm/pvmove solution to this problem is this: instead of doing
a userspace migration between disk volumes, you add your new volume into the
existing volume group, and then tell lvm to move all the physical extents
off of your old physical volume, and the migration is magically handled by
lvm, without even needing to unmount the logical volume!
# Volume group 'extra' exists on physical volume /dev/sdc1
$ lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
data extra -wi-ao 100.00G
# Add new physical volume /dev/sdd1 into volume group
$ vgextend extra /dev/sdd1
Volume group "extra" successfully extended
$ lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
data extra -wi-ao 200.00G
# Use pvmove to move physical extents off of old /dev/sdc1 (verbose mode)
$ pvmove -v /dev/sdc1
# Lots of output in verbose mode ...
# Done - remove old physical volume
$ pvremove /dev/sdc1
$ lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
data extra -wi-ao 100.00G
The joys of linux.
#Currency On-Screen Display
Here's a quick hack demonstrating a nice juxtaposition between the
power of a CPAN module - in this case Christopher Laco's
Finance::Currency::Convert::WebserviceX
- and the elegance and utility of the little known osd_cat, putting
together a desktop currency rates widget in a handful of lines:
#!/usr/bin/perl
use strict;
use IO::File;
use Finance::Currency::Convert::WebserviceX;
# Configuration
my @currencies = map { uc } @ARGV || qw(USD GBP);
my $base_currency = 'AUD';
my $refresh = 300; # seconds
my $font = '9x15bold';
# X colours: http://sedition.com/perl/rgb.html
my $colour = 'goldenrod3';
my $align = 'right';
my $pos = 'top';
my $offset = 25;
my $lines = scalar @currencies;
my $osd_refresh = $refresh + 1;
my $osd = IO::File->new(
"|osd_cat -l $lines -d $osd_refresh -c '$colour' -f $font -p $pos -A $align -o $offset"
) or die "can't open to osd_cat $!";
$osd->autoflush(1);
local $SIG{PIPE} = sub { die "pipe failed: $!" };
my $cc = Finance::Currency::Convert::WebserviceX->new;
while (1) {
my $output = '';
$output .= "$_ " . $cc->convert(1, $base_currency, $_) . "\n" for @currencies;
$osd->print($output);
sleep $refresh;
}
Most of this is just housekeeping around splitting out various osd_cat
options for tweaking, and allowing the set of currencies to display
to be passed in as arguments. I haven't bothered setting up any option
handling in order to keep the example short, but that would be
straightforward.
To use, you just run from the command line in the background:
./currency_osd &
and it shows up in the top right corner of your screen, like so:
Tweak to taste, of course.
#Delicious CSS Bookmarklet
Further to my Delicious CSS post, here's a javascript bookmarklet to make adding delicious css tags that much easier:
Just drag it to your bookmarks toolbar somewhere, and click to use.
Unfortunately, the latest version of the delicious tag form doesn't accept
tag arguments in the URL, which is what we need to preset the delicious_css
tags we need. To workaround this, you need to also install the
Auto-Fill Delicious Tag Field
greasemonkey script.
Delicious CSS
And from the quick-weekend-hack department ...
Ever wanted to quickly add a style to a page you were on to make it more usable? If you're a Firefox user with Firebug installed you can do that directly, but it's a temporary and local-only solution. User stylesheets are more permanent, but at least in Firefox (as I've complained before) they're relatively difficult to use, and they're still a single-host solution.
I've wanted a lightweight method of defining and applying user styles on the network for ages now, and this weekend it struck me that a simple and relatively elegant hack would be to just store user styles as delicious tags, applying them to a page via a greasemonkey script.
So here it is: available at userscripts.org is a
relatively trivial Delicious CSS
greasemonkey script. It looks for delicious bookmarks belonging to a list
of specified delicious usernames that are tagged with
delicious_css=<current_domain>,
and applies any 'style tags' it finds on that bookmark to the current
page.
Say if for example you wanted to hide the sidebar on my blog and make the content wider, you might do this in CSS:
div#sidebar { display: none }
div#main { width: 100% }
To define these rules for Delicious CSS you'd just create a bookmark
for www.openfusion.net with the following tags:
delicious_css
delicious_css=www.openfusion.net
div#sidebar=display:none
div#main=width:100%
Note that since delicious tags are space-separated, you have to be careful to avoid spaces.
The general format of the style tags is:
ELT[,ELT...]=STYLE[;STYLE...]
so more complex styles are fine too. Here for example are the styles I'm using for the Sydney Morning Herald:
div.header,div.sidebar,div.aside,div.ad,div.footer=display:none
div#content,div.col1,.span-11=width:100%
body=background:none
which turns this:
into this:
And to setup a new machine, all you need to do is install the Delicious CSS greasemonkey script, adjust the usernames you're trusting, and all your styles are available instantly - nice!
I'll be setting up my userstyles under my 'gavincarr' delicious account, so you should be able to find additional examples at http://delicious.com/gavincarr/delicious_css.
#Missing Delicious Feeds
I've been playing with using delicious as a lightweight URL database lately, mostly for use by greasemonkey scripts of various kinds (e.g. squatter_redirect).
For this kind of use I really just need a lightweight anonymous http interface to the bookmarks, and delicious provides a number of nice lightweight RSS and JSON feeds suitable for this purpose.
But it turns out the feed I really need isn't currently available. I mostly want to be able to ask, "Give me the set of bookmarks stored for URL X by user Y", or even better, "Give me the set of bookmarks stored for URL X by users Y, Z, and A".
Delicious have a feed for recent bookmarks by URL:
http://feeds.delicious.com/v2/{format}/url/{url md5}
and a feed for all a user's bookmarks:
http://feeds.delicious.com/v2/{format}/{username}
and feeds for a user's bookmarks limited by tag(s):
http://feeds.delicious.com/v2/{format}/{username}/{tag[+tag+...+tag]}
but not one for a user limited by URL, or for URL limited by user.
Neither alternative approach is both feasible and reliable: searching by url will only return the most recent set of N bookmarks; and searching by user and walking the entire (potentially large) set of their bookmarks is just too slow.
So for now I'm having to workaround the problem by adding a special
hostname tag to my bookmarks (e.g. squatter_redirect=www.openfusion.net),
and then using the username+tag feed as a proxy for my username+domain
search.
Any cluesticks out there? Any nice delicious folk want to whip up a shiny new feed for the adoring throngs? :-)
#Squatter Domains, Tracked with Delicious
A few weeks ago I hit a couple of domain squatter sites in quick succession and got a bit annoyed. I asked on twitter/identi.ca whether anyone knew of any kind of domain squatter database on the web, perhaps along the lines of the email RBL lists, but got no replies.
I thought at the time that delicious might be useful for this, in much the same way that Jon Udell has been exploring using delicious for collaborative event curation.
So here's the results of some hacking time this weekend: Squatter Redirect, a greasemonkey script (i.e. firefox only, sorry) that checks whether the sites you visit have been tagged on delicious as squatter domains that should be directed elsewhere, and if so, does the redirect in your browser.
Here's a few squatter domains to try it out on:
The script checks two delicious accounts - your personal account, so
you can add your own domains without having to wait for them to be
pulled into the 'official' squatter_redirect stream; and the
official squatter_redirect delicious account, into which other
people's tags are periodically pulled after checking.
Marking a new domain as a squatter simply involves creating a delicious bookmark for the squatter page with a few special tags:
squatter_redirect - flagging the bookmark for the attention of
the squatter_redirect delicious usersquatter_redirect=www.realdomain.com - setting the real domain that you
want to be redirected tosquatter_domain=www.baddomain.com - marker for the
squatter domain itself (only required if you want to use from your own
delicious account)So www.quagga.org above would be tagged:
squatter_redirect squatter_redirect=www.quagga.net
# or, optionally:
squatter_redirect squatter_redirect=www.quagga.net squatter_domain=www.quagga.org
Feedback/comments welcome.
#Quick Linux Box Hardware Overview
Note to self: here's how to get a quick overview of the hardware on a linux box:
perl -F"\s*:\s*" -ane "chomp \$F[1];
print qq/\$F[1] / if \$F[0] =~ m/^(model name|cpu MHz)/;
print qq/\n/ if \$F[0] eq qq/\n/" /proc/cpuinfo
grep MemTotal /proc/meminfo
grep SwapTotal /proc/meminfo
fdisk -l /dev/[sh]d? 2>/dev/null | grep Disk
Particularly useful if you're auditing a bunch of machines (via an ssh loop or clusterssh or something) and want a quick 5000-foot view of what's there.
#ASX20 Announcements Review
Question: you're a small investor with a handful of small share investments in Australian companies listed on the ASX. How do you keep track of the announcements those companies make to the ASX?
There are a couple of manual methods you can use. You can bookmark the announcements page on the various company websites you're interested in and go and check them periodically, but that's obviously pretty slow and labour intensive.
Or you can go to a centralised point, the ASX Announcements Page, and search for all announcements from there. Unfortunately, the ASX only lets you search for one company at a time, so that's also pretty slow, and still requires you do all the checking manually - there's no automation available without subscribing to the ASX's expensive data feed services.
There are also other third-party subscription services you can pay for that will do this for you, but it's annoying to have to pay for what is rightly public information.
The better answer is for the company themselves to provide their announcements through some sort of push mechanism. The traditional method is via email, where you subscribe to company announcements, and they show up in your inbox shortly after they're released.
But the best and most modern solution is for companies to provide a syndication feed on their website in a format like RSS or Atom, which can be monitored and read using feed readers like Google Reader, Mozilla Thunderbird, or Omea Reader. Feeds are superior to email announcments in that they're centralised and lightweight (big companies don't have to send out thousands of emails, for instance), and they're a standardised format, and so can be remixed and repurposed in lots of interesting ways.
So out of interest I did a quick survey of the current ASX20 (the top 20 list of companies on the ASX according to Standards and Poors) to see how many of them support syndicating their announcements either by email or by RSS/Atom. Here are the results:
| Company | via web | via email | via RSS/Atom |
|---|---|---|---|
| AMP |  |
|
 |
| ANZ | |
||
| BHP | |
|
|
| Brambles (BXB) | |
|
|
| Commonwealth Bank (CBA) | |
|
|
| CSL | |
|
|
| Fosters (FGL) | |
||
| Macquarie Group (MQG) | |
|
|
| NAB | |
||
| Newcrest Mining (NCM) | |
||
| Origin Energy (ORG) | |
|
|
| QBE Insurance (QBE) | |
||
| Rio Tinto (RIO) | |
|
|
| Suncorp Metway (SUN) | |
|
|
| Telstra (TLS) | |
|
|
| Wesfarmers (WES) | |
|
|
| Westfield (WDC) | |
|
|
| Westpac (WBC) | |
||
| Woodside Petroleum (WPL) | |
|
|
| Woolworths (WOW) | |
|
|
Some summary ratings:
Overall, I'm relatively impressed that 7 of the ASX20 do support RSS. On the down side, the fact that another 6 don't even provide an email announcements service is pretty ordinary, especially considering the number of retail shareholders who hold some of these stocks (e.g. three of the big four banks, bonus points to CBA, the standout).
Special bonus points go to:
Suncorp Metway and Wesfarmers, who also offer RSS feeds for upcoming calendar events;
Rio Tinto, who have their own announcements twitter account.
Corrections and updates are welcome in the comments.
#The Joy of Scripting
Was going home on the train with Hannah (8) this afternoon, and she says, "Dad, what's the longest word you can make without using any letters with tails or stalks?". "Do you really want to know?", I asked, and whipping out the trusty laptop, we had an answer within a couple of train stops:
egrep -v '[A-Zbdfghjklpqty]' /usr/share/dict/words | \
perl -nle 'chomp; push @words, $_;
END { @words = sort { length($b) cmp length($a) } @words;
print join "\n", @words[0 .. 9] }'
noncarnivorousness
nonceremoniousness
overcensoriousness
carnivorousnesses
noncensoriousness
nonsuccessiveness
overconsciousness
semiconsciousness
unacrimoniousness
uncarnivorousness
Now I just need to teach her how to do that.
#Cityrail Timetables Greasemonkey Script
I got sufficiently annoyed over last week's Cityrail Timetable fiasco that I thought I'd contribute something to the making-Cityrail-bearable software ecosystem.
So this post is to announce a new Greasemonkey script called Cityrail Timetables Reloaded [CTR], available at the standard Greasemonkey repository on userscripts.org, that cleans up and extensively refactors Cityrail's standard timetable pages.
Here's a screenshot of Cityrail's initial timetable page for the Northern line:
and here's the same page with CTR loaded:
CTR loads a configurable number of pages rather than forcing you to click through them one by one, and in fact will load the whole set if you tell it to.
It also has support for you specifying the 'from' and 'to' stations you're travelling between, and will highlight them for you, as well as omit stations well before or well after yours, and any trains that don't actually stop at your stations. This can compress the output a lot, allowing you to fit more pages on your screen:
I can't see Cityrail having a problem with this script since it's just taking their pages and cleaning them up, but we shall see.
If you're a firefox/greasemonkey user please try it out and post your comments/feedback here or on the userscripts site.
Enjoy!
#Soul Communications FAIL!
What's a blog if not a vehicle for an occasional rant?
I used to have a mobile with Soul Communications, and recently changed to another provider because Soul cancelled the plan I'd been on with them for 3 or 4 years. I ported my number, and gathered that that would close the Soul account, and all would be good. Soul has a credit card on that account that they've billed for the last 3 years or so without problems. I've had nothing from them to indicate there are any issues.
And so today I get a Notice of Demand and Disconnection from Soul advising me that my account is overdue, charging me additional debt recovery fees, and advising that if I don't pay all outstanding amounts immediately it'll be referred to debt collectors.
Nice work Soul.
So let's recap. I've had no notices that my account is overdue, no contact from anyone from Soul, no indication that there are any issues, and then a Notice of Demand?
I go and check my email, in case I've missed something. Two emails from Soul since the beginning of the year, the most recent from a week ago. They're HTML-only, of course, and I use a text email client, but hey, I'll go the extra mile and fire up an HTML email client to workaround the fact that multipart/alternative is a bit too hard.
The emails just say, "Your Soul Bill is Now Available", and point to the "MySoul Customer Portal". (Yes, it would be nice if it was a link to the actual bill, of course, rather than expecting me to navigate through their crappy navigation system, but that's clearly a bit too sophisticated as well; but I digress.) There's no indication in any of the emails that anything is amiss, like a "Your account is overdue" message or something. So no particular reason I would have bothered to go and actually login to their portal, find my bill, and review it, right? They've got the credit card, right?
So let's go and check the bill. Go to "MySoul Salvation Portal", or whatever it's called, dig out obscure customer number and sekrit password, and login. Except I can't. "This account is inactive."
Aaaargh!
So let's recap:
account has been cancelled due to move to another carrier (yippee!)
can't login to super-customer-portal to get bills
emails from Soul do not indicate there are any problems with the account
no other emails from the Soul saying "we have a problem"
maybe they could, like, phone my mobile, since they do have the number - no, too hard!
Epic mega stupendous FAIL! What a bunch of lusers.
So now I've phoned Soul, had a rant, and been promised that they'll email me the outstanding accounts. That was half an hour ago, and nothing in the inbox yet. I get the feeling they don't really want to be paid.
And I feel so much better now. :-)
#mod_auth_tkt 2.0.1
I'm happy to announce the release of mod_auth_tkt 2.0.1, the first full release of mod_auth_tkt in a couple of years. The 2.0.x release includes numerous enhancements and bugfixes, including guest login support and full support for apache 2.2.
mod_auth_tkt is a lightweight single-sign-on authentication module for apache, supporting versions 1.3.x, 2.0.x, and 2.2.x. It uses secure cookie-based tickets to implement a single-signon framework that works across multiple apache instances and servers. It's also completely repository agnostic, relying on a user-supplied script to perform the actual authentication.
The release is available as a tarball and various RPMs from the mod_auth_tkt homepage.
#Testing Disqus
I'm trying out disqus, since I like the idea of being able to track/collate my comments across multiple endpoints, rather than have them locked in to various blogging systems. So this is a test post to try out commenting. Please feel free to comment ad nauseum below (and sign up for a disqus account, if you don't already have one).
#