Fri 23 May 2008
Had to setup some simple policy-based routing on CentOS again recently, and had
forgotten the exact steps. So here's the simplest recipe for CentOS that seems
to work. This assumes you have two upstream gateways (gw1 and gw2), and that
your default route is gw1, so all you're trying to do is have packets that come
in on gw2 go back out gw2.
1) Define an extra routing table e.g.
$ cat /etc/sysconfig/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local tables
#
102 gw2
#
2) Add a default route via gw2 (here 172.16.2.254) to table gw2 on the
appropriate interface (here eth1) e.g.
$ cat /etc/sysconfig/network-scripts/route-eth1
default table gw2 via 172.16.2.254
3) Add an ifup-local script to add a rule to use table gw2 for eth1 packets e.g.
$ cat /etc/sysconfig/network-scripts/ifup-local
#!/bin/bash
#
# Script to add/delete routing rules for gw2 devices
#
GW2_DEVICE=eth1
GW2_LOCAL_ADDR=172.16.2.1
if [ $(basename $0) = ifdown-local ]; then
OP=del
else
OP=add
fi
if [ "$1" = "$GW2_DEVICE" ]; then
ip rule $OP from $GW2_LOCAL_ADDR table gw2
fi
4) Use the ifup-local script also as ifdown-local, to remove that rule
$ cd /etc/sysconfig/network-scripts
$ ln -s ifup-local ifdown-local
5) Restart networking, and you're done!
# service network restart
For more, see:
Mon 03 Mar 2008
Find goodness (with a recent-ish find for the '-delete'):
find -L . -type l
find -L . -type l -delete
Fri 28 Sep 2007
We've been chasing a problem recently with trying to use dual
nvidia 8000-series cards with four displays. 7000-series cards
work just fine (we're mostly using 7900GSs), but with 8000-series
cards (mostly 8600GTs) we're seeing an intermittent problem with
one of the displays (and only one) going badly 'fuzzy'. It's not
a hardware problem because it moves displays and cables and
cards.
Turns out it's an nvidia driver issue, and present on the latest
100.14.11 linux drivers. Lonni from nvidia got back to us saying:
This is a known bug ... it is specific to G8x GPUs ... The
issue is still being investigated, and there is not currently
a resolution timeframe.
So this is a heads-up for anyone trying to run dual 8000-series
cards on linux and seeing this. And props to nvidia for getting
back to us really quickly and acknowledging the problem. Hopefully
there's a fix soonish so we can put these lovely cards to use.
Fri 28 Sep 2007
I was building a shiny new CentOS 5.0 server today with a very nice
3ware 9650SE raid card.
Problem #1: the RedHat anaconda installer kernel doesn't support these cards
yet, so no hard drives were detected.
If you are dealing with a clueful
Linux vendor like 3ware, though, you can just go to their comprehensive
download driver page,
grab the right driver you need for your kernel, drop the files onto a
floppy disk, and boot with a 'dd' (for 'driverdisk') kernel parameter
i.e. type 'linux dd' at your boot prompt.
Problem #2: no floppy disks! So the choices were: actually exit the office
and go and buy a floppy disk, or (since this was a kickstart anyway) figure
out how to build and use a network driver image. Hmmm ...
Turns out the dd kernel parameter supports networked images out of the box.
You just specify dd=http://..., dd=ftp://..., or dd=nfs://..., giving it
the path to your driver image. So the only missing piece was putting the
3ware drivers onto a suitable disk image. I ended up doing the following:
# Decide what name you'll give to your image e.g.
DRIVER=3ware-c5-x86_64
mkdir /tmp/$DRIVER
cd /tmp/$DRIVER
# download your driver from wherever and save as $DRIVER.zip (or whatever)
# e.g. wget -O $DRIVER.zip http://www.3ware.com/KB/article.aspx?id=15080
# though this doesn't work with 3ware, as you need to agree to their
# licence agreement
# unpack your archive (assume zip here)
mkdir files
unzip -d files $DRIVER.zip
# download a suitable base image from somewhere
wget -O $DRIVER.img \
http://ftp.usf.edu/pub/freedos/files/distributions/1.0/fdboot.img
# mount your dos image
mkdir mnt
sudo mount $DRIVER.img mnt -o loop,rw
sudo cp files/* mnt
ls mnt
sudo umount mnt
Then you can just copy your $DRIVER.img somewhere web- or ftp- or
nfs-accessible, and give it the appropriate url with your dd kernel
parameter e.g.
dd=http://web/pub/3ware/3ware-c5-x86_64.img
Alternatives: here's an
interesting post
about how to this with USB keys as well, but I didn't end up going that way.
Thu 09 Aug 2007
We've been having a bit of trouble with these motherboards under linux
recently. The two S4/S5 variants are basically identical
except that the S5 has two Gbit ethernet ports where the S4 has only one,
and the S5 has a couple of extra SATA connections - we've been using both
variants. We chose these boards primarily because we wanted AM2 boards
with multiple PCIe 16x slots to use with multiple displays.
We're running on the latest BIOS, and have tested various kernels from 2.6.9
up to about 2.6.19 so far - all evidence the same the same problems. Note
that these are much more likely to be BIOS bugs, we think, than kernel
problems.
The problems we're seeing are:
kernel panics on boot due to apic problems - we can workaround by specifying
a 'noapic' kernel parameter at boot time
problems with IRQ 7 - we get the following message in the messages log
soon after boot:
kernel: irq 7: nobody cared (try booting with the "irqpoll" option)
kernel: [<c044aacb>] __report_bad_irq+0x2b/0x69
kernel: [<c044acb8>] note_interrupt+0x1af/0x1e7
kernel: [<c05700ba>] usb_hcd_irq+0x23/0x50
kernel: [<c044a2ff>] handle_IRQ_event+0x23/0x49
kernel: [<c044a3d8>] __do_IRQ+0xb3/0xe8
kernel: [<c04063f4>] do_IRQ+0x93/0xae
kernel: [<c040492e>] common_interrupt+0x1a/0x20
kernel: [<c0402b98>] default_idle+0x0/0x59
kernel: [<c0402bc9>] default_idle+0x31/0x59
kernel: [<c0402c90>] cpu_idle+0x9f/0xb9
kernel: =======================
kernel: handlers:
kernel: [<c0570097>] (usb_hcd_irq+0x0/0x50)
kernel: Disabling IRQ #7
after which IRQ 7 is disabled and whatever device is using IRQ 7 seems to
fail intermittently or just behave strangely (and "irqpoll" would just
cause hangs early in the boot process).
This second problem has been pretty annoying, and hard to diagnose because it
would affect different devices on different machines depending on what bios
settings were on and what slots devices were in. I spent a lot of time chasing
weird nvidia video card hangs which we were blaming on the binary nvidia
kernel module, which turned out to be this interrupt problem.
Similarly, if it was the sound device that happened to get that interrupt,
you'd just get choppy or garbled sound out of your sound device, when other
machines would be working flawlessly.
So after much pain, we've even managed to come up with a workaround: it turns
out that IRQ 7 is the traditional LPT port interrupt - if you ensure the
parallel port is turned on in the bios (we were religiously turning it off as
unused!) it will grab IRQ 7 for itself and all your IRQ problems just go away.
Hope that saves someone else some pain ...
