Debian PV guest from HVM install...

  • update-grub will not pick up the xen kernels, be very careful with it

  • pygrub in 3.2.1 will not work with reiserfs guest, lilo on debian stable doesn't work with XFS... use EXT3.

  • several config file changes are needed. remove 'ioemu' from vif definitions, for starters.

Start with HVM guest, run through a standard install with EXT3 for root.

  • HVM:
    name = 'samba0.vpr.rpi.edu'
    memory = 2048
    vcpus = 4
    
    on_poweroff = 'destroy'
    on_reboot   = 'restart'
    on_crash    = 'restart'
    
    disk = [ 'phy:/dev/vm1-vg/samba0.vpr-root,ioemu:hda,w',
            'phy:/dev/vm1-vg/samba0.vpr-swap,ioemu:hdb,w']
    #       'file:/home/wickbt/debian-40r3-amd64-netinst.iso,hdc:cdrom,r' ]
    
    # boot on floppy (a), hard disk (c) or CD-ROM (d)
    boot='dc'
    
    sdl=0
    vnc=1
    vncdisplay=1
    
    
    kernel='/usr/lib/xen/boot/hvmloader'
    builder='hvm'
    device_model='/usr/lib/xen/bin/qemu-dm'
    
    vif=['type=ioemu,bridge=eth0,mac=00:01:00:11:df:df']
    

  • PV:
    name = 'samba0.vpr.rpi.edu'
    memory = 2048
    vcpus = 4
    
    on_poweroff = 'destroy'
    on_reboot   = 'restart'
    on_crash    = 'restart'
    
    disk = [ 'phy:/dev/vm1-vg/samba0.vpr-root,ioemu:hda,w',
            'phy:/dev/vm1-vg/samba0.vpr-swap,ioemu:hdb,w']
    #       'file:/home/wickbt/debian-40r3-amd64-netinst.iso,hdc:cdrom,r' ]
    
    # boot on floppy (a), hard disk (c) or CD-ROM (d)
    boot='dc'
    
    sdl=0
    vnc=1
    vncdisplay=1
    
    bootloader='/usr/bin/pygrub'
    
    vif=['bridge=eth0,mac=00:01:00:11:df:df']
    

Lustre/Xen PV guest kernel built from Xen sources

apt-get install dpkg-dev quilt build-essential libncurses5-dev mercurial

apt-get build-dep linux-2.6

cd /root

# get lustre-1.6.5.1.tar.gz into /home/wickbt

cd /root
tar xzf /home/wickbt/lustre-1.6.5.1.tar.gz 

hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg

cd linux-2.6.18-xen.hg

ln -s ../lustre-1.6.5.1/lustre/kernel_patches/series/2.6.18-vanilla.series series
ln -s ../lustre-1.6.5.1/lustre/kernel_patches/patches/ patches

quilt push -av

#may need to copy in a good .config, can swipe from the vm servers if needed. has to have Arcanum.Xen enabled...

make menuconfig
make -j 5
make install
make modules_install
update-initramfs -c -k 2.6.18.8

vi /boot/grub/menu.lst

echo "xc:23:respawn:/sbin/getty 57600 xvc0" >> /etc/inttab
echo xvc0 >> /etc/securetty

halt

Change over config file!

Building Lustre

cd /root/lustre-1.6.5.1
./configure --with-linux=/root/linux-2.6.18-xen.hg

make
make install

depmod -a

modprobe lustre
modprobe lnet networks=tcp

echo "lustre" >> /etc/modules
echo "lnet networks=tcp" >>/etc/modules


Mounting / Test FS

  • MDS/MSS/OSS (samba0) *
    mkdir /mnt/mdt
    mkdir /mnt/ost1
    mkdir /mnt/ost2
    
    mount -t lustre -o loop /f1 /mnt/mdt
    
    mount -t lustre -o loop /f2 /mnt/ost1
    mount -t lustre -o loop /big /mnt/ost2
    
    
    
    

* OSS (samba1) *
mkdir /mnt/ost1
mkdir /mnt/ost2

mount -t lustre -o loop /f1 /mnt/ost1
mount -t lustre -o loop /big /mnt/ost2

References

Samba

  542  mv /usr/lib/libtdb.so /usr/lib/libtdb.so.1
  544  mv /usr/lib/libtalloc.so /usr/lib/libtalloc.so.1
  547  mv /usr/lib/libwbclient.so /usr/lib/libwbclient.so.0

 ctdbd -i --reclock=/storage/lock --logfile=/tmp/ctdblog --public-addresses=/etc/ctdb/public_addresses --nlist=/etc/ctdb/nodes &

Random Performance Notes

Difference in find performance between lustre's lfs find and normal find.

risk:~ time lfs find --type f --atime +60 --mtime +60 --ctime +60 /lustre/

real    3m18.206s
user    0m43.687s
sys     0m21.189s

risk:~ time find /lustre/ -type f -atime +60 -mtime +60 -ctime +60

real    7m47.554s
user    0m3.284s
sys     2m25.289s
Topic revision: r8 - 06 Mar 2009, TimWickberg
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback