RAID/LVM on Gentoo

Here are some notes I've made whilst building my NAS server. The server is based on Gentoo Linux and uses software RAID5, LVM and the XFS file system from Silicon Graphics.

First, install the RAID tools (mdadm), LVM (lvm2) and Samba (samba).

root@server:~

login as: root
Using keyboard-interactive authentication.
Password:
Last login: Sun Apr  1 23:38:46 2007 from 192.168.1.1
server ~ #emerge -va mdadm lvm2 samba_

Next we need to remove all existing partitions on each of the disks we want to use (in my case 5 x 320Gb PATA)

root@server:~

server ~ #fdisk /dev/hda_

The keystrokes we want to use are:

d
Delete partition, this should remove the partition if there was previously one one there. If there's more than one partition, specify the partition number and repeat for each partition on the drive
n
New partition
p
Primary
1
Partition number 1
Enter
Start at the first cylinder
Enter
End at the last cylinder (use the whole drive)
t
Change partition type
fd
Linux RAID auto
p
Print the partition table (check that all looks okay)
w
Write the new partition table and exit

You can also use 'p' within fdisk to print (to screen) the partition table, 'm' to show a list of options and 'l' to show a list of partition types. Repeat this step for each drive.

Next we zero the superblock of all our drives, which wipes all info from any previous RAID installs we've done on the drives.

root@server:~

server ~ #mdadm --zero-superblock /dev/hda1_

Repeat the above step for each drive we're using

Now we create the RAID array. In my case, I created the array as a 5 disk array with one disk missing (this disk had data on it at the time). This creates a degraded RAID5 array, and the last disk can be added later.

root@server:~

server ~ #mdadm --create --verbose /dev/md0 --level=5 --raid-devices=5 --chunk=512 /dev/hd[cegi]1 missing_

We can see in the line above that we're using verbose mode so that we can see what's going on, creating the device at /dev/md0, creating the array with RAID level 5, using 5 disks, setting a chunk size of 512Kb and using partitions hdc1, hde1, hdg1, hgi1 and 'missing', which denotes that one of our drives is not present (the one that still has data on it). Not the shorthand way that we can identify the drives using square brackets. Also note that our drives are only every other letter (c, e, g, i and eventually k). This is because I've limited the drives to one per IDE channel, to improve performance.

Now we can grab the details of our new RAID array and load them into a config file for mdadm to use in the future

root@server:~

server ~ #mdadm --detail --scan > /etc/mdadm.conf_

We can now edit this file to provide an email address to be mailed if one of the drives fails.

root@server:~

server ~ #nano /etc/mdadm.conf_

The file should look like the following:

DEVICE          /dev/hd[cegi]1
ARRAY           /dev/md0 level=raid5 num-devices=5 devices=/dev/hd[aceg]1 missing
MAILADDR        mark@honeychurch.org
DEVICE /dev/hd[cegik]1 ARRAY /dev/md0 level=raid5 num-devices=5 devices=/dev/hd[acegm]1 MAILADDR mark@honeychurch.org vgscan vgchange -a y pvcreate /dev/md0 pvdisplay vgcreate -s 32M vg0 /dev/md0 vgdisplay vg0 vgdisplay vg0 | grep "Total PE" lvcreate -l 38155 vg0 -n lv0 lvdisplay /dev/vg0/lv0 mkfs.xfs -d su=512k,sw=4 /dev/vg0/lv0 mount /dev/vg0/lv0 /raid /dev/vg0/lv0 /raid xfs defaults 0 0 mdadm /dev/md0 -a /dev/hdm1 mdadm --detail /dev/md0 mknod /dev/md0 b 9 0 mdadm --assemble /dev/md0 /dev/hd[aceg]1 mdadm --assemble /dev/md0 /dev/hd[acegm]1 mdadm --assemble --scan /dev/hd[aceg]1 mdadm --assemble --scan /dev/hd[acegm]1 vgscan vg0 vgchange -a y vg0 mkdir /raid mount /dev/vg0/lv0 /raid umount /dev/vg0/lv0 lvremove /dev/vg0/lv0 vgchange -a n vg0 vgremove vg0 Defragmenting an XFS Volume

My next filer, which I'll purchase and build once the price of 1Tb drives has stabilised, will have 4 x 1Tb Western Digital Caviar SATA drives, possibly an ST Lab A-224 4 channel SATA controller (as there seem to be no cheap SATA 3Gb cards available at the moment), and will run on Openfiler. Hopefully, Openfiler 2.3 will be out by then, although I'm not holding my breath!