Xi6.org

Xenserver doesn’t “really” support fakeraid or software raid.

Fakeraid is that of an onboard or bios type raid. All the processing is done by your CPU and is similar to software raid in that regard. But software raid is handled by the Kernel or OS.

A hardware RAID card is expensive (~$400 additional cost to a system), and is typically proprietary in the way it writes data. So if it does happen to fail, you may be SOL when replacing with a different card – or quite often the same card with a slight variation in the firmware.

These days I would say software raid is comparable in performance to hardware, but can more easily be patched or moved across different systems.

While I would like to use software raid on my xenserver installation (And it is “possible”), it’s just a little bit too much patching/hacking to get working… And there is a bit of uncertainty to what happens when the host is updated to a more recent xenserver version.

Now fakeraid is “semi-supported” on a xenserver installation… meaning you can fake raid a storage repository, but not the entire host. You’ll be better off with some raid than no raid.

You’ll need a minimum of three (3) hard drives for a fakeraid on xenserver. (one will be used to install the host, and the others will be raided for VM storage). So we’re limited to a point of failure of the host hard disk… meaning at worst if that fails you should still be able to access/restore VM’s on another xenserver (or re-install the host on a new HD and still have access to the raided storage when you add it again).

On this installation I’ll be using a 200GB drive for the host. (And may use this drive to store ISO’s and other misc files). And all my virtual machine data will be on the raided drives.

You can setup your bios raid (fakeraid) with 1 drive non-raided (the one that will hold the host), and the other drives in a raid array.

During the xenserver installation you will be asked which drive to install on. All your drives will display (non-raided), so choose the single drive you know isn’t in the array (Mine is SDA). And when asked which will be used as the default storage repository, choose the same single drive or none at all.

Once the host boots up you should get into the console.

Make sure the proper modules are installed:

# insmod /lib/modules/$(uname -r)/kernel/driver/md/dm-mod.ko
# insmod /lib/modules/$(uname -r)/kernel/driver/md/dm-mirror.ko

(This would be for a RAID1 mirror)

Execute the following command in order to init the raid:

# dmraid -ay -v

The raid device should show up as /dev/mapper/RaidVol0 (or whatever you called the raid volume in the bios).

Now we’ll want to create a new SR (storage repository) using the RAID volume:

# sm create /dev/mapper/RaidVol0

In order to have the repo avail on boot in /etc/smtab:

# xe host-sr-set -u root sr-id=rep_uuid active=true

Attach the Storage Repository:

# sm attach <rep_uuid> none

See if its active:

# sm info

If not active, restart smtab and xenagentd:

# service smtab restart
# service xenagentd restart

And that should do it.

Setting up software RAID is “fairly” easy. But first you must install the ALTERNATE installation CD for Ubuntu. (CentOS is pretty straight forward when you select “manual partitioning” in the GUI installer on the regular ISO).

Once you download and burn the alternate ubuntu ISO for your cpu architecture, boot it up on the machine you want to install. (Note: this will erase data on the drives, so make sure no external drives are connected, or any drives in the system you don’t want raided).

During the install, when you come to partitioning, select manual and delete the partitioning tables that currently exist on your drives so that each drive has “FREE SPACE” as the only option under the device.

I’m going to be setting up RAID5, which in my opinion is the best balance between performance and redundancy. (This box is going to be a media server / backup / desktop with three TB drives).

Now for my first device (HD) I’m setting up 3 partitions, (boot, root, and swap). I create the first primary partition of 100MB at the beginning of the drive and set the “use for RAID” where you might normally select ext3, boot, swap etc.  Also be sure to add the boot flag to this partition.

The second partition I setup as logical 6GB and “use for RAID” selected here as well. And the third I set the remainder as a primary partition and “use for RAID”.

I repeat these partitions on the remainder of the 2 drives so each of the 3 drives have identical partition tables.

Next we’re going to select “configure software RAID” that is at the top of the menu, and create a new MD (Multi Disk) device.

Keep in mind we have 3 partitions per disk, totalling 9. So we will have to create 3 MD devices in order to raid each partition.

Creating the first MD device we select RAID1 (mirroring) for the boot partition. This will allow the system to boot properly. (This single raid 1 is very important to our system functioning).

For number of active devices, we type in 3. And 0 spares. Selecting /dev/sda1, /dev/sdb1, and /dev/sdc1 in the device list.

We will repeat creating an MD for each “set” of partitions. (for a total of 3 times). But these subsequent partitions will be RAID5.

You will be presented with RAID MD devices showing up on the partition overview. This is when we will set these partitions to EXT3, SWAP etc…

The 100MB raid1 partition we will set up as EXT3 mounted at /boot. This raid1 will allow the system to boot and not throw a grub error 2 at us.

The 12GB raid partition will be set up as SWAP

And the 2TB partition will be set up as EXT3 mounted at root /

Once we have those set, we finish and write to partition table. It’ll format the drives, sync the arrays and install the OS.

After that’s all said and done it’ll want to reboot. And we’re finished!

After installing software RAID, or perhaps for some other reason – you’re presented with a system that cannot boot properly. In my case, I installed a RAID5 Array and was presented with a GRUB ERROR:2 on boot.

GRUB Error 2 is basically “Selected Disk does not exist“.

Which likely means that grub is pointing to a disk or device that is either not recognized or as stated, doesn’t exist. It may be pointing to (hd0 0,1)  instead of (hd0 0,0) or whichever missconfig. In my case with the newly installed RAID, it’s likely the issue of GRUB being non-existant on subsequent disks, or not mapped properly to the MD array.

To fix this error after a RAID install, I had to go back in with the alternate installation CD and modify my RAID on the /boot partion (100MB).

Instead of this being a RAID5, I changed it to be RAID1 (mirroring) instead. So instead of 1 drive having the pairity value, the 3 drives now have an exact copy of the static boot files. We’re now able to boot our system!

Otherwise you may have to manually edit GRUB and change the device it’s trying to boot. (Such as mentioned above where it may be trying to boot into hd0 0,1 instead of hd0 0,0)