Posted by: reformedmusings | December 27, 2008

Ubuntu 8.10 Intrepid and NVidia RAID

My home-built Linux box has NVidia nForce 650i SLI North Bridge and nForce 450i South Bridge chipsets. The 450i supports a number of RAID setups, including RAID 0 (striping), 1 (mirroring), 0+1 (mirrored stripe), and 5 (striped with parity). This type of setup is sometimes called FakeRAID because, although implemented in hardware, it still needs software support to make it work. The Linux driver software is in a program called “dmraid” (for “device mapper RAID”). I found a number of HOWTOs on these types of RAID setups, but didn’t need to use any as Ubuntu did all the heavy lifting for me as you’ll see below. The most complete HOWTO  is this one. It points out that Intrepid makes the setup simple just as I found out, but it also provides a manual approach that’s more complex, plus covers a number of older Ubuntu distributions.

A “pure” hardware RAID is more expensive and is implemented entirely in hardware  so that the operating system sees only one drive. There’s a nice but somewhat dated discussion of hardware RAID in Linux here.

At the other end of the spectrum is software RAID which requires no special hardware. The Linux driver for this is called “mdadm” (for “multiple-drive administration”). There are a number of online tutorials on how to setup a software RAID on Ubuntu. I found Martti Kuparinen’s guide particularly helpful when I first created my software RAID array last year. Martti includes excellent step-by-step screenshots.

The visual difference manifests in the device links they create. mdadm creates “mdx” devices in /dev, with x being replaced with drive numbers (md0, md1, etc.) dmraid creates devices in /dev/mapper like “nvidia.abcdex” where x is the device number (nvidia.abcde1, nvidia.abcde2, etc.)

When I first set up my RAID 1 array, I tried to get Linux to use my NVidia RAID support. Unfortunately, dmraid didn’t recognize the NVidia hardware support at that time. Word is that this was because of the way NVidia stored its RAID metadata. This issue was fixed in Linux kernel version 2.6.27-7 according to this launchpad entry. This caught me by surprise when I installed Ubuntu 8.10 over Kubuntu 8.04. I had the NVidia RAID support disabled in the BIOS and used mdadm  to mirror the drives.

When I first tried to preserve the software array, Ubuntu picked up on the NVidia hardware setup (probably because my earlier failed attempt last year left its NVidia RAID metadata intact on the drive) and tried to implement that with mdadm. That resulted in a degraded array of one disk. Not what I had in mind. So, I set up the BIOS to create the RAID array at the hardware level. When I rebooted, Ubuntu picked up on the hardware array and implemented dmraid automatically. Cool!

For the following description, I had already partitioned my identical hard drives to mirror each other’s partition sizes. You can do this in gparted using systemrescuecd or another tool. More naturally, you should be able to partition the RAID drives during Ubuntu’s alternate CD installation process. I always recommend separate ‘/’ and ‘/home’ partitions so that you can clean install new operating systems without disturbing your data.

For an install, you must use the alternate installation CD to set up a RAID array, whether hardware or software. Select manual partitioning when you get to that point. When the installation scans for drives in the system, it will find the NVidia array, tell you that it detected RAID disks, and ask if you want to activate them. Answer “yes”. On the manual partitioning screen, choose the “nvidia_abcdex” drive name(s) and set them up as you desire (ext file system, partition assignments, size, etc.), then proceed as normal. Ubuntu will take care of the dmraid setup for you.

To check that dmraid has properly picked up on the array after the installation is complete, go to the terminal and type:

dmraid -r

That should produce a list of arrayed disks, like:

/dev/sde: nvidia, “nvidia_figedjie”, mirror, ok, 625142446 sectors, data@ 0
/dev/sdd: nvidia, “nvidia_figedjie”, mirror, ok, 625142446 sectors, data@ 0

If it doesn’t, or produces an error, maybe your array wasn”t started. Try:

dmraid -ay

This should activate all of your arrays. If it doesn’t, then you may need to go back to your BIOS and check your settings. Or, it could be that dmraid doesn’t yet support your chipset. You can find a list of currently supported arrays by going to the terminal and typing:

dmraid -l

which will produce the currently supported hardware list:

asr     : Adaptec HostRAID ASR (0,1,10)
ddf1    : SNIA DDF1 (0,1,4,5,linear)
hpt37x  : Highpoint HPT37X (S,0,1,10,01)
hpt45x  : Highpoint HPT45X (S,0,1,10)
isw     : Intel Software RAID (0,1,10)
jmicron : JMicron ATARAID (S,0,1)
lsi     : LSI Logic MegaRAID (0,1,10)
nvidia  : NVidia RAID (S,0,1,10,5)
pdc     : Promise FastTrack (S,0,1,10)
sil     : Silicon Image(tm) Medley(tm) (0,1,10)
via     : VIA Software RAID (S,0,1,10)
dos     : DOS partitions on SW RAIDs

If your hardware isn’t on the list, you’ll have to go with a software RAID scheme using mdadm.

This post isn’t intended as a tutorial. There are a number of good ones out there depending on which Linux distribution you are using. This post merely described the process which I followed to enable my NVidia RAID 1 array on pre-partitioned disks in Ubuntu 8.10. Ubuntu made the implementation easy in Intrepid.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: