Software RAID and Linux

Part of my day to day activities is to support RedHat Enterprise Linux on the desktop. To demo software that runs natively on Linux, I firmly believe that the only bottleneck should be the system itself: no virtualization, no networking, all self contained.

Recently, I’ve been banging my head with a new deployment of desktops. Whenever I tried to format the internal drive, it would come back and say that it was in use. I could partition it, but not format it. I have never seen this type of issue before.

Some PCs ship with RAID enabled on the motherboard. Most, if not all, PCs do not ship with hardware based RAID.

Hardware based RAID is a beautiful thing. Extra silicon is on the system that manages the configuration and operation of multiple disks, making them all appear as a device that is defined by the system administrator. From the perspective of Linux, it simply looks like disk drive, while it may be many, with redundancy. The computer’s CPU is not bothered with RAID operations, and is left to do other things.

Software based RAID is completely different. It is a RAID system, but still requires the host’s CPU to do work, via a driver, thereby gobbling up resources that may be required elsewhere.

Oddly, in this situation, the solution was in part knowing what the problem was. Most search engine queries resulted in similar problems, but none related to this issue.

[root@localhost ~]# fdisk -l

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          62      497983+  83  Linux
/dev/sda2              63        2495    19543072+  82  Linux swap / Solaris
/dev/sda3            2496      121601   956718945   8e  Linux LVM

Disk /dev/sdb: 20.0 GB, 20014718976 bytes
255 heads, 63 sectors/track, 2433 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Looks fine. But see when I try to make a filesystem on /dev/sda1:

[root@localhost ~]# mke2fs -j -m 0 -L BOOT /dev/sda1
mke2fs 1.39 (29-May-2006)
/dev/sda1 is apparently in use by the system; will not make a filesystem here!

I know the disk is not in use because the system is PXE booted via ethernet.

Looking at the partitions:

[root@localhost ~]# cat /proc/partitions
major minor  #blocks  name

   8     0  976762584 sda
   8     1     497983 sda1
   8     2   19543072 sda2
   8     3  956718945 sda3
   8    16   19545624 sdb
 253     0   19503104 dm-0
 253     1  976759808 dm-1
 253     2     102400 dm-2
 253     3   67108864 dm-3
 253     4    7247872 dm-4
 253     5     102400 dm-5

So I have some devices with major number 253 and minor numbers 0 through 5. Checking to see what else is assigned with these numbers in /dev:

[root@localhost ~]# find /dev | xargs ls -ald | grep 253
brw-rw----  1 root disk  253,    0 Jan 22 01:22 /dev/mapper/isw_cehgfbiihe_Cache
brw-rw----  1 root disk  253,    1 Jan 22 01:22 /dev/mapper/isw_dccbeigfai_CACHEVOL
brw-rw----  1 root disk  253,    2 Jan 22 01:22 /dev/mapper/isw_dccbeigfai_CACHEVOLp1
brw-rw----  1 root disk  253,    3 Jan 22 01:22 /dev/mapper/isw_dccbeigfai_CACHEVOLp2
brw-rw----  1 root disk  253,    4 Jan 22 01:22 /dev/mapper/isw_dccbeigfai_CACHEVOLp3
brw-rw----  1 root disk  253,    5 Jan 22 01:22 /dev/mapper/isw_dccbeigfai_CACHEVOLp4

So the device mapper is involved here. But why? Thankfully, this post shed some light on the situation:

So dmraid is involved. If this is the case, then there is some software RAID involved. Issuing the dmraid command with the -l option (from man page, list all available metadata format handlers with their names and descriptions):

[root@localhost ~]# dmraid -l
asr     : Adaptec HostRAID ASR (0,1,10)
ddf1    : SNIA DDF1 (0,1,4,5,linear)
hpt37x  : Highpoint HPT37X (S,0,1,10,01)
hpt45x  : Highpoint HPT45X (S,0,1,10)
isw     : Intel Software RAID (0,1,5,01)
jmicron : JMicron ATARAID (S,0,1)
lsi     : LSI Logic MegaRAID (0,1,10)
nvidia  : NVidia RAID (S,0,1,10,5)
pdc     : Promise FastTrack (S,0,1,10)
sil     : Silicon Image(tm) Medley(tm) (0,1,10)
via     : VIA Software RAID (S,0,1,10)
dos     : DOS partitions on SW RAIDs

Intel Software RAID appears to be the issue, confirmed by the driver listed in /dev/mapper. Though the storage options were configure as AHCI (not RAID or IDE) in the system’s BIOS, the metadata on the disks still throws Linux into a tizzy.

The solution (and it’s painfully easy):

[root@localhost ~]# dmraid -rE
Do you really want to erase "isw" ondisk metadata on /dev/sda ? [y/n] :y
Do you really want to erase "isw" ondisk metadata on /dev/sdb ? [y/n] :y
[root@localhost ~]# reboot

After the power cycle, the disks are available.

Comments are closed.