5.8. Disk management

In respect of the storage units, as we have seen, they have a series of associated devices, depending on the type of interface:

With regard to the partitions, the number that follows the device indicates the partition index within the disk and it is treated as an independent device: /dev/hda1 first partition of the first IDE disk, or /dev/sdc2, second partition of the third SCSI device. In the case of the IDE disks, these allow four partitions, known as primary partitions, and a higher number of logical partitions. Therefore, if /dev/hdan, n is less than or equal to 4, then it will be a primary partition; if not, it will be a logical partition with n being higher than or equal to 5.

With the disks and the associated file systems, the basic processes that we can carry out are included in:

We will now mention two subjects related to the concept of storage space, which are used in various environments for the basic creation of storage space. The use of RAID software and the creation of dynamic volumes.

5.8.1. RAID software

The configuration of disks using RAID levels is currently one of the most widely-used high-availability storage schemes, when we have various disks for implementing our file systems.

The main focus on the different existing techniques is based on a fault-tolerance that is provided from the level of the device and the set of disks, to different potential errors, both physical or in the system, to avoid the loss of data or the lack of coherence in the system. As well as in some schemes that are designed to increase the performance of the disk system, increasing the bandwidth of these available for the system and applications.

Today we can find RAID in hardware mainly in corporate servers (although it is beginning to appear in desktops), where there are different hardware solutions available that fulfil these requirements. In particular, for disk-intensive applications, such as audio and/or video streaming, or in large databases.

In general, this hardware is in the form of cards (or integrated with the machine) of RAID-type disk drivers, which implement the management of one or more levels (of the RAID specification) over a set of disks administered with this driver.

In RAID a series of levels (or possible configurations) are distinguished, which can be provided (each manufacturer of specific hardware or software may support one or more of these levels). Each RAID level is applied over a set of disks, sometimes called RAID array (or RAID disk matrix), which are usually disks with equal sizes (or equal to group sizes). For example, in the case of an array, four 100 GB disks could be used or, in another case, 2 groups (at 100 GB) of 2 disks, one 30 GB disk and one 70 GB disk. In some cases of hardware drivers, the disks (or groups) cannot have different sizes; in others, they can, but the array is defined by the size of the smallest disk (or group).

We will describe some basic concepts on some levels in the following list (it should be remembered that, in some cases, the terminology has not been fully accepted, and it may depend on each manufacturer):

Some points that should be taken into account with regard to RAID in general:

In GNU/Linux, RAID hardware is supported through various kernel modules, associated to different sets of manufacturers or chipsets of these RAID drivers. This permits the system to abstract itself from the hardware mechanisms and to make them transparent to the system and the end user. In any case, these kernel modules allow us to access the details of these drivers and to configure their parameters at a very low level, which in some cases (especially in servers that support a high I/O load) may be beneficial for tuning the disks system that the server uses in order to maximise the system's performance.

The other option that we will analyse is that of carrying out these processes through software components, specifically GNU/Linux's RAID software component.

GNU/Linux has a kernel of the so-called Multiple Device (md) kind, which we can consider as a support through the driver of the kernel for RAID. Through this driver we can generally implement RAID levels 0,1,4,5 and nested RAID levels (such as RAID 10) on different block devices such as IDE or SCSI disks. There is also the linear level, where there is a lineal combination of the available disks (it doesn't matter if they have different sizes), which means that disks are written on consecutively.

In order to use RAID software in Linux, we must have RAID support in the kernel, and, if applicable, the md modules activated (as well as some specific drivers, depending on the case (see available drivers associated to RAID, such as in Debian with modconf). The preferred method for implementing arrays of RAID disks through the RAID software offered by Linux is either during the installation or through the mdadm utility. This utility allows us to create and manage the arrays.

Let's look at some examples (we will assume we are working with some SCSI /dev/sda, /dev/sdb disks... in which we have various partitions available for implementing RAID):

Creation of a linear array:

# mdadm –create –verbose /dev/md0 –level=linear –raid-devices=2 /dev/sda1 /dev/sdb1

where we create a linear array based on the first partitions of /dev/sda and /dev/sdb, creating the new device /dev/md0, which can already be used as a new disk (supposing that the mount point /media/diskRAID exists):

# mkfs.ext2fs /dev/md0 # mount /dev/md0 /media/diskRAID

For a RAID 0 or RAID 1, we can simply change the level (-level) to raid0 or raid1. With mdadm –detail /dev/md0, we can check the parameters of the newly created array.

We can also consult the mdstat entry in /proc to determine the active arrays and their parameters. Especially in the cases with mirrors (for example, in levels 1, 5...) we can examine the initial backup reconstruction in the created file; in /proc/mdstat we will see the reconstruction level (and the approximate completion time).

The mdadm utility provides many options that allow us to examine and manage the different RAID software arrays created (we can see a description and examples in man mdadm).

Another important consideration are the optimisations that should be made to the RAID arrays so as to improve the performance, through both the monitoring of its behaviour to optimise the file system parameters, as well as to use the RAID levels and their characteristics more effectively.

Example 5-15. Note

The optimisation of the RAID arrays, may be an important resource for system tuning and some questions should be examined in:

Software-RAID-Howto, or in the actual mdadm man.

5.8.2. Logical Volume Manager (LVM)

There is a need to abstract from the physical disk system and its configuration and number of devices, so that the (operating) system can take care of this work and we do not have to worry about these parameters directly. In this sense, the logical volume management system can be seen as a layer of storage virtualisation that provides a simpler view, making it simpler and smoother to use.

In the Linux kernel, there is an LVM (logical volume manager), which is based on ideas developed from the storage volume managers used in HP-UX (HP's proprietary implementation of UNIX). There are currently two versions and LVM2 is the most widely used due to a series of added features.

The architecture of an LVM typically consists of the (main) components:

For example, in the following figure, we can see volume groups where we have 7 PVs (in the form of disk partitions, which are grouped to form two logical volumes (which have been completed using /usr and /home to form the file systems):

By using logical volumes, we can treat the storage space available (which may have a large number of different disks and partitions) more flexibly, according to the needs that arise, and we can manage the space by the more appropriate identifiers and by operations that permit us to adapt the space to the needs that arise at any given moment.

Logical Volume Management allows us to:

RAID levels 1 or 5 are not implemented in LVM; if they are necessary (in other words, redundancy and fault tolerance are required), then either we use RAID software or RAID hardware drivers that will implement it and we place LVM as the upper layer.

We will provide a brief, typical example (in many cases, the distributor installer carries out a similar process if we set an LVM as the initial storage system). Basically, we must: 1) create physical volumes (PV). 2) create the logical group (VG) and 3) create the logical volume and finally use the following to create and mount a file system:

1) example: we have three partitions on different disks, we have created three PVs and started-up the contents:

# dd if=/dev/zero of=/dev/hda1 bs=1k count=1 # dd if=/dev/zero of=/dev/hda2 bs=1k count=1 # dd if=/dev/zero of=/dev/hdb1 bs=1k count=1 # pvcreate /dev/hda1 Physical volume "/dev/sda1" successfully created # pvcreate /dev/hda2 Physical volume "/dev/hda2" successfully created # pvcreate /dev/hdb1 Physical volume "/dev/hdb1" successfully created

2) placement of a VG created from the different PVs:

# vgcreate group_disks /dev/hda1 /dev/hda2 /dev/hdb1 Volume group "group_disks" successfully created

3) we create the LV (in this case, with a size of 1 GB) based on the elements that we have in group VG group (-n indicates the name of the volume):

# lvcreate -L1G -n logical_volume group_disks lvcreate -- doing automatic backup of "group_disks" lvcreate -- logical volume "/dev/group_disks/ logical_volume" successfully created

And finally, we create a file system (a ReiserFS in this case):

# mkfs.reiserfs /dev/group_disks/logical_volume

Which we could, for example, place as backup space

# mkdir /mnt/backup # mount -t reiserfs /dev/group_disks/logical_volume /mnt/backup

Finally, we will have a device as a logical volume that implements a file system in our machine.