Introduction

The lack of available disk storage frequently plagues Linux systems administrators. The most
common reasons for this are expanding databases, increasing numbers of users, and the larger
number of tasks your Linux server is expected to perform until a replacement is found.
This chapter explores how to add a disk to a Linux system in two ways. The first is by moving
directories from a full partition to an empty one made available by the new disk and then linking
the directory structures of the two disks together. The second is by merging the partitions
together to create a combined partition using the Linux Logical Volume Manager (LVM.

Debian / Ubuntu Differences

This chapter focuses on Fedora / CentOS / RedHat for simplicity of explanation. Whenever there
is a difference in the required commands for Debian / Ubuntu variations of Linux it will be
noted.
The universal difference is that the commands shown are done by the Fedora / CentOS / RedHat
root user. With Debian / Ubuntu you will either have to become root using the "sudo su "
command or you can temporarily increase your privilege level to root using the "sudo
<command>" command.
Here is an example of how to permanently become root:
user@ubuntu:~$ sudo su [sudo] password for peter:
root@ubuntu:~#

Here is an example of how to temporarily become root to run a specific command. The first
attempt to get a directory listing fails due to insufficient privileges. The second attempt succeeds
when the sudo keyword is inserted before the command.
user@ubuntu:~$ ls -l /var/lib/mysql/mysql
ls: cannot access /var/lib/mysql/mysql: Permission denied
user@ubuntu:~$ sudo ls -l /var/lib/mysql/mysql
[sudo] password for peter:
total 964
-rw-rw---- 1 mysql mysql
8820 2010-12-19 23:09 columns_priv.frm
-rw-rw---- 1 mysql mysql
0 2010-12-19 23:09 columns_priv.MYD
-rw-rw---- 1 mysql mysql
4096 2010-12-19 23:09 columns_priv.MYI
-rw-rw---- 1 mysql mysql
9582 2010-12-19 23:09 db.frm
...
...
...
user@ubuntu:~$

Now that you have got this straight, lets continue with the discussion.

Adding Disks To Linux

At some stage you'll be faced with the task of installing an additional hard drive into your Linux
server. Perhaps an existing device failed, or maybe you ran out of available space. To provide
more space, this section will cover adding a hard disk with only one partition and will then
explain how to migrate data from the full disk to the new one.

Scenario
Things are getting crowded on bigboy: Even after you removed all unwanted data, the /var
partition is full. You need to add a new hard drive to the system. You can verify this situation
with the df -k command's output, which also shows that the other partitions are too full to accept
any more data.
[root@bigboy tmp]# df -k
Filesystem
1K-blocks
/dev/hda3
505636
/dev/hda1
101089
none
63028
/dev/hda5
248895
/dev/hda7
3304768
/dev/hda2
3304768
[root@bigboy tmp]#

Used Available Use% Mounted on

118224
361307 25% /
14281
81589 15% /boot
0
63028
0% /dev/shm
6613
229432
3% /tmp
2720332
416560 87% /usr
3300536
4232 99% /var

A new hard disk was added according to the manufacturer's instructions, but you now need to
know how to proceed.

Determining The Disk Types

Linux stores the names of all known disk partitions in the /proc/partitions file. The entire hard
disk is represented by an entry with a minor number of 0, and all the partitions on the drive are
sequentially numbered after that. In the example, the system has two hard disks; disk /dev/hda
has been partitioned, but the new disk (/dev/hdb) needs to be prepared to accept data.
[root@bigboy tmp]# cat /proc/partitions
major minor #blocks name
3
0
7334145
3
1
104391
3
2
1052257
3
3
2040255
3
4
1
3
5
3582463
3
6
554211
22
0
78150744
[root@bigboy tmp]#

hda
hda1
hda2
hda3
hda4
hda5
hda6
hdb

Note: Linux hard disk device names follow a specific standard. SCSI disks all start with sd and
IDE disks with hd. After this comes a letter that identifies the unit number of the disk, so for
example, the first disk would be a, the second would be b, the third would be c, and so on.
Finally, a two-digit number defines the partition number. Using this convention the fifth partition
on the fourth IDE drive would be /dev/hdd5.

Preparing Partitions on New Disks

Linux partition preparation is very similar to that in a Windows environment, because both
operating systems share the fdisk partitioning utility. The steps are:
1) The first Linux step in adding a new disk is to partition it in preparation of adding a filesystem
to it. Type the fdisk command followed by the name of the disk. You want to run fdisk on the
/dev/hdb disk, so the command is:
[root@bigboy tmp]# fdisk /dev/hdb
The number of cylinders for this disk is set to 9729.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help):

2) Just to make sure you're on the correct device, issue the p command to print all the known
partitions on the disk. In this case, there are none which is good.
Command (m for help): p
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot

Start

End

Blocks

Id

System

Command (m for help):

3) The fdisk m command prints a small help manual of valid commands. You will see that n is
the command to add a new partition. Add a new primary partition, number 1, and use the defaults
to make the partition occupy the entire disk.
Command (m for help): n
Command action
e
extended
p
primary partition (1-4)
Partition number (1-4): 1

First cylinder (1-9729, default 1):<RETURN>

Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-9729, default 9729):

4) Run the print (p) command to confirm that you successfully created the partition partition.
Command (m for help): p
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/hdb1

Start
1

End
9726

Blocks
78148161

Id
83

System
Linux

Command (m for help):

Tip: If you make a mistake, you can use the d command to delete the partition and start over.
The t command enables you to change partition type from the default of 83 for regular Linux
partitions to something else, such as 82 for swap space. In most cases, this won't be necessary,
the default value is sufficient.
Note: When you created the new partition, you may have noticed that fdisk queried you as to
whether it was going to be a primary or secondary partition. Linux allows only four primary
partitions; if you need more, you can convert one of the primary ones into an extended one. Here
is an example of a partition table that includes an extended partition followed by two regular
partitions within it.
Command (m for help): p
Disk /dev/hda: 7510 MB, 7510164480 bytes
255 heads, 63 sectors/track, 913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/hda1
*
/dev/hda2
/dev/hda3
/dev/hda4
/dev/hda5
/dev/hda6

Start
1
14
145
399
399
845

End
13
144
398
913
844
913

Blocks
104391
1052257+
2040255
4136737+
3582463+
554211

Id
83
83
82
5
83
83

System
Linux
Linux
Linux swap
Extended
Linux
Linux

Command (m for help):

Adding more partitions is just a question of repeating the previous steps the required number of
times, while remembering that at some stage, you may need to add an extended partition.
5) Changes won't be made to the disk's partition table until you use the w command to write, or
save, the changes. Do that now, and, when finished, exit with the q command.
Command (m for help): w
Command (m for help): q

After this is complete you'll need to verify your work and start migrating your data to the new
disk. These steps will be covered next.

Verifying Your New Partition

You can take a look at the /proc/partitions file or use the fdisk -l command to see the changes to
the disk partition structure of your system:
[root@bigboy
major minor
...
...
...
22
0
22
1
[root@bigboy

tmp]# cat /proc/partitions

#blocks name

78150744 hdb
78150744 hdb1
tmp]#

[root@bigboy tmp]# fdisk -l

...
...
...
Disk /dev/hdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/hdb1
[root@bigboy tmp]#

Start
1

End
9729

Blocks
76051710

Id
83

System
Linux

Putting A Directory Structure On Your New Partition

You now need to format the partition, giving it a new directory structure by using the mkfs
command. The Fedora installation procedure defaults to an ext3 type, which is what you should
use here.
[root@bigboy tmp]# mkfs -t ext3 /dev/hdb1

Next, you must create special mount point directory, to which the new partition will be attached.
Create directory /mnt/hdb1 for this purpose.
[root@bigboy tmp]# mkdir /mnt/hdb1

When Linux boots, it searches the /etc/fstab file for a list of all partitions and their mounting
characteristics, and then it mounts the partitions automatically. You'll have to add an entry for
your new partition that looks like this:
#
# File: /etc/fstab
#

/dev/hdb1

/mnt/hdb1

ext3

defaults 1 2

The first entry is the name of the partition followed by the mount point directory and the
filesystem type. The fourth entry defines the mounting options, which need be only default for
most scenarios. The fifth entry governs whether the dump filesystem backup command can be
used for the filesystem. A value of 0 means no, and 1 means yes. The final entry defines the order
in which a filesystem check is done at boot time. The check is done twice. The root (or master)
filesystem has a value of 1 and is checked on the first pass, all other filesystems should have a
value of 2. If you are not familiar with the /etc/fstab file use the man fstab command to get a full
explanation of its various options.
You don't have to wait for a reboot to mount your partition. You can use the mount command
with the -a option to read the /etc/fstab file for new entries.
[root@bigboy tmp]# mount -a

You are now able to access your new partition as device /mnt/hdb1.

Migrating Data Over To your New Partition

As you remember from investigating with the df -k command, the /var partition is almost full.
[root@bigboy tmp]# df -k
Filesystem
1K-blocks
/dev/hda3
505636
/dev/hda1
101089
none
63028
/dev/hda5
248895
/dev/hda7
3304768
/dev/hda2
3304768
[root@bigboy tmp]#

Used Available Use% Mounted on

118224
361307 25% /
14281
81589 15% /boot
0
63028
0% /dev/shm
6613
229432
3% /tmp
2720332
416560 87% /usr
3300536
4232 99% /var

The du -sk * command shows the disk usage of all subdirectories in a directory. You can
recursively use the command by using the cd command to step down through all the
subdirectories until you discover the one with the greatest file usage. In this case, you only had to
go to the /var directory to see that the /var/transactions directory was the culprit.
[root@bigboy tmp]# cd /var
[root@bigboy var]# du -sk *
2036
cache
4
db
8
empty
...
...
133784 transactions
...
...
[root@bigboy var]#

As a solution, the /var partition will be expanded to the new /dev/hdb1 partition mounted on
the /mnt/hdb1 directory mount point. To migrate the data, use these steps:
1) Back up the data on the partition you are about to work on.
2) Use the who command to see who's logged in. If other users are present, send a message with
the wall command informing them that the system is about to shutdown:
[root@bigboy tmp]# who
root
pts/0
Nov 6 14:46
bob
pts/0
Nov 6 12:01
bunny
pts/0
Nov 6 16:25
[root@bigboy tmp]# wall The system

(192-168-1-242.my-site.com)
(192-168-1-248.my-site.com)
(192-168-1-250.my-site.com)
is shutting down now!

Broadcast message from root (pts/0) (Sun Nov

7 15:04:27 2004):

The system is shutting down now!

[root@bigboy tmp]#

3) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

4) Rename the /var/transactions directory /var/transactions-save to make sure you have an easy
to restore backup of the data, not just the tapes.
sh-2.05b# mv /var/transactions /var/transactions-save

5) Create a new, empty /var/transactions directory; this will later act as a mount point.
sh-2.05b# mkdir /var/transactions

6) Copy the contents of the /var/transactions-save directory to the root directory of /dev/hdb1,
which is actually /mnt/hdb1.
sh-2.05b# cp -a /var/transactions-save/* /mnt/hdb1

7) Unmount the new /dev/hdb1 partition.

sh-2.05b# umount /mnt/hdb1

8) Edit the /etc/fstab file, removing our previous entry for /dev/hdb1 replacing it with one using
the new mount point.
#
# File: /etc/fstab
#
#/dev/hdb1

/mnt/hdb1

ext3

defaults 1 2

/dev/hdb1

/var/transactions

ext3

defaults 1 2

9) Remount /dev/hdb1 on the new mount point using the mount -a command, which reads
/etc/fstab and automatically mounts any entries that are not mounted already.
sh-2.05b# mount -a

10) Test to make sure that the contents of the new /var/transactions directory is identical to
/var/transactions-save.
11) Return to multi-user mode by typing exit. The system will return to its default runlevel.
sh-2.05b# exit

12) Make sure your applications are working correctly and delete both the /var/transactions-save
directory and the /mnt/hdb1 mount point directory at some later date.
This exercise showed you how to migrate the entire contents of a subdirectory to a new disk.
Linux also allows you to merge partitions together, to create a larger combined one. The reasons
and steps for doing so will be explained next.

Expanding Partitions With LVM

The Logical Volume Manager (LVM) enables you to resize your partitions without having to
modify the partition tables on your hard disk. This is most useful when you find yourself running
out of space on a filesystem and want to expand into a new disk partition versus migrating all or
a part of the filesystem to a new disk.

Physical Volume: A physical volume (PV) is another name for a regular physical disk
partition that is used or will be used by LVM.
Volume Group: Any number of physical volumes (PVs) on different disk drives can be
lumped together into a volume group (VG). Under LVM, volume groups are analogous to
a virtual disk drive.
Logical Volumes: Volume groups must then be subdivided into logical volumes. Each
logical volume can be individually formatted as if it were a regular Linux partition. A
logical volume is, therefore, like a virtual partition on your virtual disk drive.
This may seem complicated, but it allows you to create new virtual partitions with sizes
you can change from groups of real disk partitions whose sizes you probably cannot
change. Another advantage of LVM is that this can all be done without disturbing other
partitions on your hard disks.
Physical Extent: Real disk partitions are divided into chunks of data called physical
extents (PEs) when you add them to a logical volume. PEs are important as you usually
have to specify the size of your volume group not in gigabytes, but as a number of
physical extents.

Make sure you understand these terms fully as they will be used repeatedly in many of the
following sections. Lets go!

Configuring LVM Devices

It is probably best to learn about the features of LVM through a scenario. Suppose a small
company needs to expand disk capacity, but there isn't the budget to purchase an adequately
sized hard drive. The /home filesystem, which resides on /dev/hde5, has become too full. You
just added a new hard drive /dev/hdf with 50% of the capacity of /dev/hde5 into which you want
to expand /home. The device /dev/hdf has a single partition named /dev/hdf1 into which
/dev/hde5 will be merged. Take a look at the required steps.

Backup Your Data

Use the tar command or some other method to backup your data in /home. The LVM process will
destroy the data on all physical volumes.

Unmount your /home filesystem

As /home stores most users' data, you'll need to do some preparatory work before unmounting
the filesystem.
1) Use the who command to see who's logged in. If other users are present, send a message with
the wall command informing them that the system is about to shutdown.
[root@bigboy tmp]# who
root
pts/0
Nov 6 14:46
bob
pts/0
Nov 6 12:01
bunny
pts/0
Nov 6 16:25
[root@bigboy tmp]# wall The system

(192-168-1-242.my-site.com)
(192-168-1-248.my-site.com)
(192-168-1-250.my-site.com)
is shutting down now!

Broadcast message from root (pts/0) (Sun Nov

7 15:04:27 2004):

The system is shutting down now!

[root@bigboy tmp]#

2) Log into the VGA console, and enter single-user mode.

[root@bigboy tmp]# init 1

3) Unmount the filesystem.

sh-2.05b# umount /home

Now we're ready to start modifying the partitions which is covered next.

Determine The Partition Types

You have to change each LVM partition used to be of type 8e (Linux LVM). You can test this
with the fdisk -l command. Here is an example using /dev/hde that shows your target partitions
are of the incorrect type.
sh-2.05b# fdisk -l /dev/hde
Disk /dev/hde: 4311 MB, 4311982080 bytes
16 heads, 63 sectors/track, 8355 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Device Boot
/dev/hde1
/dev/hde2
/dev/hde3
/dev/hde4
/dev/hde5
sh-2.05b#

Start
1
4089
5714
6608
6608

End
4088
5713
6607
8355
7500

Blocks
2060320+
819000
450576
880992
450040+

Id
fd
83
83
5
83

System
Linux raid autodetect
Linux
Linux
Extended
Linux

Start FDISK
You can change the partition type using fdisk with the disk name as its argument. Use it to
modify both partitions /dev/hde5 and /dev/hdf1. The fdisk examples that follow are for
/dev/hde5; repeat them for /dev/hdf1.
sh-2.05b# fdisk /dev/hde
The number of cylinders for this disk is set to 8355.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help):

Set The ID Type To 8e

You now need to set the partition types to the LVM value of 8e. Partitions /dev/hde5 and
/dev/hdf1 are the fifth and sixth partitions on disk /dev/hde. Modify their type using the t
command, and then specify the partition number and type code. You can also use the L command
to get a full listing of ID types in case you forget.
Command (m for help): t
Partition number (1-6): 5
Hex code (type L to list codes): 8e
Changed system type of partition 5 to 8e (Linux LVM)
Command (m for help): t
Partition number (1-6): 6
Hex code (type L to list codes): 8e
Changed system type of partition 6 to 8e (Linux LVM)

Command (m for help):

Make Sure The Change Occurred

Use the p command to get the new proposed partition table.
Command (m for help): p
Disk /dev/hde: 4311 MB, 4311982080 bytes
16 heads, 63 sectors/track, 8355 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Device Boot
/dev/hde1
/dev/hde2
/dev/hde3
/dev/hde4
/dev/hde5

Start
1
4089
5714
6608
6608

End
4088
5713
6607
8355
7500

Blocks
2060320+
819000
450576
880992
450040+

Id
fd
83
83
5
8e

System
Linux raid autodetect
Linux
Linux
Extended
Linux LVM

Command (m for help):

Save The Partition Changes

Use the w command to permanently save the changes to disk /dev/hde.
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or
resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
sh-2.05b#

The error above will occur if any of the other partitions on the disk is mounted. This shouldn't be
grave as you are already in single user mode in which most of the system's processes that would
be accessing the partition have been shutdown.

Define Each Physical Volume

After modifying the partition tables of /dev/hde and /dev/hdf, initialize the target partitions with
the pvcreate command. This wipes out all the data on them in preparation for the next step. If
you haven't backed up your data yet, do it now!
sh-2.05b# pvcreate /dev/hde5 /dev/hdf1
pvcreate -- physical volume "/dev/hde5" successfully created
pvcreate -- physical volume "/dev/hdf1" successfully created
sh-2.05b#

Create A Volume Group For the PVs

Use the vgcreate command to combine the two physical volumes into a single unit called a
volume group. The LVM software effectively tricks the operating system into thinking the
volume group is a new hard disk. In the example, the volume group is called lvm-hde.
sh-2.05b# vgcreate lvm-hde /dev/hdf1 /dev/hde5
Volume group "lvm-hde" successfully created
sh-2.05b#

Therefore, the vgcreate syntax uses the name of the volume group as the first argument followed
by the partitions that it will be comprised of as all subsequent arguments.

Run VGscan
The next step is to verify that Linux can find your new LVM disk partitions. To do this, use the
vgscan command.
sh-2.05b# vgscan
vgscan -- reading all physical volumes (this may take a while...)
Found volume group "lvm-hde" using metadata type lvm2
sh-2.05b#

Create A Logical Volume From The Volume Group

Now you're ready to partition the volume group into logical volumes with the lvcreate command.
Like hard disks, which are divided into blocks of data, logical volumes are divided into units
called physical extents (PEs).
You'll have to know the number of available PEs before creating the logical volume. This is done
with the vgdisplay command using the new lvm-hde volume group as the argument.
sh-2.05b# vgdisplay lvm-hde
--- Volume group --VG Name
lvm-hde
VG Access
read/write
VG Status
available/resizable
VG #
0
MAX LV
256
Cur LV
0
Open LV
0
MAX LV Size
255.99 GB
Max PV
256
Cur PV
2
Act PV
2
VG Size
848 MB
PE Size
4 MB
Total PE
212
Alloc PE / Size
0 / 0
Free PE / Size
212 / 848 MB
VG UUID
W7bgLB-lAFW-wtKi-wZET-jDJF-8VYD-snUaSZ

sh-2.05b#

As you can see, 212 PEs are available as free. You can now use all 212 of them to create a logical
volume named lvm0 from volume group lvm-hde.
sh-2.05b# lvcreate -l 212 lvm-hde -n lvm0
Logical volume "lvm0" created
sh-2.05b#

Note: You can also define percentages of the volume group to be used. The first example defines
the use of 100% of the volume group's free space and the second example specifies using 50% of
the total volume group.
sh-2.05b# lvcreate -l 100%FREE -n lvm0 lvm-hde
sh-2.05b# lvcreate -l 50%VG -n lvm0 lvm-hde

Format The Logical Volume

After the logical volume is created, you can format it as if it were a regular partition. In this case,
use the -t switch to specify to the mkfs formatting program that you want a type ext3 partition.
sh-2.05b# mkfs -t ext3 /dev/lvm-hde/lvm0
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
108640 inodes, 217088 blocks
10854 blocks (5.00%) reserved for the super user
First data block=0
7 block groups
32768 blocks per group, 32768 fragments per group
15520 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
sh-2.05b#

Create A Mount Point

When you formatted the /dev/hde5 partition, you lost the /home directory. Now you have to
recreate /home on which you'll later mount your new logical volume.
sh-2.05b# mkdir /home

Update The /etc/fstab File

The /etc/fstab file lists all the partitions that need to be automatically mounted when the system
boots. This snippet configures the newly labeled partition to be mounted on the /home mount
point.
/dev/lvm-hde/lvm0

/home

ext3

defaults

1 2

The /dev/hde5 and /dev/hdf1 partitions are replaced by the combined /lvm0 logical volume. You,
therefore, don't want the old partitions to be mounted again. Make sure that any reference to
them in this file has either been commented a # character at the beginning of each line or deleted
entirely.
#/dev/hde5
#/dev/hdf1

/data1
/data2

ext3
ext3

defaults
defaults

1 2
1 2

Mount The Volume

The mount -a command reads the /etc/fstab file and mounts all the devices that haven't been
mounted already. After mounting, test the volume by listing its directory contents. It should just
contain the lost+found directory
sh-2.05b# mount -a
sh-2.05b# ls /home
lost+found
sh-2.05b#

Restore Your Data

You can now restore your backed up data to /home.

Get Out Of Single User Mode

Return to your original run state by using either the init 3 or init 5 commands. The exit command
will make you return to your default runlevel.

Conclusion
The demise of the hard disk has been predicted for many years. Faster, denser memory chips
were supposed to eliminate their need, but hard disk technology has evolved, dramatically
increasing their speed and capacity too. They will be around for a long time to come.
It seems as if when drives get bigger, so does the data they are intended to store. Expanding the
existing disk capacity of your server may become an everyday occurrence and the tools
described in this chapter should make the task easier.