LVM end to end (and growing a volume)

What we're doing

When you hit Start, the VM has two spare 1 GB disks attached. You'll turn them into an LVM stack (physical volumes, a volume group, a logical volume), format and mount it at /data, fill it near-full, and then grow it online by adding the second disk.

Watch the video first, then run these as you read. Every command and flag is explained. You're root on this VM, so use sudo where shown.

Heads up: the two spare disks are simulated with loopback files, so they show up as /dev/loop0 and /dev/loop1 instead of /dev/sdb and /dev/sdc. Every LVM command is the same as on real disks.

The idea: LVM has three layers

• PV (physical volume): a disk marked for LVM to use. pvcreate.
• VG (volume group): one or more PVs pooled into one big pool of space. vgcreate.
• LV (logical volume): a slice carved out of the pool. You format and mount this, like a partition, except you can resize it later. lvcreate.

Disks become PVs, PVs pool into a VG, you cut LVs from the VG. The win: an LV isn't stuck at the size of one disk.

Step 1: See the disks

lsblk

NAME    SIZE TYPE MOUNTPOINTS
loop0     1G loop
loop1     1G loop
sda      40G disk
├─sda1   39G part /
└─sda15  99M part /boot/efi

loop0 and loop1, 1 GB each, nothing mounted. We build on loop0 first and keep loop1 for the grow.

Step 2: Build the stack

sudo pvcreate /dev/loop0                      # disk -> physical volume
sudo vgcreate data-vg /dev/loop0              # PV -> volume group named data-vg
sudo lvcreate -L 900M -n data-lv data-vg      # carve a 900M logical volume named data-lv

• pvcreate marks the disk for LVM.
• vgcreate <name> <pv> makes a volume group (the pool) from the PV.
• lvcreate -L <size> -n <name> <vg> carves a logical volume out of the pool.

The LV lives at /dev/data-vg/data-lv. Format and mount it:

sudo mkfs.ext4 /dev/data-vg/data-lv
sudo mkdir -p /data
sudo mount /dev/data-vg/data-lv /data
df -h /data

Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/data--vg-data--lv 871M   28K  805M   1% /data

/data is live, about 870 MB, on the LV. (df shows the LV by its device-mapper name, data--vg-data--lv, same thing.)

Step 3: Fill it up

Simulate the app's data with one big file:

sudo fallocate -l 750M /data/app.dat     # allocate a 750 MB file
df -h /data

/dev/mapper/data--vg-data--lv 871M  751M   77M  91% /data

91% full. On a real server, this is the alert. With a raw partition you'd be stuck. With LVM, you grow it.

Step 4: Grow it online

Add the second disk to the pool, then grow the volume and its filesystem.

sudo pvcreate /dev/loop1                       # disk2 -> physical volume
sudo vgextend data-vg /dev/loop1               # add it to the group (pool is now ~2 GB)
sudo lvextend -l +100%FREE /dev/data-vg/data-lv  # grow the LV into all the new space
sudo resize2fs /dev/data-vg/data-lv            # grow the ext4 filesystem to match
df -h /data

• vgextend adds the new PV to the existing group.
• lvextend -l +100%FREE grows the LV by all the free space in the group.
• resize2fs grows the filesystem to fill the bigger LV. It does this online, while /data is mounted.

/dev/mapper/data--vg-data--lv 1.9G  751M  1.1G  42% /data

/data is now about 1.9 GB and 42% full. The file is still there, and you never unmounted. That's the whole point of LVM.

The order matters: grow the LV first (lvextend), then the filesystem (resize2fs). The LV is the container; the filesystem only fills the space the container gives it.

Step 5: See the stack

sudo pvs    # physical volumes: both disks now
sudo vgs    # the volume group: ~2 GB
sudo lvs    # the logical volume: ~1.9 GB

pvs, vgs, and lvs list each layer. Both disks are PVs, the group is about 2 GB, and the LV grew to fill it.

Cheat sheet

# build
sudo pvcreate /dev/loop0
sudo vgcreate data-vg /dev/loop0
sudo lvcreate -L 900M -n data-lv data-vg
sudo mkfs.ext4 /dev/data-vg/data-lv
sudo mount /dev/data-vg/data-lv /data

# grow (online, no unmount)
sudo pvcreate /dev/loop1
sudo vgextend data-vg /dev/loop1
sudo lvextend -l +100%FREE /dev/data-vg/data-lv
sudo resize2fs /dev/data-vg/data-lv

# inspect
sudo pvs ; sudo vgs ; sudo lvs

The one thing to remember: LVM stacks disks into a pool and carves volumes from it, so a volume isn't stuck at one disk's size. When it fills up, add a disk (pvcreate + vgextend), grow the volume (lvextend), then grow the filesystem (resize2fs), all while it stays mounted.

Next tutorial: the weird "disk full" cases, inodes and deleted-but-open files.