Mounts and /etc/fstab explained

What we're doing

When you hit Start, you land on a Linux box where widget-api is failing because the disk it needs was never mounted. You'll learn what a mount actually is, inspect the disks and the mount table, format and mount the disk by hand, then make it permanent with a UUID-based /etc/fstab line (with the nofail safeguard), and finally reboot to prove it sticks.

Watch the video first, then run these as you read. Every command and flag is explained. You're root on this box, so use sudo where shown.

Heads up: the disk is simulated. This lab box has one SSD, so the "second disk" is a loopback file pretending to be a disk. It shows up as /dev/loop0 instead of /dev/sdb. Everything you do (format, UUID, fstab, mount, reboot) is exactly what you'd do on a real disk. The only extra piece is a tiny service that re-attaches the file at boot, standing in for physical hardware; on a real server you wouldn't have it, and your fstab line is all you'd need.

The one idea: what a mount is

Linux has one directory tree, starting at /. There are no drive letters. A disk that's plugged in but not mounted isn't in the tree at all; it's just sitting there. Mounting grafts a disk onto a folder (the mountpoint). Before you mount it, /srv/appdata is an empty folder on the root disk. After you mount the disk there, /srv/appdata is that disk, and its files show up there.

That's the whole job today: take a disk and graft it onto /srv/appdata, reliably and permanently.

Step 1: See what exists, and what's mounted

Two different questions. First, what disks exist?

lsblk

lsblk ("list block devices") draws your disks as a tree: name, size, type, and where each is mounted. Your system disk (sda or similar) is mounted on /. The one to notice is loop0: about 2G, with no mountpoint. It exists but is grafted onto nothing.

Add columns to see why it's unusable yet:

lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT

NAME      SIZE TYPE FSTYPE MOUNTPOINT
loop0       2G loop
sda        40G disk
├─sda1     39G part ext4   /

loop0 has an empty FSTYPE (no filesystem yet, it's raw) and an empty MOUNTPOINT (not mounted). Now the second question, what's mounted on the app's folder?

findmnt /srv/appdata

Nothing prints. The folder exists, but nothing is mounted on it. That's exactly why the app is down.

lsblk shows the disks that exist. findmnt shows what's actually mounted. Your job is to get the first disk to show up in the second list.

Step 2: Format the raw disk

You can't mount a raw disk, there's no filesystem on it. Create one:

sudo mkfs.ext4 /dev/loop0

mkfs.ext4 means "make an ext4 filesystem" (ext4 is the standard Linux default). This writes the structure that lets the disk hold files, and stamps a permanent UUID onto it.

This wipes the disk. Only ever run mkfs on a disk you intend to erase. Double-check the device name first with lsblk.

Step 3: Mount it by hand and bring the app back

sudo mount /dev/loop0 /srv/appdata
findmnt /srv/appdata

TARGET       SOURCE     FSTYPE OPTIONS
/srv/appdata /dev/loop0 ext4   rw,relatime

mount <disk> <folder> performs the graft. findmnt confirms /srv/appdata is now /dev/loop0. Restart the app:

sudo systemctl restart widget-api
systemctl is-active widget-api      # -> active

It goes active. It had a disk all along; it just needed one mounted where it was looking.

But this won't survive a reboot. A hand mount lives only in memory. Reboot and it's gone, and the app breaks again. That's exactly how this box arrived broken. To make it permanent, you write it into /etc/fstab.

Step 4: Get the disk's UUID

You'll refer to the disk by its UUID, not /dev/loop0. Device names (sda, sdb, loop0) are just the order the kernel found disks in this boot; they can shuffle. The UUID is stamped into the filesystem and never changes.

sudo blkid /dev/loop0

/dev/loop0: LABEL="appdata" UUID="bd53b537-8a4a-4599-85bf-07df1fc597e2" TYPE="ext4"

Copy your UUID (yours will be different).

Step 5: Add the fstab line

/etc/fstab is the table systemd reads on every boot to know which disks to mount, where, and how. Each line has six fields:

UUID=<uuid>     /srv/appdata    ext4      defaults,nofail   0   2
   what          where          type        options       dump pass

1. what to mount, the UUID
2. where, the mountpoint
3. type, the filesystem (ext4)
4. options, here defaults,nofail (more below)
5. dump, an ancient backup flag, leave 0
6. pass, filesystem-check order: 1 is the root filesystem, every other disk is 2, 0 skips the check

Append your line (swap in your UUID):

echo "UUID=bd53b537-8a4a-4599-85bf-07df1fc597e2  /srv/appdata  ext4  defaults,nofail  0  2" | sudo tee -a /etc/fstab
sudo systemctl daemon-reload     # tell systemd you changed fstab

Why nofail matters: by default, if a disk in fstab is missing at boot (a failed drive, or a typo in the UUID), the boot doesn't skip it, it stops and drops the server into emergency mode, offline until someone fixes it from the console. nofail says "if this disk isn't here, skip it and keep booting." On any non-root data disk, always include nofail.

Step 6: Test the fstab line WITHOUT rebooting

Never trust a fresh fstab line by rebooting straight into it. Test it live first:

sudo umount /srv/appdata     # undo the hand mount, /srv/appdata is empty again
sudo mount -a                # "mount all": mount everything in fstab, just like boot does
findmnt /srv/appdata         # mounted again -> your line works

mount -a reads /etc/fstab and mounts everything in it, the same thing boot does. If your line is good, the disk mounts straight back. You can also lint the file:

findmnt --verify

0 parse errors, 0 errors, 1 warning

findmnt --verify checks fstab for mistakes before they bite at boot. Aim for 0 parse errors, 0 errors. (A warning about daemon-reload just means run sudo systemctl daemon-reload, which you did.)

Step 7: Reboot and prove it

Drop a marker file, then reboot for real:

echo "here before the reboot" | sudo tee /srv/appdata/proof.txt
sudo reboot

When the box comes back, touch nothing and check:

findmnt /srv/appdata          # mounted automatically, by UUID
cat /srv/appdata/proof.txt    # "here before the reboot" -> data survived
systemctl is-active widget-api # active -> app came up on its own

The disk mounted itself before the app needed it, the data is intact, and widget-api is green. Done: a raw disk is now a permanent, named home for the app's data that returns on every boot.

Useful options to know

You change a disk's behaviour in the options field. Two worth knowing:

• noatime stops Linux recording a timestamp every time a file is merely read. On busy data or log disks that removes a lot of pointless writes. Safe and common: defaults,nofail,noatime.
• ro mounts a disk read-only, for inspecting something you must not change (a recovered disk, evidence, a reference volume).

You can test options live without editing fstab, using remount:

sudo mount -o remount,ro /srv/appdata     # now read-only (writes will be refused)
sudo mount -o remount,rw /srv/appdata     # back to read-write

See the nofail safeguard for yourself (optional)

Watch what nofail prevents, safely, on the running box (we'll add a bad line, test it, then remove it; we will not reboot into it):

# a line pointing at a UUID that doesn't exist, WITHOUT nofail:
echo "UUID=0000-dead-beef  /mnt/ghost  ext4  defaults  0  2" | sudo tee -a /etc/fstab
sudo mount -a            # ERROR: it can't find the device and complains

At boot, that error is what drops you into emergency mode. Now make it safe:

sudo sed -i 's#defaults  0  2#defaults,nofail  0  2#' /etc/fstab   # add nofail to that line
sudo mount -a            # now it's skipped quietly, no error

Clean up the demo line so it doesn't linger:

sudo sed -i '\#/mnt/ghost#d' /etc/fstab
sudo systemctl daemon-reload

Cheat sheet

lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT   # 1. find the raw disk (no FSTYPE, no MOUNTPOINT)
sudo mkfs.ext4 /dev/loop0                   # 2. format it (WIPES it)
sudo mount /dev/loop0 /srv/appdata          # 3. mount by hand, confirm with findmnt
sudo blkid /dev/loop0                       # 4. get the UUID
# add to /etc/fstab:  UUID=<uuid>  /srv/appdata  ext4  defaults,nofail  0  2
sudo systemctl daemon-reload                #    tell systemd about the change
sudo umount /srv/appdata && sudo mount -a   # 5. test fstab WITHOUT rebooting
findmnt --verify                            #    lint the fstab file
sudo reboot                                 # 6. prove it survives

The one thing to remember: a hand mount is temporary; /etc/fstab makes it permanent. Mount by UUID so you never grab the wrong disk, and add nofail so a bad line can never brick the boot.