Up

Encrypted Filesystems with LUKS and LVM on CentOS 5 (5.1, 5.2, 5.3)

This document shows an example of how to set up a CentOS 5 machine with encryption of selected filesystems

This example shown as a server configuration, but may be used in other types of machines provided that hibernate and suspend capability is not required.

Before continuing, please read the introductory page, which also deals with another configuration where all filesystems are encrypted (other than /boot) and which is suitable for use in laptops (including hibernate and suspend functionality).

Contents

• Pre-Installation Partitioning
• OS Installation
• Initial Configuration Changes
• Update mkinitrd
• Disk Preparation
• Set Up Encrypted Volumes
• [Optional] Set Up a USB-Based LUKS Key
• Copy Filesystems
• Use Crypto-Swap
• Reenable Automatic Updates
• [Optional] Move /tmp to Tmpfs
• [Optional] Disable Filesystem Checks
• Testing
• Recover Unencrypted Partition Space

Pre-Installation Partitioning

Here we assume that we'll be using software RAID-1 for our system. Unlike the laptop version, we don't need to do any pre-install partitioning, as long as we examine the partition map created later on and verify that md0 appears at the beginning of the disk, md1 appears in the middle, and md2 appears at the end, and that they are all primary partitions.

OS Installation (Unencrypted)

Proceed with the installer until you get to the partitioning screen. Select "Create custom layout" and hit "Next".

Create three md devices, in sequence, from the beginning to the end of the underlying disks:

1. /dev/md0: Create this device as the first primary partition, about 100-200 MB in size. Configure it to be of type ext3 and mounted as /boot.
2. /dev/md1: This must be the second primary partition. It should consist of all remaining space except for 10-20 GB that will be used for the third partition. Do not place any filesystems on this device.
3. /dev/md2: This must be the third primary partition, consisting of the remaining space (10-20 GB). On this partition, create a logical volume. Use a logical volume name that eventually will be thrown away. We will assume the logical volume name is vg2.

Next, create your filesystems as logical volumes under volume group vg2. Use whatever allocation scheme you want, but I prefer separate filesystems:

We'll make /home larger later. 2GB is a good size for /var as it allows lots of space for OS updates.

I'd suggest using a boot loader password. This should not be the same password that you're going to use for the disk encryption, later.

Select your sets or packages of software and perform the install. Reboot when prompted.

During the "final" configuration, you will be prompted for the SELinux setting, which is by default enabled. These instructions have only been tested with SELinux disabled. Given the history of SELinux, if you have it enabled you will probably have to do some additional configuration.

Reboot if the installation script requires you to do so.

Initial Configuration Changes

Before continuing, there are a couple of things that I'd recommend. The first is to disable any automatic updates while we're getting the encrypted setup working:

   service yum-updatesd stop
   chkconfig yum-updatesd off

The second is to use a text boot sequence so that we can see any interesting information go by. If you want to, you can reverse this later to get the default graphical boot sequence, but I leave it as-is. Note that even if you perform these steps, the system still boots into run level 5, meaning that you'll have your graphical window manager when it's time to log in:

1. Modify /etc/sysconfig/init changing GRAPHICAL=yes to GRAPHICAL=no
2. Modify /boot/grub/grub.conf (or the symlink /etc/grub.conf) thus:
   1. Comment out the splashimage line.
   2. Comment out the hiddenmenu line.
   3. Delete the word quiet from the kernel arguments line.

Update mkinitrd

At the time of this writing (CentOS 5.1), there is a mkinitrd bug that can cause kernel updates to fail. It is possible, by the time that you are reading this, that the bug has been fixed and propagated. UPDATE: This seems to have been fixed CentOS 5.3 circa early 2009.

Therefore, first update mkinitrd to its current version before we start:

• yum -y update yum
• yum -y update mkinitrd

[17 Nov 2008] The above URL seems to be a dead link. If you need the fix and it is not yet in your mkinitrd, you can apply my mkinitrd-slaves patch.

Disk Preparation

This next step is very time consuming, but important. In order to avoid certain types of cryptanalysis on your soon-to-be-encrypted disk, it should be filled with random data. We'll do this in two parts; one now, one later.

[Optional] If your disk previously had data on it that is of a sensitive nature, you can also first shred it, which helps to protect you from having someone get your disk and read your old data off of it via hysteresis effects. In this context, shredding is an repeated overwrite procedure. See Data Remanence. There is no need to shred the disk if it is new or if the old contents are not sensitive (you still need to randomize it). Shredding may seem paranoid, but you're reading this, aren't you? ;)

The preparation consists of:

1. [Optional] Shred the currently unused device, that will later hold the encrypted data.
2. Randomize the same device

The specific commands are:

1. shred -v /dev/md1
2. dd if=/dev/urandom of=/dev/md1 bs=1024

Do not reverse the two steps above; the order is important, otherwise you nullify the effect of the second one.

Do not use /dev/random as you will empty your entropy pool quickly and possibly cause your system to hang. Use /dev/urandom instead.

The one or two commands above can take a LONG time. Having your computer run them overnight is a good idea. They might still be running when you wake up in the morning. If you want to run both, then you can chain them with a semicolon:

shred -v /dev/md1; dd if=/dev/urandom of=/dev/md1 bs=1024

My laptop (IBM Thinkpad T42) performs the dd step at about 3.4 MB/s, so my 72GB /dev/hda2 partition takes almost 6 hours to randomize. The time to perform shredding is comparable.

If you want to see the progress of dd, then you can (from another window or terminal) send it a SIGUSR1. If you don't know how to do this, look at the ps(1) and kill(1) man pages. If you don't know what a man page is, you probably shouldn't be doing this ...

Set Up Encrypted Volumes

Whew! Now that you have your target partition scrubbed and randomized, it's time to get to the meat of the issue. We will now set up the LVM configuration that will be used in production.

Naming Scheme

Since we are setting up a volume group over top of an encrypted device, I use the following naming scheme: The decrypted device base name uses the same numerical identifier as does the volume group name, and has all of its letters in upper case. Volume group names use lower case (and numbers and hyphens). For example, if I have a volume group vg1 which, when encrypted, resides on /dev/md1, then:

1. The volume group name is vg1 and we will see it in the filesystem as /dev/vg1/....
2. The volume group vg1 will use the "physical volume" /dev/mapper/PV1. (If this were a server with multiple physical volumes per volume group, then I'd use PV1A, PV1B, or vg01 with PV01A, PV01B, etc)
3. The above "physical volume" is actually the unencrypted virtual device that is based on the encrypted device /dev/md1.

Edit lvm.conf

Now you should edit the /etc/lvm/lvm.conf file so that pvscan and vgscan can pick up the soon-to-be-encrypted device. Edit it to have the following line (or apply this patch):

   types = [ "device-mapper", 16 ]

   filter = [ "a|^/dev/mapper/[A-Z].*|", "r|^/dev/mapper/.*|", "a/.*/" ]

Create the Encrypted Volume

Now we're ready to create the encrypted device. We use LUKS for this. Pick a strong passphrase, but one that you're going to remember, because you'll either be typing it every time you boot your computer or (depending on what you use as your primary key location) at least using it as a backup passphrase:

   cryptsetup -c aes-cbc-essiv:sha256 luksFormat /dev/md1

Now before you go any further, make a copy of that passphrase somewhere secure. Write it on a piece of paper, seal it in an envelope, and lock it up in your safe. Make another copy of it, seal it, and put it in your safety deposit box. If you lose it, all your data is gone forever; there is no magical passphrase recovery mechanism, no back doors. (However you can have more than one passphrase that unlocks your data; see the LUKS documentation for details.)

Next, open the device as PV1, which will be the physical volume for volume group vg1:

   cryptsetup luksOpen /dev/md1 PV1

   pvcreate /dev/mapper/PV1
   vgcreate vg1 /dev/mapper/PV1

   lvcreate -L10G -n home vg1
   lvcreate -L2G -n var vg1

   mke2fs -j /dev/vg1/home
   mke2fs -j /dev/vg1/var

   lvcreate -L2G -n swap vg1
   mkswap /dev/vg1/swap

Next, edit (or create) the /etc/crypttab file and enter the following on a single line:

    PV1   /dev/md1

[OPTIONAL] Set Up a USB-Based LUKS Key

One option that will allow a system of this type to boot up without human intervention is to place a LUKS key on a device that is available to the system at boot time.

Note that doing this makes your system much less secure than if you require a person to type in the key at boot time. If your adversary gets your USB stick as well as your disk, then your data is compromised.

We assume for the sake of example that we will be using a USB flash disk that identifies itself to the kernel as usb-Manufacturer_123456790ABCDEF, has a single FAT file system, and that we'll use /flash/key as the mount point for that device.

1. Plug in your USB stick. In /var/log/messages you should see diagnositcs showing that the device has been plugged in. Verify its ID (as shown above) and that it contains a single FAT filesystem as its first partition. (If not, you'll have to ammend your procedure accordingly.)

2. Create the mount point:

       mkdir -p /flash/key
       chmod 700 /flash/key
   

3. Modify /etc/fstab and add in an entry for the device, based on the "by-id" path, with the following flags:

       /dev/disk/by-id/usb-Manufacturer_123456790ABCDEF-part1   /flash/key   vfat   noatime,umask=0377,noauto   0 0
   

   Save the fstab file and exit your editor.

4. Perform a test mount, create a file on the device, and unmount it (you will use this test file later):

       mount /flash/key
       date > /flash/key/testfile
       ls /flash/key
       umount /flash/key
   

5. Create the file /etc/sysconfig/modules/usb-crypto-key.modules (or download and edit it for your configuration. The sample download is shown here:

   #! /bin/sh
   
   FLASH_ID="usb-Manufacturer_123456790ABCDEF-part1"
   USB_FLASH_DEVICE="/dev/disk/by-id/$FLASH_ID"
   MOUNT_POINT=/flash/key
   
   modprobe usb_storage
   echo "waiting 5 seconds for USB discovery"
   sleep 5
   if [ -e "$USB_FLASH_DEVICE" ]; then
           echo "mounting USB flash drive"
           mount -n -t vfat -o ro,umask=377 "$USB_FLASH_DEVICE" "$MOUNT_POINT"
   else
           echo "USB flash device not found"
   fi
   

   Modify FLASH_ID and MOUNT_POINT in the script appropriately for your system. Change the file permissions appropriately:

       chmod 0755 /etc/sysconfig/modules/usb-crypto-key.modules
   

6. Reboot to ensure that the device is mounted at boot time. Note: You will not see the device mounted in the output of df or mount, because the device was mounted with the -n flag. Instead, look for the test file that you created previously:

       cat /flash/key/testfile
   

7. Add the following line to /etc/rc.d/rc.local

       umount /flash/key
   

8. Create another LUKS passphrase, for use on the USB device:

       dd if=/dev/random of=/flash/key/luks.key bs=35 count=1
   

9. Add the new key to your encrypted device:

       cryptsetup luksAddKey /dev/md1 /flash/key/luks.key
   

   You will be asked for your existing passphrase, twice. You should now see two keys in the LUKS header:

       cryptsetup luksDump /dev/md1
   

10. Edit /etc/crypttab and add the name of the key file to the existing line:

        PV1    /dev/md1    /flash/key/luks.key    
    

11. Unmount the device:

        umount /flash/key
    

12. Reboot your system and verify that:

    1. The encrypted disk gets set up without user intervention.
    2. The /flash/key disk is not mounted (the directory appears empty) after boot.
    3. You are able to manually mount and unmount /flash/key (as root)

Copy Filesystems

Reboot to single user mode, and copy the desired filesystems from the unencrypted volume group to the encrypted volume group.

1. Copy over the /var filesystem:

      mount /dev/vg1/var /mnt
      cd /mnt
      dump -0af - /var | restore -rf -   
      rm restoresymtable
      cd
      umount /mnt
   

2. Copy over any remaining filesystems, such as /home:

      mount /dev/vg1/home /mnt
      cd /mnt
      dump -0af - /home | restore -rf -   
      rm restoresymtable
      cd
      umount /mnt
   

3. Edit the /etc/fstab file and update the mount points.

Use Crypto-Swap

If you are keeping your swap partition on an unencrypted device, this mechanism can be used to ensure that any sensitive information held in memory is not written in clear text to the disk. This mechanism is not compatible with hibernate, suspend, and resume.

1. Comment out the swap entry in /etc/fstab

2. Reboot into single user mode, and verify that there is no swap active via swapon -s

3. Add a line to your /etc/crypttab file, similar to the following:

       swap   /dev/vg2/swap   /dev/urandom   swap,cipher=aes-cbc-essiv:sha256
   

   Do not use /dev/random.

4. Clear the existing disk:

       dd if=/dev/urandom of=/dev/vg2/swap bs=1024
   

5. Reboot (or at least bring the system to its default run level)

6. Change /etc/fstab to use /dev/mapper/swap as the swap partition.

7. Execute swapon -a

An alternative to this mechanism is to put your swap volume on an encrypted volume group.

Reenable Automatic Updates

After everything is sorted out, you should reenable automatic updates. While normally I would suggest doing the opposite of the disabling process, there was a BIG problem with updates when yum-updatesd was introduced. See the RedHat bug report for details. This was reported in FC6, but it also affected RHEL5, thus CentOS 5.0 and 5.1.

An alternative to reenabling the official (and broken) mechanism is to do what was done before yum-updatesd was introduced, namely invoke it as an overnight cron job. A script to do this is available here and can be installed like this:

    cd /etc/cron.daily
    wget ftp://ftp.gno.org/pub/tools/centos5.2/0yum-custom
    chmod 755 0yum-custom

[OPTIONAL] Move /tmp to tmpfs

In order to make access to /tmp faster and to keep it cleaned out between reboots, I like to put /tmp on tmpfs.

Regardless of whether or not you choose to perform this step, the data in /tmp (or the data that was in /tmp) will be encrypted whenever the machine is off, hibernating, or suspended. (That is, assuming that you have performed the other steps described in this document.)

1. Reboot into single user mode.
2. Add the following entry to /etc/fstab:

      tmpfs       /tmp       tmpfs     defaults     0 0
   

   If you want to limit the amount of space used by /tmp, you can specify it with the size option:

      tmpfs       /tmp       tmpfs     size=512m    0 0
   

3. Delete all the files in /tmp, including hidden files (ls -a).
4. Reboot.

[OPTIONAL] Disable Periodic Filesystem Checks

Since a journalling file system was used above, you may optionally choose to disable the periodic filesystem checks. There are differing opinions on whether this is a good or bad thing to do. Theoretically, a journalled filesystem shouldn't need a periodic check and disabling it can avoid the occasional long boot time while a check is conducted (especially annoying when running on battery). However, some people are uncomfortable with the idea of never running fsck on a filesystem unless it is marked as dirty. It's up to you.

Do disable the checks, you would use tune2fs:

   tune2fs -i0 -c0 /dev/vg0/root
   tune2fs -i0 -c0 /dev/vg0/usr
   tune2fs -i0 -c0 /dev/vg0/var
   tune2fs -i0 -c0 /dev/vg0/home

Testing

I need to write this one

Recover Unencrypted Partition Space

I need to write this one (for moved filesystems, only)