Storage DeDup (lvm & vdo)

Skill Level: Advanced

1. Overview

A Logical Volume Manager is a software layer on top of physical hard disks and partitions, which creates an abstraction of continuity and ease-of-use for managing the lifecycle of those devices (ie: addition, removal, replacement, repartitioning, backup, etc).

Over the years, the role of LVM has expanded greatly to include data redundancy (RAID), compression, deduplication, and more…​

These exercises will help you get familiar with the basic concepts of LVM and also introduce deduplication with the Virtual Data Optimizer (VDO).

2. Getting Started

For these exercises, you will be using the host node3 as user root.

From host bastion, ssh to node3.

ssh node3

Use sudo to elevate your privileges.

[[ "$UID" == 0 ]] || sudo -i

Verify that you are on the right host for these exercises.

workshop-vdo-checkhost.sh

You are now ready to proceed with these exercises.

3. Installation & Configuration

Install the required packages - this will pull in several related dependencies.

dnf install -y vdo

That’s it! All LVM components are a standard part of RHEL. Only the vdo kmod and related utilities need to be installed.

4. Why use Logical Volume Management?

  • Flexibility

  • Grow, shrink or relocate your data/filesystems

  • Aggregate or subdivide devices as needed

  • Performance

  • Striping across multiple devices

  • Caching via SSDs

  • Fault Tolerance (redundancy & resiliency)

  • RAID 0, 1, 5, 6, 10

  • Snapshots: Historical Recovery

  • Data Optimization: Compression and De-Duplication

4.1. Building Blocks of Storage Management

From the bottom up, here is a basic explanation of the layered technology stack that comprises modern storage.

File-systems

Formatted LV’s become filesystems

Logical Volume

A virtual storage device that may span multiple physical devices. Allocatable chunks (PEs) are assembled into “Logical Extents” that form the addressable space.

Volume Group

A collection of Physical Volumes that are divided into discrete allocatable chunks called “physical extents” (PEs).

Physical Volume

An LVM concept that identifies physical devices for LVM use.

Physical Device

Disks (IDE [hda], SCSI, SATA & SAS [sda], etc…​) Partitions (ex: hda1, sda1, cciss/c0d0p1, etc…​) LUNs (FCOE, SAN, etc…​) loopback

4.2. LVM CLI Toolbox

Physical Volumes Volumes Groups Logical Volumes

Core Utilities

 
pvcreate
pvdisplay
pvremove
pvs
pvscan
pvmove
 
vgcreate
vgdisplay
vgextend
vgreduce
vgremove
vgrename
vgs
vgscan
vgcfgbackup
vgcfgrestore
 
lvconvert
lvcreate
lvdisplay
lvextend
lvreduce
lvremove
lvrename
lvresize
lvs
lvscan

Other Stuff

 
fdisk
parted
partprobe
multipath
smartd
 
mkfs
mount
fsadm

5. Create a Linear Volume

5.1. Summary

In this exercise, you will perform steps to make a new filesystem available to the system using the Logical Volume Management tools.

We will begin with a simple linear volume.

5.2. Clean Up Devices

Since we will be reusing the same resources for many exercises, we will begin by wiping everything clean. Don’t worry if you get an error message.

umount /mnt/lab*
vgremove -ff vg_lab
pvremove /dev/vd{b..e}
wipefs -a /dev/vd{b..e}
partprobe

5.3. Physical Volume Creation

pvcreate /dev/vdb

5.4. Volume Group (Pool) Creation

vgcreate vg_lab /dev/vdb

5.5. Logical Volume Creation

lvcreate -y -n lab1 -l 95%FREE vg_lab

5.6. Make and Mount Filesystem

mkfs -t ext4 /dev/vg_lab/lab1
mkdir -p /mnt/lab1
mount /dev/vg_lab/lab1 /mnt/lab1
If this were going to be a persistent filesystem, you would also need to add an entry to /etc/fstab.

5.7. Examine Your Work

List general information about all logical volumes.

lvs
  LV   VG     Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lab1 vg_lab -wi-ao---- 3.79g

List general information about our specific logical volumes.

lvs vg_lab/lab1
  LV   VG     Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lab1 vg_lab -wi-ao---- 3.79g

Now let’s change some of the output information.

lvs -o lv_name,lv_size,lv_attr,segtype,devices vg_lab/lab1
  LV   LSize Attr       Type   Devices
  lab1 3.79g -wi-ao---- linear /dev/vdb(0)

Change the unit of measure to kilobytes. Also drop a few default output data fields and add 'devices' (notice the plus and minus specified with the -o output parameters).

lvs --units k -o +devices -o -copy_percent,move_pv,mirror_log vg_lab/lab1
  LV   VG     Attr       LSize       Pool Origin Data%  Meta%  Convert Devices
  lab1 vg_lab -wi-ao---- 3977216.00k                                   /dev/vdb(0)

Report file system space usage.

df /mnt/lab1
Filesystem              1K-blocks  Used Available Use% Mounted on
/dev/mapper/vg_lab-lab1   3833600    24   3618332   1% /mnt/lab1

6. Extend and Resize a Linear Volume

Create a physical volume on a physical device.

pvcreate /dev/vdc

Add the physical volume to the volume group.

vgextend vg_lab /dev/vdc

Increase the size of the logical volume to make use of the extra space in the volume group.

lvresize -l 95%VG /dev/vg_lab/lab1

It is not always optimal to allocate 100% of a volume group to the logical volumes. For example, the unused space in the volume group could be used for a temporary snapshot.

Expand the file system to the size of the logical volume.

resize2fs /dev/vg_lab/lab1

6.1. Examine Your Work

Let us take a look at the logical volume.

lvs -o vg_name,vg_free,lv_name,lv_size,seg_size,segtype,devices vg_lab/lab1
  VG     VFree   LV   LSize SSize  Type   Devices
  vg_lab 408.00m lab1 7.59g <4.00g linear /dev/vdb(0)
  vg_lab 408.00m lab1 7.59g <3.60g linear /dev/vdc(0)

Notice a few things:

  • we added seg_size to the options to report segment size

  • the logical volume is comprised of 2 devices (vdb, vdc)

  • the first segment is roughly 4GB and the second segment is roughly 3.6GB in size

  • Overall, the volume group has 408MB of free space

Report file system space usage again.

df /mnt/lab1
Filesystem              1K-blocks  Used Available Use% Mounted on
/dev/mapper/vg_lab-lab1   7756580    24   7385344   1% /mnt/lab1

Note that the number of available blocks has increased.

7. Create a RAID-10 Volume with Virtual Data Optimizer (VDO)

We will be leveraging devices /dev/vd{b..e}.

As before, we will cleanup up prior work and start fresh.

7.1. Clean Up Devices

Since we will be reusing the same resources for many exercises, we will begin by wiping everything clean. Don’t worry if you get an error message.

umount /mnt/lab*
vgremove -ff vg_lab
pvremove /dev/vd{b..e}
wipefs -a /dev/vd{b..e}
partprobe

7.2. Physical Volume Creation

pvcreate /dev/vd{b..e}
  Physical volume "/dev/vdb" successfully created.
  Physical volume "/dev/vdc" successfully created.
  Physical volume "/dev/vdd" successfully created.
  Physical volume "/dev/vde" successfully created.

7.3. Volume Group Creation

vgcreate vg_lab /dev/vd{b..e}
Volume group "vg_lab" successfully created

7.4. Logical Volume Creation

This time, we are going to use four disks to create a mirrored set of striped disks, otherwise known as RAID 10.

lvcreate -y --type raid10 -m1 -i 2 -n lv_raid10 -l 95%FREE vg_lab

7.5. Add VDO Deduplication

lvconvert --type vdo-pool --name=lab2 -V 30G vg_lab/lv_raid10 -y
  WARNING: Converting logical volume vg_lab/lv_raid10 to VDO pool volume with formatting.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
    The VDO volume can address 4.00 GB in 2 data slabs, each 2.00 GB.
    It can grow to address at most 16.00 TB of physical storage in 8192 slabs.
    If a larger maximum size might be needed, use bigger slabs.
  Logical volume "lab2" created.
  Converted vg_lab/lv_raid10 to VDO pool volume and created virtual vg_lab/lab2 VDO volume.

Create an XFS filesystem on the logical volume.

mkfs.xfs -K /dev/mapper/vg_lab-lab2
mkdir /mnt/lab2
mount /dev/mapper/vg_lab-lab2 /mnt/lab2
To make the mount persistent across reboots, you would still need to either add a systemd unit to mount the filesystem, or add an entry to /etc/fstab.

7.6. Create Sample Data

Let us now populate the filesystem with some content. Create a bunch of random subdirectories in our new filesystems with the following command.

for i in {1..20} ; do mktemp -d /mnt/lab2/XXXXXX ; done

Now we will copy the same content into each of the folders as follows.

This could take a few minutes. 
for i in /mnt/lab2/* ; do echo "${i}" ; cp -rf /usr/share/locale $i ; done

The previous command should have copied approximately 170MB in 20 folders yielding about 3.5GB of traditional filesystem consumption.

7.7. Examine Your Work

Let us now check some statistics.

du -sh /mnt/lab2
3.5G    /mnt/lab2
df -h /mnt/lab2
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/vg_lab-lab2   30G  4.1G   26G  14% /mnt/lab2
vdostats --human-readable
Device                      Size      Used Available Use% Space saving%
vg_lab-lv_raid10-vpool      7.6G      3.8G      3.8G  49%           95%

So in summary, we built a 30GB filesystem that only has ~8GB of actual physical disk capacity. We then copied roughly 3.5GB of data into the filesystem. After deduplication vdostats --human-readable shows a savings of 95%.

Why do we see 3.8GB of used space? A large portion of the physical space is being used for VDO’s metadata, such as the deduplication index and block map. The high overhead is due to the relatively small size of the VDO volume. On a very large volume (multi-terabyte), this initial overhead might be only 1-2%. However, on a small volume, this same amount of reserved space can account for a much larger percentage of the total physical blocks.

A few additional high-level things to know about VDO.

VDO uses a high-performance deduplication index called UDS to detect duplicate blocks of data as they are being stored. The deduplication window is the number of previously written blocks which the index remembers. The size of the deduplication window is configurable. The index will require a specific amount of RAM and a specific amount of disk space.

Red Hat generally recommends using a "sparse" UDS index for all production use cases. This indexing data structure requires approximately one-tenth of a byte of DRAM (memory) per block in its deduplication window. On disk, it requires approximately 72 bytes of disk space per block.

The default configuration of the index is to use a "dense" index. This index is considerably less efficient (by a factor of 10) in DRAM, but it has much lower (also by a factor of 10) minimum required disk space, making it more convenient for evaluation in constrained environments.

Please refer to the Red Hat Documentation further information on provisioning and managing your data with VDO.

8. Conclusion

This concludes the exercises related to lvm and vdo.

Time to finish this unit and return the shell to it’s home position.

workshop-finish-exercise.sh

9. Additional Resources

Red Hat Documentation:

End of Unit