Master Stratis Storage Management and VDO (Virtual Data Optimizer) for Thin Provisioning, Snapshots, and Data Deduplication
Modern storage requirements demand more than basic partitioning and filesystem management. CentOS 8.5 introduces two powerful advanced storage technologies: Stratis for simplified storage pool management with built-in snapshots, and VDO (Virtual Data Optimizer) for transparent data deduplication, compression, and thin provisioning.
This comprehensive guide covers installation, configuration, management, and real-world scenarios for both Stratis and VDO. You'll learn how to create thin-provisioned filesystems that grow dynamically, take instantaneous snapshots, implement storage-level data deduplication, and optimize storage utilization. All procedures are tested and verified on CentOS 8.5.
Understanding Advanced Storage Concepts
The Storage Management Analogy
Think of traditional storage like managing individual filing cabinets — each has fixed space and you need to plan carefully. Stratis is like a smart storage room where you can instantly create new filing cabinets of any size from a shared pool, take snapshots of entire cabinets, and they automatically expand as needed. VDO is like a magical compression system that automatically removes duplicate documents and compresses everything, giving you much more usable space than your physical storage suggests.
Why Advanced Storage Features Matter
- Storage Efficiency - Thin provisioning and deduplication maximize storage utilization
- Operational Simplicity - Automated management reduces administrative overhead
- Snapshot Capabilities - Instantaneous point-in-time copies for backup and recovery
- Dynamic Growth - Filesystems automatically expand as data is written
- Cost Savings - Reduce physical storage requirements through optimization
| Feature | Traditional LVM | Stratis | VDO |
|---|---|---|---|
| Thin Provisioning | Manual setup required | Built-in and automatic | Built-in and automatic |
| Snapshots | Manual, space-intensive | Automatic, space-efficient | Not available |
| Data Deduplication | Not available | Not available | Built-in block-level |
| Compression | Not available | Not available | Built-in (LZ4) |
| Management Complexity | High (multiple commands) | Low (single tool) | Medium (dedicated tool) |
| Use Case | Basic volume management | Application storage with snapshots | Backup targets, VM storage |
Stratis Storage Management
Stratis is a next-generation storage management solution that simplifies the creation and management of storage pools and filesystems. It provides automatic thin provisioning, snapshots, and monitoring capabilities with a simple command-line interface.
Stratis Architecture
Pool - Collection of block devices (physical storage)
Filesystem - Thin-provisioned filesystem created from pool space
Blockdev - Individual physical devices added to pools
Snapshot - Point-in-time copy of a filesystem
Installing and Configuring Stratis
- 1 Install Stratis Packages
sudo dnf install -y stratis-cli stratisdThis installs both the daemon (
stratisd) and command-line tool (stratis). - 2 Enable and Start Stratis Daemon
sudo systemctl enable --now stratisd sudo systemctl status stratisdVerify the daemon is active and running.
- 3 Verify Installation
stratis --version stratis pool listThe second command should return empty if no pools exist yet.
Practice Lab 1: Stratis Installation
Install and verify Stratis on your CentOS 8.5 system:
# Install Stratis packages
sudo dnf install -y stratis-cli stratisd
# Start and enable the daemon
sudo systemctl enable --now stratisd
# Verify the service is running
sudo systemctl status stratisd --no-pager
# Check Stratis version and list pools
stratis --version
stratis pool list
# Expected output for empty system:
# Pool Name Total Physical Size Total Physical Used
# no pools present in the pool namespaceManaging Stratis File Systems
Once Stratis is installed, you can create storage pools and filesystems to manage your data efficiently.
Creating and Managing Stratis Pools
- 1 Identify Available Block Devices
lsblk sudo fdisk -lIdentify unused block devices (e.g.,
/dev/sdb,/dev/sdc). Ensure they don't contain important data as Stratis will format them. - 2 Create a Stratis Pool
sudo stratis pool create mypool /dev/sdbYou can add multiple devices:
sudo stratis pool create mypool /dev/sdb /dev/sdc - 3 Verify Pool Creation
stratis pool list stratis blockdev list - 4 Create a Filesystem
sudo stratis filesystem create mypool myfsStratis automatically creates a thin-provisioned XFS filesystem.
- 5 List Filesystems
stratis filesystem listThis shows all Stratis filesystems with their pool, name, used size, and UUID.
Practice Lab 2: Stratis Filesystem Management
Create a Stratis pool and filesystem:
# Identify available disks (use virtual disks in lab)
lsblk
# Create pool with one or more disks
sudo stratis pool create datapool /dev/sdb
# Verify pool creation
stratis pool list
# Output should show:
# Pool Name Total Physical Size Total Physical Used
# datapool 10.00 GiB 54.00 MiB
# Create filesystem
sudo stratis filesystem create datapool webdata
# List filesystems
stratis filesystem list
# Output should show:
# Pool Name Name Used Created Device
# datapool webdata 546 MiB Nov 15 2023 10:30 /stratis/datapool/webdataFilesystem Properties and Monitoring
Stratis filesystems are thin-provisioned, meaning they can be larger than the available physical space. The filesystem automatically grows as data is written, up to the pool's capacity.
Understanding Stratis Space Usage
- Total Physical Size - Total size of all block devices in the pool
- Total Physical Used - Actual space used on physical devices
- Filesystem Used - Logical space used by the filesystem
- Thin Provisioning - Filesystem size can exceed physical space
Persistently Mounting Stratis File Systems
Stratis filesystems need to be mounted persistently to survive reboots. Unlike traditional filesystems, Stratis uses a special mount mechanism.
Mounting Stratis Filesystems
- 1 Create Mount Point
sudo mkdir -p /mnt/webdata - 2 Mount Using Filesystem Path
sudo mount /stratis/datapool/webdata /mnt/webdataStratis filesystems are accessible via
/stratis/<pool>/<filesystem>. - 3 Get Filesystem UUID
stratis filesystem listNote the UUID for the filesystem you want to mount persistently.
- 4 Add to /etc/fstab
echo "UUID=<filesystem-uuid> /mnt/webdata xfs defaults,x-systemd.requires=stratisd.service 0 0" | sudo tee -a /etc/fstabThe
x-systemd.requires=stratisd.serviceoption ensures Stratis daemon starts before mounting. - 5 Test Mount Configuration
sudo umount /mnt/webdata sudo mount -a df -h /mnt/webdata
Practice Lab 3: Persistent Mounting
Configure persistent mounting for your Stratis filesystem:
# Create mount point
sudo mkdir -p /var/www/html
# Get filesystem UUID
stratis filesystem list
# Example output UUID: 12345678-1234-1234-1234-123456789012
# Add to fstab (replace UUID with actual value)
echo "UUID=12345678-1234-1234-1234-123456789012 /var/www/html xfs defaults,x-systemd.requires=stratisd.service 0 0" | sudo tee -a /etc/fstab
# Test the configuration
sudo umount /stratis/datapool/webdata
sudo mount -a
# Verify mount
df -h /var/www/html
ls -la /var/www/htmlStratis Filesystem Snapshots
Snapshots are one of Stratis's most powerful features, allowing you to create instantaneous point-in-time copies of filesystems with minimal space overhead.
Creating and Managing Snapshots
- 1 Create a Snapshot
sudo stratis filesystem snapshot datapool webdata webdata-snap-20231115Syntax:
stratis filesystem snapshot <pool> <original-fs> <snapshot-name> - 2 Verify Snapshot Creation
stratis filesystem listThe snapshot will appear as a separate filesystem with the same pool.
- 3 Mount the Snapshot
sudo mkdir -p /mnt/snapshot sudo mount /stratis/datapool/webdata-snap-20231115 /mnt/snapshot - 4 Access Snapshot Data
ls -la /mnt/snapshot cat /mnt/snapshot/somefile.txtSnapshots are read-write by default, but changes don't affect the original filesystem.
- 5 Remove a Snapshot
sudo umount /mnt/snapshot sudo stratis filesystem destroy datapool webdata-snap-20231115
Practice Lab 4: Snapshot Management
Create and use a Stratis snapshot:
# Create some test data in original filesystem
echo "Original data created on $(date)" | sudo tee /var/www/html/testfile.txt
# Create snapshot
sudo stratis filesystem snapshot datapool webdata webdata-backup
# Mount snapshot
sudo mkdir -p /tmp/snapshot-restore
sudo mount /stratis/datapool/webdata-backup /tmp/snapshot-restore
# Verify snapshot contains original data
cat /tmp/snapshot-restore/testfile.txt
# Modify original filesystem (snapshot remains unchanged)
echo "Modified data on $(date)" | sudo tee /var/www/html/testfile.txt
# Verify snapshot still has original data
cat /tmp/snapshot-restore/testfile.txt
# Clean up
sudo umount /tmp/snapshot-restore
sudo stratis filesystem destroy datapool webdata-backupReal-World Scenario: Application Update Rollback
Use snapshots to safely update a web application with rollback capability:
# Before update, create snapshot
sudo stratis filesystem snapshot webpool production-app pre-update-snapshot
# Perform application update
sudo rsync -av /tmp/new-app/ /var/www/html/
# If update fails, rollback using snapshot
sudo systemctl stop httpd
sudo umount /var/www/html
sudo stratis filesystem destroy webpool production-app
sudo stratis filesystem snapshot webpool pre-update-snapshot production-app
# Remount and restart
sudo mount UUID=<new-fs-uuid> /var/www/html
sudo systemctl start httpd
# If update succeeds, remove snapshot after verification period
sudo stratis filesystem destroy webpool pre-update-snapshotGuided Exercises: Thin Provisioning and Dynamic Growth
These exercises demonstrate the core benefits of Stratis: thin provisioning and automatic filesystem growth.
Guided Exercise 1: Create Thin-Provisioned Filesystem
Create a filesystem that appears larger than available physical space:
# Create pool with 10GB physical disk
sudo stratis pool create thinpool /dev/sdb
# Create filesystem (appears as 1TB even with 10GB pool)
sudo stratis filesystem create thinpool largefs
# Check filesystem size
df -h /stratis/thinpool/largefs
# Output shows filesystem size much larger than pool
# Verify with Stratis command
stratis filesystem list
# Shows logical size vs physical usageGuided Exercise 2: Verify Dynamic Growth
Demonstrate automatic filesystem expansion as data is written:
# Mount the filesystem
sudo mkdir -p /mnt/largefs
sudo mount /stratis/thinpool/largefs /mnt/largefs
# Check initial usage
df -h /mnt/largefs
stratis pool list
# Create large file to consume space
sudo dd if=/dev/zero of=/mnt/largefs/bigfile bs=1M count=2000
# Check usage after writing data
df -h /mnt/largefs
stratis pool list
# Physical usage should increase while filesystem size remains constant
# Continue until pool is nearly full
sudo dd if=/dev/zero of=/mnt/largefs/bigfile2 bs=1M count=8000
# Attempt to write beyond pool capacity (should fail gracefully)
sudo dd if=/dev/zero of=/mnt/largefs/toobig bs=1M count=1000
# Write will fail when pool is exhaustedGuided Exercise 3: Create and Access Snapshot Data
Work with snapshot data in a practical scenario:
# Create database backup scenario
echo "CREATE DATABASE production;" | sudo tee /mnt/largefs/init.sql
echo "INSERT INTO users VALUES (1, 'admin');" | sudo tee -a /mnt/largefs/init.sql
# Create snapshot before major operation
sudo stratis filesystem snapshot thinpool largefs pre-migration-snap
# Simulate database migration
echo "ALTER TABLE users ADD COLUMN email VARCHAR(255);" | sudo tee -a /mnt/largefs/init.sql
# Access snapshot to verify original state
sudo mkdir -p /tmp/original-db
sudo mount /stratis/thinpool/pre-migration-snap /tmp/original-db
cat /tmp/original-db/init.sql
# Shows original data without email column
# Compare with current state
cat /mnt/largefs/init.sql
# Shows modified data with email column
# Clean up
sudo umount /tmp/original-db
sudo stratis filesystem destroy thinpool pre-migration-snapVDO - Virtual Data Optimizer
VDO (Virtual Data Optimizer) is a Linux kernel module that provides inline data deduplication, compression, and thin provisioning. It's ideal for workloads with repetitive data like virtual machines, backups, and container images.
VDO Architecture
Physical Storage - Actual block devices (HDD/SSD)
VDO Volume - Logical volume with deduplication/compression
Logical Size - Apparent size presented to filesystem (can be larger than physical)
Index Memory - RAM used for deduplication metadata (1GB per 1TB logical)
VDO Benefits
- Block-level Deduplication - Eliminates duplicate 4KB blocks across entire volume
- LZ4 Compression - Compresses unique data blocks
- Thin Provisioning - Logical size can exceed physical capacity
- Transparent Operation - Applications see normal block device
- Performance Optimized - Designed for minimal CPU overhead
Installing and Configuring VDO
VDO requires specific packages and careful configuration to achieve optimal results.
Installation Process
- 1 Install VDO Packages
sudo dnf install -y vdo kmod-kvdoThis installs the VDO management tools and kernel module.
- 2 Load VDO Kernel Module
sudo modprobe kvdo lsmod | grep kvdoVerify the module is loaded successfully.
- 3 Enable VDO Service
sudo systemctl enable --now vdo sudo systemctl status vdoThe VDO service manages VDO volumes at boot time.
Practice Lab 5: VDO Installation
Install and verify VDO on your CentOS 8.5 system:
# Install VDO packages
sudo dnf install -y vdo kmod-kvdo
# Load kernel module
sudo modprobe kvdo
# Verify module loaded
lsmod | grep kvdo
# Expected output: kvdo 2345678 0
# Enable and start VDO service
sudo systemctl enable --now vdo
# Verify service status
sudo systemctl status vdo --no-pager
# Check VDO version
sudo vdo --versionCreating and Formatting VDO Volumes
Creating a VDO volume requires careful consideration of physical size, logical size, and index memory requirements.
VDO Volume Creation
- 1 Identify Physical Device
lsblk sudo fdisk -lSelect an unused block device (e.g.,
/dev/sdc). - 2 Calculate Logical Size
For deduplication workloads, logical size can be 2-10x physical size:
# Example: 10GB physical disk, 50GB logical size # Index memory needed: ~50MB (1GB per 1TB logical) - 3 Create VDO Volume
sudo vdo create --name=vdo1 --device=/dev/sdc --vdoLogicalSize=50GKey parameters:
--name- VDO volume name--device- Physical block device--vdoLogicalSize- Logical size (optional, defaults to physical size)--indexMem- Index memory in GB (optional, auto-calculated)
- 4 Verify VDO Creation
sudo vdo status --name=vdo1 ls /dev/mapper/vdo1 - 5 Format VDO Volume
sudo mkfs.xfs -K /dev/mapper/vdo1The
-Kflag disables discard operations which can interfere with VDO. - 6 Mount VDO Volume
sudo mkdir -p /mnt/vdo1 sudo mount /dev/mapper/vdo1 /mnt/vdo1
Practice Lab 6: VDO Volume Creation
Create and format a VDO volume:
# Create VDO volume (10GB physical, 30GB logical)
sudo vdo create --name=backup-vdo --device=/dev/sdc --vdoLogicalSize=30G
# Verify creation
sudo vdo status --name=backup-vdo
# Should show physical size: 10.0G, logical size: 30.0G
# Check device mapper
ls -l /dev/mapper/backup-vdo
# Format with XFS (disable discard)
sudo mkfs.xfs -K /dev/mapper/backup-vdo
# Create mount point and mount
sudo mkdir -p /backup
sudo mount /dev/mapper/backup-vdo /backup
# Verify mount
df -h /backup
# Should show 30G sizeTesting VDO Deduplication
Deduplication is VDO's most valuable feature. This section demonstrates how to test and measure deduplication effectiveness.
Deduplication Testing Methodology
- 1 Create Test Data with Duplicates
# Create base file sudo dd if=/dev/urandom of=/tmp/basefile bs=1M count=100 # Create multiple copies (simulates VM templates, backups, etc.) for i in {1..10}; do sudo cp /tmp/basefile /backup/copy$i done - 2 Monitor VDO Statistics
sudo vdostats --human-readableKey metrics to watch:
- Used% - Physical space utilization
- Logical Used - Apparent data size
- Physical Used - Actual space consumed
- 3 Calculate Deduplication Ratio
# Before adding duplicates sudo vdostats --human-readable # After adding duplicates sudo vdostats --human-readable # Deduplication ratio = Logical Used / Physical Used - 4 Test with Real Workloads
# Copy existing data with natural duplicates sudo cp -r /var/log /backup/logs-original sudo cp -r /var/log /backup/logs-duplicate # Monitor space savings sudo vdostats --human-readable
Practice Lab 7: Deduplication Testing
Test VDO deduplication with controlled duplicate data:
# Check initial VDO stats
sudo vdostats --human-readable
# Example output:
# Device Size Used Available Use% Space saving%
# /dev/mapper/backup-vdo 30.0G 104.0M 29.9G 0% 0%
# Create base data (100MB)
sudo dd if=/dev/urandom of=/tmp/testdata bs=1M count=100
# Copy data 5 times to VDO volume
for i in {1..5}; do
echo "Copying iteration $i"
sudo cp /tmp/testdata /backup/duplicate$i
done
# Check VDO stats after duplication
sudo vdostats --human-readable
# Should show:
# Logical Used: ~500MB
# Physical Used: ~100MB
# Space saving: ~80%
# Calculate actual ratio
# Space saving % = (1 - Physical/Logical) * 100Understanding VDO Statistics
Space saving % = How much physical space is saved due to deduplication and compression
Logical Used = Total size of all data written to the VDO volume
Physical Used = Actual space consumed on the physical device
Index Memory = RAM used for deduplication metadata (critical for performance)
Extending and Removing VDO Volumes
VDO volumes can be extended to use additional physical space, and removed when no longer needed.
Extending VDO Volumes
VDO volumes can only be extended by adding more physical devices or extending the underlying device.
- 1 Extend Underlying Physical Device
If using LVM or virtual disks, extend the physical device first:
# For LVM: extend logical volume sudo lvextend -L+10G /dev/vg0/vdo-lv # For virtual machines: extend virtual disk, then rescan echo 1 | sudo tee /sys/class/block/sdc/device/rescan - 2 Grow VDO Physical Size
sudo vdo growPhysical --name=vdo1This tells VDO to use the additional physical space.
- 3 Grow VDO Logical Size (Optional)
sudo vdo growLogical --name=vdo1 --vdoLogicalSize=60GIncrease logical size if needed (requires available physical space).
- 4 Grow Filesystem
# For XFS filesystems sudo xfs_growfs /backup # Verify new size df -h /backup
Removing VDO Volumes
- 1 Unmount Filesystem
sudo umount /backup - 2 Remove from /etc/fstab
sudo sed -i '/backup/d' /etc/fstab - 3 Stop VDO Volume
sudo vdo stop --name=vdo1 - 4 Remove VDO Volume
sudo vdo remove --name=vdo1This permanently deletes all data on the VDO volume.
Practice Lab 8: VDO Volume Management
Extend and remove a VDO volume:
# Extend physical device (assuming virtual disk was extended)
echo 1 | sudo tee /sys/class/block/sdc/device/rescan
# Grow VDO physical size
sudo vdo growPhysical --name=backup-vdo
# Optionally grow logical size
sudo vdo growLogical --name=backup-vdo --vdoLogicalSize=50G
# Grow filesystem
sudo xfs_growfs /backup
# Verify new sizes
df -h /backup
sudo vdostats --human-readable
# To remove volume:
sudo umount /backup
sudo sed -i '/backup-vdo/d' /etc/fstab
sudo vdo stop --name=backup-vdo
sudo vdo remove --name=backup-vdoAdvanced Configuration and Best Practices
Stratis Best Practices
- Use dedicated disks - Don't mix Stratis pools with other data on same disks
- Monitor pool usage - Set up alerts when pool usage exceeds 80%
- Regular snapshots - Create snapshots before major system changes
- Backup critical data - Snapshots are not backups; use separate backup strategy
- Plan for expansion - Add disks to pools before they become full
VDO Best Practices
- Sufficient RAM - Allocate 1GB RAM per 1TB logical size for index
- Avoid double compression - Don't use VDO with already-compressed data
- Monitor performance - Watch CPU usage during heavy write operations
- Test deduplication ratio - Verify your workload benefits from deduplication
- Use appropriate filesystem - XFS with
-Kflag is recommended
Performance Considerations
Performance Guidelines
Stratis: Minimal performance overhead, suitable for most workloads including databases
VDO: Write operations have CPU overhead due to deduplication/compression; read operations have minimal overhead
Memory Requirements: VDO needs significant RAM for index; Stratis has minimal memory requirements
I/O Patterns: Both work best with sequential I/O; random I/O may reduce deduplication effectiveness
⚠️ Critical Considerations
- Never use VDO for root filesystem - Boot complications and recovery difficulties
- Stratis snapshots are not backups - They reside on the same pool as original data
- VDO requires careful capacity planning - Running out of physical space causes write failures
- Both technologies are relatively new - Test thoroughly in non-production environments first
- Backup your data - Advanced storage features don't replace proper backup strategies
Troubleshooting and Reference
🔧 Common Issues and Solutions
- Stratis pool won't create? → Ensure devices are unused and not mounted; check
dmesgfor errors - VDO creation fails with memory error? → Insufficient RAM for index; reduce logical size or add more RAM
- Filesystem won't mount after reboot? → Verify
/etc/fstabentries and systemd dependencies - Deduplication not working? → Test with truly duplicate data; check VDO statistics with
vdostats - Pool full but filesystem shows space? → Thin provisioning limit reached; add more physical devices to pool
📌 Comprehensive Command Reference
# STRATIS COMMANDS
stratis pool list # List all pools
stratis pool create <name> <device> # Create pool
stratis filesystem list # List filesystems
stratis filesystem create <pool> <fs> # Create filesystem
stratis filesystem snapshot <pool> <orig> <snap> # Create snapshot
stratis filesystem destroy <pool> <fs> # Remove filesystem/snapshot
# VDO COMMANDS
sudo vdo create --name=<name> --device=<dev> # Create VDO volume
sudo vdo status --name=<name> # Show VDO status
sudo vdostats --human-readable # Show space usage
sudo vdo growPhysical --name=<name> # Extend physical size
sudo vdo growLogical --name=<name> --vdoLogicalSize=<size> # Extend logical
sudo vdo stop --name=<name> # Stop VDO volume
sudo vdo remove --name=<name> # Remove VDO volume
# MONITORING
df -h # Filesystem usage
lsblk # Block device overview
free -h # Memory usage (important for VDO)
iostat -x 1 # I/O performance monitoring📚 Additional Resources
man stratis- Stratis command documentationman vdo- VDO command documentationman vdostats- VDO statistics documentation- Red Hat Storage Administration Guide - Official documentation
/var/log/stratisd.log- Stratis daemon logsjournalctl -u vdo- VDO service logs