Skip to main content

Replace Drives in the ZPOOL

As of 01JAN2025, my Proxmox is using a ZPOOL for LXC storage. Probably not the best set up, but here is what I currently have:

root:~# zpool status
  pool: ZFS1
 state: ONLINE
  scan: scrub repaired 0B in 00:28:18 with 0 errors on Sun Mar 23 17:04:26 2025
config:

        NAME                                 STATE     READ WRITE CKSUM
        ZFS1                                 ONLINE       0     0     0
          raidz2-0                           ONLINE       0     0     0
            ata-ST4000DM004-2U9104_ZW6357WK  ONLINE       0     0     0
            ata-ST4000DM004-2U9104_ZFN5MJ3H  ONLINE       0     0     0
            ata-ST4000DM004-2U9104_ZFN5PACE  ONLINE       0     0     0
            ata-ST4000DM004-2U9104_ZW633P7H  ONLINE       0     0     0

errors: No known data errors

Eventually I will be updating these to larger capacity drives. I figured I would game plan my expansion now, and document it so I don't forget the process when it comes time to actually put it into place... Don't laugh, I'm not an expert at this!

Doing some prep work to help ensure I don't dork this up, I did a drive map to show what serials are what and to show which /dev/sdx they map to, which is gonna be helpful when replacing and hopefully stops me from crossing things over and getting disk errors. As of 08APR2025, this is my drive map:

root:~# ls -l /dev/disk/by-id/ | grep ST4000
lrwxrwxrwx 1 root root  9 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5MJ3H -> ../../sdb
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5MJ3H-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5MJ3H-part9 -> ../../sdb9
lrwxrwxrwx 1 root root  9 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5PACE -> ../../sdc
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5PACE-part1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZFN5PACE-part9 -> ../../sdc9
lrwxrwxrwx 1 root root  9 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW633P7H -> ../../sdd
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW633P7H-part1 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW633P7H-part9 -> ../../sdd9
lrwxrwxrwx 1 root root  9 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW6357WK -> ../../sda
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW6357WK-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 Mar 18 01:22 ata-ST4000DM004-2U9104_ZW6357WK-part9 -> ../../sda9

 

Per-Drive Upgrade Process (X4)

This process will be repeated for each drive in the pool, one at a time

  1. Physically replace one drive
    1. Power down the server (unless hotswap is available)
    2. Remove one of the old 4TB drives
    3. Insert new drive
    4. Power the server back on

  2. Find the new drive
    1. After booting, run this, and look for a new device that looks like ata-STXXXXXX-XXXXXXXX_newserial
      ls /dev/disk/by-id/
  3. Replace the drive in ZFS
    1. Run the following command and be sure to replace <OLD_SERIAL> with <NEW_DRIVE_ID> accordingly:
      zpool replace ZFS1 ata-ST4000DM004-2U9104_ /dev/disk/by-id/
  4. Monitor the resilver
    1. Run this command, and be sure to wait until it's 100% completed before continuing:
      zpool status

When All 4 Drives Are Replaced

ZFS will not automatically increase the capacity until all drives in a RAIDZ vdev are upgraded

Once all 4 drives are swapped for larger ones and resilvering is done, do the following:

  1. Enable auto-expand (if not already) by running the following command:
    zpool set autoexpand=on ZFS1
  2. Trigger the capacity expansion with this:
    zpool online -e ZFS1
  3. Check the new size with this:
    zpool list
Back to top