private-schrijfsels-en-notities/2025/xg-infra-upgrade-2025.md

tags
freebsd homelab

NAS

• clean install newest FreeBSD 14.2
  • move OS /home into /data/home zpool/home
  • mount ISO over IPMI
• Manage old zpool:
  • (old OS) zpool export zpool
  • (new OS) zpool import -N zpool
  • zpool status
  • zpool upgrade
  • zpool upgrade zpool
  • https://docs.freebsd.org/en/books/handbook/zfs/#zfs-zpool-upgrade
• Syncthing shares on separated ZFS subvolumes
  • zfs autosnapshot retetion policies
• pyrotechnics & private data zfs copies=2?
• Applications in VMs
  • Photoprism
  • Homeassistant? or in jail?
• Applications in seperate freebsd jails
  • Syncthing
  • Transmission
  • (existing -> upgrade) Gitea
  • Template FreeBSD 14.2
  • Samba
  • VTVBB sync + go tooling
  • Cache: pkg + freebsd update (for jails)
  • Reverse caddy HTTP + SSL proxy
  • Bastion SSH host
  • Wireguard VPN
    • https://genneko.github.io/playing-with-bsd/networking/freebsd-wireguard-jail/
• Data partitioning
  • zroot (ssd OS only)
  • zpool (14TB hdd mirror)
    • /data/home

Host OS services:

• SSH
• sshguard
• pf firewall

Improvements & things to not forget:

• ZFS disable atime https://www.unixtutorial.org/zfs-performance-basics-disable-atime/
• backup settings from /etc and /usr/local/etc before SSD OS disk wipe
• private keychains daily snapshots (separate Syncthing share + copies=2?)
• Syncthing
  • per-share zfs subvolume
  • per-share needs .zfs ignore or else snapshots are propagated
• crontab(s) backup
• samba config
• gitea backup
• sshguard
• jails settings backup
• vanilla jails management with templates
• Migrate from zfstools auto-snapshot and prune to Python zfs-autobackup?
  • See blog https://it-notes.dragas.net/2024/08/21/automating-zfs-snapshots-for-peace-of-mind/
• URLs for (web)services with nanodash for homelab + quick access
• Upgrade gitea and migrate sqlite to postgres
  • https://forum.gitea.com/t/migrate-from-sqlite-to-postgresql/2269/
  • https://tutorialinux.com/today-learned-migrating-sqlite-postgres-easy-sequel/
• Migrate all automations Hue -> Home Assistant
• Samba network share
  • AVAHI/Bonjour autodiscovery
• Automount network shares on macOS
• ZFS zpool scrub monthly cron
• Homeassistant in FreeBSD jail rc.d service file for auto-start on boot
• ZFS volumes are seen as block devices zfs create -V for use with VMs for better performance
• FreeBSD pkg cache for jails
  • https://omussell.github.io/fbsd-update-cache/
  • https://forums.freebsd.org/threads/nginx-pkg-cache-help.85699/ + https://serverfault.com/a/1026574

Syncthing share enrol on ZFS subvolume

1. Create zfs subvolume: zfs create ...
2. Set zfs-auto-snapshot property (for zfstools) : zfs set ..
3. Create share in Syncthing web GUI
4. Ignore .zfs folder (to not propagate to connected peers): filter .zfs

Syncthing migrate to zfs subvolume

# zfs create zpool/data/syncthing/shared/...
# zfs set snapdir=hidden zpool/data/syncthing/shared/...
(optional) # zfs set copies=2 zpool/data/syncthing/shared/...
# rsync --perms --archive --progress /data/syncthing/shares/... /data/syncthing/shared/...
# echo ".zfs" > /data/syncthing/shares/.../.stignore
# zfs set com.sun:auto-snapshot=true zpool/data/syncthing/shared/...
# zfs list -t snap zpool/data/syncthing/shared/...

ZFS dataset datablock copies

For extra redundancy amount of datablock copies can be set and tested:

# zfs create data/test-dataset/dataset-1
# zfs list
# zfs set copies=2 data/test-dataset/dataset-1
# zfs get copies data/test-dataset/dataset-1
root@mango:/data/test-dataset/dataset-1 # dd if=/dev/random of=testfile bs=64K count=1024
1024+0 records in
1024+0 records out
67108864 bytes transferred in 0.609759 secs (110058049 bytes/sec)
root@mango:/data/test-dataset/dataset-1 # ls -lah
total 131146
drwxr-xr-x  2 root wheel    3B Dec 19 19:56 .
drwxr-xr-x  3 root wheel    3B Dec 19 19:55 ..
-rw-r--r--  1 root wheel   64M Dec 19 19:57 testfile
root@mango:/data/test-dataset/dataset-1 # zfs list | grep dataset-1
data/test-dataset/dataset-1   128M   410G   128M  /data/test-dataset/dataset-1

OpenZFS VM performance and database block size

https://klarasystems.com/articles/openzfs-storage-best-practices-and-use-cases-part-3-databases-and-vms/ https://github.com/openzfs/zfs/issues/7631

Syncthing per-share zfs subvolume

For important shares data we set copies=2 and checksum=sha256

# zfs create zpool/syncthing/shares/myshare
# zfs set copies=2 zpool/syncthing/shares/myshare
# zfs set checksum=sha256 zpool/syncthing/shares/myshare

Which shares?:

• jerry/pyrotechnics
• jerry/private

ZFS snapshot visibility

Hide the .zfs/snapshot directory in the dataset so Syncthing doesn't sync it by accident when not ignored via .stignore:

# zfs list -t snapshot zpool/data/syncthing/shared/jerry/private
# zfs get snapdir zpool/data/syncthing/shared/jerry/private
# zfs set snapdir=hidden zpool/data/syncthing/shared/jerry/private
# ls /data/syncthing/shared/jerry/private/.zfs/snapshot

To make it visible again

# zfs set snapdir=visible

See https://docs.oracle.com/cd/E78901_01/html/E78912/gprhq.html

See also

• freebsd-notes
• freebsd-jail-vanilla
• zfs-snapshots
• So you wanna do FreeBSD 14.1 and native Home Assistant?
• computers-and-machines

External docs

• https://klarasystems.com/articles/openzfs-understanding-zfs-vdev-types/
• https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/