From 2b165c7b0a080542ef7260ca90c38fb49739a840 Mon Sep 17 00:00:00 2001 From: Jerry Jacobs Date: Sat, 8 Feb 2025 11:14:08 +0100 Subject: [PATCH] vault backup: 2025-02-08 11:14:08 --- .DS_Store | Bin 8196 -> 10244 bytes .obsidian/core-plugins.json | 3 +- .obsidian/workspace.json | 20 +- ...5.sync-conflict-20250126-182027-OZPPIBC.md | 196 ------------------ 4 files changed, 19 insertions(+), 200 deletions(-) delete mode 100644 2025/xg-infra-upgrade-2025.sync-conflict-20250126-182027-OZPPIBC.md diff --git a/.DS_Store b/.DS_Store index 7a2beeacc30f49c55fdbf94f632e44ec6299631b..3adf41f72cd994923f2cc7a876d3ffaf93b29676 100644 GIT binary patch delta 165 zcmZp1XbF&DU|?W$DortDU{C-uIe-{M3-C-V6q~50$jGrVU^hP_$7UXZKK98Q#5fpD zCrb!4h?p81=_r_3n$_wkR9hMu0NG|Hlh+8CvKuiNFc>kIPVN@3+1Mb#F31ct3M3bnJ4qh1d1?0Y|{WqGco|R0m0+~ncmHeqBEJ`RwV)eGiV upgrade) Gitea - * Template FreeBSD 14.2 - * Samba - * VTVBB sync + go tooling - * Cache: pkg + freebsd update (for jails) - * Reverse caddy HTTP + SSL proxy - * Bastion SSH host - * Simple DNS server - * Wireguard VPN - * https://genneko.github.io/playing-with-bsd/networking/freebsd-wireguard-jail/ -* Data partitioning - * zroot (ssd OS only) - * zpool (14TB hdd mirror) - * /data/home - -Host OS services: - * SSH - * sshguard - * pf firewall - -Improvements & things to not forget: -* ZFS disable atime https://www.unixtutorial.org/zfs-performance-basics-disable-atime/ -* backup settings from /etc and /usr/local/etc before SSD OS disk wipe -* private keychains daily snapshots (separate Syncthing share + copies=2?) -* Syncthing - * per-share zfs subvolume - * per-share needs .zfs ignore or else snapshots are propagated -* crontab(s) backup -* samba config -* gitea backup -* sshguard -* jails settings backup -* vanilla jails management with templates -* Migrate from zfstools auto-snapshot and prune to Python zfs-autobackup? - * See blog https://it-notes.dragas.net/2024/08/21/automating-zfs-snapshots-for-peace-of-mind/ -* URLs for (web)services with nanodash for homelab + quick access -* Upgrade gitea and migrate sqlite to postgres - * https://forum.gitea.com/t/migrate-from-sqlite-to-postgresql/2269/ - * https://tutorialinux.com/today-learned-migrating-sqlite-postgres-easy-sequel/ -* Migrate all automations Hue -> Home Assistant -* Samba network share - * AVAHI/Bonjour autodiscovery -* Automount network shares on macOS -* ZFS zpool scrub monthly cron -* Homeassistant in FreeBSD jail rc.d service file for auto-start on boot -* ZFS volumes are seen as block devices `zfs create -V` for use with VMs for better performance -* ZFS scrub & status report e-mail -* FreeBSD pkg cache for jails - * https://omussell.github.io/fbsd-update-cache/ - * https://forums.freebsd.org/threads/nginx-pkg-cache-help.85699/ + https://serverfault.com/a/1026574 -* VMs managed with https://github.com/churchers/vm-bhyve - * zvol (vdev) vm instead of file: https://github.com/churchers/vm-bhyve?tab=readme-ov-file#adding-custom-disks - * https://forum.level1techs.com/t/zvol-vs-file-as-vm-backing-huge-performance-difference-on-nvme-based-zpool/182074/15 - -# Syncthing share enrol on ZFS subvolume - -1. Create zfs subvolume: `zfs create ...` -2. Set zfs-auto-snapshot property (for zfstools) : `zfs set ..` -3. Create share in Syncthing web GUI -4. Ignore .zfs folder (to not propagate to connected peers): filter `.zfs` - -# Syncthing migrate to zfs subvolume - - -``` -# zfs create zpool/data/syncthing/shared/... -# zfs set snapdir=hidden zpool/data/syncthing/shared/... -(optional) # zfs set copies=2 zpool/data/syncthing/shared/... -# rsync --perms --archive --progress /data/syncthing/shares/... /data/syncthing/shared/... -# echo ".zfs" > /data/syncthing/shares/.../.stignore -# zfs set com.sun:auto-snapshot=true zpool/data/syncthing/shared/... -# zfs list -t snap zpool/data/syncthing/shared/... -``` -# ZFS dataset datablock copies - -For extra redundancy amount of datablock copies can be set and tested: - -``` -# zfs create data/test-dataset/dataset-1 -# zfs list -# zfs set copies=2 data/test-dataset/dataset-1 -# zfs get copies data/test-dataset/dataset-1 -root@mango:/data/test-dataset/dataset-1 # dd if=/dev/random of=testfile bs=64K count=1024 -1024+0 records in -1024+0 records out -67108864 bytes transferred in 0.609759 secs (110058049 bytes/sec) -root@mango:/data/test-dataset/dataset-1 # ls -lah -total 131146 -drwxr-xr-x 2 root wheel 3B Dec 19 19:56 . -drwxr-xr-x 3 root wheel 3B Dec 19 19:55 .. --rw-r--r-- 1 root wheel 64M Dec 19 19:57 testfile -root@mango:/data/test-dataset/dataset-1 # zfs list | grep dataset-1 -data/test-dataset/dataset-1 128M 410G 128M /data/test-dataset/dataset-1 -``` - -## Automatic USB backup with devd - -* https://man.freebsd.org/cgi/man.cgi?devd.conf -# OpenZFS VM performance and database block size - -https://klarasystems.com/articles/openzfs-storage-best-practices-and-use-cases-part-3-databases-and-vms/ -https://github.com/openzfs/zfs/issues/7631 - -## Syncthing per-share zfs subvolume - -For important shares data we set copies=2 and checksum=sha256 -``` -# zfs create zpool/syncthing/shares/myshare -# zfs set copies=2 zpool/syncthing/shares/myshare -# zfs set checksum=sha256 zpool/syncthing/shares/myshare -``` - -Which shares: -* jerry/pyrotechnics -* jerry/private -## ZFS snapshot visibility - -Hide the `.zfs/snapshot` directory in the dataset so Syncthing doesn't sync it by accident when not ignored via `.stignore`: - -``` -# zfs list -t snapshot zpool/data/syncthing/shared/jerry/private -# zfs get snapdir zpool/data/syncthing/shared/jerry/private -# zfs set snapdir=hidden zpool/data/syncthing/shared/jerry/private -# ls /data/syncthing/shared/jerry/private/.zfs/snapshot -``` - -To make it visible again -``` -# zfs set snapdir=visible -``` - -See https://docs.oracle.com/cd/E78901_01/html/E78912/gprhq.html - -## Internal DNS server - -* https://blog.marcg.pizza/marcg/a-quick-and-dirty-dns-server-using-freebsd-100daystooffload-day-28 -* https://etherarp.net/dnsmasq/index.html -* https://vlads.me/post/setting-up-dns-adblocker-freebsd-jail/ -# See also - -* [[freebsd-notes]] -* [[freebsd-jail-vanilla]] -* [[zfs-snapshots]] -* [[So you wanna do FreeBSD 14.1 and native Home Assistant?]] -* [[computers-and-machines]] - -# External docs - -* https://klarasystems.com/articles/openzfs-understanding-zfs-vdev-types/ -* https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/ -* https://serverfault.com/questions/1075846/config-for-using-git-via-ssh-on-jump-host -* https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/ - -## Domains - -### Public services - -DNS at Transip - -* git.xor-gate.org -* homeassistant.xor-gate.org - -### Internal services - -Internal services use split-brain DNS - -* homelab.xor-gate.org -* git.xor-gate.org -* homeassistant.xor-gate.org \ No newline at end of file