We are using Proxmox as the primary virtualization platform for the Rachelski homelab. Specifically proxmox is used to manage the hardware to provide a VM layer we can then run cluster nodes inside of. Each VM is a node. In some rare situations, as with Home Assistant, we will just have a stand-alone VM outside of the cluster.
The cluster is currently composed of 2 Minisforum MS-01 mini workstations. Each host has 96GB of memory and at least 1 TB of SSD storage.
These are the setup notes for the first 2 servers to function as hardware for a clustered homelab. We will not be using clustering at the proxmox level itself as there does not seem to be a compelling benefit to take on the additional complexity.
The largest benefit of clustering the proxmox nodes that I have been able to work out is that the proxmox cluster can be made to function as a region indicator in Kubernetes if we finish getting the proxmox-cloud-controller installed and running. But this is not compelling enough for me to introduce the complexity of clustering.
At hard disk options choose ZFS (raid 0) and apply to the only hard disk on the system. We are using ZFS for its snapshot features.
For ranch-1 I used the following:
For ranch-2 I used the following:
Note: for reasons I do not understand, the first attempt to install proxmox on ranch-2 failed and I could not get to the web console. I had to install a 2nd time (this time plugged directly into the router extension). Thus the DNS server was left alone to avoid creating issues. But then later manually changed in /etc/resolv.conf to 192.168.200.1
This section is executed from an SSH session on the host ranch-1.rachelski.internal
Start by connecting to the non-production repo as we won't be purchasing proxmox. See Proxmox VE Helper Scripts for the latest versions of these commands starting with Proxmox VE Post Install
bash -c "$(wget -qLO - https://github.com/community-scripts/ProxmoxVE/raw/main/misc/post-pve-install.sh)"
Note: I have disabled high availability since this cluster will be running kubernetes which handles application failover. The services can be re-enabled later.
Then I considered connecting the NAS to this server so that I can use shared resources. However, upon further reading this seems to be a weak practice. In general, the proxmox host is just for serving VMs and LXC containers. The cluster will use Longhorn to back up data to the NAS. We let Pods access the NAS using PVs.
After this I prepared our first Virtual Machine template following the instructions at Perfect Proxmox template with Cloud Image and Cloud Init. Except, instead of installing ubuntu, I used debian with this command:
#wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
wget https://cdimage.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2
qm create 8000 --memory 2048 --core 2 --name debian-cloud --net0 virtio,bridge=vmbr0
qm disk import 8000 debian-12-generic-amd64.qcow2 local-zfs
qm set 8000 --scsihw virtio-scsi-pci --scsi0 local-zfs:vm-8000-disk-0
qm set 8000 --ide2 local-zfs:cloudinit
qm set 8000 --boot c --bootdisk scsi0
qm set 8000 --serial0 socket --vga serial0
One last step was to add a DNS resolver as an LXC container. This was temporary, and the DNS resolver has since moved to the net-1 host.
From here... go to the k3s project and follow the instructions found there.