Skip to content

Proxmox Cluster and VMs

First PublishedByAtif Alam

First page of the HA Kubernetes on Two Proxmox Hosts series. This page takes two bare machines to a working Proxmox cluster with six Kubernetes-ready VMs.

  • Proxmox VE 8.x ISO flashed to a USB stick.
  • The IP plan from the hub page finalized: static IPs reserved for both hosts, the tiebreaker box, all six VMs, and the API VIP (outside the DHCP pool).
  • The tiebreaker box reachable on the same subnet, with apt and Docker available.

Install Proxmox VE normally on each host (ext4 or ZFS on the single NVMe). During or right after install:

  • Set the planned static IP and a unique hostname per host.
  • Point both hosts at the same NTP source — etcd and corosync both misbehave under clock skew.

On each host:

Terminal window
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/pve-enterprise.list
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/ceph.list 2>/dev/null
apt update && apt full-upgrade -y

Suppress the subscription nag popup (cosmetic; re-run after pve-manager updates):

Terminal window
sed -i.bak "s/data.status !== 'Active'/false/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
systemctl restart pveproxy.service

Step 2: Form the 2-Node Cluster and Add the QDevice

Section titled “Step 2: Form the 2-Node Cluster and Add the QDevice”

A 2-node cluster cannot form quorum alone, so a QDevice on the tiebreaker box provides the third vote. This is Proxmox-level quorum (corosync), independent of the Kubernetes/etcd quorum built later.

On Host A:

Terminal window
pvecm create homelab-cluster

On Host B:

Terminal window
pvecm add <HOST-A-IP>

Verify both hosts see each other:

Terminal window
pvecm status

Expected: 2 nodes listed.

On the tiebreaker box:

Terminal window
apt install corosync-qnetd

On both Proxmox hosts:

Terminal window
apt install corosync-qdevice

From one host:

Terminal window
pvecm qdevice setup <TIEBREAKER-IP>

Verify:

Terminal window
pvecm status

Expected: Total votes: 3 and quorum maintained.

Build a Debian 12 cloud-init template once per host, then clone it.

Terminal window
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2
qm create 9000 --memory 2048 --cores 2 --name debian12-template --net0 virtio,bridge=vmbr0
qm importdisk 9000 debian-12-generic-amd64.qcow2 local-lvm
qm set 9000 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9000-disk-0
qm set 9000 --ide2 local-lvm:cloudinit
qm set 9000 --boot c --bootdisk scsi0
qm template 9000

Per the hub’s VM table (control planes 60 GB, workers 150 GB). Example for cp1 on Host A:

Terminal window
qm clone 9000 101 --name cp1 --full
qm set 101 --memory 4096 --balloon 0 --cores 2
qm resize 101 scsi0 60G
qm set 101 --ipconfig0 ip=<CP1-IP>/24,gw=<GATEWAY-IP>
qm start 101

Repeat for the two workers on Host A (--memory 4096 --balloon 0, qm resize ... 150G), then cp2 and two workers on Host B.

Set —balloon 0 on every VM: no memory ballooning or overcommit. Keep ~4 GB free for the Proxmox host itself; if a host shows memory pressure later, trim worker VMs to 3.5 GB instead of squeezing the hypervisor.

Pin the VMs to their hosts. Do not live-migrate or configure Proxmox HA failover for the control-plane/etcd VMs — the quorum design depends on etcd1 and etcd2 staying on separate physical machines. Migrating cp1 to Host B would put two etcd members on one host and defeat the failure model.

Apply this on every VM (via cloud-init snippets or SSH). Versions come from the hub’s version matrix.

Terminal window
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
Terminal window
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
sudo apt install -y chrony && sudo systemctl enable --now chrony
Terminal window
sudo apt install -y containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl restart containerd && sudo systemctl enable containerd

SystemdCgroup = true is required — kubelet defaults to the systemd cgroup driver, and a mismatch causes pods to restart in loops.

kubeadm, kubelet, kubectl (Pinned and Held)

Section titled “kubeadm, kubelet, kubectl (Pinned and Held)”
Terminal window
sudo apt install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

On the four worker VMs (needed later by the Longhorn page):

Terminal window
sudo apt install -y open-iscsi nfs-common
sudo systemctl enable --now iscsid
Terminal window
pvecm status # on either host: 3 votes, quorate
qm list # on each host: its 3 VMs running
ssh <any-vm> 'systemctl is-active containerd && kubeadm version -o short && free -h | grep -i swap'

Expected: quorum with 3 votes, all six VMs running with their static IPs, containerd active, kubeadm at the pinned version, swap showing 0B.

Next: External etcd and kubeadm Bootstrap.