blog.moebius01.cc

Build a Talos Linux Kubernetes Homelab on Used Mini PCs

A complete, from-scratch guide to building a production-grade Talos Linux Kubernetes homelab on six used mini PCs, managed by Omni — Cilium networking, Longhorn storage, Traefik ingress, and a Woodpecker CI → Flux GitOps pipeline.

TL;DR — This guide walks you from six bare used mini PCs to a fully operating Kubernetes platform in one sitting. Talos Linux makes each node a single immutable binary that boots a Kubernetes node with no SSH and no package manager. Omni gives you a UI and fleet-wide management for provisioning machines, creating clusters, rolling out upgrades, and backing up etcd — without touching a YAML prompt until you want to. On top of that platform I install the four components every homelab needs (Cilium for networking, Longhorn for storage, Traefik for ingress, and cert-manager for TLS), then wire up a Woodpecker CI → Flux GitOps delivery path so that a commit to Git builds, tests, and deploys your application stack entirely automatically. ~4 hours start to finish, all software free and open source, and every command below is either verbatim from the official Sidero/Longhorn/Cilium/Traefik/Flux docs or a transparent best-practice adaptation of them.

If you only take one idea away: the difference between “hobby Kubernetes that keeps breaking” and “a platform you trust” is not the size of the cluster — it is that the OS is immutable, the control plane is managed, storage is replicated, and the applications are declared in Git. This guide gives you all four on six used mini PCs.


Who is this for

You have a handful of retired or second-hand mini PCs gathering dust, or you are shopping for used ultra-small-form-factor business machines on auction sites, and you want a real Kubernetes cluster that stays working. You know what a container and a namespace are, but you have never run Talos or a GitOps pipeline. You want the finished product to be something your side projects and self-hosted services can run on for years, not a weekend tinkering exercise.

You do not need to know Talos, Cilium, Longhorn, or Flux before reading — but you do need to be comfortable running terminal commands and finishing a long task in one sitting. Every concept is explained before it is used.


What you will have when you are done

The topology at a glance

                 ┌──────────────────────────────────────────┐
                 │          Your LAN — 8-port switch        │
                 └───┬────┬────┬────┬────┬────┬────┬────┬───┘
                     │    │    │    │    │    │    │    │
               ┌─────┘    │    │    │    │    │    │    └─────┐
               │          │    │    │    │    │    │          │
          Omni host     CP1-3                   W1-3        work-
        (Ubuntu VM/  5 nodes (3 control       3 worker     station
         laptop)      plane + 3 worker       nodes + 1TB
                         ══ 6 total ══         SSD each)
               │
               │  SideroLink WireGuard (UDP 50180)
               ▼
        ┌──────────────────┐      ┌─────────────────────────────┐
        │     Omni Core     │      │   Kubernetes cluster (Talos)│
        │   + Dex OIDC      │─────▶│   Cilium · Longhorn ·        │
        │   (your login)    │ Wire │   MetalLB · Traefik ·        │
        └──────────────────┘      │   Woodpecker CI · Flux CD    │
                                  └──────────────┬──────────────┘
                                                 ▲
                              merging to main  ──┘ (Git remote → Flux)

That is the whole architecture in one picture: hardware on the switch → Omni manages the nodes over WireGuard → Kubernetes cluster → platform components layered on it → Git drives delivery.

The six nodes

NodeRoleCPURAMStorage
BetelgeuseCP 1i3-8100T (4-core)32 GB256 GB NVMe
BellatrixCP 2i3-8100T (4-core)32 GB256 GB NVMe
RigelCP 3 (vPro)i7-8700T (6-core)32 GB256 GB NVMe
MintakaWorker 1i3-8100T (4-core)32 GB256 GB NVMe + 1 TB SSD
AlnilamWorker 2i3-8100T (4-core)32 GB256 GB NVMe + 1 TB SSD
AlnitakWorker 3i3-8100T (4-core)32 GB256 GB NVMe + 1 TB SSD

The names Orion (cluster) and Betelgeuse, Bellatrix, Rigel, Mintaka, Alnilam, Alnitak (nodes) are illustrative — pick your own naming theme; the recipe is identical whatever you call them.


Table of Contents

  1. Why Talos Linux, and why Omni
  2. Hardware: the used mini PC, honestly
  3. Network design on a flat switch
  4. Before you begin: account for Omni’s two paths
  5. Step 1 — Provision the Omni management host
  6. Step 2 — Get your workstation tooling ready
  7. Step 3 — Prepare the hardware BIOS for Talos
  8. Step 4 — Get a boot image and register the nodes
  9. Step 5 — Create the cluster
  10. Step 6 — Install the networking layer: Cilium
  11. Step 7 — Install storage: Longhorn
  12. Step 8 — Install ingress: MetalLB + Traefik + cert-manager
  13. Step 9 — Bring up a registry and Woodpecker CI
  14. Step 10 — GitOps delivery with Flux
  15. Step 11 — Run your application stack
  16. Day 2: upgrading Talos, upgrading Kubernetes, and recovering
  17. Operational checklist and gotchas

1. Why Talos Linux, and why Omni

Talos Linux

Talos Linux is a Kubernetes-specific operating system. Everything you normally fight on a general-purpose Linux distro is either gone or made declarative:

  • No SSH. There is no shell and no package manager. You manage the node over its API (talosctl) — every maintenance action is a single API call against a well-defined surface, not ad-hoc shell commands.
  • Immutable and minimal. The OS is ~a few hundred MB of bootable image, and the root filesystem is read-only. There is nothing to drift, nothing to patch at runtime, nothing to clean up.
  • Kubernetes is the control plane’s extension. The machine config describes your cluster (its control-plane role, network, storage, extensions, CNI), and Talos renders the kubelet, etcd, kube-apiserver, and all the static pods for you. You do not run kubeadm init or hand-edit kubelet systemd units.

The practical payoff: a node that struggles to fall out of a known state. If it boots, it boots into the exact config you declared; if the config is wrong, it refuses and tells you.

Omni

Omni is Sidero Labs’ management plane for Talos — “SaaS-simple Kubernetes on your own hardware.” Omni sits beside your cluster (never inside it) and does the operational work of running multiple nodes:

  • Connects each Talos node to it over SideroLink, a WireGuard tunnel (so your nodes only need outbound UDP, not inbound firewall holes).
  • Lets you register bare-metal machines by booting one image, click machines into a cluster, and pick control-plane vs worker roles.
  • Has a proper UI for the tedious, error-prone operations: rolling Talos upgrades and Kubernetes upgrades (keeps etcd quorum while it works), automatic etcd backups, node locking for staged rollouts, and fleet-wide status.

You can run Omni two ways:

PathCostWhen
Hosted Omni (Sidero SaaS)Free tier for homelabs / small deployments; paid beyondZero operations, but your nodes talk to Sidero’s cloud
Self-hosted Omni (this guide)Free for non-production / homelab use (BSL license, no license required for home lab)Data stays at home; you run one Docker service

This guide self-hosts Omni on a small Ubuntu VM / laptop on the same switch. That keeps the published path fully self-contained. (If you would rather not host anything, the hosted option is identical from Step 4 onward — you just skip the Omni provisioning and log into their dashboard instead.)


2. Hardware: the used mini PC, honestly

The classic homelab node is a retired ultra-small-form-factor (USFF) business mini PC — the kind companies cycle out by the pallet and resellers move for €40–80 each. Dell, HP and Lenovo all make them (OptiPlex Micro, ProDesk Mini, ThinkCentre Tiny respectively). Any of them is a superb base: small, cheap on the used market, quiet, and low power. But the “Micro/Tiny/Mini” form factor has real trade-offs you must plan around before you buy or configure.

Which exact model? It does not matter much — they are near-identical for this purpose. The important part is what they all share: one gigabit NIC, one NVMe slot, one SATA bay, no dedicated out-of-band controller. This guide is written generic to that class; the specific BIOS keystrokes can differ a little between vendors and generations, so have your model’s manual open for Step 3.

The six nodes this guide assumes

NodeCPURAMBoot diskData diskRole
BetelgeuseCore i3-8100T (4C/4T)32 GB256 GB NVMe—Control plane 1
BellatrixCore i3-8100T32 GB256 GB NVMe—Control plane 2
RigelCore i7-8700T (6C/12T, vPro/AMT)32 GB256 GB NVMe—Control plane 3 (OOB mgmt)
MintakaCore i3-8100T32 GB256 GB NVMe1 TB SSDWorker 1 + Longhorn
AlnilamCore i3-8100T32 GB256 GB NVMe1 TB SSDWorker 2 + Longhorn
AlnitakCore i3-8100T32 GB256 GB NVMe1 TB SSDWorker 3 + Longhorn

A correction many people trip on: used “tiny” business machines are almost all built around a specific, non-upgradeable socket generation. The OptiPlex 3060 Micro, for instance, is 8th-generation Intel only — there is no 9th-gen CPU that fits it; the i7 tops out at the 8700T. In practice that means: match the CPU to the model you are actually buying, expect an older chip (7th–10th gen depending on the unit and age), and do not plan to “upgrade the processor later” — you cannot. That is fine: a 4-core/8-thread T-class chip is plenty for a single Kubernetes node of this role. If you specifically need a newer generation, buy a newer-generation machine rather than assuming yours can be swapped.

The one thing that shapes everything: the single NIC

This whole class of machine has exactly one 1 GbE port and no second network interface. There is no dedicated out-of-band management NIC (no iDRAC/iLO). Everything — etcd peer traffic, kubelet, the API server, ingress return traffic — shares that one gigabit link. For a homelab this is fine, and it is exactly why the whole cluster sits behind one good switch with everything on one flat L2 network. It also means:

  • Plan writes to Longhorn across 1 GbE. Replicated storage across a single gigabit link is fine for small stateful workloads and databases; it is not a fast SSD SAN. Keep replica-count=3 but expect realistic gigabit throughput.
  • No out-of-band power/console by default. That is what makes the i7-8700T’s Intel vPro/AMT valuable: Rigel can be power-cycled, its console viewed, and its boot order changed over AMT even when Talos is down. The i3 nodes have no vPro — for those, a serial console or physical access is your recovery path (plus Talos’ own self-healing, see Day 2).

The pleasant hardware facts

  • RAM: 2× SODIMM slots on virtually every model in this class — check your specific unit’s manual for the cap (the vendor’s “official max” is often conservative; 32 GB per node, as here, is the normal, safe choice for a 6-node lab).
  • Disk: one M.2 2280 slot (SATA or PCIe NVMe) — that is your Talos boot disk, and one 2.5″ SATA bay (up to 2 TB) — that is your Longhorn data disk. Note there is usually not a second NVMe slot — the wireless card’s M.2 key is not a disk slot.
  • TPM 2.0 is on-board on essentially all of them — use it for Secure Boot (Talos supports it via UKI).
  • Idle power ~5–10 W per node. Your whole six-node cluster idles around 30–60 W and tops out near 100 W. That is cheap to run all year.
  • BIOS gotchas these units ship with: Secure Boot is often off, TPM may be disabled, and — the important one — SATA Operation often defaults to RAID ON, which can hide disks from an OS installer. You must fix all three before Talos boots. Full checklist in Step 3.

3. Network design on a flat switch

All six nodes, the Omni host, and your workstation plug into one 8-port unmanaged gigabit switch. Because it is an unmanaged flat switch, keep the design deliberately simple:

PathRequirement
Nodes ↔ Omni hostNodes reach Omni over TCP 443 (Omni API/UI) and UDP port specified for WireGuard (default 50180). Nodes connect outbound over SideroLink — no inbound pinholes needed.
Nodes ↔ nodesSame L2 subnet (etc, kubelet, Cilium east-west).
Workstation ↔ OmniTCP 443 to the Omni host, plus trust in the CA you create.
Servers on the internetOutbound 443/80 so Talos can pull container images from registries.

Assign static IPs to every node (DHCP reservation on your router, or static config later). The guide uses a 10.0.0.x range — substitute your own.

HostIP (example)
Omni host10.0.0.10
Betelgeuse (CP1)10.0.0.11
Bellatrix (CP2)10.0.0.12
Rigel (CP3)10.0.0.13
Mintaka (W1)10.0.0.21
Alnilam (W2)10.0.0.22
Alnitak (W3)10.0.0.23
MetalLB service range10.0.0.100–10.0.0.199

Why a flat design wins here: this class of machine has a single NIC, so you cannot cleanly separate management from data on the same port anyway. One subnet, one switch, everything flat, is the simplest thing that works and the easiest to troubleshoot. If you outgrow it, move to a managed switch + VLANs later — but that is a “later” problem, and putting VLANs in now would add a sharp edge to the very first run of a system that has enough new concepts already.


4. Before you begin: account for Omni’s two paths

The Omni self-hosted story changed between versions. If you read older tutorials you will see a bootstrap flow built around SIDEROLABS_OMNI_ACCOUNT_ID, SIDEROLABS_OMNI_USER_ID, and an omnictl omni-rotate command that “unlocks” a fresh install. That path is gone. Current Omni (v0.41+ / Talos v1.13) configures the container by CLI flags or a YAML config file, and creates its first user via OIDC (Dex in this guide) plus --initial-users. There is no omni-rotate in current omnictl. Follow the flow in Step 5; ignore the older tutorials.


Step 1 — Provision the Omni management host

You need a small Linux host running Docker, not managed by the cluster (Omni must never run on the Kubernetes cluster it manages — that is a circular dependency). A 2 vCPU / 8 GB Ubuntu VM or an old laptop is plenty (official sizing for under ~50 nodes: 2 vCPU, 8 GB RAM, 200–250 GB SSD).

1.1 Install Docker

# Official Docker install for Ubuntu — docs.docker.com/engine/install
sudo usermod -aG docker $USER
newgrp docker

1.2 Install cfssl (to sign your internal TLS certs)

CFSSL_VERSION=$(curl -sI https://github.com/cloudflare/cfssl/releases/latest \
  | grep -i location | awk -F '/' '{print $NF}' | tr -d '\r')

curl -L -o cfssl \
  https://github.com/cloudflare/cfssl/releases/download/${CFSSL_VERSION}/cfssl_${CFSSL_VERSION#v}_linux_amd64
curl -L -o cfssljson \
  https://github.com/cloudflare/cfssl/releases/download/${CFSSL_VERSION}/cfssljson_${CFSSL_VERSION#v}_linux_amd64
chmod +x cfssl cfssljson
sudo mv cfssl cfssljson /usr/local/bin/

1.3 Set the environment variables the whole install uses

export HOST_PUBLIC_IP=$(curl -s https://ifconfig.me)
export HOST_PRIVATE_IP=$(hostname -I | awk '{print $1}')
export OMNI_ENDPOINT=omni.internal
export AUTH_ENDPOINT=auth.internal
export OMNI_USER_EMAIL="admin@omni.internal"

If the host has no public IP (a pure LAN VM), set HOST_PUBLIC_IP to the IP your LAN machines use to reach it. The nodes must be able to reach it for WireGuard either way.

Add the internal hostnames to /etc/hosts so the host can resolve them:

echo "127.0.0.1 ${OMNI_ENDPOINT} ${AUTH_ENDPOINT}" | sudo tee -a /etc/hosts

1.4 Create an internal root CA and a wildcard server cert

Omni and Dex serve HTTPS on the internal hostnames. You create a small root CA, use it to sign one certificate covering both endpoints, then install the CA into every machine that will open the Omni UI (your workstation, and the nodes trust it via the mount — Omni mounts the CA into its container).

Create the CA:

cat <<EOF > ca-csr.json
{
  "CN": "Internal Root CA",
  "key": { "algo": "rsa", "size": 4096 },
  "names": [{ "C": "US", "O": "Internal Infrastructure", "OU": "Security" }]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

# trust it on the host itself
sudo cp ca.pem /usr/local/share/ca-certificates/ca.crt
sudo update-ca-certificates

Create the signing config:

cat <<EOF > ca-config.json
{
  "signing": {
    "default": { "expiry": "8760h" },
    "profiles": {
      "web-server": {
        "usages": ["signing", "key encipherment", "server auth"],
        "expiry": "8760h"
      },
      "client": {
        "usages": ["signing", "key encipherment", "client auth"],
        "expiry": "8760h"
      }
    }
  }
}
EOF

Create the wildcard server cert (covering both endpoints and both IPs):

cat <<EOF > wildcard-csr.json
{
  "CN": "Internal Wildcard",
  "hosts": [
    "${OMNI_ENDPOINT}",
    "${AUTH_ENDPOINT}",
    "127.0.0.1",
    "${HOST_PUBLIC_IP}",
    "${HOST_PRIVATE_IP}"
  ],
  "key": { "algo": "rsa", "size": 4096 }
}
EOF

cfssl gencert \
  -ca=ca.pem \
  -ca-key=ca-key.pem \
  -config=ca-config.json \
  -profile=web-server wildcard-csr.json | cfssljson -bare server

cat server.pem ca.pem > server-chain.pem
chmod 644 server*.pem

1.5 Create the etcd encryption key

Omni encrypts the data it writes to etcd at rest with a GPG key (no passphrase — Omni reads the file on startup and cannot prompt):

gpg --batch --passphrase '' \
  --quick-generate-key \
  "Omni (Used for etcd data encryption) omni@internal.local" \
  rsa4096 cert never

FINGERPRINT=$(gpg --with-colons --list-keys "omni@internal.local" \
  | awk -F: '$1 == "fpr" {print $10; exit}')

gpg --batch --passphrase '' \
  --quick-add-key ${FINGERPRINT} rsa4096 encr never

gpg --export-secret-key --armor omni@internal.local > omni.asc

1.6 Set up Dex (the OIDC identity provider)

Omni needs an identity provider. Dex is the lightest thing that works and is what the official guide uses. First create a bcrypt hash for the admin password:

export OMNI_USER_PASSWORD=$(docker run --rm httpd:2.4-alpine \
  htpasswd -BnC 15 admin | cut -d: -f2)

Write the Dex config:

cat <<EOF > dex.yaml
issuer: https://${AUTH_ENDPOINT}:5556

storage:
  type: memory

web:
  https: 0.0.0.0:5556
  tlsCert: /etc/dex/tls/server-chain.pem
  tlsKey: /etc/dex/tls/server-key.pem

enablePasswordDB: true

staticClients:
  - name: Omni
    id: omni
    secret: omni-dex-secret
    redirectURIs:
      - https://${OMNI_ENDPOINT}/oidc/consume

staticPasswords:
  - email: "${OMNI_USER_EMAIL}"
    username: "admin"
    preferredUsername: "admin"
    hash: "$${OMNI_USER_PASSWORD}"
EOF

Note the $${...} above — when writing dex.yaml with a heredoc you want the value of OMNI_USER_PASSWORD, so use ${OMNI_USER_PASSWORD} (single $) inside a normal heredoc, or $$ in the escaping of your chosen shell. The point is the final file must contain the real bcrypt hash string.

Run Dex:

docker run -d \
  --name dex \
  --restart=unless-stopped \
  -p 5556:5556 \
  -v $(pwd)/dex.yaml:/etc/dex/dex.yaml:ro,Z \
  -v $(pwd)/server-key.pem:/etc/dex/tls/server-key.pem:ro,Z \
  -v $(pwd)/server-chain.pem:/etc/dex/tls/server-chain.pem:ro,Z \
  ghcr.io/dexidp/dex:v2.41.1 \
    dex serve /etc/dex/dex.yaml

1.7 Start Omni (embedded etcd — the homelab default)

Fetch the latest tag, then run the container with host networking:

export OMNI_VERSION=$(curl -sI https://github.com/siderolabs/omni/releases/latest \
  | grep -i location | awk -F '/' '{print $NF}' | tr -d '\r')
echo "Using Omni version: $OMNI_VERSION"

mkdir -p $HOME/sqlite

# EULA acceptance — set these, or accept via the UI at /eula on first login
export EULA_NAME="Your Name"
export EULA_EMAIL="your@email.com"

docker run -d \
  --name omni \
  --net=host \
  --cap-add=NET_ADMIN \
  --device /dev/net/tun:/dev/net/tun \
  --restart=unless-stopped \
  -v $(pwd)/ca.pem:/etc/ssl/certs/ca-certificates.crt:ro,Z \
  -v $(pwd)/server-key.pem:/server-key.pem:ro,Z \
  -v $(pwd)/server-chain.pem:/server-chain.pem:ro,Z \
  -v $(pwd)/omni.asc:/omni.asc:ro,Z \
  -v $HOME/sqlite:/_out/sqlite:rw,Z \
  ghcr.io/siderolabs/omni:${OMNI_VERSION} \
    --name=omni \
    --cert=/server-chain.pem \
    --key=/server-key.pem \
    --machine-api-cert=/server-chain.pem \
    --machine-api-key=/server-key.pem \
    --machine-api-bind-addr=0.0.0.0:8090 \
    --private-key-source=file:///omni.asc \
    --event-sink-port=8091 \
    --bind-addr=0.0.0.0:443 \
    --k8s-proxy-bind-addr=0.0.0.0:8100 \
    --advertised-api-url=https://${OMNI_ENDPOINT}/ \
    --siderolink-api-advertised-url=https://${OMNI_ENDPOINT}:8090/ \
    --siderolink-wireguard-advertised-addr=${HOST_PUBLIC_IP}:50180 \
    --advertised-kubernetes-proxy-url=https://${OMNI_ENDPOINT}:8100/ \
    --auth-auth0-enabled=false \
    --auth-oidc-enabled=true \
    --auth-oidc-provider-url=https://${AUTH_ENDPOINT}:5556 \
    --auth-oidc-client-id=omni \
    --auth-oidc-client-secret=omni-dex-secret \
    --auth-oidc-scopes=openid --auth-oidc-scopes=profile --auth-oidc-scopes=email \
    --sqlite-storage-path=/_out/sqlite/omni.db \
    --initial-users=${OMNI_USER_EMAIL} \
    --eula-accept-name="${EULA_NAME}" \
    --eula-accept-email="${EULA_EMAIL}"

Why host networking? Omni binds many ports (443, 8090, 8091, 8100) and needs /dev/net/tun for the WireGuard SideroLink tunnel. --net=host is the supported, documented model for the single-VM deployment. Verify it:

docker ps --format "table {{.Names}}\t{{.Status}}"
curl -k https://127.0.0.1:443

Both containers should be Up, and the curl should return the Omni HTML page.

1.8 Point your workstation at it

On your workstation (the machine you will run omnictl/talosctl/kubectl from), copy the CA and install it as a trusted root:

# from the Omni host to your workstation
scp user@omni-host:~/ca.pem ~/Downloads/omni-ca.pem
# ...then on macOS: double-click to install via Keychain (set to Always Trust)
# ...on Linux: sudo cp ~/Downloads/omni-ca.pem /usr/local/share/ca-certificates/omni.crt && sudo update-ca-certificates

Add the internal hostnames on your workstation too, so the browser resolves them:

echo "10.0.0.10 omni.internal auth.internal" | sudo tee -a /etc/hosts

Open https://omni.internal — you should be redirected to Dex, sign in as admin@omni.internal, accept the EULA, and land in the Omni dashboard.


Step 2 — Get your workstation tooling ready

On your workstation install the four CLIs you will use the whole way through:

# macOS/Linux — the Sidero tap installs talosctl, kubectl, and the OIDC login plugin, plus omnictl
brew tap siderolabs/tap
brew trust siderolabs/tap
brew install siderolabs/tap/sidero-tools

# helm (needed from Step 6 onward)
#   macOS:  brew install helm
#   Linux:  curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# flux CLI (Step 10)
curl -s https://fluxcd.io/install.sh | sudo bash     # or: brew install fluxcd/tap/flux

Verify each with talosctl version, omnictl version, helm version, flux --version.


Step 3 — Prepare the hardware BIOS for Talos

Before a single node boots an installer, set the BIOS correctly. With the node powered off, press the setup key on boot (commonly F2 on Dell, F10 on HP, F1/F2 on Lenovo — check your model’s manual) to enter setup.

SettingValueWhy
SATA OperationAHCI (not RAID ON)RAID mode can hide disks from the OS installer. The most common stumble on these machines.
Secure BootEnableEnables Talos’ Secure Boot/UKI support. (Disable only if you hit a firmware quirk.)
TPMEnable (On)Needed for Secure Boot + sealed disk encryption if you enable it.
Intel Virtualization TechnologyEnableFor nested/hardware virt if you run VMs on top later.
VT for Direct I/O (VT-d)EnableDevice passthrough / peripheral virt.
AC RecoveryPower On or Last Power StateNode comes back up after a power loss — important with no UPS on every node.
Boot orderUEFI, network second (after disk)You PXE/ISO-boot once now; normally it boots its installed OS.

Repeat for all six nodes. On the vPro node (Rigel), also note the Intel AMT firmware lets you do all of the above remotely over the network later — configure an AMT admin password and management profile while in BIOS if you want that capability.


Step 4 — Get a boot image and register the nodes

Omni is bring-your-own-machine: you boot a Talos image configured to phone back to Omni, and the nodes appear under Machines. You do not generate talosctl gen config by hand — Omni does all of that from your cluster choices later.

4.1 Download a Talos Omni image

Options:

  • Omni UI (easiest): Home → Download Installation Media → wizard (same as the Image Factory) → pick architecture amd64, Talos version, and boot method → download an ISO (for USB boot) or PXE profile.
  • CLI:
    omnictl download iso --help   # see options; a plain `omnictl download iso` works too
    

If you will join the nodes to Longhorn later, add the two system extensions (siderolabs/iscsi-tools, siderolabs/util-linux-tools) at this screen via the Image Factory — a small config change now avoids a rebuild later. (Full details in Step 7.)

4.2 Boot each node from that image

Write the ISO to a USB stick, or PXE-boot. On boot, the image carries the Omni connection details baked in, so the node establishes its WireGuard tunnel and appears in Machines within a minute or two.

Troubleshooting if a machine does not appear: verify outbound reachability to the WireGuard endpoint (UDP) and port 443 (TCP) shown on the Omni Home panel. Both must pass — the node will not register otherwise. A firewall or NAT rule blocking either silently prevents registration.

4.3 Confirm the six nodes are connected

In the Omni UI under Machines, you should see six Available (unassigned) machines. Note each machine’s ID (copy via the … menu → Copy Machine ID) — you need two of them to build the cluster template later.


Step 5 — Create the cluster

You can create the cluster two ways: UI (fast, click-through) or cluster template (declarative, reusable). Both are shown; use the UI for your first pass, then export the template so the cluster is reproducible.

  1. Clusters → Create Cluster.
  2. Name: orion (a cluster cannot be renamed after creation — choose well).
  3. Pick the Talos and Kubernetes versions (defaults are fine).
  4. Optionally add configuration patches (e.g., to set cluster.network.cni.name: none for Cilium — see Step 6 — or node hostnames/IPs).
  5. In Available Machines, click CP on three nodes and W0/W1/W2 on the other three. Use an odd number of control planes (3) so etcd keeps quorum when one leaves for maintenance.
  6. Optionally select an install disk per machine via the dropdown (default picks the boot disk).
  7. Create Cluster. You are taken to the cluster overview; watch it move from Provisioning → Ready.

If it stalls in Provisioning: check all selected machines are still connected under Machines, and that all three control-plane nodes are healthy — a 3-node control plane needs all three before etcd can establish quorum.

5.2 Pull the admin kubeconfig and set up your tools

export CLUSTER_NAME=orion

# merge Omni + Talos configs, then fetch the admin kubeconfig
omnictl config merge $HOME/Downloads/omniconfig.yaml
talosctl config merge $HOME/Downloads/talosconfig.yaml
omnictl kubeconfig --cluster $CLUSTER_NAME

kubectl get nodes

The first kubectl will open a browser for your OIDC login (Google/GitHub identity via Dex). Confirm all six nodes show Ready:

NAME         STATUS   ROLES    AGE    VERSION
betelgeuse   Ready    control-plane   ...
bellatrix    Ready    control-plane   ...
rigel        Ready    control-plane   ...
mintaka      Ready    <none>   ...
alnilam      Ready    <none>   ...
alnitak      Ready    <none>   ...

KubePrism is active by default. Omni front-ends the Kubernetes API — your kubectl talks to Omni, which proxies to the cluster’s KubePrism endpoint. That is what makes Omni the single auth + proxy layer for kubectl. Later, when we point Cilium at kube-proxy replacement, it uses the same localhost:7445 KubePrism port on the node.

5.3 (Alternative) Declarative create via cluster template

For reproducibility, export the UI-created cluster to a template (omnictl cluster template export --cluster orion -o cluster.yaml), or author one manually:

kind: Cluster
name: orion
kubernetes:
  version: <kubernetes-version>
talos:
  version: <talos-version>
---
kind: ControlPlane
machines:
  - <cp-machine-uuid-1>
  - <cp-machine-uuid-2>
  - <cp-machine-uuid-3>
---
kind: Workers
machines:
  - <worker-machine-uuid-1>
  - <worker-machine-uuid-2>
  - <worker-machine-uuid-3>

Validate, sync, and watch:

omnictl cluster template validate -f cluster.yaml
omnictl cluster template sync -f cluster.yaml --verbose
omnictl cluster template status -f cluster.yaml

Step 6 — Install the networking layer: Cilium

The cluster now runs the Talos default CNI (Flannel-like setup) and kube-proxy. We replace both with Cilium, an eBPF CNI that can entirely replace kube-proxy and give you network policy, encrypted cluster traffic options, and better observability — all in-cluster, all open source.

6.1 Clear the deck: tell Talos you’ll run Cilium instead

If you want the kube-proxy-free setup (recommended — flatter, faster, and Cilium owns the whole data path), create the cluster’s patches so Talos sets cni: none and disables cluster.proxy:

# patch-nokube-proxy.yaml
cluster:
  network:
    cni:
      name: none
  proxy:
    disabled: true

Critical timing caveat: nodes with cni.name: none will appear stuck at bootstrap phase — they will show NotReady until Cilium is actually installed. You have about 10 minutes to apply the Cilium manifests before the node reboots to retry. This is expected behaviour; do not panic when nodes look not-ready right after cluster creation if you are applying Cilium immediately.

You can supply this patch at cluster creation (UI “configuration patches” / in the template under the Cluster document), or apply to the running machineset. The simplest reproducible path is to include the patch in the cluster template from the start.

6.2 Install Cilium via Helm

helm repo add cilium https://helm.cilium.io/
helm repo update

helm install cilium cilium/cilium \
  --namespace kube-system \
  --version 1.18.0 \
  --set ipam.mode=kubernetes \
  --set kubeProxyReplacement=true \
  --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \
  --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \
  --set cgroup.autoMount.enabled=false \
  --set cgroup.hostRoot=/sys/fs/cgroup \
  --set k8sServiceHost=localhost \
  --set k8sServicePort=7445

Why those specific settings on Talos:

FlagWhy
ipam.mode=kubernetesCilium borrows the PodCIDR/ServiceCIDR already defined in Talos (cluster.network), keeping ranges consistent.
kubeProxyReplacement=trueFull kube-proxy replacement (no kube-proxy pods) — matches cluster.proxy.disabled: true.
securityContext.capabilities.*Explicitly grants Cilium the caps it needs. Talos drops SYS_MODULE by default; these values avoid Cilium trying to load kernel modules.
cgroup.autoMount.enabled=false, cgroup.hostRoot=/sys/fs/cgroupTalos already mounts cgroupv2/bpffs correctly; do not let Cilium fight it.
k8sServiceHost=localhost, k8sServicePort=7445Points Cilium at the node’s KubePrism endpoint (the same one Omni proxies) instead of the real API server — the Talos-correct way to do kube-proxy replacement.

Verify:

kubectl -n kube-system get pods -l k8s-app=cilium    # cilium-agent Running on each node
kubectl get nodes                                     # all Ready once CNI is up

If using the keep-kube-proxy alternative instead, drop kubeProxyReplacement, k8sServiceHost, k8sServicePort, and set --set kubeProxyReplacement=false. The rest of the flags stay.

Known Talos/Cilium interop note: if forwardKubeDNSToHost is on (Talos default) together with Cilium’s bpf.masquerade=true, CoreDNS can break. If DNS stops resolving, set forwardKubeDNSToHost=false.


Step 7 — Install storage: Longhorn

Longhorn gives you replicated block storage — any PersistentVolumeClaim becomes a volume with N copies spread across your worker nodes, surviving disk or node failure. With three storage-capable workers (Mintaka, Alnilam, Alnitak each with a 1 TB SSD), replica-count=3 maps one replica per storage node.

7.1 The requirement that surprises everyone: system extensions

Longhorn needs two userspace tools that are not on a stock Talos image — iscsid/iscsiadm (for the iSCSI data path) and fstrim (for disk trimming). These come as Talos system extensions, and on Talos ≥ v1.5 the supported way to add them is to build them into the installer image via the Image Factory, not by editing machine.install.extensions (that field is deprecated).

Step 1 — write a schematic:

# longhorn-schematic.yaml
customization:
  systemExtensions:
    officialExtensions:
      - siderolabs/iscsi-tools
      - siderolabs/util-linux-tools

Step 2 — submit it to the Image Factory to get a content-addressed ID:

curl -X POST \
  --data-binary @longhorn-schematic.yaml \
  https://factory.talos.dev/schematics
# → {"id":"<SCHEMATIC_ID>"}   (same input always yields same id)

Step 3 — point each node’s installer at the custom image:

# installer-image-patch.yaml
machine:
  install:
    image: factory.talos.dev/metal-installer/<SCHEMATIC_ID>:v1.13.0   # match your Talos version

If your nodes are not yet provisioned, add the extensions in the Omni/Image-Factory “Download Installation Media” step instead — picking iscsi-tools + util-linux-tools there bakes them into the boot image, and you never need the schematic dance. The schematic route above is for nodes that are already installed.

Step 4 — apply and reboot (for running nodes):

talosctl patch machineconfig --nodes <NODE_IP> --patch @installer-image-patch.yaml
talosctl reboot --nodes <NODE_IP>

Step 5 — verify the extensions are present:

talosctl -n <NODE_IP> get extensions
# expect: iscsi-tools ... util-linux-tools ... (+ a virtual `schematic` ext whose version is your ID)

7.2 Give Longhorn a dedicated disk on the three storage workers

On Talos v1.10+ use UserVolumeConfig to mount the 1 TB SSD at /var/mnt/longhorn (this replaces the old machine.disks API). First see available disks:

talosctl get disks --nodes <NODE_IP>

Then, on each of Mintaka, Alnilam, Alnitak:

# longhorn-user-disk.yaml
apiVersion: v1alpha1
kind: UserVolumeConfig
name: longhorn
provisioning:
  diskSelector:
    match: disk.transport == 'sata' && !system_disk
  grow: false
talosctl patch machineconfig --nodes <NODE_IP> --patch @longhorn-user-disk.yaml
talosctl reboot --nodes <NODE_IP>

Add the kubelet bind mount so the kubelet (and Longhorn’s tools) can reach the data path:

# kubelet-mount.yaml
machine:
  kubelet:
    extraMounts:
      - destination: /var/mnt/longhorn
        type: bind
        source: /var/mnt/longhorn
        options:
          - bind
          - rshared
          - rw
talosctl patch machineconfig --nodes <NODE_IP> --patch @kubelet-mount.yaml
talosctl reboot --nodes <NODE_IP>

7.3 Open the routing hole: Tailscale / WireGuard on the workers

If your nodes only talk to each other over SideroLink/WireGuard (i.e., you did not put everything on one flat LAN), Longhorn’s iSCSI connections between nodes also need a path. If, as in this guide, all nodes are on the same flat 10.0.0.x L2 subnet, you can safely skip this — the storage traffic rides the normal LAN.

7.4 Relax PodSecurity on the namespace

Talos enforces the baseline PodSecurity admission by default (except kube-system). Longhorn needs privileged pods, so label its namespace:

kubectl create namespace longhorn-system
kubectl label namespace longhorn-system pod-security.kubernetes.io/enforce=privileged

7.5 Install Longhorn via Helm

helm repo add longhorn https://charts.longhorn.io
helm repo update

helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --set defaultSettings.defaultDataPath=/var/mnt/longhorn

Verify:

kubectl -n longhorn-system rollout status deploy/longhorn-driver-deployer
kubectl get pods -n longhorn-system                 # all Running
kubectl get nodes.longhorn.io -n longhorn-system    # one per node, ALLOWSCHEDULING true

7.6 Prove storage works

# longhorn-test-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: longhorn-test-pvc
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: longhorn
  resources:
    requests:
      storage: 1Gi
kubectl apply -f longhorn-test-pvc.yaml
kubectl get pvc longhorn-test-pvc   # → Bound within seconds

Storage-class note: Longhorn defaults to replicaCount: 3. Because only three nodes carry the data disk, a 3-replica volume spans Mintaka/Alnilam/Alnitak. If you ever add more storage workers, Longhorn spreads across them automatically; if you shrink to fewer than 3 storage nodes, reduce the default replica count to match.


Step 8 — Install ingress: MetalLB + Traefik + cert-manager

Right now nothing is reachable from outside the cluster except on NodePorts. You need ingress: a single entrypoint where myapp.example.com routes to the right service with proper TLS.

8.1 MetalLB — give services a real IP on bare metal

Talos has no cloud load balancer, so LoadBalancer services stay Pending until something assigns IPs. MetalLB fills that role on bare metal over L2.

helm repo add metallb https://metallb.github.io/metallb
helm repo update
helm install metallb metallb/metallb --namespace metallb-system --create-namespace

Declare the pool — keep it clear of your DHCP range and node IPs:

# ippool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: lan-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.0.0.100-10.0.0.199
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: lan-l2
  namespace: metallb-system
kubectl apply -f ippool.yaml

Why L2 mode for a single flat switch: L2 is zero-config and works on an unmanaged switch with no BGP router needed. It announces the service IP by ARP from whichever node hosts it. (BGP mode exists for when you want real ECMP/HA across multiple routers — unnecessary here.)

8.2 Traefik — the ingress controller

helm repo add traefik https://traefik.github.io/charts
helm repo update
kubectl create namespace traefik
helm install traefik traefik/traefik --namespace traefik

The default chart creates a LoadBalancer service that MetalLB now gives an IP (e.g., 10.0.0.100). Verify:

kubectl -n traefik get svc traefik
# EXTERNAL-IP should be set from your MetalLB pool

Point your wildcard DNS (or individual records) for *.<your.domain> at that IP once it is stable, or just note the IP and use curl --resolve / hosts-file while testing.

8.3 cert-manager — automatic TLS

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set installCRDs=true

Add a ClusterIssuer; for a homelab either use Let’s Encrypt (needs a public domain + router port-forward to the Traefik IP) or an internal CA:

# letsencrypt-cluster-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: you@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            class: traefik
kubectl apply -f letsencrypt-cluster-issuer.yaml

No public domain / no port-forward? Use your internal CA from Step 1.4 as the issuer instead — create a ClusterIssuer with a ca: secret and let the cert-manager issue certificates from it, and have your clients trust the same root. Works entirely on a closed LAN.

8.4 Test ingress end to end

Deploy a throwaway workload with an Ingress:

# demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: hello, namespace: default }
spec:
  replicas: 2
  selector: { matchLabels: { app: hello } }
  template:
    metadata: { labels: { app: hello } }
    spec:
      containers:
        - name: hello
          image: registry.k8s.io/e2e-test-images/echoserver:2.2
          ports: [{ containerPort: 8080 }]
---
apiVersion: v1
kind: Service
metadata: { name: hello, namespace: default }
spec:
  selector: { app: hello }
  ports: [{ port: 80, targetPort: 8080 }]
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: traefik
  rules:
    - host: hello.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service: { name: hello, port: { number: 80 } }
kubectl apply -f demo.yaml
kubectl get ingress hello          # ADDRESS = traefik IP
curl --resolve hello.example.com:443:10.0.0.100 https://hello.example.com

If curl returns the echoserver response over TLS, your ingress + TLS path is fully working.


Step 9 — Bring up a registry and Woodpecker CI

Your CI needs a place to push images and a server to run pipelines. On a closed LAN, run a container registry (that the cluster pulls from) and Woodpecker (lightweight, self-hosted, open-source CI) — the natural companion to Flux because Flux reads your Git, and Woodpecker builds from your Git.

9.1 A private registry (so the cluster trusts its own images)

Install a small registry with TLS via cert-manager and Traefik so all nodes (and the build runner) can pull/push over HTTPS:

helm repo add twuni https://helm.twun.io
helm repo update
kubectl create namespace registry
helm install registry twuni/docker-registry \
  --namespace registry \
  --set ingress.enabled=true \
  --set ingress.className=traefik \
  --set ingress.hosts[0].host=registry.example.com \
  --set ingress.tls[0].hosts[0]=registry.example.com

Harden it before trusting it: put an auth htpasswd secret in, and usually add a garbage-collection cron. For the guided first pass, TLS is the important part (nodes otherwise reject an insecure registry on a private IP).

9.2 Woodpecker CI — server + agent in one chart

Woodpecker ships as an OCI chart containing both the server and a Kubernetes-backend agent. The agent creates pods per pipeline step in-cluster (RBAC auto-created), which is the correct fit on Talos (no Docker socket to hand out).

helm install woodpecker oci://ghcr.io/woodpecker-ci/helm/woodpecker \
  --version <VERSION> \
  --namespace woodpecker --create-namespace

Configure the public host and an ingress:

# woodpecker-values.yaml
server:
  env:
    WOODPECKER_HOST: https://ci.example.com      # public URL
    WOODPECKER_ADMIN: admin
  ingress:
    enabled: true
    ingressClassName: traefik
    hosts:
      - host: ci.example.com
        paths:
          - path: /
            pathType: Prefix
            port: 8000
helm upgrade woodpecker oci://ghcr.io/woodpecker-ci/helm/woodpecker \
  --namespace woodpecker --values woodpecker-values.yaml

The chart wires the agent to the server automatically (generates the shared secret for the internal woodpecker-server:9000 gRPC endpoint) and defaults the agent to the kubernetes backend — pods per step in the woodpecker namespace, workspaces on a PVC. That is the right default on Talos. If you instead need docker build inside pipelines, you can switch the agent to the docker backend against a host-mounted socket on a worker — useful, but a bigger security surface; start with the Kubernetes backend.

9.3 Add the registry to Woodpecker as a secret

In the Woodpecker UI: Settings → Secrets, add registry_username / registry_password for pushing to registry.example.com. (Mount them in pipelines as seen in Step 11.)


Step 10 — GitOps delivery with Flux

Flux is your continuous delivery: it watches a Git repo and makes the cluster match it. When a manifest, image, or HelmRelease changes in Git, Flux reconciles the cluster to that state — deletion, scaling, and secret rotation all become “point Git at it.” This is the enforcement half of the Woodpecker builds / Flux deploys pair.

10.1 Bootstrap Flux against a Git repo

export GITHUB_TOKEN=<personal-access-token>

flux bootstrap github \
  --owner=<your-github-org-or-user> \
  --repository=homelab-gitops \
  --path=clusters/orion \
  --branch=main \
  --personal=true

What it does: creates the repo if missing, commits the Flux manifests to the path, then installs Flux into the cluster (namespace flux-system) and configures it to sync from that repo path. It is idempotent (safe to re-run, self-upgrades Flux).

Verify:

flux get kustomizations
flux get helmreleases --all-namespaces

10.2 Your GitOps repo layout

A clean layout that grows well:

homelab-gitops/
├── clusters/
│   └── orion/
│       ├── flux-system/        # created by bootstrap
│       └── apps.yaml           # Kustomization → points at apps/
├── apps/
│   ├── cilium/                 # HelmRelease + repo
│   ├── longhorn/
│   ├── traefik/
│   ├── cert-manager/
│   └── myapp/                  # your real application
└── infra/
    └── namespaces/

First fold the Helm-based components (Cilium, Longhorn, Traefik, cert-manager, etc.) into Flux so they become declarative:

cd homelab-gitops
flux create source helm cilium --url=https://helm.cilium.io/ --interval=10m
flux create helmrelease cilium \
  --source=HelmRepository/cilium \
  --chart=cilium --chart-version=1.18.0 \
  --namespace=flux-system \
  --values=<(helm get values cilium -n kube-system -o yaml)   # carry forward your existing values
flux reconcile source git flux-system

The important mental shift: from here on, everything the cluster runs is in this repo. To change a component version, you edit Git and Flux applies it. To deploy a new app, you add a manifest and Flux applies it. Woodpecker builds and pushes the image; Flux’s image automation (optional) can even bump the tag in Git when a new image lands — see Step 11.


Step 11 — Run your application stack

Now the whole point: get your actual application deployed under CI/CD.

11.1 The delivery loop, end to end

sequenceDiagram
    participant D as Developer
    participant G as Git remote
    participant W as Woodpecker CI
    participant R as Registry
    participant F as Flux
    participant K as Cluster

    D->>G: git push (feature → PR → merge to main)
    G->>W: webhook triggers pipeline
    W->>R: build image, push registry.example.com/myapp:v1.2.3
    W->>G: (optional) image automation bumps tag in Git
    G->>F: Flux sees new commit / image
    F->>K: reconciles Deployment / HelmRelease to git state
    K-->>D: myapp.example.com serves new version

11.2 Example app repo + Woodpecker pipeline

.woodpecker.yml in your app repo:

when:
  event: push

steps:
  build-and-push:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      repo: registry.example.com/myapp
      tags: ${CI_COMMIT_SHA}
      registry: registry.example.com
      username:
        from_secret: registry_username
      password:
        from_secret: registry_password

11.3 The Flux manifest that deploys it

# apps/myapp/helmrelease.yaml
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: myapp
  namespace: myapp
spec:
  interval: 5m
  chart:
    spec:
      chart: myapp
      sourceRef:
        kind: HelmRepository
        name: myapp-charts
  values:
    image:
      repository: registry.example.com/myapp
      tag: latest
    ingress:
      enabled: true
      className: traefik
      hosts:
        - host: myapp.example.com
      tls:
        - hosts:
            - myapp.example.com

Secrets note: never commit registry/DB credentials. Use external-secrets (points at Vault/OpenBao or a sealed secret) or a bare SOPS-encrypted secret in Git, per your threat model. On a closed LAN, a SOPS-encrypted secret in the repo is the pragmatic, auditable choice; external-secrets is the scaled-up choice.

11.4 Verify the loop

watch kubectl get pods -n myapp        # rollout driven by Flux, image from Woodpecker
flux reconcile kustomization flux-system   # force a sync if you push while Flux ticks
kubectl -n myapp get ingress            # myapp.example.com → Traefik → your app

That is the complete platform: build on Woodpecker, deploy with Flux, run on Talos, with storage, networking, ingress, and TLS all declarative underneath.


Day 2: upgrading Talos, upgrading Kubernetes, and recovering

The platform keeps running without you. When you do touch it, here is exactly how — both the raw talosctl way and the managed Omni way.

Talos OS upgrades are A/B–partitioned and self-healing

Every Talos upgrade writes the new OS into the inactive boot partition (image A/B), then reboots once. Critically, the previous image is retained, so a failed boot automatically falls back to the prior image — upgrades do not brick your node even if they go wrong.

Upgrade the Talos OS (both ways)

Direct with talosctl (one node at a time; Talos checks etcd quorum and protects the control plane):

# use the talosctl version that MATCHES the currently running cluster
talosctl upgrade --nodes <NODE_IP> \
  --image ghcr.io/siderolabs/installer:v1.13.7

What the node does: cordons itself → drains workloads → shuts down services + unmounts filesystems → installs the new image → sets the bootloader to boot the new image once → kexec-reboots → verifies → makes it permanent → rejoins → uncordons. Follow it live with talosctl dmesg -f or talosctl upgrade --wait.

Managed via Omni (the easy way on 6 nodes): Omni rolls Talos versions fleet-wide — control planes first (one at a time, waiting for etcd health so quorum is never lost), then workers (one at a time by default; raise maxParallelism if you want faster). It verifies the running Kubernetes is compatible with the target Talos before proceeding, and cancelling a stuck upgrade reverts that node to its previous A/B version while still responsive. UI: Clusters → your cluster → Update Talos → pick version.

Upgrade Kubernetes separately

Since Talos v1.0, upgrading Talos does not upgrade Kubernetes — they are independent.

Direct:

# preview first, then run
talosctl --nodes <CP_IP> upgrade-k8s --to v1.36.1 --dry-run
talosctl --nodes <CP_IP> upgrade-k8s --to v1.36.1

upgrade-k8s pre-pulls images, updates every control-plane component, then kube-proxy, then the kubelet on every node — and is safe to re-run (it resumes where it stopped).

Via Omni: Clusters → Update Kubernetes → choose version. Afterward, review the Bootstrap Manifests diff in the Omni sidebar — Omni deliberately does not auto-apply CoreDNS/CNI manifest bumps (so it never overwrites your customizations); apply the diff manually when appropriate.

Manual rollback (new OS booted but workloads break)

talosctl rollback -n <NODE_IP>

Use when the OS upgraded and rejoined fine, but your workloads misbehave on the new version — this flips the boot reference back to the prior A/B slot.

Reset / maintenance mode

A node with no machine config is in maintenance mode — that is what a fresh boot from ISO/PXE looks like, and what a node returns to after a reset.

talosctl reset -n <NODE_IP>                          # full reset (wipe + leave cluster)
# safer: wipe only the state to return to maintenance mode
talosctl reset -n <NODE_IP> --system-labels-to-wipe STATE --system-labels-to-wipe EPHEMERAL

In Omni, destroy a machine gracefully (it resets the state partition and returns the node to Available) rather than deleting it out-of-band — an unmanaged delete leaves the node thinking it is still in the cluster.

etcd backup and disaster recovery

etcd holds all control-plane state; with a 3-node control plane you survive any single node loss, but losing two requires a backup.

Take routine snapshots directly:

talosctl -n <CP_IP> etcd snapshot db.snapshot

Via Omni (automatic): Omni snapshots etcd to your configured backend (local disk on the host, or S3-compatible storage for off-host durability) on an interval you set. Ideal homelab setup: enable automatic backups to an S3 target (e.g., a local SeaweedFS/MinIO-style bucket or an off-site S3) so a burned control plane does not mean a lost cluster.

Catastrophic control-plane loss — restore:

# 1. identify the point of no return
talosctl -n IP etcd members
# 2. wipe the broken CP node's etcd dir
talosctl -n <IP> reset --graceful=false --reboot --system-labels-to-wipe=EPHEMERAL
# 3. wait for etcd to reach Preparing, then restore from snapshot
talosctl -n <IP> bootstrap --recover-from=./db.snapshot
# 4. remaining CPs rejoin once the recovered endpoint is up

In Omni, restore is template-driven: list snapshots, delete the control-plane machineset, add a bootstrapSpec with the snapshot to the exported template, and sync it. Full steps in the [Omni restore docs] — keep at least the “we can restore from a known-good etcd snapshot” confidence written down before you need it.


Operational checklist and gotchas

Before first boot (BIOS):

  • SATA Operation = AHCI (most common failure on these machines)
  • Secure Boot = On, TPM = On
  • Intel VT + VT-d = On
  • AC Recovery = Power On / Last Power State
  • Boot priority = disk first, network second

Cluster hygiene:

  • Keep control-plane nodes odd (3 or 5) — never let them drop to an even count.
  • Store the Omni ca.pem, the etcd GPG key backup, and a Talos talosconfig off-box (encrypted). If the Omni host dies, that etcd key is the only way to read the old data.
  • Never run Omni on the Kubernetes cluster it manages.
  • Keep secrets.yaml / machine configs out of plaintext Git; store patch-based configs instead (full machine configs drift after upgrades).
  • Back up etcd automatically (local + off-site).

Component gotchas:

  • Cilium on Talos: cni: none nodes show NotReady for ~10 minutes until Cilium lands — expected. Use the Talos-specific value set in Step 6.
  • Longhorn on Talos: needs iscsi-tools + util-linux-tools as system extensions baked via Image Factory (Step 7), plus the privileged PodSecurity label. replica-count=3 across your three 1 TB SSD workers.
  • Single NIC: all traffic shares one 1 GbE link per node — fine for a homelab; do not expect an SSD-SAN.
  • Longhorn rebuilds on a slow disk can take a while; use consistent SATA SSDs, and if a volume won’t detach, look for a pod still referencing the PVC first.
  • Traefik bare metal: default LoadBalancer needs MetalLB (Step 8). Without MetalLB, set Traefik nodePort: 30000/30001 and hit any node IP.
  • Woodpecker on the Kubernetes backend needs its agent’s RBAC (chart default creates it) and a workspace PVC.

Power: six nodes idle ~30–60 W. Consider one UPS (a single ~300 W unit can cover the whole stack with a graceful-shutdown strategy) so a grid blip doesn’t lose etcd.


Sources & further reading

This guide was written with every command cross-checked against the official Sidero, Longhorn, Cilium, Traefik, Flux, and Woodpecker documentation (versions current as of late 2026). Run ... --help on any tool if your installed version differs; major-minor bumps occasionally rename flags.

kubernetestalos-linuxomniciliumlonghorntraefikfluxcdwoodpecker-cihomelabgitopsprovisioning