Home Assistant on Kubernetes

Because I need more complexity in my life.

So, I am aware that there is a certain degree of interest in this particular way of running Home Assistant, and having just refactored my Home Assistant installation to work this way, thought I’d share the relevant configuration.

I should tell you in advance though, gentle reader, that this isn’t quite the way I would set this up if I were starting with a blank slate. I’ll mention a couple of these differences as we go along, but most prominently is that my MQTT server (which binds the whole thing together) isn’t in Kubernetes, largely because it has existed since before I was running either of Kubernetes or Home Assistant, and there seemed very little point in replacing something that works perfectly well. But fortunately this is a small thing: a Mosquitto container inside the cluster would work in exactly the same way; it just means changing the hostname in various config files appropriately.

For those wondering, the cluster is an eight-node multiple-architecture cluster, incorporating six Raspberry Pis and two amd64 machines. Kubernetes is supplied by k3s, using the default containerd back-end.

Nothin’ But Core

So here’s the basic Home Assistant Core part; note, please, that this is hand-rolled. I’m not using the HA Helm Chart because it doesn’t quite fit my requirements:

	---
	apiVersion: v1
	kind: Namespace
	metadata:
	name: homeassistant
	---
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	labels:
	app: homeassistant
	name: homeassistant
	namespace: homeassistant
	spec:
	replicas: 1
	selector:
	matchLabels:
	app: homeassistant
	template:
	metadata:
	labels:
	app: homeassistant
	spec:
	volumes:
	- name: ha-storage
	nfs:
	server: mnemosyne.arkane-systems.lan
	path: "/swarm/harmony/homeassistant/ha"
	containers:
	- image: homeassistant/home-assistant:2021.2.3
	name: home-assistant
	volumeMounts:
	- mountPath: "/config"
	name: ha-storage
	---
	apiVersion: v1
	kind: Service
	metadata:
	name: homeassistant
	namespace: homeassistant
	spec:
	selector:
	app: homeassistant
	ports:
	- protocol: TCP
	port: 8123
	name: http
	---
	apiVersion: networking.k8s.io/v1
	kind: Ingress
	metadata:
	name: homeassistant-ingress
	namespace: homeassistant
	annotations:
	traefik.ingress.kubernetes.io/router.entrypoints: 'websecure'
	traefik.ingress.kubernetes.io/router.tls: 'true'
	spec:
	rules:
	- host: ha.harmony.arkane-systems.lan
	http:
	paths:
	- pathType: Prefix
	path: /
	backend:
	service:
	name: homeassistant
	port:
	number: 8123

view raw ha-base.yaml hosted with ❤ by GitHub

Nice and simple, eh? This puts the Home Assistant core container in its own namespace (homeassistant), creates a service for its UI, and then shares its UI via the Traefik instance (plus inbuilt k3s and keepalived load-balancing) I have set up for the cluster.

You’ll note the volume section of the deployment; this sets up a volume for the Home Assistant configuration which is shared from my NAS using NFS (mnemosyne:/swarm/harmony/homeassistant/ha); the “swarm” part of the name is legacy, carried over from when I was using Docker Swarm as my container orchestrator, “harmony” is the name of the cluster, and the two-level directory name at the end is because, for convenience, I store the files of other containers closely related to Home Assistant inside the greater homeassistant folder.

I generally NFS-mount storage for my k8s pods when I can to simplify backup issues and to avoid hammering the SD cards in the Pis. But I can already hear some of you muttering “But you shouldn’t do that with the recorder database (by default HA’s inbuilt SQLite), that’s a fast track to data corruption….”

Well, you’re right.

recorder:
  db_url: mssql+pyodbc://ha:<PASSWORD>@calmirie.arkane-systems.lan/ha_recorder?charset=utf8;DRIVER={FreeTDS};Port=1433;

But as it happens, I have a SQL Server installation elsewhere on the network, so it’s easy enough to put the recorder database over there.

Adding On

Next up, the former add-ons:

	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: node-red
	namespace: homeassistant
	labels:
	app: node-red
	spec:
	replicas: 1
	selector:
	matchLabels:
	app: node-red
	template:
	metadata:
	labels:
	app: node-red
	spec:
	containers:
	- name: node-red
	image: nodered/node-red:latest
	ports:
	- containerPort: 1880
	name: node-red-ui
	securityContext:
	privileged: true
	volumeMounts:
	- name: node-red-data
	mountPath: /data
	env:
	- name: TZ
	value: America/Chicago
	volumes:
	- name: node-red-data
	nfs:
	server: mnemosyne.arkane-systems.lan
	path: "/swarm/harmony/homeassistant/node-red"
	---
	apiVersion: v1
	kind: Service
	metadata:
	name: node-red
	namespace: homeassistant
	spec:
	selector:
	app: node-red
	type: ClusterIP
	ports:
	- name: node-red-ui
	port: 1880
	protocol: TCP
	targetPort: node-red-ui
	---
	apiVersion: networking.k8s.io/v1
	kind: Ingress
	metadata:
	name: node-red-ingress
	namespace: automation
	annotations:
	traefik.ingress.kubernetes.io/router.entrypoints: 'websecure'
	traefik.ingress.kubernetes.io/router.tls: 'true'
	spec:
	rules:
	- host: node.harmony.arkane-systems.lan
	http:
	paths:
	- pathType: Prefix
	path: /
	backend:
	service:
	name: node-red
	port:
	number: 1880
	---
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	labels:
	app: ring-mqtt
	name: ring-mqtt
	namespace: homeassistant
	spec:
	replicas: 1
	selector:
	matchLabels:
	app: ring-mqtt
	template:
	metadata:
	labels:
	app: ring-mqtt
	spec:
	volumes:
	- name: ring-config
	nfs:
	server: mnemosyne.arkane-systems.lan
	path: "/swarm/harmony/homeassistant/ring-mqtt"
	containers:
	- image: tsightler/ring-mqtt:4.4.0
	name: ring-mqtt
	env:
	- name: "MQTTHOST"
	value: "ariadne.arkane-systems.lan"
	- name: "ENABLEPANIC"
	value: "true"
	- name: "ENABLEVOLUME"
	value: "true"
	# - name: "RINGTOKEN"
	# value: ""
	- name: "DEBUG"
	value: "ring-mqtt"
	volumeMounts:
	- mountPath: "/data"
	name: ring-config
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	labels:
	app: grocy
	name: grocy
	namespace: homeassistant
	spec:
	replicas: 1
	selector:
	matchLabels:
	app: grocy
	template:
	metadata:
	labels:
	app: grocy
	spec:
	volumes:
	- name: grocy-storage
	nfs:
	server: mnemosyne.arkane-systems.lan
	path: "/swarm/harmony/homeassistant/grocy"
	containers:
	- image: linuxserver/grocy:version-v3.0.1
	name: grocy
	env:
	- name: PUID
	value: "1004"
	- name: PGID
	value: "1000"
	- name: TZ
	value: "America/Chicago"
	volumeMounts:
	- mountPath: "/config"
	name: grocy-storage
	---
	apiVersion: v1
	kind: Service
	metadata:
	name: grocy
	namespace: homeassistant
	spec:
	selector:
	app: grocy
	ports:
	- protocol: TCP
	port: 80
	name: http
	---
	apiVersion: networking.k8s.io/v1
	kind: Ingress
	metadata:
	name: grocy-ingress
	namespace: homeassistant
	annotations:
	traefik.ingress.kubernetes.io/router.entrypoints: 'websecure'
	traefik.ingress.kubernetes.io/router.tls: 'true'
	spec:
	rules:
	- host: grocy.harmony.arkane-systems.lan
	http:
	paths:
	- pathType: Prefix
	path: /
	backend:
	service:
	name: grocy
	port:
	number: 80
	---
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	labels:
	app: esphome
	name: esphome
	namespace: homeassistant
	spec:
	replicas: 1
	selector:
	matchLabels:
	app: esphome
	template:
	metadata:
	labels:
	app: esphome
	spec:
	hostNetwork: true
	volumes:
	- name: esphome-storage
	nfs:
	server: mnemosyne.arkane-systems.lan
	path: "/swarm/harmony/homeassistant/esphome"
	containers:
	- image: esphome/esphome:1.16.2
	name: esphome
	env:
	- name: ESPHOME_DASHBOARD_USE_PING
	value: "true"
	- name: ESPHOME_DASHBOARD_RELATIVE_URL
	value: "/"
	volumeMounts:
	- mountPath: "/config"
	name: esphome-storage
	---
	apiVersion: v1
	kind: Service
	metadata:
	name: esphome
	namespace: homeassistant
	spec:
	selector:
	app: esphome
	ports:
	- protocol: TCP
	port: 6052
	name: http
	---
	apiVersion: networking.k8s.io/v1
	kind: Ingress
	metadata:
	name: esphome-ingress
	namespace: homeassistant
	annotations:
	traefik.ingress.kubernetes.io/router.entrypoints: 'websecure'
	traefik.ingress.kubernetes.io/router.tls: 'true'
	spec:
	rules:
	- host: esphome.harmony.arkane-systems.lan
	http:
	paths:
	- pathType: Prefix
	path: /
	backend:
	service:
	name: esphome
	port:
	number: 6052
	---

view raw ha-addon.yaml hosted with ❤ by GitHub

What you see here are the Kubernetes configurations for four of the five addons that I used to use (Node-RED, Ring-MQTT, Grocy, and ESPHome). These are relatively self-explanatory. Of the obvious things to point out, you can see that Node-RED, Grocy, and ESPHome all have their own ingresses set up on their own hostnames, since in Home Assistant Core there is no supervisor to do this for them. Ring-MQTT doesn’t need one, since it only makes outgoing network connections.

Of course, you’ll need to install the Home Assistant integrations for these in HA yourself. In most cases, you these integrations are happy to talk back and forth over the MQTT server, or using the ingress URL, but when you need to reference the cluster-internal address of an add-on - since there’s no need to expose anything that doesn’t have an external use - you can do so using the normal k8s convention of

<service>.homeassistant.svc.cluster.local

Elsewhere

As I mentioned, the MQTT server for my setup isn’t in the cluster at the moment. There are also two other parts of my Home Assistant configuration that also aren’t in the cluster.

One is the Zigbee2MQTT add-on. It could have been, certainly, but since I already have the Zigbee hardware hooked up to a different machine and it wasn’t convenient to relocated it to one of the cluster nodes, I left it there. It still integrates just fine.

And the other? Well, you’ll notice that the hostnames given in the ingresses above are all internal ones, i.e., *.harmony.arkane-systems.lan. I prefer not to expose my cluster to the Internet directly, so its Traefik proxy just supports proxying from within to without the cluster itself. I have an nginx proxy set up elsewhere which handles all external HTTPS traffic destined for anywhere on the internal network, which terminates SSL connections to HA from, for example, Alexa skills at its external hostname, and then passes them on to ha.harmony.arkane-systems.lan.

Feel free to leave me a comment if you would like more details on any particular thing!