In my free time I'm running an enterprise Kubernetes cluster on my home server. I'm using it for learning and playing around. Using GitOps with a public repo forces me to think about security, backups, scalability and maintenance.
All apps are deployed via GitOps and FluxCD. I picked FluxCD over ArgoCD because it forces me to use the CLI and not a web interface. Some apps are usable outside my local network. This is done with Cloudflare Tunnel.
- Storage is managed through Longhorn.
- Everything is deployed through FluxCD (GitOps)
- Using a public repo forces me to think more about security.
- All apps are supposed to work in production. First I need a second mini pc for that.
- Secrets are stored inside this git repository. All are encrypted. I'm using CNCF SOPS for encryption.
- Make sure you install
nfs-common. It's used by Longhorn.
- Have an Ubuntu 24 VM or server
- Move into the
ansible/production/oransible/staging/folder. - Add your VM/server's IP to the
inventory.yamlwith a new line under[cpn]and[wn]or[staging].[cpn]stands for control plane node.[wn]stands for worker node.
- Run
ansible-playbook production.yaml/staging.yaml -i inventory.
For production, this will configure Talos Linux and link them into one cluster. For staging, this will configure your VM, install k3s, and install the flux operator.
But be warned. This repo makes use of encrypted secrets via CNCF SOPS. Flux will complain that you never added the correct private key. But longhorn and the prometheus stack should start.
I'm currently using the following technologies:
| Logo | Name | Description |
|---|---|---|
|
Homepage | A modern and open-source dashboard application configurable via a YAML file. It's using Longhorn for storing the config files. |
|
linkding | linkding is a self-hosted open-source bookmark manager. Storage is provided by Longhorn. |
|
Nextcloud | An open-source self-hosted file storage. Made available to the public via a Cloudflare Tunnel. To store files, it uses a volume by Longhorn and uses CloudNativePG for it's database. |
|
n8n | n8n is a workflow automation platform that combines AI capabilities with business process automation. n8n is using Longhorn for storage. On top, it uses CloudNativePG for it's database. |
|
Open-WebUI | Open-WebUI makes use of Ollama to offer a web-interface for accessing local LLMs. It's using Longhorn for storage. Flux uses Helm to install it. |
|
Ollama | Ollama is a tool for running and managing local open-source AI models. It's using the Tinyllama model and Longhorn for storage. |
| Logo | Name | Description |
|---|---|---|
|
CloudNativePG | A cloud-native Kubernetes operator for the PostgreSQL database. CloudNativePG uses Longhorn for storing data. |
| Logo | Name | Description |
|---|---|---|
|
Longhorn | Cloud-native production-ready management for Block Storage for Kubernetes. Used as the default storage option for new persistent volume claims. |
| Logo | Name | Description |
|---|---|---|
|
Talos | Talos Linux is a minimal, hardened, immutable, and open-source Linux distribution designed for Kubernetes. |
| Logo | Name | Description |
|---|---|---|
|
k3s | K3s is a highly available Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances. |
|
Ubuntu | Ubuntu is the modern, open-source operating-system on Linux for the enterprise server, desktop, cloud, and IoT by Canonical. |
Both production and staging use these tools, but also specific tools to their needs. Because I want to be able to check what's going on in staging, I'm using Ubuntu with k3s. In production, I only want to deploy what's working. Therefore I'm using Talos.
| Logo | Name | Description |
|---|---|---|
|
Proxmox | Enterprise software for virtualization. Used to play around and switch from Ubuntu to for example Talos Linux. |
|
FluxCD | A CNCF project for continuous and progressive delivery of Kubernetes applications and infrastructure. It's used to enable GitOps. |
|
Helm | A package manager for Kubernetes. With Helm you can install applications directly to Kubernetes. Used to provide templates which are then reconfigured. |
|
traefik | An ingress reverse proxy used internally by Kubernetes. |
|
Cloudflare Tunnel | Creates a secure connection to Kubernetes pods without a publicly routable IP address. |
|
CNCF SOPS | Encrypts and decrypts files like YAML, JSON, ENV, INI, etc. Used for public secrets. |
|
Ansible | Used to automate the setup of virtual machines. |
| Logo | Name | Description |
|---|---|---|
|
KubeLinter | Open-Source tool for highlighting typos and security problems for Kubernetes YAML manifests. |
I picked the Minisforum mini PC because it supports having a GPU. This will come in handy when using it with Ollama and n8n.
-
MINISFORUM 795S7
- AMD Ryzen 9 7945HX (16C/32T)
- 32GB DRR5 RAM
- 1TB SSD
- RTX 4060 OC 8GL
-
NiPoGi E3B Mini PC
- ΑΜD Ryzen 5 7430U (6C/12T)
- 64GB DDR4 RAM
- 2TB SSD
Control Plane Nodes:
- 3x Talos Linux VM:
- 4 CPU Cores
- 50GB SSD
- 4GB RAM
Worker Nodes:
- 4x Talos Linux VM:
- 3 CPU Cores
- 50GB SSD
- 3GB RAM
Scheduling is deactivated on the control plane nodes.
All nodes are setup by Ansible using ansible/production/.
Control Plane Node:
- Ubuntu 24 VM
- 10 CPU Cores
- 200GB SSD
- 50GB RAM
Scheduling is activated on the control plane node. Only used for testing.