People keep asking why my personal platform does not run on Kubernetes. Here is the full answer, with numbers.
TL;DR: my complete DevSecOps platform runs on Docker Swarm, on top of a WireGuard mesh, for about 30 euros a month, real invoice. Reliability is not the problem. The real price is rigidity: adding or removing a node is still a manual job. And the VPN becomes a hard dependency of the orchestrator. I would sign again.
This article is for small teams, solo builders and internal platforms. Anyone torn between Kubernetes and something simpler.
The setup
I run a self-hosted DevSecOps platform for my projects. On it: a Traefik reverse proxy, a shared PostgreSQL, a Keycloak for identity, and monitoring with Prometheus, Grafana and Loki. A Wazuh SIEM can be switched on. A SIEM is the tool that collects and analyzes security events.
Terraform creates the machines. Ansible configures them. Docker Swarm orchestrates the containers.
Everything lives on a handful of small VMs at a European cloud provider. They talk to each other through a WireGuard mesh VPN (NetBird). A mesh VPN is simple: every machine talks to the others through a direct encrypted tunnel.
The real bill
Catalog prices are misleading. My VMs cost a few euros each before taxes. The invoice that lands every month says 30 euros.
# The full platform, per month
1 manager cx32 (4 vCPU, 8 GB)
worker VMs on cx22 (2 vCPU, 4 GB)
1 small dedicated VPN VM
1 cx32 VM for the SIEM, optional
Real invoice: ~30 EUR per month, everything included
The gap between catalog and invoice has names: VAT, IPv4 addresses, backups, volumes.
For dev, I merge monitoring into the manager and turn the SIEM off. The same platform then fits on one or two small VMs.
A managed Kubernetes adds the control plane fee before you even pay for nodes. And a self-managed Kubernetes charges you in time what it does not charge in euros.
The Kubernetes bill also hides documented waste. According to the Cast AI 2026 report, across 23,000+ production clusters, real CPU utilization sits at 8%. And 69% of clusters are overprovisioned on CPU.
What Swarm gives you for free
Swarm ships inside Docker. There is no control plane to install, no extra component to babysit.
My init playbook is short and rerunnable. It initializes the manager, joins the workers one by one, sets labels, creates networks and secrets.
Overlay networks are encrypted. Secrets are native. Service placement is a matter of node labels.
Swarm is boring. For infrastructure, that is a compliment.
Trap 1: docker stack deploy ignores your .env
First trap, and it surprises everyone. docker compose up reads your .env file and substitutes variables. docker stack deploy ignores it completely.
The symptom is nasty. Your Traefik rules contain a literal app.${DOMAIN} instead of your domain. Nothing crashes. Routing just does not work.
This is a known, accepted limitation (moby issue #29133). It has been open for years.
My workaround: I never hand Swarm a raw compose file. Ansible renders each one from a Jinja2 template, variables already substituted. Sensitive values come from Ansible vault, not from an env file.
Trap 2: the MTU of double encapsulation
The second trap is in the network. My containers talk over VXLAN, the Swarm overlay network. And that VXLAN travels inside WireGuard. Two encapsulations, so two headers to fit in every packet.
The MTU is the maximum packet size on a link. Each encapsulation eats a piece of it.
WireGuard refuses to fragment oversized packets. That is a deliberate design choice. A packet that is too big gets dropped, not split.
The rule fits on one line:
# VXLAN adds ~50 bytes of headers
mtu_docker = mtu_vpn - 50
# in my case: 1280 - 50 = 1230
# /etc/docker/daemon.json
{ "mtu": 1230 }
And the test is reproducible with plain ping:
# 1202 bytes + 28 of ICMP header = 1230: passes
ping -M do -s 1202 100.64.0.2
# 1252 + 28 = 1280: rejected, packet too big
ping -M do -s 1252 100.64.0.2
Forget this setting and TCP mostly survives, because it negotiates its segment size. UDP does not. In my case, metrics were timing out with not one useful error line.
The real price: rigidity
My own internal audit of the setup says it in one phrase: "well-designed but architecturally static". I confirm.
Adding a node takes three steps. A VM definition in Terraform. An entry in the inventory template. A rerun of the init playbook.
Removing a node is worse, because everything is manual. Drain the services, remove the node from the Swarm, destroy the VM. In that order, without slipping. It is my biggest documented pain point.
And the matching risk: a node drops out of the Swarm and Terraform does not know. Nobody brings it back. There is no self-healing.
The dependency Kubernetes does not have: the VPN
The whole Swarm control plane rides on the mesh. The manager advertises its VPN address. Workers join through that address.
The consequence is direct: if the VPN goes down, the orchestrator becomes unreachable. The VPN is not a comfort anymore. It is a foundation.
I accepted that trade with open eyes. In exchange, I get encryption everywhere and a platform I can move to any provider.
When I would still pick Kubernetes
This setup has a domain of validity. Mine: a stable platform, one operator, nodes that almost never change.
I would pick Kubernetes the day nodes have to move on their own. Autoscaling, automatic machine replacement, ephemeral environments.
I would also pick it with several teams deploying. Namespaces, RBAC, operators: that machinery exists for exactly this.
Gartner predicted that 80% of large engineering organizations would run a platform engineering team by 2026. If that is you, take Kubernetes. If you are a team of three, that number is not about you.
This is not "Kubernetes is dead". This is "Kubernetes is a cost". Buy it when it pays you back.
The checklist before you choose
Run this list before creating your cluster, in either direction.
- [ ] Count your nodes: below five, Swarm or even docker compose is often enough
- [ ] Price both bills, the provider's and your own time
- [ ] Overlay on a VPN:
mtu_docker = mtu_vpn - 50, verified withping -M do - [ ] Never rely on
.envwithdocker stack deploy: render your compose files - [ ] Write the drain and remove-node playbooks before you need them
- [ ] List what depends on the VPN, and what stays up if it dies
- [ ] Write down today what would make you migrate to Kubernetes
What to remember
A complete platform fits on Swarm for about thirty euros a month. Reliability has not been my problem.
The price is paid elsewhere. Every topology change is a manual project. And the VPN carries everything.
Do the math for your case, not for your resume. The simplest orchestrator that covers your need is the right one.
Torn between Swarm, Kubernetes or something in between for your platform? Let's talk.
Sources: moby issue #29133 (docker stack deploy and .env files) · WireGuard whitepaper (no fragmentation) · NetBird, MTU considerations · Docker, overlay networks · Cast AI, State of Kubernetes Optimization 2026 · Gartner, platform engineering · Hetzner Cloud, pricing
Top comments (0)