I recently learned Rasperry Pi can be netbooted following this workshop by Alex Ellis. It’s amazing, you should check it out. Seriously! Invest in yourself.
After setting up the Pi cluster, I put it to work by installing kube-vip and then K3S (using k3sup).
I want my development cluster to be accessible to the Internet and accomplish that for pennies. I decided to use port-forwarding through my router to get Ingress to work with the public Internet. Here is a diagram showing my setup.
My K3S cluster consists of four Raspberry Pi’s, each runs a kube-vip pod. They elect a leader, and the leader node’s MAC address is assigned to the VIP (virtual IP address). If the leader node goes offline, a new leader is elected on another node, and I can continue using the same VIP for my port forwarding. Without a VIP, I’d be forced to forward to one node, and if that node were to go offline, I’d lose access to my cluster.
Port forwarding to one cluster is a short term solution to facilitate development w/o cost. My needs are currently simple, I can manage the router and associated risk, and as things change I will adjust accordingly.
Long term, I plan to use tunneling via Inlets Operator (check this out for a primer). There are limitations associated with my short term plan:
- I am relying on port-forwarding…I can forward the ports from my router once. What if I had many clusters at home, each with their own ingress?
- I use a cronjob to manage the public IP address associated with my Ingress’s host DNS entry. What if my IP changes? I’ll have to wait for the job to run.
Why am I sharing this post? I was really impressed with kube-vip, and that resulted in a Twitter thread. The knowledge and software in the CNCF ecosystem is powerful. Check it out!
I guess @kylos101 might be able to share his findings? 😀
— Dan (@thebsdbox) July 11, 2021
To setup Traefik to use the VIP provided by kube-vip, install it like so:
# Set service.externalIPs to your kube-vip $VIP
# For example, my $VIP is 192.168.2.200
helm upgrade --install traefik traefik/traefik \
--namespace kube-system \
--set additional.checkNewVersion=false \
--set additional.sendAnonymousUsage=false \
--set dashboard.ingressRoute=false \
--set service.externalIPs=$VIPReferences: