Setting Up Cluster Pi with Kubernetes

There are a great many things that can be done with a cluster, even a low power one like a Raspberry Pi cluster where the worker nodes are only single core Pi Zeros. I have decided to play around with Kubernetes, with Ansible to administer the whole cluster, just for the sheer fun of it!

Hat's off to Jeff Geerling as I've used his tutorials (links at the bottom) to assist in setting this all up.

However... this guide will end up with a small cluster that is using most of its limited resources to run kubernetes and nothing left over to deploy anything useful - you have been warned :-)

Hardware

I've chosen to use a Pi 3 with 4x Zero Pis (no point in using ZeroW as they are connected in USB Gadget mode anyway, so have networking that way). If you use a Pi 4 as the controller, you can't reboot the controller without power cycling the worker nodes. Of course, you'll also need a Cluster HAT! I'd also recommend the Cluster HAT Case to house the cluster.

Operating Systems

Download and install all necessary images from ClusterCTRL... remember to add an empty "ssh" file in each boot partition to automatically enable SSH on all nodes. I picked the BRIDGE controller - primarily so that I can easily access each worker from my Desktop PC.

Connect everything up and power up the cluster - only the controller Pi will power up, as the ClusterHAT needs to be instructed to apply power to all of the workers.

That's the easy bit! Now for just a few more steps...

Instructions

  1. ssh into the controller Pi and power up the workers:
    clusterctrl on
  2. ssh into all nodes to apply all updates, change the default password and node names (if desired, using raspi-config). My nodes called:
    cluster-pi-c (controller)
    cluster-pi-w1 (worker 1)
    cluster-pi-w2 (worker 2)
    cluster-pi-w3 (worker 3)
    cluster-pi-w4 (worker 4)
  3. On the controller Pi create an RSA key pair:
    ssh-keygen -b 2048
  4. Then copy the public key to each of the workers, for example:
    ssh-copy-id pi@cluster-pi-w1
    (I also do that from my desktop PC to make remote login quicker)
  5. On whichever computer you wish to administer the entire cluster from, install ansible:
    sudo apt install ansible
    And install kubectl (follow these instructions).
  6. Then download k3s-ansible from rancher's Git repo
    git clone https://github.com/rancher/k3s-ansible.git
  7. Enter the k3s-ansible directory
    cd k3s-ansible
  8. Copy the sample host inventory file and edit to list the worker hostnames or IP addresses.
    cp inventory/sample/hosts.ini inventory/.
    nano inventory/hosts.ini
  9. Copy the sample inventory/group_vars/all.yml file and edit to change the ansible_user to pi (for use with a Raspbian based distro)
    cp -R inventory/sample/group_vars inventory/.
    nano inventory/group_vars/all.yml
  10. Run the installed playbook which will download and install k3s on all nodes and then configure them appropriately:
    ansible-playbook site.yml -i inventory/hosts.ini
    (I had to run this twice, with a reset.yml playbook in between, I think that the configuration of the ClusterCTRL controller, meant that the workers failed to download the k3s image first time around. The script isn't quite tailored for a ClusterPi bridge network.)
  11. Copy the kubectl configuration from the controller pi to the administration PC:
    scp pi@cluster-pi:~/.kube/config ~/.kube/config-cluster-pi
  12. Check that the kubernetes nodes are up and running:
    export KUBECONFIG=~/.kube/config-cluster-pi
    kubectl get nodes
    All well and good, the result should look something like this:
    NAME STATUS ROLES AGE VERSION
    cluster-pi Ready master 23m v1.17.5+k3s1
    cluster-p1 Ready <none> 22m v1.17.5+k3s1
    cluster-p2 Ready <none> 22m v1.17.5+k3s1
    cluster-p3 Ready <none> 22m v1.17.5+k3s1
    cluster-p4 Ready <none> 22m v1.17.5+k3s1

Unfortunately, here's where it breaks down a bit... the Pi zeros are so stretched just running the kubernetes framework that they've got no resources left to run any deployment. You may be able to run a "hello world" type thing though :-)

An interesting experience, but I'll save it for a slightly more capable cluster :-)

Helpful Resources

More geared towards a cluster of Pi4s or the Turing Pi 1, but still can be applied to the ClusterHAT.




Del.ico.us Digg Facebook Google LinkedIn LiveJournal NewsVine reddit StumbleUpon Twitter
Valid XHTML 1.0 Transitional Valid CSS! [Valid Atom 1.0] [Valid RSS 2.0]
[ Page last updated Sun 6th Dec 2020 | viewed 86 times ]