Microk8s: No such file or directory: '/var/snap/microk8s/1908/var/kubernetes/backend.backup/info.yaml' while joining a cluster

 Kubernetes cluster with microk8s on raspberry pi

If you want to join a node and you get the following error:

microk8s join 192.168.178.57:25000/6a3ce1d2f0105245209e7e5e412a7e54
Contacting cluster at 192.168.178.57
Traceback (most recent call last):
  File "/snap/microk8s/1908/scripts/cluster/join.py", line 967, in <module>
    join_dqlite(connection_parts)
  File "/snap/microk8s/1908/scripts/cluster/join.py", line 900, in join_dqlite
    update_dqlite(info["cluster_cert"], info["cluster_key"], info["voters"], hostname_override)
  File "/snap/microk8s/1908/scripts/cluster/join.py", line 818, in update_dqlite
    with open("{}/info.yaml".format(cluster_backup_dir)) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/var/snap/microk8s/1908/var/kubernetes/backend.backup/info.yaml' 

 This error happens, if you have not enabled dns on your nodes.

So just run "microk8s.enable dns" on every machine:

microk8s.enable dns 
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
Restarting kubelet
Adding argument --cluster-domain to nodes.
Configuring node 192.168.178.57
Adding argument --cluster-dns to nodes.
Configuring node 192.168.178.57
Restarting nodes.
Configuring node 192.168.178.57
DNS is enabled

And after that the join will work like expected:

root@ubuntu:/home/ubuntu# microk8s join 192.168.178.57:25000/ed3f57a3641581964cad43f0ceb2b526
Contacting cluster at 192.168.178.57
Waiting for this node to finish joining the cluster. ..  
root@ubuntu:/home/ubuntu# kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
ubuntu   Ready    <none>   3m35s   v1.20.1-34+97978f80232b01
zigbee   Ready    <none>   37m     v1.20.1-34+97978f80232b01
 

MicroK8s: Kubernetes on raspberry pi - get nodes= NotReady

On my little kubernetes cluster with microK8s

 

 i got this problem:

kubectl get nodes
NAME     STATUS     ROLES    AGE   VERSION
zigbee   NotReady   <none>   59d   v1.19.5-34+b1af8fc278d3ef
ubuntu   Ready      <none>   59d   v1.19.6-34+e6d0076d2a0033

The solution was:

kubectl describe node zigbee

and in the output i found:

Events:
  Type     Reason                   Age                From        Message
  ----     ------                   ----               ----        -------
  Normal   Starting                 18m                kube-proxy  Starting kube-proxy.
  Normal   Starting                 14m                kubelet     Starting kubelet.
  Warning  SystemOOM                14m                kubelet     System OOM encountered, victim process: influx, pid: 3256628
  Warning  InvalidDiskCapacity      14m                kubelet     invalid capacity 0 on image filesystem
  Normal   NodeHasNoDiskPressure    14m (x2 over 14m)  kubelet     Node zigbee status is now: NodeHasNoDiskPressure
  Normal   NodeHasSufficientPID     14m (x2 over 14m)  kubelet     Node zigbee status is now: NodeHasSufficientPID
  Normal   NodeHasSufficientMemory  14m (x2 over 14m)  kubelet     Node zigbee status is now: NodeHasSufficientMemory

Hmmm - so running additional databases, processes outside of kubernetes is not such a good idea.

But as a fast solution: I ejected the SD card and did a resize + add swap on my laptop and put the SD card back to the raspberry pi...

Review: Kafka: The Definitive Guide

Last week i read the book "Kafka: The Definitive Guide" with the subtitle "Real-Time Data and Stream Processing at Scale" which was provided by confluent.io:

The book contains 11 chapters on 288 pages - let's take look on the content:

Chapter 1 "meet Kafka" start with a motivation, why moving data is important and why you should not spend your effort not into moving but into your business. In addition an introduction to the messaging concepts like publish/subscribe, queues, messages, batches, schemas, topics, partitions, ... Many technical terms are defined there, but some are specific to Kafka and some are more general definitions. One additional info: Kafka was built by linkedin - the complete story told in the last section of this chapter.

The second chapter is about installing Kafka. Nothing special. OS, Java, Zookeeper (if clustered), Kafka.

Chapter 3 is called "Kafka producers: Writing messages to Kafka". Like the title indicates: all configuration details about sending messages are listed and explained. 

Chapter 4 is the same as the previous chapter but for reading messages. Both chapters contain many java example listings.

Chapters 5 & 6 are about clusters and reliability. Here are the nifty details explained like high water marks, message replication, timeouts, indices, ... If you want to run a high available Kafka system, you should read that and in case of failures you will know what to do.

Chapter 7 introduces Kafka Connect. Here a citation, when you should use Connect (it is not possible to summarize this):

You will use Connect to connect Kafka to datastores that you did not write and whose code you cannot or will not modify. Connect will be used to pull data from the external datastore into Kafka or push data from Kafka to an external store. For datastores where a connector already exists, Connect can be used by nondevelopers, who will only need to configure the connectors.

"Cross data cluster mirroring" is the title of chapter 8 - i do not understand why this chapter is not placed before chapter 7...

In chapter 9 and 10 administration and monitoring is explained. Very impressive is the amount of CLI examples. If you have a question: here you will find the CLI command, which provides the answer.

The last chapter "stream processing" is one of the longest chapters (>40 pages).  Here two APIs are presented to do some processing based on the messages. One example is, a stream which processes stock quotes. With stream processing it is possible to calculate the number of trades for every five-second window or the average ask price for every five-second window. Of course this chapter shows much more, but i think this gives the best impression ;-)

All in all a excellent book - even if you are not implementing Kafka ;-)

 

Samsung A50: boot loop problem after last Samsung OS update

 I used a Samsung A50 for nearly 1,5 years and was very satisfied with the device. 128GB internal storage and dual sim - i do not need more :)

But last week the monthly "security" update was done by Samsung and after booting the new OS everything seems to fine. But only a few hours later (i did not install any new software - was just browsing in the web on my favourite news page) the smartphone froze and after that it keeps showing this screen for hours:

With pressing "Volume Up" and "Power" i was able to open the recovery mode, but after a factory reset, still the boot screen is shown...

Anyone else with this problem? Please leave a comment!