Dec 31, 2020

Samsung A50: boot loop problem after last Samsung OS update

 I used a Samsung A50 for nearly 1,5 years and was very satisfied with the device. 128GB internal storage and dual sim - i do not need more :)

But last week the monthly "security" update was done by Samsung and after booting the new OS everything seems to fine. But only a few hours later (i did not install any new software - was just browsing in the web on my favourite news page) the smartphone froze and after that it keeps showing this screen for hours:

With pressing "Volume Up" and "Power" i was able to open the recovery mode, but after a factory reset, still the boot screen is shown...

Anyone else with this problem? Please leave a comment!


Dec 30, 2020

My son started at blogspot.com

My son started its own blog 

https://holzgeschenkebasteln.blogspot.com/


Of course this blog is in german, but it is nice to see, that he managed to get everything running and configured.

I am curious, if he will write some more postings...

Dec 23, 2020

MicroK8s: more problems - log flooding

After getting my kubernetes nodes running on ubuntu's microK8s

i got thousands of these messages in my syslog:

Dec 22 21:15:00 ubuntu microk8s.daemon-kubelet[10978]: W1122 21:15:00.735176   10978 clientconn.go:1223] grpc: addrConn.createTransport fail
ed to connect to {unix:///var/snap/microk8s/common/run/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error whil
e dialing dial unix:///var/snap/microk8s/common/run/containerd.sock: timeout". Reconnecting...

Dec 22 21:15:00 ubuntu microk8s.daemon-kubelet[10978]: W1122 21:15:00.737524   10978 clientconn.go:1223] grpc: addrConn.createTransport fail
ed to connect to {unix:///var/snap/microk8s/common/run/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error whil
e dialing dial unix:///var/snap/microk8s/common/run/containerd.sock: timeout". Reconnecting...

Really annoying i found no solution for this problem. But there is an easy way to correct this problem:

snap disable microk8s
snap enable microk8s
Run this on both nodes and the problem is gone (i think rebooting will do the same job).



Dec 18, 2020

Review: AIOPS for dummies - the newest buzzword in town...

 Today i ran over an article in linkedin, where this book was announced:


The next big thing behind DevOps is AIOps?

Moogsoft says about themselves: "Moogsoft is a pioneer and leading provider of AIOps solutions that help IT teams work faster and smarter. With patented AI analyzing billions of events daily across the world's most complex IT environments, the Moogsoft AIOps platform helps the world's top enterprises avoid outages, automate service assurance, and accelerate digital transformation initiatives...."

So let's take a look inside this book with 43 pages and 7 chapters:

Chapter one start with the declaration of the problem: DevOps & reliability need improvements in incident resolution, meeting SLAs and accelerating digital transformation. Very nice is the short case study, which is provided there.

The beginning of chapter 2 starts with this setence: "AI is technology used to create machines that imitate intelligent human behaviour." YES! They are not talking over the almighty AI - this sounds very promising. AI is for moogsoft statistics, probabilites, calculations and algebra - as physicist i strongly agree with that "legacy" approach. Then this book covers very brief the ai learning techniques.

In chapter 3 the AIOps workflow is presented. Without going into any details here: Moogsoft uses a very nice iconic design, which explains their procedure well. At this point i would recommend you, to take a look on that...

Chapter 4 provides some more use cases for AIOps. Nice - but nothing really new.

Chapter 5 claims, that AIOps is providing a unified view for monitoring, observability and change data. Sounds good - but i think digging into details will show limits of the promise. But page 32 shows a list of systems which are already integrated - this is really a very impressive list.

In chapter 6 moogsoft advertise their small entry solution "moogsoft express".  

The last chapter closes with the typical "ten tips".

 

All in all a nice idea and let's see how this solution performs on the market!

Dec 16, 2020

zigbee: moving data from mqtt to influxdb - transforming strings to integers

After some first steps with zigbee devices and storing the data in an influxdb, i noticed that string values are suboptimal for building graphs. 

Moving the data from mqtt to influxdb was done with telegraf:

https://www.influxdata.com/time-series-platform/telegraf/

And i was wondering, how i can change string to integers, but this i very easy:

  [[processors.enum]]
    order = 2
    [[processors.enum.mapping]]
      field = "state"
      [processors.enum.mapping.value_mappings]
        "ON" = 1
        "OFF" = 0
    [[processors.enum.mapping]]
      field = "contact"
      [processors.enum.mapping.value_mappings]
        "true" = 2
        "false" = 1
    [[processors.enum.mapping]]
      field = "tamper"
      [processors.enum.mapping.value_mappings]
        "true" = 1
        "false" = 0
    [[processors.enum.mapping]]
      field = "water_leak"
      [processors.enum.mapping.value_mappings]
        "true" = 1
        "false" = 0
Next problem: if the column "water_leak" was already added inside your influxdb, you can not add numbers - so you have to drop the table and loose your data...

(This is not the full truth: you can export the data via a select to a file and insert the data afterwards - with the appropriate numbers...)
 


Dec 12, 2020

My start to a local kubernetes cluster: microK8s @ubuntu

After playing around with zigbee on raspberry pi, i decided to build up my own kubernetes cluster at home. I have to raspberry pi running ubuntu server, so i wanted to go this direction:


The start is very easy. Just follow the steps shown here:

https://microk8s.io/docs

But by adding the second node i got the following result:

root@zigbee:/home/ubuntu/kubernetes# microk8s kubectl get nodes
NAME     STATUS     ROLES    AGE   VERSION
ubuntu   NotReady   <none>   98s   v1.19.3-34+b9e8e732a07cb6
zigbee   NotReady   <none>   37m   v1.19.3-34+b9e8e732a07cb6
Hmmm.

The best way to debug this problem is

# microk8s inspect
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Service snap.microk8s.daemon-control-plane-kicker is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
Inspecting juju
  Inspect Juju
Inspecting kubeflow
  Inspect Kubeflow

# Warning: iptables-legacy tables present, use iptables-legacy to see them
WARNING:  Docker is installed.
File "/etc/docker/daemon.json" does not exist.
You should create it and add the following lines:
{
    "insecure-registries" : ["localhost:32000"]
}
and then restart docker with: sudo systemctl restart docker
WARNING:  The memory cgroup is not enabled.
The cluster may not be functioning properly. Please ensure cgroups are enabled
See for example: https://microk8s.io/docs/install-alternatives#heading--arm
Building the report tarball
  Report tarball is at /var/snap/microk8s/1794/inspection-report-20201212_194335.tar.gz
And as you can see: this contains the solution!

After adding the /etc/docker/daemon.json everything went fine:

root@zigbee:~# kubectl get nodes 
NAME     STATUS   ROLES    AGE    VERSION
ubuntu   Ready    <none>   46h    v1.19.3-34+b9e8e732a07cb6
zigbee   Ready    <none>   2d3h   v1.19.3-34+b9e8e732a07cb6

Dec 11, 2020

MicroK8s: Dashboard & RBAC

If you want to access your dashboard and you have enabled RBAC (like shown here), you will get this error, if you follow the default manual (https://microk8s.io/docs/addon-dashboard):

secrets is forbidden: User "system:serviceaccount:default:default" cannot list resource "secrets" in API group "" in the namespace "default"
error
persistentvolumeclaims is forbidden: User "system:serviceaccount:default:default" cannot list resource "persistentvolumeclaims" in API group "" in the namespace "default"
error
configmaps is forbidden: User "system:serviceaccount:default:default" cannot list resource "configmaps" in API group "" in the namespace "default"
error
services is forbidden: User "system:serviceaccount:default:default" cannot list resource "services" in API group "" in the namespace "default"
error
statefulsets.apps is forbidden: User "system:serviceaccount:default:default" cannot list resource "statefulsets" in API group "apps" in the namespace "default"
error
ingresses.extensions is forbidden: User "system:serviceaccount:default:default" cannot list resource "ingresses" in API group "extensions" in the namespace "default"
error
replicationcontrollers is forbidden: User "system:serviceaccount:default:default" cannot list resource "replicationcontrollers" in API group "" in the namespace "default"
error
jobs.batch is forbidden: User "system:serviceaccount:default:default" cannot list resource "jobs" in API group "batch" in the namespace "default"
error
replicasets.apps is forbidden: User "system:serviceaccount:default:default" cannot list resource "replicasets" in API group "apps" in the namespace "default"
error
deployments.apps is forbidden: User "system:serviceaccount:default:default" cannot list resource "deployments" in API group "apps" in the namespace "default"
error
events is forbidden: User "system:serviceaccount:default:default" cannot list resource "events" in API group "" in the namespace "default"
error
pods is forbidden: User "system:serviceaccount:default:default" cannot list resource "pods" in API group "" in the namespace "default"
error
daemonsets.apps is forbidden: User "system:serviceaccount:default:default" cannot list resource "daemonsets" in API group "apps" in the namespace "default"
error
cronjobs.batch is forbidden: User "system:serviceaccount:default:default" cannot list resource "cronjobs" in API group "batch" in the namespace "default"
error
namespaces is forbidden: User "system:serviceaccount:default:default" cannot list resource "namespaces" in API group "" at the cluster scope
 
To get the right bearer token you have to this:

export K8S_USER="system:serviceaccount:default:default"
export NAMESPACE="default"
export BINDING="defaultbinding"
export ROLE="defaultrole"
kubectl create clusterrole $ROLE  --verb="*"  --resource="*.*"    
kubectl create rolebinding $BINDING --clusterrole=$ROLE --user=$K8S_USER -n $NAMESPACE
kubectl -n ${NAMESPACE} describe secret $(kubectl -n ${NAMESPACE} get secret | (echo "$_") | awk '{print $1}') | grep token: | awk '{print $2}'\n

(create role, add a role binding and then get the token)

But there is still one error:

To fix this, you have add the cluster-admin role to this account (if you really want clusterwide permissions):

kubectl create clusterrolebinding root-cluster-admin-binding --clusterrole=cluster-admin --user=$K8S_USER

Dec 5, 2020

Securing InfluxDB

In my monitoring setup i am heavily using InfluxDB. Starting with one linux server with grafana which loads the data from its local influxdb, i wanted to setup a second linux server.

My options:

  1. new telegraf, new influxdb, new grafana
    but then i have two url (because of two grafanas and i can not copy graphs from one dashboard to the other)
  2. new telegraf, new influxdb, but grafana from first server
    grafana has to get the data over the network
  3. new telegraf, influxdb & grafana from first server
    what is happening if telegraf can not reach influxdb, because of network problem? what if the first server is down?
  4. completely remote monitoring
    what is happening if telegraf can not reach the other server? what if the first server is down? 

As you can see, option 2 is the favorite here.

But therefore InfluxDB has to be secured: SSL + user/password.

So let's start with creating some certificates:

openssl req -new -x509 -nodes -out server-cert.pem -days 3650 -keyout server-key.pem

So that you get:

zigbee:/etc/influxdb# ls -lrt *pem
-rw-r--r-- 1 influxdb root  1704 Nov  7 09:48 key.pem
-rw-r--r-- 1 influxdb root  1411 Nov  7 09:48 cert.pem

Then add this in /etc/influxdb/influxdb.conf

 https-enabled = true
 https-certificate = "/etc/influxdb/cert.pem"
 https-private-key = "/etc/influxdb/key.pem"

But still a user is missing, so we have to create users (via bash):

influx -ssl -unsafeSsl

create user admin with password 'XXXXXXX' with all privileges

After that you can test this with

root@zigbee:# influx -ssl -unsafeSsl  
Connected to https://localhost:8086 version 1.6.4
InfluxDB shell version: 1.6.4
> show databases
ERR: unable to parse authentication credentials
Warning: It is possible this error is due to not setting a database.
Please set a database with the command "use <database>".
> auth
username: admin
password:
> show databases
name: databases
name
----
_internal

 


 

Dec 4, 2020

AVM Fritz.Box: how to do an automatic login and get the active WLAN devices

The AVM Fritz.Box is really a great device - but the possibilities to get monitoring data are very limited. (Please read this posting)

Which data do i want?


I want the data, which is presented in the networking tab:

If i trace the networking with the developer tools, i the the following:

To reproduce this on my command line, i have to enter this into my bash:

curl 'http://fritz.box/data.lua' 
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0'
-H 'Accept: */*'
-H 'Accept-Language: de,en;q=0.7,en-US;q=0.3' --compressed
-H 'Content-Type: application/x-www-form-urlencoded'
-H 'Origin: http://fritz.box' -H 'Connection: keep-alive'
-H 'Referer: http://fritz.box/' -H 'Pragma: no-cache'
-H 'Cache-Control: no-cache'
--data-raw 'xhr=1&sid=cb......SID&lang=de&page=netDev&xhrId=cleanup&useajax=1&no_sidrenew='

(you have to add the line breaks and the SID in the last line).

Then you will get a JSON object beginning with these lines:

{
  "pid": "netDev",
  "hide": {
    "ssoEmail": true,
    "shareUsb": true,
    "liveTv": true,
    "faxSet": true,
    "dectMoniEx": true,
    "rss": true,
    "mobile": true,
and all the other information.

The problem: How to get this SID?

If you trace the login, it is not so easy, that the password is just send to the Fritz.Box. They use PBDFK2 to encrypt the password and then send it to the Fritz.Box.

You can find some information about that here:

https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AVM%20Technical%20Note%20-%20Session%20ID_EN%20-%20Nov2020.pdf


Inside this document a PHP program is stated, which does the login (not really - i think it does the job years ago - but now it does a fallback to md5 authentication. I fixed this, just post a comment, if you want this pbkdf2 enabled php script). I wrote a small javascript, which i execute with node and after that i was able to log the data into my influxdb and build a show it inside grafana:


If you are interested in the configuration, the js script and the collect commands, then post me a comment...

Dec 2, 2020

Kubernetes: Rights & Roles with kubectl and RBAC - How to restrict kubectl for a user to a namespace

Playing around with my MicroK8S i was thinking about restricting access to the default namespace. Why?

Every command adds something and so your default namespace gets polluted more and more and cleaning up might be a lot of work.

But:

There is neither a HOWTO nor some quickstart into this. Everything you can find is:

https://kubernetes.io/docs/reference/access-authn-authz/rbac/

But after this very detailed article you know a lot of things, but for restricting the kubectl you are as smart as before.

One thing i learned in this article:

You do not have to use these YAML files - everything can be done with commands and their options (i do not like YAML, so this was a very important understanding for me).

At the end it is very easy:

export K8S_USER="ateamuser"
export NAMESPACE="ateam"
export BINDING="ateambinding"
export ROLE="ateamrole"
kubectl create namespace $NAMESPACE
kubectl label namespaces $NAMESPACE team=a
kubectl create clusterrole ateamrole  --verb="*"  --resource="*.*"
kubectl create rolebinding $BINDING --clusterrole=$ROLE --user=$K8S_USER -n $NAMESPACE
kubectl create serviceaccount $K8S_USER -n $NAMESPACE
kubectl describe sa $K8S_USER -n $NAMESPACE
and just test it with:

root@zigbee:/home/ubuntu/kubernetes# kubectl get pods -n ateam  --as=ateamuser
NAME                  READY   STATUS    RESTARTS   AGE
web-96d5df5c8-cc9jv   1/1     Running   0          14m
root@zigbee:/home/ubuntu/kubernetes# kubectl get pods -n default  --as=ateamuser
Error from server (Forbidden): pods is forbidden: User "ateamuser" cannot list resource "pods" in API group "" in the namespace "default"
So there is not a big script needed - but building these commands was really a hard job...

If you want to know, how to restrict the kubectl on a remote computer, please write a comment. 

One last remark: In microK8s you enable RBAC with the command

microk8s.enable rbac

Check this with

microk8s.status
microk8s is running
high-availability: no
  datastore master nodes: 192.168.178.57:19001
  datastore standby nodes: none
addons:
  enabled:
    dashboard            # The Kubernetes dashboard
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    ingress              # Ingress controller for external access
    metrics-server       # K8s Metrics Server for API access to service metrics
    rbac                 # Role-Based Access Control for authorisation
  disabled:
    helm                 # Helm 2 - the package manager for Kubernetes
    helm3                # Helm 3 - Kubernetes package manager
    host-access          # Allow Pods connecting to Host services smoothly
    linkerd              # Linkerd is a service mesh for Kubernetes and other frameworks
    metallb              # Loadbalancer for your Kubernetes cluster
    registry             # Private image registry exposed on localhost:32000
    storage              # Storage class; allocates storage from host directory