Run like Hell: Search results for alpine linux

Showing posts sorted by relevance for query alpine linux. Sort by date Show all posts

Dec 5, 2017

Docker-Swarm: One manager, two nodes with Alpine Linux

After creating a Alpine Linux VM inside virtualbox and after adding docker because of the small disk footprint (Alpine Linux: 170MB | with docker: 280MB) i performed the following steps to create a docker swarm:

cloning the vm twice
assigning a static ip to the manager node
create new MACs for the network interface cards on the nodes

Then i followed the tutorial https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ but without running the docker-machine commands, because i have 3 VMs and do not want to run the node on top of docker.

manager:

alpine:~# docker swarm init --advertise-addr 192.168.178.46
Swarm initialized: current node (wy1z8jxmr1cyupdqgkm6lxhe2) is now a manager.

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-3b7f69d3wgty0u68oab8724z07fkyvgc0w8j37ng1l7jsmbghl-0yfr1eu5u66z8pinweisltmci 192.168.178.46:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

nodes

# docker swarm join --token SWMTKN-1-3b7f69d3wgty0u68oab8724z07fkyvgc0w
8j37ng1l7jsmbghl-0yfr1eu5u66z8pinweisltmci 192.168.178.46:2377
This node joined a swarm as a worker.

And then a check on the master:

alpine:~# docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
wy1z8jxmr1cyupdqgkm6lxhe2 *   alpine              Ready               Active              Leader
pusf5o5buetjqrsmx3kzusbyt     node01              Ready               Active
io3z3b6nf8xbzkyzjq6sa7cuc     node02              Ready               Active

Run a first job:

alpine:~# docker service create --replicas 1 --name helloworld alpine ping 192.168.178.1
rsn6igby4f6d7uuy8eny7sbfb
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged

But on my manager i get no output for "docker ps". But this is, because the service is not running here:

alpine:~# docker service ps helloworld
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
wrrobalt4oe7 helloworld.1 alpine:latest node01 Running Running 2 minutes ago

Node 1 shows:

node01:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
40c5e9b2ffbc alpine:latest "ping 192.168.178.1" 3 minutes ago Up 3 minutes helloworld.1.wrrobalt4oe7mrbhxjlweuxgk

If i do a kill on the ping process, it is immediately restarted:

node01:~# ps aux|grep ping
2457 root       0:00 ping 192.168.178.1
2597 root       0:00 grep ping
node01:~# kill 2597
node01:~# ps aux|grep ping
2457 root       0:00 ping 192.168.178.1
2600 root       0:00 grep ping

A scale up is no problem:

alpine:~# docker service create --replicas 2 --name helloworld alpine ping 192.168.178.1
3lrdqdpjuqml6creswdcqpn2p
overall progress: 2 out of 2 tasks
1/2: running   [==================================================>]
2/2: running   [==================================================>]
verify: Service converged
alpine:~# docker service ps helloworld
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
616scw68s8bv        helloworld.1        alpine:latest       node02              Running             Running 8 seconds ago
n8ovvsw0m4id        helloworld.2        alpine:latest       node01              Running             Running 8 seconds ago

And a shutdown of node02 is no problem:

alpine:~# docker service ps helloworld
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
bne2enbkabfo        helloworld.1        alpine:latest       alpine              Ready               Ready 2 seconds ago
616scw68s8bv         \_ helloworld.1    alpine:latest       node02              Shutdown            Running 17 seconds ago
n8ovvsw0m4id        helloworld.2        alpine:latest       node01              Running             Running about a minute ago

After a switchoff of node01 both service are running on the remaining master:

alpine:~# docker service ps helloworld
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
bne2enbkabfo        helloworld.1        alpine:latest       alpine              Running             Running about a minute ago
616scw68s8bv         \_ helloworld.1    alpine:latest       node02              Shutdown            Running about a minute ago
pd8dfp4133yw        helloworld.2        alpine:latest       alpine              Running             Running 2 seconds ago
n8ovvsw0m4id         \_ helloworld.2    alpine:latest       node01              Shutdown            Running 2 minutes ago

So failover is working.
But failback does not occur. After switching on node01 again, the service remains on the manager:

alpine:~# docker service ps helloworld
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE               ERROR                         PORTS
bne2enbkabfo        helloworld.1        alpine:latest       alpine              Running             Running 4 minutes ago
616scw68s8bv         \_ helloworld.1    alpine:latest       node02              Shutdown            Running 4 minutes ago
pd8dfp4133yw        helloworld.2        alpine:latest       alpine              Running             Running 2 minutes ago
n8ovvsw0m4id         \_ helloworld.2    alpine:latest       node01              Shutdown            Failed about a minute ago   "task: non-zero exit (255)"
alpine:~# docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
wy1z8jxmr1cyupdqgkm6lxhe2 *   alpine              Ready               Active              Leader
pusf5o5buetjqrsmx3kzusbyt     node01              Ready               Active
io3z3b6nf8xbzkyzjq6sa7cuc     node02              Down                Active

Last thing: How to stop the service?

alpine:~# docker service rm helloworld
helloworld
alpine:~# docker service ps helloworld
no such service: helloworld

Remaining open points:

Is it possible to do a failback or limit the number of a service per node?
How to do this with a server application?
(load balancer needed?)
What happens, if the manager fails / is shutdown?

Alpine Linux, Oracle Java JDK and musl?! - why it does not work...

Sometime ago i did some work with Alpine Linux (s. here) and i was impressed how tiny this Linux distro was and how fast it was running.

So i decided after nearly 6 years of running an aircraft noise measuring station (for dfld.de) with Ubuntu to change to Alpine Linux.

This station runs a software on Java and needs RXTX, because the microphone is connected via USB and is read over /dev/ttyUSB0.

What is the problem with this setup?

RXTX needs a Java which is running on glibc
Alpine Linux does not run on glibc

If you are not aware of this problem, you get some errors like

./javaash: java: command not found

and this happens even if you are in the right directory and java got the execute bit configured.

Alpine Linux changed to musl:

There are some other libc implementations (take a look here).
The homepage is https://www.musl-libc.org/:

And a comparison to other libc can be found at http://www.etalabs.net/compare_libcs.html:

There are some workarounds to get applications build with glibc running on Alpine Linux, but i did not get to run my aircraft noise measuring station - i switched to Debian - because i needed a 32bit Linux for my very old UMPC...

May 18, 2018

Virtualbox: vboxdrv.sh: failed & Run 'make oldconfig && make prepare'

After updating to ubuntu 18.04 i had to reinstall virtualbox:

# dpkg -i virtualbox-5.2_5.2.10-122088~Debian~stretch_amd64.deb 
Vormals nicht ausgewähltes Paket virtualbox-5.2 wird gewählt.
(Lese Datenbank ... 378316 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von virtualbox-5.2_5.2.10-122088~Debian~stretch_amd64.deb ...
Entpacken von virtualbox-5.2 (5.2.10-122088~Debian~stretch) ...
virtualbox-5.2 (5.2.10-122088~Debian~stretch) wird eingerichtet ...
addgroup: Die Gruppe »vboxusers« existiert bereits als Systemgruppe. Programmende.
vboxdrv.sh: failed: Look at /var/log/vbox-install.log to find out what went wrong.

There were problems setting up VirtualBox.  To re-start the set-up process, run
  /sbin/vboxconfig
as root.
Trigger für systemd (237-3ubuntu10) werden verarbeitet ...
Trigger für ureadahead (0.100.0-20) werden verarbeitet ...
Trigger für gnome-menus (3.13.3-11ubuntu1) werden verarbeitet ...
Trigger für bamfdaemon (0.5.3+18.04.20180207.2-0ubuntu1) werden verarbeitet ...
Rebuilding /usr/share/applications/bamf-2.index...
Trigger für desktop-file-utils (0.23-1ubuntu3) werden verarbeitet ...
Trigger für mime-support (3.60ubuntu1) werden verarbeitet ...
Trigger für hicolor-icon-theme (0.17-2) werden verarbeitet ...
Trigger für shared-mime-info (1.9-2) werden verarbeitet ...

But this just failed with the following error (i am running my own kernel because of suspend to disk issues) inside /var/log/vbox-install.log:

test -e include/generated/autoconf.h -a -e include/config/auto.conf || (        \
echo >&2;                            \
echo >&2 "  ERROR: Kernel configuration is invalid.";        \
echo >&2 "         include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 "         Run 'make oldconfig && make prepare' on kernel src to fix it.";    \
echo >&2 ;                            \
/bin/false)
mkdir -p /lib/modules/4.17.0-rc3/misc

and at the end:

/tmp/vbox.0/linux/VBoxPci-linux.c:92:40: error: implicit declaration of function ‘pci_get_bus_and_slot’; did you mean ‘pci_get_domain_bus_and_slot’? [-Werror=implicit-function-declaration]
 # define PCI_DEV_GET_SLOT(bus, devfn)  pci_get_bus_and_slot(bus, devfn)
                                        ^
/tmp/vbox.0/linux/VBoxPci-linux.c:397:15: note: in expansion of macro ‘PCI_DEV_GET_SLOT’
     pPciDev = PCI_DEV_GET_SLOT(uBus, uDevFn);
               ^~~~~~~~~~~~~~~~
/tmp/vbox.0/linux/VBoxPci-linux.c:397:13: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
     pPciDev = PCI_DEV_GET_SLOT(uBus, uDevFn);
             ^
/tmp/vbox.0/linux/VBoxPci-linux.c: In function ‘vboxPciOsDevInit’:
/tmp/vbox.0/linux/VBoxPci-linux.c:648:17: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
         pPciDev = PCI_DEV_GET_SLOT((pIns->HostPciAddress) >> 8,
                 ^
cc1: some warnings being treated as errors
scripts/Makefile.build:312: recipe for target '/tmp/vbox.0/linux/VBoxPci-linux.o' failed
make[2]: *** [/tmp/vbox.0/linux/VBoxPci-linux.o] Error 1
Makefile:1568: recipe for target '_module_/tmp/vbox.0' failed
make[1]: *** [_module_/tmp/vbox.0] Error 2
/tmp/vbox.0/Makefile.include.footer:101: recipe for target 'vboxpci' failed
make: *** [vboxpci] Error 2

Starting virtualbox shows:

The virtual machine 'Docker on Alpine LInux' has terminated unexpectedly during startup with exit code 1 (0x1).

Fehlercode:NS_ERROR_FAILURE (0x80004005)
Komponente:MachineWrap
Interface:IMachine {85cd948e-a71f-4289-281e-0ca7ad48cd89}

There are many postings out there how to fix this.
But is easier than everything written there:

Change to /lib/modules/4.17.0-rc3/misc and type:

modprobe vboxdrv
modprobe vboxnetadp
modprobe vboxnetflt

And after that everything works like expected.

Nov 25, 2017

Docker-CE: Creating a small Linux as base OS for running a Docker cluster

After some first steps with docker i wanted to run more than one docker node. The problem is, that i have very limited resources on my laptop.

CoreOS (see here)
consumes about 600MB diskspace
UbuntuCore (see here)
consumes about 1.9GB diskspace

To setup CoreOS you have to learn ignition - otherwise every change is lost after a reboot. For UbuntuCore you have to use an UbuntuOne account to get it running.

I heard, that Alpine Linux is an OS with a very small footprint, so i downloaded the "virtual" image and

mounted it in my virtualbox:

After a short startup i got the following:

This is completely different to all other installation procedures i know. No configuration dialog just a

localhost login:

You have to enter "root" and without a password you get a banner with one important information:

You can setup the system with the command: setup-alpine

This will guide you through a 11 step setup (read the manual). To get a installation with persistent homes, etc. you have to choose the "sys" mode.

And then after a reboot, i have a Linux VM with 170MB.
For my next steps (installing docker, cloning some nodes) this should be the right choice.

[Installing docker is quite easy: remove the # before "edge/community" in /etc/apk/repositories and the run "apk update" and "apk add docker".]

Related posts:

May 22, 2018

Docker: Lessons learned - Logging

After some time working with docker here my experiences:

Some days ago i created my own container with a minimal web service.

Here the ncweb.sh:

#!/bin/ash
sed -i  's/Hostname:.*/Hostname: '$HOSTNAME'/g' index.html
while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html;}  | nc  -l -p 8080  2>&1 >> logfile; done

This is the Dockerfile:

FROM alpine

WORKDIR /tmp

RUN mkdir ncweb

ADD .  /tmp

ENTRYPOINT [ "/tmp/ncweb.sh" ]

After building the image

docker build -t ncweb:0.4 .

And starting the container:

docker run -d -p 8080:8080 ncweb:0.4 --name ncweb0.4

I was able to connect to the container and view the log:

To get the right command:

docker ps  |grep  ncweb:0.4 |awk '{print "docker exec -it "$1" ash"}'

and then use the output:

docker exec -it e4f9960fc8e5 ash

alpine:~/ncweb# docker exec -it e4f9960fc8e5 ash
/tmp # ls
Dockerfile  hexdump     index.html  logfile     ncweb       ncweb.sh
/tmp # cat logfile 
GET / HTTP/1.1
Host: 192.168.178.46:8080
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en,de;q=0.7,en-US;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0

Thu May 10 10:01:23 UTC 2018 request done

But this is not the right way.
If i change the ncweb.sh to

#!/bin/ash
sed -i  's/Hostname:.*/Hostname: '$HOSTNAME'/g' index.html
while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html;}  | nc  -l -p 8080 ;done

then you can do the following (after building a new container version):

alpine:~/ncweb# docker run -d -p 8080:8080 ncweb:0.5 --name ncweb0.5

9589f77fc289a3713354a365f8f08098279e6d0e893de99a0431d8fbd62c834a

alpine:~/ncweb# docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES

9589f77fc289        ncweb:0.5           "/tmp/ncweb.sh --n..."   8 seconds ago       Up 7 seconds        0.0.0.0:8080->8080/tcp   gracious_archimedes

To get the logs (which are written to STDOUT):

alpine:~/ncweb# docker logs -f 9589f77fc289

GET / HTTP/1.1

Host: 192.168.178.46:8080

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en,de;q=0.7,en-US;q=0.3

Accept-Encoding: gzip, deflate

Connection: keep-alive

Upgrade-Insecure-Requests: 1

Cache-Control: max-age=0

Conclusion: It is better to use STDOUT than local logfiles. Or even better: use syslog or other central logging mechanisms.

Related posts:

Dec 10, 2017

Docker-Swarm: Running a minimal webserver in a swarm

In my last posting to docker swarm i created a swarm on virtualbox with alpine linux with a hdd footprint of 290MB per node:

There are some tutorials out there with running a nginx or a java webserver in a container but >100MB for each node seems far to much for my tests.

So i decided to create a application which listens on port 8080 with netcat. I created a directory ncweb with ncweb.sh:

ncweb# cat ncweb.sh
#!/bin/bash
sed -i 's/Hostname:.*/Hostname: '$HOSTNAME'/g' index.html
while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html;} | nc -l -p 8080; done 2&>1 >> logfile

and a index.html:

ncweb# cat index.html
<html>
<head>
    <title>"Hello, World"</title>
</head>
<body bgcolor=white>
    <table border="0" cellpadding="10">
      <tr>
        <td>
          <h1>"Hello, World"</h1>
        </td>
      </tr>
    </table>
</body>
</html>

The Dockerfile looks like this:

# cat Dockerfile
FROM alpine
WORKDIR /tmp
ADD . /tmp
ENTRYPOINT [ "/tmp/ncweb.sh" ]

After that i created the container:

ncweb# docker build -t ncweb:0.2 .
Sending build context to Docker daemon 4.096kB
Step 1/5 : FROM alpine
---> 053cde6e8953
Step 2/5 : WORKDIR /tmp
---> Using cache
---> c3e11ac3773b
Step 3/5 : RUN mkdir ncweb
---> Using cache
---> d9e634c03cd1
Step 4/5 : ADD . /tmp
---> 95f022aacc1c
Step 5/5 : ENTRYPOINT [ "/tmp/ncweb.sh" ]
---> Running in c1a9e8cee248
---> 6880521f68e4
Removing intermediate container c1a9e8cee248
Successfully built 6880521f68e4
Successfully tagged ncweb:0.2

And let's do a test without docker swarm:

ncweb# docker run -p 8080:8080 ncweb

But running this as a service fails:

# docker service create --replicas=1 --name myweb ncweb:0.2
image ncweb:0.2 could not be accessed on a registry to record
its digest. Each node will access ncweb:0.2 independently,
possibly leading to different nodes running different
versions of the image.
n0himwum38bqzd8ob1vf8zhip
overall progress: 0 out of 1 tasks
1/1: No such image: ncweb:0.2
^COperation continuing in background.
Use `docker service ps n0himwum38bqzd8ob1vf8zhip` to check progress.

and:

# docker service ps n0himwum38bqzd8ob1vf8zhip
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                 ERROR                        PORTS
8tjsuae9jv8o        myweb.1             ncweb:0.2           node01              Ready               Rejected 3 seconds ago        "No such image: ncweb:0.2"
qp24ssxb5bl5         \_ myweb.1         ncweb:0.2           alpine              Shutdown            Failed 36 seconds ago         "task: non-zero exit (2)"
zwfgcatk7zyi         \_ myweb.1         ncweb:0.2           node01              Shutdown            Rejected about a minute ago   "No such image: ncweb:0.2"
v4a7zkb85yd4         \_ myweb.1         ncweb:0.2           node01              Shutdown            Rejected about a minute ago   "No such image: ncweb:0.2"
ycjftjusv484         \_ myweb.1         ncweb:0.2           node01              Shutdown            Rejected about a minute ago   "No such image: ncweb:0.2"

# docker service rm n0himwum38bqzd8ob1vf8zhip
n0himwum38bqzd8ob1vf8zhip

The error "No such image..." is happening, because the container ncweb is only in the repository of my master.
The easiest way for my test environment is to distribute the local image to all nodes:

# docker save ncweb:0.3 | ssh 192.168.178.47 docker load
The authenticity of host '192.168.178.47 (192.168.178.47)' can't be established.
ECDSA key fingerprint is SHA256:2/8O/SE1fGJ4f5bAQls5txrKMbqZfMmiZ+Tha/WFKxA.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.178.47' (ECDSA) to the list of known hosts.
root@192.168.178.47's password:
Loaded image: ncweb:0.3

(i have to distribute a ssh-key to all nodes)

and then:

alpine:~/ncweb# docker service create --replicas=1 --name myweb ncweb:0.3
image ncweb:0.3 could not be accessed on a registry to record
its digest. Each node will access ncweb:0.3 independently,
possibly leading to different nodes running different
versions of the image.

# docker service ps myweb
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
in97xlc7azcw myweb.1 ncweb:0.3 node01 Running Running 8 seconds ago

So my nc-webserver runs on node01, but i can not access it there because i did not define any port mappings ;-(

But finally this command did the job:

# docker service create --replicas=1 --name myweb --publish 8080:8080 ncweb:0.3
image ncweb:0.3 could not be accessed on a registry to record
its digest. Each node will access ncweb:0.3 independently,
possibly leading to different nodes running different
versions of the image.
runf8u9r8719sk13mkf8hh8ec
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged

The hostname corresponds to the docker container id on node01:

node01:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c9434b08082 ncweb:0.3 "/tmp/ncweb.sh" 37 minutes ago Up 37 minutes myweb.1.lqiyb34cuxxme2141ahsg8neu

Remaining open points:

Is it possible to do a failback or limit the number of a service per node?
How to get a loadbalancing mechanism for a server application?
(load balancer needed?)
What happens, if the manager fails / is shutdown?

Docker-CE on Ubuntu 17.10 (Artful Aardvark) (2)

Three months ago i installed docker on my ubuntu 17.10. At those days there was no straight forward howto on docher.com.

Now the installation is listed on docker.com:

The installation manual can be found here.

root@zerberus:~# apt-get install apt-transport-https ca-certificates curl   software-properties-common
root@zerberus:~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - OK
root@zerberus:~# apt-key fingerprint 0EBFCD88 pub   rsa4096 2017-02-22 [SCEA]
      9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid        [ unbekannt ] Docker Release (CE deb)
sub   rsa4096 2017-02-22 [S]
root@zerberus:~# add-apt-repository \
>    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
>    $(lsb_release -cs) \
>    stable"
root@zerberus:~# apt update
root@zerberus:~# apt install docker-ce

And then a check:

root@zerberus:~# docker versionClient:
Version:    17.12.0-ce
API version:    1.35
Go version:    go1.9.2
Git commit:    c97c6d6
Built:    Wed Dec 27 20:11:14 2017
OS/Arch:    linux/amd64

Server:
Engine:
Version:    17.12.0-ce
API version:    1.35 (minimum version 1.12)
Go version:    go1.9.2
Git commit:    c97c6d6
Built:    Wed Dec 27 20:09:47 2017
OS/Arch:    linux/amd64
Experimental:    false

Docker-CE: initial configuration

After installing docker to a ubuntu server i was keen what kind of processes and files are there.

# ps x|grep docker
5852 ? Ssl 0:05 /usr/bin/dockerd -H fd://
5867 ? Ssl 0:04 docker-containerd --config /var/run/docker/containerd/containerd.toml

So the inital configuration is not stored in /etc.

The configuration file looks like this:

# cat /var/run/docker/containerd/containerd.toml
root = "/var/lib/docker/containerd/daemon"
state = "/var/run/docker/containerd/daemon"
subreaper = false
oom_score = -500

[grpc]
address = "/var/run/docker/containerd/docker-containerd.sock"
uid = 0
gid = 0

[debug]
address = "/var/run/docker/containerd/docker-containerd-debug.sock"
uid = 0
gid = 0
level = "info"

[metrics]
address = ""

[cgroup]
path = ""

[plugins]
[plugins.linux]
    shim = "docker-containerd-shim"
    runtime = "docker-runc"
    runtime_root = "/var/lib/docker/runc"
    no_shim = false
    shim_debug = false
    shim_no_newns = false

The directory /var/run/docker/containerd/daemon seems to be important, so let's take a look:

# find /var/run/docker/containerd/daemon
/var/run/docker/containerd/daemon
/var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux
/var/run/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby

But theese entries are only directories...

So what about /var/lib? Here we go:

# find /var/lib/docker/ -maxdepth 1
/var/lib/docker/
/var/lib/docker/overlay2
/var/lib/docker/swarm
/var/lib/docker/image
/var/lib/docker/tmp
/var/lib/docker/runtimes
/var/lib/docker/trust
/var/lib/docker/plugins
/var/lib/docker/containerd
/var/lib/docker/containers
/var/lib/docker/builder
/var/lib/docker/volumes
/var/lib/docker/network

Within the directory containers you can find information about the containers which run / ran on your docker system:

# ls -l
insgesamt 12
drwx------ 4 root root 4096 Nov 16 22:12 0c7567bb965449f5f2f3dfadfc38f0226bdb42cc6d46daa4641042090e0405c7
drwx------ 4 root root 4096 Nov 16 22:57 28b7f70147aabc94bd174ce7ad7f5d48ed5610ffaa21733d3549ee6e998ee7c3
drwx------ 4 root root 4096 Nov 16 21:59 2db1c3ed1dafc85369536c382e02b6a23e5d134d2ba3d56a738f3441fb624b04

The first characters of the directories match the container-id:

# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                  PORTS               NAMES
28b7f70147aa        hello-world         "/hello"            2 days ago          Exited (0) 2 days ago                       trusting_bartik
0c7567bb9654        hello-world         "/hello"            2 days ago          Exited (0) 2 days ago                       festive_wozniak
2db1c3ed1daf        hello-world         "/hello"            2 days ago          Exited (0) 2 days ago                       epic_tesla

Docker-Swarm: Running with more than one manager-node / How to add a secondary manager or multiple managers

Adding additional masters to a docker swarm is very easy. I just followed the documentation:

On the manager i ran the following command:

alpine:~# docker swarm join-token manager
To add a manager to this swarm, run the following command:

docker swarm join --token SWMTKN-1-3b7f69d3wgty0u68oab8724z07fkyvgc0w8j37ng1l7jsmbghl-5nrni6ksqnkljvqpp59m5gfh1 192.168.178.46:2377

And on the node:

node03:~# docker swarm join --token SWMTKN-1-3b7f69d3wgty0u68oab8724z07fkyvgc0w8j37ng1
l7jsmbghl-5nrni6ksqnkljvqpp59m5gfh1 192.168.178.46:2377

This node joined a swarm as a manager.

The new state of the cluster shows:

alpine:~# docker info | grep -A 23 Swarm
Swarm: active
NodeID: wy1z8jxmr1cyupdqgkm6lxhe2
Is Manager: true
ClusterID: wkbjyxbcuohgdqc3amhl9umlq
Managers: 2
Nodes: 4
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.178.46
Manager Addresses:
192.168.178.46:2377
192.168.178.50:2377

If you want to use a new token just run:

alpine:~# docker swarm join-token --rotate manager
Successfully rotated manager join token.

To add a manager to this swarm, run the following command:

docker swarm join --token SWMTKN-1-3b7f69d3wgty0u68oab8724z07fkyvgc0w8j37ng1l7jsmbghl-cgy143mwghbfoozt0b8li2587 192.168.178.46:2377

Docker-CE: Get Started Tutorial - processes / files / HTTP headers

Inside my docker installation on ubuntu server i performed the official docker tutorial part 2: "containers"

Ok. Not really interesting, because the tutorial describes all steps very well (except the point, that you have to install python3-pip and you have to run pip3 and not pip).

Here the interesting part:

After i started the container ("docker run -p 4000:80 friendlyhello") i got the following:

root@ubuntuserver:/var/lib/docker/image# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6f187950452f friendlyhello "python app.py" 2 hours ago Up 2 hours 0.0.0.0:4000->80/tcp keen_bartik

and the OS displayed

root@ubuntuserver:/var/lib/docker/image# ps x |grep docker
5852 ?        Ssl    0:18 /usr/bin/dockerd -H fd://
5867 ?        Ssl    0:10 docker-containerd --config /var/run/docker/containerd/containerd.toml
17100 ?        Sl     0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4000 -container-ip 172.17.0.2 -container-port 80
17105 ?        Sl     0:00 docker-containerd-shim --namespace moby --workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/6f187950452f501cf18d7efd673e8305d6c9752c58acfb14918ae37341952a11 --address /var/run/docker/containerd/docker-containerd.sock --runtime-root /var/run/docker/runtime-runc

So the containerid is part of the directory which is loaded with "--workdir"

The HTTP headers showup the following:

So the process "docker-proxy-proto" does only a portforwarding from port 4000 to port 80 and the webserver is provided by python:

Content-Type: text/html; charset=utf-8
Content-Length: 118
Server: Werkzeug/0.12.2 Python/2.7.14
Date: Sun, 19 Nov 2017 12:20:42 GMT

Here the process tree:

# pstree -al
systemd
├─dockerd -H fd://
│   ├─docker-containe --config /var/run/docker/containerd/containerd.toml
│   │   ├─docker-containe --namespace moby --workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/6f187950452f501cf18d7efd673e8305d6c9752c58acfb14918ae37341952a11 --address /var/run/docker/containerd/docker-containerd.sock --runtime-root /var/run/docker/runtime-runc
│   │   │   ├─python app.py
│   │   │   └─9*[{docker-containe}]
│   │   └─7*[{docker-containe}]
│   ├─docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4000 -container-ip 172.17.0.2 -container-port 80
│   │   └─4*[{docker-proxy}]
│   └─9*[{dockerd}]

Note that this container has the ip 172.17.0.2. This is part of a private network and the ubuntu server has the ip
# ip address show docker0

3: docker0: mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:f4:04:06:3d brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever

If you start the container twice

root@ubuntuserver:~/dockerapp# docker run -d -p 4000:80 friendlyhello
aebe1a25dd2dda1c36c5cdd46bbdc138099711c885a7fb270b9981301a66fbd9
root@ubuntuserver:~/dockerapp# docker run -d -p 4001:80 friendlyhello
59276d1e15d7a636dbcf4eb599f7a948365eed656d93d47097999bee9b25db6b

there are two docker-proxies running:

# ps aux|grep docker-proxy
root 1286 0.0 0.3 304928 3084 ? Sl 16:16 0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4000 -container-ip 172.17.0.2 -container-port 80
root 1385 0.0 0.2 304928 2880 ? Sl 16:16 0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4001 -container-ip 172.17.0.3 -container-port 80

Java 10 released: java with some enhancements for running inside docker

After the release of Java 9 in october 2017 with its new features

Oracle released Java 10:

A short summary of the new feature can be found
at https://blogs.oracle.com/java-platform-group/introducing-java-se-10
or you can take a look a the release notes:
http://www.oracle.com/technetwork/java/javase/10-relnote-issues-4108729.html#NewFeature

My favourites are:

JEP 307 Parallel Full GC for G1 Improves G1 worst-case latencies by making the full GC parallel. The G1 garbage collector is designed to avoid full collections, but when the concurrent collections can't reclaim memory fast enough a fall back full GC will occur. The old implementation of the full GC for G1 used a single threaded mark-sweep-compact algorithm. With JEP 307 the full GC has been parallelized and now use the same amount of parallel worker threads as the young and mixed collections.

See JDK-8172890

and the docker enhancements:

JDK-8146115 Improve docker container detection and resource configuration usage

The JVM has been modified to be aware that it is running in a Docker container and will extract container specific configuration information instead of querying the operating system. The information being extracted is the number of CPUs and total memory that have been allocated to the container. The total number of CPUs available to the Java process is calculated from any specified cpu sets, cpu shares or cpu quotas. This support is only available on Linux-based platforms. This new support is enabled by default and can be disabled in the command line with the JVM option:

-XX:-UseContainerSupport

In addition, this change adds a JVM option that provides the ability to specify the number of CPUs that the JVM will use:

-XX:ActiveProcessorCount=count

This count overrides any other automatic CPU detection logic in the JVM.

JDK-8186248 Allow more flexibility in selecting Heap % of available RAM

Three new JVM options have been added to allow Docker container users to gain more fine grained control over the amount of system memory that will be used for the Java Heap:

-XX:InitialRAMPercentage
-XX:MaxRAMPercentage
-XX:MinRAMPercentage

These options replace the deprecated Fraction forms (-XX:InitialRAMFraction, -XX:MaxRAMFraction, and -XX:MinRAMFraction).

JDK-8179498 attach in Linux should be relative to /proc/pid/root and namespace aware

This bug fix corrects the attach mechanism when trying to attach from a host process to a Java process that is running in a Docker container.

Docker: How to build you own container with your own application

atThere are many tutorials out there, how to create a docker container with a apache webserver inside or a nginx.
But you can hardly find a manual how to build your own docker container without pulling everything from a foreign repository.
Why should you not pull everything from foreign repositories?

You should read this article or this:

But since each phase of the development pipeline is built at a different time, …

…you can’t be sure that the same version of each dependency in the development version also got into your production version.

That is a good point.

As considered in this article you can put some more constraints into your docker file:

FROM ubuntu:14.04.

or even

FROM ubuntu:0bf3461984f2fb18d237995e81faa657aff260a52a795367e6725f0617f7a56c

And that is the point where i tell you: Create a process to build your own docker containers from scratch and distribute them with your own repository or copy them to all your docker nodes (s. here)

So here the steps to create your own container from a local directory (here ncweb):

# ls -l ncweb/
total 12
-rw-r--r--    1 root     root            90 Nov 26 10:06 Dockerfile
-rw-r--r--    1 root     root           255 Nov 26 11:29 index.html
-rw-r--r--    1 root     root             0 Nov 26 11:29 logfile
-rwxr--r--    1 root     root           176 Nov 26 11:29 ncweb.sh

The Dockerfile contains the following:

# ls -l ncweb/
alpine:~# cat ncweb/Dockerfile 
FROM alpine
WORKDIR /tmp
RUN mkdir ncweb
ADD .  /tmp
ENTRYPOINT [ "/tmp/ncweb.sh" ]

Into this directory you have to put everything you need, e.g. a complete JDK or your binaries or ...

And then change into this directory and build your container:

ncweb# docker build -t ncweb:0.2 .

The distribution to other docker nodes can be done like this:

# docker save ncweb:0.3 | ssh 192.168.178.47 docker load

For more details read this posting.

Related posts:

Feb 11, 2018

Docker-Machine: how to create a docker vm on a remote virtualbox server

After doing some first steps with docker, i wanted to test docker-swarm. Because of the limited resources of my notebook, i was looking for a Linux with a minimal footprint. In the context of setting up VMs for docker-swarm i found a log of articles about doing that with the tool docker-machine.
It sounds like this tool can create VMs just with one command. (here the documentation).

So let's give it a try:
(You have to install docker-machine first, but you do not need to install docker itself)

~$ docker-machine create --driver virtualbox test
Creating CA: /home/schroff/.docker/machine/certs/ca.pem
Creating client certificate: /home/schroff/.docker/machine/certs/cert.pem
Running pre-create checks...
(test) Image cache directory does not exist, creating it at /home/schroff/.docker/machine/cache...
(test) No default Boot2Docker ISO found locally, downloading the latest release...
(test) Latest release for github.com/boot2docker/boot2docker is v17.11.0-ce
(test) Downloading /home/schroff/.docker/machine/cache/boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v17.11.0-ce/boot2docker.iso...
(test) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%
Creating machine...
(test) Copying /home/schroff/.docker/machine/cache/boot2docker.iso to /home/schroff/.docker/machine/machines/test/boot2docker.iso...
(test) Creating VirtualBox VM...
(test) Creating SSH key...
(test) Starting the VM...
(test) Check network to re-create if needed...
(test) Found a new host-only adapter: "vboxnet0"
(test) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env test

Wow.
After this command inside my virtualbox a new machine shows up with 1GB RAM, 20 GB HDD (dynamic allocated), 2 network adapters (1x NAT, 1x host only).

But it is not possible to create VMs on a remote Virtualbox server. The CLI does not allow to give a remote server IP:

But for some other environments it is possible to deploy VMs on a remote site:

--vmwarevsphere-vcenter: IP/hostname for vCenter (or ESXi if connecting directly to a single host)

If your preferred virtualization engine supports remote servers, you can check here:

Nevertheless docker-machine is an excellent tool. If you are interested in creating a swarm, read this tutorial.
The homepage of the OS boot2docker can be found here.

Related posts:

Run like Hell