Review: Kafka: The Definitive Guide

Last week i read the book "Kafka: The Definitive Guide" with the subtitle "Real-Time Data and Stream Processing at Scale" which was provided by confluent.io:

The book contains 11 chapters on 288 pages - let's take look on the content:

Chapter 1 "meet Kafka" start with a motivation, why moving data is important and why you should not spend your effort not into moving but into your business. In addition an introduction to the messaging concepts like publish/subscribe, queues, messages, batches, schemas, topics, partitions, ... Many technical terms are defined there, but some are specific to Kafka and some are more general definitions. One additional info: Kafka was built by linkedin - the complete story told in the last section of this chapter.

The second chapter is about installing Kafka. Nothing special. OS, Java, Zookeeper (if clustered), Kafka.

Chapter 3 is called "Kafka producers: Writing messages to Kafka". Like the title indicates: all configuration details about sending messages are listed and explained. 

Chapter 4 is the same as the previous chapter but for reading messages. Both chapters contain many java example listings.

Chapters 5 & 6 are about clusters and reliability. Here are the nifty details explained like high water marks, message replication, timeouts, indices, ... If you want to run a high available Kafka system, you should read that and in case of failures you will know what to do.

Chapter 7 introduces Kafka Connect. Here a citation, when you should use Connect (it is not possible to summarize this):

You will use Connect to connect Kafka to datastores that you did not write and whose code you cannot or will not modify. Connect will be used to pull data from the external datastore into Kafka or push data from Kafka to an external store. For datastores where a connector already exists, Connect can be used by nondevelopers, who will only need to configure the connectors.

"Cross data cluster mirroring" is the title of chapter 8 - i do not understand why this chapter is not placed before chapter 7...

In chapter 9 and 10 administration and monitoring is explained. Very impressive is the amount of CLI examples. If you have a question: here you will find the CLI command, which provides the answer.

The last chapter "stream processing" is one of the longest chapters (>40 pages).  Here two APIs are presented to do some processing based on the messages. One example is, a stream which processes stock quotes. With stream processing it is possible to calculate the number of trades for every five-second window or the average ask price for every five-second window. Of course this chapter shows much more, but i think this gives the best impression ;-)

All in all a excellent book - even if you are not implementing Kafka ;-)

 

Linux ser2net: no connection to /dev/ttyUSB0

If you are running some java application on a Linux box (especially ARM architecture) and this application accesses the serial interface (/dev/ttyUSB0, /dev/ttyUSB1 or just /dev/ttyX), then a easy way to do this, is running ser2net.

For all who are not familiar with the serial port:
https://en.wikipedia.org/wiki/Serial_port

But there is one tricky thing, you have to consider when using ser2net:

Inside ser2net.conf you will find some lines like this here:

15000:raw:0:/dev/ttyUSB0:9600 8DATABITS NONE 1STOPBIT 

This means: on port tcp 15000 you can access the serial port /dev/ttyUSB0 (if you have a USB to serial adapter in place).

If this does not work, check the ports with

root@ubuntu:/home/ubuntu/dfld# netstat -lntup |grep ser2net
tcp6       0      0 :::15000                :::*                    LISTEN      1361/ser2net        

As you can see, it only listens on TCP6. So you have to reconfigure this to

127.0.0.1,15000:raw:0:/dev/ttyUSB0:9600 8DATABITS NONE 1STOPBIT

If you only want to access this on localhost (which is very nice security enhancement ;-) ).
And after a restart of ser2net everything works like expected:

root@ubuntu:/home/ubuntu/dfld# netstat -lntup |grep ser2net

tcp        0      0 127.0.0.1:15000         0.0.0.0:*               LISTEN      

How to run a java software which needs RXTX on a raspberry pi

In my last posting i wrote about migrating my aircraft noise measurement station to alpine linux. There i had some problems with getting the RXTX library for Java running on a linux, which uses musl and not GNU libc6.

Why does my java application require RXTX? As stated on the RXTX page:

RXTX is a Java library, using a native implementation (via JNI), providing serial and parallel communication for the Java Development Toolkit (JDK).

Now i wanted to move to a raspberry pi. But this runs on ARM and the RXTX is only provided for x86/x64 systems.

But there is another way: ser2net

With this proxy /dev/ttyUSB0 can be mapped to a tcp port and java can access this without using RXTX.

Alpine Linux, Oracle Java JDK and musl?! - why it does not work...

Sometime ago i did some work with Alpine Linux (s. here) and i was impressed how tiny this Linux distro was and how fast it was running.

So i decided after nearly 6 years of running an aircraft noise measuring station (for dfld.de) with Ubuntu to change to Alpine Linux.

This station runs a software on Java and needs RXTX, because the microphone is connected via USB and is read over /dev/ttyUSB0.

What is the problem with this setup?

• RXTX needs a Java which is running on glibc
• Alpine Linux does not run on glibc

If you are not aware of this problem, you get some errors like

./javaash: java: command not found

and this happens even if you are in the right directory and java got the execute bit configured.

Alpine Linux changed to musl:

There are some other libc implementations (take a look here).
The homepage is https://www.musl-libc.org/:

 And a comparison to other libc can be found at http://www.etalabs.net/compare_libcs.html:

There are some workarounds to get applications build with glibc running on Alpine Linux, but i did not get to run my aircraft noise measuring station - i switched to Debian - because i needed a 32bit Linux for my very old UMPC...

Java 11: JEP 333 ZGC A Scalable Low-Latency Garbage Collector

After i found this strange "No-Op Garbage Collector", i was keen, if there are some other new GC features with Java 11.

There is another JEP with the number 333:

 If you look here, the goals are:

• GC pause times should not exceed 10ms
• Handle heaps ranging from relatively small (a few hundreds of megabytes) to very large (many terabytes) in size
• No more than 15% application throughput reduction compared to using G1
• Lay a foundation for future GC features and optimizations leveraging colored pointers and load barriers
• Initially supported platform: Linux/x64

 Inside JEP 333 there are some numbers for the performance provided:

Below are typical GC pause times from the same benchmark. ZGC manages to stay well below the 10ms goal. Note that exact numbers can vary (both up and down, but not significantly) depending on the exact machine and setup used.
(Lower is better)

ZGC
                avg: 1.091ms (+/-0.215ms)
    95th percentile: 1.380ms
    99th percentile: 1.512ms
  99.9th percentile: 1.663ms
 99.99th percentile: 1.681ms
                max: 1.681ms

G1
                avg: 156.806ms (+/-71.126ms)
    95th percentile: 316.672ms
    99th percentile: 428.095ms
  99.9th percentile: 543.846ms
 99.99th percentile: 543.846ms
                max: 543.846ms

This looks very promising. But within the limitations you can read, that it will take some more time, until this can be used:

The initial experimental version of ZGC will not have support for class unloading. The ClassUnloading and ClassUnloadingWithConcurrentMark options will be disabled by default. Enabling them will have no effect.
Also, ZGC will initially not have support for JVMCI (i.e. Graal). An error message will be printed if the EnableJVMCI option is enabled.
These limitations will be addressed at a later stage in this project.

Nevertheless: You can use this GC with the command line argument

-XX:+UnlockExperimentalVMOptions -XX:+UseZGC

For more information take a look here: https://wiki.openjdk.java.net/display/zgc/Main

Java 11: JEP 318 Eplison / A No-Op Garbage Collector

This week i read about the new JDK Enhancement Proposal (JEP) of Java 11.
One of my favourites is JEP 318:

The summary of this JEP says:

Develop a GC that handles memory allocation but does not implement any actual memory reclamation mechanism. Once the available Java heap is exhausted, the JVM will shut down.

For which scenario could this garbage collector be useful?

If you skip to the "motivations" at the JEP, i found the following very interesting:

Performance testing. Having a GC that does almost nothing is a useful tool to do differential performance analysis for other, real GCs. [...]

There are other points mentioned there which are also interesting, but this performance calibration is in my opinion one of the most important ones.

One other is important too:

Extremely short lived jobs. A short-lived job might rely on exiting quickly to free the resources (e.g. heap memory). In this case, accepting the GC cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway. Note that the GC cycle might take a while, because it would depend on the amount of live data in the heap, which can be a lot.

This can be quite helpful for Oracle fn or for some special docker images, where a java funtion is called and after the execution the process is terminated.
Let's if this feature will get used in the future....

jconsole/visualvm/java applications without fonts over ssh forwarding (characters displayed as boxes)

On servers which run java applications sometimes you need to run jconsole or jvisualvm.
This is typically no problem - ssh -X and you are done.

But in some rare circumstances you will get something like this:

??

Following some other sources, you end up with tweaking xorg parameter without any success. The solution is very easy:

apt-get install ttf-dejavu

And then:

Java: Slow java with server.policy enabled - how to fix this issue

If you use Java security manager for hardening your java processes, you have to add the following JVM options:

-Djava.security.manager
-Djava.security.policy=server.policy 

Create a server.policy file (you can use jdkXXX/jre/lib/security/java.policy as a tamplate) and add the following line:

permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve"; 

Now create a small java program, which listens on a port (like this example).

If you send a message with netcat

nc -u localhost 9876

Everyhting is fine.
Now send a message from a remote host. This does not work - like expected.

Try it again with the following network tracing running (capturing all DNS packets):

tcpdump -i any port 53

Cool. For each connect a DNS-Lookup is done.
This could be a problem for high performance systems or for systems, which have to running/reachable DNS-Servers. In the latter case all requests will be sent to localhost:53 and of course, localhost will not give any answer. (This is not true - there will be a "ICMP - not reachable", but no DNS answer.).
If you add now line with *.*, to allow the connection the server.policy file should contain the following lines:

permission java.net.SocketPermission "*:*", "listen, accept, connect, resolve";
permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve"; 

Hmmm. The connection is allowed, but there still DNS requests happening. The problem is that "*:*" is behind the "localhost:*" because Java reads this file from bottom to top - so if you write it this way:

permission java.net.SocketPermission "localhost:*", "listen, accept, connect, resolve";
permission java.net.SocketPermission "*:*", "listen, accept, connect, resolve";

there are no DNS requests happening anymore.

If you still see DNS requests: Take a look at this file:

YourJDK/jre/lib/security/java.policy

there are some entries with java.net.SocketPermission like:

permission java.net.SocketPermission "localhost:0", "listen";

 Because java first checks this file, you have to remove such lines, to get rid of the DNS requests.

If you do not need to use DNS, you should remove dns in /etc/nsswitch.conf. But, then no domain lookup will succeed  on this machine anymore...

Java 10 released: java with some enhancements for running inside docker

After the release of Java 9 in october 2017 with its new features

• JShell with its special commands
• the new folder structure 
• and many more

Oracle released Java 10:

 A short summary of the new feature can be found
at https://blogs.oracle.com/java-platform-group/introducing-java-se-10
or you can take a look a the release notes:
http://www.oracle.com/technetwork/java/javase/10-relnote-issues-4108729.html#NewFeature

My favourites are:

• JEP 307 Parallel Full GC for G1  Improves G1 worst-case latencies by making the full GC parallel. The G1 garbage collector is designed to avoid full collections, but when the concurrent collections can't reclaim memory fast enough a fall back full GC will occur. The old implementation of the full GC for G1 used a single threaded mark-sweep-compact algorithm. With JEP 307 the full GC has been parallelized and now use the same amount of parallel worker threads as the young and mixed collections.
  
  See JDK-8172890

and the docker enhancements:

• JDK-8146115 Improve docker container detection and resource configuration usage

The JVM has been modified to be aware that it is running in a Docker container and will extract container specific configuration information instead of querying the operating system. The information being extracted is the number of CPUs and total memory that have been allocated to the container. The total number of CPUs available to the Java process is calculated from any specified cpu sets, cpu shares or cpu quotas. This support is only available on Linux-based platforms. This new support is enabled by default and can be disabled in the command line with the JVM option:

-XX:-UseContainerSupport

In addition, this change adds a JVM option that provides the ability to specify the number of CPUs that the JVM will use:

-XX:ActiveProcessorCount=count

This count overrides any other automatic CPU detection logic in the JVM.

• JDK-8186248 Allow more flexibility in selecting Heap % of available RAM

Three new JVM options have been added to allow Docker container users to gain more fine grained control over the amount of system memory that will be used for the Java Heap:

-XX:InitialRAMPercentage
-XX:MaxRAMPercentage
-XX:MinRAMPercentage

These options replace the deprecated Fraction forms (-XX:InitialRAMFraction, -XX:MaxRAMFraction, and -XX:MinRAMFraction).

• JDK-8179498 attach in Linux should be relative to /proc/pid/root and namespace aware

This bug fix corrects the attach mechanism when trying to attach from a host process to a Java process that is running in a Docker container.

Related posts:

• Docker on Ubuntu 
• Running through a tutorial / investigating the processes
• Creating a small linux for my docker cluster (Alpine Linux) 
• How to build a docker swarm 
• Creating a minimal webserver container and deploying it to docker swarm
• Setting up a tomcat with docker  
• Running a swarm with multiple managers 
• Mysql inside a docker container - persistency with docker volumes 
• How to change the configuration of a standard docker container(e.g. changing tomcats memory settings)

Java 9: JDK without subfolder JRE

In older JDK releases the directory structure was:

bin
db
include
jre
lib

With JEP 220 the jre folder was removed.

A JDK image no longer contains a jre subdirectory, as noted above. Existing code that assumes the existence of that directory might not work correctly.

But this does not mean that there is no JRE any more. You can still download the JRE. There are some minor changes to the JRE like:

The bin directory in a JRE image contains a few commands that were previously found only in JDK images, namely appletviewer, idlj,jrunscript, and jstatd. As with the previous item, these changes are a consequence of the way in which components that contain both APIs and tools were modularized.

For both version the files rt.jar, tools.jar and dt.jar are removed:

JDK and JRE images no longer contain the files lib/rt.jar,lib/tools.jar, lib/dt.jar, and other internal JAR files, as noted above. Existing code that assumes the existence of these files might not work correctly.

But this should not be a problem:

Class and resource files previously found in lib/dt.jar and visible only when that file was added to the class path are now visible via the bootstrap class loader and present in both the JRE and the JDK. 

Everyone who delivers Java applications should read  JEP 220 and check, what has to be changed...

OpenJDK 9: Limitations/shortcomings of the Jshell

Jshell is a quite powerful tool to do some quick explorations and to write some scripts.

If found the following limitations:

• No syntax highlighting
• Only comments with // are allowed
  if you open a script file with
   /**
    *  My comments
    */
  this will work only many warnings.
  

  |  Error:
  |  illegal start of expression
  |   * Copyright (c) 1995, 2008, Oracle and/or its affiliates. All rights reserved.
  |   ^
  |  Error:
  |  illegal start of expression
  |   *
  |   ^

• The usage of public, static, ... is quite different:
  

  -> public class MyClass {
  >> public int a;
  >> };
  |  Warning:
  |  Modifier 'public'  not permitted in top-level declarations, ignored
  |  public class MyClass {
  |  ^----^
  |  Added class MyClass this error is corrected:
  |      Modifier 'public'  not permitted in top-level declarations, ignored
  |      public class MyClass {
  |      ^----^

  
  
• If you want to load a scipt with /open then tab completion does not work. The complete path including filename has to be typed / pasted.
• If you load some example snippet, main(String[] args) is not run by default.
  Just tried with the swing tutorial from oracle:
  

  jshell Downloads/HelloWorldSwing.java  

  -> String[] mystringarray;
  |  Added variable mystringarray of type String[]
  
  -> HelloWorldSwing.main(mystringarray); 

• .. 

to be continued...

JDK 9.0 released! (but java.net is gone)

Some week ago, i downloaded jdk 9 via java.net. But this page in not available anymore:

After some searching i found the download page for the jdk: http://jdk.java.net/9/  (so java.net is not really closed...). But it is easier to download them from the official Oracle website:
http://www.oracle.com/technetwork/java/javase/downloads/index-jsp-138363.html#javasejdk

$java -version
java version "9"
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

The release note can be found here.

There you can find interesting things like:

The command line flags that were removed are:


• -Xincgc
• -XX:+CMSIncrementalMode
• -XX:+UseCMSCompactAtFullCollection
• -XX:+CMSFullGCsBeforeCompaction

• -XX:+UseCMSCollectionPassing

or

JEP 291: Deprecate the Concurrent Mark Sweep (CMS) Garbage Collector
Deprecates the Concurrent Mark Sweep (CMS) garbage collector. A warning message is issued when it is requested on the command line, using the -XX:+UseConcMarkSweepGC option. The Garbage-First (G1) garbage collector is intended to be a replacement for most uses of CMS.

 My favourite is:

JEP 222: jshell: The Java Shell (Read-Eval-Print Loop)
Adds Read-Eval-Print Loop (REPL) functionality to the Java platform.
The jshell tool provides an interactive command-line interface for evaluating declarations, statements, and expressions of the Java programming language. It facilitates prototyping and exploration of coding options with immediate results and feedback. The immediate feedback combined with the ability to start with expressions is useful for education—whether learning the Java language or just learning a new API or language feature.

(See my posting jshell and swing, jshell and scripts and jshell built-in commands) 

Ubuntu /etc/alternatives/java et.al.: Using java from external sources | update-alternatives | update-java-alternatives

After some problems with the jdk integrated in ubuntu 16.04 i installed the OpenJDK from http://jdk.java.net/9/.

The software is provided as tarball (tar.gz) and after

tar -xvf ~/Downloads/jdk-9+178_linux-x64_bin.tar.gz

the binaries can be used with

jdk-9/bin/java
...

But there are many links in /etc/alternatives still pointing to the ubuntu jdk:

lrwxrwxrwx 1 root root  41 Jul 11 21:02 /etc/alternatives/jar -> /usr/lib/jvm/java-9-openjdk-amd64/bin/jar
lrwxrwxrwx 1 root root  51 Jul 11 21:02 /etc/alternatives/jar.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/jar.1.gz
lrwxrwxrwx 1 root root  47 Jul 11 21:02 /etc/alternatives/jarsigner -> /usr/lib/jvm/java-9-openjdk-amd64/bin/jarsigner
lrwxrwxrwx 1 root root  57 Jul 11 21:02 /etc/alternatives/jarsigner.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/jarsigner.1.gz
lrwxrwxrwx 1 root root  42 Mai 31 21:09 /etc/alternatives/java -> /usr/lib/jvm/java-9-openjdk-amd64/bin/java
lrwxrwxrwx 1 root root  52 Mai 31 21:09 /etc/alternatives/java.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/java.1.gz
lrwxrwxrwx 1 root root  43 Jul 11 21:02 /etc/alternatives/javac -> /usr/lib/jvm/java-9-openjdk-amd64/bin/javac
lrwxrwxrwx 1 root root  53 Jul 11 21:02 /etc/alternatives/javac.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/javac.1.gz
lrwxrwxrwx 1 root root  45 Jul 11 21:02 /etc/alternatives/javadoc -> /usr/lib/jvm/java-9-openjdk-amd64/bin/javadoc
lrwxrwxrwx 1 root root  55 Jul 11 21:02 /etc/alternatives/javadoc.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/javadoc.1.gz
lrwxrwxrwx 1 root root  43 Jul 11 21:02 /etc/alternatives/javah -> /usr/lib/jvm/java-9-openjdk-amd64/bin/javah
lrwxrwxrwx 1 root root  53 Jul 11 21:02 /etc/alternatives/javah.1.gz -> /usr/lib/jvm/java-9-openjdk-amd64/man/man1/javah.1.gz
lrwxrwxrwx 1 root root  43 Jul 11 21:02 /etc/alternatives/javap -> /usr/lib/jvm/java-9-openjdk-amd64/bin/javap

You can change them manually by deleting and creating new links but i think this is to much work.
Update-Alternatives works, but you have to issue the following command many times:

update-alternatives --quiet --install /usr/bin/jconsole jconsole /home/data/opt/jdk-9/bin/jconsole 1191

Update-java-alternatives

update-java-alternatives.new -s java-jdk-9
update-java-alternatives.new: directory does not exist: /usr/lib/jvm/java-jdk-9

 does not work. But this can be fixed by creating your own jinfo file:

ls -la /usr/lib/jvm/.*jinfo
-rw-r--r-- 1 root root 2600 Mai  6 05:55 /usr/lib/jvm/.java-1.8.0-openjdk-amd64.jinfo
-rw-r--r-- 1 root root 2646 Apr 14  2016 /usr/lib/jvm/.java-1.9.0-openjdk-amd64.jinfo
-rw-r--r-- 1 root root 2058 Jul 14 22:05 /usr/lib/jvm/.java-jdk-9.jinfo

Just copy one of the existing files and change the paths to your JDK installation directory.
Then copy update-java-alternatives to update-java-alternatives and change the following lines:

inside the do_set function

#update-alternatives $uaopts --set $name $location
prio=$(awk -F= '/priority=/ {print $2}' $top/*$jname.jinfo)
update-alternatives $uaopts --install /usr/bin/${location##*\/} $name $location $prio

And then

update-java-alternatives.new -s java-jdk-9

will change all links in /etc/alternatives, so that running java without path will use the your installed version...

$ java -version
java version "9"
Java(TM) SE Runtime Environment (build 9+178)
Java HotSpot(TM) 64-Bit Server VM (build 9+178, mixed mode)

OpenJDK 9: Jshell - Using Swing / doing GUI stuff

After the posts buitin commands of jshell and how to load and save scripts i tried to get swing components running.

I created a script  HelloWorld.java

import javax.swing.*;       

JFrame frame = new JFrame("HelloWorldSwing");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
JLabel label = new JLabel("Hello World");
frame.getContentPane().add(label);
frame.pack();
frame.setVisible(true);

But this does not work with Ubuntu:

jshell HelloWorld.java
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f61041bb009, pid=7440, tid=7442
#
# JRE version: OpenJDK Runtime Environment (9.0) (build 9-internal+0-2016-04-14-195246.buildd.src)
# Java VM: OpenJDK 64-Bit Server VM (9-internal+0-2016-04-14-195246.buildd.src, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libjava.so+0x1d009]  JNU_GetEnv+0x19
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %P" (or dumping to /home/schroff/core.7440)
#
# An error report file with more information is saved as:
# /home/schroff/hs_err_pid7440.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
|  State engine terminated.
|  Restore definitions with: /reload restore
|  Resetting...
|  Welcome to JShell -- Version 9-internal
|  For an introduction type: /help intro

Hmmm. This does look good.

On a Windows 10 host it works without any problem:

I hope this bug will be fixed soon...
EDIT: After downloading a new version from java.net it worked:

./java -version
java version "9"
Java(TM) SE Runtime Environment (build 9+178)
Java HotSpot(TM) 64-Bit Server VM (build 9+178, mixed mode)

OpenJDK 9: Jshell - how to load scripts and how to save/persist finished snippets

In my last posting i showed the builtin jshell commands and how to start working with the java shell.

What about loading and saving scripts?

I created a file myshell.txt with this content:

class MyClass {
 private int a;
 public MyClass(){a=0;}
 int getA() {return a;};
 void setA(int var) {a=var; return;}
}
MyClass ZZ;
ZZ = new MyClass();
ZZ.setA(200);

The help shows the following:

-> /help /open
| 
|  /open
| 
|  Open a file and read its contents as snippets and commands.
| 
|  /open
|      Read the specified file as jshell input.

so i tried this one:

-> /open myjshell.txt

hmmm. No feedback inside jshell. But no news is good news:

-> /list

   1 : class MyClass {
        private int a;
        public MyClass(){a=0;}
        int getA() {return a;};
        void setA(int var) {a=var; return;}
       }
   2 : MyClass ZZ;
   3 : ZZ = new MyClass();
   4 : ZZ.setA(200);

 Saving your work is quite easy:

-> /help /save
| 
|  /save
| 
|  Save the specified snippets and/or commands to the specified file.
| 
|  /save
|      Save the source of current active snippets to the file.
| 
|  /save all
|      Save the source of all snippets to the file.
|      Includes source including overwritten, failed, and start-up code.
| 
|  /save history
|      Save the sequential history of all commands and snippets entered since jshell was launched.
| 
|  /save start
|      Save the default start-up definitions to the file.

And also no news is good news:

-> /save myjshell2.txt

and like expected:

$ cat myjshell2.txt
class MyClass {
 private int a;
 public MyClass(){a=0;}
 int getA() {return a;};
 void setA(int var) {a=var; return;}
}
MyClass ZZ;
ZZ = new MyClass();
ZZ.setA(200);

But what about this /save start ?

-> /save start myjshell3.txt

and the content of this file is:

$ cat myjshell3.txt

import java.util.*;
import java.io.*;
import java.math.*;
import java.net.*;
import java.util.concurrent.*;
import java.util.prefs.*;
import java.util.regex.*;
void printf(String format, Object... args) { System.out.printf(format, args); }

 To load a scipt on startup just type

jshell myjshell.txt

OpenJDK 9: JShell - an interactive java interpreter shell | builtin commands

One of the new features of java 9 is jshell (JEP 222).

On my ubuntu system the installation was quite easy:

# apt-get install openjdk-9-jdk-headless

and you can find

$ ls /usr/lib/jvm/java-9-openjdk-amd64/bin/
idlj       jcmd    jmap        jstatd       schemagen
jar        jdb     jmod        keytool      serialver
jarsigner  jdeps   jps         orbd         servertool
java       jhsdb   jrunscript  pack200      tnameserv
javac      jimage  jsadebugd   policytool   unpack200
javadoc    jinfo   jshell      rmic         wsgen
javah      jjs     jstack      rmid         wsimport
javap      jlink   jstat       rmiregistry  xjc

 (in the third column, sixth row: jshell)

After a startup jshell comes up with this prompt:

$ /usr/lib/jvm/java-9-openjdk-amd64/bin/jshell
|  Welcome to JShell -- Version 9-internal
|  For an introduction type: /help intro


->

 The most important command is

/exit

to leave the jshell (Strg-C works also, but i think /exit should be used).

There is no syntax highlighting but this does not matter.

The following builtin commands are allowed:

-> /help
|  Type a Java language expression, statement, or declaration.
|  Or type one of the following commands:

|     /list [all|start|]                       -- list the source you have typed
|     /edit                                    -- edit a source entry referenced by name or id
|     /drop                                    -- delete a source entry referenced by name or id
|     /save [all|history|start]                      -- Save snippet source to a file.
|     /open                                          -- open a file as source input
|     /vars                                                -- list the declared variables and their values
|     /methods                                             -- list the declared methods and their signatures
|     /classes                                             -- list the declared classes
|     /imports                                             -- list the imported items
|     /exit                                                -- exit jshell
|     /reset                                               -- reset jshell
|     /reload [restore] [quiet]                            -- reset and replay relevant history -- current or previous (restore)
|     /classpath                                     -- add a path to the classpath
|     /history                                             -- history of what you have typed
|     /help [|]                          -- get information about jshell
|     /set editor|start|feedback|newmode|prompt|format ... -- set jshell configuration information
|     /? [|]                             -- get information about jshell
|     /!                                                   -- re-run last snippet
|     /                                                -- re-run snippet by id
|     /-                                                -- re-run n-th previous snippet
| 
|  For more information type '/help' followed by the name of command or a subject.
|  For example '/help /list' or '/help intro'.  Subjects:

|     intro     -- an introduction to the jshell tool
|     shortcuts -- a description of shortcuts

With /list the source code, which you provided, is shown:

-> /list 5

   5 : class MyClass {
       private int a;
       public MyClass(){a=0;}
       int getA() {return a;};
       void setA(int var) {a=var; return;}
       }

Everytime you create an Object, you will see the following:

-> ZZ = new MyClass();
|  Variable ZZ has been assigned the value MyClass@28d25987

-> ZZ.getA();
|  Expression value is: 0
|    assigned to temporary variable $8 of type int

-> ZZ.setA(200);

-> ZZ.getA();
|  Expression value is: 200
|    assigned to temporary variable $10 of type int

With /vars the variables are shown:

-> /vars
|    MyClass ZZ = MyClass@28d25987
|    int $8 = 0
|    int $10 = 200

Listing the classes (ok it is getting boring):

-> /classes
|    class MyClass

 and last but not least /methods:

-> /methods
|    printf (String,Object...)void
|    getA ()int

Ubuntu: Eclipse does not find openjdk (JRE)

On my Ubunut linux i decided to do some Java programming.
So first step is to install java:

apt-get install openjdk-9-jdk openjdk-9-demo openjdk-9-doc openjdk-9-jre-headless openjdk-9-source

This just works like expected. But after that i installed eclipse.
Download via www.eclipse.org was easy and the installation ran without any error.
But Eclipse was not able to find the JRE for compiling, autocorrection, etc.

For all developers, which are not used to linux:
The JRE is in Ubuntu located here:

/usr/lib/jvm/java-9-openjdk-amd64/jre

And it is added like on any other platform via the preferences window:

 After that Eclipse runs fine and you can start building your java applications...

review at amazon: Apache SOLR for newbies

Last weekend this book fell into my hands:

Paul Lawson wrote a splendid introduction into Apache SOLR. At the beginning he describes the motivation behind the project SOLR:
 "Outside the firewall  [searching] is used to make money, and inside to save money."
Within 130 pages this books covers everything from installation, adminstration GUI, examples to start (importing documents, first searches, etc.) and the quick overview to the fundamental configuration files.
If you are interested in Apache SOLR and you do not know where to start: this book is the answer.
But one thing is missing: The book only covers searching XML documents. On page 42 (!) it is pointed out, that you have to configure SOLR cell. I think this was a good decision, because deep dives should be done after an introduction.

if you are interested, take a look at my review at amazon.de. (as all my reviews written in german ;-)

"Oracle has finalized the Sun transaction and the deal has closed."

The title of this blogging is from the offical website from oracle "Overview and Frequently Asked Questions for the Developer Community".
So what are the important answers for these questions?

For the near future, all these sites [Sun Developer Network, java.sun.com, and BigAdmin] will remain in their current form.

Java.net is an important part of the community, and Oracle will continue
to invest in it—as well as look for new and better ways to support its
membership

NetBeans.org will continue to be available at the usual URL—no changes.

We will also communicate important admin/Solaris-related news through
BigAdmin's existing newsletter.

But what has changed with closing the deal?
If you try www.sun.com you get redirected to www.oracle.com and you can find the hardware here:

And after some clicks you will see something like this:

If you want to get the sun hardware more quickly, you can use http://catalogs.sun.com.

Using Eclipse with Oracle's application server 11g (Weblogic)

JDeveloper 11g is fully integrated with Oracle Weblogic 11g (that is what everybody has expected). Now i tried to connect Eclipse to Oracle's new application server.
This was not easy, so here the steps, if you really want to stay with Eclipse....

1. Do a right click on the server tab and use the link Download additional server adapters:
2. After hitting the next button, i got the following error:
   org.eclipse.jst.common_core.... missing
3. So i installed the missing plugins via ->help->install new software:
   
4. But after this upgrade the installation of the Oracle Weblogic server plugins stills failed ;-(
5. So i googled around and found this page.
   

   1. Select Help > Install New Software.
   2. Click Add to add a new update site.
   3. In the Add Site dialog, enter http://download.oracle.com/otn_software/oepe/galileo, and then click OK.
   4. Select Oracle Enterprise Pack for Eclipse, verify that all of the subcomponents are selected, and then click Next.
   5. Confirm information presented in the Install Details, and then click Finish.

6. So next try:
   
   

After that you can choose inside the "new server wizard" Oracle Weblogic 10 or 11:

It works, but there is at least one big difference between JDeveloper and Eclipse: No SOA or BPEL extension is available for Eclipse...