RabbitMQ for Spring Boot with Jackson

Issue:

When you use rabbitMQ to send the java object you might get this exception. The error you will meet would like this :

Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [[B]

 

Solution:

This is caused by the MQ converter not using Jackson yet. So just add following piece to solve this issue:

@SpringBootApplication
@EnableRabbit

public class MYCloudApplication {

@Bean
 public MessageConverter jsonMessageConverter() {
 return new Jackson2JsonMessageConverter();
 }
Advertisements

RabbitMQ quick start for Spring Boot

  1. Install the rabbitmq
sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server stop
sudo service rabbitmq-server start
sudo service rabbitmq-server status
sudo rabbitmqctl status

cat /proc/$RABBITMQ_BEAM_PROCESS_PID/limits

The broker always appends to the log files, so a complete log history is retained.

/var/log/rabbitmq directory.
See /etc/logrotate.d/rabbitmq-server to configure logrotate.

The rabbbitMQ will run at port localhost:5672

 

2. Let us build a Spring Boot project to make MQ work.

2.1  Download the spring boot sample project from https://github.com/spring-projects/spring-boot/tree/master/spring-boot-samples/spring-boot-sample-amqp.

remember to change pom.xml parent as this:

 <parent>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-parent</artifactId>
 <version>1.5.10.RELEASE</version>
 <relativePath /> <!-- lookup parent from repository -->
 </parent>

 

2.2 Use eclipse to import this maven project. and use “java -jar” run it, you will see it continuously say hello.

2.3 The code use following tricks:

a. @EnableScheduling and @Scheduled(fixedDelay = 1000L)  will make the app auto start a Schedule task every 1 second and call the send() method to send a msg “hello” to the MQ queue “foo”:

@RabbitListener(queues = "foo")

defines a mq listener to listen to the queue of “foo”.

b. When a msg is in the queue, this method

@RabbitHandler
 public void process(@Payload String foo) { ...}

will be triggered to process the msg got in this queue, code just prints it our in stdout.

c. the method fooQueue must names as the fooQueue as the queue name in this example is “foo”.

public Queue fooQueue() {...}

d. This code:

 @Autowired
 private RabbitTemplate rabbitTemplate;

Will auto get a connection from the RabbitMQ so it can do the send later for you.

 

3. Access RabbitMQ GUI to make simple management

Enable the rabbitmq_management module:

sudo rabbitmq-plugins enable rabbitmq_management

The Web UI is located at: http://server-name:15672/
The user “guest” is created with password “guest” by default.

 

 

 

 

 

 

 

 

 

NLU tools – Apache Tika

https://tika.apache.org/

Apache Tika – This is a must have tool if you doing the Natural Language Understanding related work in Java. As you have to prepare your training materials  with many text and articles. Tika is a tool to help you extract the text from all kinds of the docs such as  html, PPT, word and other office doc types, and many many others.

“Tika detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF). ”

Add these dependency to your maven:

<dependency>
    <groupId>org.apache.tika</groupId>
    <artifactId>tika-core</artifactId>
    <version>1.16</version>
  </dependency>

and you can use the core tika, such as check the doc file type etc.

If you want more to extract content, you also need to add parser and also some others upon needs.

 <dependency>
    <groupId>org.apache.tika</groupId>
    <artifactId>tika-parsers</artifactId>
    <version>1.16</version>
  </dependency>

It also support running at Restful service mode with Jetty server. so you can call API through the web service. And it has simple GUI too.

 

 

How to call a c/c++ shared lib from java

You have two ways at here, JNI and JNA. JNA is based on JNI, but simpler at usage here. I will give an example for JNA usage today. It does not need use javah to create a header file.

  1. Add jna lib to you maven :
    <dependency>
    
     <groupId>net.java.dev.jna</groupId>
    
     <artifactId>jna</artifactId>
    
     <version>4.4.0</version>
    
     </dependency>
  2. Create a c file like this:
    cd /home/test
    
    touch MyTest.c
    /* MyTest.c */
    #include <stdio.h>
    
    void aSimplePrint() {
     printf("Hello world from C!\n");
    }
  3. compile the c file to the .so share lib file:
    gcc -c -fPIC MyTest.c -o MyTest.o
    gcc -shared -o libMyTest.so MyTest.o
  4. Create a java file like this, TestJNA.java
    import com.sun.jna.Library;
    import com.sun.jna.Native;
    
    public class TestJNA {
    
        static {
            System.setProperty("jna.library.path", "/home/test");
      }
    
        public interface MyTest extends Library {
            public void aSimplePrint();
         }
    
        public static void main(String[] args) {
            MyTest ctest = (MyTest) Native.loadLibrary("MyTest", MyTest.class);
            ctest.aSimplePrint();
         }
    }

    Compile to get the TestJNA.class.

  5. Then you can run the java main to test it:
    java -cp .:/path/to/jna-4.4.0.jar TestJNA

The whole process will be like this. And you could face some issue if lib is 3rd party one or others reason.
Here are some issues you may encounter in the process.

1. In linux, loadLibrary(“MyTest” will point to load a libMyTest.so file. You need pay attention name of lib at here.

2. “Exception in thread “main” java.lang.UnsatisfiedLinkError: no MyTest in java.library.path ……..”
This means java can not find the libMyTest.so in the lib folder.  This code is to solve that problem.
System.setProperty(“jna.library.path”, “/home/test”);
If you still have issue, you can try this code too:
System.load(“/home/test/libMyTest.so”);

3. If you can load lib, but can not find symbol or function name in the lib:
Exception in thread “main” java.lang.UnsatisfiedLinkError: Error looking up function ‘aSimplePrint’: /home/test/libMyTest.so: undefined symbol: …………………

Then use this command to check what a share lib .so contains in the API:
nm -D /home/test/libMyTest.so
By this way, you can know that (T) function is existing in the lib or not.

4. Command :
file /home/test/libMyTest.so
— will tell you info about this file include it is for 32bit or 64 bit will will cause some problem of some 3rd party lib files.

Multiple version JAVA in Ubuntu

Install JDK 8

sudo apt-get update ; sudo apt-get install openjdk-8-jre-headless
sudo  update-java-alternatives --list

to list off all the Java installations on a machine by name and directory, and then run

sudo  update-alternatives --config java

to choose which JRE/JDK to use.If you want to use different JDKs/JREs for each Java task, you can run can be configured (java, javac, javah, javaws, etc). And then

sudo  update-alternatives --config [javac|java|javadoc|etc.]

will associate that Java task/command to a particular JDK/JRE.

Solve the Json/Java polymorphism request by Jackson

By W.ZH

When convert between json and jave objects, you could often face the polymorphism issue.
Such as a json of Content could be:

{
  "type":"valuea",
  "value": {
    "valueaName": "corn",
    "bar":  "sweet"
  }
}

and also could be

{
  "type":"valueb",
  "value": {
    "valuebName": "toy",
    "color":  "yellow",
    "price":  "20"     
  }
}

Value objects in fact is depends on the type data to change. This requires that depending on the type data and create different value’s object at Java side, How to do?

Here is the solution at the Jackson code by using the     @JsonTypeInfo

let us define a abstract class Value and another two class ValueA and ValueB to Extends it.

public abstract class Value {
}

and ValueA to inherit Value

@JsonInclude(JsonInclude.Include.NON_NULL)
public class ValueA extends Value {

    @JsonProperty("valueaName")
    private String valueaName = "";

    @JsonProperty("bar")
    private String bar;
....................................
....................................
}

and ValueB Class to inherit Value.

@JsonInclude(JsonInclude.Include.NON_NULL)
public class ValueB extends Value {

    @JsonProperty("valuebName")
    private String valuebName = "";

    @JsonProperty("color")
    private String color;

    @JsonProperty("price")
    private String price;
....................................
....................................
}

 

Then after that we can create a Class for Content to let “type” files  work as a EXTERNAL_PROPERTY to control the Value object in side the Content:

@JsonInclude(JsonInclude.Include.NON_NULL)
public class Content {

    @JsonProperty("type")
    private String type;

    @JsonInclude(JsonInclude.Include.NON_NULL)
    @JsonProperty("value")
    private Value value = null;
    
    public Content (){
    }
    public String getType() {
        return type;
    }

    public void setType(String type) {
        this.type = type;
    }

    @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXTERNAL_PROPERTY, property = "type")
    @JsonSubTypes({ @Type(value = ValueA.class, name = "valuea"),@Type(value = ValueB.class, name = "valueb")})
    public void setValue(Value value) {
        this.value = value;
    }

    public Value getValue() {
        return value;
    }

}

In this way. @JsonTypeInfo in fact define how to rely on “type” to dynamically  serialize and de-serialize the json/java object.

To make this properly work, your Jackson version must higher than the 2.5 version. As I have faced the duplicated fields bug in the 2.5  version Jackson , here is sample dependency:

 

        <!-- Jackson -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.8.0</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-core</artifactId>
            <version>2.8.0</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>2.8.0</version>
        </dependency>

 

 

How to run Memcached on Mybatis

By W.ZH

1. Install Memcache

To start, install memcached via apt-get. such as in the Ubuntu 12.04

sudo apt-get install memcached

It auto starts the memcached

ps -ax | grep memcac
21199 ?        Sl     0:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1

Refer to official Mybatis link:  http://mybatis.github.io/memcached-cache/
Add the jar to your maven

<dependency>
    <groupId>org.mybatis.caches</groupId>
    <artifactId>mybatis-memcached</artifactId>
    <version>1.0.0</version>
  </dependency>
 and then add the memcached to your mapper if you want which MyBatis mapper to use it.
<mapper namespace="org.acme.FooMapper">
  <cache type="org.mybatis.caches.memcached.MemcachedCache" />
  ...
</mapper>
Create a memcached.properties file and put to your class path, eg resources folder.
# any string identifier
org.mybatis.caches.memcached.keyprefix=_mybatis_
# space separated list of ${host}:${port}
org.mybatis.caches.memcached.servers=127.0.0.1:11211
org.mybatis.caches.memcached.connectionfactory=net.spy.memcached.DefaultConnectionFactory
org.mybatis.caches.memcached.expiration = 600
org.mybatis.caches.memcached.asyncget = true
# the expiration time (in seconds)
org.mybatis.caches.memcached.timeout = 600
org.mybatis.caches.memcached.timeoutunit = java.util.concurrent.TimeUnit.SECONDS
# if true, objects will be GZIP compressed before putting them to Memcached
org.mybatis.caches.memcached.compression = false

In fact if you add multiple cache server at org.mybatis.caches.memcached.servers, it will has fail over ability among them. auto continue using live one if one die.

This intergration in fact based on the Spymemcached, is an asynchronous supported, single-threaded Memcached client. When you call any caching-related method on spymemcached’s MemcachedClient, it will be handled asynchronously. The client call method handles writing the details of the operation that should be performed into a queue and returning the control back to the client making the call. The actual interaction with the Memcached server, meanwhile, is handled by a separate thread that runs in the background.

My testing prove that it can improve the loading DB data at least 50% reading time if data has been in memcached. So this also proves that Mybatis self cache is not enough  big because it is not designed for cache only.