Taste of Apache Camel: Part V - Marshalling and Unmarshalling

In previous post we stored data to Apache Solr from text file written in special format. To parse this file, special parsing method was created. But we could avoid this method just by having data in some standard format like XML, JSON or CSV and use unmarshalling. I will show here how this is done with XML and JSON file formats.

XML

To do JAXB unmarshalling with Camel, new dependency must be added in pom.xml, camel-jaxb.
Next, file movies.txt from Post III must be changed. I will name it movies.xml and of course put in some tags:

For XML unmarshalling we must add JAXB annotations to our Movie POJO.

I surrounded it with Movies POJO for easier unmarshalling.

And now we will make a new route in configure() method in our route builder class.

First there must be defined JaxbDataFormat for route to know how to map XML to Object.
Route starts with loading movies.xml file and does unmarshalling with simple command and moviesData JaxbDataFormat.
Then processor sets route body from Movies to List so splitter would take whole body and we don't have to make new splitter method. And that is all there is to it.

Instead of unmarshalling whole XML, it can be done for every movie separately.

Defined JaxbDataFormat is now for Movie object.
Here file is loaded as before, but unmarshalling is done inside splitter which at first splits XML on movie tags.

JSON

Because unmarshalling JSON in Camel does not need JAXB annotation, different dependency must be added in pom.xml, camel-jackson.
Next, we will change file movies.xml into JSON format and name it movies.js:

The first route that unmarshalls XML will be changed just minimal.

As you can see, after unmarshall method, route calls json method where we specify how this JSON will be unmarshalled. Everything else is the same as in previous example. There is no need of JaxbDataFormat because here unmarshalling is done with POJO field and method names that must be the same as in JSON.

Marshalling

If you were converting XML into JSON by hand, marshalling can help you very easy and fast.

As you can see, first XML file gets unmarshalled and after that marshalling is done same way as unmarshalling of JSON. The String is then written into file movies2.js. Marshalling can be done with String as with InputStream.

My posts on Apache Camel:
 - Part I - Property Placeholder
 - Part II - Using Beans
 - Part III - More on using Beans
 - Part IV - Solr
 - Part V - Marshalling and Unmarshalling

Taste of Apache Camel: Part IV - Solr

In previous post we read file with some movie titles and actors playing in these movies and printed everything on console. But more often than not, received data should be stored somewhere. To not just save data in some file to disk, I will show how to use Apache Solr. It is enterprise search platform used for full-text search, hit highlighting, faceted search, etc.

First, we must create new Solr core (or change existing one if you like).

It has two fields important for our example (title and actors), the rest is mandatory for Solr to run properly.
To use Solr with Camel, new dependency must be added in pom.xml, camel-solr

From here on, there is more than one way to store data to Solr with Camel.

Example 1: Add Bean

For start we will continue from previous post.
First we must make some minimal changes to Movie POJO. In order to store data in this POJO to Solr, some annotations must be added.

Here I annotated fields with SolrJ annotation @Field, but setters (not getters) could be annotated, too.
In ExampleBean we don't need to make changes, at least for this example.
But our route will have to be changed too in order to call Solr. We will make a new route in configure() method in our route builder class.

This route is a copy of the last one from previous post with few changes. First, there is no more processor which was there just for printing out to console. It is replaced with setHeader and URL to Solr core. In header we must set Solr operation to ADD_BEAN since this is what we try to store. And then call Solr that will receive data from out POJO. After splitter, data must be commited and optionally optimized in order to search these in Solr.
This approach is very easy to use but with a lot of data it becomes quite slow. That is because we feed Solr with single document at the time. This can be much faster if we could send more than one document.

Example 2: Insert SolrInputDocument

To achieve that we must first change ExampleBean method giveMeMovies or make new one. I made new one.

Here we see that method does not return list of Movie object, but Solr's own SolrInputDocument. Here you must know field names in Solr schema, but they are the same as in Movie class. Insted of setting data in Movie object with setters, we put it into SolrInputDocument with setField method.
To store SolrInputDocument, route must be changed, too.

Here the change is minimal. Operation was changed from ADD_BEAN to INSERT. This should work nicely, but it still does not fix speed issue. This code still sends single document at the time to Solr. To get around this, splitter must go.

Here you see that splitter was replaced with bean call and operation INSERT was replaced with INSERT_STREAMING. Now Solr will be called only once and it will receive whole list of movies.

Camel-solr component has few more operations, like DELETE_BY_ID and DELETE_BY_QUERY, but it does not have operation for querying and retrieving data. To do that, you can query Solr using HTTP.

My posts on Apache Camel:
 - Part I - Property Placeholder
 - Part II - Using Beans
 - Part III - More on using Beans
 - Part IV - Solr
 - Part V - Marshalling and Unmarshalling

Taste of Apache Camel: Part III - More on using Beans

Let us continue with the topic about beans from previous post. I mentioned that beans can process and convert data to different types.

Example 3:

Make a file movies.txt that looks like this (movie name:actors separated with comma):

Now we'll try to read the file and put it in simple POJO.

For route to read a file, there is file component. So let's make a new route in configure() method in our route builder class.

In this route we're using file component to read a file movie.txt from some folder and with property noop=true the file will not be removed but just read. The body that component file produces will be of type java.io.File.
When invoking our bean ExampleBean there is a new method giveMeMovies that accepts body from route. In our bean we must therefore create this method.

In this method implementation I use @Body annotation. It is not necessary but I use it for easier code reading.
The method just reads file line by line and creates list of Movie objects. This list is then returned to route and becomes new body with different type. In the route there is processor to show how to get body and data from it.

Example 4:

Method giveMeMovies in ExampleGean returns a list of objects. That's why this method can be used with splitter. Splitter splits a message into a number of pieces and process them independently. This time we will not change ExampleBean since we have everything we need to show how splitter works. We just need to make a new route in configure() method in our route builder class.

This route has exactly same functionality as one in Example 3 but is few lines shorter. Be careful to close splitter with .end() or you could have problems when route does something after splitter.

My posts on Apache Camel:
 - Part I - Property Placeholder
 - Part II - Using Beans
 - Part III - More on using Beans
 - Part IV - Solr
 - Part V - Marshalling and Unmarshalling

Taste of Apache Camel: Part II - Using Beans

In my previous post I showed how to use Property Placeholder. Next we'll look into using Beans. They can be used in few ways mentioned on Apache Camel page.
Lets make a new Maven project and pom.xml should look something like this:
This time blueprint.xml will not be necessary, unless you want to run it in container like Karaf or Fuse.
To demonstare how beans work, lets create one.
The bean is very simple. It has only one method that prints out content of route body. This method also has annotation @Handler to indicate that this method should be used for Bean Binding. This has an advantage as you need not specify a method name in the Camel route, and therefore do not run into problems after renaming the method in an IDE that can't find all its references.

Example 1:

The route is simple, too.
The component timer fires and generate message exchanges. Here route is fired every 10 seconds with initial delay of 10 seconds. Parameter repeatCount specifies a maximum limit of number of fires, where zero or negative means fire forever. Timer component is part of camel-core, but for more advanced features camel-quartz could be used.
Body for the route is here set to some hardcoded string, and our bean will get it as parameter to the method. Bean is invoked as endpoint with .bean(). This is enough. Camel knows which method to use, because it is annotated with @Handler annotation.

Example 2:

Lets make another method in out ExampleBean bean. This time method is not annotated, because only one per bean is allowed. This method has two parameters and returns another string.
Now let make another route in configure() method in our route builder class. Here we specified which method from our bean route must use. In double quotes method name must be specified and what the values of parameters will be. If you would like to know how Bean Binding works in more details, go here. When route is started, it should be accessible on url http://localhost:2208/beanexample2/{name}/{surname} and it should return an answer that answerMe method in ExampleBean put together.
Here we see that bean changed body value of out route just by returning some value. The same way the body can be converted to some other type as I will demonstrate in one of my future posts.

My posts on APache Camel:
 - Part I - Property Placeholder
 - Part II - Using Beans
 - Part III - More on using Beans
 - Part IV - Solr
 - Part V - Marshalling and Unmarshalling

Taste of Apache Camel: Part I - Property Placeholder

I started learning Apache Camel about a year ago. All routes are running in Red Hat JBoss FUSE, but about that some other time.

All routes are written in Java DSL and Blueprint with help of Maven.
I will show how to use Property Placeholder in blueprint.xml to set some properties which will be used in route. The route will be simple. The route will listen on some URL on port, that is set in blueprint.xml and returns some answer.
First we must set pom.xml:

Next step is blueprint.xml. It must be in folder resources/OSGI-INF/blueprint. Here are two properties. First one is port on which this service will be accessible. Second one is answer of this service. What are all other things I will try to explain when looking at FUSE Fabric.
Finally the source code. Every class that has route defined as Java DSL, must extend org.apache.camel.builder.RouteBuilder. And that class must implement configure() method. To run this route in shell, main method must create org.apache.camel.main.Main object. When route is started, it should be accessible on url http://localhost:2013/ppexample/{yout text}.
If this works, you should notice that port is set to what was value in main method and not in blueprint.xml. If you omit setters in main method, values from blueprint.xml will be used.

My posts on APache Camel:
 - Part I - Property Placeholder
 - Part II - Using Beans
 - Part III - More on using Beans
 - Part IV - Solr
 - Part V - Marshalling and Unmarshalling