Serving the Mule Tools XSD using the Apache Rewrite Module and PHP

After I created the NamespaceHandler and XSD for the Integration Cocktail Mule Tools module, I wanted to make the XSD available at the URL http://www.integrationcocktail.org/schema/mule/tools/3.2/mule-tools.xsd so that users of the module will have code completion using any half decent XML editor. I bought the domain and was going to use a simple LAMP server to provide the XSD. It was straightforward enough if I wanted to simply put the XSD file, however I know myself, I will forget to update the XSD file after I will do any form of updates to the file. So why not provide the schema directly from the git repository? So, after years not touching any form of PHP I was going to hack a bit with it.

First I added a .htaccess file to rewrite all requests for .xsd files which do not exist to my php script:



After that all I had to do is create a simple script to proxy. I used PHP cURL library to do the requests and voila, simple enough:



Anybody out there who is creating custom mule modules and as they should, to encourage use, creates a custom namespace, please, do make the XSD available for your fellow developers so that you do not have to stay adding it to the IDE.

Twittering new Posts using Mule Studio and ION

I must admit that personally I am not really much into social media and rarely find myself twittering or updating my status. However anyone must admit that they are valuable channels to pass on information and want to start tweeting every time I publish a new post. So this was a great opportunity to try out the new Mule Studio and MuleION and deploy this simple application.

I was involved in the testing of the earlier Mule Studio releases and must admit that this has been polished quite a lot. It is quite easy to create a Mule Project and deploy it to Mule. Another nice feature is that it is easy to run the application in Mule and also, every time you save the configuration, the mule application is reloaded. It is also just as easy to deploy to MuleION (Mule's cloud offering), just a simple click. Now lets take a quick look at the application:

Blog Notifier Flow from Mule Studio

1. Using the poll component and an http endpoint we can poll the atom url of this blog which is http://integrationcocktail.blogspot.com/atom.xml
2. The http receiver returns a stream which we need to transform into a String so that then we can split.
3. We use the splitter in order to split the atom.xml into the different blog entries. Actually, we only care of the entry link since it contains the URL and the title. We use the xpath expression /atom:feed/atom:entry/atom:link[@rel='alternate']. We also need to add the atom namespace to the namespace manager.
4. Here we simply log the information of the entry being processed  : Processing post  title=#[xpath://@title] and path=#[xpath://@href]
5. Here we check of the entry is already in MongoDB. If it is present already, then this entry has already twittered, otherwise it still needs to be twittered. We use the count-objects-using-query-map method and pass as parameter the path. The target of the enrichment is #[header:existsInMongoDb] and the source is the expression #[groovy:payload>0]. So existsInMongoDb header is true if Mongo DB returned a count larger than 0.
6. When the header evaluator with expression existsInMongoDb=true returns true, then we simply log that this entry has already been processed: Post already twittered:title=#[xpath://@title] and path=#[xpath://@href].
7. Otherwise we log that we are about to twitter: Twittering:title=#[xpath://@title] and path=#[xpath://@href]
8. We update the status with the title and the path
9. We insert the title and the path into Mongo DB since it has now been published

Finally, to upload to ION, you can do it very easily from the IDE.

The following is the full configuration:



Mule Studio is quite mature, and it was fun to just drag and drop what you want... most of the time. There are obviously some items missing, for example, the namespace manager is not part of the xml elements, so I had to do that in the xml configuration mode, which did not know about it either. Another slight disappointment was that the sources are not attached, which will make it impractical to debug. I would also have preferred to have the project be a maven project, but then, I do agree that not everybody likes maven so it does not make sense to tie the IDE with any particular build tool. I guess the next step would be to find a way for Mule Studio to complement my development approach which focuses more on automated tests rather then running the application in a server and find a way for Mule Studio to play nice with maven. Once I have that figured out I will probably start using Mule Studio exclusively when developing Mule applications since it makes it so easy to deploy the applications to a standalone server or upload it to ION.

Book Review: Spring Integration in Action

Spring Integration in Action

Finally I have gotten some time to spend on Spring Integration and started by reading Spring Integration in Action MEAP Edition version 7. It is the first time that I am reading a MEAP Edition so will not be able to accurately compare the book, however, from my point of view, it is already in pretty good shape and the mistakes where minimal.

This book is great for anybody who wants to start delving in the world of integration. The book isn't simply a list of recipes of things you can do with Spring Integration but rather discuss the integration issues in their own right and then describe how you would do it using Spring Integration. The first two parts are quite heavy on theory which is great for beginners however developers who have been doing integration with other tools might want to skim through these chapters and look at the labs. I was quite happy to see the discussion of transaction boundaries discussed as early as Chapter 3 since it is a topic which I feel is sometimes ignored.

The third part starts with a discussion about splitting and aggregating messages then focused more on the different transports. The most interesting part here is the file based collaborative trip diary editor which is described in this section. This lab describes how you can have a collaborative text editor by writing “diffs” to the file system and have multiple clients writing to the same directory and thus working on the same file concurrently.

Personally I found the final part quite interesting. Well, to be honest, I was a bit confused why the chapter about twitter and chatting was not placed in the previous part, but apart from that, the rest is great. I loved the introduction to Spring Batch and the discussion on OSGI & scaling up applications. Something which I found surprising was how easy it is in Spring Integration to monitor the message path. The testing chapter is also quite interesting but I would have moved it earlier in the book since it can help with understanding and explaining.

All in all, this is a great book and can't wait to get the final version. I give this book a score of 4 out of 5 but has the potential to be 5 out of 5 once it is ready.

Rethrowing Caught Exceptions in Camel

Compared to the RouteController, this post will be much simpler. I had a number of situations where I wanted to perform some processing when an exception is thrown, however I did not want to simply catch the exception and "gobble it up". I wanted to catch the exception, do some processing and then re-throwing the exception so that the the exceptional flow continues. One example of such use was that when a specific exception is thrown, I wanted to stop the current route, give a job to quartz to restart this route withing 15minutes and then re-throw the exception so that the exceptional logging and flow continues (for example, rolling back the transaction).

For this we made use of the RethrowException bean which has a very simple implementation. A single void method which takes an Exchange as a parameter. The only slightly interesting thing about this post is how we get our hands on the Exception that Camel caught for us. That is quite straightforward because luckily for us, Camel sets the caught exception as a Property on the exchange with the key CamelExceptionCaught. So the code for the RethrowingException bean is:

Controlling Flows in Mule

After we saw how we can control routes in camel, we can now take a look at how to get the same behaviour in Mule. In mule instead of having routes we have flows. One thing to note is that a flow can be started or stopped but there is no notion of suspending and resuming flows. Apart from flows there are the deprecated Services which were use pre Mule 3. We should handle them as well and services do actually support pause and resume.

So what we will do is query the registry to get the service or flow. If it is a service then we can perform all 4 actions. If it is a flow, we can only perform stop and start. If we get a request for a pause, we will log a warning and perform a stop instead, whilst if we get a resume we will log a warning and perform a start.

The code of the FlowController is as follows:



One thing to note is that we are using the Lifecycle interface in order to control the flow. This actually can allow us to generalise our FlowController to start/stop not only flows and services but any component in the registry which implements the Lifecycle interface. This means that you will be able to control Connectors as well for example. For this we will make a minor change as follows:



Even though now our FlowController can do more than just control Flows and Services, I will still leave it called the FlowController since that is its main use, LifecycleController does not really cut it in my opinion ;).

As always you can browse the FlowController code and for usage examples you can look at the test implementation and test configuration.

Routes Stopping and Suspending themselves in Camel

As we seen in the post "Controlling Routes" in Camel it is quite straightforward to control other routes, but what about controlling ”yourself”? What if you want to stop yourself?

In such situations, things get slightly more complicated. The main problem is the DefaultShutdownStrategy which when configured for graceful shut-down (this is the default) it will wait for all in-flight messages to finish before shutting down. Why would this be a problem? Well, if a route is trying to stop itself using the Route Controller we defined earlier, the Route Controller will call Stop, however there are still in-flight exchanges (the same exchange that is currently being processed by the Route Controller) which will cause the DefaultShutdownStrategy to wait until the time-out before shutting down the route. Don't get me wrong, the DefaultShutdownStrategy is doing its work correctly, it is the Route Controller which needs to handle itself better.

The Camel documentation suggests that in such situations, you should remove the current exchange from the list of current in-flight messages. This might seem at first glance to have solved the problem, but it will not always work well with all transports. Polling transports like the file transport will still cause the DefaultShutdownStrategy to wait because there is still a polling thread working (The same thread that is trying to do the stopping). Also, even if you wait for the time-out, the file which cause the time-out is not removed since technically the file has not yet been completely processed since the processing was stopped in mid way. In short, I could not see a clean way to do it from within the same thread.

The simplest solution that I found was to call the stop from a different thread. This will allow the ”current” flow to finish normally (the exchange processing is completed) while the stopping is done in a different thread. The simplest way would be to spawn off a new thread to do the stopping, and that is what I have done here.



Spawning off threads like this is not ideal since you could run in situations where you end up spawning threads out of control. A further improvement is to use the ExecutorService to do this for you (Similar to how the DefaultShutdownStrategy gives the task of shutting down to the ExecutorService in order to be able to support time-out).



After this change, a Route should be able to gracefully stop itself using the Route Controller

Controlling Routes in Camel

I have had the need to be able to control routes from different routes quite a number of times (both in Mule and Camel). Typically this is useful if you would like to be able to remotely control your routes using any of the available transports. Yes, some might say that you have JMX available to do this and even if your administrators do not want to open up any new ports on the test environment you can always overload a port already being used by the ESB and proxy for example the MX4J interface, however that is a different story.

But why not control your routes using the same way as you interact with your application. If you are sending JMS messages to your application, why not have a queue which receives control messages. Or if you are using web services, why not expose a web service which will allow you to control the routes. This is one beauty of an ESB, you have so much flexibility when it comes to transports to use.

So what I did for camel was create a simple bean, which will perform the control action (START, STOP, RESUME, SUSPEND) on a specific route depending on the Route Id.


The bean has two attributes, routeId and action which can be set to give the bean a default value and a method which takes an Exchange as parameter which is the entry point. From the exchange we attempt to get headers "IntegrationCocktail.RouteController.RouteId" and "IntegrationCocktail.RouteController.Action". If these headers are not set, we use the defaults set directly on the bean. Finally, depending on the action we simply call context.startRoute/stopRoute/resumeRoute/suspendRoute(routeId).

To use this bean, you can choose to either give it static behaviour by setting the attributes on the bean, or dynamically by setting the appropriate message headers. The source code is availale on the integration-cocktail github repository and I created a camel-tools project where I will be putting such tit bits. For examples for how to use this bean you can look at the tests.

Finally, a question that inquisitive readers might ask is, will it be possible to stop or suspend the current route? Can you use this bean to for example stop the current route when an exception is thrown? Technically, it will work but not elegantly since by default the DefaultShutdownStrategy kicks in to enforce a graceful shutdown which will wait for all in flight exchanges to finish. However, the current exchange which initiated the stop request is still in flight so you will end up waiting until the timeout. We will polish the behaviour of the RouteController for such situations in another post.

Until then, feel free to to suggest other possible ”tools” for both Camel and Mule either by commenting on the post or at integrationcocktail@google.com.