TomcatExpert

Basic Apache Tomcat clustering for Grails applications

posted by pledbrook on July 20, 2010 06:43 AM

Grails is a rapid application development framework for web applications on the Java platform. It's similar in many respects to Ruby on Rails, but it's based on Java libraries, like Spring and Hibernate, and the Groovy language. It also produces standard WAR files that can be deployed to a servlet container like Tomcat. That means you can deploy Grails applications to a Tomcat cluster and in this article I'll show you how.

Getting started

In order to demonstrate clustering, we need an application to deploy. Assuming that Grails is installed (I was using Grails 1.3.3 - the latest) we can create a brand new application with the command

grails create-app my-cluster-app

This will create a my-cluster-app directory containing the project. If we then switch to that directory, we can generate a WAR file straight away by running

grails war

Of course, the application doesn't do anything yet. Nor is it ready for clustering.

Those in the know are aware that a Java web application can't be clustered unless it is marked as 'distributable' and Grails applications are not by default marked as such. The easy solution is to add the <distributable/> element to the web descriptor, but you won't find a web.xml file in a standard Grails application. That's because Grails generates the web descriptor on the fly when it needs one. Fortunately, a web.xml that you can edit is only a command away:

grails install-templates

This will install several template files in your project, but the one we're interested in is src/templates/war/web.xml. Open this up and add the <distributable/> element:

<web-app ...>
  <display-name>/@grails.project.key@</display-name>
 
  <!-- Add this line -->
  <distributable/>
  &hellip;
</web-app>

Now when Grails generates the WAR file, the web application will be distributable.

It would be great if this was all we needed to do to make a Grails application ready for clustering, but that bane of deployments everywhere, configuration, must be tackled first. In a standard Grails application, the relevant files are packaged with the WAR file. This is convenient for development since you don't have to worry about managing external configuration files and their locations, but it prevents you from customising the configuration per deployment. What's the solution?

Local configuration

There's no getting away from the fact that the simplest way to provide per-deployment configuration is via external configuration files located on the deployment servers. This allows you to reconfigure the application at any time with a quick edit and server restart rather than a redeployment of the WAR. Grails allows for external configuration files through a special grails.config.locations setting in the grails-app/conf/Config.groovy file:

grails.config.locations = [ "file:./${appName}-config.groovy", "classpath:${appName}-config.groovy" ]

This tells Grails to load two configuration files, one from the local filesystem and one from the classpath. Why the reference to 'appName'? It's so that you can easily host multiple web applications on a server/cluster without the names of the configuration files conflicting with each other. At runtime, ${appName} is replaced by the name of the application, i.e. the name passed to the create-app command - 'my-cluster-app' in this case.

The first path is for development when you're executing grails run-app - the configuration file just goes in the root of the project. The second item is for WAR deployment and causes Grails to look for the file on the classpath. You often don't know at development time where the current working directory is or where configuration files go on the deployment server's file systemm. So where can you put the configuration file so that it goes on the classpath? Since Tomcat add its lib directory to the classpath, that's the ideal location. Don't worry if Grails can't find one or other of the specified files: it will simply print a warning to stderr but continue to start up normally.

We'll return to these configuration files and what to put in them later when we come to the actual deployment. That story isn't over yet. For now, I want to consider features that would be affected by clustering, such as caching, scheduling, etc. It's easy to overlook them because Grails typically makes it very easy to include features without explicit configuration, but if you deploy to a cluster with their default settings, they will typically break.

Distributed caching

When you first create a Grails application, it has Hibernate's 2nd-level cache already enabled and configured to use Ehcache. You can take a look at the grails-app/conf/DataSource.groovy file to confirm this - you should see the following:

hibernate {
    cache.use_second_level_cache = true
    cache.use_query_cache = true
    cache.provider_class = 'net.sf.ehcache.hibernate.EhCacheProvider'
}

As you can see, it configures Ehcache as the cache provider. What you won't see is an Ehcache configuration file (ehcache.xml) in the project. That means Ehcache will use its default settings. This isn't recommended for a standalone server, let alone a clustered one. An additional concern is that Ehcache should be configured differently for local development (using grails run-app) and deployment to the cluster. How do we manage that?

Ehcache will load the ehcache.xml file from the classpath. We can easily make sure that the file ends up on the application's classpath by placing it in either the grails-app/conf or src/java directories. The trouble is, that file will be used in both development and the deployed web application. If you want different files for development and deployment you have to resort to a clever trick: replacing the file at packaging time.

The Grails build system allows the developer to interact with the build via an event system. One of those events is triggered just before the WAR is created, which allows us to influence what goes into the final package. All we have to do is create a scripts/_Events.groovy file containing this code:

eventCreateWarStart = { name, stagingDir ->
    ant.copy(file: 'cluster_resources/ehcache_distributed.xml',
             tofile: "$stagingDir/WEB-INF/classes/ehcache.xml", overwrite: true)
}

This event handler will copy the distributed cache configuration file from the cluster_resources directory to the staging directory, replacing the standard ehcache.xml. Once its done, Grails zips up the staging directory as the WAR file. Job done. Note that cluster_resources is not a standard Grails directory, so it must be created manually. You can find it along with the Ehcache configuration files in the attached project.

Caching isn't the only case where you have to do something special for cluster deployment, but external configuration files and tinkering with the generated WAR file will typically work for other cases as well. For example, you can use the same technique we used for ehcache.xml to ensure that an appropriate quartz.properties file (if you're using Quartz) is used by the application on the cluster.

We're now almost ready for deployment, but we don't yet have a way to test the session replication. For that, we need some web pages that utilise the HTTP session.

Creating the test pages

This being Grails, creating test pages is straightforward. We're going to use a single controller with three actions (which correspond to three distinct pages): 'arrive', 'leave', and 'counter'. Create the file grails-app/controllers/HomeController.groovy and give it this content:

class HomeController {
    def arrive = {
        if (session.counter) {
            render "You are already here"
        }
        else {
            session.counter = 1
            render "You have arrived - counter initialised"
        }
    }
 
    def leave = {
        if (!session.counter) {
            render "You have already left"
        }
        else {
            session.counter = null
            render "You have left - counter cleared"
        }
    }
 
    def counter = {
        if (session.counter) {
            session.counter++
            render "Counter: ${session.counter}"
        }
        else {
            render "You're not here!"
        }
    }
}

When we hit the page /home/arrive the counter variable will be added to the session. This counter will then be incremented and displayed each time we go to the /home/counter page. Finally, the variable is removed from the session when we hit /home/leave. Simple, pointless, but it does demonstrate the session behaviour of the application. We can now run the grails war command to generate a WAR file.

Deployment

Now that we have a WAR file, we can focus on deploying it to a Tomcat cluster. It's not the focus of this article to detail how to set up a Tomcat cluster, but in case you haven't done it before I have attached a package of scripts based on Burt Beckwith's work and using Tomcat 6.0.28. Just download the tomcat-cluster-scripts.zip archive and unpack it to any directory, for example in ~/tomcat-cluster-scripts. In the root of the archive you will see several shell scripts which make creating and managing a Tomcat cluster almost trivial.

Let's start by creating a cluster in a /opt/java/tomcat-cluster directory:

cd ~/tomcat-cluster-scripts
export CR=/opt/java/tomcat-cluster
sudo ./createCluster.sh my-cluster-app

This will create a cluster for the application 'my-cluster-app'. You can pass the location of the cluster to all the scripts, but it's easier to create a CR environment variable containing the location and run the scripts without that argument. Saves typing! Our cluster is now ready and all we need to do is add some Tomcat instances:

sudo ./createInstance.sh 1 1
sudo ./createInstance.sh 1 2

These commands will create two instances (1 & 2) on server 1 - the first argument being the server number, the second argument the instance number. You can find the instances at $CR/instance_1 and $CR/instance_2. In this example, I'm only creating a single server cluster, but you can run the scripts on multiple servers to create a multi-server cluster - just make sure you use a different server number on each one. As a side note, the scripts only work if you use single digit numbers for the instance numbers, so don't try to create more than nine instances on a server with them!

Before we start the Tomcat instances, we must first deploy our application and configure it. For the physical deployment, we can use the deploy.sh script:

sudo ./deploy.sh /path/to/myapp-0.1.war

This will unpack the WAR into the $CR/shared/webapps/ROOT directory, so the application will be served from the root context. This makes it easier to load balance the application with something like Apache httpd. The next step is to configure the database and logging for our instances. If you look in the $CR/shared/lib directory, you will see a my-cluster-app-config.groovy. What's this for? If you cast your mind back to earlier in the article, you will remember that we added the line

grails.config.locations = [ "file:./${appName}-config.groovy", "classpath:${appName}-config.groovy" ]

to the application's Config.groovy file. Since the shared/lib directory is on the classpath of all the cluster instances, we can add extra configuration via this my-cluster-app-config.groovy file. This could include database connection settings, logging configuration, or anything else. In this case, we'll configure the database connection settings and make sure that log files are created per instance. Open up the file up and add the following:

dataSource {
    dbCreate = 'update'
    username = 'grails'
    password = 'grails'
    driverClassName = 'com.mysql.jdbc.Driver'
    dialect = org.hibernate.dialect.MySQL5InnoDBDialect
    url = 'jdbc:mysql://localhost/myapp'
}
 
log4j = {
    appenders {
        rollingFile name: 'main', file: "${System.getProperty('catalina.base')}/logs/main.log"
    }
 
    root {
        info 'main'
    }
}

On the database side, the above configures all instances to use the 'myapp' MySQL database with the 'grails' user. Of course, you need the MySQL JDBC driver on the classpath, but that's as easy as copying the JAR into the shared/lib directory. On the logging side, we want to make sure that the application log file is created in the logs directory of each instance. Since the location of the instance directory is available in the 'catalina.base' system property, we can easily include it in the log file's path as we have done above.

Assuming that the MySQL database already exists, we're ready to test the cluster. Start the Tomcat instances with these commands:

sudo ./run.sh start 1
sudo ./run.sh start 2

After a little while you should be able to point your browser at http://localhost:8091/home/arrive and see the message "You have arrived - counter initialised". Now replace the 'arrive' in the URL with 'counter' and reload the page a few times. The counter will increment on each refresh. So that's the first Tomcat instance working. What about session replication? To test that, go to this URL: http://localhost:8092/home/counter - you are now hitting the second server. If everything is working properly, you should see the counter incremented from the previous number you saw - the session replication is working!

The next step would be to put a load balancer in front of the Tomcat cluster, but that's beyond the scope of this article. Suffice it to say that you don't have to do anything special for Grails applications. Should you choose to use Apache httpd with mod_proxy_balancer, you may be interested in the modproxybalancer Grails plugin which allows you to redeploy your application via a single Grails command, although you need to have Tomcat's manager application installed in all your instances.

Wrap up

Since Grails applications are packaged as WAR files, you can deploy them to a servlet container just as you would any other Java web application. That's one of the key strengths of Grails. Tomcat clusters are no different, but they do force you to do a bit more custom configuration of a Grails application than would normally be the case. As you've seen, this is straightforward using externalised configurations. When you need more flexibility, you can always make use of the Grails build system with its events or use JNDI.

I hope this post proves useful to both Grails developers who have never set up a Tomcat cluster and those people who have to deploy Grails applications in production systems. Although their configuration is different to typical Java web applications, once you know the techniques, it's pretty straightforward.

As a final note, I have made the sample application available to download.

I'm the Grails Advocate at SpringSource (a division of VMware) and one of the core committers of the project. I have been using Grails since 2006 and have contributed several plugins on top of the work I have done on the core. These days I write about the framework and guide people through basic and advanced usage.

Comments

Complexity

Grails was created to ease the Java web development, but it looks like deployment to cluster was not simplified. Sounds quite complex to me...

I just found this blog and

I just found this blog and have high hopes for it to continue. Keep up the great work, its hard to find good ones. rebelmouse

It is very good, but look at

It is very good, but look at the information at this address. escort curico

At this point you'll find out

At this point you'll find out what is important, it all gives a url to the appealing page: escort antofagasta

I personally use them

I personally use them exclusively high-quality elements : you will notice these folks during: escort santiago

Another fascinating blog post

Another fascinating blog post submitted here by you. You've been giving us some really amazing stuff over the past few months and that's good to see. I hope you'll keep giving us more stuff like this. Printer Drum

This is the most brilliant

This is the most brilliant article that I've read in a very long time. You've done a really good job here. I just wish you can continue to do more jobs like this. Cheers, mate! how to save power bills

Once again, you've shown here

Once again, you've shown here why you are one of the best writers at the moment. This is just a brilliant article. Thanks a lot for publishing this today. Keep it going, mate. débarras paris

If more people that write

If more people that write articles really concerned themselves with writing great content like you. more readers would be interested in their writings. Thank you for caring about your content.
english translations service

I am definitely enjoying your

I am definitely enjoying your website. You definitely have some great insight and great stories.
tattoo salon

It’s appropriate time to make

It’s appropriate time to make some plans for the future and it is time to be happy. I have read this post and if I could I wish to suggest you few interesting things or advice. Perhaps you could write next articles referring to this article. I desire to read even more things about it!
WordPress themes

I am unable to read articles

I am unable to read articles online very often, but I’m glad I did today. This is very well written and your points are well-expressed. Please, don’t ever stop writing.
tree service omaha

You have a real ability for

You have a real ability for writing unique content. I like how you think and the way you represent your views in this article. I agree with your way of thinking. Thank you for sharing.
john overdurf

Exactly, you're very kind of

Exactly, you're very kind of us about comment!.
San Jose Movers

Regardless of type of area or

Regardless of type of area or practice, research is the key in law. Law students must keep themselves busy in knowing about cases and judgements so as to enhance their presenting skills and gain experience. Research or studies are all what takes a law student to a level where he/she can be declared or awarded with a degree of lawyer.
top lawyers in India

Super-Duper site! I am Loving

Super-Duper site! I am Loving it!! Will come back again, Im taking your feed also, Thanks.
Holiday villa Phuket

I have recently started a

I have recently started a blog, the info you provide on this site has helped me greatly. Thanks for all of your time & work.
Phuket villa rental

As far back as Dimebag was

As far back as Dimebag was shaking a Dean ML. The main thing that he changed on those guitars were the pickups. As we have specified some time recently, he was exploring different avenues regarding different humbuckers while pursuing the ideal tone. His fundamental setup came down to a Dean ML, the one he won from the challenge, fitted with a Bill Lawrence XL500 at the extension, and a Seymour Duncan '59 at the neck. He utilized that guitar as his essential until the end.
Dimebag Darrell Guitar Setup

i am for the first time here.

i am for the first time here. I found this board and I in finding It truly helpful & it helped me out a lot. I hope to present something back and help others such as you helped me.
Psicóloga em Sao Paulo

This is a great inspiring

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post.
Trickprime

I am very much pleased with

I am very much pleased with the contents you have mentioned. I wanted to thank you for this great article.
how to get rid of piles

Nice post! This is a very

Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post.
gout uric acid

I just want to let you know

I just want to let you know that I just check out your site and I find it very interesting and informative..
plus size swimwear

This is a great post. I like

This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again.
dry cough bronchitis

Nice post! This is a very

Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post.
obat kuat tradisional

This is a smart blog. I mean

This is a smart blog. I mean it. You have so much knowledge about this issue, and so much passion. You also know how to make people rally behind it, obviously from the responses.
uric acid levels

This is a great post. I like

This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again.
Laguna Beach Subzero refrigerator repair

Excellent article. Very

Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking.
best guitars in india

This is highly information,

This is highly information, crisp and clear. I think that everything has been described in systematic manner so that reader could get maximum information and learn many things.
AXIS Q1614-E IP Camera Dubai

Thank you very much for this

Thank you very much for this useful article. I like it.
phenq reviews 2017

thanks this is good

thanks this is good blog.
cupom de desconto

This content is written very

This content is written very well. Your use of formatting when making your points makes your observations very clear and easy to understand. Thank you.
shopping

Superbly written article, if

Superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place..
phenq reviews

This is such a great resource

This is such a great resource that you are providing and you give it away for free. I love seeing websites that understand the value of providing a quality resource for free. It is the old what goes around comes around routine.
products

You have done a great job on

You have done a great job on this article. It’s very readable and highly intelligent. You have even managed to make it understandable and easy to read. You have some real writing talent. Thank you.
day tours Zurich

This is my first time i visit

This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you.
Mona Media

thank you for a great

Thank you again for all the knowledge you distribute,Good post. I was very interested in the article, it's quite inspiring I should admit. I like visiting you site since I always come across interesting articles like this one.Great Job, I greatly appreciate that.Do Keep sharing! Regards,
encanador

Thank you for taking the time

Thank you for taking the time to publish this information very useful!
guest post

Love what you're doing here

Love what you're doing here guys, keep it up!..
learn here

Thank you very much for the

Thank you very much for the sharing! COOL..
serum tri mun serumkieu

Exactly, you're very kind of

Exactly, you're very kind of us about comment!.
Real Estate News Gateway

DANI

hello!! Very interesting discussion glad that I came across such informative post. Keep up the good work friend. Glad to be part of your net community. temp mail

This article gives the light

This article gives the light in which we can observe the reality. This is very nice one and gives in depth information. Thanks for this nice article.
spy app

Great job for publishing such

Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too. There tend to be not many people who can certainly write not so simple posts that artistically. Continue the nice writing
Mondial Assistance

I got what you mean , thanks

I got what you mean , thanks for posting .Woh I am happy to find this website through google.
Baseball Drills

I just found this blog and

I just found this blog and have high hopes for it to continue. Keep up the great work, its hard to find good ones. I have added to my favorites. Thank You.
West Virginia Mesothelioma Law Firm

dani

It is a great website.. The Design looks very good.. Keep working like that!.
new futura 5 bedroom

Thanks for sharing this

Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post with people..
new futura location

Cool stuff you have got and

Cool stuff you have got and you keep update all of us.
new futura sale

Nice post! This is a very

Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post.
SEO Chauncey Riddle

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.