Clustering Cloud-friendly Apache Tomcat Sessions with RabbitMQ: Part I

posted by jbrisbin on June 21, 2010 07:11 AM

The existing solutions for Tomcat session clustering are viable for moderate clusters and straightforward failover in traditional deployments. Several are well developed with a mature code-base, such as Apache's own multicast clustering. But there's no getting around the fact that today's cloud architectures place new and different expectations on Tomcat that didn't exist several years ago when those solutions were being written. In cloud architectures, where horizontal scalability is king, a dozen or more Tomcat instances is not unusual.

In my hybrid, private cloud, first described in my earlier post on keeping track of the availability and states of services across the cloud, I wanted to fully leverage all my running Tomcat instances on every user request. I didn't want to use sticky sessions as I wanted each request (including AJAX requests on a page) to be routed to a different application server. This is how virtual machines work at a fundamental level. Increased throughput is obtained by parallelizing as much of the work as possible and utilizing more of the available resources. I wanted the same thing for my dynamic pages. Not finding a solution in the wild, I resolved to roll my own session manager. The result is the "vcloud" (or virtual cloud) session manager. I've Apache-licensed the source code and put it on GitHub--mostly to try and elicit free help to make it better. You can find the source code here:

The Dilemma

I dug into the code of the available session managers and found they simply would not do what I was asking them to do. My expectations were (and are) quite high. I was looking for something that was, simultaneously:

  1. Lightweight (sorry Terracotta)
  2. Reliable
  3. Scalable
  4. No single point of failure
  5. Asynchronous (for speed and efficiency)
  6. Easy to configure
  7. Zero (or nearly so) maintenance

I don't have to give the standard session managers a second thought when deploying a web application onto Tomcat. I wanted the same out of any other session manager. I also didn't want user sessions "copied" throughout the network all the time. I use lots of little Tomcat instances that have small heap sizes. If there was any way around it, I didn't want to keep entire copies of every active session in every server. But if I wanted the session loaded on demand, I'd need something screamingly fast. I use a cluster of RabbitMQ servers internally, so it only made sense to leverage that great asynchronous message broker to solve this problem.

Ideally, a user session would always be available in-memory. This is the fastest possible way to serve the user request. But it's inherently limiting. What if a new server comes online? Or, why keep precious resources locked up caching user sessions if that application server never sees that user?

The Trade-Off

Ain't that just life? Always having to trade one good thing for something else that you hope will really be better. Having user sessions available on-demand in any server that needs them is definitely trading off the speed of having a user session in-memory with the flexibility of not knowing (or caring) where that session object comes from. But realistically, it's only trading off the time it takes to handle the bytes that make up the object going from one server to the next. Since all my servers are connected by either a VMware virtual switch (because they're running on the same host) or are directly connected to our core Cisco gigabit switch; and because the only object stored in those user sessions is a Spring Security user object, I felt okay taking that hit on page load time if it meant I could load-balance all my servers, all the time.

The Method Behind the Madness

As a Java programmer, I found myself doing what nearly every Java programmer alive does: I tried to complicate things by adding to the code all kinds of stuff I didn't need. It took me a while to filter that out but I eventually settled on only the components that are actually required: a fast, thread-safe list of session IDs that are currently active in the cloud, and a convention of subscribing and publishing standard messages to queues with names that include the session ID.

The Process (at 30,000 feet)

When a Tomcat server is asked to load a user session, it passes that ID into the CloudStore, which checks the internal list of session IDs. If it finds that ID listed there, it knows it has an active session and checks the internal Map to see if that session is actually local to the application server. If it's not, it publishes a "load" message to the appropriate queue. The thread is then blocked for a configurable amount of time until it's either got the loaded session or it times out and assumes the session is no longer valid (maybe because the server that had it in-memory crashed).

As a somewhat-related aside: the Tomcat server doesn't know what's on the other end of that queue. It could be another Tomcat server that has that session in its internal Map. It could also be a dedicated session replicator written in Ruby and backed by a Redis (or other NoSQL) store. Going the other direction: code running in standalone mode, like a system utility, could load the session object that was created when a user logged into your initial web application, exposing a single user object to any Java code running anywhere in your cloud. Pretty cool, huh? ;)

When the server responsible for that session gets this load message, it serializes the session it has in-memory and sends those bytes back to the sender. The sender's listener gets this "update" message and checks to see if any loaders are waiting on this session. If it finds one, it gives the deserialized session to the loader, who, in turn, passes the session back to Tomcat, who continues to serve the page.

The Code (at 1,000 feet)

We won't have time in just one article to cover the important parts of the CloudStore. In my next article, I'll discuss the "load" and "update" event handlers in detail as well as issue the obligatory caveats and acknowledge the known shortcomings. But to start with, let's take a quick peek at the event dispatcher, which is the backbone of the CloudStore session manager.

There are several different programming models when writing asynchronous applications. There's no one way to tackle the problem, so it often comes down to simply personal taste. I can get my mind around the dispatching model more easily than some of the alternatives, so that's the route I chose. Fundamentally, the dispatcher has a custom EventListener class that runs in a separate thread which is responsible for pulling messages off the queue, interrogating them, and dispatching them to the appropriate handler based on their type. There is a message type for every action the CloudStore performs.

  • A touch message adds that session ID to the master list, which marks it as a valid session throughout the cloud. One of these is emitted whenever a new session is first saved by the session manager.
  • A destroy message not only deletes the session object from the internal Map, it unbinds the queue for that ID. These are emitted by the store when the session has expired or is otherwise invalidated.
  • A setattr message contains incoming changes made to the session while it was on the remote server. These changes have to be replicated to the real session object in the internal Map. One of these is emitted by the session whenever the "setAttribute" method is called with a different object than what the attribute is currently set to.
  • A delattr message is emitted whenever the session's "removeAttribute" method is called. Its handler removes the attribute from the real session object.

Almost without exception, I dispatch EventHandlers into separate threads to do the actual work requested by the incoming message. I do this because I have only one QueueingConsumer per queue. I want as much throughput as possible, so the thread that pulls messages off the queue can very quickly get back to doing what it's supposed to concentrate on: processing messages as quickly as possible. Because I'm using the official RabbitMQ Java client, I have to do this in a separate thread so I can publish messages, bind and unbind queues, and perform other operations that require communicating back to a RabbitMQ server; otherwise, those operations would cause a deadlock if I tried to do them in the EventListener's thread.

The Teaser

The other event message types ("load" and "update") get emitted whenever the session manager attempts to load a session or when the server that has the actual object in memory responds to that load message, respectively. I'll cover those in my next article.

Jon Brisbin is an Architect/Analyst/Java Guru at NPC International, the world's largest Pizza Hut franchisee. He's been deploying web applications on Tomcat for over 10 years and currently focuses on cloud computing (virtual, hybrid, and private). He built a private cloud from scratch using VMware ESX, Ubuntu Linux, packing tape, and rusty baling wire. He's done consulting work with industry leaders and Mom-and-Pops alike. Prior to NPC, Jon developed new application frameworks, integrated the AS/400 with UNIX and Windows systems, developed Lotus Domino applications, hacked websites together with Perl CGI and a text editor, and served with US Air Force Intelligence in a very hot and sandy, but undisclosed, location. He lives in the rural Midwest.

He blogs on Web 2.0 (and sundry topics) on his website:



i was looking for something similar. i was thinking about storing session (that contains only user id) in redis and having absolutely no stateful data on tomcats. this would be more simple than anything else with no need for some MQ

less is more

I would actually prefer to not even store the authentication object in the session, either. But at the moment, I don't see any way around it. The only thing I'm really using session replication for is to keep a user logged in throughout the cloud.

I thought about using Redis early on. I think I'll still use it as part of the system, but in the role of session store replicator for failover when if a Tomcat dies.

I have been checking out a

I have been checking out a few of your stories and i can state pretty good stuff. I will definitely bookmark your blog read more

If you are looking for

If you are looking for Affordable transportation service, then you have reached at the right spot. We are one of the best Affordable transportation providing company he has a good point .

Thanks for your insight for

Thanks for your insight for your fantastic posting. I’m glad I have taken the time to see this Seaside Residences Frasers Centrepoint Homes.

thanks for j sharing with

thanks for j sharing with that awesome articles thanks u a lot
rasoio elettrico | regolabarba | ferro da stiro con caldaia | Friggitrice ad Aria | scopa a vapore | ripetitore wifi | ripetitore wifi amplificatore wifi piastra a vapore | miglior aspirapolvere | skateboard elettrico | passeggino leggero | miglior smartphone | tablet per bambini èqq

Thank you for some other

Thank you for some other informative website. The place else may just I get that kind of information written in such a perfect method? I have a venture that I am simply now running on, and I’ve been at the glance out for such info! coffee beans

I appreciate everything you

I appreciate everything you have added to my knowledge base.. great post . Thanks for share with us

Very interesting blog. What

Very interesting blog. What else could I get that kind of information written in such a perfect approach, I am very glad to read visiting this blog. fernglaskaufen

Hello there. This is an

Hello there. This is an extremely well written article. I will be sure to bookmark it and return to read more of your useful information. Thanks for the post. I’ll certainly comeback. kokoswasser kaufen -


I have similar requirements. My design (not implemented yet) consists of a Terracotta cluster at the core (for replicating light sessions on demand, in a cluster of tomcat-nodes). You are implying that Terracotta might not be lightweight enough for this - do you mind explaining why you think this would be the case?
In respect to this, is it the implementation overhead or actual runtime overhead (memory footprint on light virtualized nodes) you are concerned about?

Alton Brown says the only

Alton Brown says the only uni-tasker you should have in your kitchen is a fire extinguisher. I have a similar philosophy in the data center. RabbitMQ is central to a lot of things we do. It's not just our session store, it's an all-purpose messaging bus for a lot of components. Introducing Terracotta means installing an entire server solely for the purpose of session clustering. Multiple RabbitMQ servers take up far less resources (CPU and memory both) on my VMware hosts than would the same number of Terracotta servers. Honestly, I'm not sure how the memory footprint compares on the session store end.

I'm always searching for

I'm always searching for articles that will help me. There's clearly a great deal to learn about this. I believe you've made good quality points in Features also. Keep working, congrats!

I really enjoyed reading this

I really enjoyed reading this post, big fan. Keep up the good work and please tell me when can you publish more articles or where can I read more on the subject drug test kits.

This is just classic work

This is just classic work from your side. You have a can do attitude and you do your work brilliantly. There is not one writer right now who is better than you. You are the best of this era. You can become the best of all time even save on power bill.

Great info! I recently came

Great info! I recently came across your blog and have been reading along. I thought I would leave my first comment. I don’t know what to say except that I have

One of the problems that I

One of the problems that I used to face while using the previous versions of the Apache Tomcat was delay it had in processing data. Now, after I learnt how to integrate it with RabbitMQ, I see significant improvement in the way it operates. house cleaning services san jose

I think that thanks for the

I think that thanks for the valuable information and insights you have so provided here review of har vokse.


hack para monster legends You possess lifted an essential offspring..Blesss for using..I would want to study better latest transactions from this blog..preserve posting.. chat gratuit coco

This type of message always

This type of message always inspiring and I prefer to read quality content, so happy to find good place to many here in the post, the writing is just great, thanks for the post.

Nice post! This is a very

Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post male extra pill.

Next time I read a blog, I

Next time I read a blog, I hope that it does not fail me just as much as this one. I mean, I know it was my choice to read, but I really thought you would probably have something interesting to talk about. All I hear is a bunch of moaning about something you could fix if you weren't too busy looking for attention.
Mortgage Broker Calgary

I believe one of your

I believe one of your advertisings caused my web browser to resize, you may well want to put that on your blacklist.
Edmonton Mortgage Broker
Life Insurance Vancouver
Life Insurance Calgary

That's so great to hear

That's so great to hear Cloud-friendly Apache Tomcat Sessions with RabbitMQ is released. Now Cloud friendly softwqare and applications are more required a most people are changing to cloud feature. Me too using the same, so it is good I can use Tomcat upper back spine pain also with the cloud.

I have read a few of the

I have read a few of the articles on your website now. and I really like your style of blogging look what I found. I added it to my favorites blog site list and will be checking back soon. Please check out my site as well and let me know what you think .

One essential way in which

One essential way in which technical innovation is affecting perform is by lowering the value of range. In many areas, the geographic submission of perform is modifying considerably Discover More.

Weblink Affiliate may

Weblink Affiliate may extremely integrate the design and also feel of their back-links website with your website so that it looks custom-made her explanation.

This is a great inspiring

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post read more.

Insurance strategy policies

Insurance strategy policies are a little more complicated than an auto insurance useful site. This is because insurance organizations allow you to add (or subtract) oral techniques so that it fits your unique circumstances.

This is my first time i visit

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here! keep up the good work discover this info here.

I’m gone to tell my little

I’m gone to tell my little brother, that he should also visit this blog on regular basis to get updated from most up-to-date information you can look here.

Much obliged such a cool post

Much obliged such a cool post which is truly extremely well written.will be alluding a great deal of companions about this.Keep blogging look what I found.

Another beautifully written

Another beautifully written blog post submitted by you here today. Your posts are becoming better and better. I wish you'll publish your next post on espiar moviles. Keep up the brilliant work Bonuses.

I will definitely dig it and

I will definitely dig it and individually suggest to my buddies. I am assured they will be helped from this website Bonuses

By taking their promotion

By taking their promotion needs on the internet choosing the help an knowledgeable SEO organization, a industry is able to achieve thousands, or even many individuals which they would have not been able to otherwise he has a good point.

Further up the look for motor

Further up the look for motor outcomes. They cannot assurance an surge in revenue or brings, because that factor is determined by your own revenue funnel. It is not the SEO provider's job to make sure that the extra website guests you receive will turn to more brings or revenue read here.

Online companies are

Online companies are increasing daily & SEO is enjoying an important part in providing the guests to the site and all the companies are investing a huge sum in SEO go to website.

I would recommend confirming

I would recommend confirming what you can, but don't get confused with recommendations because it may be the sketchy SEO company providing self compliment discover this info here.

This blog is really great.

This blog is really great. The information here will surely be of some help to me. Thanks! judi mesin slot online.

Many seo companies depend

Many seo companies depend heavily on on the internet look for motor research - even in their web page duplicate - but don't let this alone scare you off discover this info here.

In conclusion, there is no

In conclusion, there is no right or incorrect when selecting instrument pedals. One of the best factors of the background songs business, is that someone is always trying something new and smashing the Normal guidelines. That being said, with these 3 pedals you should be able to take most excellent appears to be with the right amp/guitar mixture discover this info here.

SEO information mill facing

SEO information mill facing great competition in the SEO field. However, they introduce assured SEO solutions to cope with the competition refinansiering.

I've been contemplating

I've been contemplating composing an extremely tantamount post in the course of the last couple of weeks, I'll most likely keep it straightforward and connection to this rather if thats cool

If you dig through the a

If you dig through the a large number of available "sources" you can usually identify the actual resource, and this is the details and method one should use when exercising how to go about it. This is the correct way to understand and apply anything read here.

Great Article it its really

Great Article it its really informative and innovative keep us posted with new updates. its was really valuable. thanks a lot more learn.

Whoever has seen deeply into

Whoever has seen deeply into the world has doubtless divined what wisdom there is in the fact that men are superficial. It is their preservative instinct which teaches them to be flighty, lightsome, and false. Here and there one finds a rolex submariner passionate and exaggerated adoration of "pure forms" in philosophers as well as in artists: it is not to be doubted that whoever has NEED of the cult of the superficial to that extent, has at one time or another made an unlucky dive BENEATH it. Perhaps there is even an order of rank with respect to those burnt children, the born artists who find the enjoyment of life only in trying to FALSIFY its image rolex datejust (as if taking wearisome revenge on it), one might guess to what degree life has disgusted them, by the extent to which they wish to see its image falsified, attenuated, ultrified, and deified,—one might reckon the homines religiosi among the artists, as their HIGHEST rank.

It is the profound, suspicious fear of an incurable pessimism which compels whole centuries breitling to fasten their teeth into a religious interpretation of existence: the fear of the instinct which divines that truth might be attained TOO soon, before man has become strong enough, hard enough, artist enough.... Piety, the "Life in God," regarded in this light, would appear as the most elaborate and ultimate product of the FEAR of truth, as artist-adoration and artist-intoxication in presence of the most logical of all falsifications, as the will to the inversion of truth, to untruth at any price. Perhaps there has hitherto been no more effective means of beautifying man than piety, by means of it man can oyster perpetual air-king become so artful, so superficial, so iridescent, and so good, that his appearance no longer offends.

To love mankind FOR GOD'S SAKE—this has so far been the noblest and remotest sentiment to which mankind has attained. That love to mankind, without any redeeming intention in the background, is only an ADDITIONAL folly and brutishness, that replica burberry handbags the inclination to this love has first to get its proportion, its delicacy, its gram of salt and sprinkling of ambergris from a higher inclination—whoever first perceived and "experienced" this, however his tongue may have stammered as it attempted to express such a delicate matter, let him for all time be holy and respected, as the man who has so far flown highest and gone astray in the finest fashion00007!

Useful information

I am a regular user of this apache server. While using its old version, I felt delay in processing data. I have read this post and understood how to integrate it with RabbitMQ. I came to know that by this integration we can solve this problem. Thanks for this information. wholesale tablets for sale

I love the way you write and

I love the way you write and share your niche! Very interesting and different! Keep it coming Lucas Porter

This program enables you to

This program enables you to become an in-demand voice over artist with ease. ranging from the plain old auditions to your own marketing efforts fifa 17.

PR is about developing

PR is about developing interest and utilizing connections Awesome Games through various programs and marketplaces.

Post new comment

This question is for testing whether you are a human visitor and to prevent automated spam submissions.