Scalable, Cloud-friendly Tomcat Sessions with RabbitMQ: Part II

posted by jbrisbin on July 1, 2010 06:12 AM

In my first article on taking advantage of RabbitMQ's asynchronous messaging to implement a cloud-friendly session manager, I covered the method behind the madness and introduced you to some of the functions a session manager has to perform in the course of doing its job. In this, the second of two articles covering this topic, I'll describe how loading and updating session objects is handled and, in the interest of fairness, give some disclaimers and acknowledge the trade-offs made to get here.

The Code (at 1,000 feet) cont'd

Whenever code wants to interact with a user session, whether that be to load it into its own memory or to replicate attributes on that session, it sends these messages to a queue whose name contains the session ID. The pattern used to create the queue name is configurable so you can partition user sessions by application, or even by something more fine-grained (without resorting to using multiple, non-clustered RabbitMQ servers).

The session store first checks its internal Map to see if the requested session happens to be local to this store. If it is, the store simply hands that session back to the manager. If it's not, then the store sends a "load" message to the session's queue. It doesn't just fire off a message, though. It turns out that a user's session can be requested from the store many times during the course of a request. If one were to send a load message every time this happens, then we would see a serious performance degradation. To get around this, before a load message is sent, the store checks to see if it's already trying to load this session. If it is, it simply waits until that process is finished and uses the session being loaded in that other thread.

Whatever is listening on the session queue responds to the requestor with a serialized version of the session object. For the purposes of failover, it would be good to have some replication queue listeners that kept a copy of all the session objects so that load messages don't go unanswered if the original Tomcat server on which the user's session was in memory dies or is stopped. There is some rudimentary support for sessions to be replicated off the server on which they exist whenever they are created or when attributes are set or removed from them. The process used to manage this replication and failover, though, is beyond the scope of this discussion.

The store or replication listener that has a subscriber listening to the session's queue will serialize the session object and send an "update" message back to the requestor. The update message comes back into the main event handler, which doesn't know anything about loading sessions. It simply dumps the message into an UpdateEventHandler, which is responsible for checking to see if anyone is currently waiting on this session to be loaded. If it is, it hands that session off and lets those other threads continue on their merry way.

The First Caveat

It's at this point I have to caveat what I said last time about only having one session that is available to the cloud. There is a mode of operation that allows for leaving session objects on every server that user session has touched. If the user goes back to that server, then their session will still be local to that box and the store will have a lot less work to do. But any time there's a need for replication, there's an exponential increase in headaches to keep things updated. Since we haven't personally needed this to work, we haven't spent a lot of time fleshing out how it should.

The Bleeding Edge Speech

There's a reason your boss doesn't want you to run bleeding edge software. Doing so can be painful. This session manager hasn't seen widespread adoption, hasn't been battle-tested, and is the only child of a proud, but over-worked single father. It's not for the faint-of-heart. That said, hybrid, private clouds are an area of our industry poised for explosive growth in the next couple of years. The more companies that deploy dynamic, private clouds, the more developers will come knocking, looking for ways to solve the complex problems introduced by a highly-virtualized and highly-dynamic cloud environment. Having an 8-CPU box does you no good if you only have one server running multiple threads. In order to make full use of those resources, you need to spread the load across those CPUs by loading it down with lots of little processes. Being able to load-balance across those Tomcat instances without resorting to sticky sessions has been the driving force behind the development and adoption of this session manager.

Nothing is perfect and neither is this particular solution. In order to gain flexibility, we've had to trade-off in-memory caching. In order to get a tailor-made solution, we've had to trade-off stability of the codebase. We don't mind bleeding a little if we're confident we'll be better off in the end. Not everyone will be so bold.

As mentioned in the first article, I have made all of the code for this session manager available on Github under the Apache 2.0 license. Patches are encouraged and welcomed!

Jon Brisbin is an Architect/Analyst/Java Guru at NPC International, the world's largest Pizza Hut franchisee. He's been deploying web applications on Tomcat for over 10 years and currently focuses on cloud computing (virtual, hybrid, and private). He built a private cloud from scratch using VMware ESX, Ubuntu Linux, packing tape, and rusty baling wire. He's done consulting work with industry leaders and Mom-and-Pops alike. Prior to NPC, Jon developed new application frameworks, integrated the AS/400 with UNIX and Windows systems, developed Lotus Domino applications, hacked websites together with Perl CGI and a text editor, and served with US Air Force Intelligence in a very hot and sandy, but undisclosed, location. He lives in the rural Midwest.

He blogs on Web 2.0 (and sundry topics) on his website: http://jbrisbin.com/web2