We're probably like a lot of folks, though, and we have to have an Apache server somewhere to support the PHP applications we have (bad idea, long story). Since we already have an Apache server, we also use it to serve our static resources, just like everyone's been doing for years. We started with mod_jk because that was the only viable option at the time. When mod_proxy got AJP support, we switched to using Apache's mod_proxy and saw a nice performance boost. We stayed with this configuration for several years just because it's not terribly complicated and it just works.
But fronting Tomcat servers with Apache is pretty limiting when you start talking private cloud architectures. I discovered some big problems when I first fired up our SpringSource tcServer instances fronted by a couple of DNS round-robin load-balanced Apache servers. Tomcat was getting confused because requests were coming through different front-end proxies. I wrote my own session manager, which I discussed recently in two blog posts (<a data-cke-saved-href=" http:="" www.tomcatexpert.com="" blog="" 2010="" 06="" 21="" clustering-cloud-friendly-tomcat-sessions-rabbitmq-part-i"="">Part I & Part II), to get around this problem. But I still had the limitation of forcing users to go to an Apache server before being able to access the dynamic resources located in my tcServer backends. This created issues when I needed to take down the server Apache was running on.
It's pretty unrealistic to think we can ever really achieve a high-availability solution that has zero single points of failure, especially if you're building your own cloud in your own data center. There's always going to be a DNS server or a proxy server or a switch that is the lynchpin of the enterprise. But it seemed to make more sense to use a more robust proxy solution as our web application entry point so we could point our users at the proxy instead of at an actual Apache server. This lets us take servers down and bring new ones up to replace them without affecting overall availability. Since I'm trying to do everything in at least twos, having a second HAProxy server with identical configuration gives me a little bit of failover.
HAProxy is a lightweight proxy server that advertises obscenely high throughput. Judging from the places it's being used, and the amount of TCP/IP traffic being passed through it, relying on it to handle all our production traffic didn't seem to be much of a stretch. Not only will HAProxy handle HTTP traffic, but it's a general-purpose TCP/IP proxy. We also run our RabbitMQ traffic through it to get failover and load-balancing for our messaging infrastructure. I'm even experimenting with shoving our PostgreSQL database traffic through it. This reusability appeals to my "the only uni-tasker you need is a fire extinguisher" sensibilities I talked about in a previous article.
I see my HAProxy server not as another type of application server but as a router. If I want to send web traffic bound for server A to server B while I update the packages on server A, I can tweak my HAProxy configuration and ask it to reload this configuration (and this is the deal-breaker for me not using Apache) without having to break existing connections.
Apache configuration changes result in killing hundreds of active connections. This is a killer to service availability. A human accessing your web application while Apache is restarted might experience random errors but at least they can hit "Reload" in the browser or click the link again if it doesn't do what they expect. When your services are exposed to non-human users, interrupting those requests can mean headaches for everyone involved. We have an AS/400 in our department and we do some pretty low-level integration between RPG and Java code that accesses web services provided through our tcServer application servers. RPG is not a flexible language that can handle bad input data or, quite frankly, anything other than what it expects. Service interruptions were cascading errors through this application chain which creates issues for a couple programmers and probably someone downstairs in Accounts Payable.
If one of the primary assumptions about your environment is that services like Apache or Tomcat can come up or go down at any time, then you simply have to put something in front of those services that can handle routing the traffic correctly, based on the cloud topology at the moment.
Although we're not using sticky sessions for reasons I've previously explained, you can use HAProxy to route traffic from a particular user back to the server they were previously on. The configuration is similar to Apache mod_proxy's in that you define a text string that HAProxy will prefix to the cookie value to determine where to send that request. This works great if you now want to do DNS load-balancing and have your users hitting several different front-end servers instead of coming up with subdomains and unique server names and the like.
To configure HAProxy to do sticky sessions, you need to set up a "backend". In HAProxy, a backend is a set of servers you want to route traffic to. This would be equivalent to mod_proxy's <Proxy> directive. I have two backends configured: one for Apache and one for tcServer/Tomcat. The Apache backend is quite simple, but here's the important parts:
backend apache mode http ... option httpclose option redispatch server www1 172.23.12.24:80 check inter 2000 server www2 172.23.12.25:80 check inter 2000 server www3 172.23.12.26:80 check inter 200
I've omitted some of the configuration directives that relate to checking the status of the Apache servers. That's covered in the excruciatingly-detailed README.
This configuration tells HAProxy that this backend is an HTTP backend. The "httpclose" and "redispatch" options are critical when proxying HTTP servers, though I honestly couldn't tell you exactly why because the explanation given in the README was a little over my head. Suffice it to say this won't work without them.
The "server" directives tell HAProxy which backend servers to load-balance. You can configure each backend differently, but I've chosen to configure "roundrobin" load-balancing globally. You can also use a weighted balancing policy. In a dynamic environment, though, I think you'd want to generate this configuration file, with load balancer weights calculated from the specifics of the machine joining the cloud. That's way beyond the scope of this article. For most private clouds, where backend servers use known IP addresses, it would suffice to configure all of the potential backend servers. Any that are not up won't be part of the load-balancing equation, so there's no negative impact to leaving servers that are currently down configured in HAProxy. When they come back up, HAProxy will see that and start sending them traffic again. All with no restarts.
The tcServer backend configuration is also pretty simple, though this example diverges from my real configuration in that I've added the necessary cookie settings to make sticky sessions work. I have these commented out in my HAProxy configuration so I can have a backup plan in case something goes horribly wrong with RabbitMQ or I introduce a bug into my session manager. Reverting back to a sticky session configuration to restore service is as simple as uncommenting the right directives and reloading HAProxy.
backend tcserver mode http ... option httpclose option redispatch cookie JSESSIONID prefix server vm33-tc1 172.23.13.33:8180 cookie vm33tc1 check inter 5000 server vm33-tc2 172.23.13.33:8280 cookie vm33tc2 check inter 5000 server vm33-tc3 172.23.13.33:8380 cookie vm33tc3 check inter 5000
You'll notice we've added some references to the session cookie. In Java web applications, this should be JSESSIONID. HAProxy will do what mod_proxy does to manage sticky sessions: it will strip out part of the JSESSIONID cookie value and match it against the configured backends. If you use a tool like Firebug and look at the cookies coming back from Tomcat with this HAProxy configuration in place, you'll see something like "vm33tc1~REALLYLONGMD5HASH" for the JSESSIONID cookie value. If you don't want to use sticky sessions, delete the references to "cookie" configuration. No other changes are required.
In one of my earlier articles, I discussed hooking into JMX to monitor server lifecycle events. If you had a lifecycle listener on that queue running on the same VM as HAProxy, it could generate this list of servers and issue a soft reload. Otherwise, you can configure every server you might load balance with, disregarding their current up or down status. When they come up at some point in the future, HAProxy will see them and start load balancing against them.
We're not quite finished. We haven't yet told HAProxy what traffic to send to Apache and what traffic to send to tcServer. In mod_proxy, you'd set a ProxyPass and ProxyPassReverse directive on the path to your webapp. In HAProxy, you do something similar, but HAProxy uses "acls" (Access Control Lists).
To tell HAProxy to listen on a particular port, you need to set up a "frontend". HAproxy is a very powerful proxy server, so setting up an HTTP frontend can get pretty complicated, depending on how you're wanting to route traffic. HAProxy knows a lot about HTTP requests because it treats them differently than other TCP/IP traffic. HAProxy actually interrogates the HTTP requests and responses and can do all kinds of things to them. The full options for this are in the README, which I strongly encourage you to read.
A basic frontend for splitting up HTTP traffic bound for a tcServer application on context path "/rest" and letting all other HTTP traffic go to Apache would need at least something like:
frontend www mode http bind :80 option forwardfor capture request header ... capture response header ... acl rest path_beg /rest use_backend apache unless rest default_backend tcserver There are a number of headers that need to be sent to the backend servers so Tomcat sees the client's IP address and Cache-Control headers get passed back correctly. Since what you might actually need depends on your particular setup, it's probably best you set this part up yourself. Just be aware that until you get these "capture" directives set up, your Tomcat server will think the HAProxy box is requesting the page, not the client on the other end of the proxy.
The acl configuration is pretty self-explanatory. If the request path doesn't begin with the string "/rest", send it to the Apache backend, otherwise let the tcServer backend handle it.
HAProxy includes a built-in HTML report on what's going on inside your proxy server. To access it, you'll need to set up a special "listen" section:
listen stats :7583 mode http stats uri /
If you pull up http://haproxy1.mycloud.mycompany.com:7583/ you'll see gobs of statistics and really low-level information on what's happening. Securing this URL is covered in the fore-mentioned HAProxy README.
Fronting Tomcat with Apache is a stable and traditional configuration. It's also pretty static. If you're trying to jump on the cloud computing bandwagon and your environment just got very dynamic, or you simply want more flexibility in how you serve Tomcat-based content, consider replacing Apache and mod_proxy with the more robust HAProxy.