TomcatExpert

WebSockets in Tomcat 7

posted by fhanik on April 23, 2012 10:22 PM

With the 7.0.27 release the Apache Tomcat team introduced a WebSocket implementation. WebSocket has received a lot of hype, and has been much anticipated by Tomcat users. Let’s take a quick look at what web sockets are, what benefits and limitations they have and how they are implemented in Apache Tomcat 7.

What is a WebSocket?

WebSocket is considered the next step in evolution of web communication. Over time, communication has evolved in steps to reduce the time and data throughput for the application to update a user’s browser. The evolution has looked a little like this:

  • Entire page reloads
  • Component reloads using AJAX Processing
  • Comet communication
    • Long poll– similar to AJAX, but not holding a thread on the server
    • Bi directional- two way communication over the same TCP

Each of these steps had their benefits and challenges. Apache Tomcat 6 implements bi-directional communication over HTTP using its Comet Processor. This implementation allowed for asynchronous event driven request processing as well as bi-directional communication. This implementation had a few limitations.

  1. Given that HTTP was intended to be request/response protocol rather than a bi directional protocol, proxies and other intermediaries may not work properly, and only forward packets in one direction at any given time.
  2. The programming model becomes more difficult as this introduces multi threading to the servlet developer
  3. Non-standard APIs made adoption difficult

Comet proved that the proverb “build it and they will come” is not necessarily true. Each major servlet container had its own implementation and non-standard APIs. Adoption was fairly sporadic.

The Servlet 3.0 release introduced a new feature called Asynchronous Servlets. This implementation standardized and simplified most Comet implementations by removing the bi-directional requirement of the protocol but retaining the ability to disassociate a container thread from the HTTP request. We can compare this feature to the server version of client AJAX calls. It lets a request hang until a response is ready to be delivered without using worker thread on the container. This feature offers little new over Apache Tomcat’s Comet Processor in terms of functionality. The fact that it is a standard Servlet API, gives frameworks an easier way to build on top of it.

Asynchronous servlets address some of the need in web communication, but not all, and the lack of bi-directional communication made its use still limited. If you take standard API aside, it may have been considered step back from most Comet implementations.

WebSockets is another attempt to standardize an idea of having asynchronous, event driven and bi-directional communication over HTTP. The standardization comes in form of a JavaScript API as well as a communication protocol. While this highlights the lack of a standard server-side API the Servlet 3.1 expert group are already discussing including some basic level of support for this and discussions are active as we are writing this article. Whether that will evolve into a full WebSocket API remains to be seen. In the meantime major servlet container are publishing non- standard APIs, Tomcat being no different.

So let’s summarize what a WebSocket implementation gives us:

  1. Bi-directional communication over a HTTP port by upgrading/switching protocols. Initiated by a HTTP request.
  2. Message/Frame based communication
  3. Intended to work with proxies and intermediaries

What does it look Like?

On the protocol level there is one major difference from what has been done previously, and that is that the WebSocket switches the protocol away from HTTP. This is where a proxy can change its behavior and support this type of messaging as well. Whether this is in theory only still remains to be seen. If we look at a WebSocket client initiating a request we can see that the intermediary has to, at a minimum,

GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade

Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13

understand the “Connection:upgrade” headers. The response contains the critical status code of 101

HTTP/1.1 101
Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

indicating that a protocol switch has been confirmed and is now expected. After this handshake is complete the client and server move away from request/response communication and can send messages independently of each other.

Does the WebSocket protocol address all the limitations of previous Comet implementations?

In short: not completely. First, proxies and intermediaries may still not work – so deployments using Wide Area Networks (WANs) are questionable. Secondly, there are still no standard Java APIs. However, given the buzz around WebSocket, this may be addressed soon.

Another thing to be careful for is that every HTTP/TCP connection requires the initial handshake and introduces a message roundtrip before you even send or receive data over the WebSocket protocol. If you’re not careful, and have very short interactions, WebSocket may be too much overhead. WebSocket are ideal for longer running conversations between client and server.

The good news is that WebSocket has benefited from a lot more attention, especially client side and cross language than any other previous attempt at enhancing web communication. It has also received a competitor for bi-directional communication, and that is SPDY, an implementation also underway within the Apache Tomcat project.

Summary

WebSocket is a straightforward implementation of bi-directional communication over HTTP. The API is part of the HTML 5 specification and so far many different server side implementations are available. It is not without risks, since the Java APIs are yet to be standardized so implementations can vary. Additionally, working with proxies and intermediaries may still not work, however it is important to note that WebSocket utilizes the Upgrade feature of HTTP to switch protocols after the initial request/response handshake has been completed. This gives future proxies an option to reject or support the WebSocket protocol over WAN.

The first released version of WebSocket in Apache Tomcat was released with Apache Tomcat 7.0.27.

Filip Hanik is a Senior Software Engineer for the SpringSource Division of VMware, Inc. (NYSE: VMW) and a key participant in the company's Apache Tomcat initiatives. Filip brings 15 years of extensive experience in architecture, design and development of distributed application frameworks and containers and is recognized for his top-quality system development skills and continuous participation of Open Source development projects. Filip is an Apache Software Foundation member and a committer to the Apache Tomcat project where he is a leading authority on Tomcat clustering and a key contributor to the core of the platform. Filip has made contributions to software initiatives for Walmart.com, Sony Music, France Telecom and has held a variety of senior software engineering positions with technology companies in both the United States and Sweden. He received his education at Chalmers University of Technology in Gothenburg, Sweden where he majored in Computer Science and Computer Engineering.

Comments

Tomcat maven plugin for version 7.0.27

Hello Filip,

I wonder where to download a maven tomcat plugin that works with the referred version.

BTW, awesome functionality delivered by the Tomcat team guys.

jD @ http://pragmatikroo.blogspot.com

threading with websockets

Hi,
Thanks for the article. From a quick look at a thread dump a websocket servlet on a NIO connector with an open connection seems to occupy an executor thread for the life of the connection?
Seems like an obvious scalability problem?
Also the connections seem to be closed automatically on some sort of timeout, do you know what the configuration value for that is?

The stack of the occupied thread is from a thread dump it (note its an exec thread):

"http-nio-8443-exec-14" daemon prio=5 tid=7f9b571ec000 nid=0x119dab000 waiting on condition [119daa000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <7b94fd1b0> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:253)
at org.apache.tomcat.util.net.NioEndpoint$KeyAttachment.awaitLatch(NioEndpoint.java:1519)
at org.apache.tomcat.util.net.NioEndpoint$KeyAttachment.awaitReadLatch(NioEndpoint.java:1521)
at org.apache.tomcat.util.net.NioBlockingSelector.read(NioBlockingSelector.java:174)
at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:246)
at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:227)
at org.apache.coyote.http11.upgrade.UpgradeNioProcessor.readSocket(UpgradeNioProcessor.java:137)
at org.apache.coyote.http11.upgrade.UpgradeNioProcessor.read(UpgradeNioProcessor.java:110)
at org.apache.catalina.websocket.WsFrame.nextFrame(WsFrame.java:213)
at org.apache.catalina.websocket.WsInputStream.nextFrame(WsInputStream.java:68)
at org.apache.catalina.websocket.StreamInbound.onData(StreamInbound.java:112)
at org.apache.coyote.http11.upgrade.UpgradeProcessor.upgradeDispatch(UpgradeProcessor.java:83)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:563)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1620)
- locked <7b8111018> (a org.apache.tomcat.util.net.SecureNioChannel)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

blocked threads

I see a similar stack with websockets holding threads open on 7.0.30 and 7.0.39. This could be a real problem

Name: http-bio-4415-exec-5
State: RUNNABLE
Total blocked: 50 Total waited: 112

Stack trace:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:161)
java.net.SocketInputStream.read(SocketInputStream.java:132)
org.apache.coyote.http11.upgrade.UpgradeBioProcessor.read(UpgradeBioProcessor.java:81)
org.apache.catalina.websocket.WsFrame.nextFrame(WsFrame.java:214)
org.apache.catalina.websocket.WsInputStream.nextFrame(WsInputStream.java:68)
org.apache.catalina.websocket.StreamInbound.onData(StreamInbound.java:149)
org.apache.coyote.http11.upgrade.UpgradeProcessor.upgradeDispatch(UpgradeProcessor.java:83)
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:587)
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
- locked org.apache.tomcat.util.net.SocketWrapper@ddb0d168
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:626)
java.lang.Thread.run(Thread.java:780)

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.