In this article we are going to explore one of the modules that is being developed in the Tomcat development trunk. The module named, jdbc-pool, is exactly that, an implementation of pooling of JDBC resources.
Connection pools have been around for a long time. I wrote one myself in 1997 when I first was introduced to Java, and used that for a long time in different projects I worked on. When JDBC came up, developers quickly realized that pooling JDBC connections would be beneficial for performance, and also a way to siphon connections and control the resource usage on the database server. This was especially important in horizontally scaled clusters where the database was always a risk of being overloaded. In those days we typically ran 1, 2 and sometimes 4 CPU machines for the application servers.
Connection pooling quickly became something that J2EE vendors developed in-house to integrate with their transaction services and became proprietary and integrated into the server itself. Outside of the J2EE containers a number of open source projects popped up. A few examples are Proxool, Apache Commons DBCP, C3P0 and Poolman. These are the most common ones we encounter in the field. Apache Tomcat has for very many years relied on DBCP to provide Tomcat users with a connection pooling implementation.
All of these open source projects have their pros and cons, but they all have something in common. They have become fairly stagnant and the code bases have not continued to evolve to meet new features in the Java language and are not optimized for multicore hardware that is increasingly becoming the common platform to deploy Java applications on. Some of the projects have also suffered from unneeded complexity in the source code, most likely one of the main reasons the project code is not moving forward, making it hard to maintain and easy to break.
Out of the four open source projects mentioned above, the two we see in Apache Tomcat deployments today are Commons DBCP and C3P0. A short summary of these two projects would be:
Both of these implementations are fairly complex. DBCP is built on top of the Apache Commons Pool project, and all together forms a code base of over 60 source files. C3P0 has grown to over 200 source files. Yet both pools fall short on features, to mention a few:
So back to our question, why yet another pool project? The answer was simple, we needed better performance and more features. We also believed that taking on one of the legacy implementations would require rewriting a majority of the code base. So, it came to make a judgement call, do we want to rewrite 60+ sources files, or even 200+.
I know what you're probably thinking, here is yet another engineer that believes in rewriting from scratch rather than contributing to the existing code base. In almost every case, I prefer to reuse and fix than rewrite. However, after carefully judging the following questions:
The next challenge we faced with such a project, within the Apache Software Foundation, was how to build a community around the project. Community is often grown by having lots of code that is not quite finished, meaning, there is work to be done in multiple areas making it enjoyable for a group to work together.
By this time, we had a prototype, it was 8 source files, and provided all the features of the other pools combined. Not all that exciting for a group of Java developers. Trying to fire off a separate project would probably not be very fruitful from a community perspective. We've since then grown it to 24 source files mostly by creating prebuilt addons that can be configured.
And so the jdbc-pool module was started.
The jdbc-pool did get some early adoption and the number of bugs was very low. Most early bugs were actual workarounds for typical use of the java.sql interfaces, calling close multiple times on a connection etc.
Today we see jdbc-pool, even in its unreleased state, several cases of production use. SpringSource ships the pool module with its tc Server product and you can already read about it in blog posts.
The jdbc-pool has an almost identical configuration as the Commons DBCP pool, in order to provide an easy migration path. For users already using Apache Tomcat and DBCP, the only difference would be the factory attribute factory="org.apache.tomcat.jdbc.pool.DataSourceFactory".
For users of dependency injection frameworks, like the Spring Framework it's easy to just configure the org.apache.tomcat.jdbc.pool.DataSource bean.
Folks, that is why we have yet another pool module for JDBC connections. Over the next few articles, I will be digging deeper into: