TomcatExpert

Deploying Apache Tomcat Applications With Puppet

posted by mpdehaan on April 29, 2010 11:34 AM

I've heard on many occasions that Java applications are deployed 'differently' than other applications, that they are difficult, and they need Java specific management tools. Ultimately when you're managing a datacenter, you need to be able to manage applications regardless of the source language. Using administration tools that only manage one type of application or code base means an admin needs a sledgehammer in his arsenal, as well as a ball peen hammer. One tool should be sufficient and can make everyone's life easier.

The ultimate open source sledgehammer (chainsaw?) in today's datacenter automation world is Puppet. Built to be cross-platform, it works on most Linux and Unix based operating systems (including OS X), and will be taking on Windows support in the near future. It is a model driven solution that requires no coding knowledge to use. The Java community has long known the power of cross-platform tools, and systems administration tools should work the same way. If I want to create a user, do I need to know the differences between user creation on OS X and Linux? Not really, I'd just like to specify the attributes and share the same deployment instructions. Puppet helps you do this.

Puppet allows for centralized management of distributed datacenters, with many options present to account for variance in configurations and roles (queue buzzwords like "heterogenous", "geographic affinity", "cloud", and "facebook"). It allows for the definition of classes of machines ("foo.example.org" is a "database-server"), and mapping what machines belong in what classes. There are mechanisms for assigning variables and using them as conditionals and as templates. There is also a pluggable framework for customizing Puppet to interact with new systems of all kinds (such as LVM). Most importantly, Puppet is written around the concept of "modules", which are units of work that are easily shareable between developers and administrators—both inside an organization and across the internet. It is for this reason that Puppet has spawned one of the richest communities of any open source management framework—it allows admins to work together and share knowledge between workplaces, rather than inventing their own deployment tools when they take up residence in a new place. It also allows for concentrating on strategic business projects because the infrastructure automation code is already written.

In a little while, I'll show how to tell Puppet to manage the configuration of a simple Java deployment. It's been said that deploying Java apps is hard for Linux packages, but Puppet makes it very easy. This is only the tip of the iceberg—you can use the same tool to deploy mailservers and databases as well as appservers. It fits in well whether you have 20 machines or thousands. It's agnostic to cloud vs physical hardware, and plays nicely in all places. The example that follows below is designed to be executed locally, though in a typical deployment, you'll host it on a central server, called a puppetmaster, and then roll your configuration out to the nodes using puppet. Let's get started with an example.

Getting Ready

Our example will assume Ubuntu 9.10 Server. If you're running a different Linux, changes may be required, so you may want to follow along after installing this version in a Virtual Machine such as VMware, Parallels, Xen or KVM. From a base install, we'll install both puppet and the git version control system (since I'm hosting my demo on github), and check out the source in a pre-set location. (For now, don't change the paths or things won't work without modification).

apt-get install puppet git
mkdir /srv
cd /srv
git clone git@github.com:mpdehaan/puppet-tomcat-demo.git
cd puppet-tomcat-demo

Running the Demo

Ubuntu 9.10 comes with a slightly older version of Puppet, but this is ok for our uses. At the time of writing this article the current version is 0.25.X. If you already have Tomcat installed on your machine, we're going to be reconfiguring it, so again, you may want to use a virtualized instance.

Run the following command as root to evaluate the Puppet code and install Tomcat, as well as one WAR inside Tomcat.

puppet --modulepath /srv/puppet-tomcat-demo/modules -v /srv/puppet-tomcat-demo/site.pp

You should see (approximately) the following output:

info: Autoloaded module tomcat
notice: Scope(Class[tomcat]): Establishing http://bojangles:735/
notice: Scope(Tomcat::Deployment[SimpleServlet]): Establishing http://bojangles:735/SimpleServlet/
info: mount[localhost]: Mounted /
info: mount[modules]: Mounted 
info: mount[plugins]: Mounted 
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/File[/var/lib/tomcat6/webapps/SimpleServlet.war]/ensure: created
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/Package[tomcat6]/ensure: created
info: Filebucket[/var/lib/puppet/clientbucket]: Adding /etc/tomcat6/tomcat-users.xml(08bebb9e6afab7920c9ceb6135095d02)
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/tomcat-users.xml]: Filebucketed to  with sum 08bebb9e6afab7920c9ceb6135095d02
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/tomcat-users.xml]/content: content changed '{md5}08bebb9e6afab7920c9ceb6135095d02' to '{md5}e6feb938bbcf04b37d13e53f14d0611d'
info: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/tomcat-users.xml]: Scheduling refresh of Service[tomcat6]
info: Filebucket[/var/lib/puppet/clientbucket]: Adding /etc/tomcat6/server.xml(7d1044d5e1d96437196d127bf2d4cdb0)
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/server.xml]: Filebucketed to  with sum 7d1044d5e1d96437196d127bf2d4cdb0
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/server.xml]/content: content changed '{md5}7d1044d5e1d96437196d127bf2d4cdb0' to '{md5}2613eb765743c5bae041a9c10237e264'
 
info: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/File[/etc/tomcat6/server.xml]: Scheduling refresh of Service[tomcat6]
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/Service[tomcat6]: Triggering 'refresh' from 2 dependencies
notice: //Node[default]/Tomcat::Deployment[SimpleServlet]/tomcat/Package[tomcat6-admin]/ensure: created

That's clearly a lot of output, but we were running it verbosely on purpose (-v) so it was easy to see what you happened. This time, we're just running Puppet locally, though if you were running it centrally, this log information would be collected on the central server. Normally you'd only care about failures.

Visit the two http:// URLs mentioned above and verify that they work. If you click through the first one to the admin page, you'll see we've set the username/password to 'admin/badwolf'—this was done from a Puppet template. The admin console and the SimpleServlet page should both be fully functional. This example just deployed one Servlet, but it would be easy to deploy many more from the same configuration.

Analyzing Our Code

Explore the code tree you've checked out from github. The most important file here is the Puppet module manifest that describes the Tomcat installation rules. This is 'modules/tomcat/manifests/ init.pp'. If you are familiar with Ruby, you will notice the syntax is similar, but this is not Ruby—it is an intentionally simple domain specific language, not a programming language. Not using a programming language is important—we want to construct a predictable model of our data center using basic building blocks—we don't want to simply execute code against that data center. In a centrally managed server context, this manifest is generated into a 'catalog' centrally, and the catalog is distributed out to the managed Puppet nodes, rather than simply sending out the code. This allows for greater security, auditability, and manageability. The configuration language is also not XML, which means your admins won't try to kill you in your sleep for suggesting this to them!

class tomcat {
 
  $tomcat_port = 735
  $tomcat_password = 'badwolf'
 
  notice("Establishing http://$hostname:$tomcat_port/")
 
  Package { # defaults
    ensure => installed,
  }
 
  package { 'tomcat6':
  }
 
  package { 'tomcat6-user':
    require => Package['tomcat6'],
  }
 
  package { 'tomcat6-admin':
    require => Package['tomcat6'],
  }
 
  file { "/etc/tomcat6/tomcat-users.xml":
    owner => 'root',
    require => Package['tomcat6'],
    notify => Service['tomcat6'],
    content => template('tomcat/tomcat-users.xml.erb')
  }
 
  file { '/etc/tomcat6/server.xml':
     owner => 'root',
     require => Package['tomcat6'],
     notify => Service['tomcat6'],
     content => template('tomcat/server.xml.erb'),
  }
 
  service { 'tomcat6':
    ensure => running,
    require => Package['tomcat6'],
  }
 
}

As you can see above, the first part of the manifest file establishes the relationships between the services, packages, and files involved in deploying base tomcat. Both tomcat-users.xml and server.xml are templated out from Puppet, allowing substitution of variables '$tomcat_port' and 'tomcat_password'. Templating is an incredibly useful feature of Puppet, though it can also be used to deploy simple files, as we will use to deploy the WAR files. In your environment, you might choose to copy the WAR files from an NFS share, or they may arrive on the managed machines via other means. For extra bonus points, you could include them in OS packages (like debs) but this is not required. Puppet Labs has also developed a type that allows checking out code directly from version control, if you wanted to go that route. (see github).

There are some clever tricks above. If the template file is ever changed in Puppet for either file, the service will restart automatically. If the user changes the file outside of Puppet, a later Puppet run will fix the file back. Commands are run as needed—Puppet will never try to restart the service until the package is installed. If we were to change the package 'ensure' line from 'installed' to 'latest', we'd also make sure that the Tomcat application we have installed is always the latest version available in the package repository. This is very useful for security updates. Another fact that isn't quite obvious from the above example is that Puppet's concept of resources are at a higher level than operating system resources, in similar ways that the JVM abstracts away details of the OS. While we didn't demo Puppet's "User" class, using "User" in Puppet can manipulate multiple files and directories. These constructs do not correspond merely 1:1 with the OS constructs, but allow for higher level abstraction. In fact, the class above is an example of this abstraction—we use the language of Puppet to model what it means to be a Tomcat server, and can then talk about Tomcat servers instead of the files, packages, and services that make up Tomcat.

Astute readers may question "what happens if this recipe is run more than once?". Puppet in this case performs only the steps that need to change. Unneccessary commands in Puppet are never executed, as it constructs a declarative model of your datacenter configuration. Just as "drive north five hours" is a valid instruction for making it Chicago if you live 5 hours South of Chicago, it's a very useless thing to do if you are already in Chicago, or live in Tokyo (you might drown). Other automation frameworks frequently fail to get this correct so you have to write a lot of if-constructs. Not in Puppet.

define tomcat::deployment($path) {
 
  include tomcat
  notice("Establishing http://$hostname:${tomcat::tomcat_port}/$name/")
 
  file { "/var/lib/tomcat6/webapps/${name}.war":
    owner => 'root',
    source => $path,
  }
 
}

The remainder of the definition sets up a 'defined type'. While a class can only appear once on a given system (they are Singletons), a defined type can appear more than once. In this case, the defined type is simple automation around deploying a WAR file into the 'webapp' directory of Tomcat.

Giving Orders From site.pp

The aforementioned manifest and template files do not by themselves indicate orders to the system. In a server deployment, Puppet would use site.pp (or a web tool such as Puppet Dashboard) to apply clases and definitions to nodes. Similarly, for this example, we're using a very simplistic site.pp that applies the same configuration to all nodes, regardless of hostname.

node default {
 
   tomcat::deployment { "SimpleServlet":
      path => '/srv/puppet-tomcat-demo/java_src/SimpleServlet.war'
   }
 
   # repeat as desired for different servlets ...
 
}

The orders for the node are actually very simple in this case. The managed Puppet node is merely told that it is to be running a given Servlet, and the implementation is abstracted away through the use of the common module. In this way, it should be evident that Puppet is a powerful abstraction layer that allows seperating out the 'how' something is deployed than the 'what' something is to be.

Conclusion

I hope this article has shown how easy deploying on Tomcat with Puppet can be, and how Puppet can abstract away low level details to allow administrators and developers to save time and manage complexity in their deployments. Puppet works very well for this task as it can accomodate the needs of multiple applications, regardless of language, and is built on a very powerful cross platform resource abstraction layer. More importantly, it's built on a community of contributors, so if you are deploying a new application (whether an app on Tomcat, a new message bus, etc), chances are someone has already written a Puppet module you can use.

The next time someone tells you Java apps are deployed differently and are a challenge to automate with the rest of the operating system, show them Puppet.

About Puppet Labs

Puppet Labs provides next-gen IT automation, enabling IT organizations to manage infrastructure as code so they can provide higher service levels with less staff. Puppet Labs releases Puppet as an open source (GPL) application and also offers training, consulting, and support to Puppet users.

Michael DeHaan has developed management applications for IBM, Motorola, and Red Hat's Emerging Technologies Group.   During his tenure at Red Hat, Michael created the Cobbler (http://fedorahosted.org/cobbler)  project and co-authored Func(http://fedorahosted.org/func), which are both frequently used in conjunction with Puppet.  He now serves as the Product Manager for Puppet Labs (http://puppetlabs.com), the creators of Puppet.

 

Comments

Several questions on this example.

I am a very new user of Puppet and am trying to understand your example. What is the purpose of having the tomcat6-user, and tomcat6-admin packages defined? They don't appear to do anything except require the tomcat package itself (which itself doesn't do anything).

In the middle example I notice you are using the $name variable, but I don't find a value being provided for this. What am I missing?

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.