Juju Java Cluster – Part 1

In my previous post I gave an introduction to Juju, the new deployment tool in Ubuntu 12.04 Precise Pangolin.  This post is the first of four demonstrating how you can deploy a typical Java web application into your own Juju cluster.  I’ll start the series by deploying an initial cluster of HAProxy, Tomcat and MySQL to Amazon EC2, shown in the diagram below.  You can always deploy to a different environment than EC2 such as MAAS or locally using LXC.  The Juju commands are equivalent.

Java web application cluster

For this demo I’ll build a sample application using the excellent Grails framework (grails.org).  You can of course use traditional tools of Maven, Ant, etc. to produce your final WAR file.  If you want to try the demo yourself, you’ll need to install Grails and Bazaar.

Firstly let’s demonstrate how to deploy Tomcat using Juju (jujucharms.com/~robert-ayres/precise/tomcat).

Open a terminal on any Ubuntu Precise machine and follow the instructions for bootstrapping a Juju cluster – juju.ubuntu.com/docs/getting-started.html.

With a bootstrapped cluster, let’s deploy a Tomcat service:

juju deploy cs:~robert-ayres/precise/tomcat

This will deploy a Tomcat unit under the service name ‘tomcat’.  Like the bootstrap instance, it will take a short time to launch a new instance, install Tomcat, configure defaults and start.  You can check the progress with ‘juju status’.  When deployed you should see the following output (‘machines’ information purposely removed):

services:
  tomcat:
    charm: cs:~robert-ayres/precise/tomcat-1
    relations:
      cluster:
      - tomcat
    units:
      tomcat/0:
        agent-state: started
        machine: 1
        public-address: xxx.compute.amazonaws.com

Should you wish to investigate the details of any unit you can ssh in – ‘ssh ubuntu@xxx.compute.amazonaws.com’ (Juju will have transferred your public key).

The Tomcat manager applications are installed and secured by default, requiring an admin password to be set.  We can apply configuration to Juju services using ‘juju set <service> “<key>=<value>” …’.  To set the ‘admin’ user password on our Tomcat unit:

juju set tomcat "admin_password=<password>"

Our Tomcat unit isn’t initially exposed to the Internet, we can only access it over a ssh tunnel (see ssh ‘-L’ option).  To expose our Tomcat unit to the Internet:

juju expose tomcat

Now you should be able to open your web browser at http://xxx.computer.amazonaws.com:8080/manager and login to Tomcat’s manager using the credentials we just set.
If we prefer our unit to run on a more traditional web port:

juju set tomcat http_port=80

After a small time of configuration you should now be able to access http://xxx.computer.amazonaws.com/manager with the same credentials.
Over HTTP, our credentials aren’t transmitted securely, so let’s enable HTTPS:

juju set tomcat https_enabled=True https_port=443 [1]

Our Tomcat unit will listen for HTTPS connections on the traditional 443 port using a generated self-signed certificate (to use CA signed certificates, see the Tomcat charm README).  Now we can securely access our manager application at https://xxx.computer.amazonaws.com/manager (you need to ignore any browser warning about a self-signed certificate).  We now have a deployed Tomcat optimised and secured for production use!

Now let’s turn our attention to evolving a simple Grails application to demonstrate further Juju abilities.

With a working Grails installation, create ‘juju-example’ application:

grails create-app juju-example

This will create your application in a directory ‘juju-example’.  Inside is a shell of a Grails application, enough for demonstration purposes.

To suit the directory layout of our deployed Tomcat, we should adjust our application to store stacktrace logs in a designated, writable directory.  Edit ‘juju-example/grails-app/conf/Config.groovy’ and inside the ‘log4j’ block add the following ‘appenders’ block:

log4j = {
    ...
    appenders {
        rollingFile name: "stacktrace", maxFileSize: 1024,
                    file: "logs/juju-example-stacktrace.log"
    }
    ...
}

To build a WAR file run:

(within 'juju-example' directory)
grails dev war

This will build a deployable WAR file ‘juju-example/target/juju-example-0.1.war’.

You have secure access to deploy WAR files directly using the Tomcat manager, but there is a better way – using the J2EE Deployer charm.

The J2EE Deployer charm is a subordinate charm that essentially provides a Juju controlled wrapper around deploying your WAR file into a Juju cluster.  This has the distinct advantage of allowing you to upgrade multiple units using a single command as is shown later.  To use the J2EE Deployer, first download a copy of the wrapper for our example application using bzr:

mkdir precise
bzr export precise/j2ee-deployer lp:~robert-ayres/charms/precise/j2ee-deployer/trunk

This will create a local copy of the wrapper under a directory ‘precise/j2ee-deployer’.  The ‘precise’ parent directory is necessary for Juju when using locally deployed charms.
Copy our war file to the ‘deploy’ directory within:

cp juju-example/target/juju-example-0.1.war precise/j2ee-deployer/deploy

Now deploy our application into Juju:

juju deploy --repository . local:j2ee-deployer juju-example

As with other charms, this will securely upload our application into S3 storage for use by any of our Juju services.  Once the deploy command returns, our application should be available within the cluster under the service name ‘juju-example’.  To deploy to Tomcat, we relate the services:

juju add-relation tomcat juju-example

Our Tomcat unit will download our application locally, stop Tomcat, deploy the application and then start Tomcat.
Issue ‘juju status’ commands to check progress.  Once deployment is complete, you can access http://xxx.computer.amazonaws.com/juju-example-0.1/ and see the default Grails welcome page (screenshot below).

juju-example application

We can use ‘juju set’ to change configuration of our application as we did with the Tomcat service.  For example, to change the deployed path to something simpler:

juju set juju-example war-config=juju-example-0.1:/

Our application will now be redeployed and Tomcat restarted so we can access our application at http://xxx.computer.amazonaws.com/.  Now we have a deployed application!

A web application typically requires access to a RDBMS, so let’s demonstrate how we can connect our application to MySQL.
Firstly, deploy a MySQL service:

juju deploy mysql

Whilst this is deploying, we can set the configuration of the imminent relation between Tomcat and MySQL:

juju set tomcat "jndi_db_config=jdbc/JujuDB:mysql:juju:initialSize=20;maxActive=20;maxIdle=20"

This is a colon separated value that maps the requested database ‘juju’ of the ‘mysql’ service under a JNDI name of ‘jdbc/JujuDB’.  The set of values after the final colon set DBCP connection pooling options.  Here we specify a dedicated pool of 20 connections.
Once our MySQL unit is deployed, we relate our Tomcat service:

juju add-relation tomcat mysql

During this process, our Tomcat unit will request the use of database juju.  Our MySQL unit will create the database and return a set of generated credentials for Tomcat to use.  Once complete, our pooled datasource connection is available to our Tomcat application under JNDI – ‘java:comp/env/jdbc/JujuDB’.  To demonstrate its use within our application, firstly configure Grails to use JNDI for its datasource connection.  Within ‘juju-example/grails-app/conf/DataSource.groovy’, inside the ‘production’/'dataSource’ block, add ‘jndiName = “java:comp/env/jdbc/JujuDB”‘ so it reads as follows:

production {
    dataSource {
        dbCreate = "update"
        jndiName = "java:comp/env/jdbc/JujuDB"
    }
}

Next create a domain class which will serve as an example database object:

(within 'juju-example' directory)
grails create-domain-class Book

Edit ‘juju-example/grails-app/domain/juju/example/Book.groovy’ so it contains the following:

package juju.example

class Book {

    static constraints = {
    }

    String author
    String isbn
    Integer pages
    Date published
    String title
}

Now we can use Grails ‘scaffolding’ to generate pages that allow us to insert Books into our database:

grails generate-all Book

Recompile our application to produce a new WAR file:

grails clean
grails war
(Note: 'grails war' now, no 'dev' option)

Now upgrade our application in Juju:

# copy across new war file
cp juju-example/target/juju-example-0.1.war precise/j2ee-deployer/deploy
# upgrade Juju deployment
juju upgrade-charm --repository . juju-example

This will upload our revised application into S3 again and then deploy to all related services, restarting them in the process.
With our newly deployed application utilising its local JNDI datasource, we can now open our web browser at http://xxx.compute.amazonaws.com/juju/book/list and use the generated page to perform CRUD operations on our Book objects, all persisted to our MySQL database.

A key point to be made is how you should develop your application to be cloud deployable.  If the application is developed to utilise external resources via runtime lookups, the application may be deployed to any number of Juju clusters.  You can observe this yourself by adding a relation between your application and any other Tomcat services.

For this post’s finale, let’s show how we can scale Tomcat.
First, deploy the HAProxy load balancer:

juju deploy haproxy

And associate with Tomcat:

juju add-relation haproxy tomcat

Unexpose Tomcat and expose HAProxy:

juju unexpose tomcat
juju expose haproxy

We can now use the public address of HAProxy to access our application.
Now we’re behind a load balancer, its simple to bolster our web traffic capacity by adding a further Tomcat unit:

juju add-unit tomcat

A second Tomcat unit will be deployed and configured as the first.  Same open ports, same MySQL connection, same web application.  Once deployed, HAProxy will serve traffic to both instances in round robin fashion.  Any future application upgrades will occur on both Tomcat units.  If we want to remove a unit:

juju remove-unit tomcat/<n>

where ‘<n>’ is the unit number (shown in status output).

That’s the end of the demo.  Should you wish to destroy your cluster, run:

juju destroy-environment

This will terminate all EC2 instances including the bootstrap instance.

To summarise, I’ve shown how you can create a Juju cluster containing a load balanced Tomcat with MySQL, serving your web application.  We’ve seen how important it is for the application to be cloud deployable allowing it to utilise managed relations.  I’ve also demonstrated how you can upgrade your application once deployed.

In my next post I shall write about adding Memcached to our cluster.


[1] Due to a current Juju bug (https://bugs.launchpad.net/juju/+bug/979859) with command line boolean variables, you may need to create a separate ‘config.yaml’ file containing the contents:

tomcat:
 https_enabled=True
 https_port=443

and then use:

juju set --config config.yaml tomcat