Tomcat Failover Setup with Apache2+modjk

Tomcat works fairly well under heavy loads. For critical services we will need some type of backup/failover mechanism. This tutorial is for those who want to configure apache to failover between two tomcat instances. When the first tomcat service fails, apache-httpd automatically routes the requests to second tomcat.

This tutorial shows how to configure failover load balancing between two tomcat instances in the same server (i have also shown how to do this for tomcat instances in two different servers). For this i am assuming you know how to install multiple tomcat instances in the same machine. This is only slightly different from load balancing between two tomcat instances as you see below in the sample workers.properties file.

#
# workers.properties
#
# Below Entry Specifies the load balance factor when used with a load balancing worker.
# Note: ----> lbfactor must be > 0,  Low lbfactor means less work done by the worker.
# worker.tomcat1.lbfactor=100

# In Unix, we use forward slashes:
ps=/

# list the workers by name

# Workers that are member of lb should not be in the list.
worker.list = failover

# ------------------------
# Tomcat1 Primary 
# ------------------------
worker.tomcat1.port=8009        # You can specify your tomcat instance's ajp port here
worker.tomcat1.host=localhost # Specify any hostname/IP here if tomcat is running in a remote server
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=1
worker.tomcat1.connection_pool_timeout=600
worker.tomcat1.socket_timeout=600
worker.tomcat1.redirect=tomcat2

# ------------------------
# Tomcat2 Failover
# ------------------------
worker.tomcat2.port=9009        # You can specify your tomcat instance's ajp port here
worker.tomcat2.host=localhost # Specify any hostname/IP here if tomcat is running in a remote server
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=1
worker.tomcat2.connection_pool_timeout=600
worker.tomcat2.socket_timeout=600
worker.tomcat2.activation=disabled # Requests are sent to the disabled worker only if first worker fails

# ------------------------
# Failover worker
# ------------------------
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# ----> If a worker dies, the load balancer will check its state
# once in a while. Until then all work is redirected to peer
# worker.
worker.failover.type=lb
worker.failover.balance_workers=tomcat1,tomcat2
worker.failover.sticky_session=False
worker.failover.sticky_session_force=False

# END workers.properties
 

Load Balancing tomcat with Apache and mod jk

Tomcat works fairly under heavy loads. But many a times we would like to distribute the requests to more than one server to minimize response time and there by improving the user experience. This tutorial shows how to configure load balancing (distribute the load) between two tomcat instances in the same system (i have also shown how to do this for tomcat instances in two different servers). For this i am assuming you know how to install multiple tomcat instances in the same machine and also know how to connect apache with tomcat using mod_jk.

All we will do here is to add a new worker to the worker.properties file we already saw in the previous tutorials with Apache httpd 2.x, tomcat 6.x and tomcat-connector 1.2.27.

Below is the content of workers.properties file.

#
#workers.properties
#
# Below Entry Specifies the load balance factor when used with a load balancing worker.
# Note: ----> lbfactor must be > 0, Low lbfactor means less work done by the worker.
# worker.tomcat1.lbfactor=100 

# In Unix, we use forward slashes:
ps=/

# list the workers by name

# Workers that are member of lb should not be in the list.
worker.list = loadbalancer   

# ------------------------
# First worker
# ------------------------
worker.tomcat1.port=8009      # Give the ajp port of the first tomcat instance here
worker.tomcat1.host=localhost # You can mention any other hostname or IP here
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=100    
worker.tomcat1.connection_pool_timeout=600
worker.tomcat1.socket_timeout=600
worker.tomcat1.recovery_options=16

# ------------------------
# Second worker
# ------------------------
worker.tomcat2.port=9009      # Give the ajp port of the first tomcat instance here
worker.tomcat2.host=localhost # You can mention any other hostname or IP here
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=100
worker.tomcat2.connection_pool_timeout=600
worker.tomcat2.socket_timeout=600
worker.tomcat2.recovery_options=16

# ------------------------
# Load Balancer worker
# ------------------------
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# ----> If a worker dies, the load balancer will check its state
#once in a while. Until then all work is redirected to peer
# worker.
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=tomcat1, tomcat2
worker.loadbalancer.sticky_session=False
worker.loadbalancer.sticky_session_force=False

# END workers.properties
 

Note: You can have as many tomcat instances(workers) as you want in the same box as long as your box can withstand the load.

Exporting tomcat keys to Apache httpd

Apache Tomcat SSL keys created with keytool are, by default, in der format. These keys cannot be used in Apache httpd since httpd, be default, expects the keys in pem(X509) format. Using the below steps you can export the tomcat's keys to apache httpd format and use it for apache.

I assume that you already have a working copy of tomcat with SSL. You need a copy of the tomcat's keystore file and the keystore password.

Let us start....

First copy the existing tomcat's keystore file to a new directory so that we don't break anything that is working.

# mkdir /root/SSL_export/
# cp <path/to/tomcat/keystore/file> /root/SSL_export/tomcat.keystore
# cd /root/SSL_export/
# keytool -list -keystore tomcat.keystore
Enter keystore password: 

The above command will print all the keys in the keystore.

Now we will export the key in DER format
# keytool -export -alias tomcat -keystore tomcat.keystore -file exported-der.crt
Enter keystore password: 
Output will be: Certificate stored in file

The certificate will be stored in exported-der.crt

Verify the certificate with this command:
# openssl x509 -noout -text -in exported-der.crt -inform der
Output will be: The whole certificate saying - who issued it  and other info like your company name etc.

Now Convert the key to PEM format so that apache can understand it:
# openssl x509 -out exported-pem.crt -outform pem -in exported-der.crt -inform der

The exported key will be in the file exported-pem.crt.

We have exported the public key and now are going to export the private key.

Download these files:
# wget  http://richfreedman.googlepages.com/ExportPrivateKey.java
# wget http://www.source-code.biz/snippets/java/Base64Coder.java.txt
# mv Base64Coder.java.txt Base64Coder.java
# javac ExportPrivateKey.java Base64Coder.java

Below command will save the exported private key in the file it is redirected to.
# java ExportPrivateKey tomcat.keystore tomcat G10BalPass > exported-pkcs8.key

We have to change it to RSA format so that apache can recognise it.
# openssl pkcs8 -inform PEM -nocrypt -in exported-pkcs8.key -out exported.key

Private key is now exported to the file exported.key.
Download the intermediate certificate from your certificate authority and keep it in the same location.

Now we should configure apache accordingly:
# cd /etc/httpd/conf.d/
# cp ssl.conf ssl.conf.orig
# vi ssl.conf (Change the following line only after commenting the old lines):
SSLCertificateFile /root/SSL_export/exported-pem.crt
SSLCertificateKeyFile /root/SSL_export/exported.key
SSLCertificateChainFile /root/SSL_export/<intermediate.crt>

Restart apache. You must be able to browse https://yourdomain.com/

Apache and Tomcat with modjk using tar

I have already put together a tutorial to connect Apache HTTP with Apache tomcat. In that i had installed httpd with rpm. This tutorial explains the steps to connect apache(http) with tomcat using modjk ;only that this time i am going to explain how to do everything from tar bundles. Here i am not going to explain how to install tomcat. Another tutorial on that is here: http://techsk.blogspot.com/2008/04/installing-tomcat.html.

In the below tutorial i will be installing apache in the default directory (/usr/local/apache2) you can change that location in your server to any other by adding --prefix=<your-install-directory> in the configuration step before calling make. Also i am installing Apache HTTP with SSL support.

When i get some time i will put together the steps for enabling SSL in apache and for using tomcat's SSL keys in Apache.

Here we go:

First download httpd:
# cd /usr/local/src/
# wget http://apache.mirrors.redwire.net/httpd/httpd-2.2.10.tar.gz
# wget http://www.apache.org/dist/httpd/httpd-2.2.10.tar.gz.md5
# md5sum httpd-2.2.10.tar.gz (was equal to the downloaded sum)
# tar zxfv httpd-2.2.10.tar.gz
# cd httpd-2.2.10

Below command will configure (prepare) apache installation with most of the shared modules. Type ./configure --help to see what they are
# ./configure --enable-so --enable-mods-shared=most --enable-ssl    (Hint: --prefix=</path/to/install/directory>)
# make
# make install
# cd /usr/local/src/
# wget http://www.trieuvan.com/apache/tomcat/tomcat-connectors/jk/source/jk-1.2.27/tomcat-connectors-1.2.27-src.tar.gz
# wget http://www.apache.org/dist/tomcat/tomcat-connectors/jk/source/jk-1.2.27/tomcat-connectors-1.2.27-src.tar.gz.md5
# md5sum tomcat-connectors-1.2.27-src.tar.gz (was equal to the downloaded sum).
# tar zxfv tomcat-connectors-1.2.27-src.tar.gz
# cd tomcat-connectors-1.2.27-src/native/
#  ./configure --with-apxs=/usr/local/apache2/bin/apxs
# make
# make install

That is it for the installation. Now we will configure apache for connecting it with tomcat
# cd /usr/local/apache2/conf/
# cp httpd.conf httpd.conf_18Nov08
# vi httpd.conf
Add the following code below the LoadModule section.

# Load mod_jk
# 
LoadModule jk_module modules/mod_jk.so

# Configure mod_jk
#
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info

JkMount *.jsp test   <- This must be the worker name you give in workers.properties. This will at(many) times go into your Virtual Host configuration.

Uncomment the line: Include conf/extra/httpd-ssl.conf
Save and Exit.
# cd /usr/local/apache2/conf/
# vi workers.properties (create new file and paste the below code)

# workers.properties
#

# In Unix, we use forward slashes:
ps=/

# list the workers by name

worker.list=test

# ------------------------
# First tomcat server
# ------------------------
worker.test.port=8009
worker.test.host=localhost
worker.test.type=ajp13

#
# END workers.properties
#

Save and exit.

Start Apache.
# /usr/local/apache2/bin/apachectl start

Now you will be able to browse all jsp pages through apache. Apache will silently be connecting with tomcat through modjk to serve those.

Blogged with the Flock Browser

Connecting Apache with tomcat using modjk

Apache HTTP and Tomcat are web servers meant for different purposes. While Apache HTTP can be used to serve static (html) and dynamic (cgi/perl) contents ,  tomcat is used to serve Java pages (jsp/servlets). We can link both Apache HTTP & Apache Tomcat using modjk (connector):

What can you do by linking them both?

1. Effectively serve both static (html) and dynamic (jsp/servlet) pages.
2. Load balance/Cluster between two or more tomcat services.

There may be more ....

I did the installation in CentOS 4.4 (32bit). I installed Apache http (rpm) using yum. I assume that you have already installed Tomcat.

Below is the tutorial:

# yum install httpd httpd-devel (httpd-devel is needed to install any module in httpd)
# wget http://apache.siamwebhosting.com/tomcat/tomcat-connectors/jk/source/jk-1.2.26/tomcat-connectors-1.2.26-src.tar.gz (download any latest version).
# wget http://www.apache.org/dist/tomcat/tomcat-connectors/jk/source/jk-1.2.26/tomcat-connectors-1.2.26-src.tar.gz.md5 (md5 check sum for the above).

Check if you have downloaded a genuine package
# md5sum tomcat-connectors-1.2.26-src.tar.gz (This must be equal to the sum downloaded with the second wget).

# tar zfxv tomcat-connectors-1.2.26-src.tar.gz
# cd tomcat-connectors-1.2.26-src
# cd native
# ./configure --with-apxs=/usr/sbin/apxs
# make
# make install

# cd /etc/httpd/conf/
# cp httpd.conf httpd.conf.orig (I always keep a copy of the original).
# vi httpd.conf

Add the following code below the LoadModule section.

# Load mod_jk
# 
LoadModule jk_module modules/mod_jk.so

# Configure mod_jk
#
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info

JkMount *.jsp test   <- This must be the worker name you give in workers.properties. This will at(many) times go into your Virtual Host configuration.

Save and Exit.

# cd /etc/httpd/conf/
# vi workers.properties (create new file and paste the below code)

# workers.properties
#

# In Unix, we use forward slashes:
ps=/

# list the workers by name

worker.list=test

# ------------------------
# First tomcat server
# ------------------------
worker.test.port=8009
worker.test.host=localhost
worker.test.type=ajp13

#
# END workers.properties
#

Restart apache. Now you can access tomcat through apache. Any request to jsp files will go to test-tomcat.

Blogged with the Flock Browser

Enabling https (SSL) in Tomcat

For enabling ssl in tomcat you need to buy signed certificate/s from a Certificate Authority (some hosting companies also sell SSL certificates). The work doesn't end with that. After buying a certificate you have to create a certificate request, which your CA will sign and authorize.

Below are the steps i followed to enable SSL in tomcat.

Note: Remember to replace the full path of the filename. Usually i create these files a new subdirectory.

Creating Certificate Request

$ keytool -genkey -alias tomcat -keyalg RSA -keystore tomcat.key

The above command generates a key store file (tomcat.key) with RSA algorithm and aliased as tomcat. A keystore is nothing but a file which has your key and the CA's signature. The above command gets all the following information from you, so please keep them handy
Keystore password: <any thing you want>
First & Last Name: <your name/any authorized person's name>
Organizational Unit: <Dept in your company>
Organization Name: <Your company name>
Your City:
State:
Country Code: <eg: US>

Following command actually creates the csr
$ keytool -certreq -keyalg RSA -alias tomcat -file certreq.csr -keystore tomcat.key

Depending on whom you bought the certificate from, you may have to copy the contents of the certreq.csr from your CA's site or your CA will send it as email attachment.

Keys from CA:

You will get the keys (issued) from CA. You may also have to download two more key's (root &amp; intermediate) from your CA's website. First add root cert, then add intermediate followed by your site's certificate. Below are the steps:

First import root cert.
$ keytool -import -alias root -keystore tomcat.key -trustcacerts -file <root.crt>

Then the intermediate cert
$ keytool -import -alias intermed -keystore tomcat.key -trustcacerts -file <intermediate.crt>

Finally your site's cert
$ keytool -import -alias tomcat -keystore tomcat.key -trustcacerts -file <yourdomain.com.crt>

yourdomain.com.crt is the cert file issued by CA for your domain.

How to list the imported Certificates?

$ keytool -list -keystore tomcat.key

The above keystore file is now ready. Let us know configure tomcat to use this key file.

If you have already installed tomcat, edit the server.xml of the tomcat instance you want to turn on https. Search for the below lines:

<Connector port="8443" maxHttpHeaderSize="8192"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" disableUploadTimeout="true"
               acceptCount="100" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />

The above lines are commented by default. Remove the comment signs <!-- before and  --> after them.

Add the below line anywhere in between them:
keystoreFile="/path/to/keystore/tomcat.key" keystorePass="changeit"

Restart the tomcat service. We are done.

Now you must be able to browse https://localhost:8443 from the same machine where tomcat is installed. You can view the certificate and cross check it.

Blogged with the Flock Browser