This is the second part of a series on clustering and load balancing in Tomcat 5 server. In part 1, I provided an overview of large-scale J2EE system design as well as various factors to be considered when designing the system for scalability and high availability. I also discussed Tomcat's support for clustering, load-balancing, fault-tolerance, and session-replication capabilities. In this part, we'll cover the architecture of a proposed cluster setup and go over the installation and configuration details in deploying the cluster (by running multiple Tomcat server instances).
Listed below are the main objectives that I wanted to achieve in the proposed Tomcat cluster:
The cluster should be highly scalable.
It should be fault-tolerant.
It should be dynamically configurable, meaning it should be easy to manage the cluster declaratively (changing a configuration file) rather than programmatically (changing Java code).
It should provide automatic cluster member discovery.
Fail-over and load-balancing features for session data with in-memory session state replication.
Pluggable/configurable load-balancing policies.
Group membership notification when a member of the cluster joins or leaves a group.
No loss of message transmission through multicast.
Clustering should be seamless to the web application and the server. It should provide both client and server transparency. Client Transparency means that the client is not aware of clustered services or how the cluster is set up. The cluster is identified and accessed as a single thing rather than individual services. And server transparency means that the application code in a server is not aware that it's in a cluster. The application code cannot communicate with the other members of cluster.
Four instances of Tomcat server were installed to set up the clustering environment. Tomcat was used for both load balancing and clustering requirements. The cluster setup was done using vertical scaling method (multiple Tomcat server instances running on a single machine). One server group and two clones were configured in the cluster (a server group is a logical presentation of the application server). The clones had same exact configuration (in terms of web application directory structure and contents) as the server group to optimize session replication. To follow are the main components in the proposed cluster setup:
Load Balancer: A Tomcat instance is configured to distribute the traffic among cluster nodes. This instance is given a code name TC-LB.
Clustering: Three Tomcat server instances were run as part of the cluster. The code names for these instances are TC01, TC02, and TC03.
Session Persistence: In-memory session replication was chosen as the session persistence mechanism. The session data was copied to all three cluster members whenever the session object is modified.
Fail-Over: The balancer application that comes with Tomcat installation is not designed to handle fail over. I wrote a utility class called
ServerUtil to check the server status before forwarding any requests to it. It has two methods to verify the status of a cluster node. In the first method, it uses
McastService to check if a specified server instance is currently running or not. The second method verifies a cluster node's availability by creating a URL object based on the web page URL passed in as a parameter. To use this class make sure
catalina-cluster.jar (located in %TOMCAT_HOME%/server/lib directory) and
commons-logging-api.jar (%TOMCAT_HOME%/bin directory) files are specified in the classpath.
The application architecture diagram in Figure 1 shows the main components of the cluster.
Figure 1. Tomcat cluster architecture diagram
Table 1. Hardware/software specifications of the machine used to set up Tomcat clustering.
|Processor||HP Pavilion Pentium III with 800 MHz|
|Memory||512 MB RAM|
|Hard Disk||40 GB|
|Operating System||Windows 2000 server with Service Pack 4|
|JDK Version||1.4.0_02 (Note: JDK version 1.4 or a later version is required to enable Tomcat Clustering)|
|Tools Used||Ant 1.6.1, Log4J, JMeter, JBuilder|
This is an abstract class created to encapsulate the common rules logic in the custom Rule classes. The custom load balancing rules used in the sample web application extend this base class.
This class defines the logic to forward the web requests to an available server in a random manner. It uses current system time as seed to generate random numbers.
This class defined the load-balancing logic based on a round robin rule. When a request comes in it forwards the request to the next member in the list. It uses a static variable to track the next available cluster member and increments this value by one for every new request.
A utility class was created to check if a specific cluster node is available or not, to receive requests. This class uses
org.apache.catalina.cluster.mcast package) to detect if a cluster member has left the group.
The relationship of these Java classes is shown in the class diagram in Figure 2.
Figure 2. Cluster application class diagram
This file is used to configure clustering in a Tomcat server instance. The version that comes with Tomcat installation has the configuration details included in the file but they are commented out.
This configuration file is used to specify that a web application session data needs to be replicated.
This file is used to define the custom load-balancing rules. This is the file we use to specify which load-balancing rule we want to use to distribute the load among the cluster members.
A simple test JSP script was used to check the server status. It displays the system time and name of the Tomcat instance from which it's launched.
This is the startup page in our sample web application. It forwards the web requests to a load balancer filter using HTML redirect.
This script is used to verify that no session data is lost when a cluster node goes down. It displays session details and also has HTML fields to manipulate HTTP session object.
An Ant build script was created to automate the tasks of starting and stopping Tomcat instances (the latest version of Ant 1.6.1 was used to run this script). Once a Tomcat instance has started successfully, you can call the
test.jsp to verify which Tomcat instance is running by specifying IP address and port number. The JSP page displays the current system time and the name of the Tomcat instance. (You need to change the home directories for Tomcat servers specified in
build.properties file to run the script in your own environment).
Several targets in the build script start or stop Tomcat instances:
start.tomcat5xto start a specific Tomcat instance (for example:
stop.tomcat5xto stop a specific Tomcat instance.
stop.alltomcatsto stop all running Tomcat instances.
The sample code used in this article is provided as tomcatclustering.zip. After installing Tomcat server instances (you need four of them), extract the contents of the zip file to the tomcat directories. The sample code provided uses
RoundRobinRule as the load-balancing policy. If you want to use the random redirect policy, modify the
rules.xml file located in "tomcat50/webapps/balancer/WEB-INF/config" directory. Just comment-out the rule elements for
RoundRobinRule and uncomment rule elements specifying
RandomRedirectRule. Also, if you want to use two instances in the cluster instead of three, comment out the third rule and change
maxServerInstances attribute to 2 (instead of 3).
Note: I removed all other web applications (jsp-examples, etc.) that were included in the Tomcat installation, keeping only the balancer and the sample web applications.
The web request flow in the sample cluster environment is as follows:
sessiondata.jsp(located in "clusterapp" web application) in the selected cluster member is called.
ClusterAppSessionListenerare called to log the session modification events.
sessiondata.jspdisplays the session details (such as session id, last access time, etc.) on the web browser.
stop.tomcat5x" target in Ant script).
Figure 3 shows the web request flow represented in a Sequence diagram.
Figure 3. Cluster application sequence diagram (click on the screenshot to open a full-size view).
A sample web application called "clusterapp" was created to run in the cluster. All the instances have the same directory structure and contents to optimize session replication.
Since Tomcat server instances in the cluster use IP Multicast to transmit sessions, we need to make sure that IP multicast is enabled on the machine where the cluster is set up. To verify this, you can run the sample Java program
MulticastNode provided in the book Tomcat: The Definitive Guide or refer to the sample tutorial available on JavaSoft web site on how to write multicast server and client programs.
When a cluster node is started the other members in the cluster show a log message on the server console that a member has been added to the cluster. Similarly, when a cluster node goes down, the remaining members display a log message on the console that a member has disappeared from the cluster. Figure 4 shows the log messages displayed on Tomcat console when a cluster node is removed from the cluster or a new member is added to the cluster.
Figure 4. Log messages when a member is added or removed from cluster
Follow the steps below to enable clustering and session replication in Tomcat server:
All session attributes must implement the
Cluster element in
server.xml file. The attributes
replicationMode in the
Cluster element are used to optimize frequency and the session replication mechanism.
ReplicationValve by uncommenting
Valve element in server.xml. The
ReplicationValve is used to intercept the HTTP request and replicate the session data among the cluster members if the session has been modified by the web client. The
Valve element has a "filter" attribute that can be used to filter out requests that could not modify the session (such as HTML pages and image files).
Since all three Tomcat instances are running on the same machine, the
tcpListenPort attribute is set to be unique for each Tomcat instance. It is important to know that the attributes starting with mcastXXX (
mcastDropTime) are for the cluster membership IP multicast ping and those starting with tcpXXX (
tcpSelectorTimeout) are for TCP session replication ("Clustering configuration parameters" table below shows the configuration of different settings in the Tomcat server instances to enable clustering).
web.xml meta file (located in clusterapp\WEB-INF directory) should have
<distributable/> element. In order to replicate session state for a specific web application, the
distributable element needs to be defined for that application. This means if you have more than one web application that need session replication, then you need to add
distributable in all those web applications'
web.xml files. The Tomcat clustering chapter in the book Tomcat: The Definitive Guide provides a very good explanation on this topic.
Table 2. Clustering configuration parameters
|Configuration Parameter||Instance 1||Instance 2||Instance 3||Instance 4|
|Instance Type||Load Balancer||Cluster Node 1||Cluster Node 2||Cluster Node 3|
|Coyote/JK2 AJP Connector||8009||9009||10009||11009|
Note: Since all the cluster members are running on the same physical machine they use the same IP address (
If you are not using the Ant script to start and stop Tomcat instances, do not set up the
CATALINA_HOME environment variable on your computer. If this variable is set, all instances will try to use the same directory (specified in
CATALINA_HOME variable) to start Tomcat instances. As a result, only the first instance will successfully start and the other instances will crash with a bind exception message saying that the port is already in use: "
java.net.BindException: Address already in use: JVM_Bind:8080".
I wrote two simple, custom load-balancing rules extending the rules API to redirect incoming web requests (
RandomRedirect). These rules are based on load-balancing algorithms such as round robin and random redirect. You can write similar custom load balancing rules based on other factors like weight based and last access time, etc. The Tomcat load balancer provides a sample parameter-based load balancing rule. It redirects web requests to different URLs depending on the parameter specified in the HTTP request.
Leave the cluster and valve elements in
server.xml (in TC-LB instance) commented out since we are not using this Tomcat instance as a cluster member.
In session persistence testing, the main objective is to verify that session data is not lost when a cluster member crashes in the middle of a web request. The JSP
sessiondata.jsp was used to display the session details. This script also provides HTML text fields to add/modify/remove the session attributes. After adding couple of attributes to HTTP session, I randomly stopped the cluster nodes and check the session data on the available cluster members.
The objective of load testing is to study the custom load balancing algorithms and see how effectively the web requests are distributed to the nodes in the cluster especially when one or more of the nodes go down. JMeter load testing tool was used to simulate multiple concurrent web users.
Steps to test load balancing in the cluster setup:
All log messages were directed to a text file called
tomcat_cluster.log (located in tomcat50/webapps/balancer directory). The response times for all the web objects shown in the sequence diagram (Figure 2) were logged using Log4J messages. The elapsed times (in milliseconds) were collected and tabulated into a matrix as shown in Table 3. I followed a similar instrumentation methodology described in Designing Performance Testing Metrics into Distributed J2EE Apps in collecting the response times during the tests.
The following tables show the elapsed times in load testing (using
RoundRobinRule) and load distribution percentages (using
Table 3. Elapsed times for load testing
|1||All three server instances running||54||76||12||142|
|2||Two server instances running (TC02 was stopped)||55||531||14||600|
|3||Only one server instance running
(TC01 and TC02 were stopped)
Note: All elapsed times are average values based on a load of 100 concurrent users.
Table 4. Load distribution when using random LB policy
|#||Scenario||TC01 (%)||TC02 (%)||TC03 (%)|
|1||All three server instances running||30||46||24|
|2||Two server instances running (TC02 was stopped)||56||0||44|
Note: Load distribution percentages are based on a load of 100 concurrent users.
In session persistence testing, after adding session attributes one of the cluster nodes was brought down and it was verified that the session attributes were not lost due to the server outage. The session details logged in the text file were used to study the details of session attributes.
In load testing, when one or two server instances were stopped and only one Tomcat instance was running, the response times took longer compared to when all three instances were available. When the stopped instances were restarted, the load balancer automatically found that the server is again available to take the requests and redirected the next web request, which significantly improved the response times.
The mechanism I used to find if a cluster member is available (using
ServerUtil) is not the fastest way to do this. More sophisticated and robust fail-over techniques should be used in real world scenarios.
One of the limitations of the proposed cluster setup is that it only provides a single load balancer. What happens if the Tomcat instance acting as load balancer goes down? There is no way to forward the requests to any of the cluster members and the result is a so-called Single Point of Failure (SPoF). One solution to this problem is to have a second Tomcat instance as the standby load balancer to take over if the primary load balancer crashes. Typical HA options involve having two load balancers to prevent SPoF situations.
In the sample cluster setup, all Tomcat instances (including load balancer) were configured to run on the same computer. A better design is to run a load balancer instance on a separate machine from the cluster members. Also, we should limit two cluster nodes per machine to take advantage of horizontal scaling method and to improve cluster performance.
HTTP session replication is an expensive operation for a J2EE web application server. The implications of session management in a clustered J2EE environment should be considered during the analysis and design phases of a project rather than waiting until the web application is implemented in the production environment. The application code must be designed keeping the cluster environment in mind. If the clustering implications are not considered at the design phase, the code may need to be completely rewritten to make it work in the cluster setup, which could be a very expensive effort.
If a web application supports any kind of object caching mechanism, then caching of objects in a cluster environment should be considered at the initial stages of the application development. This is very important because keeping cached data in all the cluster nodes in sync is critical for providing accurate and up-to-date business data to web users. Another important consideration is clearing the expired session data in a cluster.
Once the J2EE cluster has been successfully set up and is running, its management and maintenance will become very important to provide the benefits of scalability and high availability. With many nodes (members) in a cluster, maintenance revolves around keeping the cluster running and pushing out application changes to all cluster nodes. One way to provide these services is to implement a monitoring service that periodically checks the server availability and notifies if any of the nodes in the cluster become unavailable. This service should check the nodes at regular intervals detecting failed nodes and removing them from the active cluster node list so no requests go to those nodes. It should include, as changes and updates occur, the ability to update and synchronize all servers in the cluster. Since all requests to a web application must pass through the load-balancing system, the system can determine the number of active sessions, the number of active sessions connected in any instance, response times, peak load times, the number of sessions during peak load, the number of sessions during minimum load, and more. All this audit information can be used to fine-tune the entire system for optimal performance. Reports showing all these metrics should be generated on a regular basis to assess the effectiveness of load-balancing policies and cluster nodes.
Currently, all the configuration required to set up the cluster and load balancer is done manually by manipulating the configuration files (
rules.xml). It would be very helpful if the Jakarta group were to provide a web-based cluster administration GUI tool to perform the configuration changes needed to manage the clustering and load-balancing setup.
Srini Penchikala is an information systems subject matter expert at Flagstar Bank.
Return to ONJava.com.
Copyright © 2009 O'Reilly Media, Inc.