Session Replication in Tomcat 5 Clusters, Part 2by Srini Penchikala
This is the second part of a series on session replication in the Tomcat 5 container. In part one, I provided an overview of sticky versus replicated sessions, and session replication design considerations. I also discussed how session replication works in a Tomcat cluster, especially when a server is started up or shut down. In this part, I will cover the configuration details of a sample Tomcat cluster setup and compare different session replication scenarios and their respective impacts on cluster performance.
To enable session replication in the Tomcat 5 container, the following steps should be completed:
- For clustering to work, you need to enable multicasting on your system.
- To use session replication efficiently, all Tomcat instances should be configured the same. This means the web applications should be deployed homogeneously to every server instance in the cluster. This configuration also simplifies cluster administration, maintenance, and troubleshooting tasks.
ReplicationValve) elements in server.xml. Enabling the
Clusterelement in server.xml means that the session manager for all the web contexts will be changed. So when running a cluster, make sure that you only have the web apps that need to be clustered and remove the other ones.
- If the server instances are running on the same machine, make sure the
tcpListenPortattribute is unique for each instance.
- Make sure web.xml has the
- All the session attributes must implement the
The sample cluster setup has two Tomcat instances and a load balancer to distribute the requests between the servers. All three servers were launched on a single machine. The following table lists the configuration parameters of the cluster and load balancer.
|Parameter||Load Balancer||Replication Node 1||Replication Node 2|
|Web App Directory||n/a||c:/dev/projects/replication/
|Coyote/JK2 AJP Connector||n/a||9009||10009|
Editor's note: the values for Web App Directory above have linefeeds inserted to suit this article's page layout. In your deployment, each should be a single, unbroken line.
Note: Do not set 127.0.0.1 as the
tcpListenAddress, since 127 is used as the
loopback address and may cause problems during session replication.
Load Balancer Setup
Tomcat server doesn't provide the failover capability to redirect any incoming
requests to the next available server in case one of the cluster nodes goes
down. So I used a separate load balancer to take care of load balancing the
web requests. Some popular load balancers are Apache/
and Pen. I used Pen as the load balancer in
the sample clustering setup.
Pen is a simple load balancer for TCP-based protocols such as HTTP or SMTP. It allows several servers to appear as one to the outside, and automatically detects servers that are down and distributes clients across the available servers. It provides both load balancing and failover capabilities. Pen is easy to install and configure, is very simple to use, and works on Windows and UNIX operating systems. The sample configuration uses the round-robin load balancing algorithm for distributing the load between the cluster nodes.
The command to run the load balancer is as follows:
pen -r -a -f -d localhost:8080 192.168.0.10:9080 192.168.0.20:10080
The following is the explanation of each command-line parameter used to launch the load balancer.
-r: Use round robin for load balancing
-a: Prints the data sent back and forth in ASCII format
-f: Stay in foreground
-d: Enable debug mode
For more details on the other parameters used to launch Pen, refer to the Pen web site.
Figure 1 shows the topology diagram with two cluster nodes, the load balancer, the instrumentation layer, and the test client.
I created a web application called
clusterapp to demonstrate
the clustering setup and the session replication mechanisms. To test the actual
session replication, I wrote a sample Java client called
measure how long it took to copy the session data from one server to the other.
This client uses Jakarta Commons HttpClient framework to create and manipulate
the HTTP session and calls a servlet on the Tomcat server. The hardware/software
specifications of the machine used to test the session replication are listed
- CPU: HP Pavilion Pentium III 800MHz
- Memory: 512MB RAM
- Hard disk: 40GB
- Operating system: Windows 2000 server
- JDK version: 1.4.2_05 (Note: JDK 1.4 or later version is required to use clustering and session replication)
- Tomcat version: 5.0.28
- Tools: Pen, Log4J, Eclipse, Commons HttpClient
When the replication client program is run, it first sets an action command
on how to handle the session (i.e. add a new attribute to the session, remove
an existing attribute from the session, or invalidate the session). The
client then calls the
ReplicationServlet by using the
in the Commons HttpClient framework. Based on the session command, the servlet
either generates a new session or modifies the existing session and forwards
to a web page to display the session details. If there is an error in managing
the session, it forwards to an error page. I wrote a custom session listener
ClusterAppSessionListener) to track the session management
details as new sessions are created and the existing ones are altered or expired.
The flow of the session replication is represented in the sequence diagram in Figure 2.
The session state on the server is tracked by a cookie for each web request, so the request URL must be the same from the client side to keep using the same session. Otherwise, a new HTTP session will be created every time a request is made. I tested the replication with two types of tests based on the type of attributes added to the session. The first category had 100 lightweight objects (1K each) added to the session. In the second category, I added a single large object (100K) to compare the session replication time taken based on the session attribute size.
Listed below are the test case specifications of
- Number of client threads: 2
- Number of repetitions: 1000
- Delay between requests: 1000 milliseconds (2000ms for large sessions)
- Number of test samples: 1000
- Session Attributes: Small (100 1K objects) or large (one 100K object)
The command used to run the test client with the required arguments is as follows.
java -Dlog4j.configuration=log4j.xml com.clusterapp.test.SessionReplicationClient 2 192.168.0.10 9080 1000 1000 lite
The test scenarios included using different session managers (
and session replication modes (Async, Sync, or Pooled). The following table shows
the list of scenarios tested in the Tomcat cluster.
|Test #||Code||Session Manager||Replication Mode||Session Type||# of Nodes||Delay (ms)|
To change replication mode from pooled to synchronous to asynchronous, just
replicationMode attribute value in the
in the server.xml file. Also, to change the session manager type,
managerClassName attribute in the
The following parameters were used to compare the response times and effectiveness of session replication.
- Average response time (ms)
- Average request time (ms)
- Cluster overhead time (ms)
- Replication time (ms)
- Ratio (bytes/ms)
- Ratio (bytes/request)
The combination of using delta manager and pooled replication mode was the best criteria for efficient session replication. Also, keeping the session size small proved to be two to three times faster than replicating large sessions. The results from the session replication tests are listed in the table below.
|Test #||Code||Avg. Response Time (ms)||Avg. Request Time (ms)||Cluster Overhead Time (ms)||Avg. Replication Time (ms)||Ratio (bytes/ms)||Ratio (bytes/request)|
DeltaManager proved to be more efficient in replicating sessions
because it only deals with session deltas, not the entire session's data. Session
DeltaManager were 30 (for large sessions) to
50 (small sessions) percent more efficient, compared to using the simple replication
Mode of Replication
Pooled replication mode took less time to replicate the sessions compared
to the other two options (Synchronous or Asynchronous). The pooled option was almost
four times as fast as the async option, in terms of replication time. But the response
time and cluster overhead times for pooled and async modes were almost same
because in the asynchronous mode,
Cluster doesn't wait for the session to be
completely replicated before returning.
Based on the results from session replication tests we can conclude that
should be used whenever possible. Since the overhead in replicating session
data is significant, make sure that you don't store too much data in the session.
Also, the size of the attributes added to the session is another factor that
impacts the session replication time. When I ran the test to store big objects
(100K) in the session, the replication times were very high compared to the
test with small objects (1K) in the session. The other best practices to
minimize the session replication overhead are to avoid calling
to store data in the request object instead of the session. This is a better
option because the request attributes are reset when the web request is completed.
Also, if there is no business reason to store the data on the server, we can
store it on the client in the form of cookies. This approach completely eliminates
the need for using any session at all.
One way to minimize the time taken for session replication is to limit the number of server instances in the cluster to two (or maybe three) servers so that session data on the first server is copied to the second server in case the first one goes down. In order to keep the network traffic down in an all-to-all environment, the cluster can be split into smaller groups, with each group consisting of two or three server instances. Remember, the more server instances in the cluster, the longer the session replication time will be. Tomcat 5 currently doesn't support this concept of primary/secondary replication. It will be available in a future release, where we will be able to store the session on one or two backup servers. With this feature, Tomcat will provide a more complete session replication solution for running web applications in cluster environments.
Some of the other features that Tomcat will support in the future are:
- The ability to have attributes that are non-serializable in the session, which the cluster will ignore.
- The ability to replicate context attributes and non-session related data, such as JDBC connection pools and object caches.
These new features will make the session replication in Tomcat clusters more robust and flexible.
I want to thank Filip Hanik for his valuable suggestions and feedback on the replication modes, session replication web application setup, and the test client sections of this article.
- Session Replication sample code for this article
- Tomcat Clustering/Session Replication Home Page
- Tomcat Load Balancer Home Page
- "Clustering and Load Balancing in Tomcat 5, Part 1"
- "Clustering and Load Balancing in Tomcat 5, Part 2"
- "BEA WebLogic Server: Using WebLogic Server Clusters"
- Patterns of Enterprise Application Architecture
Srini Penchikala is an information systems subject matter expert at Flagstar Bank.
Return to ONJava.com