In a recent article, Bill Burke and Sacha Labourey explained the key components of JBoss 3 clustering. We will now present several new clustered services recently introduced in JBoss 3.2.2, which was not yet released at the time of the writing of that article. Basic knowledge of JBoss 3 clustering and farming is a prerequisite for this article.
Several design patterns commonly occur in clustered enterprise applications. Though they are not covered by the current J2EE specification, JBoss provides several built-in services to help the development of components that follow these patterns.
A clustered singleton service is deployed on multiple nodes in a cluster but runs on only one of the nodes. The node running the singleton service is typically called the master node. When the master fails, another master is selected from the remaining nodes and the service is restarted on the new master.
Figure 1. Clustered singleton service
Many times, an application needs to ensure that a certain task is run exactly once. Thus, only one of the nodes in a cluster should execute the task. The other nodes should knowingly remain passive. Examples of singleton tasks include:
|
Related Reading 
Java Enterprise Best Practices |
While it is fairly easy to implement such singleton tasks in a single VM, the solution will usually not work immediately in a clustered environment. Even in the simple case of a task activated upon startup on one of the nodes in a two-node cluster, several problems must be addressed:
The logic to solve these problems is unlikely to be included in the design of a single-VM solution. However, a solution can be found to address the case at hand and it can be patched on the startup task. This is an acceptable approach for a few startup tasks and two-node clusters.
As the application grows and becomes more successful, more startup tasks may be necessary. The application may also need to scale to more than two nodes. The clustered singleton problem can quickly become mind-boggling for larger clusters, where the different node startup scenarios are far more difficult to enumerate than in the two-node case. Another complicating factor is communication efficiency. While two nodes can directly connect to each other and negotiate, 10 nodes will have to establish 45 total connections to use the same technique.
This is where JBoss comes in handy. It eliminates most of the complexity and allows application developers to focus on building singleton services regardless of the cluster topology.
We will illustrate how the JBoss clustered singleton facility works with an example. First, we will need a service archive descriptor. Let's use the one that ships with JBoss under server/all/farm/cluster-examples-service.xml. The following is an excerpt:
<!-- | This MBean is an example of a cluster Singleton -->
<mbean code="org.jboss.ha.singleton.examples.HASingletonMBeanExample"
name="jboss.examples:service=HASingletonMBeanExample">
</mbean>
<!- - -->
<!-- | This is a singleton controller which works similarly to the
| SchedulerProvider (when a MBean target is used) -->
<mbean code="org.jboss.ha.singleton.HASingletonController"
name="jboss.examples:service=HASingletonMBeanExample-HASingletonController">
<depends>jboss:service=DefaultPartition</depends>
<depends>jboss.examples:service=HASingletonMBeanExample</depends>
<attribute name="TargetName">jboss:service=HASingletonMBeanExample</attribute>
<attribute name="TargetStartMethod">startSingleton</attribute>
<attribute name="TargetStopMethod">stopSingleton</attribute>
</mbean>
<!- - --> This file declares two MBeans, HASingletonMBeanExample and
HASingletonController. The first one is a singleton service that
contains the custom code. It is a simple JavaBean with the following source
code:
public class HASingletonMBeanExample
implements HASingletonMBeanExampleMBean {
private boolean isMasterNode = false;
public void startSingleton() {
isMasterNode = true;
}
public boolean isMasterNode() {
return isMasterNode;
}
public void stopSingleton() {
isMasterNode = false;
}
} All of the custom logic for this particular singleton service is contained
within this class. Our example is not too useful; it simply indicates, via the
isMasterNode member variable, whether the master node
is running the singleton. This value will be true only on the one node in the cluster
where it is deployed.
HASingletonMBeanExampleMBean exposes this variable as an MBean
attribute. It also exposes startSingleton() and
stopSingleton() as managed MBean operations. These methods control
the lifecycle of the singleton service. JBoss invokes them automatically when
a new master node is elected.
How does JBoss control the singleton lifecycle throughout the cluster? The
answer to this question is in the MBean declarations. Notice that the
HASingletonMBeanExample-HASingletonController MBean also takes the
name of the sample singleton MBean and its start and stop methods.
On each node in the cluster where these MBeans are deployed, the controller will work with all of the other controllers with the same MBean name deployed in the same cluster partition to oversee the lifecycle of the singleton. The controllers are responsible for tracking the cluster topology. Their job is to elect the master node of the singleton upon startup, as well as to elect a new master should the current one fail or shut down. In the latter case, when the master node shuts down gracefully, the controllers will wait for the singleton to stop before starting another instance on the new master node.
A singleton service is scoped in a certain cluster partition via its
controller. Notice that, in the declaration above, the controller MBean
depends on the MBean service DefaultPartition. If the partition
where the singleton should run is different than the default, its name can be
provided to the controller via the MBean attribute PartitionName.
Clustered singletons are usually deployed via the JBoss farming service. To test this example, just drop the service file above in the server/all/farm directory. You should be able to see the following in the JBoss JMX web console:
Figure 2. Controller MBean view. The MasterNode attribute will have value True on only one of the nodes.
Figure 3. Sample singleton MBean view. The MasterNode attribute will have the same value as
the MasterNode attribute on the controller MBean.
|
The JBoss 3 Scheduler service is covered in detail by the JBoss 3: Administration and Development book. It has three interdependent components:
ScheduleManager, which serves as a centralized registry for
registering and executing schedules.ScheduleProvider, which abstracts the custom logic that
creates schedules.Schedulable, which represents a task that is executed at
scheduled times.Often, applications need to schedule tasks that have to be executed once in the scope of the application, whether it is running standalone or in a cluster of multiple nodes. Examples of such tasks include regular database cleanup, email notifications, and scheduled reports.
All JBoss ScheduleProvider services accept an MBean attribute
that enables them to schedule tasks on only one node in the cluster (of one or
more nodes). The attribute name is HASingleton and its value is a
Boolean type. When set to True, all ScheduleProvider
MBeans registered with the same name and deployed in the same cluster partition
will coordinate and make sure that the schedule is only provided to the
schedule manager on one of the nodes. When set to False, each of
the schedule providers will act independently and as a result, they will all
schedule tasks with their local scheduler managers. The default value of the
attribute is True, which allows transparent transition of standalone schedule
providers to a clustered environment.
The name of the partition where a singleton schedule provider service will
be deployed can be set via the PartitionName attribute. By default,
the value assumes the default JBoss partition name.
Here is an excerpt from the example service archive descriptor cluster-examples-service.xml, located in the server/all/farm directory in the JBoss installation.
<!--
| This MBean is an example of an HA Schedule Target
| which is identical to a regular Schedule Target
| (the example class is the same, just the MBean has different names)
- ->
<mbean code="org.jboss.varia.scheduler.example.SchedulableMBeanExample"
name="jboss.examples:service=HASchedulableMBeanExample">
</mbean>
<!- - -->
<!--
| The Schedule Manager has to be started whenever
| schedules are needed.
|
| Uncomment only if not started by
| another service (e.g. schedule-manager-service.xml)
- ->
<mbean code="org.jboss.varia.scheduler.ScheduleManager"
name="jboss:service=ScheduleManager">
<attribute name="StartAtStartup">true</attribute>
</mbean>
<!- - -->
<!--
| This is a single schedule Provider which works like the
| one in schedule-manager-service.xml
|
| The key difference is the explicit use of the HASingleton MBean attribute
| to make the provider a clustered singleton.
| When HASingleton is set to true the MBean will usually declare dependency
| on a cluster partition. In this case it is the DefaultPartition.
| When not explicitly set the attribute defaults to true.
|
| The same attribute can also be used for the other schedule providers as well:
| DBScheduleProvider and XMLScheduleProvider
|
|
- ->
<mbean code="org.jboss.varia.scheduler.SingleScheduleProvider"
name="jboss:service=HASingleScheduleProvider">
<depends>jboss:service=DefaultPartition</depends>
<depends>jboss:service=ScheduleManager</depends>
<depends>jboss.examples:service=HASchedulableMBeanExample</depends>
<attribute name="HASingleton">true</attribute>
<attribute name="ScheduleManagerName">jboss:service=ScheduleManager</attribute>
<attribute name="TargetName">jboss:service=HASchedulableMBeanExample</attribute>
<attribute name="TargetMethod">
;hit( NOTIFICATION, DATE, REPETITIONS, SCHEDULER_NAME, java.lang.String )
</attribute>
<attribute name="DateFormat"></attribute>
<attribute name="StartDate">NOW</attribute>
<attribute name="Period">10000</attribute>
<attribute name="Repetitions">10</attribute>
</mbean>
<!- - --> The deployment descriptor above is almost identical to the one provided by
schedule-manager-service.xml but is intended to be deployed via
farming. Although not necessary, since the default value is True, the
descriptor specifies the HASingleton attribute of the schedule
provider for illustration of its usage.
When it comes to reliable, mission-critical, and sophisticated J2EE messaging, one API stands out: JMS. The JBossMQ service is a robust JMS implementation that meets the high standards commanded by the API for distributed, transactional, and secure messaging. Then why another notification service?
Most typical clustered applications are deployed in a symmetric, homogeneous environment where components need to notify each other about various lifecycle or domain-change events, much like AWT and Swing widgets exchange events. This is where the JBoss cluster notifications come in. They fill the need for reliable, quick, and lightweight events. Cluster notifications do not require new skills from developers familiar with JMX, because they comply with the JMX notifications API.
Figure 4. Clustered notification service
If you have used JMX notifications before, the clustering extension should
be a breeze. There are two common ways to employ clustered notifications: by
extending HAServiceMBeanSupport or by delegating the work to
instances of this class.
Let's look at an example that comes with the JBoss distribution. Here is an excerpt from the familiar descriptor cluster-examples-service.xml, located in server/all/farm.
<!--
| This MBean is an example showing how to extend a cluster notification broadcaster
| Use the sendNotiication() operation to trigger new clustered notifications.
| Observe the status of each instance of this mbean in the participating cluster partition nodes.
- ->
<mbean code="org.jboss.ha.jmx.examples.HANotificationBroadcasterExample"
name="jboss.examples:service=HANotificationBroadcasterExample">
<depends>jboss:service=DefaultPartition</depends>
</mbean>
<!- - -->
<!--
| This MBean is an example that shows how to delegate notification services
| to a HANotificationBroadcaster.
| Use the sendNotiication() operation to trigger new clustered notifications.
| Observe the status of each instance of this mbean in the participating cluster partition nodes.
- ->
<mbean code="org.jboss.ha.jmx.examples.HANotificationBroadcasterClientExample"
name="jboss.examples:service=HANotificationBroadcasterClientExample">
<depends>jboss.examples:service=HANotificationBroadcasterExample</depends>
<attribute name="HANotificationBroadcasterName">
jboss.examples:service=HANotificationBroadcasterExample
</attribute>
</mbean>
<!- - --> The MBean implemented by the HANotificationBroadcasterExample
class provides common clustering services like shared distributed state,
replication management, and notifications. It is a convenience class that can
be extended and customized or used as is. Its most important attribute is
PartitionName, which fixates the partition of nodes where all
MBeans with the same name will work together.
The other MBean in this example is implemented by the
HANotificationBroadcasterClientExample class. It uses the former
MBean's services to broadcast notifications to the local listeners, as well
as remote cluster listeners that have subscribed to the
HANotificationBroadcasterExample MBean. At the same time, the
client MBean will also receive all notifications sent through any instance of
the broadcaster. Clear as mud?
|
Maybe a picture will help. Here is a screenshot of the client MBean attributes:
Figure 5. HANotification Client Attributes
The first attribute points to the name of the cluster notification
broadcaster to which we will subscribe. The next attribute of interest is
ReceivedNotifications. When a message is sent to any deployed
instance of the broadcaster MBean on any of the partition nodes, it will be
received and listed in the Value column for this attribute. In the
screenshot above, there are two received notifications, iivanov2
and mau. Even though it is not obvious from the picture, one of
these messages was received from a remote host.
Now, a screenshot of the MBean operation of interest:
Figure 6. HANotification client operation
The name is not very friendly, but it is distinct and will do for this
example. When this operation is invoked with a text message as its argument,
the value will appear in the ReceivedNotification attribute on
each of the client MBeans deployed in the same partition.
Let's look at the code of the client MBean to see that it is actually straightforward. I made an attempt to annotate the code and keep it simple, so that there is no need for additional analysis.
public class HANotificationBroadcasterClientExample
extends ServiceMBeanSupport
implements HANotificationBroadcasterClientExampleMBean, NotificationListener
{
/**
*
* On service start, subscribes to notification sent by this broadcaster or
* its remote peers.
*
*/
protected void startService() throws Exception
{
super.startService();
addHANotificationListener(this);
}
/**
*
* On service stop, unsubscribes to notification sent by this broadcaster or
* its remote peers.
*
*/
protected void stopService() throws Exception
{
removeHANotificationListener(this);
super.stopService();
}
/**
* Broadcasts a notification to the cluster partition.
*
* This example does not ensure that a notification sequence number
* is unique throughout the partition.
*
*/
public void sendTextMessageViaHANBExample(String message)
throws InstanceNotFoundException, MBeanException, ReflectionException
{
long now = System.currentTimeMillis();
Notification notification =
new Notification(
"hanotification.example.counter",
super.getServiceName(),
now,
now,
message);
server.invoke(
broadcasterName_,
"sendNotification",
new Object[] { notification },
new String[] { Notification.class.getName() }
);
}
/**
* Lists the notifications received on the cluster partition
*/
public Collection getReceivedNotifications()
{
return messages_;
}
/**
* @return the name of the broadcaster MBean
*/
public String getHANotificationBroadcasterName()
{
return broadcasterName_ == null ? null : broadcasterName_.toString();
}
/**
*
* Sets the name of the broadcaster MBean.
*
* @param
*/
public void setHANotificationBroadcasterName(String newBroadcasterName)
throws InvalidParameterException
{
if (newBroadcasterName == null)
{
throw new InvalidParameterException("Broadcaster MBean must be specified");
}
try
{
broadcasterName_ = new ObjectName(newBroadcasterName);
}
catch (MalformedObjectNameException mone)
{
log.error("Broadcaster MBean Object Name is malformed", mone);
throw new InvalidParameterException("Broadcaster MBean is not correctly formatted");
}
}
protected void addHANotificationListener(NotificationListener listener)
throws InstanceNotFoundException
{
server.addNotificationListener(broadcasterName_, listener,
/* no need for filter */ null,
/* no handback object */ null);
}
protected void removeHANotificationListener(NotificationListener listener)
throws InstanceNotFoundException, ListenerNotFoundException
{
server.removeNotificationListener(broadcasterName_, listener);
}
public void handleNotification(
Notification notification,
java.lang.Object handback)
{
messages_.add(notification.getMessage());
}
// Attributes ----------------------------------------------
Collection messages_ = new LinkedList();
/**
* The broadcaster MBean that this class listens to and delegates HA notifications to
*/
ObjectName broadcasterName_ = null;
}The key points of the code are highlighted. Notice how subscription to the cluster broadcaster is accomplished via the MBean server API.
It is also important to note that the subscription is local. Both the
client and the MBean server reside in the same VM. The
sendNotification() invocation is also local to the VM. The cluster
broadcaster hides the implementation details of working together with its
remote peers to deliver notifications throughout all nodes.
If you aren't too concerned with dependencies on JBoss-specific classes, you can directly extend the cluster broadcaster class, in which case all method calls will be direct instead of proxied through the MBean Server. Here is the code for an extended broadcaster:
public class HANotificationBroadcasterExample
extends HAServiceMBeanSupport
implements HANotificationBroadcasterExampleMBean
{
/**
*
* On service start, subscribes to notification sent by this broadcaster or
* its remote peers.
*
*/
protected void startService() throws Exception
{
super.startService();
addNotificationListener(listener_, /* no need for filter */ null, /* no handback object */ null);
}
/**
*
* On service stop, unsubscribes to notification sent by this broadcaster or
* its remote peers.
*
*/
protected void stopService() throws Exception
{
removeNotificationListener(listener_);
super.stopService();
}
/**
* Broadcasts a notification to the cluster partition.
*
* This example does not ensure that a notification sequence number
* is unique throughout the partition.
*
*/
public void sendTextMessage(String message)
{
long now = System.currentTimeMillis();
Notification notification =
new Notification("hanotification.example.counter", super.getServiceName(), now, now, message);
sendNotification(notification);
}
/**
* Lists the notifications received on the cluster partition
*/
public Collection getReceivedNotifications()
{
return messages_;
}
Collection messages_ = new LinkedList();
NotificationListener listener_ = new NotificationListener()
{
public void handleNotification(Notification notification,
java.lang.Object handback)
{
messages_.add( notification.getMessage() );
}
};
} This class behaves similarly to the previous one. If you examine its MBean
View in the JMX console, you will notice that it has the same
ReceivedNotifications attribute. The messaging operation is called
sendTextMessage().
This text shows more examples where clustering is both powerful and complex. Fortunately, JBoss eliminates much of the complexity, allowing developers to enjoy the benefits of high availability. Writing singleton services in a distributed environment with JBoss is almost as easy as writing single VM singletons. If you previously invested in the JBoss Scheduler for a standalone server environment, you can keep your code and make it automatically valid in a clustered environment. Finally, lightweight notifications are no longer the prerogative of standalone applications. JBoss cluster notifications are yet another step towards practical horizontal scalability.
Ivelin Ivanov is a frequent contributor to the open source community and has recently participated in projects that include Apache Cocoon, JBoss, GNU XQuery and Jakarta Commons.
Return to ONJava.com.
Copyright © 2009 O'Reilly Media, Inc.