madbean

Tip: Clustering and your Servlet application

27 Jun 2003

Someone posted a question on Slashdot asking "Building a Stable and Clustered J2EE Environment?". Clustering is just hard.

Here is a tip for young players:

Good-old, trusty Session

Many Servlet applications make use of the Session. You have to love that Session. In one Servlet you can do this:

HttpSession session = ...;
List names = new ArrayList();
names.add("Matt");
names.add("Oliver");
session.setAttribute("names", names);

Then at a later date, in some other request, in some other Servlet, you can get to that data, and modify it:

HttpSession session = ...;
List names = (List) session.getAttribute("names");
// add someone else
names.add("Conor");

The Session is like a HashMap. It's good for squirrelling things away for later.

And the survey says...

The problem comes when you deploy your Servlet application into a clustered Servlet container. When clustering (or in a "distributed" container), two caveats pop up:

  1. The objects you put into the session must be Serializable1. Not a problem for our code above, fortunately. List and String are both Serializable.
  2. The Servlet spec makes no provision regarding how objects bound as attributes in a Session are replicated in a cluster. And in practice, this turns out into a huge bug in our code above.

"What is the huge bug" you say? Consider a cluster with two nodes. For fail-over reasons, you want the data in the Session replicated between the two nodes. The Servlet spec says nothing about when the Session attributes should get replicated in a cluster. So it is up to the particular container implementation. And many implement a policy much like this:

When you add an object to a Session using setAttribute(), then that object will be (at some point) serialized; and it is that serialized representation that will be replicated through the cluster.

Note that no provision is made for replicating our List of names when we call names.add("Conor"). So, in the current JVM, we have a List with three names. But the "backup" Sessions in the other JVMs in the cluster will only have two names.

The trigger to say "this attribute/session is dirty, please replicate it" is never fired. So we need to change our code to this:

HttpSession session = ...;
List names = (List) session.getAttribute("names");
// add someone else
names.add("Conor");
// trigger the replication of 'names' through cluster
session.setAttribute("names", names);

I know2 this is a real pain. YMMV depending on your Servlet container. But if you ever hope to run your Servlet app in a cluster, then it's a good idea to get into this habit:

If you modify (mutate) an object in a Session, put it back in the Session.

"Are you sure it works this way, Spud?"

Yep. Firstly, read this article by Abraham Kang (skip to the second bullet under "Session-storage guidelines"). It is really annoying that the Servlet specs don't even address this as a "MAY" behaviour; but if it's on developer.java.sun.com then it has a little more credibility than just "Matt said so".

But at the end of the day, this is going to be a vendor-specific policy. So, secondly, here are some relevant pointers to particular Servlet containers.

BEA Weblogic

I've done a lot of miles with BEA's container. Read bullet starting "Use setAttribute to Change Session State" on http://edocs.bea.com/wls/docs81/cluster/failover.html#1022133

Orion Application Server

I couldn't find a specific reference to Orion's policy in their documentation, but this mail list post is telling enough. It actually highlights an even more insidious potential bug when using <jsp:useBean ... scope="session"...>.

Macromedia's JRun

I found the following in this whitepaper (page 9):

Use setAttribute() to change session state
Session replication occurs only at the end of a request when setAttribute or removeAttribute is called on the current session.
...
Or ensure that setAttribute is called on a different object. It does not have to be on the Object that you changed.

It seems that JRun will replicate the whole session when just one attribute is set. I'm pretty sure that in Weblogic, only the changed attributes get replicated, not the whole session. So it's best to put every object you change back into the Session.

Caucho's Resin

Looks like, by default, Resin doesn't replicate values unless setAttribute() is called. But it also seems to have a "always-save-session" configuration parameter that replicates the whole session at the end of every request.

IBM's WebSphere

Same deal with WebSphere. It is set to only "Write changed" by default, but has a "Write all" too (includes a "Time-based write" option).

JBoss+Jetty

JBoss has a Distributed HTTPSession service, which as a HttpSessionSnapshotFrequency setting with values "never", "idle", "request" and "number of seconds" (from the JBoss Administration and Development Second Edition book for JBoss 3.0.4; my work bought it). Not exactly sure what the semantics are there, or how it works in later version of JBoss. If you really must know, just read the source.


1 ... Serializable, or something like it. See section 7.7.2 of the Servlet2.3 spec.

2 Oh baby! I know it.

  • Home
  • Blog