ONJava.com    
 Published on ONJava.com (http://www.onjava.com/)
 See this if you're having trouble printing code examples


The REST of the Web

by Jason R. Briggs
04/27/2005

My team has recently been working to refactor our existing, traditional web interface, in order to expose web services instead. In doing so, we've spun the old web front end out into a separate application, which calls the various web services to perform the critical work (most of this web code is automated, with just a few special cases so far).

Our web services are based on a RESTian framework, which has forced the refactoring down into more layers than just the web tier, since it is very much a paradigm shift in thinking. REST (Representational State Transfer) is a collection of design principles best embodied by the Web--stateless client/server protocol, a small set of well-defined operations, a universal syntax for resource identification, and hyperlinking between resources--and the exclusion of RPC-like ideas. The end result is, I believe, a much tighter codebase, and we've been able to chop out large chunks of now-unnecessary source; imagine an Army barber with a large pair of shears and a nervous, long-haired hippy, and you'll get an idea of the amount of virtual detritus we've managed to discard.

I've lately enjoyed a number of books and articles, such as Better, Faster, Lighter Java and The Pragmatic Programmer, which argue against complexity, or at the very least, urge developers to approach problems from a different point of view than the traditionally accepted enterprise architectures pushed by the big corporations. I believe a RESTian simplification fits comfortably within the precepts advanced by many of these texts. Of course, a significant percentage of the code reduction can be attributed to the benefit of hindsight you only get from refactoring, but certainly around 20-30 percent is a result of working within the constraints of a REST framework. We have added some complexity to the separated web application, but that's a small price to pay for the added flexibility you gain from decoupling (and considering the overall picture, we're still better off).

Related Reading

Better, Faster, Lighter Java
By Justin Gehtland

The refactoring has also forced me to reevaluate some past coding decisions, particularly in regard to following standard protocols (i.e., HTTP). For example, up until this project, I cannot recall ever implementing a servlet with more than doGet and doPost--certainly in the projects I've been involved with, I've never come across other source containing more than a bastardization of the fundamental uses of those two methods (by which I mean inserts, updates, and deletes performed by either doGet or doPost, or both). My feeling now is that this has unnecessarily complicated a number of my projects that would have been far more elegant, had I paid more attention to the standards.

In this article, I will present an alternative to basic web development taking into account some of these ideals: a RESTafarian adherence to the HTTP protocol, and using the flexibility proferred by Jython and Velocity in order to simplify servlet development.

Browser Support

One of the main problems we face with implementing more than just GET and POST is the lack of browser support. If you create a form with anything other than method="get" or method="post", most browsers will send an HTTP GET; this at least goes part way to explaining the historical abuse of the doGet and doPost servlet methods.

Luckily, browser support is relatively easy to subvert. One approach is to override the service method in a subclass of HttpServlet, and then have your servlets inherit from that class:

 public void service(HttpServletRequest request,
      HttpServletResponse response)
      throws IOException, ServletException {

    String override = 
        request.getParameter("method");
    if (!StringUtils.isEmpty(override)) {
      if (override.equals("delete")) {
        doDelete(request, response);
        return;
      }
      else if (override.equals("put")) {
        doPut(request, response);
        return;
      }
    }

    super.service(request, response);
  }

The service method looks for the parameter method, and if the value is delete or put, the requisite servlet methods are then called; otherwise, control is passed up to the HttpServlet superclass. This procedure will work; however, I also want to introduce more flexibility into my servlet development, so I'll be using a combination of Jython servlets and Velocity to create the web content instead.

Setting up the Web Server

One of the advantages that web-based scripting languages like PHP offer is a considerable flexibility in development--removing the necessity for the compile, package, deploy, and restart cycle of web development, which we've grown so accustomed to with Java. One alternative is Jython (based on Python), one of many scripting languages available for the JVM, which can provide a level of flexibility similar to PHP, without necessarily having to lose the power of Java.

The first step in this process is to download the Jython distribution. After downloading (at the time of this writing, Jython is distributed as a class file), you then run the Java class (e.g., java jython_21) to extract the Jython directory structure.

I use Jetty both during development and in production (just personal preference, but I've always found it much less hassle to play with than Tomcat). The next step is to copy jython.jar to Jetty's ext directory. Create a new directory for your web app (I'm calling mine test, so I've created $JETTY_HOME/webapps/test and $JETTY_HOME/webapps/test/WEB-INF directories accordingly), and then put the following web.xml (see the Resources section for this article's sample code) in the WEB-INF directory:

<?xml version = '1.0' encoding = 'UTF-8'?>
<!DOCTYPE web-app PUBLIC
"-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN" 
"http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
  <display-name>test</display-name>
  <description></description>
  
  <servlet>  
    <servlet-name>PyServlet</servlet-name>  
    <servlet-class>org.python.util.PyServlet</servlet-class>  
    <load-on-startup>1</load-on-startup>  
  </servlet>

  <servlet-mapping>
    <servlet-name>PyServlet</servlet-name>
    <url-pattern>*.py</url-pattern>
  </servlet-mapping>

</web-app>

Without going into too much detail about the workings of a web.xml file, since it's pretty much self-explanatory, this sets up a servlet provided in the Jython distribution, and then maps all URLs matching with an extension of .py to that servlet.

Jython also requires a number of supporting modules. At least with Jetty, you should be able to accomplish this by copying the entire Lib directory from the Jython distribution into the lib directory of your web app (in my case, $JETTY_HOME/webapps/test/WEB-INF/lib).

In the home directory of your web app, create the following Jython servlet in a file called test.py (see sample code in Resources):

from javax.servlet.http import HttpServlet

class test(HttpServlet):
    def doGet(self, request, response):
        w = response.getWriter()
        w.println("hello world")
        w.close()

Start Jetty (usually with the command java -jar start.jar from $JETTY_HOME), and point your browser at http://localhost:8080/test/test.py--you should see hello world displayed.

Sean McGrath, CTO of Propylon, gives an excellent tutorial on setting up Tomcat to run Jython servlets (as well as a basic introduction to Jython), if you want to delve more into the nitty-gritty detail. If you're new to Jython, there are a number of books available (O'Reilly has chapter 1 of Jython Essentials here) and the Jython website has a lot of useful information. You will also find the Python docs useful, bearing in mind that Jython is an implementation of version 2.1 of Python.

The Modified Servlet, Refactored

Now that we have Jython up and running on Jetty, we can deploy the modified servlet previously mentioned, so that PUT and DELETE are supported. However, rather than compiling a Java class, and sticking it somewhere in Jetty's classpath, we can recode as Python in a file called utils.py (see Resources):

from javax.servlet.http import HttpServlet

class enhancedservlet(HttpServlet):
    def service(self, request, response):
        override = request.getParameter('method')
        if override and override != '':
            getattr(self, 
              'do' + 
                override.lower().capitalize())\
                  (request, response)
        else:
            HttpServlet.service(self, request, 
              response)

There are a couple of things to note about the Jython version:

  1. I've made it slightly more generic than the original Java version--you can pass any method to override and it will attempt to handle it (if you pass in something meaningless it will, of course, fail).
  2. Python doesn't have a super keyword as such, so a call to the superclass, to handle cases where the method parameter is not passed, will be via HttpServlet.service(...).
  3. utils.py is accessible from the browser in this example--it throws an error, but it is on the browser path. If this doesn't fit into your worldview of acceptable web application etiquette, you can also move the file into the $YOUR_WEBAPP_DIR/WEB-INF/lib/Lib directory, and it will still be available to your servlets.


A simple subclass of enhancedservlet will look like this (in the file test2.py):

from utils import enhancedservlet

class test2(enhancedservlet):
    def doGet(self, request, response):
        w = response.getWriter()
        w.println("hello world again")
        w.close()

In case the ease of development isn't immediately obvious, change the println message and refresh your browser to see an instant change. In the words of Sean: Welcome to Java rapid development.

It's worth noting that you may need to restart your server in certain cases. Changes to dependent modules occasionally do not get picked up if there has been no change to the caller class/module. So if a servlet has not changed, but relies on a module that has been changed, this may require a web server restart.

To test that the override is working correctly, we can use the following servlet (which should be in a file named test3.py), again found in Resources):

from javax.servlet.http import HttpServlet

from utils import enhancedservlet

class test3(enhancedservlet):
    def doGet(self, request, response):
        w = response.getWriter()
        w.println("I'm a GET")
        w.close()

    def doPost(self, request, response):
        w = response.getWriter()
        w.println("I'm a POST")
        w.close()

    def doPut(self, request, response):
        w = response.getWriter()
        w.println("I'm a PUT")
        w.close()

    def doDelete(self, request, response):
        w = response.getWriter()
        w.println("I'm a DELETE")
        w.close()

And the HTML page (called methodtest.html):

<html>
<form action="test3.py" method="GET">
<p>Test GET <input type="submit" /></p>
</form>

<form action="test3.py" method="POST">
<p>Test POST <input type="submit" /></p>
</form>

<form action="test3.py" method="POST">
<input type="hidden" name="method" value="PUT" />
<p>Test PUT <input type="submit" /></p>
</form>

<form action="test3.py" method="POST">
<input type="hidden" name="method" value="DELETE" />
<p>Test DELETE <input type="submit" /></p>
</form>
</html>

While neither very awe-inspiring nor particularly edifying, these examples do give us a starting point for a REST-style web application, where we can create HTML forms that call the protocol-correct servlet methods depending upon the operation (more on that below).

Adding Velocity to the Mix

Velocity is a templating engine I've written about before, and including a template engine with Jython makes a big difference in simplifying development.

To use Velocity with Jython, we'll need to add a few more .jar files to Jetty's ext directory: log4j, Velocity and commons-collections. Also, Velocity will not work out-of-the-box with Jython-generated objects, so we need to customize the Velocity engine slightly. JythonUberspect is the customizer in question, and should be compiled into a .jar and also added to $JETTY_HOME/ext as well. To make your life easier, I've created the jyvel.jar file for you (which just contains JythonUberspect.java and the class files it produces when compiled) and can be found with the sample code. See Resources.

Once JythonUberspect has been included, we can modify enhancedservlet as follows, adding an init method:

from javax.servlet.http import HttpServlet

from org.apache.velocity.app import VelocityEngine
from org.apache.velocity.tools.\
  generic.introspection import JythonUberspect

class enhancedservlet(HttpServlet):
  ve = None

  def init(self, config=None):
      if config:
        HttpServlet.init(self, config)
      else:
        HttpServlet.init(self)

      if enhancedservlet.ve is None:
        ve = VelocityEngine()
        ve.setProperty(VelocityEngine.\
          FILE_RESOURCE_LOADER_PATH,
          self.getServletConfig().\
            getServletContext().getRealPath('/') \
              + '/WEB-INF/templates')
        ve.setProperty(VelocityEngine.\
          FILE_RESOURCE_LOADER_CACHE, 'true')
        ve.setProperty('input.encoding', 'UTF-8')
        ve.setProperty(VelocityEngine.\
          RUNTIME_LOG_LOGSYSTEM_CLASS,
          'org.apache.velocity.runtime.log.\
          SimpleLog4JLogSystem')
        ve.setProperty(\
          'runtime.log.logsystem.log4j.category', 
          VelocityEngine.getName())
        ve.setProperty(VelocityEngine.\
          UBERSPECT_CLASSNAME, 
          JythonUberspect.getName())
        ve.init()
        enhancedservlet.ve = ve

One point to note in this excerpt is that ve is a variable of the class, and therefore similar to a Java static. So rather than creating a new Velocity engine per servlet, we'll have one global engine. Templates are served from a templates directory in WEB-INF (i.e., ${YOUR_WEBAPP_DIR}/WEB-INF/templates). Keeping them in the WEB-INF directory keeps nosy surfers out of your template code. The completed enhancedservlet can be found in the sample code.

RESTified Servlets

Now I want to create a (reasonably) standards-compliant servlet, using the correct HTTP methods for my operations. I'll start with a simple screen to display a data list, and then a screen to display a single record in editable form.

list.vm (see Resources) is the template for displaying lists of my data:

<html>
<body>
<table border="1">
  #foreach ( $row in $rows )
    <tr>
    #foreach ( $col in $row.cols )
      <td>
        #if ( $col == $row.key )
        <a href="test5.py?key=${col}">
          ${col}</a>
        #else${col}#end</td>
    #end
    </tr>
  #end
</table>
<a href="test5.py?key=None">Add</a>
</body>
</html>

You'll notice that I have a $rows object (which is a list), and each $row object has attributes for columns and its key. If the column matches the key, then I create a link for editing that row using the column value.

edit.vm is the template for editing a row of data:

<html>
<head>
<body>
<form action="test5.py" method="POST">
<table border="1">
  #foreach ( $col in $obj.cols )
    <tr>
      <td>Column #${velocityCount}</td>
      <td>
        <input name="col${velocityCount}" 
          type="text" 
          #if ( $col == $obj.key )
          disabled="disabled"
          #end value="${col}" /></td>
    </tr>
  #end
</table>
<br />
<input type="hidden" name="key" 
          value="${obj.key}" />
<input type="hidden" name="method" 
          value="${method}" /> 
<input type="submit" value="Save" />
</form>
#if ( $method == 'POST' )
<form action="test5.py" method="GET">
<input type="hidden" name="key" 
          value="${obj.key}" />
<input type="hidden" name="method" 
          value="DELETE" /> 
<input type="submit" value="Delete" />
</form>
#end
<form action="test5.py" method="GET">
<input type="submit" value="Cancel" />
</form>
</body>
</html>

This displays the data columns, disabling the key value so it can't be edited. While I could use JavaScript to modify a form parameter for overriding the method, in this case I use multiple forms. The first gets a method parameter, which is passed from the servlet depending upon whether this is an existing record (therefore an UPDATE or doPost) or a new record (an INSERT or doPut). The second form--only displayed if this is an existing record, and therefore with the method set to POST--has the method override parameter set to DELETE, while the last is just a call back to the servlet with no parameters so we can display the list again.

The servlet (with the file name test5.py) looks like this, so far:

from javax.servlet.http import HttpServlet
from org.apache.velocity import VelocityContext
from utils import *

# a simple data object with a key and some columns

class dataobj:
    def __init__(self, key, cols):
        self.key = key
        self.cols = cols

global sequence
sequence = 1
        
# a map containing data objects
mydata = { }

# prepopulate with a few rows
for sequence in xrange(1, 4):
    mydata['row%s_key' % sequence] = 
        dataobj('row%s_key' % sequence, 
                  [ 'row%s_key' % sequence, 
                    'row%s_col2' % sequence, 
                    'row%s_col3' % sequence ])

class test5(enhancedservlet):

    def doGet(self, request, response):
        ctx = VelocityContext()
        
        key = request.getParameter('key')
        
        if key:
            t = enhancedservlet.ve.\
                getTemplate('edit.vm')
            if key != 'None':
                ctx.put('obj', mydata[key])
                ctx.put('method','POST')
            else:
                ctx.put('obj', dataobj('None', 
                    [ 'None', '', '' ]))
                ctx.put('method','PUT')
        else:
            t = enhancedservlet.ve.\
                getTemplate('list.vm')
            ctx.put('rows', mydata.values())
        
        response.setContentType(\
            'text/html;charset=UTF-8') 
        pw = response.getWriter()
        t.merge(ctx, pw)
        pw.close()

From the code above you can see that if the parameter key is present we use the template edit.vm; otherwise, we use list.vm. The Velocity context is populated with data depending upon whether key is an actual key value or set to None. In the case of None, we construct a blank data object and put that in the context. This much allows us to display a list of rows, and display a single row for editing.

Next I'll implement a doDelete method:

def doDelete(self, request, response):
    key = request.getParameter('key')
    if not key or key == '':
        response.sendError(\
            response.SC_BAD_REQUEST, 
            'no key specified')
        return
        
    if not mydata.has_key(key):
        response.sendError(response.SC_NOT_FOUND,
            '%s was not found' % key)
        return
        
    del mydata[key]

    wrappedrequest = modifiablerequest(request)
    wrappedrequest.delParameter('key')
    self.doGet(wrappedrequest, response)

This method performs some simple validation (checking for the presence of a key parameter, and checking for the existence of that key), before deleting the data from the map. modifiablerequest (also found in utils.py) is a subclass of HttpServletRequestWrapper, and provides set and delete parameter methods, so a request may be modified and then pushed on to another method for processing. (In the above example, we perform the delete, then pass the request on to doGet after deleting the parameter key, so that doGet displays the main data list.)

The doPut method again performs some simple validation (also checking for the existence of a key parameter), before creating a data object and then adding it to the map. Again, processing is passed on to doGet:

def doPut(self, request, response):
    key = request.getParameter('key')
    if key and key != 'None':
        response.sendError(\
            response.SC_BAD_REQUEST,
            'there is an existing key')
        return
        
    # generate a new key
    global sequence
    sequence = sequence + 1
    key = 'row%s_key' % sequence
        
    # create the data object
    data = dataobj( key, [ key, 
            request.getParameter('col2'), 
            request.getParameter('col3') ])
    
    mydata[key] = data
    
    wrappedrequest = modifiablerequest(request)
    wrappedrequest.delParameter('key')
    self.doGet(wrappedrequest, response)

Finally, the doPost method checks for the existence of a key, and updates the requisite data object, before calling doGet:

def doPost(self, request, response):
    key = request.getParameter('key')
    if not key or key == 'None' or \
            not mydata.has_key(key):
        response.sendError(\
            response.SC_BAD_REQUEST, 
            'invalid or missing key')
        return
        
    data = mydata[key]
    
    data.cols[1] = request.getParameter('col2')
    data.cols[2] = request.getParameter('col3')
    
    wrappedrequest = modifiablerequest(request)
    wrappedrequest.delParameter('key')
    self.doGet(wrappedrequest, response)

Figures 1 to 3 show the final result when running the Jython servlet under Jetty. Figure 1 shows the initial list of data rows provided by the doGet, Figure 2 shows the edit screen when adding a record, and Figure 3 shows the list screen again after the row has been added.

The List data screen
Figure 1. The list data screen

The Edit screen
Figure 2. The edit screen

Back to the List screen
Figure 3. Back to the list screen with a new row

Conclusion

This is a simple example, but hopefully gives you some idea of the flexibility of this approach. There is a clean delineation of operations and responsibilities within the servlet: Velocity gives you a nice separation of view and, combined with Jython, gives the flexibility of rapid web development. The amount of code you are required to write for a typical web app, particularly traditional GET/POST servlets handling a variety of different operations, can be prodigious, so sticking to the HTTP protocol does force you to cut to the essentials. But certainly, in my experience, simplifying your code to basic HTTP operations does not necessarily mean losing functionality for your end users, and it can help focus your efforts on the important fundamentals.

Resources

Jason R. Briggs is an architect/developer working in the United Kingdom.


Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.