Template-Based Code Generation with Apache Velocity, Part 2

As described in part one of this series, code generation typically uses a template engine to transform some kind of "model" into compilable code, given the formatting specified by a template. In this second part of the series, I will show how to use Velocity inside of a code generator that is based on a well-defined Internal Object Model (IOM). You will see two different strategies:

• Using the Internal Object Model as the data model for the Velocity transformation.
• Using a Platform-Specific Model (PSM) in order to balance the logic between generator and template.

I'm going to show how to implement both approaches and explain why the latter is a better solution.

Internal Object Model Generator

In my article "Writing a Code Generator in Java," I proposed the architecture for a code generator shown in Figure 1.

Figure 1. The IOM code generator architecture

You can see three different modules in the code generator:

• Importer: Reads the data model as input and translates it into a platform-independent internal format based on an object model.

• Internal Object Model: Is the platform-independent internal format and could be considered the core of the code-generator architecture. IOM contains a set of classes that make it easy to manipulate the information coming from the model to generate outputs. The structure of the object model is a very important issue -- when well designed, it can be a powerful representation, independent of the target technology (Java, C++, etc.), that can easily be converted into source code.

• Exporter: Accesses the IOM and takes the relevant information to generate code. As we will see, it can use templates in order to drive the generation process.

The class diagram in Figure 2 shows a basic Internal Object Model.

Figure 2. Class diagram for a basic Internal Object Model

As a convention, IOM classes start with the "IOM" prefix. The IOMClass class models a class. It has an aggregation with the IOMAttribute and IOMOperation classes because classes have attributes and operations. Since operations have parameters, an aggregation is modelled between the IOMOperation and the IOMParameter classes. The IOMAssociation class represents an association. It has two roles (start and end) that are modelled by the IOMRole class, which is associated to the IOMClass' class that represents the start or end class of the UML association. Although this design is very simple, it will give you an idea of how to implement a fully functional IOM. In a real context, each class should provide specific attributes that somehow will drive the code generation. For example, for the IOMClass class you would specify these attributes: name, stereotype, visibility, type (abstract, transient, persistent) and documentation. In addition, important aspects such as base classes, derived classes, exception classes, and packages must be supported by adding new classes and associations to the model.

In order to put Velocity in the game, you have to implement the exporter module in a way such that that it uses Velocity, as explained in part one of this series, to generate code. The exporter module is represented by this interface:

public interface Exporter {

public void initialize() throws Exception;
public void startClass(IOMClass cl) throws Exception;
public void endClass(IOMClass cl) throws Exception;
public void startAssociation(IOMAssociation association, 
  IOMClass currentClass) throws Exception;
public void endAssociation(IOMAssociation association, 
  IOMClass currentClass) throws Exception;
public void startOperation(IOMOperation operation)
  throws Exception;
public void endOperation(IOMOperation operation)
  throws Exception;
public void startAttribute(IOMAttribute attribute)
  throws Exception;
public void endAttribute(IOMAttribute attribute)
  throws Exception;
public void startParameter(IOMParameter parameter)
  throws Exception;
public void endParameter(IOMParameter parameter)
  throws Exception;
public void finalize() throws Exception;

}

The generator has also an important component, called navigator, that navigates through the Internal Object Model and invokes the methods of the Exporter interface. The idea is closely bound to the SAX parser. The navigator calls the startClass method each time a IOMClass object is processed, and in general, the startXxx method each time an instance of IOMXxx is processed. The navigator then invokes the endXxx methods when an instance of IOMXxx is finished processing. The following IOMNavigator class implements the navigator:

package com.codegenerator;

import java.util.*;

public class IOMNavigator {

private Exporter exporter = null;

public IOMNavigator(Exporter exporter) {
  this.exporter = exporter;
}

public void start() throws Exception {

  exporter.initialize();

  // Class navigation
  for (int i = 0;
       i < IOMController.getClasses().size();
       i++) {
    IOMClass cl =
        (IOMClass) IOMController.getClasses().get(i);
    exporter.startClass(cl);

    // Attribute navigation
    for (int j = 0; j < cl.getAttributes().size(); j++) {
      IOMAttribute attribute = 
        (IOMAttribute) cl.getAttributes().get(j);
      exporter.startAttribute(attribute);
      exporter.endAttribute(attribute);
    }

    // Associtaion Navigation
    ArrayList associations = cl.getMyAssociations();
    for (int j = 0; j < associations.size(); j++) {
      IOMAssociation ass =
          (IOMAssociation) associations.get(j);
      exporter.startAssociation(ass,cl);
      exporter.endAssociation(ass,cl);
    }

    // Operation navigation
    for (int k = 0;
         k < cl.getOperations().size();
         k++) {
      IOMOperation operation = 
       (IOMOperation) cl.getOperations().get(k);
      exporter.startOperation(operation);

      // Parameter navigation
      for (int t = 0;
           t < operation.getParameters().size();
           t++) {
        IOMParameter param = 
         (IOMParameter) operation.getParameters().get(t);
        exporter.startParameter(param);
        exporter.endParameter(param);
      }
      exporter.endOperation(operation);
    }

    exporter.endClass(cl);
  }

  exporter.finalize();
}

}

The constructor takes an Exporter object as input and stores it into the exporter private member. The start method starts the navigation processes; it first processes classes, then attributes, associations (using the getAllMyAssocitiations user method), operations, and parameters.

Now that it's clear how that framework works, we can concentrate on the implementation of the Exporter interface that uses Velocity as template engine.

Implementing the Exporter Using Velocity

As you have seen in part one, a VTL template references information belonging to a Velocity context. Therefore, in order to generate Java classes, you can insert instances of IOMClass into the context so that they can provide data such as the names of the classes, the attributes, etc. Proceeding like that, the transformation data model for the engine will be the Internal Object Model, as shown in Figure 3.

Figure 3. IOM as data model for Velocity

Thus, since the template knows the IOM, it will be possible extract the information for each class and generate the proper code.

Basically, we just need to implement the startClass method of the Exporter interface, assigning the current IOMClass instance to the Velocity context and invoking the transformation. The code is as follows:

public class IOMVelocityExporter implements Exporter {

private final static String TEMPLATE = "IOMTemplate.vm";
private GeneratorUtility utility = null;

public void initialize() throws Exception {
  Velocity.init();
  utility = new GeneratorUtility();
}

public void startClass(IOMClass cl) throws Exception {

  VelocityContext context = new VelocityContext();
  context.put("class", cl);
  context.put("utility", utility);

  Template template = Velocity.getTemplate(TEMPLATE);

  BufferedWriter writer =
    new BufferedWriter(new FileWriter(cl.getName()+".java"));

  template.merge(context, writer);
  writer.flush();
  writer.close();

  System.out.println("Class " + cl.getName() + " generated!");

}

/* Non-implemented methods not reported */

}

Note: The utility object is an instance of GeneratorUtility, containing the firstInUpperCase method, which will be used by the template. That way, it is also associated with the context.

As you'll see, techniques and syntax to generate code are delegated to the template. Before explaining how you can write that template, I'd like to clarify the way in which we want to implement the associations. In this example, I consider four different cases of relations between two classes. Figure 4 shows those cases.

Figure 4. Different types of multiplicity

As you can see, when the class A has a bidirectional association with just one class, B, the generator creates the setXxx/getXxx methods for both A and B. If the association is unidirectional, B won't have the setA and getA methods, because that navigation is not allowed. When the A class can be associated to more B classes, A contains addB and getAllB, while B contains setA and getA, if the association is bidirectional, and no method otherwise.

The template has the responsibility to judge, for each association, the right case, and then to generate the proper code according to Figure 4. To do that, it can use the isNavigable and multiplicity attributes of the IOMRole classes aggregated to IOMAssociation (see the detailed class diagram in Figure 5).

Figure 5. Detailed class diagram for IOMAssociation

Here is the template:

## IOMTemplate.vm
// Generated by IOMVelocityExporter

import java.util.*;

public abstract class $class.Name {

#foreach( $att in $class.Attributes ) 
#set($javaType = $utility.getJavaType($att.Type))
// $att.Name
private $javaType $att.Name;
public $javaType get$utility.firstToUpperCase($att.Name)() {
  return this.$att.Name;
} 
public void set$utility.firstToUpperCase($att.Name)($javaType $att.Name) {
  this.$att.Name = $att.Name;
}

#end

#set($associations = $class.getMyAssociations())
#foreach( $assoc in $associations )
#if($assoc.getStartRole().getClassInvolved()==$class)
  #set($role=$assoc.getEndRole())
#else
  #set($role=$assoc.getStartRole())
#end
#set($targetClass=$role.getClassInvolved().Name)
#set($paramClass=$utility.firstToLowerCase($targetClass))

#if($role.isNavigable())

// Association
// $class.getName() -- $targetClass
// Navigable   : $role.isNavigable()
// Multiplicity: $role.getMultiplicity()

  #if($role.Multiplicity=="1")

public abstract $targetClass get$targetClass();
public abstract void set$targetClass($targetClass $paramClass);
  #else

public abstract ArrayList getAll$targetClass();
public abstract void add$targetClass($targetClass $paramClass);
  #end
#end
#end  

}

The template creates attribute definitions and related accessor methods in the same way we saw in part one. Then it defines the way to create methods for handling the associations. First, it figures out whether the current class is the end or the start role of the association; eventually $targetClass will be a reference to the associated class, and $paramClass contains the identifier used for instances and parameters of the target class (if $targetClass="Order", then $paramClass="order").

Then the template checks whether or not the association is navigable. If it isn't, no method will be generated. If it is, and multiplicity is "1", the template defines the getXxx and setXxx methods. Otherwise, it defines the getAllXxx and addXxx methods, because the multiplicity is "*".

An Example

To provide a real example of code generation, let's consider the class diagram in Figure 6.

Figure 6. Class diagram for Order/OrderItem/Article

We can represent it with XML in the following way:

<?xml version="1.0" encoding="UTF-8"?>

<!-- order.xml -->

<Content>
    
<Class name="Order">
  <Attribute name="number" type="integer"/>
  <Attribute name="date" type="date"/>    
</Class>

<Class name="OrderItem">
  <Attribute name="number" type="integer"/>
  <Attribute name="quantity" type="integer"/>   
</Class>

<Class name="Article">
  <Attribute name="code" type="integer"/>
  <Attribute name="description" type="integer"/>    
</Class>
  
<Association name="">
  <Role class="Order" multiplicity="1" name="" 
    type="start" navigable="true"/>
  <Role class="OrderItem" multiplicity="*" name="" 
    type="end" navigable="true"/>
</Association>

<Association name="">
  <Role class="OrderItem" multiplicity="*" name="" 
    type="start" navigable="false"/>
  <Role class="Article" multiplicity="1" name="" 
    type="end" navigable="true"/>
</Association>
        
</Content>

Then we can create an importer that creates the Internal Object Model starting from that XML information -- inside of the source code for this article (in the "Resources" section below) you can find an implementation of such an importer. Finally, we can create a main program having three command-line parameters: the input file, an alias for the importer implementation, and an alias for the exporter implementation. Here's the code:

import com.codegenerator.velocity.*;

import java.util.*;

public class Generator {

public static void main(String args[]) throws Exception {

  if (args.length!=3) {
    System.out.println(
      "Syntax: Generator <input-file> <importer> <exporter>");
    System.exit(1);
  }

  Importer importer = null;
  if (args[1].equals("XML"))
    importer = new XMLImporter(args[0]);
  else {
    System.out.println("Importer " + args[1] +
      " non found!");
    System.exit(1);
  }

  Exporter exporter = null;
  if (args[2].equals("IOMVelocityExporter"))
    exporter = new IOMVelocityExporter();
  else if (args[2].equals("PSMVelocityExporter"))
    exporter = new PSMVelocityExporter();
  else {
    System.out.println("Exporter " + args[2] +
       " non found!");
    System.exit(1);
  }

  importer.start();
  IOMNavigator navigator = new IOMNavigator(exporter);
  navigator.start();

}

}

Note: We will see later what PSMVelocityExporter is.

Launching the application in this way:

java com.codegenerator.Generator order.xml XML IOMVelocityExporter

creates, by using IOMVelocityExporter, the Java classes indicated by order.xml: Order, OrderItem, and Article. The generator, for example, creates OrderItem.java as the following:

// Generated by IOMVelocityExporter

import java.util.*;

public abstract class OrderItem {

// number
private int number;
public int getNumber() {
  return this.number;
} 
public void setNumber(int number) {
  this.number = number;
}

// quantity
private int quantity;
public int getQuantity() {
  return this.quantity;
} 
public void setQuantity(int quantity) {
  this.quantity = quantity;
}
  
// Association
// OrderItem -- Order
// Navigable   : true
// Multiplicity: 1
   
public abstract Order getOrder();
public abstract void setOrder(Order order);
          
  
// Association
// OrderItem -- Article
// Navigable   : true
// Multiplicity: 1
    
public abstract Article getArticle();
public abstract void setArticle(Article article);
    
}

Note: The classes are defined as abstract because the implementations of the association methods have not been specified.

The Platform-Specific Model

Using the Internal Object Model as the input data model for Velocity makes, as we have seen, the Exporter implementation very straightforward. The downside of this approach is that all of the logic to access the internal model is delegated to the template. That's not so good, because a template language is less powerful that an imperative language like Java. Thus, you'll have a number of limitations in manipulating the data model. In production work, when the code to be generated is very complex, the template has to perform lots of checks and interpret the logic behind the design. As result, we get messy templates, which are very difficult to handle.

For this reason, many generators, along with the platform-independent model, also uses a Platform-Specific Model. It is written by using an imperative language but, unlike the internal model, the PSM knows about the target platform. It has the duty to break out all of the data into the fields required for the templates, which do little more than placement and formatting.

The exporter module I'm going to describe will use a PSM, which contains information from the Internal Object Model in a ready-to-use simplified format. The PSM will be the data model for the Velocity transformation; therefore, it will be associated to a Velocity context. Figure 7 shows that architecture.

Figure 7. Code generation with a Platform-Specific Model

Figure 8 shows the class diagram related to the PSM.

Figure 8. Class diagram for the Platform-Specific Model

The PSMClass class is aggregated to the PSMAttribute class. It has also two associations to the PSMAssociation class: singleAssociations and multiAssociations. This means that the template, without performing any check, will receive separate lists for single or multiple associations. The PSMAttribute class represents an attribute; it has a name and a type and also another member called upperName, which contains the name of the attribute with the first letter in upper case. That will prevent the template from invoking a context function for doing that operation in generating setXxx/getXxx methods. The PSMAssociation class contains the attribute targetClass (the target class name) and paramClass, which is the same as targetClass but with the first letter in lower case; this will be used as an identifier for instances and parameters of targetClass. You can find the implementation of the PSM classes inside of the source code for this article.

In order to place the PSM in the picture, you have to implement a new exporter that converts the IOM into the PSM and starts the transformation process using Velocity. This exporter, called PSMVelocityExporter, is implemented as follows:

package com.codegenerator.velocity;

import com.codegenerator.*;

import java.io.*;
import java.util.*;

import org.apache.velocity.app.*;
import org.apache.velocity.*;
import org.apache.velocity.exception.*;

public class PSMVelocityExporter implements Exporter {

private final static String TEMPLATE = "PSMTemplate.vm";
private PSMClass currentClass = null;;

public void initialize() throws Exception {
  Velocity.init();
}

public void startClass(IOMClass cl) throws Exception {
  currentClass = new PSMClass();
  currentClass.setName(cl.getName());
}

public void endClass(IOMClass cl) throws Exception {

  VelocityContext context = new VelocityContext();
  context.put("class", currentClass);

  Template template = Velocity.getTemplate(TEMPLATE);

  BufferedWriter writer =
    new BufferedWriter(new FileWriter(cl.getName() +
                      ".java"));

  template.merge(context, writer);
  writer.flush();
  writer.close();

  System.out.println("Class " + cl.getName() +
                     " generated!");

}

public void startAttribute(IOMAttribute attr)throws Exception {
  PSMAttribute at = new PSMAttribute();
  at.setName(attr.getName());
  at.setNameUpper(GeneratorUtility.firstToUpperCase(attr.getName()));
  at.setType(GeneratorUtility.getJavaType(attr.getType()));
  currentClass.addAttribute(at);
}

public void startAssociation(IOMAssociation association, 
  IOMClass sourceClass) throws Exception {

  IOMRole role = association.getStartRole();

  if (role.getClassInvolved()==sourceClass)
    role = association.getEndRole();

  if (role.isNavigable()) {

    PSMAssociation ast = new PSMAssociation();
    ast.setTargetClass(role.getClassInvolved().getName());                
    ast.setRefClass(GeneratorUtility.firstToLowerCase(
      role.getClassInvolved().getName()));

    if (role.getMultiplicity().equals("1"))
      currentClass.addSingleAssociation(ast);
    else
      currentClass.addMultiAssociation(ast);
  }
}

/* Non-implemented method not reported */


}

The currentClass data member is a reference to the PSMClass currently in process. The startClass method takes the name from the IOMClass in input and assigns it to currentClass. The endClass method will be invoked once the PSMClass is ready; therefore, it starts the transformation by using currentClass as the Velocity context. The startAttribute method creates a PSMAttribute from the IOMAttribute and assigns it to currentClass. The startAssociation method creates a PSMAssociation instance and performs all of the needed checks to judge whether or not the association is navigable and to find out the multiplicity. As you will see, those checks will no longer be inside of the template.

The template used to work with this exporter is implemented as follows:

## PSMTemplate.vm

// Generated by PSMVelocityExporter

import java.util.*;

public abstract class $class.name {

#foreach( $att in $class.Attributes ) 
// $att.Name
private $att.Type $att.Name;
public $att.Type get${att.NameUpper}() {
  return this.$att.Name;
} 
public void set${att.NameUpper}($att.Type $att.Name) {
  this.$att.Name = $att.Name;
}

#end
#foreach( $assoc in $class.SingleAssociations)

// Association
// $class.Name -- $assoc.TargetClass
public abstract $assoc.TargetClass get${assoc.TargetClass}();
public abstract void set${assoc.TargetClass}($assoc.TargetClass $assoc.paramClass);
#end
#foreach( $assoc in $class.MultiAssociations)

// Association
// $class.Name -- $assoc.TargetClass
public abstract ArrayList getAll${assoc.TargetClass}();
public abstract void add${assoc.TargetClass}($assoc.TargetClass $assoc.paramClass);
#end  

}

It is very simple and does not perform any checks. Since the data model is the PSM, the template can just retrieve all of the attributes, single associations, and multiple associations, and create the methods to handle them.

You can launch the generator using the PSMVelocityExporter like this:

java com.codegenerator.Generator order.xml XML PSMVelocityExporter

It creates the Order, OrderItem, and Article classes.

Conclusion

A real-world code generator must use templates. In these two articles, we have analyzed the huge benefits of using templates and provided examples with Java and Apache Velocity. In particular, in this second part, we saw how to integrate templates with a IOM-based code generator by using the IOM itself as data model for the transformation or resorting to a Platform-Specific Model. The second solution, which we can call a hybrid model, is better than the hard-coded model, because you don't have the formatting portion embedded as print statements in the generator code. It is also better than templates alone, because the PSM is written with an imperative language, which is better for doing complex model transformations and validations.

Resources

[1] Apache Velocity

[2] F. Aliverti-Piuri, "Bug Prevention with Code Generation: A J2EE Case Study," ONJava.com, March 2004.

[3] J. Herrington, "Code-Generation Techniques for Java," ONJava.com, September 2003.

[4] J. Herrington, Code Generation In Action, Manning, 2003.

[5] Code Generation Network

[6] G. Naccarato, "Writing a Code Generator in Java," JavaPro Online, March 2004.

[7] G. Naccarato, "Implement an IOM-Based Code Generator," JavaPro Online, March 2004.

[8] G. Naccarato, "Template-Based Code Generation with Apache Velocity, Part 1," ONJava.com, May 2004.

[9] Source code for this article

Giuseppe Naccarato has a degree in computer science and works as software developer for an IT company based in Glasgow (UK). His main interests are J2EE- and .Net-related technologies. Contact Giuseppe at http://www.giuseppe-naccarato.com.

Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.