Linux DevCenter    
 Published on Linux DevCenter (http://www.linuxdevcenter.com/)
 See this if you're having trouble printing code examples


Advanced makefiles

by Jennifer Vesperman
02/14/2002

In this article, we analyze a fairly complicated makefile. This makefile was captured from actual use in the wild, and sections specific to that project have been removed for this article.

This is a companion article to Introduction to make, and it builds on the topics covered there. See Introduction for a simple makefile and a guide on how to use and construct simple makefiles.

Both the introduction and this advanced article focus on make as a tool for developers. Sysadmins, make can be useful to you, too! As we step through the makefile, think about how the techniques here can be applied to rebuilding a configuration script, or an installation script, or as an auto-updating tool.

Thanks to Mitch Davis for allowing me to use his file. The modified version won't run as presented because I've removed lines, but it will allow me to display various techniques. The original is in frequent use.

Header comments

# TODO: Add a section for installation

# Makefile for (the project)
# Version 3.0
# Jennifer Vesperman
# 14 August 2001
#
# Version 4.0
# Mitch Davis
# 10 January 2002

It's always useful to include headers in a makefile and to record the wish list of improvements. My version of the makefile was one of those quick-and-dirty jobs that we find ourselves doing.

Location definitions

# Set up the compiler directories
xmlc_j:=            /usr/local/lib/xmlc2.0.1/xmlc.jar
install_d:=         /usr/local/apps/jakarta-tomcat-3.2.2/webapps/example
servlet_d:=         $(tomcathome)/lib
class_d:=           $(install_d)/WEB-INF/classes
class_example_d=    $(class_d)/example
# (additional directory definitions snipped.)

The paths necessary for compilation are set up as variables. This allows them to be easily changed if the installation is moved--and you can use conditionals and shell functions to change things automatically for different machines. Conditionals and shell functions are discussed in more detail later in this article.

Using a conditional to define a machine-specific variable

ifeq ($(shell uname -n),install)
  # For compiling on install machine
  install_d:= /usr/local/jakarta-tomcat-3.2.2/webapps/example
endif

Recursively expanded variables

There are two forms of variable. Recursively expanded variables use an "=" between the name and the value and are evaluated at the time the variable is needed. They cannot be defined recursively, as this would create an infinite loop. (make checks for this and forbids this usage.) Functions will be calculated each time make needs the value of the variable.

Simply expanded variables

Simply expanded variables use a ":=" between the name and the value and are evaluated immediately when make encounters the variable. Recursive definition is fine, and the right-hand-side value will be the former value of the variable. Simply expanded variables are a feature of GNU make that may not exist in other versions of make.

Environment variables

# These exports are for xmlc, which is 
# a java program which calls javac.
export JAVAC:=  $(java_h)/bin/javac
export JAVA:=   $(java_h)/bin/java

Sometimes it's necessary to set environment variables within the makefile, usually when you have a project that must use a specific version of a compiler. Choose your variables carefully--changing the EDITOR variable will annoy your development team!

Make functions

# Set up the classpaths for Javac
classpath:= \
        $(cookie_d)/cookiebuster.jar \
        $(example_j) \
        $(hsqldb_d)/hsqldb.jar \
        $(xmlc_j) \
        $(xml_j) \
        $(jetty_d)/com.mortbay.jetty.jar

# Convert the spaces to colons.  This trick is from 
# the make info file.
empty:=
space:= $(empty) $(empty)
classpath:=     $(subst $(space),:,$(classpath))

The classpath variable is a standard, simply expanded variable, with its definition split over several lines with the new lines escaped. Mitch then uses a make function to replace the spaces separating paths with colons. He considers this to be a much more readable variable definition than a single line, colon-separated classpath. Don't use this if you have spaces inside the classpath.

Running Linux

Related Reading

Running Linux
By Matt Welsh, Matthias Kalle Dalheimer, Lar Kaufman

make has many useful built-in functions. The syntax of a function call is $(function arguments). Functions are evaluated when the surrounding construct is evaluated: functions in simply expanded variables are evaluated when make first encounters the variable; functions in recursively expanded variables are evaluated when make needs the value of the variable; functions in rules are evaluated when make runs that part of the rule.

Conditionals

# If there's already a CLASSPATH, put it on the front
ifneq ($(CLASSPATH),)
        classpath:=     $(CLASSPATH):$(classpath)
endif

# Re-export the CLASSPATH.
export CLASSPATH:=$(classpath)

Make has conditionals, often used when setting variables. Available conditionals are ifeq, ifneq, ifdef, and ifndef. The else clause is optional, and the arguments may be surrounded by brackets, apostrophes, or quote marks.

In the example, we could have used an ifdef rather than the ifneq to determine whether the classpath environment variable is defined.

The syntax used in this makefile is:

ifeq (arg1, arg2) 
	statement/s
else
	statement/s
endif

Shell functions and more make functions

Make is not an ideal tool for Java compilation. The Java compiler likes to resolve dependencies itself, which can cause make some confusion. When resolving dependencies, Java works from the source files and make works from the target. Java's structure invites the casual creation of new source files, and make is designed for new source files to be added manually.

In an ideal world, each new file would be added to the make script as part of a variable definition. This project is done in the real world, so make searches the source directories and constructs the list of targets automatically. Because of Java's idiosyncrasies and some aspects of our project, this requires four variables and a mix of shell and make functions.

In the previous version of this makefile, I generated the targets with an external shell script, exporting the list to a file as make variables, then importing the make variables into the makefile. Mitch improved my makefile so targets are generated in a much neater way.


# Search the directory containing the sources, and 
# generate a corresponding name for the generated 
# .class file. Make sure the classes in example/util/* 
# are compiled first.
CLASSFILES_view:=
  $(patsubst %.java,$(class_d)/%.class,
  $(shell find example/util -name '*.java'))

# Find the non-util, non-views sources.  Uses 
# -path -prune instead of grep -v, from an example 
# on the find(1) man page.
CLASSFILES_nonview:=    
  $(patsubst %.java,$(class_d)/%.class,
  $(shell find example -path example/util -prune -o 
  -path example/views -prune -o -name '*.java' -a -print))

# Find the classes generated by xmlc from the 
# .html files, except the ones in the mockup 
# directory.
CLASSFILES:=    
  $(CLASSFILES_view) $(CLASSFILES_nonview)
VIEWFILES:=     
  $(patsubst %.html,$(class_d)/%.class,$(shell find 
  example -path example/views/mockups -prune -o -name 
  '*.html' -a -print))

This makefile includes shell functions as well as standard make functions. The syntax for a shell function is $(shell command). This returns the output of the shell function (stripping new lines).

The patsubst function has the syntax $(patsubst pattern,replacement,text). It uses the percent symbol (%) the same way pattern rules do--as a string which matches in both the pattern and the replacement text. It searches the text for whitespace-separated words that match the pattern and substitutes the replacement for them.

Makefile variables as commands, and automatic variables

# How to compile the .html and .java files.
htmlcompile=$(XMLC) -d $(class_d) -class $(subst /,.,
  $(basename $<)) $<
javacompile=javac -sourcepath . -d $(class_d) 
  $(filter %.java,$?)

These are recursively expanded variables, calculated when the variable is actually needed. The variables need to be recursively expanded because they include automatic variables that rely on prerequisite lists and will be used as command strings in rules. (Automatic variables are explained in Introduction to make.)

$< and $? are the automatic variables. $< expands to the name of the first prerequisite. $? expands to a space-separated list of prerequisites that are newer than the target.

The $(filter %.java,$?) is necessary because when javacompile is used later in a rule, one prerequisite of the rule is the directory into which the Java files are compiled. The filter removes the directory, leaving only the .java files.

Note the use of make functions in the variable definition.

Pattern rules

$(VIEWFILES): $(class_example_view_d)/%.class: 
  example/views/%.html $(class_d)
        $(htmlcompile)

$(CLASSFILES): $(class_example_d)/%.class: 
  example/%.java $(class_d)
        $(javacompile)

The first line of these rules has three parts. These are a special type of GNU make rule called static pattern rules. There is a simple pattern rule described in Introduction to make. The syntax of a static pattern rule is:

targets: target-pattern: dependency-patterns	commands

A .class file in the $(class_example_view_d) directory can be created from a corresponding .html file in the example/views/%.html directory, but only if the .class file is part of the $(VIEWFILES) list. Use static pattern rules if you have multiple ways of converting file type .A to .B and the way you choose depends on the files being converted.

The critical part of a pattern rule is the percent symbol. It refers to the same stem for both the target and the prerequisite file.

Simple rules

# A few things Make needs to know
.SUFFIXES : .html .java .class
.PHONY : clean all show_classpath

# The primary target: make all
all: $(VIEWFILES) $(CLASSFILES)

$(class_d):
        mkdir $@

show_classpath:
        @echo Here is the CLASSPATH passed to javac:
        @echo $$CLASSPATH
# note the $$ expansion

clean:
        -rm -rf $(class_d)

The all target relies on the pattern rules. The other rules are simple rules in the format:

target: prerequisites
	command

Simple rules and phony rules are both described in the article Introduction to make.

Caveats and gotchas

Final words

This makefile demonstrates many of make's advanced features. When writing your own makefiles, start with something simple and add new features one at a time.

make can be used anywhere one file needs to be generated from another. Try experimenting with using make for configurations and update scripts, as well as for software development.

Further reading

Jennifer Vesperman is the author of Essential CVS. She writes for the O'Reilly Network, the Linux Documentation Project, and occasionally Linux.Com.


Return to the Linux DevCenter.

Copyright © 2009 O'Reilly Media, Inc.