Recursive Makefile Example

I wanted to come up with generic way using only make (no autoconf/automake/etc) to solve the following problem:

Given a source directory with several subdirectories, have it so that running make in the top directory runs the make in all the subdirectories, potentially with dependencies so that things are built in the correct order. This should work for "make", "make install", "make clean", etc.

The make manual has a good description of this (in the "Phony targets" section of all places. It says

Another example of the usefulness of phony targets is in conjunction with recursive invocations of make (for more information, see Recursive Use of make). In this case the makefile will often contain a variable which lists a number of subdirectories to be built. One way to handle this is with one rule whose command is a shell loop over the subdirectories, like this:

     SUBDIRS = foo bar baz
     
     subdirs:
             for dir in $(SUBDIRS); do \
               $(MAKE) -C $$dir; \
             done

There are a few problems with this method, however. First, any error detected in a submake is not noted by this rule, so it will continue to build the rest of the directories even when one fails. This can be overcome by adding shell commands to note the error and exit, but then it will do so even if make is invoked with the -k option, which is unfortunate. Second, and perhaps more importantly, you cannot take advantage of make's ability to build targets in parallel (see Parallel Execution), since there is only one rule.

By declaring the subdirectories as phony targets (you must do this as the subdirectory obviously always exists; otherwise it won't be built) you can remove these problems:

     SUBDIRS = foo bar baz
     
     .PHONY: subdirs $(SUBDIRS)
     
     subdirs: $(SUBDIRS)
     
     $(SUBDIRS):
             $(MAKE) -C $@
     
     foo: baz

Here we've also declared that the foo subdirectory cannot be built until after the baz subdirectory is complete; this kind of relationship declaration is particularly important when attempting parallel builds.

The point about not using loops and instead making the subdirs themselves targets seems like a good idea and works for our "make" case. Well what about "make install"? We're already using the subdir names as targets for the normal "make", so we can't do the same trick exactly, but we can invent new target names

INSTALLDIRS = $(SUBDIRS:%=install-%)

install: $(INSTALLDIRS)
$(INSTALLDIRS): 
	$(MAKE) -C $(@:install-%=%) install

.PHONY: subdirs $(INSTALLDIRS)
.PHONY: install

In the first line, for each directory in SUBDIRS, we create a corresponding list of directories with "install-" prefixed (but could just as easily be a list of dirs). Next we have an install target that depends on all of them, and then we have a generic target that for each of them determines the the subdir name (by stripping off the "install-") and does a "make install" in that directory. Last we declare these made up targets as phony.

OK, so can we use this technique for other targets too? Sure, and while we're at it lets make our original solution for "make" use "build-" as a prefix for consistency (we are using .PHONY for all these targets, but it's probably a good idea to have the target names not correspond to any real file/dir).

So here's the end result, a sample Makefile for a ficticious project that has a few directories and some dependencies along with some other good ideas suggested by the GNU Coding Standards.

# example Makefile
#
SHELL = /bin/sh
INSTALL = /usr/bin/install
INSTALL_PROGRAM = $(INSTALL)
INSTALL_DATA = $(INSTALL) -m 644
include Makefile.conf

DIRS = dev ui utils doc
# the sets of directories to do various things in
BUILDDIRS = $(DIRS:%=build-%)
INSTALLDIRS = $(DIRS:%=install-%)
CLEANDIRS = $(DIRS:%=clean-%)
TESTDIRS = $(DIRS:%=test-%)

all: $(BUILDDIRS)
$(DIRS): $(BUILDDIRS)
$(BUILDDIRS):
	$(MAKE) -C $(@:build-%=%)

# the utils need the libraries in dev built first
build-utils: build-dev

install: $(INSTALLDIRS) all
$(INSTALLDIRS):
	$(MAKE) -C $(@:install-%=%) install

test: $(TESTDIRS) all
$(TESTDIRS): 
	$(MAKE) -C $(@:test-%=%) test

clean: $(CLEANDIRS)
$(CLEANDIRS): 
	$(MAKE) -C $(@:clean-%=%) clean


.PHONY: subdirs $(DIRS)
.PHONY: subdirs $(BUILDDIRS)
.PHONY: subdirs $(INSTALLDIRS)
.PHONY: subdirs $(TESTDIRS)
.PHONY: subdirs $(CLEANDIRS)
.PHONY: all install clean test

The above still seems too complicated to me for doing something that seems like such a common thing to do. If you have better ways to do it or other suggestions, please let me know.