Improving Makefile and general C++ project structure

Question

I am a single developer on a numerics project. As it is growing, I would like to improve

• the Makefile
• the general organisation of the project
• structuring of the files
• workflow

As it stands now I have a few classes (where a few depend on each other) and a main.cc. In total maybe 20 files. From that the executable is build via make and ./projectName creates data files, which are later processed with gnuplot script files. This whole process can also be compressed in a shell script run.sh.

I am working under Linux using gnu make and g++ compiler.

What bothers me is:

1. As I add new classes make takes more and more time.
2. Everything is build from scratch each time make runs, even if only one class file is changed.
3. Related to that I have seen others using a build/ directory where object files files are stored. I haven't used .obj files so far. Does that solve the problem of (2.) and if yes how do I create them using make?
4. A debug and release build would be nice too, but I have seen questions on that on Stack Overflow, so I'll get to that.

What I have is the following:

Dir Tree

ProjectName
|+data/ <-- data files generated by program
|+scripts/ <-- scripts that handle data processing
|~includes/
| |-headerClass1.h
| |-headerClass2.h
| |-...
| |-main.h
| |-parameters.h
|~src/
| |-main.cc
| |-class1.cc
| |-class2.cc
| |...
|-Makefile
|-tags
|-projectname <--executable

Makefile

workdir:=$(shell pwd)
SRCS = $(workdir)/src/main.cc \
 $(workdir)/src/class1.cc \
 $(workdir)/src/class2.cc \
 ...
INCLUDE = -I$(workdir)/include/\
 -I$(workdir)/src/
CTALL = $(workdir)/include/*\
 $(workdir)/src/*
CTAGS = ctags
DEL = /bin/rm -f
RESULT = projectname
CC = g++
LIBS= -lm -lboost_timer -lboost_filesystem -lboost_system
OPT = -O3 -Wall -ggdb -D__STDC_FORMAT_MACROS -std=c++0x -fipa-matrix-reorg
all: clean $(RESULT) ctags
$(RESULT): $(SRCS)
 $(CC) $(LIBS) $(OPT) $(INCLUDE) $(SRCS) -o $@
# Rules for generating the tags.
# #-------------------------------------
ctags: $(CTALL)
 $(CTAGS) $(CTALL)
clean:
 @echo "cleaning ..."
 $(DEL) $(RESULT)

Answer 1

Common Conventions

1. The C++ compiler is named CXX
2. What you call RESULT is usually named TARGET
3. In addition to SRC you probably need OBJ
4. Commands are usually replaced by uppercase version varaiable names that name themselves.
   • ie. RM not DEL

Building issues

It rebuilds everything every time for two reasons.

The all: rule performs a clean: before it starts. So obviously it has to rebuild everything:

all: clean $(RESULT) ctags
 # ^^^^^^ Get rid of this

RESULT is dependent on SRC and rebuilds the executable from scratch each time. What you need to do is depend on OBJ and then you only need to link the objects together to build the executable. Each object will have its own dependency and only be re-built if the source file changes.

Note: Any common commands should follow the conventions set by the implicit rules:

The implicit rule for linking is:

$(CC) $(LDFLAGS) n.o $(LOADLIBES) $(LDLIBS)

You have a slightly more complex case but I would thus make my link command:

$(RESULT): $(OBJ)
 $(CXX) $(LDFLAGS) -o $@ $(OBJ) $(LOADLIBES) $(LDLIBS) 

In your case you have a set of extra libs I would then add them as follows

LDLIBS += -lm -lboost_timer -lboost_filesystem -lboost_system
 # ^^^^ Notice the +=
 # This makes it easy to add future enhancements to the makefile

The default rule for building *.o files from C++ files is:

$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c

Your header files inclusion is part of the pre-processor stage so should be added to the CPPFLAGS macro.

CPPFLAGS += $(INCLUDE)

Now your object files will build as you expect.

But normally you don't want to build your object files into the same directory as your source files. This is because you can have different build versions. eg you can have a debug build and a release build. You should never mix and match object from different builds. In fact no compiler manufacturer makes any guarantees about object files built with different compiler flags. So any differences in compiler flags can potentially make the object files incompatible. As a result build your object files into different sub directories based on the type of build.

Personally I build release into a directory called release and debug into a directory called debug. I know very boring. I also suffix any targets with their type.

Say I am building the lib plop.

libplop.<version>.so // release version of plop
libplopD.<version>.so // debug version of plop
libplopS.<version>.so // Single threaded version (only build this if there is a specific difference for a single threaded version of the library that can be exploited (rare))
libplopSD.<version>.so // Single threaded debug version

This way I can install both normal and debug version of a library and explicitly link other code against them.

Anyway back to your make file. To achieve this I normally do

OBJ = $(patsubst %.cc,$(BUILD)/%.o, $(SRC))
BUILD ?= .
TARGET = projectname$(BUILD_EXT)
 # BUIlD_EXT is the suffix I place on executable to tell if it is debug/relese and version information
 # BUILD defines the build type. If this is not specified an unadulterated
 # build is done into the current directory (rather than debug/release)
 # You can do this by directory building target `make target`
## You can add build specific flags like this
## Set BUILD_EXT using the same technique.
CXX_BUILD_FLAGS_debug = -g
CXX_BUILD_FLAGS_release = -O3
CXXFLAGS += $(CXX_BUILD_FLAGS_$(BUILD))
all: debug ctags
 # by default build debug version
 # if you want release you must explicitly ask for it
debug:
 $(MAKE) BUILD=debug target
release:
 $(MAKE) BUILD=release target
target: $(TARGET)
$(TARGET): $(OBJ)
 $(CXX) $(LDFLAGS) -o $@ $(OBJ) $(LOADLIBES) $(LDLIBS) 
$(BUILD)/%.o: %.cc
 $(CXX) $^ -c -o $@ $(CPPFLAGS) $(CXXFLAGS)

Organization of files.

I normally put header and source files into the same directory. And I normally break these into directories based on libraries or executable.

But I add an install target into to my makefile that uploads the binary and required header files to a build directory for other projects so they can use it.

# Note this is not real makefile (pseudo make)
# It is designed to show intention not real make instructions.
install:
 $(generate_source_control_tag)
 $(CP) $(TARGET) $(BUILD)/bin # used for executables
 $(CP) $(TARGET) $(BUILD)/lib # used for libs
 $(CP) $(TARGET_INCLUDES) $(BUILD)/include/$(TARGET_BASE)/

This way if I am working on a project I can leave it (potentially in mid modification) and work on another library without the two being affected. The two will only ever interact after they have been installed into the build directory.

This means I also add the following too my makefile

 CPPFLAGS += -I$(BUILD)/include
 LDFLAGS += -L$(BUILD)/lib

Now if project B wants to use the library from project A. Then the convention becomes.

 #include "A/A_HeaderFile_1.h"
 LDLIBS += -lA

• Da Frenk: Hi Loki, thanks for the in-depth answer. I tried to implement you suggestions as good as I could, however i am not able to build my project. Could you have a look at my EDIT2 above? That would be great.
• Loki Astari: @DaFrenk: That is because your makefile is not in the directory with the source files (which is where I would normally place the make file). Thus the following OBJ = $(patsubst %.cc,$(BUILD)/%.o, $(SRC)) is not matching correctly. Here % will be matching $(workdir)/src/main and thus your output file will be named: $(BUILD)/$(workdir)/src/main.o

Answer 2

Note that some of the commands below may be specific to GNU Make - you haven't said what tools you're using (although I guess you're on Windows since you mention .obj files).

Rules and object files

Currently, you have $(RESULT): $(SRCS), so it has to re-run your whole (single) compile line every time anything in $(SRCS) changes. Cacheing the compiler output for each individual file can indeed save time here:

$(OBJS) = $(patsubst %.cc, %.obj, $(SRCS))
$(RESULT): $(OBJS)
 $(CC) $(LIBS) $(OPT) $(INCLUDE) -o $@ $^

would do this (it just links all the .obj files to form your executable).

Now you need to tell make how to build those .obj files, which you can either leave up to the implicit rule (setting CXXFLAGS appropriately) or explicitly with:

%.obj : %.cc
 $(CC) $(OPT) $(INCLUDE) -c -o $@ $^

---

Directory structure

Note that leaving your build objects and output data inside your project tree isn't ideal - specifically it can add noise to whatever SCM tool you use, risks adding .obj files to source code control, which you almost certainly don't want, and makes it generally harder than necessary to clean your source tree.

You can use the patsubst command to change the path as well as the suffix of your object files, eg.

$(OBJS) = $(patsubst src/%.cc, build/%.obj, $(SRCS))

---

Dependencies

One major thing you're missing is header file dependencies: if you change a .h file, your makefile should be able to figure out which .o files need re-building. I don't know your toolchain, but for example gcc can automate this.