Make file example

The make utility automatically determines which pieces of a large program need to be recompiled, and issues commands to recompile them.

Most of the engineers who started working on projects in the middle or supporting the product which are in maintenance phase has less visibility to Make files. For them, make files are of least concern. I want to present Make file basics with simple examples here. I was in this situation in my previous job.  I am into software development for almost 5 years,   would like to explain the basics of makefile I learnt while creating new projects.

In any Make file, we see mainly these sections: Variable, targets.

Variable syntax:

NAME = value

Target syntax:

target … : prerequisitesrecipe
        …
        …

A target is usually the name of a file that is generated by a program.

A prerequisite is a file that is used as input to create the target.

A recipe is an action that make carries out. A recipe may have more than one command, either on the same line or each on its own line.

By default, Makefile targets are “file targets” – they are used to build files from other files. Make assumes its target is a file, and this makes writing Makefiles relatively easy.

However, sometimes you want your Makefile to run commands that do not represent physical files in the file system. Good examples for this are the common targets “clean” and “all”.These special targets are called phony and you can explicitly tell Make they’re not associated with files. In terms of Make, a phony target is simply a target that is always out-of-date, so whenever you ask make clean, it will run, independent from the state of the file system

.PHONY: clean all product-daemon subproduct-daemon iotest framework metastore

The below is an example of a makefile of a top directory in a product:

TARGET=production

include makefiles/$(TARGET).make
export CC
export CC_PERMISIVE
export CXX
export CXX_PERMISIVE
export OPTFLAGS
export CXXFLAGS
export LDFLAGS

PROTOC = protoc
GRPC_CPP_PLUGIN = grpc_cpp_plugin
GRPC_PYTHON_PLUGIN = grpc_python_plugin
GRPC_CPP_PLUGIN_PATH ?= `which $(GRPC_CPP_PLUGIN)`
GRPC_PYTHON_PLUGIN_PATH ?= `which $(GRPC_PYTHON_PLUGIN)`

PROTOS_PATH = ./protos
PROTOS_OUTPUT_CPP = ./protos/src
PROTOS_OUTPUT_PYTHON = ./build/python

PROTO_FILES = product_daemon_rpc.proto \
 subproduct_daemon_rpc.proto \
 metadb_kv.proto

PROTO_GEN_HEADERS = $(patsubst %,$(PROTOS_OUTPUT_CPP)/%, $(PROTO_FILES:.proto=.pb.h) $(PROTO_FILES:.proto=.grpc.pb.h))
PROTO_GEN_SOURCES = $(patsubst %,$(PROTOS_OUTPUT_CPP)/%, $(PROTO_FILES:.proto=.pb.cc) $(PROTO_FILES:.proto=.grpc.pb.cc))

all: common/common.a framework/framework.a meta-store/metastore.a product-daemon subproduct-daemon iotest

common/common.a:
 cd common && $(MAKE)

framework/framework.a:
 cd framework && $(MAKE)

meta-store/metastore.a: $(PROTO_GEN_HEADERS) $(PROTO_GEN_SOURCES)
 cd meta-store && $(MAKE) STATS=1

clean:
 cd common && $(MAKE) clean
 cd framework && $(MAKE) clean
 cd meta-store && $(MAKE) clean
 cd subproduct-daemon && $(MAKE) clean
 cd product-daemon && $(MAKE) clean
 cd utils/iotest && $(MAKE) clean
 rm -f $(PROTOS_OUTPUT_CPP)/*.cc
 rm -f $(PROTOS_OUTPUT_CPP)/*.h
 rm -f $(PROTOS_OUTPUT_CPP)/*.d
 rm -f $(PROTOS_OUTPUT_PYTHON)/*.py

$(PROTOS_OUTPUT_CPP)/%.pb.h $(PROTOS_OUTPUT_CPP)/%.pb.cc: $(PROTOS_PATH)/%.proto
 $(PROTOC) -I $(PROTOS_PATH) --cpp_out=$(PROTOS_OUTPUT_CPP) $<

$(PROTOS_OUTPUT_CPP)/%.grpc.pb.h $(PROTOS_OUTPUT_CPP)/%.grpc.pb.cc: $(PROTOS_PATH)/%.proto
 $(PROTOC) -I $(PROTOS_PATH) --grpc_out=$(PROTOS_OUTPUT_CPP) --plugin=protoc-gen-grpc=$(GRPC_CPP_PLUGIN_PATH) $<

product-daemon: $(PROTO_GEN_HEADERS) $(PROTO_GEN_SOURCES) common/common.a framework/framework.a meta-store/metastore.a
 cd product-daemon && $(MAKE)

subproduct-daemon: $(PROTO_GEN_HEADERS) $(PROTO_GEN_SOURCES) common/common.a framework/framework.a
 cd subproduct-daemon && $(MAKE)

iotest:
 cd utils/iotest && $(MAKE)

framework: framework/framework.a

metastore: metastore/metastore.a

.PHONY: clean all product-daemon subproduct-daemon iotest framework metastore
$ cat makefiles/production.make
CC=gcc
CC_PERMISIVE=gcc
CXX=g++
CXX_PERMISIVE=g++

OPTFLAGS=-g3 -ggdb -O3 -DNDEBUG
  • include makefiles/$(TARGET).make

    The include directive tells make to suspend reading the current makefile and read one or more other makefiles before continuing.

  • export CC 
    export CC_PERMISIVE 
    export CXX
    export CXX_PERMISIVE
    export OPTFLAGS
    export CXXFLAGS
    export LDFLAGS

    When make runs a recipe, variables defined in the makefile are placed into the environment of each shell. This allows you to pass values to sub-make invocations . By default, only variables that came from the environment or the command line are passed to recursive invocations. You can use the export directive to pass other variables. To pass down, or export, a variable, make adds the variable and its value to the environment for running each line of the recipe. The sub-make, in turn, uses the environment to initialize its table of variable values.

  • GRPC_CPP_PLUGIN_PATH ?= which $(GRPC_CPP_PLUGIN)

This is called a conditional variable assignment operator, because it only has an effect if the variable is not yet defined. This is equivalent to

ifeq ($(origin GRPC_CPP_PLUGIN_PATH), undefined)
  GRPC_CPP_PLUGIN_PATH = which $(GRPC_CPP_PLUGIN)
endif
  • PROTO_GEN_HEADERS = $(patsubst %,$(PROTOS_OUTPUT_CPP)/%,
     $(PROTO_FILES:.proto=.pb.h) $(PROTO_FILES:.proto=.grpc.pb.h))

A substitution reference substitutes the value of a variable with alterations that you specify. It has the form ‘$(var:a=b)’ (or ‘${var:a=b}’) and its meaning is to take the value of the variable var, replace every a at the end of a word with b in that value, and substitute the resulting string.

For example:

foo := a.o b.o c.o
bar := $(foo:.o=.c)

sets ‘bar’ to ‘a.c b.c c.c’. This case is equivalent to ‘$(patsubst %.o,%.c,$(foo))’

  • meta-store/metastore.a: $(PROTO_GEN_HEADERS) $(PROTO_GEN_SOURCES)
     cd meta-store && $(MAKE) STATS=1

    For generating target meta-store/metastore.a , make has to make sure the dependencies $(PROTO_GEN_HEADERS) and $(PROTO_GEN_SOURCES) are available. The recipe is “cd meta-store && make STATS=1”, This sub-make gets all the exported variables in its environment.

  • $(PROTOS_OUTPUT_CPP)/%.pb.h $(PROTOS_OUTPUT_CPP)/%.pb.cc: $(PROTOS_PATH)/%.proto
     $(PROTOC) -I $(PROTOS_PATH) --cpp_out=$(PROTOS_OUTPUT_CPP) $<

    Target is: A rule with multiple targets is equivalent to writing many rules, each with one target, and all identical aside from that. The same recipe applies to all the targets. In above case for each maching target, it makes sures that the corresponding prerequisite exits and corresponding rule is executed.

    $(PROTOS_OUTPUT_CPP)/%.pb.h $(PROTOS_OUTPUT_CPP)/%.pb.cc
  • Dependency:
    $(PROTOS_PATH)/%.proto

    Recipe:

    $(PROTOC) -I $(PROTOS_PATH) --cpp_out=$(PROTOS_OUTPUT_CPP) $<

    Make has some special variables called, automatic variables. These variables have values computed afresh for each rule that is executed, based on the target and prerequisites of the rule. The widely used ones are:

    • $@ The file name of the target of the rule.
    • $%The target member name.

Example1:

foolib(hack.o) : hack.o
        ar cr foolib hack.o

foo.a(bar.o) then ‘$%’ is bar.o and ‘$@’ is foo.a. ‘$%’ is empty when the target is not an archive member.

  • $< The name of the first prerequisite.
  • $^The names of all the prerequisites

Example2:

all: library.cpp main.cpp

In this case:

  • $@ evaluates to all
  • $< evaluates to library.cpp
  • $^ evaluates to library.cpp main.cpp

 

common directory  makefile:

CXX?=g++
CC?=gcc
ASM?=g++
OPTFLAGS?=-g3 -ggdb -O0

PACKAGES=libconfig++ libglog
LIBS=-pthread -lgtest

CXXFLAGS+=-std=c++14 -MMD -Wall -I. -I../init -I/usr/local/include `pkg-config --cflags ${PACKAGES}` $(OPTFLAGS)
CFLAGS+=-MMD -Wall $(OPTFLAGS)
LDFLAGS+=-L/usr/local/lib `pkg-config --libs ${PACKAGES}` ${LIBS}

ASM_SRC=codeword_utils.o
CRC_SRC=crc.o

OBJECTDIR=build

ASM_OBJS=$(patsubst %,$(OBJECTDIR)/%,$(ASM_SRC))
CRC_OBJS=$(patsubst %,$(OBJECTDIR)/%,$(CRC_SRC))
VPATH+=.

SRC=$(CRC_OBJS) $(ASM_OBJS)

all: codeword_utils mpmcq

codeword_utils: $(CRC_OBJS) $(ASM_OBJS)
 ar rcs common.a $(SRC)

mpmcq:
 cd mpmcq && $(MAKE)

$(OBJECTDIR)/%.o: %.cpp
 $(CXX) $(CXXFLAGS) -c -o $@ $<

$(OBJECTDIR)/%.o: %.s
 $(ASM) $(CXXFLAGS) -c -o $@ $<

-include $(OBJECTDIR)/*.d

clean :
 rm -f $(OBJECTDIR)/*.o $(OBJECTDIR)/*.d common.a codeword_utils
 cd mpmcq && $(MAKE) clean

.PHONY: clean all codeword_utils mpmcq

most of the above sections are already discussed. The new ones are:

  • VPATH+=.

    The value of the make variable VPATH specifies a list of directories that make should search. Most often, the directories are expected to contain prerequisite files that are not in the current directory; however, make uses VPATH as a search list for both prerequisites and targets of rules.

  • -include $(OBJECTDIR)/*.d

    This acts like include in every way except that there is no error (not even a warning) if any of the filenames (or any prerequisites of any of the filenames) do not exist or cannot be remade.

meta-store makefile

CXX?=g++
CC?=gcc
CXX_PERMISIVE?=g++
OPTFLAGS?=-g3 -ggdb -O0

PACKAGES=libconfig++

CXXFLAGS+=-std=c++14 -MMD -Wall -I. -I/usr/local/include -Iapi -Istore -Icluster -Itransactions -Igen-cpp2 -I../protos/src `pkg-config --cflags ${PACKAGES}` $(OPTFLAGS)
CXXFLAGS+= -I transactions/meta_keys -Itransactions/sql -Itransactions/key-value

ifeq ($(STATS), 1)
CXXFLAGS+= -D_ENABLE_STATS_
endif

CFLAGS+=-MMD -Wall $(OPTFLAGS)

SRCPROTOS = metadb_kv.pb.o

VPATH+=api::store::cluster::transactions::tests::gen-cpp2
VPATH+=../protos/src
VPATH+=transactions/sql \
 transactions/key-value

SRC = meta1.o \
 meta2.o \
 

THRIFT_CPP_FILES = $(wildcard gen-cpp2/*.cpp)
THRIFT_CPP_FILES := $(filter-out gen-cpp2/non-shared_constants.cpp, $(THRIFT_CPP_FILES))
SRC += $(patsubst gen-cpp2/%.cpp,%.o,$(THRIFT_CPP_FILES))

SRC += $(SRCPROTOS)

OBJECTDIR=build

OBJS=$(patsubst %,$(OBJECTDIR)/%,$(SRC))

all: db test

db: $(OBJS)
 ar rcs metastore.a $(OBJS)

test: db
 cd tests && $(MAKE)

$(OBJECTDIR)/%.o: %.cpp
 $(CXX) $(CXXFLAGS) -c -o $@ $<

$(OBJECTDIR)/%.o: %.cc
 $(CXX) $(CXXFLAGS) -c -o $@ $<

-include $(OBJECTDIR)/*.d

clean :
 rm -f $(OBJECTDIR)/*.o $(OBJECTDIR)/*.d
 rm -f metastore.a
 cd tests && $(MAKE) clean

.PHONY: clean all db test
  • THRIFT_CPP_FILES = $(wildcard gen-cpp2/*.cpp)

    Wildcard expansion does not happen when you define a variable.However, if you use the value of objects in a target or prerequisite, wildcard expansion will take place there. To do the wildcard expansion we use the keyword wildcard.

Advertisements