Why Mock the Unix Make Utility?
There are many build systems, and even more uses for build systems (see [Powers], sections 11.10 and 11.11).
I have been using the
unix make utility when
developing R packages since 2012. But sometimes I get caught on a machine where
make is not available and where I am not entitled to install it
[This is a nice example of what restrictive software policies are good for:
you end up with a buggy imitation like fakemake instead of the
well established original.
You should not regulate software installations for programmers, unless you take
away their interpreters/compilers.]
This is why I wrote fakemake: to build an R package conditionally on the modification times of (file) dependencies without having to rely on external software. If you have any proper build system at hand: stick to it, do not use fakemake.
withr and knitr
Throughout this vignette I use R's temporary directory, often by using
withr::with_dir(tempdir(), …). Because this is a vignette and the codes are
examples. In real life, we would skip the temporary directory stuff.
This vignette is built using knitr, which itself uses sink(). As sink() is
central to fakemake for redirecting output to files in the make chain,
I have to disable some of knitr's output here and there.
Don’t worry, it’s just because knitr and fakemake both want to use sink()
exclusively and it only affects vignettes built with knitr.
Makelists
A makelist is fakemake's representation of a Makefile. It’s just a list of lists. Look at the minimal makelist provided by fakemake:
str(fakemake::provide_make_list("minimal", clean_sink = TRUE))## List of 4
## $ :List of 3
## ..$ target : chr "all.Rout"
## ..$ prerequisites: chr [1:2] "a1.Rout" "a2.Rout"
## ..$ code : chr "print(\"all\")"
## $ :List of 2
## ..$ target: chr "a2.Rout"
## ..$ code : chr "print(\"a2\")"
## $ :List of 3
## ..$ target : chr "a1.Rout"
## ..$ prerequisites: chr "b1.Rout"
## ..$ code : chr "print(\"a1\")"
## $ :List of 2
## ..$ target: chr "b1.Rout"
## ..$ code : chr "print(\"b1\")"Each sublist represents a Makefile’s target rule and has several items: at least a target and either code or prerequisites, possibly both. This makelist would still be a Makefile’s valid representation if target rule #3 with target "a1.Rout" had no (or an empty) code item.
Other possible target rule entries are:
-
alias: An alias to target that would be easier to remember and/or type (we will come back to his later).
-
sink: By default, all output of the code item is dumped (well, with R's
sinkfunction) into a file with the name given by target. If target should be created by the code you will want to redirect the output into sink (we will come back to his later). -
.PHONY: If set to TRUE, the target is rebuilt unconditionally every time the target is hit (trying to mock GNU make’s .PHONY-extension) (we will come back to his later).
Using fakemake
Suppose we would have a minimal makelist:
ml <- fakemake::provide_make_list("minimal", clean_sink = TRUE)We can visualize the makelist
(giving the root is optional, in this case it just makes a neater plot):
fakemake::visualize(ml, root = "all.Rout")
Building and Rebuilding
Now build the "all.Rout" target:
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))## [1] "b1.Rout" "a1.Rout" "a2.Rout" "all.Rout"We can see the files created:
show_file_mtime <- function(files = list.files(tempdir(), full.names = TRUE,
pattern = "^.*\\.Rout")) {
return(file.info(files)["mtime"])
}
show_file_mtime()## mtime
## /tmp/RtmpUIPXNi/a1.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/a2.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/all.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/b1.Rout 2023-08-15 23:48:00If we wait for a second and rerun the build process, we get:
# ensure the modification time would change if the files were recreated Sys.sleep(1) withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))
## NULLshow_file_mtime()
## mtime
## /tmp/RtmpUIPXNi/a1.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/a2.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/all.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/b1.Rout 2023-08-15 23:48:00Nothing changed. Good. Now, we change one file down the build chain:
fakemake::touch(file.path(tempdir(), "b1.Rout")) withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))
## [1] "a1.Rout" "all.Rout"show_file_mtime()
## mtime
## /tmp/RtmpUIPXNi/a1.Rout 2023-08-15 23:48:01
## /tmp/RtmpUIPXNi/a2.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/all.Rout 2023-08-15 23:48:01
## /tmp/RtmpUIPXNi/b1.Rout 2023-08-15 23:48:01Since a1.Rout depends on b1.Rout and all.Rout depends on a1.Rout, these targets get rebuilt while a2.Rout stays untouched.
Had we touched a1.Rout, b1.Rout would not have been rebuilt:
fakemake::touch(file.path(tempdir(), "a1.Rout")) withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))
## [1] "all.Rout"show_file_mtime()
## mtime
## /tmp/RtmpUIPXNi/a1.Rout 2023-08-15 23:48:02
## /tmp/RtmpUIPXNi/a2.Rout 2023-08-15 23:48:00
## /tmp/RtmpUIPXNi/all.Rout 2023-08-15 23:48:02
## /tmp/RtmpUIPXNi/b1.Rout 2023-08-15 23:48:01Forcing the Build
If you set the force option, you can force the target and all its prerequisites down the build chain to be built:
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, force = TRUE)))## [1] "b1.Rout" "a1.Rout" "a2.Rout" "all.Rout"If you want to force the target itself, but not all its prerequisites, set
recursive = FALSE:
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, force = TRUE,
recursive = FALSE)))## [1] "all.Rout"Faking the Build
If you don’t actually want to run the recipes but would rather like to know what
would happen if you ran the build chain (this mocks GNU make’s -n option),
you can set dry_run = TRUE:
file.remove(dir(tempdir(), pattern = ".*\\.Rout", full.names = TRUE))
## [1] TRUE TRUE TRUE TRUEwithr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, dry_run = TRUE)))## [1] "b1.Rout" "a1.Rout" "a2.Rout" "all.Rout"Note that no files have been created:
dir(tempdir(), pattern = ".*\\.Rout")
## character(0)So we recreate them now:
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))## [1] "b1.Rout" "a1.Rout" "a2.Rout" "all.Rout"dir(tempdir(), pattern = ".*\\.Rout")
## [1] "a1.Rout" "a2.Rout" "all.Rout" "b1.Rout"Using Aliases
If you find a target rule’s target too hard to type, you can use an alias:
i <- which(sapply(ml, "[[", "target") == "all.Rout") ml[[i]]["alias"] <- "all" withr::with_dir(tempdir(), print(fakemake::make("all", ml, force = TRUE)))
## [1] "b1.Rout" "a1.Rout" "a2.Rout" "all.Rout"This is pointless here, but _target_s might be files down a
directory tree like log/roxygen2.Rout when building R packages:
you might want to alias that target to roxygen
Diverting Output / Programmatically Creating a Target Rule’s Target
Target rule b1 dumps its output to b1.Rout:
cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")
## [1] "b1"Suppose it would programmatically create the target:
i <- which(sapply(ml, "[[", "target") == "b1.Rout") ml[[i]]["code"] <- paste(ml[[i]]["code"], "cat('hello, world\n', file = \"b1.Rout\")", "print(\"foobar\")", sep = ";") withr::with_dir(tempdir(), print(fakemake::make("b1.Rout", ml, force = TRUE)))
## [1] "b1.Rout"cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")
## hello, wo[1] "foobar"You end up with a broken target file, so you need to add a *sink*:
ml[[i]]["sink"] <- "b1.txt" withr::with_dir(tempdir(), print(fakemake::make("b1.Rout", ml, force = TRUE)))
## [1] "b1.Rout"Now you get what you wanted:
cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")
## hello, worldcat(readLines(file.path(tempdir(), "b1.txt")), sep = "\n")
## [1] "b1"
## [1] "foobar"We need sinks when the target’s code creates the target, for example when it builds a package’s tarball: we would want to get the output of building the tarball to be written to a file the path of which we specify via the target’s sink.
No Code Targets
Rule a1 has code
i <- which(sapply(ml, "[[", "target") == "a1.Rout") ml[[i]]["code"]
## $code
## [1] "print(\"a1\")"that prints "a1" into "a1.Rout":
cat(readLines(file.path(tempdir(), "a1.Rout")), sep = "\n")
## [1] "a1"If we remove that code and its output file and rerun
ml[[i]]["code"] <- NULL withr::with_dir(tempdir(), print(fakemake::make("a1.Rout", ml, force = TRUE)))
## [1] "b1.Rout" "a1.Rout"the file is still created (note that target rule b1 down the make chain is run
since we did not set recursive = FALSE ) but empty:
file.size(file.path(tempdir(), "a1.Rout"))## [1] 0