Main entry point for running Shake build systems. For an example see the top of the module Development.Shake. Use ShakeOptions to specify how the system runs, and Rules to specify what to build. The function will throw an exception if the build fails.

To use command line flags to modify ShakeOptions see shakeArgs .

The default set of ShakeOptions .

Define an alias for the six type classes required for things involved in Shake Rule s. This alias is only available in GHC 7.4 and above, and requires the ConstraintKinds extension.

To define your own values meeting the necessary constraints it is convenient to use the extensions GeneralizedNewtypeDeriving and DeriveDataTypeable to write:

 newtype MyType = MyType (String, Bool) deriving (Show,Typeable,Eq,Hashable,Binary,NFData)

Define a pair of types that can be used by Shake rules. To import all the type classes required see Development.Shake.Classes.

Methods

storedValue :: key -> IO (Maybe value)Source

Define a set of rules. Rules can be created with calls to rule , defaultRule or action . Rules are combined with either the Monoid instance, or (more commonly) the Monad instance and do notation.

Instances

Like rule , but lower priority, if no rule exists then defaultRule is checked. All default rules must be disjoint.

Add a rule to build a key, returning an appropriate Action . All rules must be disjoint. To define lower priority rules use defaultRule .

Run an action, usually used for specifying top-level requirements.

 main = shake  shakeOptions  $ do
 action  $ do
 b <- doesFileExist  "file.src"
 when b $ need  ["file.out"]

This action builds file.out, but only if file.src exists. The action will be run in every build execution (unless withoutActions is used), so only cheap operations should be performed.

For the standard requirement of only need ing a fixed list of files in the action , see want .

Remove all actions specified in a set of rules, usually used for implementing command line specification of what to build.

The Action monad, use liftIO to raise IO actions into it, and need to execute files. Action values are used by rule and action .

Instances

Execute a rule, returning the associated values. If possible, the rules will be run in parallel. This function requires that appropriate rules have been added with rule or defaultRule .

Apply a single rule, equivalent to calling apply with a singleton list. Where possible, use apply to allow parallelism.

Write an action to the trace list, along with the start/end time of running the IO action. The cmd and command functions automatically call traced . The trace list is used for profile reports (see shakeReport ).

If an exception is raised by the Action , perform some IO .

After an Action , perform some IO , even if there is an exception.

Error representing all expected exceptions thrown by Shake. Problems when executing rules will be raising using this exception type.

Constructors

Instances

Options to control the execution of Shake, usually specified by overriding fields in shakeOptions :

 shakeOptions {shakeThreads =4, shakeReport =Just "report.html"}

The Data instance for this type reports the shakeProgress and shakeOutput fields as having the abstract type Function, because Data cannot be defined for functions.

Constructors

Instances

The current assumptions made by the build system, used by shakeAssume . These options allow the end user to specify that any rules run are either to be treated as clean, or as dirty, regardless of what the build system thinks.

These assumptions only operate on files reached by the current action commands. Any other files in the database are left unchanged.

Constructors

Instances

Run a build system using command line arguments for configuration. The available flags are those from shakeOptDescrs , along with a few additional make compatible flags that are not represented in ShakeOptions , such as --print-directory. If there are no file arguments then the Rules are used directly, otherwise the file arguments are want ed (after calling withoutActions ). As an example:

 main = shakeArgs  shakeOptions {shakeFiles  = "_make/", shakeProgress  = progressSimple } $ do
 phony  "clean" $ removeFilesAfter  "_make" ["//*"]
 want  ["_make/neil.txt","_make/emily.txt"]
 "_make/*.txt" *>  \out ->
 ... build action here ...

This build system will default to building neil.txt and emily.txt, while showing progress messages, and putting the Shake files in locations such as _make/.database. Some example command line flags:

• main --no-progress will turn off progress messages.
• main -j6 will build on 6 threads.
• main --help will display a list of supported flags.
• main clean will not build anything, but will remove the _make directory, including the any shakeFiles .
• main _make/henry.txt will not build neil.txt or emily.txt, but will instead build henry.txt.

A version of shakeArgs with more flexible handling of command line arguments. The caller of shakeArgsWith can add additional flags (the second argument) and chose how to convert the flags/arguments into rules (the third argument). Given:

 shakeArgsWith  opts flags (\flagValues argValues -> result)

• opts is the initial ShakeOptions value, which may have some fields overriden by command line flags. This argument is usually shakeOptions , perhaps with a few fields overriden.
• flags is a list of flag descriptions, which either produce a String containing an error message (typically for flags with invalid arguments, .e.g. Left "could not parse as int"), or a value that is passed as flagValues. If you have no custom flags, pass [].
• flagValues is a list of custom flags that the user supplied. If flags == [] then this list will be [].
• argValues is a list of non-flag arguments, which are often treated as files and passed to want .
• result should produce a Nothing to indicate that no building needs to take place, or a Just providing the rules that should be used.

As an example of a build system that can use either gcc or distcc for compiling:

import System.Console.GetOpt
data Flags = DistCC deriving Eq
flags = [Option "" ["distcc"] (NoArg $ Right DistCC) "Run distributed."]
main = shakeArgsWith  shakeOptions  flags $ \flags targets -> return $ Just $ do
 if null targets then want  ["result.exe"] else want  targets
 let compiler = if DistCC `elem` flags then "distcc" else "gcc"
 "*.o" *>  \out -> do
 need  ...
 cmd compiler ...
 ...

Now you can pass --distcc to use the distcc compiler.

A list of command line options that can be used to modify ShakeOptions . Each option returns either an error message (invalid argument to the flag) or a function that changes some fields in ShakeOptions . The command line flags are make compatible where possbile, but additional flags have been added for the extra options Shake supports.

Information about the current state of the build, obtained by passing a callback function to shakeProgress . Typically a program will use progressDisplay to poll this value and produce status messages, which is implemented using this data type.

Constructors

Instances

A simple method for displaying progress messages, suitable for using as shakeProgress . This function writes the current progress to the titlebar every five seconds using progressTitlebar , and calls any shake-progress program on the $PATH using progressProgram .

Given a sampling interval (in seconds) and a way to display the status message, produce a function suitable for using as shakeProgress . This function polls the progress information every n seconds, produces a status message and displays it using the display function.

Typical status messages will take the form of 1m25s (15%), indicating that the build is predicted to complete in 1 minute 25 seconds (85 seconds total), and 15% of the necessary build time has elapsed. This function uses past observations to predict future behaviour, and as such, is only guessing. The time is likely to go up as well as down, and will be less accurate from a clean build (as the system has fewer past observations).

The current implementation is to predict the time remaining (based on timeTodo ) and the work already done (timeBuilt ). The percentage is then calculated as remaining / (done + remaining), while time left is calculated by scaling remaining by the observed work rate in this build, roughly done / time_elapsed.

Set the title of the current console window to the given text. If the environment variable $TERM is set to xterm this uses xterm escape sequences. On Windows, if not detected as an xterm, this function uses the SetConsoleTitle API.

Call the program shake-progress if it is on the $PATH. The program is called with the following arguments:

• --title=string - the string passed to progressProgram.
• --state=Normal, or one of NoProgress, Normal, or Error to indicate what state the progress bar should be in.
• --value=25 - the percent of the build that has completed, if not in NoProgress state.

The program will not be called consecutively with the same --state and --value options.

Windows 7 or higher users can get taskbar progress notifications by placing the following program in their $PATH: https://github.com/ndmitchell/shake/releases.

The verbosity data type, used by shakeVerbosity .

Constructors

Instances

Get the current verbosity level, as set by shakeVerbosity . If you want to output information to the console, you are recommended to use putLoud / putNormal / putQuiet , which ensures multiple messages are not interleaved.

Write a message to the output when the verbosity (shakeVerbosity ) is appropriate. The output will not be interleaved with any other Shake messages (other than those generated by system commands).

Write a message to the output when the verbosity (shakeVerbosity ) is appropriate. The output will not be interleaved with any other Shake messages (other than those generated by system commands).

Write a message to the output when the verbosity (shakeVerbosity ) is appropriate. The output will not be interleaved with any other Shake messages (other than those generated by system commands).

Run an action with Quiet verbosity, in particular messages produced by traced (including from cmd or command ) will not be printed to the screen.

Execute a system command. Before running command make sure you need any files that are required by the command.

This function takes a list of options (often just [], see CmdOption for the available options), the name of the executable (either a full name, or a program on the $PATH) and a list of arguments. The result is often (), but can be a tuple containg any of Stdout , Stderr and Exit . Some examples:

 command_  [] "gcc" ["-c","myfile.c"] -- compile a file, throwing an exception on failure
 Exit  c <- command  [] "gcc" ["-c",myfile] -- run a command, recording the exit code
 (Exit  c, Stderr  err) <- command  [] "gcc" ["-c","myfile.c"] -- run a command, recording the exit code and error output
 Stdout  out <- command  [] "gcc" ["-MM","myfile.c"] -- run a command, recording the output
 command_  [Cwd  "generated"] "gcc" ["-c",myfile] -- run a command in a directory

Unless you retrieve the ExitCode using Exit , any ExitFailure will throw an error, including the Stderr in the exception message. If you capture the Stdout or Stderr , that stream will not be echoed to the console, unless you use the option EchoStdout or EchoStderr .

If you use command inside a do block and do not use the result, you may get a compile-time error about being unable to deduce CmdResult . To avoid this error, use command_ .

A version of command where you do not require any results, used to avoid errors about being unable to deduce CmdResult .

A variable arity version of command .

• String arguments are treated as whitespace separated arguments.
• [String] arguments are treated as literal arguments.
• CmdOption arguments are used as options.

To take the examples from command :

 () <- cmd  "gcc -c myfile.c" -- compile a file, throwing an exception on failure
 Exit  c <- cmd  "gcc -c" [myfile] -- run a command, recording the exit code
 (Exit  c, Stderr  err) <- cmd  "gcc -c myfile.c" -- run a command, recording the exit code and error output
 Stdout  out <- cmd  "gcc -MM myfile.c" -- run a command, recording the output
 cmd  (Cwd  "generated") "gcc -c" [myfile] :: Action  () -- run a command in a directory

When passing file arguments we use [myfile] so that if the myfile variable contains spaces they are properly escaped.

If you use cmd inside a do block and do not use the result, you may get a compile-time error about being unable to deduce CmdResult . To avoid this error, bind the result to (), or include a type signature.

Collect the stdout of the process. If you are collecting the stdout, it will not be echoed to the terminal, unless you include EchoStdout .

Constructors

Instances

Collect the stderr of the process. If you are collecting the stderr, it will not be echoed to the terminal, unless you include EchoStderr .

Constructors

Instances

Collect the ExitCode of the process. If you do not collect the exit code, any ExitFailure will cause an exception.

Constructors

Instances

A class for specifying what results you want to collect from a process. Values are formed of Stdout , Stderr , Exit and tuples of those.

Instances

Options passed to command or cmd to control how processes are executed.

Constructors

Instances

Execute a system command. This function will raise an error if the exit code is non-zero. Before running system' make sure you need any required files.

Deprecated: Please use command or cmd instead, with Cwd. This function will be removed in a future version.

Execute a system command with a specified current working directory (first argument). This function will raise an error if the exit code is non-zero. Before running systemCwd make sure you need any required files.

 systemCwd  "/usr/MyDirectory" "pwd" []

Deprecated: Please use command or cmd instead, with Stdout or Stderr. This function will be removed in a future version.

Execute a system command, returning (stdout,stderr). This function will raise an error if the exit code is non-zero. Before running systemOutput make sure you need any required files.

copyFile' old new copies the existing file from old to new. The old file is has need called on it before copying the file.

Read a file, after calling need .

A version of readFile' which also splits the result into lines.

Write a file, lifted to the Action monad.

A version of writeFile' which writes out a list of lines.

Write a file, but only if the contents would change.

Remove all empty directories and files that match any of the patterns beneath a directory. Some examples:

 removeFiles  "output" ["//*"]
 removeFiles  "." ["//*.hi","//*.o"]

This function is often useful when writing a clean action for your build system, often as a phony rule.

Remove files, like removeFiles , but executed after the build completes successfully. Useful for implementing clean actions that delete files Shake may have open for building.

Require that the following files are built before continuing. Particularly necessary when calling cmd or command . As an example:

 "//*.rot13" *>  \out -> do
 let src = dropExtension  out
 need  [src]
 cmd  "rot13" [src] "-o" [out]

Require that the following are built by the rules, used to specify the target.

 main = shake  shakeOptions $ do
 want  ["Main.exe"]
 ...

This program will build Main.exe, given sufficient rules.

This function is defined in terms of action and need , use action if you need more complex targets than want allows.

Define a rule that matches a FilePattern . No file required by the system must be matched by more than one pattern. For the pattern rules, see ?== .

 "*.asm.o" *>  \out -> do
 let src = dropExtension  out
 need  [src]
 cmd  "as" [src] "-o" [out]

To define a build system for multiple compiled languages, we recommend using .asm.o, .cpp.o, .hs.o, to indicate which language produces an object file. I.e., the file foo.cpp produces object file foo.cpp.o.

Note that matching is case-sensitive, even on Windows.

Define a set of patterns, and if any of them match, run the associated rule. See *> .

Define a rule to build files. If the first argument returns True for a given file, the second argument will be used to build it. Usually *> is sufficient, but ?> gives additional power. For any file used by the build system, only one rule should return True .

 (all isUpper . takeBaseName ) ?>  \out -> do
 let src = replaceBaseName  out $ map toLower $ takeBaseName out
 writeFile'  out . map toUpper =<< readFile'  src

Declare a phony action -- an action that does not produce a file, and will be rerun in every execution that requires it. You can demand phony rules using want / need . Phony actions are never executed more than once in a single build run.

Phony actions are intended to define command-line abbreviations. If you need a phony action in a rule then every execution where that rule is required will rerun both the rule and the phony action.

Infix operator alias for phony , for sake of consistency with normal rules.

Define a rule for building multiple files at the same time, a more powerful and more dangerous version of *>> .

Given an application test ?>> ..., test should return Just if the rule applies, and should return the list of files that will be produced. This list must include the file passed as an argument and should obey the invariant:

 forAll $ \x ys -> test x == Just ys ==> x `elem` ys && all ((== Just ys) . test) ys

As an example of a function satisfying the invariaint:

test x | takeExtension  x `elem` [".hi",".o"]
 = Just [dropExtension  x <.>  "hi", dropExtension  x <.>  "o"]
test _ = Nothing

Regardless of whether Foo.hi or Foo.o is passed, the function always returns [Foo.hi, Foo.o].

Define a rule for building multiple files at the same time. As an example, a single invokation of GHC produces both .hi and .o files:

 ["*.o","*.hi"] *>>  \[o,hi] -> do
 let hs = o -<.>  "hs"
 need  ... -- all files the .hs import
 cmd  "ghc -c" [hs]

However, in practice, it's usually easier to define rules with *> and make the .hi depend on the .o. When defining rules that build multiple files, all the FilePattern values must have the same sequence of // and * wildcards in the same order.

A type synonym for file patterns, containing // and *. For the syntax and semantics of FilePattern see ?== .

Match a FilePattern against a FilePath , There are only two special forms:

• * matches an entire path component, excluding any separators.
• // matches an arbitrary number of path components.

Some examples that match:

 "//*.c" ?==  "foo/bar/baz.c"
 "*.c" ?==  "baz.c"
 "//*.c" ?==  "baz.c"
 "test.c" ?==  "test.c"

Examples that don't match:

 "*.c" ?==  "foo/bar.c"
 "*/*.c" ?==  "foo/bar/baz.c"

An example that only matches on Windows:

 "foo/bar" ?==  "foo\\bar"

Returns True if the file exists.

Returns True if the directory exists.

Get the contents of a directory. The result will be sorted, and will not contain the entries . or .. (unlike the standard Haskell version). The resulting paths will be relative to the first argument.

It is usually simpler to call either getDirectoryFiles or getDirectoryDirs .

Get the files anywhere under a directory that match any of a set of patterns. For the interpretation of the patterns see ?== . All results will be relative to the FilePath argument. Some examples:

 getDirectoryFiles "Config" ["//*.xml"]
 -- All .xml files anywhere under the Config directory
 -- If Config/foo/bar.xml exists it will return ["foo/bar.xml"]
 getDirectoryFiles "Modules" ["*.hs","*.lhs"]
 -- All .hs or .lhs in the Modules directory
 -- If Modules/foo.hs and Modules/foo.lhs exist, it will return ["foo.hs","foo.lhs"]

If you require a qualified file name it is often easier to use "" as FilePath argument, for example the following two expressions are equivalent:

 fmap (map ("Config" </>)) (getDirectoryFiles "Config" ["//*.xml"])
 getDirectoryFiles "" ["Config//*.xml"]

Get the directories in a directory, not including . or ... All directories are relative to the argument directory.

 getDirectoryDirs "/Users"
 -- Return all directories in the /Users directory
 -- e.g. ["Emily","Henry","Neil"]

Return Just the value of the environment variable, or Nothing if the variable is not set.

Add extra information which rules can depend on. An oracle is a function from a question type q, to an answer type a. As an example, we can define an oracle allowing you to depend on the current version of GHC:

 newtype GhcVersion = GhcVersion () deriving (Show,Typeable,Eq,Hashable,Binary,NFData)
 rules = do
 addOracle  $ \(GhcVersion _) -> fmap fromStdout  $ cmd  "ghc --numeric-version"
 ... rules ...

If a rule calls askOracle (GhcVersion ()), that rule will be rerun whenever the GHC version changes. Some notes:

• We define GhcVersion with a newtype around (), allowing the use of GeneralizedNewtypeDeriving. All the necessary type classes are exported from Development.Shake.Classes.
• Each call to addOracle must use a different type of question.
• Actions passed to addOracle will be run in every build they are required, but if their value does not change they will not invalidate any rules depending on them. To get a similar behaviour using data stored in files, see alwaysRerun .
• If the value returned by askOracle is ignored then askOracleWith may help avoid ambiguous type messages. Alternatively, use the result of addOracle , which is askOracle restricted to the correct type.

As a more complex example, consider tracking Haskell package versions:

newtype GhcPkgList = GhcPkgList () deriving (Show,Typeable,Eq,Hashable,Binary,NFData)
newtype GhcPkgVersion = GhcPkgVersion String deriving (Show,Typeable,Eq,Hashable,Binary,NFData)
rules = do
 getPkgList <- addOracle  $ \GhcPkgList{} -> do
 Stdout out <- cmd  "ghc-pkg list --simple-output"
 return [(reverse b, reverse a) | x <- words out, let (a,_:b) = break (== '-') $ reverse x]
 --
 getPkgVersion <- addOracle  $ \(GhcPkgVersion pkg) -> do
 pkgs <- getPkgList $ GhcPkgList ()
 return $ lookup pkg pkgs
 --
 "myrule" *> \_ -> do
 getPkgVersion $ GhcPkgVersion "shake"
 ... rule using the shake version ...

Using these definitions, any rule depending on the version of shake should call getPkgVersion $ GhcPkgVersion "shake" to rebuild when shake is upgraded.

Get information previously added with addOracle . The question/answer types must match those provided to addOracle .

Get information previously added with addOracle . The second argument is not used, but can be useful to fix the answer type, avoiding ambiguous type error messages.

Always rerun the associated action. Useful for defining rules that query the environment. For example:

 "ghcVersion.txt" *>  \out -> do
 alwaysRerun 
 Stdout  stdout <- cmd  "ghc --numeric-version"
 writeFileChanged  out stdout

A type representing an external resource which the build system should respect. There are two ways to create Resource s in Shake:

• newResource creates a finite resource, stopping too many actions running simultaneously.
• newThrottle creates a throttled resource, stopping too many actions running over a short time period.

These resources are used with withResource when defining rules. Typically only system commands (such as cmd ) should be run inside withResource , not commands such as need .

Be careful that the actions run within withResource do not themselves require further resources, or you may get a "thread blocked indefinitely in an MVar operation" exception. If an action requires multiple resources, use withResources to avoid deadlock.

Instances

Create a finite resource, given a name (for error messages) and a quantity of the resource that exists. Shake will ensure that actions using the same finite resource do not execute in parallel. As an example, only one set of calls to the Excel API can occur at one time, therefore Excel is a finite resource of quantity 1. You can write:

 shake  shakeOptions {shakeThreads =2} $ do
 want  ["a.xls","b.xls"]
 excel <- newResource  "Excel" 1
 "*.xls" *>  \out ->
 withResource  excel 1 $
 cmd  "excel" out ...

Now the two calls to excel will not happen in parallel.

As another example, calls to compilers are usually CPU bound but calls to linkers are usually disk bound. Running 8 linkers will often cause an 8 CPU system to grid to a halt. We can limit ourselves to 4 linkers with:

 disk <- newResource  "Disk" 4
 want  [show i <.>  "exe" | i <- [1..100]]
 "*.exe" *>  \out ->
 withResource  disk 1 $
 cmd  "ld -o" [out] ...
 "*.o" *>  \out ->
 cmd  "cl -o" [out] ...

A version of newResource that runs in IO, and can be called before calling shake . Most people should use newResource instead.

Run an action which uses part of a finite resource. For more details see Resource . You cannot call apply / need while the resource is acquired.

Run an action which uses part of several finite resources. Acquires the resources in a stable order, to prevent deadlock. If all rules requiring more than one resource acquire those resources with a single call to withResources , resources will not deadlock.

Create a throttled resource, given a name (for error messages) and a number of resources (the Int ) that can be used per time period (the Double in seconds). Shake will ensure that actions using the same throttled resource do not exceed the limits. As an example, let us assume that making more than 1 request every 5 seconds to Google results in our client being blacklisted, we can write:

 google <- newThrottle  "Google" 1 5
 "*.url" *>  \out -> do
 withResource  google 1 $
 cmd  "wget" ["http://google.com?q=" ++ takeBaseName  out] "-O" [out]

Now we will wait at least 5 seconds after querying Google before performing another query. If Google change the rules to allow 12 requests per minute we can instead use newThrottle "Google" 12 60, which would allow greater parallelisation, and avoid throttling entirely if only a small number of requests are necessary.

In the original example we never make a fresh request until 5 seconds after the previous request has completed. If we instead want to throttle requests since the previous request started we can write:

 google <- newThrottle  "Google" 1 5
 "*.url" *>  \out -> do
 withResource  google 1 $ return ()
 cmd  "wget" ["http://google.com?q=" ++ takeBaseName  out] "-O" [out]

However, the rule may not continue running immediately after withResource completes, so while we will never exceed an average of 1 request every 5 seconds, we may end up running an unbounded number of requests simultaneously. If this limitation causes a problem in practice it can be fixed.

A version of newThrottle that runs in IO, and can be called before calling shake . Most people should use newResource instead.

Run an action without counting to the thread limit, typically used for actions that execute on remote machines using barely any local CPU resources. Unsafe as it allows the shakeThreads limit to be exceeded. You cannot call apply / need while the extra thread is executing.

Given a way of loading information from a file, produce a cached version that will load each file at most once. Using the cached function will still result in a dependency on the original file. The argument function should not access any files other than the one passed as its argument. Each call to newCache creates a separate cache that is independent of all other calls to newCache .

This function is useful when creating files that store intermediate values, to avoid the overhead of repeatedly reading from disk, particularly if the file requires expensive parsing. As an example:

 digits <- newCache  $ \file -> do
 src <- readFile file
 return $ length $ filter isDigit src
 "*.digits" *>  \x -> do
 v1 <- digits (dropExtension x)
 v2 <- digits (dropExtension x)
 writeFile'  x $ show (v1,v2)

To create the result MyFile.txt.digits the file MyFile.txt will be read and counted, but only at most once per execution.

A version of newCache that runs in IO, and can be called before calling shake . Most people should use newCache instead.