SBT with an external build tool

The SBT does a great job in building many Scala projects. One of the key features of SBT is to create custom build steps very easily. We will show this by an hopefully very generic example, where the specific use cases should be easy to extract.
Even with the great number of plugins, SBT doesn't provide everything needed. Especially when it comes to other languages with their own build tool or file formats that aren't part of a normal Scala project.
To have a concrete example I will assume a Play application where all images are created by Gimp are placed inside src/main/gimp/. All Gimp files (*.xcf) should be converted to PNG-files during the compile step so that a browser can display them. In addition we also like to test that the images named *.icon.xcf are squares.
Both tasks are very easy to archive by using the tools xcf2png and xcfinfo. On the following lines I will explain how to integrate booth tools into SBT and visit thereby useful concepts of SBT and explain them.
Both tools are only examples. Other command line tools like make, NPM, or custom script could be integrated the same way.
The complete result is available at Github.

Creating custom tasks

At first lets create an empty task definition. Later on we will fill the place holder printlns with an actual implementation.

val checkGimpTools = taskKey[Unit]("Check the commandline tools xcf2png and xcfinfo")
val compileImages = taskKey[Unit]("Converts Gimp files into png assets")
val testImages = taskKey[Unit]("Check size of images")
checkGimpTools := println("check images Gimp tools")
compileImages := println("convert images")
testImages := println("check images")

Here we see an important concept of SBT: Keys are things on it's own, they can exists without a concrete meaning. This allows to give keys different definitions in different contexts.
There are also different kinds of keys. A TaskKey does something and should be evaluated every time it is called. Another very common kind of key is SettingKey. They will be only once evaluated during the initialization of SBT. All keys have a type, e.g. scalaVersion is a SettingKey[String]. Our tasks above won't return nothing, just do something.
More to read: SBT Doc

Calling external command line tools

At first we like to make sure that all required tools are installed. To do that we just want to start the tools with the command line parameter --version and see if we get an answer. SBT has an already build mechanism to call and handle command line processes like a bash. It's very similar to the Scala Process DSL. For what we want to archive, we need only Process(<command> :: <arg1> :: <arg2> :: ... :: Nil).!!. The method !! returns the output of program as String and throws an exception if the program couldn't be found or returns with a none zero value.

val checkGimpTools = taskKey[Unit]("Check the commandline tools xcf2png and xcfinfo")
checkGimpTools := {
  val missing = Seq("xcf2png", "xcfinfo") filter { name =>
    scala.util.Try {
      Process(name :: "--version" :: Nil).!! == ""
    } getOrElse true
  }
  missing foreach { m =>
    println(s"Command line tool $m is missing")
  }
  assert(missing.isEmpty, "Required command line tools are missing")
}

For those more used to the command line there are also more DSL-ish constructs to invoke an external tool: The Process word is optional, (name :: "--version" :: Nil).!! would be absolutely sufficient. Even shorter is s"$name --version".!!, but you need to be sure there is no white space within your parameters. Keep in mind that embedded DSL is sometimes hard to read. I recommend to use this style only if you use command line tools very often from within SBT.
For more advanced stuff see External Processes in the SBT documentation.

Working with files

Now that we checked that all necessary tools are available on the building machine, we would like to use them. A quick look into the man-page of xcf2png shows that it handles only a single file per call. So we need to traverse the files in the source directory by ourselves. SBT comes with an rich embedded DSL for file handling that works out of the box.

val compileImages = taskKey[Unit]("Converts Gimp files into png assets")
compileImages := {
  val sourceFolder = sourceDirectory.value / "main" / "gimp"
  val sourceFiles = sourceFolder ** "*.xcf"
  val targetFolder = target.value / "web" / "public" / "main" / "images"
  val toPngExtension = (f: File) =>
    file(f.getAbsolutePath.replaceAll("xcf$", "png"))
  val mappings = sourceFiles pair (toPngExtension andThen rebase(sourceFolder, targetFolder))
  println(s"Converting ${sourceFiles.size} gimp files...")
  mappings.foreach { case (sourceFile, targetFile) =>
    targetFile.getParentFile.mkdirs()
    Process("xcf2png" :: sourceFile.getAbsolutePath :: "--output" :: targetFile.getAbsolutePath :: Nil).!
  }
}

At first we take the sourceDirectory (the /src folder not /public nor /app) go to the sub folder main/gimp and retrieve all files within the folder and sub-folders with the extension *.xcf. In the next step we map all source files to a corresponding target file path. Since we want to keep the relation between the source and the target file we use pair instant of map. Finally we call xcf2png for each file to generate the PNG files.
At this stage we already can use SBT to generate PNG files, but we need to call commands explicitly, which is tedious and error-prone since we could forget to call compileImages after we updated a Gimp file.
More on working with files: http://www.scala-sbt.org/release/docs/Mapping-Files.html

Wiring the task into the workflow

Finally we link compileImages with checkGimpTools as a dependency and as a dependency for the predefined compile tasks. Since the compile task has different meanings, a different context like test:compile we need to specify the scope Compile here.

compileImages <<= compileImages dependsOn checkGimpTools
compile in Compile <<= compile in Compile dependsOn compileImages

Conclusion

What we have so far? We integrated two external tools into our build and feed them with the desired files of our project. This was done with a total of 30 lines of code SBT-configuration, where the majority actually do stuff (20 lines) and 10 lines for the integration into SBT.
In this blog-post we left the testImages unimplemented. In a later blog-post we will fix this and address some other shortcomings of our current implementation: continues run in play, better output, a fast incremental compilation and a better organization of the build configuration.