Exorcising the Gradle Daemon

What is a build tool? A build tool compiles source code, creates a binary, and maybe runs tests. But if you are a Java developer, a build tool is a biblical being of pure evil and darkness: the Gradle Daemon.

For Java, build tools usually compile classes (via the JDK), zip up compiled classes/resources into a .jar (via JDK APIs), and run tests (via a framework such as JUnit or TestNG). All of the difficult aspects of build tools are effectively outsourced. So theoretically Gradle should be extremely simple right? Unfortunately, Gradle is one of the most over-engineered, overcomplicated, and overhyped tools that I’ve ever had to use.

Gradle is a product. The dark priests of Gradle are incentivized to make using Gradle as complicated and slow as possible in order to sell support and cloud builds. Ideally, you would completely exorcise the Gradle Daemon from your projects. You could replace Gradle with a simple, holy build tool such as a Java shebang build file. But due to the Gradle Daemon’s insidious influence and industry by-in, your company and coworkers might already be possessed by the Gradle Daemon. In that case, you alone will need to free your mind from the clutches of the Gradle Daemon.

Cleanse your mind of Gradle’s teachings. There is no “right way” to use Gradle. Gradle is a poorly-documented assortment of overloaded terms and functionality. If you need Gradle to do something and someone tells you “tHaT’S NoT tHE RiGHt wAY tO uSE GraDLE!”, know that they are possessed. They are true-believers. They accept complex builds in the vain hope that their builds will somehow be fast. Their foolish hearts are darkened. Your build tool should do what your project specifically needs and not conform to the whims and capricious standards of some arcane being and his unholy followers.

I once needed to change our build to upload results to Amazon S3. I wanted to upload these results with Gradle so I would not be reliant on another evil demon called Github Actions. This seemed straightforward: after the build passed or failed, run code to upload test results to S3. There is even a nice callback called buildFinished. One problem: that API does not work with configuration caching. Gradle’s satanic bible provides zero alternatives. Given Gradle’s design, it is effectively impossible to run a task or function on all failed and/or successful builds. This is a very common use-case, but Gradle doesn’t support it. Tasks can run in any order and any task could fail. I could not even use finalizedBy since I would need to apply it to every task in the build (and skip running on task success to avoid duplicate uploads). Instead I had to run my code as a different command after the build completed. In other words, I had to skirt the Gradle Daemon just to handle the simple task of uploading test results.

There is no right way to interact with a Daemon. However, there are effective ways to prevent the Daemon from completely destroying your codebase and productivity.

Languages

Gradle build files can be written in both Groovy and Kotlin; two evil options. Groovy is a dynamic language with a syntax somewhat like Java but with enough differences to encourage mistakes and mix ups. So you might say: “I’ll take Kotlin then.” You have fallen for a trap! Kotlin support has worse documentation. Kotlin must (slowly) compile before builds. Kotlin even allows broken patterns. In Kotlin, you can write code before the buildscript block even though the buildscript block always executes first. Kotlin gives Gradle even more dark powers, so it is better to simply avoid it. Should you use Groovy then? Not quite. You can also use Java with Gradle, but only for plugins. So I recommend using Java for plugins and Groovy for build scripts. This has the added benefit of making your build plugin code more potentially portable if you decide to completely exorcise Gradle in the future. And it minimizes code in Groovy which is a niche language popularized by Gradle itself.

Groovy

Gradle is much easier to contain when you understand the Groovy incantations that control it. Gradle uses Groovy closures¹ everywhere, so understanding them is critical.

tasks.named('test') { // Closure 1

    systemProperty "foo", "bar"
  
    doFirst { // Closure 2

        systemProperty "baz", "qux"

    }

}

afterEvaluate { // Closure 3

    tasks.register('hello') { // Closure 4

        doFirst { // Closure 5

          println("Hellope!")

        }

    }

}

dependencies { // Closure 6
    implementation "foo:bar:123"
}

tasks.register('helloAgain') { // Closure 7

    doFirst { // Closure 8

        println("Hellope Again!")

    }

}

I have found that translating Groovy or Kotlin code to Java was what finally allowed me to understand what was going on with Gradle.

project.getTasks().named("test", Test.class)
  .configure(testConfig -> { // Closure 1

    test.systemProperty("foo", "bar");

    test.doFirst(testConfigDoFirst -> { // Closure 2

        testConfigDoFirst.systemProperty("baz", "qux");

    });

});

project.afterEvaluate(projectAfterEvaluate -> { // Closure 3

    tasks.register("hello", DefaultTask.class)
      .configure(helloConfig -> { // Closure 4

        helloConfig.doFirst(helloConfigDoFirst -> { // Closure 5

            println("Hellope!");

        }); 

    });

});

// "Closure" 6
// Some evil is truly incomprehensible.
project.getDependencies().add("implementation", "foo:bar:123");


tasks.register("helloAgain", DefaultTask.class)
    .configure(helloConfig -> { // Closure 7

    helloConfig.doFirst(helloConfigDoFirst -> { // Closure 8

        println("Hellope Again!");

    });

});

So why does the Gradle Daemon torture us with these closures within closures within closures? And how can we understand when a closure will execute? The easy answer is that Gradle is a being of pure malice. It loves to spread discord and confusion.

Gradle Phases

However, we can comprehend this evil, and we must. Gradle effectively has 3 phases: initialization, configuration, and execution. During initialization, the build.gradle files and plugins run, mostly registering configuration closures for tasks. The configuration phase runs configuration closures. The execution phase finally runs doFirst closures, task code, and doLast closures.

So given the above example, the outermost code will run during initialization. The configure and afterEvaluate closures will run during the configuration phase. afterEvaluate will run after other configuration in the project and then the configuration it applies will run. Finally doFirst and task executions will run during the execution phase.

So if we run gradle test hello helloAgain, the closures will run in this order:

1. Closure 6
2. Closure 1
3. Closure 3
4. Closure 7
5. Closure 4
6. Closure 2
7. Closure 5
8. Closure 8

Understanding these closures is critical to preventing Gradle from crippling your build times. The daemon promotes the deadly sin: sloth. Your build logic must be as lazy as possible. Do not resolve dependencies or configuration except at task execution time. If you are calling resolve, get, or create earlier than the execution phase, you are (likely unnecessarily) slowing down the build. Use lazy methods like register and map as much as possible. Pass configurations to dependsOn or from instead of passing the resolved files. Follow the tips in this mini-guide to avoid writing Gradle code that is slow or unstable.

Plugins

Avoid plugins if you can. Write your build logic as custom tasks in build.gradle if possible. If you must write a plugin, minimize the configuration options. Do not provide extensibility or abstraction. Allow configuration of tasks directly rather than providing plugin/task extensions. Extensions are a bizarre plugin feature that decouples the configuration of a task or tasks from the task itself. This only adds more indirection to plugins and there is no harm in simply providing the same APIs on the tasks themselves. Simplify and minimize your plugins and tasks as much as possible.

Inputs and Outputs

In order to take advantage of caching, your tasks will need to configure inputs and outputs. Specifying task dependencies is not (always) enough if you want your build to use caching. Use methods like inputs#dir and outputs#files to tell Gradle which files and directories your task depends on and produces.

Dependencies

Like the snake in the garden, the Gradle Daemon will tempt you to eat the forbidden fruit. “It’s easy,” he whispers, “Just download a dependency to do it for you.” Now your build relies on the Internet, and your project is infected with someone else’s code. Dependencies are a liability. Gradle encourages adding them without thought, but dependencies should be heavily scrutinized. Don’t add a dependency unless you absolutely need it. And consider committing dependencies to your source control so your build won’t fail when the Internet is down.

Modularization

Gradle encourages modularity. You can split your project into many subprojects to enforce boundaries between your own APIs. Don’t do it. The fewer subprojects you have, the more understandable your build. Build a single artifact. Minimize source sets. Prune subprojects. Modularize using Java packages, not Gradle.

Parallelization

The Gradle Daemon is Legion. It can run your build tasks in parallel. Parallelizing your build can speed it up, but it comes with a cost: random resource usage spikes, irreproducible build ordering, and intermittent problems. Instead of parallelizing the build, consider parallelizing the individual tasks (for example run your tests in parallel) in order to speed up the build without introducing instability.

Upgrading

The Gradle Daemon will lie. It will promise new features. It will promise speed. If you believe these lies and upgrade an existing Gradle build, your build will break. Gradle is an ever-changing miasma of APIs, tasks, and plugins. It never gets faster. It never gets easier. Upgrading can only break things. Never do it.

Conclusion

I have used Gradle the wrong way. I have used symlinks to share code between build plugins and Gradle projects. I have used JavaExec to run code that could have been a Gradle plugin. I have written 500 lines of code in a single build.gradle file. If you have been pronounced guilty of not following Gradle’s best practices by the priestly caste of Gradle, I am here to absolve you. There’s no evil you can do to a Daemon. Use hacks. Use kludges. Use bodges. Gradle is evil because of its complexity, steep learning curve, and instability. As long as your hacks are less complex, more stable, and faster than Gradle, you have done no wrong.

“Resist the devil and he will flee from you.”

James 4:7