Make build depend on `bufGenerate`

[remoun]

We use buf generate for generating code from protobufs. I've been integrating this step in build scripts with:

// Add a task dependency for compilation
tasks.named("compileJava").configure { dependsOn("bufGenerate") }
tasks.named("compileKotlin").configure { dependsOn("bufGenerate") }

It would be great if the buf plugin did this automatically, perhaps with a config flag? e.g. buf { generateInBuild = true }

[andrewparmet]

Just thinking out loud for a moment: the way to do this is probably via implicit dependencies on task output. It looks like that's how the protobuf-gradle-plugin does it. This would work nicely as part of a larger effort to use proper Gradle task APIs instead of doFirst all over the place.

[andrewparmet]

The first step - using custom task classes - is complete: #94.

[andrewparmet]

Another problem is that there's no great way to know from the buf.gen.yaml which generators are being used and their target directories, and therefore which tasks ought to depend on buf generate and where the output should be for the compile task's new source directory.

Support for this might require a proper config DSL that generates a YAML itself, which I was hoping to avoid.

[remoun]

Why not add it to the CompileJava and CompileKotlin tasks/types, e.g.

tasks.withType(JavaCompile) { dependsOn("bufGenerate") }
tasks.withType(KotlinCompile) { dependsOn("bufGenerate") }

[andrewparmet]

I don't think that's the right path forward since it only addresses half the issue. I think a more elegant solution may be to introduce a function on the extension like addGeneratedSource<KotlinCompile>("path-to-generated-code") that adds the task dependency and the source set location in one shot. I'm not sure if it's possible to easily determine the SourceSet based on the task type though.

[remoun]

I'm not sure if it's possible to easily determine the SourceSet based on the task type though.

FWIW the protobuf plugin configures its extension for all SourceSets, although it switches paths based on whether the project is an Android project.

Generally, I think making all {Java,Kotlin}Compile tasks depend on bufGenerate is a reasonable default, and we could add an escape hatch (e.g. config flag) for folks who want to customize that behaviour.

[milgner]

One thing I noticed recently that after adding

tasks.withType<KotlinJvmCompile>().configureEach {
    dependsOn("bufGenerate")
}

to my build file, buf is invoked on any compilation-related command and it will contact the BSR - even if no input files changed. I only noticed when I ran into rate limiting issues. Now I don't want to waste anyones resources but I also feel like this should be possible to automate so one cannot forget to compile .proto files in case they actually do change.

As someone who practices TDD a lot, having buf run a full compile step including remote services for every change (even in non-buf-related files) is quite an overhead. Unfortunately I also couldn't find anything that would allow me to tune this behaviour. Gradle build cache is already enabled.