Skip to content

Setting up GameTest

Jump to bottom
Shadew edited this page Aug 30, 2021 · 2 revisions

GameTest is a framework integrated in Minecraft for in-game automated testing. JEDT provides an easy API to use the framework. Make sure you use JEDT 0.3.2 or higher.

Setting up

You probably want your tests to be in a separate source set. Due to the way the test source set works, your tests will not be on the class path, even when you run Minecraft with the test source set. Instead, you have to create a new source set. This source set can be any name, but a recommended name is gametest.

sourceSets {
    gametest {
        // Inherit main class path
        compileClasspath += main.compileClasspath
        runtimeClasspath += main.runtimeClasspath
    }
}

dependencies {
    // Use main class path as dependency
    gametestImplementation sourceSets.main.output
}

This creates a new source set that extends from the main source set, like test, but does not exclude itself from the classpath. Now, you probably want to run the game with your gametest sources so you can create and run your tests at any time. To do so, modify the run configurations in Loom.

loom {
    runs {
        client {
            name "Minecraft Client"
            source sourceSets.gametest

            // This sets the folder where the GameTest framework imports and exports test structures from and to
            // By default, this folder resides in the 'run' folder, but you likely want your structures to be
            // in VCS, so export them to a folder visible to VCS instead.
            vmArg "-Djedt.test_structures_path=${file("gametest/gameteststructures")}"
        }
        server {
            name "Minecraft Server"
            source sourceSets.gametest

            vmArg "-Djedt.test_structures_path=${file("gametest/gameteststructures")}"
        }
    }
}

If you already have run configurations, make sure to remove those first, otherwise Loom will not re-generate your run configurations.

Now, once you refreshed Gradle, you have new gametest source set in your project. Set up a simple tests mod in here by adding a fabric.mod.json, this is required by JEDT to find your tests, because it will not find tests that are not part of a mod. Once you configured your test mod, you can proceed to adding tests.

Adding tests

Making your test classes goes pretty much entirely via the GameTest framework. A tutorial will follow on how to write tests. However, you need to register your tests. JEDT assumes you use JEDT to register tests, so to register your tests, create a jedt.tests.json in the same folder as your test mod's fabric.mod.json.

The jedt.tests.json has a simple format. The root of the JSON is an object ({}), with in there various arrays ([]) by name, defining sets of tests. Tests are specified by either specifying a method ("path.to.Class::method"), a class ("path.to.Class"), or another set to inherit ("#setname"). In most cases you want to register an entire test class. An example jedt.tests.json below:

{
  "tests": [
    "#inherit_another_set",

    "test.Class",
    "another.test.Class",
    "single.Method::fromClass"
  ]
}

Say you have made a class my.tests.MyTests, your jedt.tests.json would look like this:

{
  "tests": ["my.tests.MyTests"]
}

However, when running the game and invoking /test runall, it will run no tests. This is because JEDT will only load tests from specific sets when running the client or dedicated server.

  • On the client, it will load _client and _runtime.
  • On the dedicated server, it will load _server and _runtime.

Hence, to make your tests available, you have to add them to the _runtime set. The easiest and recommended way to do this is to make _runtime inherit the test sets you want to have available when running the client or dedicated server:

{
  "_runtime": ["#tests"],
  "tests": ["my.tests.MyTests"]
}

Now, when you run /test runall your tests will be invoked.

The GameTest server

JEDT provides a way to run tests in a server instance and generate a test report. To run a test server, you have to create a new run configuration in Loom.

loom {
    runs {
        // ... client and server ...

        gametest {
            server() // Run a server instance

            name "Minecraft GameTest"
            source sourceSets.gametest

            // Tell JEDT to redirect the Dedicated server to a GameTest server
            vmArg "-Djedt.gametest=true"

            // This is our test configuration file, herein we specify which tests need to be ran, and how. It's a bit more complicated
            // than specifying tests for the client or dedicated server. You probably want this to be in VCS too, so use the same
            // 'gametest' directory.
            vmArg "-Djedt.test_config=${file("gametest/gametest.json")}"

            // Test structures path
            vmArg "-Djedt.test_structures_path=${file("gametest/gameteststructures")}"

            serverWithGui() // The GameTest server has no nogui property, this removes it
        }
    }
}

Note how we specify a jedt.test_config property. This is a path to a JSON file containing the configuration of the GameTest server. It contains various options that specify how the GameTest server must be run, which test sets must be included and which mods tests must be loaded from. A test configuration JSON looks like this:

{
  "test_structures_dir": "[jedt.test_structures_path]",
  "reporter": { "jedt:junit": "./test_results.xml" },
  "export": "test_world.zip"
}

There are various other options (to be documented), but these three are the most important ones.

  • test_structures_dir specifies where the test structures are located. This must be set in the config JSON, setting the JVM property doesn't work. However, you can get the value from the JVM property by wrapping the property name in braces.
  • reporter specifies how test results must be reported. The jedt:junit reporter generates an XML that is similar to the JUnit XML output, and requires an output file as property.
  • export specifies that the world where the tests were performed in must be exported as a zip file, and where.

By default, the GameTest server loads all test sets from all mods. You can configure which sets and mods must be loaded using the sets and mods properties.

When running the GameTest server via the new run configuration, it will start running your tests if you properly set it up. If you used the above configuration, you will notice that it generated a test_results.xml and test_world.zip in your run folder. They don't need to be in VCS.

GitHub Actions

When you have configured your GameTest server, you might want to run your tests in GitHub Actions. All you need to do for that is set up your Actions workflow. The actions EnricoMi/publish-unit-test-result-action@v1 and actions/upload-artifact@v2 come in handy here.

  • EnricoMi/publish-unit-test-result-action@v1 is used to upload a visual diagram of the generated JUnit-like test results (test_results.xml).
  • actions/upload-artifact@v2 is used to upload the exported world zip (test_world.zip) after testing.

Your workflow would more or less follow the following steps:

  1. Clone and set up Java
  2. Run the test server via the gradle runGametest command (you do not need an action for that).
  3. Use EnricoMi/publish-unit-test-result-action@v1 to upload the test results. Make sure it runs regardless of the previous step's status (if: always()).
  4. Use actions/upload-artifact@v2 to upload the exported world zip. Again, make sure it runs regardless of the previous step's status (if: always()).

In workflow format:

name: Run GameTest

on: [ push ]

jobs:
  gametest:
    runs-on: ubuntu-latest

    steps:
      # Check out the repository
      - uses: actions/checkout@v2

      # Set up Java
      - name: Set up JDK 16
        uses: actions/setup-java@v1
        with:
          java-version: 16
          server-id: github
          settings-path: ${{ github.workspace }}

      # Run your test server
      # When one of the required tests fails, the Gradle task and hence this step will fail
      - name: Run GameTests
        run: gradle runGametest

      # Publish the test results
      - name: Publish GameTest Results
        uses: EnricoMi/publish-unit-test-result-action@v1
        if: always()  # Otherwise it will not run when testing fails
        with:
          files: run/test_results.xml

      # Upload the test world
      - name: Upload Test World
        uses: actions/upload-artifact@v2
        if: always()  # Otherwise it will not run when testing fails
        with:
          name: test-world
          path: run/test_world.zip

Clone this wiki locally