Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,22 @@ jobs:
with:
distribution: 'temurin'
java-version: '21'
- name: PMD
uses: pmd/pmd-github-action@v2.0.0
id: pmd
- name: Set up Workspace Environment Variable
run: echo "WORKSPACE=${{ github.workspace }}" >> $GITHUB_ENV
- name: PMD Check
run: mvn pmd:pmd pmd:cpd pmd:check pmd:cpd-check -f ./ddk-parent/pom.xml --batch-mode --fail-at-end
checkstyle:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- uses: actions/setup-java@v5
with:
version: '7.21.0'
rulesets: 'ddk-configuration/pmd/ruleset.xml'
analyzeModifiedFilesOnly: false
- name: Fail build if there are violations
if: steps.pmd.outputs.violations != 0
run: exit 1
distribution: 'temurin'
java-version: '21'
- name: Set up Workspace Environment Variable
run: echo "WORKSPACE=${{ github.workspace }}" >> $GITHUB_ENV
- name: Checkstyle Check
run: mvn checkstyle:checkstyle checkstyle:check -f ./ddk-parent/pom.xml --batch-mode --fail-at-end
maven-verify:
runs-on: ubuntu-24.04
steps:
Expand All @@ -36,10 +42,10 @@ jobs:
distribution: 'temurin'
java-version: '21'
- name: Set up Workspace Enviroment Variable
run: echo "WORKSPACE=${{ github.workspace }}" >> $GITHUB_ENV
run: echo "WORKSPACE=${{ github.workspace }}" >> $GITHUB_ENV
- name: Cache Maven dependencies
uses: actions/cache@v5
with:
with:
path: /home/runner/.m2/repository
key: ${{ runner.os }}-maven-0-${{ hashFiles('**/pom.xml') }}
- name: Build with Maven within a virtual X Server Environment
Expand Down
125 changes: 125 additions & 0 deletions TEST-REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# CI Test Report — Non-UI Test Modules

Results from enabling `tycho-surefire-plugin` (`skip=false`) for 7 non-UI test modules.

CI Run: https://github.com/dsldevkit/dsl-devkit/actions/runs/21770300335

## Summary

| Module | Tests | Failures | Errors | Status |
|--------|------:|:--------:|:------:|:------:|
| `xtext.export.test` | 14 | 0 | 0 | PASS |
| `typesystem.test` | 55 | 0 | 0 | PASS |
| `check.runtime.core.test` | 16 | 0 | 0 | PASS |
| `xtext.test` *(existing)* | — | — | — | SKIPPED |
| `xtext.format.test` | 22 | 6 | 7 | FAIL |
| `xtext.generator.test` | 27 | 0 | 2 | FAIL |
| `checkcfg.core.test` | 10 | 5 | 1 | FAIL |
| `check.core.test` | 65 | 0 | 63 | FAIL |
| **Total** | **209** | **11** | **73** | |

> **Note:** `xtext.test` was SKIPPED by Maven due to earlier module failures (`--fail-at-end` mode). It normally runs 74 tests and passes on `master`.

## Root Cause Analysis

### `check.core.test` — 63 errors (Guice injector needs UI)

Nearly all tests fail with `com.google.inject.CreationException`. Tests use `CheckUiInjectorProvider`, which requires the Eclipse UI workbench. Despite being in a "core" test module, these tests depend on the UI injector.

**Fix:** Run with `useUIHarness=true`, or move tests to `check.ui.test`.

### `xtext.format.test` — 6 failures, 7 errors (mixed: injector + resource loading)

Two distinct failure modes:

1. **`FormatScopingTest` (6 errors):** `NullPointerException` — grammar fields are null because `FormatUiInjectorProvider` (UI injector) fails to initialize without Eclipse workbench.
2. **`FormatValidationTest` (5 failures, 1 error):** Assertion failures and a `ClassCastException` (`XMIResourceImpl` cannot be cast to `XtextResource`) — also caused by incorrect injector initialization.
3. **`FormatParsingTest` (1 failure):** `loadModel` assertion failure.

**Fix:** Run with `useUIHarness=true`.

### `checkcfg.core.test` — 5 failures, 1 error (language registration + linking)

1. **`CheckCfgScopeProviderTest` (1 error):** `java.lang.Exception: Scope has no elements` — the scope provider cannot find any check catalogs.
2. **`CheckCfgSyntaxTest` (3 failures):** Unexpected issues `unknown_language` and `Diagnostic.Linking` — the `com.avaloq.tools.ddk.check.TestLanguage` is not registered in the runtime, so references cannot be resolved.
3. **`CheckCfgConfiguredParameterValidationsTest` (1 failure):** Same `unknown_language` issue.
4. **`CheckCfgTest` (1 failure):** `testValidLanguageOk` — `Expected no issues, but got: ERROR (unknown_language) 'Unknown language'`.

**Fix:** These tests require the Check language runtime to be registered. Likely needs `useUIHarness=true` or explicit language registration in test setup.

### `xtext.generator.test` — 2 errors (test setup bug)

**`XbaseGeneratorFragmentTest` (2 errors):**
- `testUsesXImportSectionWhenNotUsed` and `testUsesXImportSectionWhenUsed` both fail with `NullPointerException: Cannot invoke "org.eclipse.emf.ecore.resource.Resource.getResourceSet()" because the return value of "org.eclipse.xtext.AbstractRule.eResource()" is null`.

**Fix:** This is a genuine test setup bug — the grammar rule is not attached to a resource. Needs investigation of test fixture creation in `XbaseGeneratorFragmentTest`.

## Detailed Failures

### `check.core.test`

All 63 errors share the same root cause:
```
java.lang.RuntimeException: Failed to create injector for com.avaloq.tools.ddk.check.Check
Caused by: com.google.inject.CreationException: Unable to create injector, see the following errors:
```

Affected test classes:
- `BasicModelTest` — 9 errors (all methods)
- `ProjectBasedTests` — 1 error (`ExceptionInInitializerError`)
- `IssueCodeToLabelMapGenerationTest` — 1 error (`ExceptionInInitializerError`)
- `BugDsl27Test` — 1 error (`ExceptionInInitializerError`)
- `CheckScopingTest` — 1 error (`ExceptionInInitializerError`)
- `IssueCodeValueTest` — 1 error (`ExceptionInInitializerError`)
- `CheckFormattingTest` — 3 errors (`testFormattedSource`, `testWSAdded`, `testWSRemoved`)
- `CheckJavaValidatorUtilTest` — 13 errors (all methods)
- Remaining ~34 errors follow the same pattern across other test classes

Note: `BugAig1314Test` (2 tests) PASSES — it does not depend on the UI injector.

### `xtext.format.test`

```
FormatScopingTest.allGrammarsScoped:107 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarC" is null
FormatScopingTest.ruleSelfScoped:152 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarA" is null
FormatScopingTest.assignmentScoped:129 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarA" is null
FormatScopingTest.groupScoped:158 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarA" is null
FormatScopingTest.ruleCallScoped:141 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarA" is null
FormatScopingTest.keywordScoped:113 — NullPointerException: Cannot invoke "Grammar.getRules()" because "this.grammarA" is null

FormatValidationTest.extendedGrammarCompatible — ClassCastException: XMIResourceImpl cannot be cast to XtextResource
FormatValidationTest.missingGrammarRuleOverride — assertion failure
FormatValidationTest.grammarRuleOverrideOK — assertion failure
FormatValidationTest.requiredRulesImplemented — assertion failure
FormatValidationTest.extendedGrammarCompatibleOK — assertion failure
FormatValidationTest.testNegativeACF1000 — assertion failure

FormatParsingTest.loadModel — assertion failure
```

### `checkcfg.core.test`

```
CheckCfgScopeProviderTest.testCatalogsAreInCorrectPackage — java.lang.Exception: Scope has no elements

CheckCfgSyntaxTest.testPropertiesOnAllLevels — Unexpected issue: 'unknown_language' + Diagnostic.Linking
CheckCfgSyntaxTest.testSyntax — Unexpected diagnostic: Diagnostic.Linking
CheckCfgSyntaxTest.testSyntaxConfiguredLanguage — Unexpected issue: 'unknown_language' + Diagnostic.Linking

CheckCfgConfiguredParameterValidationsTest.testConfiguredParameterValues — Unexpected issue: 'unknown_language'

CheckCfgTest.testValidLanguageOk — Expected no issues, but got: ERROR (unknown_language) 'Unknown language' on ConfiguredLanguageValidator
```

### `xtext.generator.test`

```
XbaseGeneratorFragmentTest.testUsesXImportSectionWhenNotUsed:162 — NullPointerException: Cannot invoke "Resource.getResourceSet()" because the return value of "AbstractRule.eResource()" is null
XbaseGeneratorFragmentTest.testUsesXImportSectionWhenUsed:180 — NullPointerException: Cannot invoke "Resource.getResourceSet()" because the return value of "AbstractRule.eResource()" is null
```

## Recommendations

1. **Enable immediately** (3 new modules, +85 tests): `xtext.export.test`, `typesystem.test`, `check.runtime.core.test`
2. **Move to UI harness** (3 modules): `check.core.test`, `xtext.format.test`, `checkcfg.core.test` — these are misclassified as non-UI but depend on UI injectors
3. **Fix test setup** (1 module): `xtext.generator.test` — `XbaseGeneratorFragmentTest` has 2 genuine test bugs
21 changes: 20 additions & 1 deletion com.avaloq.tools.ddk.check.core.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,23 @@

<artifactId>com.avaloq.tools.ddk.check.core.test</artifactId>
<packaging>eclipse-test-plugin</packaging>
</project>

<build>
<plugins>
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-surefire-plugin</artifactId>
<version>${tycho.version}</version>
<configuration>
<skip>false</skip>
<includes>
<include>**/*Test.java</include>
<include>**/*Tests.java</include>
</includes>
<failIfNoTests>false</failIfNoTests>
<useUIThread>false</useUIThread>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ public abstract class AbstractCheckTestCase {
private Provider<XtextResourceSet> resourceSetProvider;

@BeforeEach
public void setUp() throws Exception {
void setUp() throws Exception {
getInjector().injectMembers(this);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
@ExtendWith(InjectionExtension.class)
@InjectWith(CheckInjectorProvider.class)
@SuppressWarnings("nls")
public class BugAig1314 {
class BugAig1314Test {

/** Constructor of super class is protected... */
private static class TestScope extends CatalogFromExtensionPointScope {
Expand Down Expand Up @@ -74,7 +74,7 @@ private ModelLocation createModelLocation(final URL url) {
return new ModelLocation(url, TEST_CATALOG_FILE + TEST_CATALOG_EXTENSION) {
@Override
public InputStream getCatalogStream() {
return BugAig1314.class.getResourceAsStream(TEST_CATALOG_FILE);
return BugAig1314Test.class.getResourceAsStream(TEST_CATALOG_FILE);
}
};
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@
@InjectWith(CheckInjectorProvider.class)
@ExtendWith(InjectionExtension.class)
@SuppressWarnings("nls")
public class BugDsl27 extends AbstractCheckGenerationTestCase {
class BugDsl27Test extends AbstractCheckGenerationTestCase {

/**
* Tests that our test source compiles fine.
*/
@Test
void testGeneratedCodeHasNoErrors() {
try (InputStream sourceStream = BugDsl27.class.getResourceAsStream("bugdsl27/BugDsl27")) {
try (InputStream sourceStream = BugDsl27Test.class.getResourceAsStream("bugdsl27/BugDsl27")) {
generateAndCompile(sourceStream);
} catch (IOException exception) {
LOGGER.info("Failed to close the test file");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,20 @@
*******************************************************************************/
package com.avaloq.tools.ddk.check.test.core;

import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;

import com.avaloq.tools.ddk.check.core.generator.IssueCodeValueTest;
import com.avaloq.tools.ddk.check.core.test.BasicModelTest;
import com.avaloq.tools.ddk.check.core.test.BugAig1314;
import com.avaloq.tools.ddk.check.core.test.BugAig830;
import com.avaloq.tools.ddk.check.core.test.BugDsl27;
import com.avaloq.tools.ddk.check.core.test.CheckScopingTest;
import com.avaloq.tools.ddk.check.core.test.IssueCodeToLabelMapGenerationTest;
import com.avaloq.tools.ddk.check.core.test.ProjectBasedTests;
import com.avaloq.tools.ddk.check.formatting.CheckFormattingTest;


/**
* Junit5 version of test suites. does not implement the logic in our DiscerningSuite.
*/
@Suite
@SelectClasses({
// @Format-Off
IssueCodeValueTest.class,
BasicModelTest.class,
BugAig830.class,
CheckScopingTest.class,
IssueCodeToLabelMapGenerationTest.class,
ProjectBasedTests.class,
BugAig1314.class,
BugDsl27.class,
CheckFormattingTest.class
// @Format-On
@SelectPackages({
"com.avaloq.tools.ddk.check.core.test"
})
public class CheckCoreTestSuite {
@IncludeClassNamePatterns(".*Test.*")
class CheckCoreTestSuite {

}
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ public InputStream createInputStream(final URI uri, final Map<?, ?> options) thr
};
}

@SuppressWarnings("restriction")
@Override
public void load(final Map<?, ?> options) throws IOException {
modelLocation = (IModelLocation) options.get(MAYBE_LOCATION_DATA);
Expand Down
21 changes: 20 additions & 1 deletion com.avaloq.tools.ddk.check.runtime.core.test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,23 @@

<artifactId>com.avaloq.tools.ddk.check.runtime.core.test</artifactId>
<packaging>eclipse-test-plugin</packaging>
</project>

<build>
<plugins>
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-surefire-plugin</artifactId>
<version>${tycho.version}</version>
<configuration>
<skip>false</skip>
<includes>
<include>**/*Test.java</include>
<include>**/*Tests.java</include>
</includes>
<failIfNoTests>false</failIfNoTests>
<useUIThread>false</useUIThread>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@
* Provides some tests of the reflective {@link AbstractCheckContext} framework.
*/
@SuppressWarnings("nls")
public class CheckContextTest {
class CheckContextTest {

public static final String ENABLED_ISSUE_CODE = "Enabled.Issue.Code";
public static final String DISABLED_ISSUE_CODE = "Disabled.Issue.Code";
public static final String DISABLED_AND_ENABLED_ISSUE_CODE = "Disabled.Enabled.Issue.Code";
public static final String ENABLED_AND_DISABLED_ISSUE_CODE = "Enaabled.Disabled.Issue.Code";
public static final String NOT_MENTIONED_ISSUE_CODE = "Not.Mentioned.Issue.Code";
static final String ENABLED_ISSUE_CODE = "Enabled.Issue.Code";
static final String DISABLED_ISSUE_CODE = "Disabled.Issue.Code";
static final String DISABLED_AND_ENABLED_ISSUE_CODE = "Disabled.Enabled.Issue.Code";
static final String ENABLED_AND_DISABLED_ISSUE_CODE = "Enaabled.Disabled.Issue.Code";
static final String NOT_MENTIONED_ISSUE_CODE = "Not.Mentioned.Issue.Code";
private static final EObject DUMMY_CONTEXT = null; // Wrap up null nicely

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
* Provides some test cases for the Check extension point.
*/
@SuppressWarnings("nls")
public class CheckExtensionPointTests {
class CheckExtensionPointTests {

private static final String DUMMY_EXTENSION_ID = "com.avaloq.tools.ddk.check.runtime.core.test";
private static final String CHECK_EXTENSION_ID = "com.avaloq.tools.ddk.check.runtime.core";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
* Performs some basic validation tests on the {@link AbstractCheckValidator}.
*/
@SuppressWarnings("nls")
public class CheckValidatorTest extends AbstractCheckValidator {
class CheckValidatorTest extends AbstractCheckValidator {
/**
* Represents the dummy language which is also registered in the plugin.xml.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
* Unit test for {@link DefaultCheckRuleLabelProvider}.
*/
@SuppressWarnings("nls")
public class CheckRuleLabelProviderTest {
class CheckRuleLabelProviderTest {

// Test data
private static final int NUM_VALIDATORS = 3;
Expand Down
Loading
Loading