Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions CHANGELOG.MD
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,24 @@ This project adheres to [Semantic Versioning](https://semver.org/). Version numb
- **MINOR**: New features that are backward-compatible.
- **PATCH**: Bug fixes or minor changes that do not affect backward compatibility.


## [1.14.0]

_released 04-01-2026

### Added
- <to_update>

### Fixed
-

## [1.13.4]

_released 03-17-2026
_released 03-18-2026

### Added
- Extended glob support for robot parser
- Added `--clear-run-assigned-to-id` flag to `add_run` command for clearing test run assignees during updates
- Extended glob support for robot parser for efficient multiple test result file processing
- Added `--run-assigned-to-id` and `--clear-run-assigned-to-id` options to `add_run` command for clearing and setting test run assignees during run creation and updates

### Fixed
- Cannot add empty runs via add_run command due to empty test case validation.
Expand Down
62 changes: 57 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ trcli
```
You should get something like this:
```
TestRail CLI v1.13.4
TestRail CLI v1.14.0
Copyright 2025 Gurock Software GmbH - www.gurock.com
Supported and loaded modules:
- parse_junit: JUnit XML Files (& Similar)
Expand All @@ -51,7 +51,7 @@ CLI general reference
--------
```shell
$ trcli --help
TestRail CLI v1.13.4
TestRail CLI v1.14.0
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli [OPTIONS] COMMAND [ARGS]...

Expand Down Expand Up @@ -196,7 +196,7 @@ For further detail, please refer to the

### Using Glob Patterns for Multiple Files

TRCLI supports glob patterns to process multiple report files in a single command. This feature is available for **JUnit XML** and **Cucumber JSON** parsers.
TRCLI supports glob patterns to process multiple report files in a single command. This feature is available for **JUnit XML**, **Robot Framework**, and **Cucumber JSON** parsers.

#### Important: Shell Quoting Requirement

Expand Down Expand Up @@ -239,6 +239,7 @@ When a glob pattern matches **multiple files**, TRCLI automatically:
3. **Merges test results** into a single combined report
4. **Writes merged file** to current directory:
- JUnit: `Merged-JUnit-report.xml`
- Robot Framework: `Merged-Robot-report.xml`
- Cucumber: `merged_cucumber.json`
5. **Processes the merged file** as a single test run upload

Expand All @@ -263,6 +264,23 @@ trcli parse_junit \
--case-matcher auto
```

**Robot Framework - Multiple output files:**
```bash
# Merge multiple Robot Framework test runs
trcli -y \
-h https://example.testrail.com \
--project "My Project" \
parse_robot \
-f "reports/robot-*.xml" \
--title "Merged Robot Tests"

# Recursive search for all Robot outputs
trcli parse_robot \
-f "test-results/**/output.xml" \
--title "All Robot Results" \
--case-matcher property
```

**Cucumber JSON - Multiple test runs:**
```bash
# Merge multiple Cucumber JSON reports
Expand Down Expand Up @@ -864,6 +882,40 @@ the `--special-parser saucectl` command line option.
Please refer to the [SauceLabs and saucectl reports](https://support.gurock.com/hc/en-us/articles/12719558686484)
documentation for further information.

#### Cross-suite test plans with multisuite parser

If your test automation spans multiple TestRail suites, the multisuite parser allows you to create a single test plan with one run per suite from a single JUnit XML report. The CLI automatically detects which suite each test case belongs to, groups them accordingly, and creates or updates a test plan with the appropriate structure.

**Requirements:**
- All test cases must have case IDs (C123 format in test names or `test_id` properties)
- Must use `--case-matcher name` or `--case-matcher property` (not `auto`)
- All cases must belong to the same project (cross-project cases are skipped with warnings)

**Basic usage (create new plan):**
```bash
trcli parse_junit \
--special-parser multisuite \
--title "Cross-Suite Test Plan" \
--file results.xml \
--case-matcher property
```

**Add to existing plan:**
```bash
trcli parse_junit \
--special-parser multisuite \
--plan-id 1234 \
--file results.xml \
--case-matcher property
```

The parser automatically:
- Fetches suite information for each case ID concurrently (fast performance)
- Groups cases by their suite
- Creates a test plan with one run per suite
- Uploads results to the correct run within the plan
- Includes suite names and test counts in the plan description

#### Creating new test runs

When a test run MUST created before using one of the parse commands, use the `add_run` command. For example, if
Expand Down Expand Up @@ -1623,7 +1675,7 @@ Options:
### Reference
```shell
$ trcli add_run --help
TestRail CLI v1.13.4
TestRail CLI v1.14.0
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli add_run [OPTIONS]

Expand Down Expand Up @@ -1782,7 +1834,7 @@ providing you with a solid base of test cases, which you can further expand on T
### Reference
```shell
$ trcli parse_openapi --help
TestRail CLI v1.13.4
TestRail CLI v1.14.0
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli parse_openapi [OPTIONS]

Expand Down
46 changes: 46 additions & 0 deletions tests/test_data/XML/example-multisuite-sp.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Cross-Suite Tests" tests="6" failures="1">
<testsuite name="Test Suite" tests="6">

<!-- Cases from Authentication Suite (Suite ID: 1) -->
<testcase name="test_login" classname="LoginTest" time="2.5">
<properties>
<property name="test_id" value="C1374639"/>
</properties>
</testcase>

<testcase name="test_logout" classname="LogoutTest" time="1.2">
<properties>
<property name="test_id" value="C1374640"/>
</properties>
</testcase>

<!-- Cases from Shopping Cart Suite (Suite ID: 2) -->
<testcase name="test_add_item" classname="CartTest" time="3.5">
<properties>
<property name="test_id" value="C1374641"/>
</properties>
</testcase>

<testcase name="test_remove_item" classname="CartTest" time="2.8">
<properties>
<property name="test_id" value="C1374642"/>
</properties>
<failure message="Item not removed"/>
</testcase>

<!-- Cases from Payment Suite (Suite ID: 3) -->
<testcase name="test_checkout" classname="PaymentTest" time="7.5">
<properties>
<property name="test_id" value="C1374643"/>
</properties>
</testcase>

<testcase name="test_refund" classname="PaymentTest" time="12.5">
<properties>
<property name="test_id" value="C1374638"/>
</properties>
</testcase>

</testsuite>
</testsuites>
77 changes: 77 additions & 0 deletions tests/test_data/XML/multisuite_report.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Multisuite Test Report" tests="9" failures="2" errors="0" time="45.0">
<testsuite name="Cross-Suite Tests" tests="9" failures="2" errors="0" time="45.0">

<!-- Cases from Suite A (Authentication Suite) -->
<testcase classname="com.example.auth.LoginTest" name="test_login_valid_credentials" time="2.5">
<properties>
<property name="test_id" value="C100"/>
<property name="testrail_result_comment" value="Login test passed successfully"/>
<property name="testrail_attachment" value="/path/to/screenshot1.png"/>
</properties>
</testcase>

<testcase classname="com.example.auth.LoginTest" name="test_login_invalid_password" time="1.8">
<properties>
<property name="test_id" value="C101"/>
</properties>
<failure type="AssertionError" message="Expected error message not displayed">
Expected: "Invalid password" message to be displayed
Actual: No error message shown
</failure>
</testcase>

<testcase classname="com.example.auth.LogoutTest" name="test_logout_normal" time="1.2">
<properties>
<property name="test_id" value="C102"/>
</properties>
</testcase>

<!-- Cases from Suite B (Shopping Cart Suite) -->
<testcase classname="com.example.cart.AddItemTest" name="test_add_single_item" time="3.5">
<properties>
<property name="test_id" value="C200"/>
<property name="testrail_result_field" value="custom_steps:Step 1: Add item\nStep 2: Verify cart"/>
</properties>
</testcase>

<testcase classname="com.example.cart.AddItemTest" name="test_add_multiple_items" time="5.0">
<properties>
<property name="test_id" value="C201"/>
</properties>
</testcase>

<testcase classname="com.example.cart.RemoveItemTest" name="test_remove_item" time="2.8">
<properties>
<property name="test_id" value="C202"/>
</properties>
<failure type="AssertionError" message="Item not removed from cart">
Expected: Cart count to decrease by 1
Actual: Cart count unchanged
</failure>
</testcase>

<!-- Cases from Suite C (Payment Suite) -->
<testcase classname="com.example.payment.CheckoutTest" name="test_checkout_credit_card" time="7.5">
<properties>
<property name="test_id" value="C300"/>
<property name="testrail_case_field" value="custom_preconds:User must be logged in with items in cart"/>
<property name="testrail_attachment" value="/path/to/payment_screenshot.png"/>
</properties>
</testcase>

<testcase classname="com.example.payment.CheckoutTest" name="test_checkout_paypal" time="8.2">
<properties>
<property name="test_id" value="C301"/>
</properties>
</testcase>

<testcase classname="com.example.payment.RefundTest" name="test_process_refund" time="12.5">
<properties>
<property name="test_id" value="C302"/>
<property name="testrail_result_comment" value="Refund processing test completed"/>
</properties>
</testcase>

</testsuite>
</testsuites>
31 changes: 31 additions & 0 deletions tests/test_glob_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,37 @@ def test_glob_junit_multiple_results_scenario_2(self):
if merged_file.exists():
merged_file.unlink()

@pytest.mark.parse_robot
def test_glob_robot_duplicate_automation_ids(self):
"""Test Robot Framework glob pattern with duplicate automation_ids."""
env = Environment()
env.case_matcher = MatchersParser.AUTO
env.file = Path(__file__).parent / "test_data/XML/testglob_robot/*.xml"

# Check if test files exist
if not list(Path(__file__).parent.glob("test_data/XML/testglob_robot/*.xml")):
pytest.skip("Robot test data not available")

parser = RobotParser(env)
parsed_suites = parser.parse_file()
suite = parsed_suites[0]

# Similar verification as JUnit tests
data_provider = ApiDataProvider(suite)
cases_to_add = data_provider.add_cases()

# Verify deduplication occurred if there were duplicates
total_cases = sum(len(section.testcases) for section in suite.testsections)
automation_ids = [c.custom_automation_id for c in cases_to_add if c.custom_automation_id]

# Cases to add should have unique automation_ids
assert len(automation_ids) == len(set(automation_ids)), "Cases to add should have unique automation_ids"

# Clean up merged file
merged_file = Path.cwd() / "Merged-Robot-report.xml"
if merged_file.exists():
merged_file.unlink()

@pytest.mark.parse_cucumber
def test_cucumber_glob_filepath_not_pattern(self):
"""Test Scenario 3: Cucumber glob pattern uses correct filepath (not pattern string).
Expand Down
Loading
Loading