Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.MD
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ This project adheres to [Semantic Versioning](https://semver.org/). Version numb
- **MINOR**: New features that are backward-compatible.
- **PATCH**: Bug fixes or minor changes that do not affect backward compatibility.

## [1.13.2]

_released 02-24-2025

### Fixed
- Fixed an issue where automation_id matching fails due to wrapped values in HTML paragraph tags, causing case mismatch and duplicate test cases

## [1.13.1]

_released 02-19-2025
Expand Down
28 changes: 19 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ trcli
```
You should get something like this:
```
TestRail CLI v1.13.1
TestRail CLI v1.13.2
Copyright 2025 Gurock Software GmbH - www.gurock.com
Supported and loaded modules:
- parse_junit: JUnit XML Files (& Similar)
Expand All @@ -51,7 +51,7 @@ CLI general reference
--------
```shell
$ trcli --help
TestRail CLI v1.13.1
TestRail CLI v1.13.2
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli [OPTIONS] COMMAND [ARGS]...

Expand Down Expand Up @@ -1509,7 +1509,7 @@ Options:
### Reference
```shell
$ trcli add_run --help
TestRail CLI v1.13.1
TestRail CLI v1.13.2
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli add_run [OPTIONS]

Expand Down Expand Up @@ -1633,7 +1633,7 @@ providing you with a solid base of test cases, which you can further expand on T
### Reference
```shell
$ trcli parse_openapi --help
TestRail CLI v1.13.1
TestRail CLI v1.13.2
Copyright 2025 Gurock Software GmbH - www.gurock.com
Usage: trcli parse_openapi [OPTIONS]

Expand Down Expand Up @@ -1815,18 +1815,23 @@ The TestRail CLI includes a comprehensive logging infrastructure designed specif

#### Automatic Logging

TRCLI now automatically logs all operations using structured logging. Simply configure using environment variables:
TRCLI supports structured logging (disabled by default). To enable logging, configure using environment variables:

```bash
# Enable JSON logs on stderr (default)
# Enable logging
export TRCLI_LOG_ENABLED=true

# Configure log level and format
export TRCLI_LOG_LEVEL=INFO
export TRCLI_LOG_FORMAT=json

# Run any TRCLI command - logging happens automatically
# Run any TRCLI command - logging will be active
trcli parse_junit --file report.xml --project "My Project" \
--host https://example.testrail.io --username user --password pass
```

**Note:** Logging is disabled by default. Set `TRCLI_LOG_ENABLED=true` to enable structured logging output.

#### Direct API Usage (Advanced)

For custom integrations or scripts:
Expand Down Expand Up @@ -1855,9 +1860,12 @@ logger.info("Custom operation", status="success")

#### Configuration

Configure logging using environment variables:
Logging is **disabled by default**. To enable, configure using environment variables:

```bash
# Enable logging (master switch - required)
export TRCLI_LOG_ENABLED=true # true/false, yes/no, 1/0, on/off (default: false)

# Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
export TRCLI_LOG_LEVEL=INFO

Expand All @@ -1872,7 +1880,7 @@ export TRCLI_LOG_FILE=/var/log/trcli/app.log
export TRCLI_LOG_MAX_BYTES=10485760 # 10MB
export TRCLI_LOG_BACKUP_COUNT=5

# Run TRCLI
# Run TRCLI with logging enabled
trcli parse_junit --file report.xml
```

Expand All @@ -1881,6 +1889,7 @@ Or use a YAML configuration file:
```yaml
# trcli_config.yml
logging:
enabled: true # Must be true to enable logging (default: false)
level: INFO
format: json
output: file
Expand Down Expand Up @@ -1952,6 +1961,7 @@ trcli parse_junit --file report.xml

| Variable | Description | Values | Default |
|----------|-------------|--------|---------|
| `TRCLI_LOG_ENABLED` | Enable/disable all logging (master switch) | true, false, yes, no, 1, 0, on, off | **false** |
| `TRCLI_LOG_LEVEL` | Minimum log level | DEBUG, INFO, WARNING, ERROR, CRITICAL | INFO |
| `TRCLI_LOG_FORMAT` | Output format | json, text | json |
| `TRCLI_LOG_OUTPUT` | Output destination | stderr, stdout, file | stderr |
Expand Down
230 changes: 230 additions & 0 deletions tests/test_api_request_handler_case_matcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -551,5 +551,235 @@ def test_performance_name_matcher_with_missing_ids(self, environment, api_client
print("=" * 60)


class TestFroalaParagraphTagStripping:
"""Test suite for Froala HTML paragraph tag stripping in automation_id matching"""

@pytest.mark.parametrize(
"input_value,expected_output",
[
# Basic Froala wrapping
("<p>com.example.Test.method</p>", "com.example.Test.method"),
("<p>automation_id_value</p>", "automation_id_value"),
# Case insensitive tags
("<P>value</P>", "value"),
("<p>value</P>", "value"),
("<P>value</p>", "value"),
# With whitespace
("<p>value</p>\n", "value"),
("<p> value </p>", "value"),
(" <p>value</p> ", "value"),
("<p>\nvalue\n</p>", "value"),
# Without tags (should pass through)
("plain_value", "plain_value"),
("com.example.Test.method", "com.example.Test.method"),
# Partial tags (still stripped for safety - unlikely to have legitimate automation IDs with these)
("<p>value", "value"),
("value</p>", "value"),
# Empty/None values
("", ""),
(None, None),
# Complex automation IDs
("<p>com.example.MyTests.test_name_C120013()</p>", "com.example.MyTests.test_name_C120013()"),
("<p>User Login.@positive.@smoke.Valid credentials</p>", "User Login.@positive.@smoke.Valid credentials"),
("<p>Sub-Tests.Subtests 1.Subtest 1a</p>", "Sub-Tests.Subtests 1.Subtest 1a"),
# HTML entities (already unescaped before this function)
("<p>test&amp;value</p>", "test&amp;value"), # Should be handled by html.unescape first
],
)
def test_strip_froala_paragraph_tags_unit(self, input_value, expected_output):
"""
Unit test for _strip_froala_paragraph_tags static method.
Tests various scenarios of Froala HTML wrapping.
"""
from trcli.api.case_matcher import AutomationIdMatcher

result = AutomationIdMatcher._strip_froala_paragraph_tags(input_value)
assert result == expected_output, f"Expected '{expected_output}', got '{result}'"

@pytest.mark.api_handler
def test_auto_matcher_matches_cases_with_froala_tags(self, environment, api_client, mocker):
"""
Integration test: AUTO matcher correctly matches cases even when TestRail returns
automation_id values wrapped in Froala <p> tags.

Simulates the real scenario:
1. Test report has automation_id: "com.example.Test.method1"
2. TestRail returns: "<p>com.example.Test.method1</p>" (after user edited the case)
3. Should still match correctly
"""
# Setup: AUTO matcher
environment.case_matcher = MatchersParser.AUTO

# Create test suite with plain automation IDs (as they come from test reports)
test_case_1 = TestRailCase(
title="Test Method 1",
custom_automation_id="com.example.Test.method1",
result=TestRailResult(status_id=1),
)
test_case_2 = TestRailCase(
title="Test Method 2",
custom_automation_id="com.example.Test.method2",
result=TestRailResult(status_id=1),
)
test_case_3 = TestRailCase(
title="Test Method 3",
custom_automation_id="com.example.Test.method3",
result=TestRailResult(status_id=1),
)

section = TestRailSection(name="Test Section", section_id=1, testcases=[test_case_1, test_case_2, test_case_3])
test_suite = TestRailSuite(name="Test Suite", suite_id=1, testsections=[section])

api_request_handler = ApiRequestHandler(environment, api_client, test_suite)

# Mock TestRail responses: return automation_id values wrapped in Froala <p> tags
# (simulates what happens when user edits cases in TestRail with Text field type)
mock_cases = [
{
"id": 1,
"custom_automation_id": "<p>com.example.Test.method1</p>", # Froala wrapped
"title": "Test Method 1",
"section_id": 1,
},
{
"id": 2,
"custom_automation_id": "<p>com.example.Test.method2</p>\n", # With newline
"title": "Test Method 2",
"section_id": 1,
},
{
"id": 3,
"custom_automation_id": "com.example.Test.method3", # Plain (not edited yet)
"title": "Test Method 3",
"section_id": 1,
},
]

mocker.patch.object(api_request_handler, "_ApiRequestHandler__get_all_cases", return_value=(mock_cases, None))

mock_update_data = mocker.patch.object(api_request_handler.data_provider, "update_data")

# Execute
project_id = 1
has_missing, error = api_request_handler.check_missing_test_cases_ids(project_id)

# Assert: All 3 cases should match successfully (no missing cases)
assert not has_missing, "Should not have missing cases - Froala tags should be stripped"
assert error == "", "Should not have errors"

# Verify that update_data was called with all 3 matched cases
mock_update_data.assert_called_once()
call_args = mock_update_data.call_args[1]
matched_cases = call_args["case_data"]

assert len(matched_cases) == 3, "All 3 cases should be matched"

# Verify correct case IDs were matched
matched_ids = {case["case_id"] for case in matched_cases}
assert matched_ids == {1, 2, 3}, "Should match all case IDs correctly"

@pytest.mark.api_handler
def test_auto_matcher_handles_both_automation_id_field_names(self, environment, api_client, mocker):
"""
Test that Froala tag stripping works for both automation_id field names:
- custom_automation_id (legacy)
- custom_case_automation_id (current)
"""
# Setup: AUTO matcher
environment.case_matcher = MatchersParser.AUTO

test_case_1 = TestRailCase(
title="Test 1",
custom_automation_id="test.method1",
result=TestRailResult(status_id=1),
)
test_case_2 = TestRailCase(
title="Test 2",
custom_automation_id="test.method2",
result=TestRailResult(status_id=1),
)

section = TestRailSection(name="Test Section", section_id=1, testcases=[test_case_1, test_case_2])
test_suite = TestRailSuite(name="Test Suite", suite_id=1, testsections=[section])

api_request_handler = ApiRequestHandler(environment, api_client, test_suite)

# Mock cases: one with legacy name, one with current name (both Froala wrapped)
mock_cases = [
{
"id": 1,
"custom_automation_id": "<p>test.method1</p>", # Legacy field name
"title": "Test 1",
"section_id": 1,
},
{
"id": 2,
"custom_case_automation_id": "<p>test.method2</p>", # Current field name
"title": "Test 2",
"section_id": 1,
},
]

mocker.patch.object(api_request_handler, "_ApiRequestHandler__get_all_cases", return_value=(mock_cases, None))

mock_update_data = mocker.patch.object(api_request_handler.data_provider, "update_data")

# Execute
project_id = 1
has_missing, error = api_request_handler.check_missing_test_cases_ids(project_id)

# Assert: Both cases should match
assert not has_missing, "Should match cases with both field name variants"
assert error == "", "Should not have errors"

matched_cases = mock_update_data.call_args[1]["case_data"]
assert len(matched_cases) == 2, "Both cases should be matched"

@pytest.mark.api_handler
def test_froala_tags_cause_mismatch_without_fix(self, environment, api_client, mocker):
"""
Negative test: Demonstrate that WITHOUT the fix, Froala tags would cause mismatches.
This test documents the bug being fixed.
"""
# Setup
environment.case_matcher = MatchersParser.AUTO

test_case = TestRailCase(
title="Test Method",
custom_automation_id="com.example.Test.method",
result=TestRailResult(status_id=1),
)

section = TestRailSection(name="Test Section", section_id=1, testcases=[test_case])
test_suite = TestRailSuite(name="Test Suite", suite_id=1, testsections=[section])

api_request_handler = ApiRequestHandler(environment, api_client, test_suite)

# Return automation_id with tags from TestRail
mock_cases = [
{
"id": 1,
"custom_automation_id": "<p>com.example.Test.method</p>",
"title": "Test Method",
"section_id": 1,
}
]

mocker.patch.object(api_request_handler, "_ApiRequestHandler__get_all_cases", return_value=(mock_cases, None))

mock_update_data = mocker.patch.object(api_request_handler.data_provider, "update_data")

# Execute
has_missing, _ = api_request_handler.check_missing_test_cases_ids(1)

# Assert: With the fix, this should NOT have missing cases
assert not has_missing, "With fix applied, Froala tags should be stripped and case should match"

# Without the fix, this test would fail because:
# - Report has: "com.example.Test.method"
# - TestRail returns: "<p>com.example.Test.method</p>"
# - No match → missing case → new duplicate case created


if __name__ == "__main__":
pytest.main([__file__, "-v", "-s"])
2 changes: 2 additions & 0 deletions tests/test_logging/test_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ def test_config_file_integration(self):
config_file.write_text(
f"""
logging:
enabled: true
level: DEBUG
format: json
output: file
Expand Down Expand Up @@ -262,6 +263,7 @@ def test_env_var_integration(self):
"""Test environment variable configuration"""
log_file = Path(self.temp_dir) / "env_test.log"

os.environ["TRCLI_LOG_ENABLED"] = "true"
os.environ["TRCLI_LOG_LEVEL"] = "WARNING"
os.environ["TRCLI_LOG_FORMAT"] = "json"
os.environ["TRCLI_LOG_OUTPUT"] = "file"
Expand Down
2 changes: 1 addition & 1 deletion trcli/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.13.1"
__version__ = "1.13.2"
Loading