Skip to content

ITEP-84743 CI All Tests Workflow rework#915

Draft
dpitulax wants to merge 37 commits intomainfrom
dpitulax_test-workflow-rework
Draft

ITEP-84743 CI All Tests Workflow rework#915
dpitulax wants to merge 37 commits intomainfrom
dpitulax_test-workflow-rework

Conversation

@dpitulax
Copy link
Contributor

📝 Description

Reworked how All Tests executes - split functional and ui test groups into sub-groups to help with debug and reruns.
Removed BATs, and put remaining tests into their respective groups, this gets rid of unnecessary duplication in executions for metric and unit test groups.

Link to a full execution for this branch: https://github.com/open-edge-platform/scenescape/actions/runs/21143582166

JIRA: ITEP-84743

✨ Type of Change

Select the type of change your PR introduces:

  • 🐞 Bug fix – Non-breaking change which fixes an issue
  • 🚀 New feature – Non-breaking change which adds functionality
  • 🔨 Refactor – Non-breaking change which refactors the code base
  • 💥 Breaking change – Changes that break existing functionality
  • 📚 Documentation update
  • 🔒 Security update
  • 🧪 Tests
  • 🚂 CI

🧪 Testing Scenarios

Describe how the changes were tested and how reviewers can test them too:

  • ✅ Tested manually
  • 🤖 Ran automated end-to-end tests

✅ Checklist

Before submitting the PR, ensure the following:

  • 🔍 PR title is clear and descriptive
  • 📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
  • 💬 I have commented my code, especially in hard-to-understand areas
  • 📄 I have made corresponding changes to the documentation
  • ✅ I have added tests that prove my fix is effective or my feature works

@dpitulax dpitulax self-assigned this Jan 21, 2026
ltalarcz

This comment was marked as resolved.

ltalarcz

This comment was marked as resolved.

@dpitulax dpitulax enabled auto-merge (squash) January 27, 2026 18:55
@dpitulax dpitulax requested a review from ltalarcz January 27, 2026 18:55
@dpitulax dpitulax disabled auto-merge January 27, 2026 18:55
@dpitulax dpitulax enabled auto-merge (squash) January 27, 2026 19:13
@dpitulax dpitulax disabled auto-merge January 29, 2026 16:55
@dpitulax dpitulax enabled auto-merge (squash) January 30, 2026 00:09
@dpitulax
Copy link
Contributor Author

here is the latest execution of All Tests: https://github.com/open-edge-platform/scenescape/actions/runs/21493175765

@ltalarcz ltalarcz changed the title CI All Tests Workflow rework ITEP-84743 CI All Tests Workflow rework Jan 30, 2026
@ltalarcz ltalarcz requested a review from Copilot January 30, 2026 09:37
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reorganizes the CI test workflow to improve debugging and rerun capabilities by splitting functional and UI test groups into smaller, focused sub-groups. The changes remove the Basic Acceptance Tests (BATs) job and redistribute remaining tests into their respective categories, eliminating test execution duplication.

Changes:

  • Split functional and UI tests into 6 sub-groups each (scenes, sensors, cameras, mqtt, objects, generic) with matrix strategy for parallel execution
  • Added parameterizable test group selection via FUNC_GROUP and UI_GROUP variables
  • Enhanced artifact naming with unique identifiers per job/matrix combination to prevent conflicts
  • Updated copyright years to 2026 across modified files

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
tests/Makefile Reorganizes test targets by splitting functional-tests and ui-tests into 6 sub-groups each, moves calibrate-camera-3d-ui-2d-ui to broken features
Makefile Adds FUNC_GROUP and UI_GROUP variables for parameterized test execution
.github/workflows/tests-all.yml Removes BATs job, converts functional and UI tests to matrix strategy with 6 sub-groups each, adds unique artifact naming
.github/actions/upload-artifacts/action.yaml Parameterizes artifact naming and file paths with configurable inputs and retention policy

Comment on lines 163 to 166
name: 'UI Tests (${{ fromJSON(''{"ui-scenes":"Scenes",
"ui-sensors":"Sensors","ui-cameras":"Cameras",
"ui-mqtt":"MQTT","ui-objects":"Objects",
"ui-generic":"Generic"}'')[matrix.group] }})'
Copy link

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a duplicate of the same JSON mapping pattern used in functional tests. The mapping between test groups and display labels should be defined once and reused to prevent inconsistencies if labels need to change.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@ltalarcz ltalarcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In artifacts from All Tests run I can see only logs from the last test

Image

@dpitulax dpitulax disabled auto-merge January 30, 2026 11:15
@dpitulax
Copy link
Contributor Author

In artifacts from All Tests run I can see only logs from the last test
This is an intended behaviour. In the summary view only logs/reports from the latest run attempt are shown, meaning if you want to access specific logs you need to either choose in the top right corner the attempt that chosen group last passed, or directly enter the group and download logs from "Upload Logs and Reports" section.

@ltalarcz
Copy link
Contributor

ltalarcz commented Feb 2, 2026

In artifacts from All Tests run I can see only logs from the last test
This is an intended behaviour. In the summary view only logs/reports from the latest run attempt are shown, meaning if you want to access specific logs you need to either choose in the top right corner the attempt that chosen group last passed, or directly enter the group and download logs from "Upload Logs and Reports" section.

Please clarify the logic here, why am I seeing only chosen reports and logs with some postfix "attemp", while all tests are running?
Examples:
only UI objects
only Metrics

I'd expect the same behavior as it is currently on main branch, where I can see the reports and logs from all test runs.

Edit. Clarified offline. Looks like it's the same behavior as on main branch.

@Irakus
Copy link
Contributor

Irakus commented Feb 2, 2026

In artifacts from All Tests run I can see only logs from the last test
This is an intended behaviour. In the summary view only logs/reports from the latest run attempt are shown, meaning if you want to access specific logs you need to either choose in the top right corner the attempt that chosen group last passed, or directly enter the group and download logs from "Upload Logs and Reports" section.

Please clarify the logic here, why am I seeing only chosen reports and logs with some postfix "attemp", while all tests are running? Examples: only UI objects only Metrics

I'd expect the same behavior as it is currently on main branch, where I can see the reports and logs from all test runs.

This is the same as in main branch. e.g. https://github.com/open-edge-platform/scenescape/actions/runs/21483221999

@Irakus
Copy link
Contributor

Irakus commented Feb 2, 2026

Can we not add attemptX to artifacts names? There's only 1 artifact per attempt nevertheless.

@dpitulax
Copy link
Contributor Author

Can we not add attemptX to artifacts names? There's only 1 artifact per attempt nevertheless.

I removed attempt suffix from the log names in a previous commit

@dpitulax dpitulax marked this pull request as draft February 19, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants