Feat: devcontainer pytest integration by junkataoka · Pull Request #110 · microsoft/dstoolkit-devcontainers

junkataoka · 2025-12-16T07:58:38Z

This pull request introduces a major refactor and modernization of the CI pipeline infrastructure for both Azure DevOps and GitHub Actions. The main improvements are the switch to modular, template-driven pipeline definitions, enhanced support for devcontainer-based testing, and improved code coverage reporting. The changes streamline CI setup, make it easier to add new projects, and improve compatibility across platforms (Linux and macOS). Documentation and configuration files have also been updated for clarity and consistency.

Azure DevOps Pipeline Modernization

Both .azuredevops/ado-ci-pipeline-ms-hosted.yml and .azuredevops/ado-ci-pipeline-self-hosted.yml are refactored to use a two-stage pipeline (Lint and Test), with jobs and steps defined via new templates for devcontainer testing and coverage publishing. This replaces legacy scripts and direct task definitions, making the pipeline more maintainable and extensible. [1] [2] [3]

Template-Driven CI Architecture

New reusable templates are introduced under .azuredevops/templates/, including test-devcontainer-job.yml for per-project devcontainer testing, run-devcontainer.yml for platform-aware container execution, setup-devcontainer.yml for environment setup, publish-test-results.yml for publishing test results and coverage, and merge-coverage.yml for merging and publishing code coverage reports. [1] [2] [3] [4] [5]

GitHub Actions Pipeline Improvements

.github/workflows/ci.yaml is reworked to use matrix jobs for devcontainer-based testing, artifact upload for results and coverage, and a dedicated lint job. A new job aggregates and publishes combined test results and coverage reports, improving visibility and maintainability. [1] [2]

Documentation and Configuration Updates

References to legacy scripts (e.g., ci-tests.sh) are removed from README.md and .env.example is updated for clarity, reflecting the new devcontainer and pipeline workflow. [1] [2] [3]

Platform Compatibility and Usability Enhancements

The new pipeline templates include logic for handling both Linux and macOS (Apple Silicon) agents, and ensure clean workspace setup and artifact handling for reliable CI runs. [1] [2]

Does this introduce a breaking change?

Yes
No

Author pre-publish checklist

No PII in logs or output
Made corresponding changes to the documentation
All new packages used are included in requirements.txt
Functions use type hints, and there are no type hint errors

Pull Request Type

What kind of change does this Pull Request introduce?

junkataoka · 2025-12-16T08:21:58Z

@junkataoka please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement

@microsoft-github-policy-service agree [company="microsoft"]

junkataoka · 2025-12-16T16:39:44Z

@microsoft-github-policy-service agree
@microsoft-github-policy-service agree company="Microsoft"

- Add GitHub Actions workflow using official devcontainers/ci@v0.3 action - Add Azure DevOps pipeline with custom templates for macOS compatibility - Configure pytest with coverage reporting and JUnit XML output - Handle workspace permission issues using /tmp with sudo - Support CPU, GPU, and notebooks project configurations - Add coverage merging and test result publishing for both platforms - Skip pre-commit hook installation in CI when .git not available - Add sync-to-azdo workflow for pipeline synchronization - Update documentation with CI/CD setup instructions GitHub Actions uses the official action which works well on Linux. Azure DevOps uses custom templates to handle macOS-specific Docker issues (path translation and working directory mount namespace errors).

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

fujikosu · 2026-02-06T10:18:29Z

                }
-            },
+            }
        }
-    },
+    }


is this your vscode's setting? it'd be good to keep diff minimal as this PR is quite large and format change may not be worth including?

Reverted here as well — trailing commas restored to match the original style.

fujikosu · 2026-02-06T10:21:47Z

+    "workspaceMount": "source=${localWorkspaceFolder},target=/workspaces/repo,type=bind",
+    "workspaceFolder": "/workspaces/repo",
+    "remoteUser": "devuser",
+    "postCreateCommand": "cd /workspaces/repo && pre-commit install --overwrite || echo 'pre-commit installation failed; continuing without pre-commit hooks'",


is there any benefit of still going ahead when pre-commit fails? I'd rather it fail there as something would be wrong in that case

Agreed — removed the || echo fallback so pre-commit failures are surfaced properly instead of being silently swallowed.

fujikosu · 2026-02-06T15:47:54Z

-# but could be adapated to other OSs.
-
+# Azure DevOps CI Pipeline (Self-Hosted Agent)
+#


this 10GB part was the reason why we needed two different yamls. Could you check if it's ok to delete this part? If we still have this restriction, we likely need some cleanup method we had for self-hosted version specifically

Good catch — restored the 10GB context comment. This is the key reason the self-hosted pipeline exists, so it's important to keep that documented.

fujikosu · 2026-02-06T15:50:04Z

-      - name: Checkout code
-        uses: actions/checkout@v6
+      - name: Checkout repository
+        uses: actions/checkout@v4


why does this degrade version?

Fixed — upgraded action versions to match main branch: checkout@v6, setup-python@v6, upload-artifact@v7. The downgrades were unintentional.

fujikosu · 2026-02-06T15:52:53Z

+          filename: 'coverage/**/*.xml'

-      - name: Add code coverage summary markdown to github step summary
+      - name: Add code coverage summary to job summary


I think this is not accurate?

Reverted the step name back to the original: "Add code coverage summary markdown to github step summary".

fujikosu · 2026-02-06T15:58:55Z

@@ -13,44 +13,109 @@ permissions:
  contents: read


I wonder if it's really worth splitting into several jobs instead of having having them as multiple steps based on job complextity we have. One thing I noticed is this does repo clone 4 times as it used to be just one for example and that's one of downsides of going with this structure

Agreed — consolidated everything back into a single build job to avoid multiple repo clones. Lint runs first as steps, followed by sequential devcontainer tests (cpu, gpu, notebooks), then publish results. Keeps the structure simple while using the new devcontainer-based testing approach.

Thanks. This simple structure looks better. Should we follow the same approach on AzDO side and remove templates and revert back to the original approach while updating test step only to keep the update simple?

fujikosu

Thanks for your PR! I left several comments so please check them

- Remove .github/agents/ .github/prompts/ and speckit entries from .gitignore - Restore PYTHONPATH in .env.example - Remove containerEnv (PYTHONPATH via .env auto-loads) - Remove redundant remoteUser (set by Dockerfile USER) - Remove pre-commit failure suppression (|| echo) - Restore trailing commas in JSONC files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

junkataoka · 2026-03-31T00:36:21Z

@fujikosu Thank you for the thorough review! I've addressed all the comments — really appreciate you taking the time to go through everything in detail. The feedback helped improve the PR significantly.

fujikosu · 2026-04-02T08:32:14Z

          name: pytest
-          path: |
-            **/test-results-*.xml
+          path: 'test-results-*.xml'


what's this path udpate for?

fujikosu · 2026-04-02T08:32:31Z

          output: both
          format: markdown
-          filename: coverage.xml
+          filename: 'coverage-*.xml'


what's this filename update for?

fujikosu · 2026-04-02T08:32:48Z

+            test-results-*.xml
+            coverage-*.xml


what are these path update for?

fujikosu

thanks for the update! I added comments to your update

junkataoka force-pushed the feature/devcontainer-pytest-integration branch from 44faaca to ec09cba Compare December 16, 2025 08:43

junkataoka force-pushed the feature/devcontainer-pytest-integration branch 5 times, most recently from 2523a23 to 0ebed45 Compare December 18, 2025 15:06

junkataoka changed the title ~~Feature/devcontainer pytest integration~~ feat: devcontainer pytest integration Dec 18, 2025

junkataoka changed the title ~~feat: devcontainer pytest integration~~ Feat: devcontainer pytest integration Dec 18, 2025

junkataoka force-pushed the feature/devcontainer-pytest-integration branch from d73bb8e to 82fb7b9 Compare December 19, 2025 00:10

junkataoka requested a review from Copilot December 19, 2025 00:20

Copilot started reviewing on behalf of junkataoka December 19, 2025 00:20 View session

This comment was marked as outdated.

Sign in to view

junkataoka force-pushed the feature/devcontainer-pytest-integration branch from fe5a82e to 78f787f Compare December 19, 2025 01:29

junkataoka requested a review from Copilot December 19, 2025 01:48

Copilot started reviewing on behalf of junkataoka December 19, 2025 01:48 View session

This comment was marked as outdated.

Sign in to view

junkataoka force-pushed the feature/devcontainer-pytest-integration branch 3 times, most recently from f22f8ef to d94a2d6 Compare December 19, 2025 02:55

junkataoka force-pushed the feature/devcontainer-pytest-integration branch from 672b9c6 to 164e6b9 Compare December 19, 2025 04:51

patch: remove azdo sync pipeline as it was for testing purpose

5f6316e

junkataoka force-pushed the feature/devcontainer-pytest-integration branch from a279838 to 5f6316e Compare December 19, 2025 05:03

junkataoka requested a review from Copilot December 19, 2025 05:04

Copilot started reviewing on behalf of junkataoka December 19, 2025 05:05 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

Comment thread pyproject.toml

Comment thread .github/workflows/ci.yaml Outdated

junkataoka marked this pull request as ready for review December 19, 2025 05:21

junkataoka requested a review from bhavikm December 19, 2025 05:21