feat(dev-exp): FTRS-3011 Central management of Python dependencies, packages & workspaces

[Tomdango]

Description

Migrates the entire monorepo from per-service Poetry environments to a single uv workspace.

Previously every service had its own poetry.lock, its own .venv, and its own poetry install step. Internal packages were referenced via fragile relative paths ({path = "../../application/packages/python", develop = true}), there was no unified lockfile, and CI had to rebuild environments from scratch on every job. This replaces all of that with a single .venv at the repo root, a single uv.lock, and {workspace = true} sources for internal packages.

What's changed:

• Root pyproject.toml added: declares all workspace members and exclusions, shared dev tooling (ruff, pytest, mypy), and shared [tool.uv.sources]
• All service pyproject.toml files migrated from poetry-core to hatchling build backend; [tool.poetry.group.dev.dependencies] → [dependency-groups] dev
• Per-service .tool-versions files removed (they only ever pinned the same python/poetry versions as the root anyway)
• scripts/services/python-service.mk updated throughout: poetry install → uv sync, poetry run → uv run, poetry build → uv build --wheel, poetry export → uv export --no-dev --no-hashes --no-emit-workspace
• poetry.lock files deleted from all services, replaced by the root uv.lock
• .tool-versions updated: poetry 2.1.1 → uv 0.10.9; also added allure, java, and jmeter for performance test tooling

CI:

• New .github/actions/setup-asdf and .github/actions/setup-uv actions to stop the 3-step asdf setup being copy-pasted across every workflow
• Inline pip install steps in composite actions replaced with setup-uv + uvx --with <packages> for one-off script runs
• New warm-dependency-caches.yaml reusable workflow that runs before quality-checks and service-automation-test to pre-warm caches rather than each job doing it independently
• Service automation cache key updated from poetry.lock → uv.lock

crud-apis:

• poethepoet removed; start-* scripts are now proper [project.scripts] entry points backed by uvicorn run() functions in each module's local.py
• fastapi upper bound relaxed to <1.0.0

Other fixes snuck in:

• OrganizationMapper tests were asserting date values as ISO strings ("2020-01-15"), corrected to date objects
• gitleaks.toml allowlist updated from poetry.lock → uv.lock
• ftrs-aws-local CLI fixed to use a module-level logger instead of calling logging.info directly

Developer migration guide added at docs/developer-guides/Migrating_to_UV.md.

---

Context

The per-service Poetry setup was getting increasingly painful. Running tests across services meant juggling multiple shells with different activated environments, VS Code would only resolve imports correctly for whichever service you'd last pointed it at, and CI was slow because there was no good caching story. uv workspaces fix all of this cleanly.

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

[Copilot]

Pull request overview

This PR migrates the monorepo’s Python dependency and build tooling from Poetry to uv workspaces + Hatchling, updating service configs, Makefiles, and CI workflows to use uv sync/run/export/build and centralising tool versions at the repo root.

Changes:

• Introduces a root pyproject.toml with a uv workspace and shared dev tooling, plus a generated uv.lock.
• Replaces Poetry configuration across multiple services/tests with PEP 621 + Hatchling and uv dependency groups/sources.
• Updates Makefiles and GitHub Actions workflows/composite actions to install and run tooling via uv (and removes many per-project .tool-versions / poetry.lock files).

Reviewed changes

Copilot reviewed 71 out of 83 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/service_automation/tests/step_definitions/data_migration_steps/test_audit_event_tracking_steps.py	Minor Python simplification (or []).
tests/service_automation/pyproject.toml	Switch to Hatchling + uv workspace sources/dependency groups.
tests/service_automation/README.md	Formatting tweaks.
tests/service_automation/Makefile	Move install/lint/test commands from Poetry to uv.
tests/service_automation/.tool-versions	Removed in favour of repo root tool versions.
tests/performance/scripts/data_generation/pyproject.toml	Switch scripts/config to PEP 621 + Hatchling.
tests/performance/scripts/data_generation/poetry.lock	Removed.
tests/performance/scripts/data_generation/.tool-versions	Removed.
tests/performance/Makefile	Use uv to run data generation.
tests/performance/.tool-versions	Removed; tool versions moved to root.
services/slack-notifier/pyproject.toml	Move from Poetry to Hatchling + uv sources/dev deps.
services/slack-notifier/Makefile	Replace Poetry version extraction with parsing pyproject.toml.
services/slack-notifier/.tool-versions	Removed.
services/sandbox-dos-search/pyproject.toml	Move from Poetry to Hatchling + dependency groups.
services/sandbox-dos-search/poetry.lock	Removed.
services/sandbox-dos-search/Dockerfile	Replace Poetry install with uv-based install.
services/sandbox-dos-search/.dockerignore	Ignore lock files; allow README for build backend.
services/sandbox-dos-ingest/pyproject.toml	Move from Poetry to Hatchling/uv settings.
services/etl-ods/pyproject.toml	Move from Poetry to Hatchling + uv sources/dev deps.
services/etl-ods/README.md	Update instructions from Poetry to uv.
services/etl-ods/Makefile	Use uv for tests and exporting requirements.
services/dos-search/pyproject.toml	Move from Poetry to Hatchling + uv sources/dev deps.
services/dos-search/opensearch_local_index_populator/README.md	Update run command to uv run.
services/dos-search/README.md	Update dependency and test commands to uv.
services/dos-search/Makefile	Remove service-specific Poetry export logic; rely on shared uv build flow.
services/dos-search/.tool-versions	Removed.
services/data-migration/tests/unit/common/test_secret_utils.py	Simplify os.getenv mocking using .get.
services/data-migration/src/queue_populator/lambda_handler.py	Minor Python simplification (or 0).
services/data-migration/pyproject.toml	Move from Poetry to Hatchling + uv sources/dev deps.
services/data-migration/events/README.md	Update local run instructions away from Poetry env activation.
services/data-migration/README.md	Update install/lint/test/docs to uv.
services/data-migration/Makefile	Update CLI invocations to uv run.
services/data-migration/.tool-versions	Removed.
services/crud-apis/pyproject.toml	Move from Poetry (+ poe tasks) to Hatchling + uv sources/dev deps + scripts.
services/crud-apis/organisations/tests/unit/test_models_organisation.py	Update date assertions to datetime.date.
services/crud-apis/organisations/local.py	Add uvicorn-based local entry point.
services/crud-apis/organisations/README.md	Update install/lint/test commands to uv.
services/crud-apis/location/local.py	Add uvicorn-based local entry point.
services/crud-apis/location/README.md	Update install/lint/test commands to uv.
services/crud-apis/healthcare_service/local.py	Add uvicorn-based local entry point.
services/crud-apis/healthcare_service/README.md	Update install/lint/test commands to uv.
services/crud-apis/handler_main.py	Add uvicorn-based local entry point.
services/crud-apis/Makefile	Switch from Poetry to uv for coverage and local run targets.
services/crud-apis/.tool-versions	Removed.
scripts/workflow/tests/open_search_index/pyproject.toml	Move to uv dependency groups and disable packaging.
scripts/workflow/tests/open_search_index/poetry.lock	Removed.
scripts/workflow/init.mk	Stop installing Poetry via asdf.
scripts/services/python-service.mk	Replace Poetry-based install/lint/test/build/export with uv equivalents.
scripts/config/vale/styles/config/vocabularies/words/accept.txt	Allow “uv” in Vale vocabulary.
scripts/config/gitleaks.toml	Update allowlist paths to prefer uv.lock.
requirements/service-specs/etl-ods.md	Update dependency install instructions to uv.
requirements/service-specs/crud-apis.md	Update setup/test instructions to uv.
pyproject.toml	New repo root PEP 621 project + uv workspace + shared tooling configuration.
application/packages/python/pyproject.toml	Move common packages project to Hatchling + uv dev groups.
application/packages/python/ftrs_common/tests/fhir/test_organisation_mapper.py	Update date assertions to datetime.date.
application/packages/python/README.md	Update instructions from Poetry to uv.
application/packages/python/Makefile	Replace Poetry build/install/test commands with uv equivalents.
application/packages/ftrs_aws_local/pyproject.toml	Move from Poetry to Hatchling + uv sources/dev deps + scripts.
application/packages/ftrs_aws_local/dynamodb/cli.py	Switch to module logger instead of root logging.
application/packages/ftrs_aws_local/README.md	Update instructions from Poetry to uv.
.tool-versions	Add uv and centralise allure/java/jmeter versions at repo root.
.github/workflows/warm-dependency-caches.yaml	New workflow to warm uv caches.
.github/workflows/service-automation-test.yaml	Replace Poetry caching/setup with composite asdf+uv setup and uv sync.
.github/workflows/quality-checks.yaml	Update permissions and switch service automation lint job to uv sync.
.github/workflows/prepare-toggle-artifacts.yaml	Install Python deps via setup-uv instead of pip steps.
.github/workflows/pipeline-deploy-application.yaml	Add warm cache job and update dependencies between jobs.
.github/workflows/build-project.yaml	Switch asdf setup to local composite action.
.github/actions/setup-uv/action.yaml	New composite action for uv install/caching (+ optional venv/package install).
.github/actions/setup-asdf/action.yaml	New composite action to centralise asdf install/cache/install-tools.
.github/actions/populate-open-search-index/action.yaml	Use uvx for ephemeral deps instead of pip installs.
.github/actions/deploy-to-apim/action.yaml	Use setup-uv for pyyaml instead of pip installs.
.github/actions/create-open-search-index/action.yaml	Use uvx for ephemeral deps instead of pip installs.
.github/actions/authenticate-apim/action.yaml	Use setup-uv for pyjwt/cryptography instead of pip installs.