diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index d6bd845..433ca42 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -18,6 +18,7 @@ "nefrob.vscode-just-syntax" ], "settings": { + "terminal.integrated.defaultProfile.linux": "bash", "python.defaultInterpreterPath": "/root/assets/workspace/.venv/bin/python", "[python]": { "editor.defaultFormatter": "charliermarsh.ruff", diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e3432b..27c1a6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Cross-language conformance test suite** ([#155](https://github.com/vig-os/fd5/issues/155)) + - 6 canonical fixture generators: minimal, sealed, with-provenance, multiscale, tabular, complex-metadata + - 3 invalid fixture generators: missing-id, bad-hash, no-schema + - Expected-result JSON files defining the format contract for any language binding + - 39 pytest conformance tests covering structure, hash verification, provenance, multiscale, tabular, metadata, schema validation, and negative tests + - README documenting how to use the suite and add new cases + - **Preflight feedback and status dashboard for devc-remote** ([#149](https://github.com/vig-os/fd5/issues/149)) - Each preflight check now prints a success/warning/error status line as it completes - New checks: container-already-running, runtime version, compose version, SSH agent forwarding diff --git a/_typos.toml b/_typos.toml new file mode 100644 index 0000000..2a2836c --- /dev/null +++ b/_typos.toml @@ -0,0 +1,3 @@ +[default.extend-words] +OME = "OME" +tre = "tre" diff --git a/docs/issues/issue-155.md b/docs/issues/issue-155.md index 95cbacc..44ecb1a 100644 --- a/docs/issues/issue-155.md +++ b/docs/issues/issue-155.md @@ -2,17 +2,17 @@ type: issue state: open created: 2026-02-26T01:04:00Z -updated: 2026-02-26T01:04:00Z +updated: 2026-02-26T10:09:09Z author: gerchowl author_url: https://github.com/gerchowl url: https://github.com/vig-os/fd5/issues/155 -comments: 0 +comments: 3 labels: priority:high, area:testing, effort:medium, area:core -assignees: none +assignees: gerchowl milestone: none projects: none relationship: none -synced: 2026-02-26T04:15:42.619Z +synced: 2026-02-27T04:11:40.452Z --- # [Issue 155]: [[TEST] Cross-language conformance test suite for fd5 format](https://github.com/vig-os/fd5/issues/155) @@ -86,3 +86,115 @@ This is a black-box test — it doesn't test internal APIs, only the format cont - Depends on the format spec document (see prerequisite issue) for normative requirements - Fixture files should be small (KBs, not MBs) to keep the repo lightweight - Inspired by JSON Schema Test Suite: https://github.com/json-schema-org/JSON-Schema-Test-Suite +--- + +# [Comment #1]() by [gerchowl]() + +_Posted on February 26, 2026 at 09:47 AM_ + +## Design + +### Overview + +A Python script (`tests/conformance/generate_fixtures.py`) will use the existing `fd5.create()` API and direct `h5py` calls to generate canonical fixture files and corresponding expected-result JSON files. A pytest-based conformance runner (`tests/conformance/test_conformance.py`) will validate that the Python implementation passes all cases. The suite is designed so any future language binding can load the same fixtures + JSON and assert equivalence. + +### Architecture + +``` +tests/conformance/ +├── README.md # How to use the suite +├── generate_fixtures.py # Script that creates all .fd5 + .json files +├── fixtures/ # Generated .fd5 files (gitignored, regenerated in CI) +├── expected/ # Expected-result JSON files (checked in) +├── invalid/ # Invalid .fd5 files + expected-errors.json +└── test_conformance.py # Pytest runner that validates fixtures vs expected +``` + +### Design Decisions + +1. **Fixtures are generated, not checked in.** HDF5 is binary; checking in binaries is fragile and bloats the repo. Instead, `generate_fixtures.py` regenerates them deterministically. The expected JSON files ARE checked in since they define the contract. A conftest fixture runs the generator before tests. + +2. **Use a test/conformance product schema.** Register a minimal `ConformanceSchema` via `register_schema()` in the generator and test module. This avoids coupling to imaging-specific schemas (recon) while exercising the full fd5 create/seal pipeline. + +3. **Expected JSON format.** Each expected JSON file is a dict with keys matching the test categories from the issue: `root_attrs`, `datasets`, `groups`, `content_hash_prefix`, `verify`, plus fixture-specific keys like `provenance`, `metadata_tree`, etc. + +4. **Test categories mapped to fixtures:** minimal.fd5 (structure), with-provenance.fd5 (provenance DAG), multiscale.fd5 (pyramid levels), tabular.fd5 (compound dataset), complex-metadata.fd5 (nested metadata), sealed.fd5 (hash verification). + +5. **Invalid fixtures.** Created with direct h5py: missing-id.fd5, bad-hash.fd5, no-schema.fd5, with expected-errors.json. + +6. **Multiscale fixture uses ReconSchema.** Only fixture needing a real product schema with pyramid support. All others use the simple conformance schema. + +### Testing Strategy + +The conformance tests ARE the tests. `test_conformance.py` covers structure, round-trip, hash verification, provenance, schema validation, and negative tests. No separate unit tests for the generator. + +### Constraints + +- Fixture files stay small (< 10 KB each) +- No new dependencies +- Generator uses only public fd5 API where possible, h5py directly for invalid fixtures + +--- + +# [Comment #2]() by [gerchowl]() + +_Posted on February 26, 2026 at 09:47 AM_ + +## Implementation Plan + +Issue: #155 +Branch: feature/155-cross-language-conformance-tests + +### Tasks + +- [ ] Task 1: Create conformance directory structure and README — `tests/conformance/README.md`, `tests/conformance/__init__.py` — verify: files exist +- [ ] Task 2: Write expected JSON files for valid fixtures — `tests/conformance/expected/minimal.json`, `with-provenance.json`, `multiscale.json`, `tabular.json`, `complex-metadata.json`, `sealed.json` — verify: valid JSON, all test categories covered +- [ ] Task 3: Write expected-errors JSON for invalid fixtures — `tests/conformance/invalid/expected-errors.json` — verify: valid JSON with error patterns for missing-id, bad-hash, no-schema +- [ ] Task 4: Write failing conformance tests for structure tests (minimal fixture) — `tests/conformance/test_conformance.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k structure -v` fails (no fixtures yet) +- [ ] Task 5: Write fixture generator for minimal.fd5 — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k structure -v` passes +- [ ] Task 6: Write failing tests for hash verification (sealed fixture) — `tests/conformance/test_conformance.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k hash -v` fails +- [ ] Task 7: Write fixture generator for sealed.fd5 — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k hash -v` passes +- [ ] Task 8: Write failing tests for provenance (with-provenance fixture) — `tests/conformance/test_conformance.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k provenance -v` fails +- [ ] Task 9: Write fixture generator for with-provenance.fd5 — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k provenance -v` passes +- [ ] Task 10: Write failing tests for multiscale fixture — `tests/conformance/test_conformance.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k multiscale -v` fails +- [ ] Task 11: Write fixture generator for multiscale.fd5 — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k multiscale -v` passes +- [ ] Task 12: Write failing tests for tabular and complex-metadata fixtures — `tests/conformance/test_conformance.py` — verify: fails +- [ ] Task 13: Write fixture generators for tabular.fd5 and complex-metadata.fd5 — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k "tabular or complex" -v` passes +- [ ] Task 14: Write failing tests for invalid/negative fixtures — `tests/conformance/test_conformance.py` — verify: fails +- [ ] Task 15: Write fixture generators for invalid files (missing-id, bad-hash, no-schema) — `tests/conformance/generate_fixtures.py` — verify: `uv run pytest tests/conformance/test_conformance.py -k invalid -v` passes +- [ ] Task 16: Wire conformance tests into full test suite and add .gitignore for fixtures/ — verify: `just test` passes including conformance + +--- + +# [Comment #3]() by [gerchowl]() + +_Posted on February 26, 2026 at 10:09 AM_ + +## Implementation Complete — PR #157 + +**PR**: https://github.com/vig-os/fd5/pull/157 + +### What was implemented + +A cross-language conformance test suite for the `fd5` format, comprising: + +**Fixture generator** (`tests/conformance/generate_fixtures.py`): +- Generates 6 valid `.fd5` fixture files: `minimal`, `sealed`, `with-provenance`, `multiscale`, `tabular`, `complex-metadata` +- Generates 3 invalid fixtures: `invalid-missing-id`, `invalid-bad-hash`, `invalid-no-schema` +- Uses a dedicated `_ConformanceSchema` to avoid polluting the global schema registry + +**Expected results** (`tests/conformance/expected/`): +- JSON files defining expected structure, metadata, hash verification, and schema validation results for each fixture +- `invalid/expected-errors.json` for invalid fixture error expectations + +**Conformance tests** (`tests/conformance/test_conformance.py`): +- 40 parameterized pytest tests across 6 categories: structure, metadata, content hash, verification, schema validation, and invalid file handling +- Session-scoped fixture generation with proper registry cleanup + +**Documentation** (`tests/conformance/README.md`): +- Describes the suite's purpose, fixture inventory, JSON contract format, and how other language implementations can use the fixtures + +### CI Note + +CI failures are pre-existing across all repo branches (missing optional deps `pydicom`/`nibabel`/`pyarrow` in CI environment). See PR comment for details. All conformance tests pass locally. + diff --git a/docs/issues/issue-156.md b/docs/issues/issue-156.md new file mode 100644 index 0000000..b440a17 --- /dev/null +++ b/docs/issues/issue-156.md @@ -0,0 +1,66 @@ +--- +type: issue +state: open +created: 2026-02-26T08:09:28Z +updated: 2026-02-26T08:09:45Z +author: gerchowl +author_url: https://github.com/gerchowl +url: https://github.com/vig-os/fd5/issues/156 +comments: 0 +labels: bug, area:workflow, effort:small, semver:patch +assignees: gerchowl +milestone: none +projects: none +relationship: none +synced: 2026-02-27T04:11:40.009Z +--- + +# [Issue 156]: [[BUG] devc-remote.sh compose commands run from repo root instead of .devcontainer/](https://github.com/vig-os/fd5/issues/156) + +## Description + +`scripts/devc-remote.sh` runs all `podman compose` / `docker compose` commands from `$REMOTE_PATH` (the repo root, e.g. `~/fd5`), but the compose files (`docker-compose.yml`, `docker-compose.project.yaml`, `docker-compose.local.yaml`) live in `$REMOTE_PATH/.devcontainer/`. The standalone `docker-compose` binary (used as podman's external compose provider) fails with "no configuration file provided: not found". + +## Steps to Reproduce + +1. Run `just devc-remote ksb-meatgrinder:~/fd5` +2. Pre-flight passes successfully +3. `remote_compose_up()` executes `cd ~/fd5 && podman compose up -d` +4. `docker-compose` (external provider) can't find any compose file in `~/fd5` + +## Expected Behavior + +Compose commands should `cd` into `$REMOTE_PATH/.devcontainer` where the compose files reside, so `podman compose up -d` succeeds. + +## Actual Behavior + +``` +>>>> Executing external compose provider "/usr/local/bin/docker-compose". <<<< +no configuration file provided: not found +Error: executing /usr/local/bin/docker-compose up -d: exit status 1 +``` + +## Environment + +- **OS**: macOS 24.5.0 (host) → Linux (remote: ksb-meatgrinder) +- **Container Runtime**: Podman 4.9.3 (remote) +- **Compose**: docker-compose v5.1.0 (standalone, used as podman's external compose provider) + +## Additional Context + +All compose-related SSH commands in the script use `cd $REMOTE_PATH` instead of `cd $REMOTE_PATH/.devcontainer`: +- Line 218/220 (preflight container check) +- Line 351 (`compose_ps_json`) +- Line 383 (`check_existing_container` down) +- Line 409 (`remote_compose_up`) +- Line 411 (error hint message) + +## Possible Solution + +Change all `cd $REMOTE_PATH` to `cd $REMOTE_PATH/.devcontainer` in compose-related commands. + +## Changelog Category + +Fixed + +- [ ] TDD compliance (see .cursor/rules/tdd.mdc) diff --git a/docs/issues/issue-158.md b/docs/issues/issue-158.md new file mode 100644 index 0000000..db54e85 --- /dev/null +++ b/docs/issues/issue-158.md @@ -0,0 +1,58 @@ +--- +type: issue +state: open +created: 2026-02-26T10:44:02Z +updated: 2026-02-26T10:44:28Z +author: gerchowl +author_url: https://github.com/gerchowl +url: https://github.com/vig-os/fd5/issues/158 +comments: 0 +labels: chore, area:workflow, effort:medium +assignees: gerchowl +milestone: none +projects: none +relationship: none +synced: 2026-02-27T04:11:39.605Z +--- + +# [Issue 158]: [[CHORE] Add opt-in Tailscale SSH to devcontainer](https://github.com/vig-os/fd5/issues/158) + +### Chore Type + +Configuration change + +### Description + +Add opt-in Tailscale SSH support to the devcontainer so developers can connect via direct mesh SSH instead of the devcontainer protocol. This is a workaround for Cursor GUI's inability to execute agent shell commands when connected via the devcontainer protocol. + +When `TAILSCALE_AUTHKEY` is set (via `docker-compose.local.yaml`), the devcontainer installs Tailscale on first create and connects to the tailnet on every start with SSH enabled. When the env var is unset, the scripts are a no-op — zero impact on normal usage. + +### Acceptance Criteria + +- [ ] New `setup-tailscale.sh` script with `install` and `start` subcommands +- [ ] `post-create.sh` calls `setup-tailscale.sh install` (no-op without `TAILSCALE_AUTHKEY`) +- [ ] `post-start.sh` calls `setup-tailscale.sh start` (no-op without `TAILSCALE_AUTHKEY`) +- [ ] `.devcontainer/README.md` updated with quick-start instructions +- [ ] Detailed design doc at `docs/tailscale-devcontainer.md` covering architecture decisions, user setup, known gaps, and upstream considerations +- [ ] `uv.lock` updated (incidental dependency sync) + +### Implementation Notes + +Files changed: +- **New:** `.devcontainer/scripts/setup-tailscale.sh` — single script, two subcommands (`install` / `start`), idempotent, uses userspace networking (`--tun=userspace-networking`) +- **Modified:** `.devcontainer/scripts/post-create.sh` — hooks `setup-tailscale.sh install` +- **Modified:** `.devcontainer/scripts/post-start.sh` — adds `SCRIPT_DIR` resolution, hooks `setup-tailscale.sh start` +- **Modified:** `.devcontainer/README.md` — new "Tailscale SSH" section +- **New:** `docs/tailscale-devcontainer.md` — full design doc with architecture table, setup guide, known gap (git signing), and upstream notes + +### Related Issues + +None + +### Priority + +Medium + +### Changelog Category + +Added diff --git a/docs/pull-requests/pr-153.md b/docs/pull-requests/pr-153.md index 0fbc22e..97f339f 100644 --- a/docs/pull-requests/pr-153.md +++ b/docs/pull-requests/pr-153.md @@ -1,9 +1,9 @@ --- type: pull_request -state: open +state: closed (merged) branch: feature/149-preflight-feedback → dev created: 2026-02-26T00:26:40Z -updated: 2026-02-26T00:26:42Z +updated: 2026-02-26T08:03:42Z author: gerchowl author_url: https://github.com/gerchowl url: https://github.com/vig-os/fd5/pull/153 @@ -13,7 +13,8 @@ assignees: gerchowl milestone: none projects: none relationship: none -synced: 2026-02-26T04:16:02.459Z +merged: 2026-02-26T08:03:42Z +synced: 2026-02-27T04:11:44.195Z --- # [PR 153](https://github.com/vig-os/fd5/pull/153) feat(devc-remote): add --yes flag, container prompt, and SSH agent improvements (#149) @@ -92,3 +93,18 @@ This is a follow-up to PR #151 which implemented the initial preflight feedback Refs: #149 + + +--- +--- + +## Commits + +### Commit 1: [0521e3f](https://github.com/vig-os/fd5/commit/0521e3fffbd6ca00a30c989d223b6ab04c0a9e46) by [gerchowl](https://github.com/gerchowl) on February 26, 2026 at 12:22 AM +test: add failing tests for --yes flag, path annotation, container prompt, SSH agent check, 231 files modified (tests/test_devc_remote_preflight.sh) + +### Commit 2: [0096477](https://github.com/vig-os/fd5/commit/0096477d415470872789df638c7f537ae6342c26) by [gerchowl](https://github.com/gerchowl) on February 26, 2026 at 12:24 AM +feat(devc-remote): add --yes flag, path annotations, container prompt, improved SSH agent check, 139 files modified (scripts/devc-remote.sh, tests/test_devc_remote_preflight.sh) + +### Commit 3: [7f636ac](https://github.com/vig-os/fd5/commit/7f636ac2e061a5cb4217c8da62dd7a0fbd3178b7) by [gerchowl](https://github.com/gerchowl) on February 26, 2026 at 12:25 AM +docs: update changelog for preflight feedback improvements, 4 files modified (CHANGELOG.md) diff --git a/docs/pull-requests/pr-157.md b/docs/pull-requests/pr-157.md new file mode 100644 index 0000000..a8cc5c1 --- /dev/null +++ b/docs/pull-requests/pr-157.md @@ -0,0 +1,117 @@ +--- +type: pull_request +state: open +branch: feature/155-cross-language-conformance-tests → dev +created: 2026-02-26T10:04:47Z +updated: 2026-02-26T10:08:56Z +author: gerchowl +author_url: https://github.com/gerchowl +url: https://github.com/vig-os/fd5/pull/157 +comments: 1 +labels: none +assignees: gerchowl +milestone: none +projects: none +relationship: none +synced: 2026-02-27T04:11:42.857Z +--- + +# [PR 157](https://github.com/vig-os/fd5/pull/157) feat: cross-language conformance test suite (#155) + +## Description + +Add a cross-language conformance test suite for the fd5 format. The suite defines canonical fixture files and expected-result JSON files that any fd5 implementation (Python, Rust, Julia, C/C++, TypeScript) must pass to prove format conformance. This is a prerequisite for multi-language fd5 bindings (#144). + +## Type of Change + +- [x] `feat` -- New feature +- [ ] `fix` -- Bug fix +- [ ] `docs` -- Documentation only +- [ ] `chore` -- Maintenance task (deps, config, etc.) +- [ ] `refactor` -- Code restructuring (no behavior change) +- [x] `test` -- Adding or updating tests +- [ ] `ci` -- CI/CD pipeline changes +- [ ] `build` -- Build system or dependency changes +- [ ] `revert` -- Reverts a previous commit +- [ ] `style` -- Code style (formatting, whitespace) + +### Modifiers + +- [ ] Breaking change (`!`) -- This change breaks backward compatibility + +## Changes Made + +- `tests/conformance/generate_fixtures.py` — Fixture generator producing 6 valid and 3 invalid fd5 files using the reference implementation +- `tests/conformance/test_conformance.py` — 39 pytest conformance tests across structure, hash verification, provenance, multiscale, tabular, metadata, schema validation, and negative tests +- `tests/conformance/expected/*.json` — Expected-result JSON files defining the format contract (minimal, sealed, with-provenance, multiscale, tabular, complex-metadata) +- `tests/conformance/invalid/expected-errors.json` — Expected error patterns for invalid fixtures (missing-id, bad-hash, no-schema) +- `tests/conformance/README.md` — Documentation on how to use the suite and add new cases +- `tests/conformance/fixtures/.gitignore`, `tests/conformance/invalid/.gitignore` — Exclude generated binary files from version control +- `CHANGELOG.md` — Added conformance test suite entry under Unreleased/Added + +## Changelog Entry + +### Added + +- **Cross-language conformance test suite** ([#155](https://github.com/vig-os/fd5/issues/155)) + - 6 canonical fixture generators: minimal, sealed, with-provenance, multiscale, tabular, complex-metadata + - 3 invalid fixture generators: missing-id, bad-hash, no-schema + - Expected-result JSON files defining the format contract for any language binding + - 39 pytest conformance tests covering structure, hash verification, provenance, multiscale, tabular, metadata, schema validation, and negative tests + - README documenting how to use the suite and add new cases + +## Testing + +- [x] Tests pass locally (`just test`) +- [ ] Manual testing performed (describe below) + +### Manual Testing Details + +N/A + +## Checklist + +- [x] My code follows the project's style guidelines +- [x] I have performed a self-review of my code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have updated the documentation accordingly (edit `docs/templates/`, then run `just docs`) +- [x] I have updated `CHANGELOG.md` in the `[Unreleased]` section (and pasted the entry above) +- [x] My changes generate no new warnings or errors +- [x] I have added tests that prove my fix is effective or that my feature works +- [x] New and existing unit tests pass locally with my changes +- [ ] Any dependent changes have been merged and published + +## Additional Notes + +The conformance suite is designed to be language-agnostic. The `expected/*.json` files define the format contract — they specify what root attributes, dataset shapes, dtypes, group hierarchies, and provenance structures any compliant fd5 reader must be able to extract. Other languages implement their own test runner that opens the same fixtures and asserts against the same JSON. + +Fixture files are generated (not checked in) to avoid binary bloat. A session-scoped pytest fixture runs the generator before tests execute. + +The with-provenance fixture has `verify: false` because compound datasets with vlen strings produce non-deterministic `tobytes()` across file close/reopen, which is a known HDF5 behavior documented in `test_integration.py`. + +Refs: #155 + + + +--- +--- + +## Comments (1) + +### [Comment #1](https://github.com/vig-os/fd5/pull/157#issuecomment-3965564449) by [@gerchowl](https://github.com/gerchowl) + +_Posted on February 26, 2026 at 10:08 AM_ + +## CI Status Note + +The CI failures on this PR are **pre-existing** and affect all open PRs in this repo: + +- **Lint**: `uv.lock` drift — the lockfile on `dev` doesn't include optional deps (`pydicom`, `nibabel`, `pyarrow`) that were recently added to `pyproject.toml`. This causes pre-commit's lockfile check to fail. +- **Tests**: `ModuleNotFoundError` for `pydicom`, `nibabel`, `pyarrow` — the CI workflow doesn't install optional dependency groups, so `test_ingest_dicom.py`, `test_ingest_nifti.py`, and `test_ingest_parquet.py` fail at collection. + +The same failures are present on other branches (e.g., `feature/149-preflight-feedback` run [#22422258963](https://github.com/vig-os/fd5/actions/runs/22422258963)). + +**No conformance test failures** — our new tests were not reached due to the pre-existing collection errors, but they pass locally with no issues. + +--- + diff --git a/scripts/devc-remote.sh b/scripts/devc-remote.sh index 56b4f6e..3c44cd9 100755 --- a/scripts/devc-remote.sh +++ b/scripts/devc-remote.sh @@ -215,9 +215,9 @@ else echo "OS_TYPE=linux" fi # Check for a running devcontainer (compose project in REPO_PATH) -if command -v podman &>/dev/null && cd "$REPO_PATH" 2>/dev/null && podman compose ps --format json 2>/dev/null | grep -q '"running"'; then +if command -v podman &>/dev/null && cd "$REPO_PATH/.devcontainer" 2>/dev/null && podman compose ps --format json 2>/dev/null | grep -q '"running"'; then echo "CONTAINER_RUNNING=1" -elif command -v docker &>/dev/null && cd "$REPO_PATH" 2>/dev/null && docker compose ps --format json 2>/dev/null | grep -q '"running"'; then +elif command -v docker &>/dev/null && cd "$REPO_PATH/.devcontainer" 2>/dev/null && docker compose ps --format json 2>/dev/null | grep -q '"running"'; then echo "CONTAINER_RUNNING=1" else echo "CONTAINER_RUNNING=0" @@ -348,7 +348,31 @@ remote_init_if_needed() { compose_ps_json() { # shellcheck disable=SC2029 - ssh "$SSH_HOST" "cd $REMOTE_PATH && $COMPOSE_CMD ps --format json 2>/dev/null" || true + ssh "$SSH_HOST" "cd $REMOTE_PATH/.devcontainer && $COMPOSE_CMD ps --format json 2>/dev/null" || true +} + +resolve_remote_path_absolute() { + local path="$1" + local remote_home="" + + # shellcheck disable=SC2088 + if [[ "$path" == "~" || "$path" == "~/"* || "$path" != /* ]]; then + # shellcheck disable=SC2029 + remote_home=$(ssh "$SSH_HOST" 'printf %s "$HOME"') + fi + + # shellcheck disable=SC2088 + if [[ "$path" == "~" || "$path" == "~/"* ]]; then + if [[ "$path" == "~" ]]; then + path="$remote_home" + else + path="${remote_home}/${path#\~/}" + fi + elif [[ "$path" != /* ]]; then + path="${remote_home}/${path#./}" + fi + + echo "$path" } check_existing_container() { @@ -380,7 +404,7 @@ check_existing_container() { if [[ "${choice:-R}" == "r" ]]; then log_info "Recreating container..." # shellcheck disable=SC2029 - ssh "$SSH_HOST" "cd $REMOTE_PATH && $COMPOSE_CMD down" || true + ssh "$SSH_HOST" "cd $REMOTE_PATH/.devcontainer && $COMPOSE_CMD down" || true SKIP_COMPOSE_UP=0 else log_info "Reusing existing container" @@ -406,21 +430,23 @@ remote_compose_up() { log_info "Starting devcontainer on $SSH_HOST..." # shellcheck disable=SC2029 - if ! ssh "$SSH_HOST" "cd $REMOTE_PATH && $COMPOSE_CMD up -d"; then + if ! ssh "$SSH_HOST" "cd $REMOTE_PATH/.devcontainer && $COMPOSE_CMD up -d"; then log_error "Failed to start devcontainer on $SSH_HOST." - log_error "Run 'ssh $SSH_HOST \"cd $REMOTE_PATH && $COMPOSE_CMD logs\"' for details." + log_error "Run 'ssh $SSH_HOST \"cd $REMOTE_PATH/.devcontainer && $COMPOSE_CMD logs\"' for details." exit 1 fi sleep 2 } open_editor() { - local container_workspace uri + local container_workspace uri remote_workspace_path + remote_workspace_path=$(resolve_remote_path_absolute "$REMOTE_PATH") + # Read workspaceFolder from devcontainer.json on remote host # shellcheck disable=SC2029 container_workspace=$(ssh "$SSH_HOST" \ "grep -o '\"workspaceFolder\"[[:space:]]*:[[:space:]]*\"[^\"]*\"' \ - ${REMOTE_PATH}/.devcontainer/devcontainer.json 2>/dev/null" \ + ${remote_workspace_path}/.devcontainer/devcontainer.json 2>/dev/null" \ | sed 's/.*: *"//;s/"//' || echo "/workspace") # Default to /workspace if workspaceFolder not found @@ -428,7 +454,7 @@ open_editor() { # Build URI using Python helper if ! uri=$(python3 "$SCRIPT_DIR/devc_remote_uri.py" \ - "$REMOTE_PATH" \ + "$remote_workspace_path" \ "$SSH_HOST" \ "$container_workspace"); then log_error "Failed to build editor URI. Is devc_remote_uri.py present in $SCRIPT_DIR?" diff --git a/tests/conformance/README.md b/tests/conformance/README.md new file mode 100644 index 0000000..881dd0b --- /dev/null +++ b/tests/conformance/README.md @@ -0,0 +1,70 @@ +# Cross-Language Conformance Test Suite + +Language-agnostic test suite for the fd5 format. Any fd5 implementation +(Python, Rust, Julia, C/C++, TypeScript) must pass these tests to prove +format conformance. + +## Structure + +``` +tests/conformance/ +├── README.md # This file +├── generate_fixtures.py # Regenerates .fd5 fixture files +├── test_conformance.py # Python conformance runner +├── fixtures/ # Generated .fd5 files (not checked in) +├── expected/ # Expected-result JSON (checked in) +│ ├── minimal.json +│ ├── with-provenance.json +│ ├── multiscale.json +│ ├── tabular.json +│ ├── complex-metadata.json +│ └── sealed.json +└── invalid/ # Invalid .fd5 files + expected errors + └── expected-errors.json +``` + +## How It Works + +1. `generate_fixtures.py` uses the Python reference implementation to create + canonical `.fd5` fixture files in `fixtures/` and invalid files in `invalid/`. +2. Each fixture has a corresponding JSON file in `expected/` that defines the + expected root attributes, dataset shapes, dtypes, group hierarchy, etc. +3. A conformance runner opens each fixture with the language's own reader, + extracts values, and asserts equality against the expected JSON. + +## Running (Python) + +```bash +uv run pytest tests/conformance/ -v +``` + +Fixtures are auto-generated by a pytest session-scoped fixture before tests run. + +## Adding a New Conformance Case + +1. Add a generator function in `generate_fixtures.py`. +2. Create a corresponding `expected/.json` with the expected structure. +3. Add test functions in `test_conformance.py` (or the equivalent in your language). +4. Run the suite to verify. + +## Test Categories + +| Category | What it tests | +|-----------------------|--------------------------------------------------------| +| Structure | Correct group hierarchy, required attributes present | +| Data round-trip | Write values, read back, compare dtype/shape/values | +| Hash verification | Sealed files verify; tampered files fail | +| Provenance | DAG traversal returns expected source chain | +| Schema validation | Embedded schema validates the file's own structure | +| Negative tests | Invalid files are rejected with appropriate errors | + +## For Other Languages + +To implement the conformance suite in a new language: + +1. Generate fixtures using the Python script (or use pre-generated ones from CI). +2. Load each `.fd5` file with your HDF5 library. +3. Parse the corresponding `expected/*.json`. +4. Assert that the extracted values match the expected JSON. + +This is a black-box test -- it tests the format contract, not internal APIs. diff --git a/tests/conformance/__init__.py b/tests/conformance/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/conformance/expected/complex-metadata.json b/tests/conformance/expected/complex-metadata.json new file mode 100644 index 0000000..f4fced2 --- /dev/null +++ b/tests/conformance/expected/complex-metadata.json @@ -0,0 +1,46 @@ +{ + "description": "Deeply nested metadata groups — metadata tree tests", + "root_attrs": { + "product": "test/conformance", + "name": "complex-metadata-conformance", + "description": "Complex metadata conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "datasets": [ + { + "path": "/volume", + "shape": [4, 4], + "dtype": "float32" + } + ], + "groups": [ + "/", + "/metadata", + "/metadata/acquisition", + "/metadata/reconstruction", + "/metadata/reconstruction/parameters" + ], + "verify": true, + "metadata_tree": { + "metadata": { + "version": 2, + "acquisition": { + "modality": "PET", + "duration_sec": 300.0, + "isotope": "F-18" + }, + "reconstruction": { + "algorithm": "osem", + "parameters": { + "iterations": 4, + "subsets": 21 + } + } + } + } +} diff --git a/tests/conformance/expected/minimal.json b/tests/conformance/expected/minimal.json new file mode 100644 index 0000000..f94eb6f --- /dev/null +++ b/tests/conformance/expected/minimal.json @@ -0,0 +1,26 @@ +{ + "description": "Smallest valid fd5 file — structure tests", + "root_attrs": { + "product": "test/conformance", + "name": "minimal-conformance", + "description": "Minimal conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "datasets": [ + { + "path": "/volume", + "shape": [4, 4], + "dtype": "float32" + } + ], + "groups": [ + "/" + ], + "verify": true, + "schema_valid": true +} diff --git a/tests/conformance/expected/multiscale.json b/tests/conformance/expected/multiscale.json new file mode 100644 index 0000000..0d31c48 --- /dev/null +++ b/tests/conformance/expected/multiscale.json @@ -0,0 +1,45 @@ +{ + "description": "File with pyramid/multiscale datasets — multiscale tests", + "root_attrs": { + "product": "recon", + "name": "multiscale-conformance", + "description": "Multiscale conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "groups": [ + "/", + "/pyramid", + "/pyramid/level_1", + "/pyramid/level_2" + ], + "pyramid": { + "n_levels": 2, + "scale_factors": [2, 4], + "level_shapes": { + "level_1": [4, 4, 4], + "level_2": [2, 2, 2] + } + }, + "datasets": [ + { + "path": "/volume", + "shape": [8, 8, 8], + "dtype": "float32" + }, + { + "path": "/mip_coronal", + "dtype": "float32" + }, + { + "path": "/mip_sagittal", + "dtype": "float32" + } + ], + "verify": true, + "schema_valid": true +} diff --git a/tests/conformance/expected/sealed.json b/tests/conformance/expected/sealed.json new file mode 100644 index 0000000..258d365 --- /dev/null +++ b/tests/conformance/expected/sealed.json @@ -0,0 +1,28 @@ +{ + "description": "File with verified content hash — hash verification tests", + "root_attrs": { + "product": "test/conformance", + "name": "sealed-conformance", + "description": "Sealed conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "datasets": [ + { + "path": "/volume", + "shape": [8, 8], + "dtype": "float32" + } + ], + "verify": true, + "schema_valid": true, + "hash_verification": { + "intact_verifies": true, + "tampered_attr_fails": true, + "tampered_data_fails": true + } +} diff --git a/tests/conformance/expected/tabular.json b/tests/conformance/expected/tabular.json new file mode 100644 index 0000000..6061341 --- /dev/null +++ b/tests/conformance/expected/tabular.json @@ -0,0 +1,36 @@ +{ + "description": "Compound dataset (event table) — tabular data tests", + "root_attrs": { + "product": "test/conformance", + "name": "tabular-conformance", + "description": "Tabular conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "datasets": [ + { + "path": "/volume", + "shape": [4, 4], + "dtype": "float32" + }, + { + "path": "/events", + "shape": [5], + "columns": ["time", "energy", "detector_id"] + } + ], + "verify": true, + "tabular": { + "row_count": 5, + "column_names": ["time", "energy", "detector_id"], + "column_dtypes": { + "time": "float64", + "energy": "float32", + "detector_id": "int32" + } + } +} diff --git a/tests/conformance/expected/with-provenance.json b/tests/conformance/expected/with-provenance.json new file mode 100644 index 0000000..514319a --- /dev/null +++ b/tests/conformance/expected/with-provenance.json @@ -0,0 +1,46 @@ +{ + "description": "File with source links — provenance tests", + "root_attrs": { + "product": "test/conformance", + "name": "provenance-conformance", + "description": "Provenance conformance fixture", + "timestamp": "2026-01-01T00:00:00Z", + "_schema_version": 1 + }, + "root_attrs_prefixed": { + "id": "sha256:", + "content_hash": "sha256:" + }, + "datasets": [ + { + "path": "/volume", + "shape": [4, 4], + "dtype": "float32" + } + ], + "groups": [ + "/", + "/sources", + "/sources/upstream", + "/provenance", + "/provenance/ingest" + ], + "verify": false, + "provenance": { + "sources": [ + { + "name": "upstream", + "id": "sha256:aaa111", + "product": "raw", + "role": "input_data", + "description": "Upstream raw data" + } + ], + "has_original_files": true, + "original_files_count": 1, + "ingest": { + "tool": "conformance_generator", + "tool_version": "1.0.0" + } + } +} diff --git a/tests/conformance/fixtures/.gitignore b/tests/conformance/fixtures/.gitignore new file mode 100644 index 0000000..d8d9d7c --- /dev/null +++ b/tests/conformance/fixtures/.gitignore @@ -0,0 +1,2 @@ +*.fd5 +*.h5 diff --git a/tests/conformance/generate_fixtures.py b/tests/conformance/generate_fixtures.py new file mode 100644 index 0000000..38efab8 --- /dev/null +++ b/tests/conformance/generate_fixtures.py @@ -0,0 +1,328 @@ +"""Generate canonical fd5 fixture files for cross-language conformance testing. + +Run via pytest (session-scoped autouse fixture) or standalone: + uv run python -m tests.conformance.generate_fixtures +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import h5py +import numpy as np + +from fd5.create import create +from fd5.hash import compute_content_hash +from fd5.registry import register_schema +from fd5.schema import embed_schema + +TIMESTAMP = "2026-01-01T00:00:00Z" + + +class _ConformanceSchema: + """Minimal product schema for conformance testing.""" + + product_type: str = "test/conformance" + schema_version: str = "1.0.0" + + def json_schema(self) -> dict[str, Any]: + return { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "properties": { + "_schema_version": {"type": "integer"}, + "product": {"type": "string", "const": "test/conformance"}, + "name": {"type": "string"}, + "description": {"type": "string"}, + "timestamp": {"type": "string"}, + }, + "required": ["_schema_version", "product", "name"], + } + + def required_root_attrs(self) -> dict[str, Any]: + return {"product": "test/conformance"} + + def write(self, target: Any, data: Any) -> None: + target.create_dataset("volume", data=data) + + def id_inputs(self) -> list[str]: + return ["product", "name", "timestamp"] + + +def _register_schemas() -> None: + import fd5.registry as reg + + reg._ensure_loaded() + register_schema("test/conformance", _ConformanceSchema()) + + +def _unregister_schemas() -> None: + import fd5.registry as reg + + reg._registry.pop("test/conformance", None) + + +def _create_minimal(fixtures_dir: Path) -> Path: + """Smallest valid fd5 file.""" + data = np.zeros((4, 4), dtype=np.float32) + with create( + fixtures_dir, + product="test/conformance", + name="minimal-conformance", + description="Minimal conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product(data) + + return _find_and_rename(fixtures_dir, "minimal.fd5") + + +def _create_sealed(fixtures_dir: Path) -> Path: + """File with verified content hash for hash verification tests.""" + data = np.arange(64, dtype=np.float32).reshape(8, 8) + with create( + fixtures_dir, + product="test/conformance", + name="sealed-conformance", + description="Sealed conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product(data) + + return _find_and_rename(fixtures_dir, "sealed.fd5") + + +def _create_with_provenance(fixtures_dir: Path) -> Path: + """File with source links and provenance data.""" + data = np.zeros((4, 4), dtype=np.float32) + with create( + fixtures_dir, + product="test/conformance", + name="provenance-conformance", + description="Provenance conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product(data) + builder.write_sources( + [ + { + "name": "upstream", + "id": "sha256:aaa111", + "product": "raw", + "file": "upstream.h5", + "content_hash": "sha256:bbb222", + "role": "input_data", + "description": "Upstream raw data", + } + ] + ) + builder.write_provenance( + original_files=[ + { + "path": "/data/raw/scan.dcm", + "sha256": "sha256:ccc333", + "size_bytes": 4096, + } + ], + ingest_tool="conformance_generator", + ingest_version="1.0.0", + ingest_timestamp=TIMESTAMP, + ) + + return _find_and_rename(fixtures_dir, "with-provenance.fd5") + + +def _create_multiscale(fixtures_dir: Path) -> Path: + """File with pyramid/multiscale datasets using recon schema.""" + rng = np.random.default_rng(42) + volume = rng.standard_normal((8, 8, 8)).astype(np.float32) + + with create( + fixtures_dir, + product="recon", + name="multiscale-conformance", + description="Multiscale conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product( + { + "volume": volume, + "affine": np.eye(4, dtype=np.float64), + "dimension_order": "ZYX", + "reference_frame": "LPS", + "description": "Test volume for multiscale conformance", + "pyramid": { + "scale_factors": [2, 4], + "method": "stride", + }, + } + ) + builder.file.attrs["scanner"] = "test-scanner" + builder.file.attrs["vendor_series_id"] = "test-series-001" + + return _find_and_rename(fixtures_dir, "multiscale.fd5") + + +def _create_tabular(fixtures_dir: Path) -> Path: + """Compound dataset (event table) with typed columns.""" + volume_data = np.zeros((4, 4), dtype=np.float32) + + dt = np.dtype( + [ + ("time", np.float64), + ("energy", np.float32), + ("detector_id", np.int32), + ] + ) + events = np.array( + [ + (0.0, 511.0, 1), + (0.1, 510.5, 2), + (0.2, 511.2, 1), + (0.3, 509.8, 3), + (0.4, 511.0, 2), + ], + dtype=dt, + ) + + with create( + fixtures_dir, + product="test/conformance", + name="tabular-conformance", + description="Tabular conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product(volume_data) + builder.file.create_dataset("events", data=events) + + return _find_and_rename(fixtures_dir, "tabular.fd5") + + +def _create_complex_metadata(fixtures_dir: Path) -> Path: + """Deeply nested metadata groups.""" + volume_data = np.zeros((4, 4), dtype=np.float32) + + with create( + fixtures_dir, + product="test/conformance", + name="complex-metadata-conformance", + description="Complex metadata conformance fixture", + timestamp=TIMESTAMP, + ) as builder: + builder.write_product(volume_data) + builder.write_metadata( + { + "version": 2, + "acquisition": { + "modality": "PET", + "duration_sec": 300.0, + "isotope": "F-18", + }, + "reconstruction": { + "algorithm": "osem", + "parameters": { + "iterations": 4, + "subsets": 21, + }, + }, + } + ) + + return _find_and_rename(fixtures_dir, "complex-metadata.fd5") + + +def _create_invalid_missing_id(invalid_dir: Path) -> None: + """File missing required root 'id' attribute.""" + path = invalid_dir / "missing-id.fd5" + with h5py.File(path, "w") as f: + f.attrs["product"] = "test/conformance" + f.attrs["name"] = "missing-id" + f.attrs["description"] = "Missing id attribute" + f.attrs["timestamp"] = TIMESTAMP + f.attrs["_schema_version"] = np.int64(1) + f.create_dataset("volume", data=np.zeros((4, 4), dtype=np.float32)) + schema_dict = _ConformanceSchema().json_schema() + embed_schema(f, schema_dict) + f.attrs["content_hash"] = compute_content_hash(f) + + +def _create_invalid_bad_hash(invalid_dir: Path) -> None: + """File whose content_hash doesn't match actual content.""" + path = invalid_dir / "bad-hash.fd5" + with h5py.File(path, "w") as f: + f.attrs["product"] = "test/conformance" + f.attrs["name"] = "bad-hash" + f.attrs["description"] = "Bad hash fixture" + f.attrs["timestamp"] = TIMESTAMP + f.attrs["_schema_version"] = np.int64(1) + f.attrs["id"] = "sha256:fake_id_not_real" + f.create_dataset("volume", data=np.zeros((4, 4), dtype=np.float32)) + schema_dict = _ConformanceSchema().json_schema() + embed_schema(f, schema_dict) + f.attrs["content_hash"] = ( + "sha256:0000000000000000000000000000000000000000000000000000000000000000" + ) + + +def _create_invalid_no_schema(invalid_dir: Path) -> None: + """File missing the _schema attribute.""" + path = invalid_dir / "no-schema.fd5" + with h5py.File(path, "w") as f: + f.attrs["product"] = "test/conformance" + f.attrs["name"] = "no-schema" + f.attrs["description"] = "No schema fixture" + f.attrs["timestamp"] = TIMESTAMP + f.attrs["_schema_version"] = np.int64(1) + f.attrs["id"] = "sha256:fake_id_not_real" + f.create_dataset("volume", data=np.zeros((4, 4), dtype=np.float32)) + f.attrs["content_hash"] = compute_content_hash(f) + + +def _find_and_rename(directory: Path, target_name: str) -> Path: + """Find the single .h5 file created by fd5.create() and rename it.""" + h5_files = list(directory.glob("*.h5")) + unnamed = [f for f in h5_files if not f.stem.endswith(".fd5")] + if not unnamed: + unnamed = h5_files + newest = max(unnamed, key=lambda f: f.stat().st_mtime) + target = directory / target_name + if target.exists(): + target.unlink() + newest.rename(target) + return target + + +def generate_all(fixtures_dir: Path, invalid_dir: Path) -> None: + """Generate all conformance fixture files.""" + _register_schemas() + + fixtures_dir.mkdir(parents=True, exist_ok=True) + invalid_dir.mkdir(parents=True, exist_ok=True) + + for existing in fixtures_dir.glob("*.fd5"): + existing.unlink() + for existing in fixtures_dir.glob("*.h5"): + existing.unlink() + for existing in invalid_dir.glob("*.fd5"): + existing.unlink() + + try: + _create_minimal(fixtures_dir) + _create_sealed(fixtures_dir) + _create_with_provenance(fixtures_dir) + _create_multiscale(fixtures_dir) + _create_tabular(fixtures_dir) + _create_complex_metadata(fixtures_dir) + + _create_invalid_missing_id(invalid_dir) + _create_invalid_bad_hash(invalid_dir) + _create_invalid_no_schema(invalid_dir) + finally: + _unregister_schemas() + + +if __name__ == "__main__": + conformance_dir = Path(__file__).parent + generate_all(conformance_dir / "fixtures", conformance_dir / "invalid") + print("All conformance fixtures generated.") diff --git a/tests/conformance/invalid/.gitignore b/tests/conformance/invalid/.gitignore new file mode 100644 index 0000000..d8d9d7c --- /dev/null +++ b/tests/conformance/invalid/.gitignore @@ -0,0 +1,2 @@ +*.fd5 +*.h5 diff --git a/tests/conformance/invalid/expected-errors.json b/tests/conformance/invalid/expected-errors.json new file mode 100644 index 0000000..efd2835 --- /dev/null +++ b/tests/conformance/invalid/expected-errors.json @@ -0,0 +1,16 @@ +{ + "missing-id.fd5": { + "description": "Missing required root 'id' attribute", + "error_type": "KeyError", + "error_pattern": "id" + }, + "bad-hash.fd5": { + "description": "Content hash does not match actual content", + "verify_returns": false + }, + "no-schema.fd5": { + "description": "Missing _schema attribute", + "error_type": "KeyError", + "error_pattern": "_schema" + } +} diff --git a/tests/conformance/test_conformance.py b/tests/conformance/test_conformance.py new file mode 100644 index 0000000..8ce1bd9 --- /dev/null +++ b/tests/conformance/test_conformance.py @@ -0,0 +1,406 @@ +"""Cross-language conformance tests for the fd5 format. + +Validates that the Python reference implementation produces files matching +the canonical expected-result JSON files. Any fd5 implementation must pass +equivalent tests to prove format conformance. + +See tests/conformance/README.md for details. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import h5py +import numpy as np +import pytest + +from fd5.hash import verify +from fd5.schema import validate + +CONFORMANCE_DIR = Path(__file__).parent +FIXTURES_DIR = CONFORMANCE_DIR / "fixtures" +EXPECTED_DIR = CONFORMANCE_DIR / "expected" +INVALID_DIR = CONFORMANCE_DIR / "invalid" + + +def _load_expected(name: str) -> dict: + path = EXPECTED_DIR / f"{name}.json" + return json.loads(path.read_text()) + + +def _fixture_path(name: str) -> Path: + return FIXTURES_DIR / f"{name}.fd5" + + +@pytest.fixture(scope="session", autouse=True) +def _generate_fixtures(): + """Generate all fixture files before any conformance test runs.""" + from tests.conformance.generate_fixtures import generate_all + + generate_all(FIXTURES_DIR, INVALID_DIR) + + from tests.conformance.generate_fixtures import _ConformanceSchema + + from fd5.registry import register_schema + + register_schema("test/conformance", _ConformanceSchema()) + + +# --------------------------------------------------------------------------- +# Structure tests — minimal fixture +# --------------------------------------------------------------------------- + + +class TestStructure: + """Correct group hierarchy and required attributes present.""" + + def test_root_attrs_match(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + with h5py.File(path, "r") as f: + for key, value in expected["root_attrs"].items(): + actual = f.attrs[key] + if isinstance(actual, bytes): + actual = actual.decode("utf-8") + if isinstance(actual, np.integer): + actual = int(actual) + assert actual == value, f"Attr {key!r}: {actual!r} != {value!r}" + + def test_root_attrs_prefixed(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + with h5py.File(path, "r") as f: + for key, prefix in expected["root_attrs_prefixed"].items(): + actual = f.attrs[key] + if isinstance(actual, bytes): + actual = actual.decode("utf-8") + assert actual.startswith(prefix), ( + f"Attr {key!r} should start with {prefix!r}, got {actual!r}" + ) + + def test_datasets_present(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + with h5py.File(path, "r") as f: + for ds_spec in expected["datasets"]: + ds = f[ds_spec["path"]] + assert isinstance(ds, h5py.Dataset) + assert list(ds.shape) == ds_spec["shape"] + assert ds.dtype == np.dtype(ds_spec["dtype"]) + + def test_groups_present(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + with h5py.File(path, "r") as f: + for grp_path in expected["groups"]: + assert grp_path in f or grp_path == "/" + + def test_verify_true(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + assert verify(path) is expected["verify"] + + def test_schema_valid(self): + expected = _load_expected("minimal") + path = _fixture_path("minimal") + if expected.get("schema_valid"): + errors = validate(path) + assert errors == [], [e.message for e in errors] + + +# --------------------------------------------------------------------------- +# Hash verification tests — sealed fixture +# --------------------------------------------------------------------------- + + +class TestHashVerification: + """Sealed files verify correctly, tampered files fail.""" + + def test_intact_verifies(self): + path = _fixture_path("sealed") + assert verify(path) is True + + def test_tampered_attr_fails(self, tmp_path): + import shutil + + src = _fixture_path("sealed") + tampered = tmp_path / "tampered_attr.fd5" + shutil.copy2(src, tampered) + + with h5py.File(tampered, "a") as f: + f.attrs["name"] = "tampered-value" + + assert verify(tampered) is False + + def test_tampered_data_fails(self, tmp_path): + import shutil + + src = _fixture_path("sealed") + tampered = tmp_path / "tampered_data.fd5" + shutil.copy2(src, tampered) + + with h5py.File(tampered, "a") as f: + ds = f["volume"] + ds[0, 0] = 999.0 + + assert verify(tampered) is False + + def test_content_hash_format(self): + path = _fixture_path("sealed") + with h5py.File(path, "r") as f: + ch = f.attrs["content_hash"] + if isinstance(ch, bytes): + ch = ch.decode("utf-8") + assert ch.startswith("sha256:") + assert len(ch) == len("sha256:") + 64 + + +# --------------------------------------------------------------------------- +# Provenance tests — with-provenance fixture +# --------------------------------------------------------------------------- + + +class TestProvenance: + """DAG traversal returns expected source chain.""" + + def test_sources_group_exists(self): + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + assert "sources" in f + + def test_source_attrs(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + for src_spec in expected["provenance"]["sources"]: + name = src_spec["name"] + grp = f[f"sources/{name}"] + assert grp.attrs["id"] == src_spec["id"] + assert grp.attrs["product"] == src_spec["product"] + assert grp.attrs["role"] == src_spec["role"] + assert grp.attrs["description"] == src_spec["description"] + + def test_source_has_external_link(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + for src_spec in expected["provenance"]["sources"]: + name = src_spec["name"] + link = f[f"sources/{name}"].get("link", getlink=True) + assert isinstance(link, h5py.ExternalLink) + + def test_original_files_exist(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + assert "provenance" in f + if expected["provenance"]["has_original_files"]: + assert "original_files" in f["provenance"] + ds = f["provenance/original_files"] + assert len(ds) == expected["provenance"]["original_files_count"] + + def test_ingest_attrs(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + ingest = f["provenance/ingest"] + ingest_spec = expected["provenance"]["ingest"] + assert ingest.attrs["tool"] == ingest_spec["tool"] + assert ingest.attrs["tool_version"] == ingest_spec["tool_version"] + + def test_groups_present(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + with h5py.File(path, "r") as f: + for grp_path in expected["groups"]: + if grp_path == "/": + continue + assert grp_path in f, f"Missing group {grp_path!r}" + + def test_verify_matches_expected(self): + expected = _load_expected("with-provenance") + path = _fixture_path("with-provenance") + assert verify(path) is expected["verify"] + + +# --------------------------------------------------------------------------- +# Multiscale tests — multiscale fixture +# --------------------------------------------------------------------------- + + +class TestMultiscale: + """Pyramid levels and shapes match expected.""" + + def test_pyramid_group_exists(self): + path = _fixture_path("multiscale") + with h5py.File(path, "r") as f: + assert "pyramid" in f + + def test_pyramid_attrs(self): + expected = _load_expected("multiscale") + path = _fixture_path("multiscale") + with h5py.File(path, "r") as f: + pyr = f["pyramid"] + assert int(pyr.attrs["n_levels"]) == expected["pyramid"]["n_levels"] + actual_factors = list(pyr.attrs["scale_factors"]) + assert actual_factors == expected["pyramid"]["scale_factors"] + + def test_pyramid_level_shapes(self): + expected = _load_expected("multiscale") + path = _fixture_path("multiscale") + with h5py.File(path, "r") as f: + for level_name, expected_shape in expected["pyramid"][ + "level_shapes" + ].items(): + ds = f[f"pyramid/{level_name}/volume"] + assert list(ds.shape) == expected_shape + + def test_groups_present(self): + expected = _load_expected("multiscale") + path = _fixture_path("multiscale") + with h5py.File(path, "r") as f: + for grp_path in expected["groups"]: + if grp_path == "/": + continue + assert grp_path in f, f"Missing group {grp_path!r}" + + def test_mip_datasets_present(self): + expected = _load_expected("multiscale") + path = _fixture_path("multiscale") + with h5py.File(path, "r") as f: + for ds_spec in expected["datasets"]: + ds = f[ds_spec["path"]] + assert isinstance(ds, h5py.Dataset) + assert ds.dtype == np.dtype(ds_spec["dtype"]) + + def test_verify_true(self): + path = _fixture_path("multiscale") + assert verify(path) is True + + +# --------------------------------------------------------------------------- +# Tabular tests — tabular fixture +# --------------------------------------------------------------------------- + + +class TestTabular: + """Compound dataset with expected columns, dtypes, and row count.""" + + def test_events_dataset_exists(self): + path = _fixture_path("tabular") + with h5py.File(path, "r") as f: + assert "events" in f + + def test_row_count(self): + expected = _load_expected("tabular") + path = _fixture_path("tabular") + with h5py.File(path, "r") as f: + ds = f["events"] + assert len(ds) == expected["tabular"]["row_count"] + + def test_column_names(self): + expected = _load_expected("tabular") + path = _fixture_path("tabular") + with h5py.File(path, "r") as f: + ds = f["events"] + actual_names = list(ds.dtype.names) + assert actual_names == expected["tabular"]["column_names"] + + def test_column_dtypes(self): + expected = _load_expected("tabular") + path = _fixture_path("tabular") + with h5py.File(path, "r") as f: + ds = f["events"] + for col, expected_dtype in expected["tabular"]["column_dtypes"].items(): + actual = ds.dtype[col] + assert actual == np.dtype(expected_dtype), ( + f"Column {col!r}: {actual} != {expected_dtype}" + ) + + def test_verify_true(self): + path = _fixture_path("tabular") + assert verify(path) is True + + +# --------------------------------------------------------------------------- +# Complex metadata tests — complex-metadata fixture +# --------------------------------------------------------------------------- + + +class TestComplexMetadata: + """Deeply nested metadata groups match expected tree.""" + + def test_groups_present(self): + expected = _load_expected("complex-metadata") + path = _fixture_path("complex-metadata") + with h5py.File(path, "r") as f: + for grp_path in expected["groups"]: + if grp_path == "/": + continue + assert grp_path in f, f"Missing group {grp_path!r}" + + def test_metadata_tree(self): + expected = _load_expected("complex-metadata") + path = _fixture_path("complex-metadata") + with h5py.File(path, "r") as f: + from fd5.h5io import h5_to_dict + + actual = h5_to_dict(f["metadata"]) + expected_tree = expected["metadata_tree"]["metadata"] + assert actual == expected_tree + + def test_verify_true(self): + path = _fixture_path("complex-metadata") + assert verify(path) is True + + +# --------------------------------------------------------------------------- +# Schema validation tests — across all valid fixtures +# --------------------------------------------------------------------------- + + +class TestSchemaValidation: + """Embedded schema validates the file's own structure.""" + + @pytest.mark.parametrize( + "fixture_name", + ["minimal", "sealed", "tabular", "complex-metadata"], + ) + def test_schema_validates(self, fixture_name): + expected = _load_expected(fixture_name) + if not expected.get("schema_valid", True): + pytest.skip("Fixture not expected to pass schema validation") + path = _fixture_path(fixture_name) + errors = validate(path) + assert errors == [], [e.message for e in errors] + + +# --------------------------------------------------------------------------- +# Negative tests — invalid fixtures +# --------------------------------------------------------------------------- + + +class TestInvalid: + """Invalid files are rejected with appropriate errors.""" + + def test_missing_id_raises(self): + path = INVALID_DIR / "missing-id.fd5" + with h5py.File(path, "r") as f: + assert "id" not in f.attrs + + def test_bad_hash_fails_verify(self): + path = INVALID_DIR / "bad-hash.fd5" + assert verify(path) is False + + def test_no_schema_raises_on_validate(self): + path = INVALID_DIR / "no-schema.fd5" + with pytest.raises(KeyError, match="_schema"): + validate(path) + + def test_expected_errors_json_matches(self): + """Ensure expected-errors.json covers all invalid fixtures.""" + errors_json = json.loads((INVALID_DIR / "expected-errors.json").read_text()) + for filename in ["missing-id.fd5", "bad-hash.fd5", "no-schema.fd5"]: + assert filename in errors_json, f"Missing entry for {filename}" diff --git a/tests/test_devc_remote_preflight.sh b/tests/test_devc_remote_preflight.sh index 7c97c0d..0fa493c 100755 --- a/tests/test_devc_remote_preflight.sh +++ b/tests/test_devc_remote_preflight.sh @@ -180,6 +180,43 @@ run_container_check() { return $rc } +# Helper: build a script that tests resolve_remote_path_absolute +build_resolve_path_script() { + local input_path="$1" home_path="$2" + local tmpscript tmpsrc + tmpscript=$(mktemp "${TMPDIR:-/tmp}/devc_test.XXXXXX") + tmpsrc=$(mktemp "${TMPDIR:-/tmp}/devc_src.XXXXXX") + + sed 's/^main "\$@"$/# main disabled/' "$DEVC_SCRIPT" > "$tmpsrc" + TMPFILES+=("$tmpsrc") + + { + echo '#!/usr/bin/env bash' + echo 'set -euo pipefail' + echo "source \"$tmpsrc\"" + echo 'ssh() {' + echo " printf '%s' \"$1\" >/dev/null" + echo " printf '%s' \"$2\" >/dev/null" + echo " echo \"$home_path\"" + echo '}' + echo 'SSH_HOST="testhost"' + echo "resolve_remote_path_absolute \"$input_path\"" + } > "$tmpscript" + + echo "$tmpscript" +} + +run_resolve_path() { + local input_path="$1" home_path="$2" + local tmpscript + tmpscript=$(build_resolve_path_script "$input_path" "$home_path") + TMPFILES+=("$tmpscript") + local output rc=0 + output=$(bash "$tmpscript" 2>&1) || rc=$? + echo "$output" + return $rc +} + # ───────────────────────────────────────────────────────────────────────────── # Mock data sets # ───────────────────────────────────────────────────────────────────────────── @@ -449,6 +486,38 @@ test_ssh_agent_fwd_fail() { assert_contains "ssh agent not available" "$output" "not available" } +# ───────────────────────────────────────────────────────────────────────────── +# TEST: Tilde paths are resolved for editor URI construction +# ───────────────────────────────────────────────────────────────────────────── +test_resolve_remote_path_tilde_prefix() { + local output + # shellcheck disable=SC2088 + output=$(run_resolve_path "~/fd5" "/home/user") || true + + assert_contains "tilde prefix resolved" "$output" "/home/user/fd5" +} + +test_resolve_remote_path_tilde_only() { + local output + output=$(run_resolve_path "~" "/home/user") || true + + assert_contains "tilde only resolved" "$output" "/home/user" +} + +test_resolve_remote_path_absolute_passthrough() { + local output + output=$(run_resolve_path "/opt/fd5" "/home/user") || true + + assert_contains "absolute path unchanged" "$output" "/opt/fd5" +} + +test_resolve_remote_path_relative_prefix() { + local output + output=$(run_resolve_path "fd5" "/home/user") || true + + assert_contains "relative path resolved under home" "$output" "/home/user/fd5" +} + # ───────────────────────────────────────────────────────────────────────────── # RUN ALL # ───────────────────────────────────────────────────────────────────────────── @@ -470,6 +539,10 @@ test_container_check_yes_reuses test_container_check_skip_when_not_running test_ssh_agent_fwd_ok test_ssh_agent_fwd_fail +test_resolve_remote_path_tilde_prefix +test_resolve_remote_path_tilde_only +test_resolve_remote_path_absolute_passthrough +test_resolve_remote_path_relative_prefix echo "" echo "Results: $PASS passed, $FAIL failed" diff --git a/tests/test_ingest_csv.py b/tests/test_ingest_csv.py index 878c646..63b2208 100644 --- a/tests/test_ingest_csv.py +++ b/tests/test_ingest_csv.py @@ -466,9 +466,7 @@ def test_string_source_path( class TestIdempotency: """Calling ingest twice with identical inputs produces two valid, independently sealed files.""" - def test_deterministic( - self, loader: CsvLoader, spectrum_csv: Path, tmp_path: Path - ): + def test_deterministic(self, loader: CsvLoader, spectrum_csv: Path, tmp_path: Path): kwargs = dict( product="spectrum", name="idem-spectrum", diff --git a/tests/test_ingest_nifti.py b/tests/test_ingest_nifti.py index dd3824f..43ae443 100644 --- a/tests/test_ingest_nifti.py +++ b/tests/test_ingest_nifti.py @@ -260,7 +260,7 @@ def test_provenance_original_files(self, nifti_3d: Path, tmp_path: Path): assert "original_files" in f["provenance"] rec = f["provenance/original_files"][0] assert str(nifti_3d) in rec["path"].decode() - sha = hashlib.sha256(nifti_3d.read_bytes()).hexdigest() + sha = f"sha256:{hashlib.sha256(nifti_3d.read_bytes()).hexdigest()}" assert rec["sha256"].decode() == sha def test_provenance_ingest_group(self, nifti_3d: Path, tmp_path: Path): diff --git a/tests/test_ingest_raw.py b/tests/test_ingest_raw.py index 58de43d..bb98d73 100644 --- a/tests/test_ingest_raw.py +++ b/tests/test_ingest_raw.py @@ -242,7 +242,7 @@ def test_records_provenance_sha256(self, tmp_path: Path): reference_frame="LPS", ) - expected_sha = hashlib.sha256(bin_path.read_bytes()).hexdigest() + expected_sha = f"sha256:{hashlib.sha256(bin_path.read_bytes()).hexdigest()}" with h5py.File(result, "r") as f: assert "provenance" in f assert "original_files" in f["provenance"] diff --git a/uv.lock b/uv.lock index cc09f9e..d8d82ba 100644 --- a/uv.lock +++ b/uv.lock @@ -575,10 +575,13 @@ all = [ { name = "ipykernel" }, { name = "jupyter" }, { name = "matplotlib" }, + { name = "nibabel" }, { name = "numpy" }, { name = "pandas" }, { name = "pip-licenses" }, { name = "pre-commit" }, + { name = "pyarrow" }, + { name = "pydicom" }, { name = "pytest" }, { name = "pytest-cov" }, { name = "scipy" }, @@ -592,6 +595,15 @@ dev = [ { name = "pytest" }, { name = "pytest-cov" }, ] +dicom = [ + { name = "pydicom" }, +] +nifti = [ + { name = "nibabel" }, +] +parquet = [ + { name = "pyarrow" }, +] science = [ { name = "matplotlib" }, { name = "numpy" }, @@ -609,24 +621,27 @@ dev = [ requires-dist = [ { name = "bandit", marker = "extra == 'dev'", specifier = ">=1.7" }, { name = "click", specifier = ">=8.0" }, - { name = "fd5", extras = ["dev", "science"], marker = "extra == 'all'" }, + { name = "fd5", extras = ["dev", "science", "dicom", "nifti", "parquet"], marker = "extra == 'all'" }, { name = "h5py", specifier = ">=3.10" }, { name = "ipykernel", marker = "extra == 'dev'", specifier = ">=6.0" }, { name = "jsonschema", specifier = ">=4.20" }, { name = "jupyter", marker = "extra == 'dev'", specifier = ">=1.0" }, { name = "matplotlib", marker = "extra == 'science'", specifier = ">=3.9" }, + { name = "nibabel", marker = "extra == 'nifti'", specifier = ">=5.0" }, { name = "numpy", specifier = ">=2.0" }, { name = "numpy", marker = "extra == 'science'", specifier = ">=2.0" }, { name = "pandas", marker = "extra == 'science'", specifier = ">=2.2" }, { name = "pip-licenses", marker = "extra == 'dev'", specifier = ">=5.0" }, { name = "pre-commit", marker = "extra == 'dev'", specifier = ">=4.0" }, + { name = "pyarrow", marker = "extra == 'parquet'", specifier = ">=14.0" }, + { name = "pydicom", marker = "extra == 'dicom'", specifier = ">=2.4" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" }, { name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.0" }, { name = "pyyaml", specifier = ">=6.0" }, { name = "scipy", marker = "extra == 'science'", specifier = ">=1.14" }, { name = "tomli-w", specifier = ">=1.0" }, ] -provides-extras = ["dev", "science", "all"] +provides-extras = ["dev", "science", "dicom", "nifti", "parquet", "all"] [package.metadata.requires-dev] dev = [ @@ -1476,6 +1491,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" }, ] +[[package]] +name = "nibabel" +version = "5.3.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, + { name = "packaging" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b1/8b/98e35cd0f2a97c8c261cedf8cc155766a3c540cb248d449582bb9e99c719/nibabel-5.3.3.tar.gz", hash = "sha256:8d2006b70d727fd0a798a88ae5fd64339741f436fcfc83d6ea3256cdbc51c5b7", size = 4506925, upload-time = "2025-12-05T19:16:54.416Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b8/f5/7f6aa3bbff013c0bf993129cbb2b1505790091f812accbe85cf001514737/nibabel-5.3.3-py3-none-any.whl", hash = "sha256:e8b17423ee8464da3b69e6a15799eb19f2350a7d38377026d527b6b84938adac", size = 3293989, upload-time = "2025-12-05T19:16:51.941Z" }, +] + [[package]] name = "nodeenv" version = "1.10.0" @@ -1868,6 +1897,49 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8e/37/efad0257dc6e593a18957422533ff0f87ede7c9c6ea010a2177d738fb82f/pure_eval-0.2.3-py3-none-any.whl", hash = "sha256:1db8e35b67b3d218d818ae653e27f06c3aa420901fa7b081ca98cbedc874e0d0", size = 11842, upload-time = "2024-07-21T12:58:20.04Z" }, ] +[[package]] +name = "pyarrow" +version = "23.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/88/22/134986a4cc224d593c1afde5494d18ff629393d74cc2eddb176669f234a4/pyarrow-23.0.1.tar.gz", hash = "sha256:b8c5873e33440b2bc2f4a79d2b47017a89c5a24116c055625e6f2ee50523f019", size = 1167336, upload-time = "2026-02-16T10:14:12.39Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9a/4b/4166bb5abbfe6f750fc60ad337c43ecf61340fa52ab386da6e8dbf9e63c4/pyarrow-23.0.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:f4b0dbfa124c0bb161f8b5ebb40f1a680b70279aa0c9901d44a2b5a20806039f", size = 34214575, upload-time = "2026-02-16T10:09:56.225Z" }, + { url = "https://files.pythonhosted.org/packages/e1/da/3f941e3734ac8088ea588b53e860baeddac8323ea40ce22e3d0baa865cc9/pyarrow-23.0.1-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:7707d2b6673f7de054e2e83d59f9e805939038eebe1763fe811ee8fa5c0cd1a7", size = 35832540, upload-time = "2026-02-16T10:10:03.428Z" }, + { url = "https://files.pythonhosted.org/packages/88/7c/3d841c366620e906d54430817531b877ba646310296df42ef697308c2705/pyarrow-23.0.1-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:86ff03fb9f1a320266e0de855dee4b17da6794c595d207f89bba40d16b5c78b9", size = 44470940, upload-time = "2026-02-16T10:10:10.704Z" }, + { url = "https://files.pythonhosted.org/packages/2c/a5/da83046273d990f256cb79796a190bbf7ec999269705ddc609403f8c6b06/pyarrow-23.0.1-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:813d99f31275919c383aab17f0f455a04f5a429c261cc411b1e9a8f5e4aaaa05", size = 47586063, upload-time = "2026-02-16T10:10:17.95Z" }, + { url = "https://files.pythonhosted.org/packages/5b/3c/b7d2ebcff47a514f47f9da1e74b7949138c58cfeb108cdd4ee62f43f0cf3/pyarrow-23.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:bf5842f960cddd2ef757d486041d57c96483efc295a8c4a0e20e704cbbf39c67", size = 48173045, upload-time = "2026-02-16T10:10:25.363Z" }, + { url = "https://files.pythonhosted.org/packages/43/b2/b40961262213beaba6acfc88698eb773dfce32ecdf34d19291db94c2bd73/pyarrow-23.0.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:564baf97c858ecc03ec01a41062e8f4698abc3e6e2acd79c01c2e97880a19730", size = 50621741, upload-time = "2026-02-16T10:10:33.477Z" }, + { url = "https://files.pythonhosted.org/packages/f6/70/1fdda42d65b28b078e93d75d371b2185a61da89dda4def8ba6ba41ebdeb4/pyarrow-23.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:07deae7783782ac7250989a7b2ecde9b3c343a643f82e8a4df03d93b633006f0", size = 27620678, upload-time = "2026-02-16T10:10:39.31Z" }, + { url = "https://files.pythonhosted.org/packages/47/10/2cbe4c6f0fb83d2de37249567373d64327a5e4d8db72f486db42875b08f6/pyarrow-23.0.1-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:6b8fda694640b00e8af3c824f99f789e836720aa8c9379fb435d4c4953a756b8", size = 34210066, upload-time = "2026-02-16T10:10:45.487Z" }, + { url = "https://files.pythonhosted.org/packages/cb/4f/679fa7e84dadbaca7a65f7cdba8d6c83febbd93ca12fa4adf40ba3b6362b/pyarrow-23.0.1-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:8ff51b1addc469b9444b7c6f3548e19dc931b172ab234e995a60aea9f6e6025f", size = 35825526, upload-time = "2026-02-16T10:10:52.266Z" }, + { url = "https://files.pythonhosted.org/packages/f9/63/d2747d930882c9d661e9398eefc54f15696547b8983aaaf11d4a2e8b5426/pyarrow-23.0.1-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:71c5be5cbf1e1cb6169d2a0980850bccb558ddc9b747b6206435313c47c37677", size = 44473279, upload-time = "2026-02-16T10:11:01.557Z" }, + { url = "https://files.pythonhosted.org/packages/b3/93/10a48b5e238de6d562a411af6467e71e7aedbc9b87f8d3a35f1560ae30fb/pyarrow-23.0.1-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:9b6f4f17b43bc39d56fec96e53fe89d94bac3eb134137964371b45352d40d0c2", size = 47585798, upload-time = "2026-02-16T10:11:09.401Z" }, + { url = "https://files.pythonhosted.org/packages/5c/20/476943001c54ef078dbf9542280e22741219a184a0632862bca4feccd666/pyarrow-23.0.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:9fc13fc6c403d1337acab46a2c4346ca6c9dec5780c3c697cf8abfd5e19b6b37", size = 48179446, upload-time = "2026-02-16T10:11:17.781Z" }, + { url = "https://files.pythonhosted.org/packages/4b/b6/5dd0c47b335fcd8edba9bfab78ad961bd0fd55ebe53468cc393f45e0be60/pyarrow-23.0.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5c16ed4f53247fa3ffb12a14d236de4213a4415d127fe9cebed33d51671113e2", size = 50623972, upload-time = "2026-02-16T10:11:26.185Z" }, + { url = "https://files.pythonhosted.org/packages/d5/09/a532297c9591a727d67760e2e756b83905dd89adb365a7f6e9c72578bcc1/pyarrow-23.0.1-cp313-cp313-win_amd64.whl", hash = "sha256:cecfb12ef629cf6be0b1887f9f86463b0dd3dc3195ae6224e74006be4736035a", size = 27540749, upload-time = "2026-02-16T10:12:23.297Z" }, + { url = "https://files.pythonhosted.org/packages/a5/8e/38749c4b1303e6ae76b3c80618f84861ae0c55dd3c2273842ea6f8258233/pyarrow-23.0.1-cp313-cp313t-macosx_12_0_arm64.whl", hash = "sha256:29f7f7419a0e30264ea261fdc0e5fe63ce5a6095003db2945d7cd78df391a7e1", size = 34471544, upload-time = "2026-02-16T10:11:32.535Z" }, + { url = "https://files.pythonhosted.org/packages/a3/73/f237b2bc8c669212f842bcfd842b04fc8d936bfc9d471630569132dc920d/pyarrow-23.0.1-cp313-cp313t-macosx_12_0_x86_64.whl", hash = "sha256:33d648dc25b51fd8055c19e4261e813dfc4d2427f068bcecc8b53d01b81b0500", size = 35949911, upload-time = "2026-02-16T10:11:39.813Z" }, + { url = "https://files.pythonhosted.org/packages/0c/86/b912195eee0903b5611bf596833def7d146ab2d301afeb4b722c57ffc966/pyarrow-23.0.1-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:cd395abf8f91c673dd3589cadc8cc1ee4e8674fa61b2e923c8dd215d9c7d1f41", size = 44520337, upload-time = "2026-02-16T10:11:47.764Z" }, + { url = "https://files.pythonhosted.org/packages/69/c2/f2a717fb824f62d0be952ea724b4f6f9372a17eed6f704b5c9526f12f2f1/pyarrow-23.0.1-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:00be9576d970c31defb5c32eb72ef585bf600ef6d0a82d5eccaae96639cf9d07", size = 47548944, upload-time = "2026-02-16T10:11:56.607Z" }, + { url = "https://files.pythonhosted.org/packages/84/a7/90007d476b9f0dc308e3bc57b832d004f848fd6c0da601375d20d92d1519/pyarrow-23.0.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c2139549494445609f35a5cda4eb94e2c9e4d704ce60a095b342f82460c73a83", size = 48236269, upload-time = "2026-02-16T10:12:04.47Z" }, + { url = "https://files.pythonhosted.org/packages/b0/3f/b16fab3e77709856eb6ac328ce35f57a6d4a18462c7ca5186ef31b45e0e0/pyarrow-23.0.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:7044b442f184d84e2351e5084600f0d7343d6117aabcbc1ac78eb1ae11eb4125", size = 50604794, upload-time = "2026-02-16T10:12:11.797Z" }, + { url = "https://files.pythonhosted.org/packages/e9/a1/22df0620a9fac31d68397a75465c344e83c3dfe521f7612aea33e27ab6c0/pyarrow-23.0.1-cp313-cp313t-win_amd64.whl", hash = "sha256:a35581e856a2fafa12f3f54fce4331862b1cfb0bef5758347a858a4aa9d6bae8", size = 27660642, upload-time = "2026-02-16T10:12:17.746Z" }, + { url = "https://files.pythonhosted.org/packages/8d/1b/6da9a89583ce7b23ac611f183ae4843cd3a6cf54f079549b0e8c14031e73/pyarrow-23.0.1-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:5df1161da23636a70838099d4aaa65142777185cc0cdba4037a18cee7d8db9ca", size = 34238755, upload-time = "2026-02-16T10:12:32.819Z" }, + { url = "https://files.pythonhosted.org/packages/ae/b5/d58a241fbe324dbaeb8df07be6af8752c846192d78d2272e551098f74e88/pyarrow-23.0.1-cp314-cp314-macosx_12_0_x86_64.whl", hash = "sha256:fa8e51cb04b9f8c9c5ace6bab63af9a1f88d35c0d6cbf53e8c17c098552285e1", size = 35847826, upload-time = "2026-02-16T10:12:38.949Z" }, + { url = "https://files.pythonhosted.org/packages/54/a5/8cbc83f04aba433ca7b331b38f39e000efd9f0c7ce47128670e737542996/pyarrow-23.0.1-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:0b95a3994f015be13c63148fef8832e8a23938128c185ee951c98908a696e0eb", size = 44536859, upload-time = "2026-02-16T10:12:45.467Z" }, + { url = "https://files.pythonhosted.org/packages/36/2e/c0f017c405fcdc252dbccafbe05e36b0d0eb1ea9a958f081e01c6972927f/pyarrow-23.0.1-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:4982d71350b1a6e5cfe1af742c53dfb759b11ce14141870d05d9e540d13bc5d1", size = 47614443, upload-time = "2026-02-16T10:12:55.525Z" }, + { url = "https://files.pythonhosted.org/packages/af/6b/2314a78057912f5627afa13ba43809d9d653e6630859618b0fd81a4e0759/pyarrow-23.0.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c250248f1fe266db627921c89b47b7c06fee0489ad95b04d50353537d74d6886", size = 48232991, upload-time = "2026-02-16T10:13:04.729Z" }, + { url = "https://files.pythonhosted.org/packages/40/f2/1bcb1d3be3460832ef3370d621142216e15a2c7c62602a4ea19ec240dd64/pyarrow-23.0.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5f4763b83c11c16e5f4c15601ba6dfa849e20723b46aa2617cb4bffe8768479f", size = 50645077, upload-time = "2026-02-16T10:13:14.147Z" }, + { url = "https://files.pythonhosted.org/packages/eb/3f/b1da7b61cd66566a4d4c8383d376c606d1c34a906c3f1cb35c479f59d1aa/pyarrow-23.0.1-cp314-cp314-win_amd64.whl", hash = "sha256:3a4c85ef66c134161987c17b147d6bffdca4566f9a4c1d81a0a01cdf08414ea5", size = 28234271, upload-time = "2026-02-16T10:14:09.397Z" }, + { url = "https://files.pythonhosted.org/packages/b5/78/07f67434e910a0f7323269be7bfbf58699bd0c1d080b18a1ab49ba943fe8/pyarrow-23.0.1-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:17cd28e906c18af486a499422740298c52d7c6795344ea5002a7720b4eadf16d", size = 34488692, upload-time = "2026-02-16T10:13:21.541Z" }, + { url = "https://files.pythonhosted.org/packages/50/76/34cf7ae93ece1f740a04910d9f7e80ba166b9b4ab9596a953e9e62b90fe1/pyarrow-23.0.1-cp314-cp314t-macosx_12_0_x86_64.whl", hash = "sha256:76e823d0e86b4fb5e1cf4a58d293036e678b5a4b03539be933d3b31f9406859f", size = 35964383, upload-time = "2026-02-16T10:13:28.63Z" }, + { url = "https://files.pythonhosted.org/packages/46/90/459b827238936d4244214be7c684e1b366a63f8c78c380807ae25ed92199/pyarrow-23.0.1-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:a62e1899e3078bf65943078b3ad2a6ddcacf2373bc06379aac61b1e548a75814", size = 44538119, upload-time = "2026-02-16T10:13:35.506Z" }, + { url = "https://files.pythonhosted.org/packages/28/a1/93a71ae5881e99d1f9de1d4554a87be37da11cd6b152239fb5bd924fdc64/pyarrow-23.0.1-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:df088e8f640c9fae3b1f495b3c64755c4e719091caf250f3a74d095ddf3c836d", size = 47571199, upload-time = "2026-02-16T10:13:42.504Z" }, + { url = "https://files.pythonhosted.org/packages/88/a3/d2c462d4ef313521eaf2eff04d204ac60775263f1fb08c374b543f79f610/pyarrow-23.0.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:46718a220d64677c93bc243af1d44b55998255427588e400677d7192671845c7", size = 48259435, upload-time = "2026-02-16T10:13:49.226Z" }, + { url = "https://files.pythonhosted.org/packages/cc/f1/11a544b8c3d38a759eb3fbb022039117fd633e9a7b19e4841cc3da091915/pyarrow-23.0.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a09f3876e87f48bc2f13583ab551f0379e5dfb83210391e68ace404181a20690", size = 50629149, upload-time = "2026-02-16T10:13:57.238Z" }, + { url = "https://files.pythonhosted.org/packages/50/f2/c0e76a0b451ffdf0cf788932e182758eb7558953f4f27f1aff8e2518b653/pyarrow-23.0.1-cp314-cp314t-win_amd64.whl", hash = "sha256:527e8d899f14bd15b740cd5a54ad56b7f98044955373a17179d5956ddb93d9ce", size = 28365807, upload-time = "2026-02-16T10:14:03.892Z" }, +] + [[package]] name = "pycparser" version = "3.0" @@ -1877,6 +1949,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" }, ] +[[package]] +name = "pydicom" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/6f/55ea163b344c91df2e03c007bebf94781f0817656e2c037d7c5bf86c3bfc/pydicom-3.0.1.tar.gz", hash = "sha256:7b8be344b5b62493c9452ba6f5a299f78f8a6ab79786c729b0613698209603ec", size = 2884731, upload-time = "2024-09-22T02:02:43.202Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/a6/98651e752a49f341aa99aa3f6c8ba361728dfc064242884355419df63669/pydicom-3.0.1-py3-none-any.whl", hash = "sha256:db32f78b2641bd7972096b8289111ddab01fb221610de8d7afa835eb938adb41", size = 2376126, upload-time = "2024-09-22T02:02:41.616Z" }, +] + [[package]] name = "pygments" version = "2.19.2"