Conversation
…_reuse (Phase 2a) Phase 2a of the migration plan in ``juniper-ml/notes/observability/REGISTER_OR_REUSE_HELPER_DESIGN_2026-05-05.md``. Drops the inline ``_get_or_create`` helper added in PR #87 and calls the canonical ``juniper_observability.register_or_reuse`` shipped in juniper-observability ``0.2.0``. ### What changes - ``juniper_data/api/observability.py``: ``_ensure_dataset_metrics`` now imports ``register_or_reuse`` from ``juniper_observability`` (lazy, inside the ``if _dataset_metrics is None`` branch) and uses it for all four collectors (Counter ×2 + Histogram + Gauge). The inline ``_get_or_create`` closure and the manual ``REGISTRY._names_to_collectors`` lookup are gone. - ``pyproject.toml``: ``juniper-observability>=0.1.1`` → ``juniper-observability>=0.2.0``. The new helper is the reason for the bump; existing 0.1.1 callers' behaviour is unchanged. ### Drive-by fix: storage circular import PR #92 (``audit-fixup: wire juniper_data_datasets_cached gauge to cache layer``) added ``from juniper_data.api.observability import set_datasets_cached`` at the top of ``juniper_data/storage/cached.py``. That triggers the import chain: juniper_data.storage.__init__ → cached.py → juniper_data.api.observability → juniper_data.api.__init__ → app.py → juniper_data.storage (← still initialising!) → ImportError: cannot import name 'LocalFSDatasetStore' …which broke ``pytest --collect-only`` on origin/main as of 2026-05-06: 31 collection errors covering every test that imports ``juniper_data.storage`` directly or transitively. ``--collect-only`` amplifies the breakage, but a normal full pytest run also hits the same path during fixture collection. Fix: defer the ``set_datasets_cached`` import to inside ``CachedDatasetStore._emit_cached_count`` so the cycle never fires at module-import time. The function is best-effort with a ``except Exception`` swallowing failures, so a deferred import that raises ``ImportError`` (e.g. during a ``CachedDatasetStore`` instantiation in a non-API context) gets logged at DEBUG and skipped the same way any other observability failure does. ### Verification Full juniper-data suite under JuniperData env: **950 passed** (was ``31 errors during collection`` on origin/main pre-fix). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2a of the migration plan in
juniper-ml/notes/observability/REGISTER_OR_REUSE_HELPER_DESIGN_2026-05-05.md.Drops the inline
_get_or_createhelper added in PR #87 and calls the canonicaljuniper_observability.register_or_reuseshipped in juniper-observability0.2.0.What changes
juniper_data/api/observability.py_ensure_dataset_metricsswaps inline_get_or_createforregister_or_reusefromjuniper_observability. Drops the inline closure +REGISTRY._names_to_collectorslookup.pyproject.tomljuniper-observability>=0.1.1→>=0.2.0Drive-by: storage circular import (separate-but-blocking)
PR #92 added a top-level
from juniper_data.api.observability import set_datasets_cachedinjuniper_data/storage/cached.py. That triggers a circular import viajuniper_data.api.__init__→app.py→juniper_data.storage(still initialising), which brokepytest --collect-onlyon origin/main as of 2026-05-06 with 31 collection errors across every test that importsjuniper_data.storagedirectly or transitively.Surfaced during verification of this PR's main change. Fix is a deferred import inside
CachedDatasetStore._emit_cached_count— the function is best-effort with anexcept Exceptionswallowing failures, so the deferred import slot in already-protected. Could be split into its own PR but the migration was wedged on it, so bundled here with explicit call-out.Verification
Full juniper-data suite under JuniperData env: 950 passed (was
31 errors during collectionon origin/main pre-fix).Test plan
test_observability.py(the SEC-16 / dataset-metrics regressions) still passes>=0.2.0so the helper resolvesCompanion PRs
Part of Phase 2 — also coming: juniper-canopy and juniper-cascor migrations. juniper-cascor-client deliberately excluded from Phase 2 (would add pydantic + starlette as transitive deps to a leaf library; the lone PR #37 inline guard stays).
🤖 Generated with Claude Code