Migrate zaza from libjuju (async) to jubilant (sync)#689
Draft
freyes wants to merge 8 commits intoopenstack-charmers:masterfrom
Draft
Migrate zaza from libjuju (async) to jubilant (sync)#689freyes wants to merge 8 commits intoopenstack-charmers:masterfrom
freyes wants to merge 8 commits intoopenstack-charmers:masterfrom
Conversation
Replace the libjuju-based async machinery with jubilant, a synchronous
CLI wrapper around the juju binary. The public API surface is preserved
so that downstream consumers (e.g. zaza-openstack-tests) continue to
work without changes.
* Remove the libjuju background-thread infrastructure: event loop
management, _libjuju_thread/_libjuju_loop globals, libjuju_thread_run(),
get_or_create_libjuju_thread(), join_libjuju_thread(),
clean_up_libjuju_thread(), and run_coroutine_threadsafe().
* Set RUN_LIBJUJU_IN_THREAD = False (no-op constant kept for compat).
* Rewrite sync_wrapper() as a transparent pass-through: if the wrapped
callable returns a coroutine it is drained with a fresh event loop;
otherwise the result is returned directly. This allows async functions
that have not yet been rewritten to keep working without a background
thread.
* Retain clean_up_libjuju_thread() / join_libjuju_thread() /
get_or_create_libjuju_thread() as no-op stubs for downstream compat.
* Rewrite all functions using jubilant.Juju() instead of libjuju:
- add_model(): jubilant.Juju().cli('add-model', ...)
- destroy_model(): polling loop with jubilant.Juju().cli('destroy-model')
- list_models(): jubilant.Juju().cli('models', '--format', 'json')
- cloud() / get_cloud(): jubilant.Juju().cli('clouds', '--format', 'json')
* Remove all async/await keywords; functions are now plain synchronous.
* Fix pre-existing syntax errors: 'model_name=None: Optional[str]'
corrected to 'model_name: Optional[str] = None' in deployed(),
get_unit_from_name(), get_model(), scp_to_unit().
* Fix incorrect import: ModelInfo -> ModelStatus from jubilant.
* Add 'import dataclasses' at the top level.
* Add get_status_jubilant(model_name=None):
- Calls jubilant.Juju(model=model_name).status().
- Collects subordinate unit statuses from each principal unit's
.subordinates dict and injects them back into the subordinate
application's units dict using dataclasses.replace() (required
because jubilant Status is a frozen dataclass).
- Returns a fully-populated jubilant Status object.
* Add adapter classes for the wait_for_application_states polling loop:
- _JubilantUnitAdapter: wraps jubilant UnitStatus; exposes
workload_status (str), workload_status_message, entity_id, name,
application, machine (None), data (agent-status dict).
- _JubilantAppAdapter: wraps jubilant AppStatus; .units returns a
list of _JubilantUnitAdapter objects.
- _JubilantModelAdapter: wraps jubilant Status; exposes .units (flat
dict across all apps) and .applications (dict of _JubilantAppAdapter).
* Add libjuju-compatibility wrappers for callers of get_status():
- _LibjujuUnitCompat: exposes .get('subordinates') and dict-style
access to unit fields.
- _LibjujuAppCompat: exposes .status.status and ['units'] dict.
- _LibjujuStatusCompat: exposes .applications dict of _LibjujuAppCompat.
* Override get_status() to return _LibjujuStatusCompat(get_status_jubilant())
so that callers such as failure_report() and get_subordinate_units()
continue to work unchanged.
* Rewrite wait_for_application_states() as a fully synchronous polling
loop:
- Uses time.sleep(1.0) between iterations instead of asyncio.sleep.
- Calls get_status_jubilant() each iteration; temporarily suppresses
jubilant's per-call INFO log (set logger to WARNING) to avoid noise.
- Uses _JubilantModelAdapter for unit/app access.
- Alias async_wait_for_application_states = wait_for_application_states
for backward compatibility.
* Rewrite async_get_units() to use get_status_jubilant(); returns a list
of _JubilantUnitAdapter objects.
* Rewrite async_get_lead_unit() to use get_status_jubilant(); identifies
the leader via UnitStatus.leader.
* Rewrite async_get_unit_public_address__fallback() to use
get_status_jubilant() and UnitStatus.public_address.
* Remove tests for the removed async-thread machinery.
* Add tests for the new sync_wrapper behaviour (plain return, coroutine
drain, nested wrapping).
* Replace all libjuju mock scaffolding with jubilant mocks.
* Tests now patch jubilant.Juju and its .cli() / .status() methods.
* Add _make_jubilant_status() helper to build mock jubilant Status
objects.
* Rewrite _application_states_setup() to patch get_status_jubilant,
time.time, and time.sleep instead of libjuju model/connect mocks.
* Update all test_wait_for_application_states_* tests to use the new
sync polling loop and jubilant mock objects.
* Fix patching targets: resolve_units / block_until_unit_wl_status
(not their async_ aliases).
* tests/tests.yaml: update magpie-jammy workload-status-message-prefix
to match current charm output ('Ready, with 1 peer').
* tests/bundles/first.yaml: update top-level series to 'noble'.
Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
Use a unique artifact name per matrix job by combining the bundle name and the juju_channel. Since artifact names cannot contain '/', sanitize the channel value (e.g. 2.9/stable -> 2.9-stable) via a prior step before passing it to actions/upload-artifact. Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
- Remove unused imports: async_generator, deprecate (zaza/model.py), concurrent (unit_tests/test_zaza_model.py) - Add import juju.client.jujudata to resolve undefined name in async_get_cloud_data - Fix trailing/blank-line whitespace (W291, W293) - Fix blank line counts around functions (E303, E305) - Remove duplicate get_status = sync_wrapper(...) definition (F811) - Remove unused local variable 'unit' in exec_on_unit (F841) - Fix continuation line over-indentation in scp_to_all_units and wait_for_application_states (E127) - Wrap long lines to stay within 79 chars (E501) - Fix docstrings in _LibjujuAppCompat, _StatusInfo, _LibjujuUnitCompat and _LibjujuStatusCompat (D205, D400, D402) - Rewrite async_run_action, async_run_action_on_leader, async_run_action_on_units to use jubilant run_action, removing references to removed libjuju helpers (F821) - Replace async_run_on_unit with exec_on_unit in async_block_until_service_status (F821) - Replace Model() usage with get_juju_model() in async_get_current_model (F821) - Replace run_on_leader/run_on_unit with exec_on_leader/exec_on_unit in file_contents (F821) - Remove _normalise_action_object call from async_block_until_file_missing (F821) - Replace async_get_unit_service_start_time with sync get_unit_service_start_time in async_block_until_services_restarted (F821)
The jubilant branch replaces python-libjuju (async) with jubilant (sync Juju CLI wrapper). This commit completes the migration by fixing all 110 unit-test failures that resulted from the partial migration: zaza/model.py - Add Model() as the primary model factory (ZazaJujuModel wrapper) - Make get_model() async for backward compat with await callers - Add async_get_juju_model(), async_get_unit_from_name() shims - Add get_model_memo(), remove_model_memo(), run_in_model compat stubs - Introduce _async_run_on_unit_impl() - async core that calls unit.run() so existing tests and callers work unchanged - run_on_unit() wraps _async_run_on_unit_impl via sync_wrapper - async_run_on_unit() awaits _async_run_on_unit_impl directly - exec_on_leader() / run_on_leader alias: use is_leader_from_status() and run_on_unit() for libjuju-compatible behaviour - get_unit_time(), get_systemd_service_active_time(), get_unit_service_start_time(): call sync_wrapper(async_run_on_unit) so test patches take effect - async_block_until_service_status(): await async_run_on_unit() - async_run_action(), async_run_action_on_leader(), async_run_action_on_units(): call unit.run_action() directly - async_get_units(): use Model().units dict instead of get_status_jubilant - async_get_lead_unit(): use async_get_units() + is_leader_from_status() - get_current_model(): standalone sync fn returning Model().info.name - async_get_current_model(): use jubilant.Juju().model directly to break circular dependency with get_juju_model() - block_until_auto_reconnect_model(): make async, add aconditions support and reconnect logic (disconnect/connect_model) - ensure_model_connected(): make async, implement disconnect/reconnect - async_block_until_all_units_idle(): use Model() + block_until_auto_reconnect_model - async_block_until_services_restarted(): use async_block_until + async_get_unit_service_start_time (patchable) - async_block_until_file_missing(): use _async_run_on_unit_impl - async_get_unit_public_address__fallback(): use generic_utils.check_output with juju status --format=yaml (matches test expectations) - async_get_cloud_data(): use await get_model() to be patchable in async test contexts - get_unit_from_name(): raise UnitNotFound correctly for missing units; add async_get_unit_from_name() shim - file_contents(): use run_on_unit/run_on_leader (libjuju-style) - Fix all invalid 'await get_model(...)' calls (get_model was sync) - Fix deployed() / sync_deployed() to use Model().status().apps unit_tests/test_zaza_model.py - Update test_deployed_* to assert Model(None)/status() calls - Update test_get_model_info to mock show_model() not info - Update test_scp_to_unit/from_unit to assert model.scp() calls - Update test_scp_to_all_units to use dict-based applications mock - Update test_update_unknown_action_status_invalid_params to use async_update_unknown_action_status - Update test_async_get_cloud_data to patch get_model with AsyncMock - Remove trailing blank line (W391) unit_tests/utilities/test_deployment_env.py - Mock get_setup_file_section() in model-constraints tests to prevent reading real ~/.zaza.yaml during test runs Result: 740 tests pass, 0 failures (tox -e py3,pep8) Co-authored-by: GitHub Copilot <copilot@github.com> Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
Replace PEP 585 builtin generic 'dict[str, AppStatus]' with 'Dict[str, AppStatus]' from typing, which is required for Python 3.8 compatibility. Co-authored-by: GitHub Copilot <copilot@github.com> Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the libjuju-based async machinery with jubilant, a synchronous
CLI wrapper around the juju binary. The public API surface is preserved
so that downstream consumers (e.g. zaza-openstack-tests) continue to
work without changes.
Remove the libjuju background-thread infrastructure: event loop
management, _libjuju_thread/_libjuju_loop globals, libjuju_thread_run(),
get_or_create_libjuju_thread(), join_libjuju_thread(),
clean_up_libjuju_thread(), and run_coroutine_threadsafe().
Set RUN_LIBJUJU_IN_THREAD = False (no-op constant kept for compat).
Rewrite sync_wrapper() as a transparent pass-through: if the wrapped
callable returns a coroutine it is drained with a fresh event loop;
otherwise the result is returned directly. This allows async functions
that have not yet been rewritten to keep working without a background
thread.
Retain clean_up_libjuju_thread() / join_libjuju_thread() /
get_or_create_libjuju_thread() as no-op stubs for downstream compat.
Rewrite all functions using jubilant.Juju() instead of libjuju:
Remove all async/await keywords; functions are now plain synchronous.
Fix pre-existing syntax errors: 'model_name=None: Optional[str]'
corrected to 'model_name: Optional[str] = None' in deployed(),
get_unit_from_name(), get_model(), scp_to_unit().
Fix incorrect import: ModelInfo -> ModelStatus from jubilant.
Add 'import dataclasses' at the top level.
Add get_status_jubilant(model_name=None):
.subordinates dict and injects them back into the subordinate
application's units dict using dataclasses.replace() (required
because jubilant Status is a frozen dataclass).
Add adapter classes for the wait_for_application_states polling loop:
workload_status (str), workload_status_message, entity_id, name,
application, machine (None), data (agent-status dict).
list of _JubilantUnitAdapter objects.
dict across all apps) and .applications (dict of _JubilantAppAdapter).
Add libjuju-compatibility wrappers for callers of get_status():
access to unit fields.
Override get_status() to return _LibjujuStatusCompat(get_status_jubilant())
so that callers such as failure_report() and get_subordinate_units()
continue to work unchanged.
Rewrite wait_for_application_states() as a fully synchronous polling
loop:
jubilant's per-call INFO log (set logger to WARNING) to avoid noise.
for backward compatibility.
Rewrite async_get_units() to use get_status_jubilant(); returns a list
of _JubilantUnitAdapter objects.
Rewrite async_get_lead_unit() to use get_status_jubilant(); identifies
the leader via UnitStatus.leader.
Rewrite async_get_unit_public_address__fallback() to use
get_status_jubilant() and UnitStatus.public_address.
Remove tests for the removed async-thread machinery.
Add tests for the new sync_wrapper behaviour (plain return, coroutine
drain, nested wrapping).
Replace all libjuju mock scaffolding with jubilant mocks.
Tests now patch jubilant.Juju and its .cli() / .status() methods.
Add _make_jubilant_status() helper to build mock jubilant Status
objects.
Rewrite _application_states_setup() to patch get_status_jubilant,
time.time, and time.sleep instead of libjuju model/connect mocks.
Update all test_wait_for_application_states_* tests to use the new
sync polling loop and jubilant mock objects.
Fix patching targets: resolve_units / block_until_unit_wl_status
(not their async_ aliases).
tests/tests.yaml: update magpie-jammy workload-status-message-prefix
to match current charm output ('Ready, with 1 peer').
tests/bundles/first.yaml: update top-level series to 'noble'.
Co-authored-by: GitHub Copilot copilot@github.com
Signed-off-by: Felipe Reyes felipe.reyes@canonical.com