Skip to content

Migrate zaza from libjuju (async) to jubilant (sync)#689

Draft
freyes wants to merge 8 commits intoopenstack-charmers:masterfrom
freyes:jubilant
Draft

Migrate zaza from libjuju (async) to jubilant (sync)#689
freyes wants to merge 8 commits intoopenstack-charmers:masterfrom
freyes:jubilant

Conversation

@freyes
Copy link
Copy Markdown
Member

@freyes freyes commented Apr 7, 2026

Replace the libjuju-based async machinery with jubilant, a synchronous
CLI wrapper around the juju binary. The public API surface is preserved
so that downstream consumers (e.g. zaza-openstack-tests) continue to
work without changes.

  • Remove the libjuju background-thread infrastructure: event loop
    management, _libjuju_thread/_libjuju_loop globals, libjuju_thread_run(),
    get_or_create_libjuju_thread(), join_libjuju_thread(),
    clean_up_libjuju_thread(), and run_coroutine_threadsafe().

  • Set RUN_LIBJUJU_IN_THREAD = False (no-op constant kept for compat).

  • Rewrite sync_wrapper() as a transparent pass-through: if the wrapped
    callable returns a coroutine it is drained with a fresh event loop;
    otherwise the result is returned directly. This allows async functions
    that have not yet been rewritten to keep working without a background
    thread.

  • Retain clean_up_libjuju_thread() / join_libjuju_thread() /
    get_or_create_libjuju_thread() as no-op stubs for downstream compat.

  • Rewrite all functions using jubilant.Juju() instead of libjuju:

    • add_model(): jubilant.Juju().cli('add-model', ...)
    • destroy_model(): polling loop with jubilant.Juju().cli('destroy-model')
    • list_models(): jubilant.Juju().cli('models', '--format', 'json')
    • cloud() / get_cloud(): jubilant.Juju().cli('clouds', '--format', 'json')
  • Remove all async/await keywords; functions are now plain synchronous.

  • Fix pre-existing syntax errors: 'model_name=None: Optional[str]'
    corrected to 'model_name: Optional[str] = None' in deployed(),
    get_unit_from_name(), get_model(), scp_to_unit().

  • Fix incorrect import: ModelInfo -> ModelStatus from jubilant.

  • Add 'import dataclasses' at the top level.

  • Add get_status_jubilant(model_name=None):

    • Calls jubilant.Juju(model=model_name).status().
    • Collects subordinate unit statuses from each principal unit's
      .subordinates dict and injects them back into the subordinate
      application's units dict using dataclasses.replace() (required
      because jubilant Status is a frozen dataclass).
    • Returns a fully-populated jubilant Status object.
  • Add adapter classes for the wait_for_application_states polling loop:

    • _JubilantUnitAdapter: wraps jubilant UnitStatus; exposes
      workload_status (str), workload_status_message, entity_id, name,
      application, machine (None), data (agent-status dict).
    • _JubilantAppAdapter: wraps jubilant AppStatus; .units returns a
      list of _JubilantUnitAdapter objects.
    • _JubilantModelAdapter: wraps jubilant Status; exposes .units (flat
      dict across all apps) and .applications (dict of _JubilantAppAdapter).
  • Add libjuju-compatibility wrappers for callers of get_status():

    • _LibjujuUnitCompat: exposes .get('subordinates') and dict-style
      access to unit fields.
    • _LibjujuAppCompat: exposes .status.status and ['units'] dict.
    • _LibjujuStatusCompat: exposes .applications dict of _LibjujuAppCompat.
  • Override get_status() to return _LibjujuStatusCompat(get_status_jubilant())
    so that callers such as failure_report() and get_subordinate_units()
    continue to work unchanged.

  • Rewrite wait_for_application_states() as a fully synchronous polling
    loop:

    • Uses time.sleep(1.0) between iterations instead of asyncio.sleep.
    • Calls get_status_jubilant() each iteration; temporarily suppresses
      jubilant's per-call INFO log (set logger to WARNING) to avoid noise.
    • Uses _JubilantModelAdapter for unit/app access.
    • Alias async_wait_for_application_states = wait_for_application_states
      for backward compatibility.
  • Rewrite async_get_units() to use get_status_jubilant(); returns a list
    of _JubilantUnitAdapter objects.

  • Rewrite async_get_lead_unit() to use get_status_jubilant(); identifies
    the leader via UnitStatus.leader.

  • Rewrite async_get_unit_public_address__fallback() to use
    get_status_jubilant() and UnitStatus.public_address.

  • Remove tests for the removed async-thread machinery.

  • Add tests for the new sync_wrapper behaviour (plain return, coroutine
    drain, nested wrapping).

  • Replace all libjuju mock scaffolding with jubilant mocks.

  • Tests now patch jubilant.Juju and its .cli() / .status() methods.

  • Add _make_jubilant_status() helper to build mock jubilant Status
    objects.

  • Rewrite _application_states_setup() to patch get_status_jubilant,
    time.time, and time.sleep instead of libjuju model/connect mocks.

  • Update all test_wait_for_application_states_* tests to use the new
    sync polling loop and jubilant mock objects.

  • Fix patching targets: resolve_units / block_until_unit_wl_status
    (not their async_ aliases).

  • tests/tests.yaml: update magpie-jammy workload-status-message-prefix
    to match current charm output ('Ready, with 1 peer').

  • tests/bundles/first.yaml: update top-level series to 'noble'.

Co-authored-by: GitHub Copilot copilot@github.com
Signed-off-by: Felipe Reyes felipe.reyes@canonical.com

freyes and others added 8 commits December 9, 2025 15:28
Replace the libjuju-based async machinery with jubilant, a synchronous
CLI wrapper around the juju binary.  The public API surface is preserved
so that downstream consumers (e.g. zaza-openstack-tests) continue to
work without changes.

* Remove the libjuju background-thread infrastructure: event loop
  management, _libjuju_thread/_libjuju_loop globals, libjuju_thread_run(),
  get_or_create_libjuju_thread(), join_libjuju_thread(),
  clean_up_libjuju_thread(), and run_coroutine_threadsafe().
* Set RUN_LIBJUJU_IN_THREAD = False (no-op constant kept for compat).
* Rewrite sync_wrapper() as a transparent pass-through: if the wrapped
  callable returns a coroutine it is drained with a fresh event loop;
  otherwise the result is returned directly.  This allows async functions
  that have not yet been rewritten to keep working without a background
  thread.
* Retain clean_up_libjuju_thread() / join_libjuju_thread() /
  get_or_create_libjuju_thread() as no-op stubs for downstream compat.

* Rewrite all functions using jubilant.Juju() instead of libjuju:
  - add_model(): jubilant.Juju().cli('add-model', ...)
  - destroy_model(): polling loop with jubilant.Juju().cli('destroy-model')
  - list_models(): jubilant.Juju().cli('models', '--format', 'json')
  - cloud() / get_cloud(): jubilant.Juju().cli('clouds', '--format', 'json')
* Remove all async/await keywords; functions are now plain synchronous.

* Fix pre-existing syntax errors: 'model_name=None: Optional[str]'
  corrected to 'model_name: Optional[str] = None' in deployed(),
  get_unit_from_name(), get_model(), scp_to_unit().
* Fix incorrect import: ModelInfo -> ModelStatus from jubilant.
* Add 'import dataclasses' at the top level.
* Add get_status_jubilant(model_name=None):
  - Calls jubilant.Juju(model=model_name).status().
  - Collects subordinate unit statuses from each principal unit's
    .subordinates dict and injects them back into the subordinate
    application's units dict using dataclasses.replace() (required
    because jubilant Status is a frozen dataclass).
  - Returns a fully-populated jubilant Status object.
* Add adapter classes for the wait_for_application_states polling loop:
  - _JubilantUnitAdapter: wraps jubilant UnitStatus; exposes
    workload_status (str), workload_status_message, entity_id, name,
    application, machine (None), data (agent-status dict).
  - _JubilantAppAdapter: wraps jubilant AppStatus; .units returns a
    list of _JubilantUnitAdapter objects.
  - _JubilantModelAdapter: wraps jubilant Status; exposes .units (flat
    dict across all apps) and .applications (dict of _JubilantAppAdapter).
* Add libjuju-compatibility wrappers for callers of get_status():
  - _LibjujuUnitCompat: exposes .get('subordinates') and dict-style
    access to unit fields.
  - _LibjujuAppCompat: exposes .status.status and ['units'] dict.
  - _LibjujuStatusCompat: exposes .applications dict of _LibjujuAppCompat.
* Override get_status() to return _LibjujuStatusCompat(get_status_jubilant())
  so that callers such as failure_report() and get_subordinate_units()
  continue to work unchanged.
* Rewrite wait_for_application_states() as a fully synchronous polling
  loop:
  - Uses time.sleep(1.0) between iterations instead of asyncio.sleep.
  - Calls get_status_jubilant() each iteration; temporarily suppresses
    jubilant's per-call INFO log (set logger to WARNING) to avoid noise.
  - Uses _JubilantModelAdapter for unit/app access.
  - Alias async_wait_for_application_states = wait_for_application_states
    for backward compatibility.
* Rewrite async_get_units() to use get_status_jubilant(); returns a list
  of _JubilantUnitAdapter objects.
* Rewrite async_get_lead_unit() to use get_status_jubilant(); identifies
  the leader via UnitStatus.leader.
* Rewrite async_get_unit_public_address__fallback() to use
  get_status_jubilant() and UnitStatus.public_address.

* Remove tests for the removed async-thread machinery.
* Add tests for the new sync_wrapper behaviour (plain return, coroutine
  drain, nested wrapping).

* Replace all libjuju mock scaffolding with jubilant mocks.
* Tests now patch jubilant.Juju and its .cli() / .status() methods.

* Add _make_jubilant_status() helper to build mock jubilant Status
  objects.
* Rewrite _application_states_setup() to patch get_status_jubilant,
  time.time, and time.sleep instead of libjuju model/connect mocks.
* Update all test_wait_for_application_states_* tests to use the new
  sync polling loop and jubilant mock objects.
* Fix patching targets: resolve_units / block_until_unit_wl_status
  (not their async_ aliases).

* tests/tests.yaml: update magpie-jammy workload-status-message-prefix
  to match current charm output ('Ready, with 1 peer').
* tests/bundles/first.yaml: update top-level series to 'noble'.

Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
Use a unique artifact name per matrix job by combining the bundle name
and the juju_channel. Since artifact names cannot contain '/', sanitize
the channel value (e.g. 2.9/stable -> 2.9-stable) via a prior step
before passing it to actions/upload-artifact.

Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
- Remove unused imports: async_generator, deprecate (zaza/model.py),
  concurrent (unit_tests/test_zaza_model.py)
- Add import juju.client.jujudata to resolve undefined name in
  async_get_cloud_data
- Fix trailing/blank-line whitespace (W291, W293)
- Fix blank line counts around functions (E303, E305)
- Remove duplicate get_status = sync_wrapper(...) definition (F811)
- Remove unused local variable 'unit' in exec_on_unit (F841)
- Fix continuation line over-indentation in scp_to_all_units and
  wait_for_application_states (E127)
- Wrap long lines to stay within 79 chars (E501)
- Fix docstrings in _LibjujuAppCompat, _StatusInfo, _LibjujuUnitCompat
  and _LibjujuStatusCompat (D205, D400, D402)
- Rewrite async_run_action, async_run_action_on_leader,
  async_run_action_on_units to use jubilant run_action, removing
  references to removed libjuju helpers (F821)
- Replace async_run_on_unit with exec_on_unit in
  async_block_until_service_status (F821)
- Replace Model() usage with get_juju_model() in
  async_get_current_model (F821)
- Replace run_on_leader/run_on_unit with exec_on_leader/exec_on_unit
  in file_contents (F821)
- Remove _normalise_action_object call from
  async_block_until_file_missing (F821)
- Replace async_get_unit_service_start_time with sync
  get_unit_service_start_time in async_block_until_services_restarted
  (F821)
The jubilant branch replaces python-libjuju (async) with jubilant
(sync Juju CLI wrapper).  This commit completes the migration by
fixing all 110 unit-test failures that resulted from the partial
migration:

zaza/model.py
- Add Model() as the primary model factory (ZazaJujuModel wrapper)
- Make get_model() async for backward compat with await callers
- Add async_get_juju_model(), async_get_unit_from_name() shims
- Add get_model_memo(), remove_model_memo(), run_in_model compat stubs
- Introduce _async_run_on_unit_impl() - async core that calls
  unit.run() so existing tests and callers work unchanged
- run_on_unit() wraps _async_run_on_unit_impl via sync_wrapper
- async_run_on_unit() awaits _async_run_on_unit_impl directly
- exec_on_leader() / run_on_leader alias: use is_leader_from_status()
  and run_on_unit() for libjuju-compatible behaviour
- get_unit_time(), get_systemd_service_active_time(),
  get_unit_service_start_time(): call sync_wrapper(async_run_on_unit)
  so test patches take effect
- async_block_until_service_status(): await async_run_on_unit()
- async_run_action(), async_run_action_on_leader(),
  async_run_action_on_units(): call unit.run_action() directly
- async_get_units(): use Model().units dict instead of get_status_jubilant
- async_get_lead_unit(): use async_get_units() + is_leader_from_status()
- get_current_model(): standalone sync fn returning Model().info.name
- async_get_current_model(): use jubilant.Juju().model directly to
  break circular dependency with get_juju_model()
- block_until_auto_reconnect_model(): make async, add aconditions
  support and reconnect logic (disconnect/connect_model)
- ensure_model_connected(): make async, implement disconnect/reconnect
- async_block_until_all_units_idle(): use Model() + block_until_auto_reconnect_model
- async_block_until_services_restarted(): use async_block_until +
  async_get_unit_service_start_time (patchable)
- async_block_until_file_missing(): use _async_run_on_unit_impl
- async_get_unit_public_address__fallback(): use generic_utils.check_output
  with juju status --format=yaml (matches test expectations)
- async_get_cloud_data(): use await get_model() to be patchable in
  async test contexts
- get_unit_from_name(): raise UnitNotFound correctly for missing units;
  add async_get_unit_from_name() shim
- file_contents(): use run_on_unit/run_on_leader (libjuju-style)
- Fix all invalid 'await get_model(...)' calls (get_model was sync)
- Fix deployed() / sync_deployed() to use Model().status().apps

unit_tests/test_zaza_model.py
- Update test_deployed_* to assert Model(None)/status() calls
- Update test_get_model_info to mock show_model() not info
- Update test_scp_to_unit/from_unit to assert model.scp() calls
- Update test_scp_to_all_units to use dict-based applications mock
- Update test_update_unknown_action_status_invalid_params to use
  async_update_unknown_action_status
- Update test_async_get_cloud_data to patch get_model with AsyncMock
- Remove trailing blank line (W391)

unit_tests/utilities/test_deployment_env.py
- Mock get_setup_file_section() in model-constraints tests to prevent
  reading real ~/.zaza.yaml during test runs

Result: 740 tests pass, 0 failures (tox -e py3,pep8)

Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
Replace PEP 585 builtin generic 'dict[str, AppStatus]' with
'Dict[str, AppStatus]' from typing, which is required for
Python 3.8 compatibility.

Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: Felipe Reyes <felipe.reyes@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant