Skip to content

Coordinator goes into error state on logging-relation-departed #208

@mmkay

Description

@mmkay

Bug Description

Noticed in integration tests for test_core_cos_integrations: test_teardown is failing when any_error is set to True. It's because the logging-relation-departed hook fails. There doesn't seem to be any relevant logs in the CI run.

To Reproduce

tox -re integration -- -k test_core_cos_integrations

Environment

local and CI, integration tests

Relevant log output

E                   'pyroscope': AppStatus(
E                     charm='local:pyroscope-coordinator-k8s-0',
E                     charm_origin='local',
E                     charm_name='pyroscope-coordinator-k8s',
E                     charm_rev=0,
E                     exposed=False,
E                     base=FormattedBase(name='ubuntu', channel='24.04'),
E                     scale=1,
E                     provider_id='dc81fc91-9e73-4369-af36-332d725d955d',
E                     address='10.152.183.221',
E                     app_status=StatusInfo(current='error', message='hook failed: "logging-relation-departed"', since='08 Jul 2025 11:45:27Z'),
E                     relations={
E                       'catalogue': [
E                         AppStatusRelation(related_app='catalogue', interface='catalogue', scope='global'),
E                       ],
E                       'charm-tracing': [
E                         AppStatusRelation(related_app='tempo', interface='tracing', scope='global'),
E                       ],
E                       'logging': [
E                         AppStatusRelation(related_app='loki', interface='loki_push_api', scope='global'),
E                       ],
E                       'metrics-endpoint': [
E                         AppStatusRelation(related_app='prometheus', interface='prometheus_scrape', scope='global'),
E                       ],
E                       'peers': [
E                         AppStatusRelation(related_app='pyroscope', interface='pyroscope_peers', scope='global'),
E                       ],
E                       'pyroscope-cluster': [
E                         AppStatusRelation(related_app='pyroscope-worker-ad-hoc-profiles', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-compactor', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-distributor', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-ingester', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-querier', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-query-frontend', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-query-scheduler', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-store-gateway', interface='pyroscope_cluster', scope='global'),
E                         AppStatusRelation(related_app='pyroscope-worker-tenant-settings', interface='pyroscope_cluster', scope='global'),
E                       ],
E                       's3': [
E                         AppStatusRelation(related_app='s3-integrator', interface='s3', scope='global'),
E                       ],
E                       'workload-tracing': [
E                         AppStatusRelation(related_app='tempo', interface='tracing', scope='global'),
E                       ],
E                     },
E                     units={
E                       'pyroscope/0': UnitStatus(
E                         workload_status=StatusInfo(current='error', message='hook failed: "logging-relation-departed" for loki:logging', since='08 Jul 2025 11:45:27Z'),
E                         juju_status=StatusInfo(current='idle', since='08 Jul 2025 11:45:27Z', version='3.6.8'),
E                         leader=True,
E                         address='10.1.137.105',
E                         provider_id='pyroscope-0',
E                       ),
E                     },
E                     endpoint_bindings={
E                       '': 'alpha',
E                       'catalogue': 'alpha',
E                       'certificates': 'alpha',
E                       'charm-tracing': 'alpha',
E                       'grafana-dashboard': 'alpha',
E                       'ingress': 'alpha',
E                       'logging': 'alpha',
E                       'metrics-endpoint': 'alpha',
E                       'peers': 'alpha',
E                       'pyroscope-cluster': 'alpha',
E                       's3': 'alpha',
E                       'send-datasource': 'alpha',
E                       'workload-tracing': 'alpha',
E                     },
E                   ),

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions