Steps to reproduce
- Clone the source code for
cassandra-operator.
If this issue is reviewed after a temporary bug fix on the operator side, remove the following lines from src/core/state.py:
if (self.model.juju_version <= JujuVersion("3.5.7")) and os.getenv("JUJU_HOOK_NAME", "") == f"{DATA_STORAGE}-storage-detaching":
unit_additional_secret_fields = None
- Build and deploy 2 units:
juju deploy ./cassandra_ubuntu@24.04-amd64.charm cassandra --config profile=testing --num-units=2
- Remove a secondary unit:
juju remove-unit <secondary-unit>
-
Wait until the storage-detaching hook completes on the departing unit.
-
Observe that the hook fails on any remaining unit with the error:
cannot apply changes: creating secrets: cannot create secret for owner "<secondary-unit>" which is not alive
Expected behavior
- After a unit begins removal (life =
dying), no new secrets should be created on its behalf, neither by the departing unit nor by surviving peers.
DataPeerUnitData should skip secret creation for units whose life is dying or defer until the unit is alive again.
- The unit should be removed cleanly after the
storage-detaching hook completes successfully.
Actual behavior
DataPeerUnitData automatically attempts to create Juju secrets for all fields in additional_secret_fields.
- Juju rejects requests for units that are not alive, causing the hook to fail and retry indefinitely, which blocks the application.
Versions
Operating system: Ubuntu 24.04.1 LTS
Juju CLI: 3.5.7-genericlinux-amd64
Juju agent: 3.5.7
Log output
unit-cassandra-1: 17:10:48 ERROR juju.worker.uniter.context cannot apply changes: creating secrets: cannot create secret for owner "unit-cassandra-1" which is not alive
unit-cassandra-1: 17:10:48 ERROR juju.worker.uniter.operation hook "data-storage-detaching" (via hook dispatching script: dispatch) failed: creating secrets: cannot create secret for owner "unit-cassandra-1" which is not alive
Additional context
- This issue occurs only on Juju v3.5; Juju v3.6 does not have this problem.
- The issue can be worked around by removing
additional_secret_fields from DataPeerUnitData during unit removal:
if (self.model.juju_version <= JujuVersion("3.5.7")) and os.getenv("JUJU_HOOK_NAME", "") == f"{DATA_STORAGE}-storage-detaching":
unit_additional_secret_fields = None
Steps to reproduce
cassandra-operator.If this issue is reviewed after a temporary bug fix on the operator side, remove the following lines from
src/core/state.py:Wait until the
storage-detachinghook completes on the departing unit.Observe that the hook fails on any remaining unit with the error:
Expected behavior
dying), no new secrets should be created on its behalf, neither by the departing unit nor by surviving peers.DataPeerUnitDatashould skip secret creation for units whose life isdyingor defer until the unit is alive again.storage-detachinghook completes successfully.Actual behavior
DataPeerUnitDataautomatically attempts to create Juju secrets for all fields inadditional_secret_fields.Versions
Operating system: Ubuntu 24.04.1 LTS
Juju CLI: 3.5.7-genericlinux-amd64
Juju agent: 3.5.7
Log output
Additional context
additional_secret_fieldsfromDataPeerUnitDataduring unit removal: