fix: prefix ClusterId with project_id to prevent collisions across projects (GTI-685)#24
Merged
ymendez-redis merged 3 commits intomainfrom Apr 24, 2026
Merged
Conversation
…vent collisions across projects Valkey and Redis Cluster instances use short names as ClusterId (e.g., 'memorystore-valkey') which can collide when output from multiple GCP projects is combined. Redis standalone is unaffected as it already uses a full GCP resource path. - Extract _resolve_inst_key() helper that detects short vs full GCP paths - Short names get prefixed with project_id (e.g., 'my-project/memorystore-valkey') - Full GCP paths (projects/...) pass through unchanged - Update _attach_memory_usage and _attach_capacity_scalar to use the helper - Add 10 unit tests covering all scenarios
…ts into gti-608-fix-clusterid-collision # Conflicts: # memorystore.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GTI-685
When
memorystore.pyoutput from multiple GCP projects is combined,redis2re.pymerges instances that share the sameClusterIdandRegioninto one row — losing data and producing incorrect memory calculations.This happens because Valkey and Redis Cluster instances use a short name as ClusterId (e.g.,
memorystore-valkey), which can be identical across projects. Redis standalone is unaffected because GCP returns a full resource path (projects/proj-id/locations/.../instances/name).Real production example
10 input rows → 3 output rows instead of 4.
Root Cause
GCP returns different identifier formats per instance type:
instance_idprojects/proj-id/locations/.../instances/nameinstance_idmemorystore-valkey(short name)cluster_idmemorystore-redis-cluster(short name)Fix
Extracted a
_resolve_inst_key()helper that detects short vs full GCP paths:projects/) → used as-is (already unique)project_id/(e.g.,my-project/memorystore-valkey)Updated
_accumulate_commands,_attach_memory_usage, and_attach_capacity_scalarto use the helper.Why fix in memorystore.py and not redis2re.py
redis2re.pyprocesses multiple integrations (ElastiCache, OSStats). Changing its grouping logic risks breaking them.memorystore.py— it's the only source producing non-unique ClusterIds.Verified end-to-end pipeline
memorystore.pyoutputproject-id/cluster-namefor Valkey/Clusterredis2re.pygroupingmetric_key = (clusterId, region)— now unique across projectsredis2re.pyget_db_name/via regex — Salesforce name still validmsstats.pyTesting
_attach_memory_usageand_attach_capacity_scalarintegrationredislabs-sales-pivotalandredislabs-universityblackformatting clean