Skip to content

fix: prefix ClusterId with project_id to prevent collisions across projects (GTI-685)#24

Merged
ymendez-redis merged 3 commits intomainfrom
gti-608-fix-clusterid-collision
Apr 24, 2026
Merged

fix: prefix ClusterId with project_id to prevent collisions across projects (GTI-685)#24
ymendez-redis merged 3 commits intomainfrom
gti-608-fix-clusterid-collision

Conversation

@ymendez-redis
Copy link
Copy Markdown
Collaborator

@ymendez-redis ymendez-redis commented Apr 23, 2026

Problem

GTI-685

When memorystore.py output from multiple GCP projects is combined, redis2re.py merges instances that share the same ClusterId and Region into one row — losing data and producing incorrect memory calculations.

This happens because Valkey and Redis Cluster instances use a short name as ClusterId (e.g., memorystore-valkey), which can be identical across projects. Redis standalone is unaffected because GCP returns a full resource path (projects/proj-id/locations/.../instances/name).

Real production example

Project ClusterId Region Result
project-019 cluster-011 europe-west2 Unique — different region
project-010 cluster-011 us-central1 Merged with project-004
project-006 cluster-011 europe-west1 Unique — different region
project-004 cluster-011 us-central1 Merged with project-010

10 input rows → 3 output rows instead of 4.

Root Cause

GCP returns different identifier formats per instance type:

Instance Type GCP Label Format Collision risk
Redis standalone instance_id projects/proj-id/locations/.../instances/name ✅ Globally unique
Valkey instance_id memorystore-valkey (short name) ❌ Collides across projects
Redis Cluster cluster_id memorystore-redis-cluster (short name) ❌ Collides across projects

Fix

Extracted a _resolve_inst_key() helper that detects short vs full GCP paths:

  • Full GCP paths (starting with projects/) → used as-is (already unique)
  • Short names → prefixed with project_id/ (e.g., my-project/memorystore-valkey)

Updated _accumulate_commands, _attach_memory_usage, and _attach_capacity_scalar to use the helper.

Why fix in memorystore.py and not redis2re.py

  • redis2re.py processes multiple integrations (ElastiCache, OSStats). Changing its grouping logic risks breaking them.
  • The problem originates in memorystore.py — it's the only source producing non-unique ClusterIds.
  • ElastiCache and OSStats are unaffected (their IDs are already unique).

Verified end-to-end pipeline

Stage Impact
memorystore.py output ClusterId becomes project-id/cluster-name for Valkey/Cluster
Conversion to xlsx Passes through as-is
redis2re.py grouping metric_key = (clusterId, region) — now unique across projects
redis2re.py get_db_name Strips / via regex — Salesforce name still valid
msstats.py Not affected — only processes Redis standalone

Testing

  • 10 new unit tests covering: Redis full path, Valkey/Cluster short name prefixing, cross-project collision prevention, empty project_id, unknown fallback, _attach_memory_usage and _attach_capacity_scalar integration
  • Verified against live GCP instances in redislabs-sales-pivotal and redislabs-university
  • All 23 tests pass, black formatting clean

…vent collisions across projects

Valkey and Redis Cluster instances use short names as ClusterId (e.g.,
'memorystore-valkey') which can collide when output from multiple GCP
projects is combined. Redis standalone is unaffected as it already uses
a full GCP resource path.

- Extract _resolve_inst_key() helper that detects short vs full GCP paths
- Short names get prefixed with project_id (e.g., 'my-project/memorystore-valkey')
- Full GCP paths (projects/...) pass through unchanged
- Update _attach_memory_usage and _attach_capacity_scalar to use the helper
- Add 10 unit tests covering all scenarios
@ymendez-redis ymendez-redis changed the title fix: prefix ClusterId with project_id to prevent collisions across projects (GTI-608) fix: prefix ClusterId with project_id to prevent collisions across projects (GTI-685) Apr 24, 2026
@ymendez-redis ymendez-redis merged commit a2d0b33 into main Apr 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants