Skip to content

Conversation

@bkirwi
Copy link
Contributor

@bkirwi bkirwi commented Nov 27, 2025

Traditionally, different types of objects may end up with what appears to be the same ID: so the index u1 may be created on replica u1 of cluster u1. This is I think generally considered to be confusing.

This change switches to assigning IDs that are unique across different types - so we will avoid creating a new cluster u2 if an object u2 already exists, and vice-versa. This is done by giving ourselves a way to assign a new id that will not conflict with multiple different types, and adopting that incrementally.

This does not guarantee that there are no ID collisions across types, and in particular no legacy ID will change... but we do avoid conflicts in many cases, as the test changes show.

Motivation

First proposed in https://github.com/MaterializeInc/database-issues/issues/6336#issuecomment-2369825541.

I don't know if it has consensus as the right end state, but hopefully it is uncontroversial as an incremental step?

Tips for reviewer

One upshot of this is that IDs of things will change, including for some system things. I did try and ensure that the system cluster IDs wouldn't change, since I suspect we have those hardcoded in some alerts and other things, but as you can see from the test rewrites there are lots of IDs that will shift around by default. I think this is probably fine, but if you disagree, I am interested to hear about it!

I updated the regular CI tests, but there are probably some nightly failures that will crop up. I'll wait to resolve those until I think this is likely to be approved!

There are some other ID types that I haven't merged into the merged lists yet. I would prefer to save that for followup work, again assuming folks think this is a positive path.

An alternative to all this would be a catalog migration that collapses down all the different keys. This version is both more incremental and easier to reverse, but eventually something like that seems like it would make sense!

@bkirwi bkirwi force-pushed the mzid branch 5 times, most recently from ba4fcd3 to 8f8cc56 Compare November 27, 2025 22:18
@bkirwi bkirwi force-pushed the mzid branch 6 times, most recently from 0319cff to 5db9fdf Compare November 28, 2025 20:51
@bkirwi bkirwi marked this pull request as ready for review November 28, 2025 22:34
@bkirwi bkirwi requested a review from a team as a code owner November 28, 2025 22:34
@bkirwi bkirwi requested a review from SangJunBak November 28, 2025 22:34
@bkirwi bkirwi changed the title [adapter] Share allocators for all system / user id types [skunkwork] [adapter] Share allocators for all system / user id types Nov 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant