Draft
Conversation
Phase 1 of the V3-native engine: a composition root with a single child, a MetadataLoader bound at construction time. PeriodicMetadataLoader mirrors Graph.startMetastoreReload but the reload body is empty until phase 2 fills it in. Engine.create() is the canonical entry point — Spring config, CLI, and tests all use it, and the engine.runtime package has zero Spring import and zero v2 import. The Engine class itself only delegates: new responsibilities arrive as new children rather than as new methods on Engine. This is the structural defense against the Graph.kt god-class pattern. Highlights: - MetadataLoader is a small interface (bind + AutoCloseable). The bind contract is documented: called once by Engine's ctor, must not call back into the engine synchronously, idempotent close. - PeriodicMetadataLoader runs a single Reactor pipeline. Periodic mode uses Flux.interval(initialDelay, interval); disabled mode uses Mono.delay(initialDelay).flux() for a one-shot. Both branches share the same operators, error handler, and disposable so the engine gets exactly one initial reload regardless of configuration. - bind / close are synchronized; the bound flag prevents double-binding and unbound-close noise. - ServerProperties.MetastoreProperties groups the two new settings under actionbase.metastore.*; application.yaml mirrors the existing kc.graph.metastoreReloadInterval value and notes the two configs run side by side until phase 2 completes the cutover. - Tests cover construction binding, lifecycle delegation, periodic scheduling, bind-triggers-exactly-one-reload, the disabled-but-still- initial path, and that the configured initial delay actually defers the first reload.
dosubot
Bot
added
size:L
- Drop KDoc blocks from Engine, MetadataLoader, and PeriodicMetadataLoader. - Rename Engine's ctor parameter from `loader` to `metadataLoader` for symmetry with the property in `create`. - Reformat `Engine.create` as a block body with a named local for clarity.
The engine runtime tracks Database, Table, and Alias definitions — exactly what the industry calls a catalog (Spark/Trino/Iceberg/Glue all use this term). Metadata was too broad; Catalog names what we actually load. Renames: - MetadataLoader -> CatalogLoader - PeriodicMetadataLoader -> PeriodicCatalogLoader - Engine.metadataLoader -> catalogLoader - metastoreReloadInitialDelay/Interval -> catalogReloadInitialDelay/Interval - ServerProperties.metastore -> catalog (CatalogProperties) - actionbase.metastore.* -> actionbase.catalog.* - log strings: 'reloading metastore' -> 'reloading catalog' etc. Metastore (the JDBC backing store) keeps its name; the catalog is what lives in memory and is loaded *from* the metastore.
em3s
changed the title
- com.kakao.actionbase.engine.Engine (was engine.runtime.Engine).
Engine now sits next to QueryEngine, MutationEngine, and the
Actionbase stub it will eventually replace.
- com.kakao.actionbase.engine.catalog.{CatalogLoader, PeriodicCatalogLoader}
(was engine.runtime.*). Future catalog types (CatalogSnapshot,
TableId, push-based loaders) get a natural home.
- The empty engine.runtime package is removed.
The interface IS the catalog (the registry of databases / tables / aliases), matching how Spark, Iceberg, Trino, and Glue use the term. `bind` is part of the catalog's lifecycle, the same way Spark's ExternalCatalog has `init`/`close` next to its query methods. - CatalogLoader -> Catalog - PeriodicCatalogLoader -> PeriodicCatalog - log strings and field names follow the rename.
- Catalog interface exposes three maps directly (DatabaseDescriptor, TableDescriptor, AliasDescriptor) so callers can write catalog.tables[id] instead of catalog.snapshot.tables[id]. - PeriodicCatalog keeps a private nested Snapshot data class holding all three maps in one `@Volatile` reference, so reload swaps them atomically (a single reference write). Each getter delegates to the current snapshot, giving consistent per-call reads without putting the snapshot type on the public API. - EngineTest's FakeLoader implements the three maps directly (empty).
- Catalog: restore KDoc covering the bind/close contract, single- threaded lifecycle assumption, and the reader contract for databases/ tables/aliases (immutable or snapshot-at-read; single-getter consistency guaranteed, cross-map best-effort). - PeriodicCatalog: mark `engine` as @volatile so the null-check in reload() has a happens-before with close()'s write — the code now actually matches what the guard comment claims. - PeriodicCatalog: regroup fields into state / public views so readers see a single ordering.
The runtime object for a table was called TableBinding, which is an unusual term. The industry-standard pair for what we have is TableDescriptor (declarative spec) and Table (live runtime, cached) — see Spark, Iceberg, Trino, Glue. Renames (interface + concrete + test + callers): - engine.binding.TableBinding -> engine.binding.Table - v2.engine.v3.V2BackedTableBinding -> V2BackedTable - v2.engine.v3.NilTableBinding -> NilTable - V2BackedTableBindingTest -> V2BackedTableTest - QueryEngine.getTableBinding -> getTable - MutationEngine.getTableBinding -> getTable - HBaseIndexedLabel.tableBinding -> table The package `engine.binding` is kept as-is for now — moving it to `engine.table` or `engine.catalog` is a separate cleanup. Prepares for the Catalog to cache Table instances in phase 3.
The catalog snapshot now holds fully-resolved runtime objects instead of raw descriptors, matching Iceberg's Table / Catalog model. - Add engine.catalog.Database (thin wrapper over DatabaseDescriptor). - Add engine.catalog.Alias (AliasDescriptor + resolved Table). - Move engine.binding.Table -> engine.catalog.Table and add `val descriptor: TableDescriptor<*>` so Table carries both its spec and its runtime ops — the core of the user's 'Table = Descriptor + runtime' model. - V3TableDescriptor gains `toTableDescriptor(entity)` so the v2 adapters can build the full core.metadata.TableDescriptor from a LabelEntity without pulling in V3MetadataConverter. - V2BackedTable and NilTable implement the new `descriptor` field using the V3TableDescriptor conversion. NilTable now takes the LabelEntity too so it can build its descriptor. - Catalog interface exposes Database / Table / Alias maps; Snapshot in PeriodicCatalog swaps runtime types atomically via a single reference. Phase 2 will read the metastore through `engine` inside reload() and build fresh Database / Table / Alias instances, reusing existing Table instances when their descriptors haven't changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 of the V3-native engine. A new
Engineclass lives inengine.runtimewith zerov2.*and zero Spring imports.PeriodicCatalogLoadermirrorsGraph.startMetastoreReloadbut the reload body is empty until phase 2.Graphis unchanged. The new bean runs alongside it underactionbase.catalog.reload-interval.Refs #247
Changes
Engine— composition root,AutoCloseable,Engine.create(...)factory.CatalogLoader— small interface (bind+ idempotentclose). Catalog = the registry of Database / Table / Alias definitions, mirroring industry terms (Spark / Trino / Iceberg / Glue).PeriodicCatalogLoader— single Reactor pipeline.Flux.intervalfor periodic,Mono.delay(...).flux()for one-shot. Both share the same operators, error handler, and disposable.EngineConfiguration— single@BeancallingEngine.create(...). OnlyEngineis imported.ServerProperties.CatalogProperties— nested under the existingactionbaseprefix.application.yaml—actionbase.catalog.reload-interval: 1malongside the existingkc.graph.metastoreReloadInterval.How to Test
Boot the server and look for:
Starting Flux.interval for reloading catalog every 60000 ms after 0 ms delay.