Pre-delete dependency guard for DROP TABLE

[jogrogan]

Problem

DROP TABLE on a Kafka topic / Venice store / MySQL table currently succeeds even when an active Pipeline still reads from or writes to the resource. The downstream pipeline gets silently orphaned, it keeps trying to consume a topic that no
longer exists.

LogicalTableDeployer.delete() was unimplemented (threw SQLFeatureNotSupported) for the same reason: there was no safe way to verify a logical table's tier resources weren't still in use.

Approach

A pre-delete dependency check that runs in the SQL DDL path before any deployer-level state change:

1. At MV creation or logical table creation, every underlying Pipeline CRD is stamped with depends-on-<slug> labels — one per source and one per sink — and a collision-guard annotation listing the same identifiers verbatim.
2. At DROP time, ValidationService.validateOrThrow(new PendingDelete<>(source), connection) runs an indexed label-selector query against Pipeline CRDs. Any match (after self-uid filtering and annotation cross-check) becomes a validation error and fails the DROP.

The mechanism is a thin layer on Hoptimator's existing Validator/ValidatorProvider/ValidationService framework, extended to thread a Connection so validators can do external lookups.

What's in the diff (3 commits)

feat: pre-delete dependency guard for DROP TABLE

The core feature, end-to-end:

Layer	Change
Validator framework	Validated.validate(Issues, Connection), ValidatorProvider.validators(T, Connection), ValidationService.validateOrThrow(T, Connection) and validateOrThrow(Collection<T>, Connection). Connection-aware on a single, breaking signature (similar to how deployers take connection)
Intent signal	New PendingDelete<T> wrapper in hoptimator-api. Carries an optional selfOwnerUid so cascade-deleted children can be excluded from the dependent set.
K8s indexed lookup	PipelineDependencyLabels (slug + identifier + label/annotation builders), PipelineDependencyChecker (label-selector query + annotation collision guard + self-owner filter).
Stamping	K8sPipelineDeployer.toK8sObject() stamps labels and annotation; K8sPipelineBundle and K8sMaterializedViewDeployer thread sources/sink through.
Dispatch	K8sValidatorProvider returns a K8sPipelineDependencyValidator for PendingDelete<Source>; registered via META-INF/services/com.linkedin.hoptimator.ValidatorProvider.
Wiring	HoptimatorDdlExecutor.execute(SqlDropObject) calls validateOrThrow(new PendingDelete<>(source), connection) before DeploymentService.delete in the table branch.

feat: support DROP TABLE for logical tables

LogicalTableDeployer.delete() is now implemented as N-tier-DROPs + 1-CRD-delete:

1. Per-tier pre-flight: validateOrThrow(new PendingDelete<>(tierSource, logicalTableUid), connection). Active external pipelines block. The selfOwnerUid (the LogicalTable CRD's UID) ensures the implicit inter-tier pipelines don't self-block.
2. Delete the LogicalTable CRD — owner-ref cascade removes its owned Pipeline and TableTrigger CRDs.
3. Best-effort per-tier physical cleanup (Kafka topic / Venice store). Failures log and continue; a stranded tier resource is recoverable, aborting mid-DROP isn't.
4. Per-tier schema cleanup: deregister the TemporaryTable only when its physical delete succeeded.

test: integration scenarios + cleanup test warnings

• kafka-ddl-create-table.id — cross-driver scenarios exercising the new check (DROP-while-pipeline-depends-on-it, source-side and partial-view-sink-side).
• The bulk of the file count is unrelated test-warning cleanup that was queued up against this branch — unused imports, tightened generics. No semantic change.

Test plan

• ./gradlew test — all unit tests pass.
• kafka-ddl-create-table.id — cross-driver DROP scenarios exercised end-to-end against a fixture-loaded JDBC topic.
• logical-ddl.id — DROP TABLE on a logical table now succeeds; cascading nearline-to-online pipeline removal verified.
• Integration tests against a live K8s environment.
• Manual: try DROP-ing a table while an MV depends on it — confirm the new error message names the dependent pipeline.

Known caveats

• No backfill for pre-existing pipelines. The label-selector query only matches Pipelines that have the depends-on-* labels stamped on them, which only happens at create time post-feature. Pipelines deployed before this PR ships are invisible
  to the check and won't block a DROP. To make this a hard correctness invariant, an upgrade-time backfill job needs to re-stamp labels on every existing Pipeline CRD by re-deriving sources/sink from its spec.sql. Out of scope here.
• MV-on-MV is not specifically guarded. Calcite inlines view sources at plan time, so a Pipeline backing ads.audience2 AS SELECT * FROM ads.audience ends up labeled with the leaf tables (ads.page_views, profile.members), not ads.audience.
  Operationally that's fine — audience2's Flink job never references audience at runtime, so dropping audience doesn't orphan anything. The dangling SQL on audience2's View CRD is self-healing on next CREATE OR REPLACE.

[github-actions]

Code Coverage

Overall Project	84.51% -0.16%	🟢
Files changed	93.86%	🟢

File	Coverage
K8sValidatorProvider.java	100%	🟢
K8sYamlApi.java	100%	🟢
K8sPipelineDependencyValidator.java	100%	🟢
AvroValidatorProvider.java	100%	🟢
Validator.java	100%	🟢
PendingDelete.java	100%	🟢
ValidationService.java	100%	🟢
DefaultValidatorProvider.java	100%	🟢
CompatibilityValidatorProvider.java	100%	🟢
K8sApi.java	99.64%	🟢
K8sPipelineDeployer.java	97.67% -2.33%	🟢
KafkaDeployer.java	97.04%	🟢
MySqlDeployer.java	96.84%	🟢
PipelineDependencyLabels.java	96.41% -3.59%	🟢
AvroTableValidator.java	96.35%	🟢
K8sMaterializedViewDeployer.java	95.85% -4.15%	🟢
K8sViewTable.java	95.42%	🟢
CompatibilityValidatorBase.java	92.96%	🟢
HoptimatorDdlExecutor.java	92.83%	🟢
PipelineDependencyChecker.java	90.96% -9.04%	🟢
K8sPipelineBundle.java	87.79% -7.63%	❌
HoptimatorDdlUtils.java	85.63% -0.14%	🟢
LogicalTableDeployer.java	76.67% -0.93%	🟢
VeniceDeployer.java	52.92%	🟢