Skip to content

Add topic join deps and fix Omni asset grouping for full lineage#186

Merged
vbalalian merged 2 commits intomainfrom
fix/omni-lineage-connection
Mar 11, 2026
Merged

Add topic join deps and fix Omni asset grouping for full lineage#186
vbalalian merged 2 commits intomainfrom
fix/omni-lineage-connection

Conversation

@vbalalian
Copy link
Copy Markdown
Owner

@vbalalian vbalalian commented Mar 11, 2026

Summary

  • Parse Omni topic YAML files (omni/BigQuery/*.topic.yaml) to discover join dependencies that the Omni API doesn't expose, adding them to workbook asset specs so the full lineage graph connects correctly (e.g. Product Category Performance now shows deps on dim_products, dim_users, dim_user_rfm in addition to fct_events)
  • Replace group_name: omni with tags: { layer: bi } in the Omni component translation config so workbook assets flow downstream from dbt models instead of appearing as isolated nodes in a separate group
  • Update tests to cover topic YAML parsing and recursive join extraction

Test plan

  • All 8 tests pass in dagster-project/tests/test_definitions.py
  • dg list defs shows correct deps for all three workbooks after dg utils refresh-defs-state
  • Verify in Dagster UI that Omni workbook assets flow downstream from dbt model assets
  • Verify dim_products appears in Product Category Performance workbook lineage

@vbalalian vbalalian merged commit 20d4bb8 into main Mar 11, 2026
3 checks passed
@vbalalian vbalalian deleted the fix/omni-lineage-connection branch March 11, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant