Skip to content

Consolidate source + remove tests#31

Merged
fivetran-joemarkiewicz merged 11 commits intomainfrom
MagicBot/consolidate-source-package
Aug 18, 2025
Merged

Consolidate source + remove tests#31
fivetran-joemarkiewicz merged 11 commits intomainfrom
MagicBot/consolidate-source-package

Conversation

@fivetran-data-model-bot
Copy link
Contributor

@fivetran-data-model-bot fivetran-data-model-bot commented Aug 13, 2025

Summary

This PR is to combine the source and transform packages and remove tests not compatible with dbt Fusion 1.10.6+.

Please check the below automated updates were successful:

MANUAL UPDATES

  • If applicable, merge in the Q2 FY26 Automatic Package Updates PR.
  • If there are 2 docs.md files, consolidate the definitions the file in the main folder and remove the 2nd file.
  • Scan the README for non-standard references to the source package and remove or update.
  • Copy the passthrough vars or other non-standard vars from the source dbt_project.yml to the transform dbt_project.yml.
  • Check BK run logs for at least 2 warehouses
    • No further deprecation warnings
    • No union data "table not found" type warnings
  • Make sure below automatic updates were applied correctly.

Source + Transform Consolidation

  • Moved models/macros to models/staging/ and macros/staging/.
  • Updated all var('') references within models to use ref('stg_zuora__').
  • Updated vars in dbt_project.yml from refs to sources.
  • Removed the source package from packages.yml.
  • Major version bump to v1.0.0 (or higher if applicable)

dbt Fusion Compatibility

  • Removed dbt_utils.unique_combination_of_columns test
  • Accepted values tests were removed where applicable (Shopify, NetSuite, QuickBooks).

YAML & Config Updates

  • Moved loaded_at_field under config: in src.yml, grouped with freshness
  • Added +schema and +materialization for the staging models in dbt_project.yml.

README

  • Removed references to the source package in:
    • Overview
    • Installation instructions
    • Dependencies list
    • Schema override examples

@fivetran-data-model-bot fivetran-data-model-bot added the next-release Include in the next release. label Aug 13, 2025
* Q2 FY26: Apply automated update.

* Q2 FY26: Update auto-release workflow only.

* Update CHANGELOG.md

---------

Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
@fivetran-joemarkiewicz fivetran-joemarkiewicz added the docs:ready Triggers the docs generator workflow. label Aug 13, 2025
Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're in a time crunch, approved after addressing the comments below, plus:

We should add this freshness line to run_models.sh after the dbt seed command (example link)

dbt source freshness --target "$db" || echo "...Only verifying freshness runs…

@@ -160,17 +162,15 @@ By default this, package will build the Zuora staging models within a schema tit

Copy link
Contributor

@fivetran-avinash fivetran-avinash Aug 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't apply a direct suggestion to the above line:

"By default this package will build the Zuora staging models within a schema titled (<target_schema> + _zuora_source), the Zuora intermediate models within a schema titled (<target_schema> + _zuora_int), and the Zuora final models within a schema titled (<target_schema> + _zuora) in your target database. If this is not where you would like your modeled Zuora data to be written to, add the following configuration to your dbt_project.yml file:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update applied

fivetran-joemarkiewicz and others added 3 commits August 15, 2025 09:53
Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>
@fivetran-joemarkiewicz fivetran-joemarkiewicz added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 15, 2025
@fivetran-joemarkiewicz fivetran-joemarkiewicz merged commit f0f17b4 into main Aug 18, 2025
9 checks passed
@fivetran-joemarkiewicz fivetran-joemarkiewicz deleted the MagicBot/consolidate-source-package branch August 18, 2025 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs:ready Triggers the docs generator workflow. next-release Include in the next release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Deprecation] Update package for dbt Fusion and dbt Core 1.10.6 compatibility

4 participants