Skip to content

feauture/union-data#34

Merged
fivetran-catfritz merged 8 commits intomainfrom
feauture/union-data
Nov 11, 2025
Merged

feauture/union-data#34
fivetran-catfritz merged 8 commits intomainfrom
feauture/union-data

Conversation

@fivetran-catfritz
Copy link
Contributor

@fivetran-catfritz fivetran-catfritz commented Oct 31, 2025

PR Overview

Package version introduced in this PR:

  • v1.1.0

This PR addresses the following Issue/Feature(s):

  • GA-1007471

Summary of changes:

  • Add union data feature.

Submission Checklist

  • Alignment meeting with the reviewer (if needed)
    • Timeline and validation requirements discussed
  • Provide validation details:
    • Validation Steps: Check for unintentional effects (e.g., add/run consistency & integrity tests)
    • Testing Instructions: Confirm the change addresses the issue(s)
    • Focus Areas: Complex logic or queries that need extra attention
  • Merge any relevant open PRs into this PR

Changelog

  • Draft changelog for PR
  • Final changelog for release review

@fivetran-catfritz fivetran-catfritz self-assigned this Oct 31, 2025
Copy link
Contributor

@fivetran-savage fivetran-savage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of questions and a few places in the yml where you need to add the source_relation column.

created_by_id,
cast(created_date as {{ dbt.type_timestamp() }}) as created_date,
current_term,
cast(current_term as {{ dbt.type_int() }}) as current_term,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were some of the _term fields coming through as different data types?

Copy link
Contributor Author

@fivetran-catfritz fivetran-catfritz Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was testing the missing table capability of union data, and noticed that even though the fields CTE includes explicit casting, the warehouse sometimes still assigns its own datatype when a column is entirely null. This casting was necessary to prevent downstream errors in these lines of zuora__subscription_overview, where multiplications can fail if the warehouse interprets a null column as a string.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I saw this issue in recharge as well and did some extra casting.

status,
subscription_end_date,
subscription_start_date,
cast(subscription_end_date as {{ dbt.type_timestamp() }}) as subscription_end_date,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this field previously a date or has it always been a timestamp?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above but for these lines.

subscription_end_date,
subscription_start_date,
cast(subscription_end_date as {{ dbt.type_timestamp() }}) as subscription_end_date,
cast(subscription_start_date as {{ dbt.type_timestamp() }}) as subscription_start_date,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as end_date.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above but for these lines.

@@ -261,7 +263,6 @@ models:
- name: invoice_item_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add source_relation column name and description.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have updated.

@@ -408,7 +409,6 @@ models:
- name: account_monthly_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add source_relation column name and description.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have updated.

models/zuora.yml Outdated
@@ -450,10 +450,9 @@ models:
- name: zuora__subscription_overview
description: Each record is a subscription, enhanced with additional details from rate plan charges and amendments.
columns:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add source_relation column name and description.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have updated.

fivetran-catfritz and others added 2 commits November 10, 2025 11:04
Co-authored-by: Savage Fivetran <sarah.savage@fivetran.com>
@fivetran-catfritz fivetran-catfritz added the docs:ready Triggers the docs generator workflow. label Nov 10, 2025
@fivetran-catfritz fivetran-catfritz merged commit c5cb5e4 into main Nov 11, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs:ready Triggers the docs generator workflow.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants