From 35dfcbc83eab7561f186d097f91cb437524307fa Mon Sep 17 00:00:00 2001
From: Joe Markiewicz
<74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Date: Wed, 16 Jul 2025 10:58:10 -0500
Subject: [PATCH 01/10] feature/text-column-addition
---
.../maintainer_pull_request_template.md | 52 +++++++++++--------
.github/workflows/generate-docs.yml | 13 +++++
CHANGELOG.md | 14 +++--
LICENSE | 2 +-
README.md | 7 +--
dbt_project.yml | 2 +-
integration_tests/dbt_project.yml | 2 +-
integration_tests/seeds/question_response.csv | 12 ++---
macros/get_question_response_columns.sql | 1 +
models/docs.md | 4 ++
models/src_qualtrics.yml | 2 +
models/stg_qualtrics.yml | 2 +
models/stg_qualtrics__question_response.sql | 1 +
13 files changed, 76 insertions(+), 38 deletions(-)
create mode 100644 .github/workflows/generate-docs.yml
diff --git a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
index 3220674..99c450c 100644
--- a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
+++ b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
@@ -1,29 +1,35 @@
-## PR Overview
-**This PR will address the following Issue/Feature:**
+
+- `dbt run --full-refresh && dbt test`
+- `dbt run` && `dbt test` (if incremental models are present)
+- The related issue is linked, tagged, and appropriately assigned
+- Documentation and version updates are included, if applicable
+- `docs` have been regenerated (unless there are no code or YAML changes)
+- BuildKite integration tests are passing
+-->
-**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:**
-
+## PR Overview
+**Package version introduced in this PR:**
+
+**This PR addresses the following Issue/Feature(s):**
+
-## PR Checklist
-### Basic Validation
-Please acknowledge that you have successfully performed the following commands locally:
-- [ ] dbt run –full-refresh && dbt test
-- [ ] dbt run (if incremental models are present) && dbt test
+**Summary of changes:**
+
-Before marking this PR as "ready for review" the following have been applied:
-- [ ] The appropriate issue has been linked, tagged, and properly assigned
-- [ ] All necessary documentation and version upgrades have been applied
-- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
-- [ ] BuildKite integration tests are passing
-- [ ] Detailed validation steps have been provided below
-### Detailed Validation
-Please share any and all of your validation steps:
-
+### Submission Checklist
+- [ ] Alignment meeting with the reviewer (if needed)
+ - [ ] Timeline and validation requirements discussed
+- [ ] Provide validation details:
+ - [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests)
+ - [ ] **Testing Instructions:** Confirm the change addresses the issue(s)
+ - [ ] **Focus Areas:** Complex logic or queries that need extra attention
-### If you had to summarize this PR in an emoji, which would it be?
-
-:dancer:
\ No newline at end of file
+### Changelog
+
+- [ ] Draft changelog for PR
+- [ ] Final changelog for release review
\ No newline at end of file
diff --git a/.github/workflows/generate-docs.yml b/.github/workflows/generate-docs.yml
new file mode 100644
index 0000000..c9f92f7
--- /dev/null
+++ b/.github/workflows/generate-docs.yml
@@ -0,0 +1,13 @@
+name: 'generate dbt docs'
+on:
+ pull_request:
+ types:
+ - labeled
+
+jobs:
+ generate-docs:
+ if: github.event.label.name == 'docs:ready'
+ uses: fivetran/dbt_package_automations/.github/workflows/generate-docs.yml@main
+ secrets: inherit
+ with:
+ schema_var_name: qualtrics_schema
\ No newline at end of file
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fe40538..2e61feb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,7 +1,15 @@
-# dbt_qualtrics_source version.version
+# dbt_qualtrics_source v0.4.0
-## Documentation
-- Corrected references to connectors and connections in the README. ([#14](https://github.com/fivetran/dbt_qualtrics_source/pull/14))
+## Schema & Data Updates
+**1 new column introduced and 1 schema change**
+
+| Data Model | Change Type | Old Name | New Name | Notes |
+| ---------- | ----------- | -------- | -------- | ----- |
+| `stg_qualtrics__question_response` | New column | | `response_text` | Captures the free text response associated with the question. |
+
+## Under the Hood
+- Introduced the generate-docs github workflow for consistent docs generation.
+- Included an updated version of the maintainer pull request template.
# dbt_qualtrics_source v0.3.0
[PR #12](https://github.com/fivetran/dbt_qualtrics_source/pull/12) includes the following update:
diff --git a/LICENSE b/LICENSE
index 261eeb9..f9554a9 100644
--- a/LICENSE
+++ b/LICENSE
@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
- Copyright [yyyy] [name of copyright owner]
+ Copyright © 2025 Fivetran Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
diff --git a/README.md b/README.md
index 57d4b4c..306364f 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,6 @@
-
+# Qualtrics Source dbt Package ([Docs](https://fivetran.github.io/dbt_qualtrics_source/))
+
+
@@ -10,7 +12,6 @@
-# Qualtrics Source dbt Package ([Docs](https://fivetran.github.io/dbt_qualtrics_source/))
## What does this dbt package do?
- Materializes [Qualtrics staging tables](https://fivetran.github.io/dbt_qualtrics_source/#!/overview/github_source/models/?g_v=1) which leverage data in the format described by [this ERD](https://fivetran.com/docs/applications/qualtrics/#schemainformation). These staging tables clean, test, and prepare your Qualtrics data from [Fivetran's connector](https://fivetran.com/docs/applications/qualtrics) for analysis by doing the following:
@@ -45,7 +46,7 @@ If you are **not** using the [Qualtrics transformation package](https://github.
```yml
packages:
- package: fivetran/qualtrics_source
- version: [">=0.3.0", "<0.4.0"] # we recommend using ranges to capture non-breaking changes automatically
+ version: [">=0.4.0", "<0.5.0"] # we recommend using ranges to capture non-breaking changes automatically
```
### Step 3: Define database and schema variables
diff --git a/dbt_project.yml b/dbt_project.yml
index 6480038..fc7932a 100644
--- a/dbt_project.yml
+++ b/dbt_project.yml
@@ -1,5 +1,5 @@
name: 'qualtrics_source'
-version: '0.3.0'
+version: '0.4.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml
index 9d68da7..5ff3e44 100644
--- a/integration_tests/dbt_project.yml
+++ b/integration_tests/dbt_project.yml
@@ -1,5 +1,5 @@
name: 'qualtrics_source_integration_tests'
-version: '0.3.0'
+version: '0.4.0'
profile: 'integration_tests'
config-version: 2
diff --git a/integration_tests/seeds/question_response.csv b/integration_tests/seeds/question_response.csv
index 28eb338..fc3b622 100644
--- a/integration_tests/seeds/question_response.csv
+++ b/integration_tests/seeds/question_response.csv
@@ -1,6 +1,6 @@
-_fivetran_id,_fivetran_synced,loop_id,question,question_id,question_option_key,response_id,sub_question_key,sub_question_text,value
-vnzZKokL2ijFL+3cv266gKI+R3U=,2023-05-02 00:18:29.039000,,Drink,QID2,,R_2w6q9pgBpebvjCg,,,2
-ZOmuYJzFNtYua4Iwbtk474SXBsk=,2023-05-02 00:18:29.040000,,Drink,QID2,,R_3RvEQNuWciDhUpl,,,2
-KeKECHYR7V5729skoyowWs4Ah4o=,2023-05-02 00:18:29.041000,,Drink,QID2,,R_2SqSR1Q6pLJZplH,,,1
-j2kXb68qeBgQ9Tdsj/nyiypvEZ0=,2023-05-02 00:18:29.042000,,Drink,QID2,,R_2xyo8lOlBsJkqNT,,,3
-tm3O8dMY4osydnC+ZJuS+O5u8QI=,2023-05-02 00:25:43.221000,,state,QID58,,R_1Qnonc63Wkk2Fur,,,
\ No newline at end of file
+_fivetran_id,_fivetran_synced,loop_id,question,question_id,question_option_key,response_id,sub_question_key,sub_question_text,text,value
+vnzZKokL2ijFL+3cv266gKI+R3U=,2023-05-02 00:18:29.039000,,Drink,QID2,,R_2w6q9pgBpebvjCg,,,example a,2
+ZOmuYJzFNtYua4Iwbtk474SXBsk=,2023-05-02 00:18:29.040000,,Drink,QID2,,R_3RvEQNuWciDhUpl,,,example b,2
+KeKECHYR7V5729skoyowWs4Ah4o=,2023-05-02 00:18:29.041000,,Drink,QID2,,R_2SqSR1Q6pLJZplH,,,example c,1
+j2kXb68qeBgQ9Tdsj/nyiypvEZ0=,2023-05-02 00:18:29.042000,,Drink,QID2,,R_2xyo8lOlBsJkqNT,,,example d,3
+tm3O8dMY4osydnC+ZJuS+O5u8QI=,2023-05-02 00:25:43.221000,,state,QID58,,R_1Qnonc63Wkk2Fur,,,example e,
diff --git a/macros/get_question_response_columns.sql b/macros/get_question_response_columns.sql
index cd5410d..3ead061 100644
--- a/macros/get_question_response_columns.sql
+++ b/macros/get_question_response_columns.sql
@@ -10,6 +10,7 @@
{"name": "response_id", "datatype": dbt.type_string()},
{"name": "sub_question_key", "datatype": dbt.type_string()},
{"name": "sub_question_text", "datatype": dbt.type_string()},
+ {"name": "text", "datatype": dbt.type_string()},
{"name": "value", "datatype": dbt.type_string()}
] %}
diff --git a/models/docs.md b/models/docs.md
index 45d2a9b..f279001 100644
--- a/models/docs.md
+++ b/models/docs.md
@@ -360,6 +360,10 @@ The key of the `QUESTION_OPTION` that was chosen for the sub-question.
Sub question text.
{% enddocs %}
+{% docs response_text %}
+Text response provided by the respondent.
+{% enddocs %}
+
{% docs response_value %}
Value of the question response.
{% enddocs %}
diff --git a/models/src_qualtrics.yml b/models/src_qualtrics.yml
index 45b5887..1d3f43f 100644
--- a/models/src_qualtrics.yml
+++ b/models/src_qualtrics.yml
@@ -176,6 +176,8 @@ sources:
description: "{{ doc('sub_question_key') }}"
- name: sub_question_text
description: "{{ doc('sub_question_text') }}"
+ - name: response_text
+ description: "{{ doc('response_text') }}"
- name: value
description: "{{ doc('response_value') }}"
diff --git a/models/stg_qualtrics.yml b/models/stg_qualtrics.yml
index 81d8cb5..781503e 100644
--- a/models/stg_qualtrics.yml
+++ b/models/stg_qualtrics.yml
@@ -199,6 +199,8 @@ models:
description: "{{ doc('sub_question_key') }}"
- name: sub_question_text
description: "{{ doc('sub_question_text') }}"
+ - name: text
+ description: "{{ doc('response_text') }}"
- name: value
description: "{{ doc('response_value') }}"
- name: source_relation
diff --git a/models/stg_qualtrics__question_response.sql b/models/stg_qualtrics__question_response.sql
index 050839b..d989ab8 100644
--- a/models/stg_qualtrics__question_response.sql
+++ b/models/stg_qualtrics__question_response.sql
@@ -34,6 +34,7 @@ final as (
response_id,
sub_question_key,
sub_question_text,
+ text as response_text,
value,
_fivetran_synced,
source_relation
From 6498268a5062e15fa23a4e9ab82fe09b7ed83960 Mon Sep 17 00:00:00 2001
From: Joe Markiewicz
<74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Date: Wed, 16 Jul 2025 11:00:36 -0500
Subject: [PATCH 02/10] documenation updates
---
CHANGELOG.md | 2 ++
models/src_qualtrics.yml | 2 +-
models/stg_qualtrics.yml | 2 +-
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2e61feb..c3f9707 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,7 @@
# dbt_qualtrics_source v0.4.0
+[PR #16](https://github.com/fivetran/dbt_qualtrics_source/pull/16) includes the following updates:
+
## Schema & Data Updates
**1 new column introduced and 1 schema change**
diff --git a/models/src_qualtrics.yml b/models/src_qualtrics.yml
index 1d3f43f..3f7bcfc 100644
--- a/models/src_qualtrics.yml
+++ b/models/src_qualtrics.yml
@@ -176,7 +176,7 @@ sources:
description: "{{ doc('sub_question_key') }}"
- name: sub_question_text
description: "{{ doc('sub_question_text') }}"
- - name: response_text
+ - name: text
description: "{{ doc('response_text') }}"
- name: value
description: "{{ doc('response_value') }}"
diff --git a/models/stg_qualtrics.yml b/models/stg_qualtrics.yml
index 781503e..ca8a191 100644
--- a/models/stg_qualtrics.yml
+++ b/models/stg_qualtrics.yml
@@ -199,7 +199,7 @@ models:
description: "{{ doc('sub_question_key') }}"
- name: sub_question_text
description: "{{ doc('sub_question_text') }}"
- - name: text
+ - name: response_text
description: "{{ doc('response_text') }}"
- name: value
description: "{{ doc('response_value') }}"
From 75436176cf67745534daba5d46ebdcab246343e7 Mon Sep 17 00:00:00 2001
From: Joe Markiewicz
<74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Date: Wed, 16 Jul 2025 12:45:48 -0500
Subject: [PATCH 03/10] requirements update
---
integration_tests/requirements.txt | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/integration_tests/requirements.txt b/integration_tests/requirements.txt
index 87ae0f9..04b5ebf 100644
--- a/integration_tests/requirements.txt
+++ b/integration_tests/requirements.txt
@@ -4,4 +4,5 @@ dbt-redshift>=1.3.0,<2.0.0
dbt-postgres>=1.3.0,<2.0.0
dbt-spark>=1.3.0,<2.0.0
dbt-spark[PyHive]>=1.3.0,<2.0.0
-dbt-databricks>=1.3.0,<2.0.0
\ No newline at end of file
+dbt-databricks>=1.3.0,<2.0.0
+certifi==2025.1.31
\ No newline at end of file
From 9feb219d7d61b2113e36c5b90af3d8c59e78f62f Mon Sep 17 00:00:00 2001
From: Fivetran Maintainer Bot
<129092423+fivetran-data-model-bot@users.noreply.github.com>
Date: Wed, 16 Jul 2025 14:58:19 -0500
Subject: [PATCH 04/10] Q2 FY26 Automatic Package Updates (#17)
* Q2 FY26: Apply automated update.
* Update CHANGELOG.md
---------
Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
---
.buildkite/pipeline.yml | 10 +--
.../maintainer_pull_request_template.md | 7 +-
.github/workflows/auto-release.yml | 13 ++--
.gitignore | 75 +++++++++++++++++--
CHANGELOG.md | 13 +++-
README.md | 14 ++--
integration_tests/ci/sample.profiles.yml | 6 +-
integration_tests/dbt_project.yml | 6 ++
integration_tests/requirements.txt | 2 +-
9 files changed, 112 insertions(+), 34 deletions(-)
diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
index 281f1d4..2603602 100644
--- a/.buildkite/pipeline.yml
+++ b/.buildkite/pipeline.yml
@@ -3,7 +3,7 @@ steps:
key: "run-dbt-postgres"
plugins:
- docker#v3.13.0:
- image: "python:3.8"
+ image: "python:3.10.13"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
@@ -18,7 +18,7 @@ steps:
key: "run_dbt_snowflake"
plugins:
- docker#v3.13.0:
- image: "python:3.8"
+ image: "python:3.10.13"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
@@ -35,7 +35,7 @@ steps:
key: "run_dbt_bigquery"
plugins:
- docker#v3.13.0:
- image: "python:3.8"
+ image: "python:3.10.13"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
@@ -47,7 +47,7 @@ steps:
key: "run_dbt_redshift"
plugins:
- docker#v3.13.0:
- image: "python:3.8"
+ image: "python:3.10.13"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
@@ -62,7 +62,7 @@ steps:
key: "run_dbt_databricks"
plugins:
- docker#v3.13.0:
- image: "python:3.8"
+ image: "python:3.10.13"
shell: [ "/bin/bash", "-e", "-c" ]
environment:
- "BASH_ENV=/tmp/.bashrc"
diff --git a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
index 99c450c..9b72163 100644
--- a/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
+++ b/.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md
@@ -12,13 +12,15 @@ Before marking this PR as "ready for review":
## PR Overview
**Package version introduced in this PR:**
-
+-
+
**This PR addresses the following Issue/Feature(s):**
+-
**Summary of changes:**
-
+-
### Submission Checklist
- [ ] Alignment meeting with the reviewer (if needed)
@@ -27,6 +29,7 @@ Before marking this PR as "ready for review":
- [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests)
- [ ] **Testing Instructions:** Confirm the change addresses the issue(s)
- [ ] **Focus Areas:** Complex logic or queries that need extra attention
+- [ ] Merge any relevant open PRs into this PR
### Changelog