BI-8806-create-gh-workflow-for-testing-on-dbt-improvado-utils#48
Merged
Solovechik merged 11 commits intomasterfrom Jul 24, 2025
Merged
Conversation
Contributor
Solovechik
commented
Jul 23, 2025
Solovechik
commented
- Adjusted datetime retrieval to remove timezone info for consistency.
- Simplified interval counts calculation by removing unnecessary input lookback window reference.
- Enhanced logging for batch insertions and improved color coding for schema consistency checks.
- Updated lookback window calculations to use interval start and end directly for clarity.
…use integration - Created .dockerignore and .gitignore files to exclude unnecessary files. - Added .env file for ClickHouse connection settings. - Configured dbt_project.yml with profile and macro paths. - Implemented Dockerfile for Python environment setup. - Created docker-compose.yml to define dbt and ClickHouse services. - Added profiles.yml for dbt ClickHouse connection configuration. - Included requirements.txt for necessary Python packages. - Established initial dbt models and tests for microbatch processing. - Set up GitHub Actions workflow for CI to run tests in Docker environment.
…g and interval calculations - Adjusted datetime retrieval to remove timezone info for consistency. - Simplified interval counts calculation by removing unnecessary input lookback window reference. - Enhanced logging for batch insertions and improved color coding for schema consistency checks. - Updated lookback window calculations to use interval start and end directly for clarity.
- Updated test structure to use class-based fixtures for better organization. - Introduced constants for SQL queries to enhance readability and maintainability. - Adjusted test cases for microbatch processing with varying batch sizes (1h, 8h, 24h). - Ensured consistent ClickHouse client setup across tests.
- Updated test methods to utilize a common execution function for improved code reuse. - Simplified parameter handling for batch size and query conditions in test cases. - Maintained consistent setup for ClickHouse client across different batch size tests.
Solovechik
deleted the
BI-8806-create-gh-workflow-for-testing-on-dbt-improvado-utils
branch
Solovechik
restored the
BI-8806-create-gh-workflow-for-testing-on-dbt-improvado-utils
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.