Merged
Conversation
Feature/sensor parameters
upgrade version of tenacity
…-deferrable-batch Feature/data 2449 improve failed deferrable batch
…_version Relaxing click version to avoid conflict with dbt
…_dagger Feature/data 2567 python upgrade dagger
Required for the new DLT creator which uses DatabricksRunNowOperator and DatabricksHook from apache-airflow-providers-databricks.
Required for DLT creator to work in production.
Add Claude to the repository
- Add type hints and docstrings to DatabricksIO - Improve error handling in DatabricksDLTCreator with ImportError support - Add validation for empty job_name in DatabricksDLTCreator - Add comprehensive test coverage for DatabricksIO, DatabricksDLTTask, and DatabricksDLTCreator - All Databricks components now have 100% test coverage
Prefer explicit properties over getattr for type safety and better IDE support.
DATA-2637 Implement DLT creator for Databricks Delta Live Tables
Rename task type and operator creator to match the generated task YAML files from the DLT task generator plugin. Files renamed: - databricks_dlt_task.py -> declarative_pipeline_task.py - databricks_dlt_creator.py -> declarative_pipeline_creator.py - test_databricks_dlt_task.py -> test_declarative_pipeline_task.py - test_databricks_dlt_creator.py -> test_declarative_pipeline_creator.py - databricks_dlt_task.yaml -> declarative_pipeline_task.yaml Changes: - Class names: DatabricksDLTTask -> DeclarativePipelineTask - Class names: DatabricksDLTCreator -> DeclarativePipelineCreator - ref_name: "databricks_dlt" -> "declarative_pipeline" - Updated imports in factory files
Add declarative_pipeline task type for DLT pipelines
The on_failure_callback on DatabricksRunNowOperator was overriding the DAG-level callback (Slack alerts) set in default_args. Build a composite callback that cancels the Databricks run AND invokes the DAG-level alert callback.
…lerts Fix Slack alerts not firing for declarative pipeline tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merging back to original