-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Labels
bugSomething isn't workingSomething isn't workingdependenciesPull requests that update a dependency filePull requests that update a dependency filepythonPull requests that update python codePull requests that update python codetriaged: noHasn't been approved for future implementationHasn't been approved for future implementation
Description
Describe the bug
I am using dbt covers to generator python dag from yml file. it is very helpful however, there is no option to add additional import
To Reproduce
have custom function to send alert and want to include in onfailure. custom python module needs to be imported.
`description: dbt-hourly
tags:
- teama
- dbt-fiv
default_args:
start_date: 2023-01-01
schedule: null
owner: teama
access_control:
fiv:- can_edit
- can_read
- can_delete
catchup: false
nodes:
task_1:
type: task
imports: []
operator: airflow.providers.amazon.aws.operators.ecs.EcsRunTaskOperator
image: idap-fiv:latest
aws_conn_id: aws_default
cluster: idap-dbt-cluster
task_definition: idap-fiv-task:3
launch_type: FARGATE
overrides:
containerOverrides:- name: sd-fiv
command:- run
- -s
- clean
network_configuration:
awsvpcConfiguration:
subnets: - '{{ var.value.IDAP_ECS_SUBNET }}'
securityGroups: - '{{ var.value.IDAP_ECS_SECURITY_GROUP }}'
assignPublicIp: DISABLED
task_2:
type: task
dependencies: ["task_1"]
operator: airflow.providers.amazon.aws.operators.ecs.EcsRunTaskOperator
aws_conn_id: aws_default
cluster: ibmdp-tbv-cluster
task_definition: sd_fiv-task:10
launch_type: FARGATE
overrides:
containerOverrides:
- name: sd-fiv
command:- test
- -s
- clean
network_configuration:
awsvpcConfiguration:
subnets: - '{{ var.value.IDAP_ECS_SUBNET }}'
securityGroups: - '{{ var.value.IDAP_ECS_SECURITY_GROUP }}'
assignPublicIp: DISABLED
- name: sd-fiv
`
Error:
"on_failure_callback": trigger_lambda_on_failure,
"on_retry_callback": trigger_lambda_on_failure but import doesnt get added through Yml
dbt-coves generate airflow-dags --yml-path example_dag_with_imports.yml --dags-path ./dags
Expected behaviour
It should generate dag successfully.
`
# Use list instead of set for imports to maintain order and prevent duplicates
-
unique_imports = [] -
for imp in self.dag_output["imports"]: -
if imp not in unique_imports: -
unique_imports.append(imp) -
final_output = (
-
"".join(set(self.dag_output["imports"]))
-
"".join(unique_imports)`
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingdependenciesPull requests that update a dependency filePull requests that update a dependency filepythonPull requests that update python codePull requests that update python codetriaged: noHasn't been approved for future implementationHasn't been approved for future implementation