Skip to content

[BUG]Unable to add custom import #564

@sivankumar86

Description

@sivankumar86

Describe the bug
I am using dbt covers to generator python dag from yml file. it is very helpful however, there is no option to add additional import

To Reproduce

have custom function to send alert and want to include in onfailure. custom python module needs to be imported.

`description: dbt-hourly
tags:

  • teama
  • dbt-fiv
    default_args:
    start_date: 2023-01-01
    schedule: null
    owner: teama
    access_control:
    fiv:
    • can_edit
    • can_read
    • can_delete
      catchup: false
      nodes:
      task_1:
      type: task
      imports: []
      operator: airflow.providers.amazon.aws.operators.ecs.EcsRunTaskOperator
      image: idap-fiv:latest
      aws_conn_id: aws_default
      cluster: idap-dbt-cluster
      task_definition: idap-fiv-task:3
      launch_type: FARGATE
      overrides:
      containerOverrides:
      • name: sd-fiv
        command:
        • run
        • -s
        • clean
          network_configuration:
          awsvpcConfiguration:
          subnets:
        • '{{ var.value.IDAP_ECS_SUBNET }}'
          securityGroups:
        • '{{ var.value.IDAP_ECS_SECURITY_GROUP }}'
          assignPublicIp: DISABLED
          task_2:
          type: task
          dependencies: ["task_1"]
          operator: airflow.providers.amazon.aws.operators.ecs.EcsRunTaskOperator
          aws_conn_id: aws_default
          cluster: ibmdp-tbv-cluster
          task_definition: sd_fiv-task:10
          launch_type: FARGATE
          overrides:
          containerOverrides:
      • name: sd-fiv
        command:
        • test
        • -s
        • clean
          network_configuration:
          awsvpcConfiguration:
          subnets:
        • '{{ var.value.IDAP_ECS_SUBNET }}'
          securityGroups:
        • '{{ var.value.IDAP_ECS_SECURITY_GROUP }}'
          assignPublicIp: DISABLED

`

Error:
"on_failure_callback": trigger_lambda_on_failure,
"on_retry_callback": trigger_lambda_on_failure but import doesnt get added through Yml

dbt-coves generate airflow-dags --yml-path example_dag_with_imports.yml --dags-path ./dags

Expected behaviour

It should generate dag successfully.

`
# Use list instead of set for imports to maintain order and prevent duplicates

  •        unique_imports = []
    
  •        for imp in self.dag_output["imports"]:
    
  •            if imp not in unique_imports:
    
  •                unique_imports.append(imp)
    
  •        final_output = (
    
  •            "".join(set(self.dag_output["imports"]))
    
  •            "".join(unique_imports)`
    

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdependenciesPull requests that update a dependency filepythonPull requests that update python codetriaged: noHasn't been approved for future implementation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions