Skip to content

Conversation

@javiergaitan
Copy link

@javiergaitan javiergaitan commented Oct 4, 2024

Context

This change rebases broadinstitute#7545 as well as broadinstitute#7518 on top of the deepgenomics/dg-ci branch. The goal is to port only the functionality necessary to support arbitrary machine types in wdl workflows into our fork of the cromwell repository.

Changes included

Testing

  1. Ran unit and integration tests
# Start sbt console
sbt
# Only relevant tests
testOnly cromwell.backend.google.batch.util.GcpBatchMachineConstraintsSpec
# All tests
test
  1. Deployed and ran a wdl workflow to validate the change
version 1.0

task nvidia_smi {
    input {
        String docker_version
    }

    command <<<
        nvidia-smi

        touch .done
        echo "Finished at $(date)"
    >>>

    runtime {
        docker: <internal image>
        disks: "local-disk 50 SSD"
        memory: "32G"
        preemptible: 0
        gpuCount: 1
        gpuType: "nvidia-tesla-a100"
        standardMachineType: "a2-highgpu-1g"
    }

    output {
        File done = ".done"
    }
}

workflow nvidia_smi_wf {
    input {
        String docker_version
    }
    
    call nvidia_smi  as nvidia_smi_call {
        input:
            docker_version = docker_version
    }

    output {
        File done = ".done"
    }
}
  1. Verified the machine type on GCP batch as A2 and the output of nvidia-smi showing the requested A100 accelerator.

@ovesh ovesh merged commit ae4e6aa into dg-ci Oct 10, 2024
5 of 13 checks passed
@ovesh ovesh deleted the support-standard-machine-type-google-backend-rebased branch October 10, 2024 15:16
SophiaPerzan-DG pushed a commit that referenced this pull request Feb 12, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
SophiaPerzan-DG pushed a commit that referenced this pull request Feb 12, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
SophiaPerzan-DG pushed a commit that referenced this pull request Feb 20, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
SophiaPerzan-DG pushed a commit that referenced this pull request Feb 21, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
SophiaPerzan-DG pushed a commit that referenced this pull request Mar 3, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
SophiaPerzan-DG pushed a commit that referenced this pull request Jun 4, 2025
…e backend (#1)

* WX-1810 WX-1830 n1/n2/n2d machine types, cpuPlatform on GCPBATCH (broadinstitute#7518)
* feat: [GCP Batch] Support passing standard machine types to the Google backend

---------

Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com>
@ Conflicts:
@	CHANGELOG.md
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/api/GcpBatchRequestFactoryImpl.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributes.scala
@	supportedBackends/google/batch/src/main/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraints.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/models/GcpBatchRuntimeAttributesSpec.scala
@	supportedBackends/google/batch/src/test/scala/cromwell/backend/google/batch/util/GcpBatchMachineConstraintsSpec.scala
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants