Problem Launching Checker Workflow

[garyluu]

Encountering issues running the checker workflow with cromwell:

java -jar cromwell-39.jar run variant-caller/variant-caller-wdl-checker/topmed_freeze3_calling_checker.wdl -i variant-caller/variant-caller-wdl-checker/topmed_freeze3_calling_checker.json 
...
- NoSuchFileException: /home/gluu/topmed-workflows/https:/storage.googleapis.com/topmed_workflow_testing/topmed_variant_caller/reference_files/hg38/1000G_omni2.5.b38.sites.PASS.vcf.gz
...

and with dockstore:

gluu@ODL-GARYLUU:~/topmed-workflows$ ./dockstore workflow launch --local-entry variant-caller/variant-caller-wdl-checker/topmed_freeze3_calling_checker.wdl --json variant-caller/variant-caller-wdl-checker/topmed_freeze3_calling_checker.json 
dockstore needs to be on PATH for certain commands to work (like --upgrade)
13:17:39.796 [main] ERROR io.dockstore.client.cli.ArgumentUtility - java.lang.NoSuchMethodException: wdl4s.wdl.expression.WdlStandardLibraryFunctionsType.flatten(scala.collection.Seq)
	at java.lang.Class.getMethod(Class.java:1786)
	at wdl4s.wdl.expression.WdlFunctions$class.getFunction(WdlFunctions.scala:10)
	at wdl4s.wdl.expression.WdlStandardLibraryFunctionsType.getFunction(WdlStandardLibraryFunctions.scala:331)
	at wdl4s.wdl.expression.TypeEvaluator.evaluate(TypeEvaluator.scala:113)
	at wdl4s.wdl.WdlExpression$.evaluateType(WdlExpression.scala:91)
	at wdl4s.wdl.WdlExpression.evaluateType(WdlExpression.scala:172)
	at wdl4s.wdl.WdlNamespace$$anonfun$typeCheckDeclaration$1.apply(WdlNamespace.scala:451)
	at wdl4s.wdl.WdlNamespace$$anonfun$typeCheckDeclaration$1.apply(WdlNamespace.scala:450)
	at scala.Option.flatMap(Option.scala:171)
	at wdl4s.wdl.WdlNamespace$.typeCheckDeclaration(WdlNamespace.scala:450)
	at wdl4s.wdl.WdlNamespace$.validateDeclaration(WdlNamespace.scala:430)
	at wdl4s.wdl.WdlNamespace$$anonfun$35$$anonfun$apply$16.apply(WdlNamespace.scala:345)
	at wdl4s.wdl.WdlNamespace$$anonfun$35$$anonfun$apply$16.apply(WdlNamespace.scala:344)
	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
	at scala.collection.Iterator$class.foreach(Iterator.scala:891)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
	at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
	at wdl4s.wdl.WdlNamespace$$anonfun$35.apply(WdlNamespace.scala:344)
	at wdl4s.wdl.WdlNamespace$$anonfun$35.apply(WdlNamespace.scala:343)
	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
	at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:316)
	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:972)
	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:972)
	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
	at wdl4s.wdl.WdlNamespace$.apply(WdlNamespace.scala:343)
	at wdl4s.wdl.WdlNamespace$$anonfun$wdl4s$wdl$WdlNamespace$$load$1.apply(WdlNamespace.scala:179)
	at wdl4s.wdl.WdlNamespace$$anonfun$wdl4s$wdl$WdlNamespace$$load$1.apply(WdlNamespace.scala:179)
	at scala.util.Try$.apply(Try.scala:192)
	at wdl4s.wdl.WdlNamespace$.wdl4s$wdl$WdlNamespace$$load(WdlNamespace.scala:178)
	at wdl4s.wdl.WdlNamespace$$anonfun$20.apply(WdlNamespace.scala:209)
	at wdl4s.wdl.WdlNamespace$$anonfun$20.apply(WdlNamespace.scala:207)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.immutable.List.map(List.scala:296)
	at wdl4s.wdl.WdlNamespace$.apply(WdlNamespace.scala:207)
	at wdl4s.wdl.WdlNamespace$$anonfun$wdl4s$wdl$WdlNamespace$$load$1.apply(WdlNamespace.scala:179)
	at wdl4s.wdl.WdlNamespace$$anonfun$wdl4s$wdl$WdlNamespace$$load$1.apply(WdlNamespace.scala:179)
	at scala.util.Try$.apply(Try.scala:192)
	at wdl4s.wdl.WdlNamespace$.wdl4s$wdl$WdlNamespace$$load(WdlNamespace.scala:178)
	at wdl4s.wdl.WdlNamespace$.loadUsingSource(WdlNamespace.scala:175)
	at wdl4s.wdl.WdlNamespaceWithWorkflow$.load(WdlNamespace.scala:555)
	at io.dockstore.common.Bridge.getInputFiles(Bridge.scala:140)
	at io.github.collaboratory.wdl.WDLClient.launch(WDLClient.java:153)
	at io.dockstore.client.cli.nested.WorkflowClient.checkEntryFile(WorkflowClient.java:490)
	at io.dockstore.client.cli.nested.WorkflowClient.launch(WorkflowClient.java:451)
	at io.dockstore.client.cli.nested.AbstractEntryClient.processEntryCommands(AbstractEntryClient.java:232)
	at io.dockstore.client.cli.Client.run(Client.java:706)
	at io.dockstore.client.cli.Client.main(Client.java:608)

this is with 1.32.0

[garyluu]

Also this one too:

cwltool aligner/sbg-alignment-cwl/topmed-alignment-checker.cwl aligner/sbg-alignment-cwl/topmed-alignment-checker.sample.json
...
[job topmed_pre_align] /tmp/qc4ic3kt$ docker \
    run \
    -i \
    --volume=/tmp/qc4ic3kt:/sMxvLA:rw \
    --volume=/tmp/ko2uaics:/tmp:rw \
    --volume=/tmp/tmpj_7amlfu:/var/lib/cwl/stgd76628e7-0851-4af6-901d-e450262d2206/NWD176325.0005.recab.cram:ro \
    --volume=/tmp/tmpy44zrze9:/sMxvLA/hs38DH.fa:ro \
    --workdir=/sMxvLA \
    --read-only=true \
    --user=1001:1001 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/sMxvLA \
    --cidfile=/tmp/tmp95uz_u4i/20190403152434-055029.cid \
    statgen/alignment:1.0.0 \
    /bin/sh \
    -c \
    'samtools' 'view' -uh -F 0x900 --threads 1 '-T' '/sMxvLA/hs38DH.fa' /var/lib/cwl/stgd76628e7-0851-4af6-901d-e450262d2206/NWD176325.0005.recab.cram | bam-ext-mem-sort-manager squeeze --in -.ubam --keepDups --rmTags AS:i,BD:Z,BI:Z,XS:i,MC:Z,MD:Z,NM:i,MQ:i --out -.ubam | samtools sort -l 1 -@ 1 -n -T NWD176325.0005.recab.samtools_sort_tmp  - | samtools fixmate - - | bam-ext-mem-sort-manager bam2fastq --in -.bam --outBase NWD176325.0005.recab  --maxRecordLimitPerFq 20000000 --sortByReadNameOnTheFly --readname --gzip
/bin/sh: 1: samtools: Permission denied
/bin/sh: 1: samtools: Permission denied
/bin/sh: 1: samtools: Permission denied
myTmpOutBase is empty, using myOutBase as myTmpOutBase
Using sortByReadNameOnTheFly process...
Ready to FindAllBeginPointer... 
Ready to Dispatch...
Exiting due to ERROR:
	FAIL_IO: Failed to read the BAM header length, read 0 bytes instead of 4
[job topmed_pre_align] Max memory used: 0MiB
[job topmed_pre_align] completed permanentFail
[step topmed_pre_align] completed permanentFail
[workflow topmed_alignment] completed permanentFail
[step topmed_alignment] completed permanentFail
[workflow ] completed permanentFail
{
    "aligned_out": null,
    "stdout": null,
    "stderr": null
}
Final process status is permanentFail