Error on run

[muntean-alex]

Hello,

I'm having an issue running pling. When not using "--sourmash"
pling input.txt output_dir align --cores 48
i get the following error, for which I am attaching the last correct ran and the error:

[Sun Oct  5 22:24:17 2025]
Finished job 19.
15 of 23 steps (65%) done
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "STRAIN1-nanopore-autocycler-contigs-4~STRAIN2-nanopore-autocycler-contigs-4.ntref" of length 43381
# construct suffix tree for sequence of length 43381
# (maximum reference length is 2305843009213693948)
# (maximum query length is 18446744073709551615)
# CONSTRUCTIONTIME ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-nanopore-autocycler-contigs-4~STRAIN2-nanopore-autocycler-contigs-4.ntref 0.01
# reading input file "/mnt/storage/DIR/autocycler/plasmids/STRAIN2-nanopore-autocycler-contigs-4.fasta" of length 43380
# matching query-file "/mnt/storage/DIR/autocycler/plasmids/STRAIN2-nanopore-autocycler-contigs-4.fasta"
# against subject-file "STRAIN1-nanopore-autocycler-contigs-4~STRAIN2-nanopore-autocycler-contigs-4.ntref"
# COMPLETETIME ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-nanopore-autocycler-contigs-4~STRAIN2-nanopore-autocycler-contigs-4.ntref 0.02
# SPACE ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-nanopore-autocycler-contigs-4~STRAIN2-nanopore-autocycler-contigs-4.ntref 0.08
4: FINISHING DATA
[Sun Oct  5 22:24:19 2025]
Finished job 17.
16 of 23 steps (70%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2025-10-05T222128.227358.snakemake.log

Command 'snakemake --snakefile ~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/Snakefile --configfile output_dir/tmp_files/config.yaml --cores 48 --rerun-incomplete --nolock  ' returned non-zero exit status 1.
Traceback (most recent call last):
  File "~/miniconda3/envs/pling/bin/pling", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/run_pling.py", line 183, in main
    pling(args)
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/run_pling.py", line 151, in pling
    raise e
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/run_pling.py", line 146, in pling
    subprocess.run(f"snakemake --snakefile {get_pling_path()}/align_snakemake/Snakefile {snakemake_args}", shell=True, check=True, capture_output=True)
  File "~/miniconda3/envs/pling/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'snakemake --snakefile ~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/Snakefile --configfile output_dir/tmp_files/config.yaml --cores 48 --rerun-incomplete --nolock  ' returned non-zero exit status 1.

The logfile shows this:

$ cat .snakemake/log/2025-10-05T222128.227358.snakemake.log
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 48
Rules claiming more threads will be scaled down.
Job stats:
job                   count
------------------  -------
all                       1
cat_containment           1
create_genomes_tsv        1
get_communities           1
make_unimogs             19
total                    23

Select jobs to execute...

[Sun Oct  5 22:21:28 2025]
localrule create_genomes_tsv:
    output: output_dir/tmp_files/genomes.tsv
    jobid: 22
    reason: Missing output files: output_dir/tmp_files/genomes.tsv
    resources: tmpdir=/tmp

[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_17.txt
    output: output_dir/tmp_files/containment_batchwise/batch_17_containment.tsv, output_dir/unimogs/batch_17_align.unimog, output_dir/unimogs/batch_17_map.txt
    jobid: 19
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_17_containment.tsv
    wildcards: batch=17
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_9.txt
    output: output_dir/tmp_files/containment_batchwise/batch_9_containment.tsv, output_dir/unimogs/batch_9_align.unimog, output_dir/unimogs/batch_9_map.txt
    jobid: 11
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_9_containment.tsv
    wildcards: batch=9
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_18.txt
    output: output_dir/tmp_files/containment_batchwise/batch_18_containment.tsv, output_dir/unimogs/batch_18_align.unimog, output_dir/unimogs/batch_18_map.txt
    jobid: 20
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_18_containment.tsv
    wildcards: batch=18
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_6.txt
    output: output_dir/tmp_files/containment_batchwise/batch_6_containment.tsv, output_dir/unimogs/batch_6_align.unimog, output_dir/unimogs/batch_6_map.txt
    jobid: 8
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_6_containment.tsv
    wildcards: batch=6
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_2.txt
    output: output_dir/tmp_files/containment_batchwise/batch_2_containment.tsv, output_dir/unimogs/batch_2_align.unimog, output_dir/unimogs/batch_2_map.txt
    jobid: 4
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_2_containment.tsv
    wildcards: batch=2
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_8.txt
    output: output_dir/tmp_files/containment_batchwise/batch_8_containment.tsv, output_dir/unimogs/batch_8_align.unimog, output_dir/unimogs/batch_8_map.txt
    jobid: 10
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_8_containment.tsv
    wildcards: batch=8
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_3.txt
    output: output_dir/tmp_files/containment_batchwise/batch_3_containment.tsv, output_dir/unimogs/batch_3_align.unimog, output_dir/unimogs/batch_3_map.txt
    jobid: 5
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_3_containment.tsv
    wildcards: batch=3
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_13.txt
    output: output_dir/tmp_files/containment_batchwise/batch_13_containment.tsv, output_dir/unimogs/batch_13_align.unimog, output_dir/unimogs/batch_13_map.txt
    jobid: 15
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_13_containment.tsv
    wildcards: batch=13
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_10.txt
    output: output_dir/tmp_files/containment_batchwise/batch_10_containment.tsv, output_dir/unimogs/batch_10_align.unimog, output_dir/unimogs/batch_10_map.txt
    jobid: 12
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_10_containment.tsv
    wildcards: batch=10
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_11.txt
    output: output_dir/tmp_files/containment_batchwise/batch_11_containment.tsv, output_dir/unimogs/batch_11_align.unimog, output_dir/unimogs/batch_11_map.txt
    jobid: 13
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_11_containment.tsv
    wildcards: batch=11
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_14.txt
    output: output_dir/tmp_files/containment_batchwise/batch_14_containment.tsv, output_dir/unimogs/batch_14_align.unimog, output_dir/unimogs/batch_14_map.txt
    jobid: 16
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_14_containment.tsv
    wildcards: batch=14
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_4.txt
    output: output_dir/tmp_files/containment_batchwise/batch_4_containment.tsv, output_dir/unimogs/batch_4_align.unimog, output_dir/unimogs/batch_4_map.txt
    jobid: 6
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_4_containment.tsv
    wildcards: batch=4
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_0.txt
    output: output_dir/tmp_files/containment_batchwise/batch_0_containment.tsv, output_dir/unimogs/batch_0_align.unimog, output_dir/unimogs/batch_0_map.txt
    jobid: 2
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_0_containment.tsv
    wildcards: batch=0
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_15.txt
    output: output_dir/tmp_files/containment_batchwise/batch_15_containment.tsv, output_dir/unimogs/batch_15_align.unimog, output_dir/unimogs/batch_15_map.txt
    jobid: 17
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_15_containment.tsv
    wildcards: batch=15
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_5.txt
    output: output_dir/tmp_files/containment_batchwise/batch_5_containment.tsv, output_dir/unimogs/batch_5_align.unimog, output_dir/unimogs/batch_5_map.txt
    jobid: 7
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_5_containment.tsv
    wildcards: batch=5
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_7.txt
    output: output_dir/tmp_files/containment_batchwise/batch_7_containment.tsv, output_dir/unimogs/batch_7_align.unimog, output_dir/unimogs/batch_7_map.txt
    jobid: 9
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_7_containment.tsv
    wildcards: batch=7
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_16.txt
    output: output_dir/tmp_files/containment_batchwise/batch_16_containment.tsv, output_dir/unimogs/batch_16_align.unimog, output_dir/unimogs/batch_16_map.txt
    jobid: 18
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_16_containment.tsv
    wildcards: batch=16
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_12.txt
    output: output_dir/tmp_files/containment_batchwise/batch_12_containment.tsv, output_dir/unimogs/batch_12_align.unimog, output_dir/unimogs/batch_12_map.txt
    jobid: 14
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_12_containment.tsv
    wildcards: batch=12
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537


[Sun Oct  5 22:21:28 2025]
rule make_unimogs:
    input: output_dir/batches/batch_1.txt
    output: output_dir/tmp_files/containment_batchwise/batch_1_containment.tsv, output_dir/unimogs/batch_1_align.unimog, output_dir/unimogs/batch_1_map.txt
    jobid: 3
    reason: Missing output files: output_dir/tmp_files/containment_batchwise/batch_1_containment.tsv
    wildcards: batch=1
    resources: tmpdir=/tmp, mem_mb=10000, mem_mib=9537

[Sun Oct  5 22:21:30 2025]
Finished job 22.
1 of 23 steps (4%) done
[Sun Oct  5 22:22:09 2025]
Finished job 20.
2 of 23 steps (9%) done
[Sun Oct  5 22:23:30 2025]
Finished job 12.
3 of 23 steps (13%) done
[Sun Oct  5 22:23:32 2025]
Finished job 16.
4 of 23 steps (17%) done
[Sun Oct  5 22:23:32 2025]
Finished job 7.
5 of 23 steps (22%) done
[Sun Oct  5 22:23:41 2025]
Finished job 9.
6 of 23 steps (26%) done
[Sun Oct  5 22:23:50 2025]
Finished job 18.
7 of 23 steps (30%) done
[Sun Oct  5 22:23:55 2025]
Finished job 14.
8 of 23 steps (35%) done
[Sun Oct  5 22:23:59 2025]
Finished job 10.
9 of 23 steps (39%) done
[Sun Oct  5 22:24:00 2025]
Finished job 15.
10 of 23 steps (43%) done
[Sun Oct  5 22:24:01 2025]
Finished job 11.
11 of 23 steps (48%) done
[Sun Oct  5 22:24:05 2025]
Finished job 13.
12 of 23 steps (52%) done
[Sun Oct  5 22:24:10 2025]
Finished job 6.
13 of 23 steps (57%) done
[Sun Oct  5 22:24:15 2025]
Finished job 8.
14 of 23 steps (61%) done
[Sun Oct  5 22:24:17 2025]
Finished job 19.
15 of 23 steps (65%) done
[Sun Oct  5 22:24:19 2025]
Finished job 17.
16 of 23 steps (70%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2025-10-05T222128.227358.snakemake.lo

[iqbal-lab]

Sorry I missed this. @babayagaofficial is out of the office, but back tomorrow

[babayagaofficial]

Hi, could you please run for batches 0-3 (replace {number} in the command below with 0,1, 2 and 3 respectively):

python ~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/unimog.py \
            --genomes_list input.txt \
            --batch output_dir/batches/batch_{number}.txt\
            --identity_threshold 80 \
            --containment_distance 0.5 \
            --outputpath output_dir \
            --containment_output output_dir/tmp_files/containment_batchwise/batch_{number}_containment.tsv \
            --unimog_output output_dir/unimogs/batch_{number}_align.unimog \
            --map_output output_dir/unimogs/batch_{number}_map.txt \

and let me know what happens?
Sorry about the hassle -- this should help me isolate where exactly it's failing.

[muntean-alex]

Dear all,

For all the batches, the error was at the same contig, for the same strain.
Removing that one plasmid file made it work. Since all the plasmids were extracted from the "parent" fasta file using a script, I don't know why that one file would differ?
Can I send over the file so you can take a look at it to find the error?

1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.ntref" of length 5734
# construct suffix tree for sequence of length 5734
# (maximum reference length is 2305843009213693948)
# (maximum query length is 18446744073709551615)
# CONSTRUCTIONTIME ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.ntref 0.00
# reading input file "~/Downloads/plasmids/STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.fasta" of length 499739
# matching query-file "~/Downloads/plasmids/STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.fasta"
# against subject-file "STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.ntref"
# COMPLETETIME ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.ntref 0.04
# SPACE ~/miniconda3/envs/pling/opt/mummer-3.23/mummer STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.ntref 0.49
4: FINISHING DATA
ERROR: Query input does not match delta file
Traceback (most recent call last):
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/unimog.py", line 86, in <module>
    main()
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/unimog.py", line 82, in main
    batchwise_unimog(fastafiles, pairs, args.unimog_output, args.map_output, args.containment_output, args.identity_threshold, args.containment_distance, topologies)
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/unimog.py", line 25, in batchwise_unimog
    unimog, containment, blocks_ref, blocks_query = unimogs_to_ilp_core(genome_1_fasta, genome_2_fasta, genome_1, genome_2, containment_threshold, identity_threshold, topology_1, topology_2)
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/unimog.py", line 9, in unimogs_to_ilp_core
    plasmid_1_unimogs, plasmid_2_unimogs, containment_distance, blocks_ref, blocks_query = integerise_plasmids(genome_1_fasta, genome_2_fasta,
                                                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/miniconda3/envs/pling/lib/python3.11/site-packages/pling/align_snakemake/integerise_plasmids.py", line 139, in integerise_plasmids
    show_snps_output = subprocess.check_output(f"show-snps -TrH {prefix}.1delta", shell=True).strip().split(b'\n')
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/miniconda3/envs/pling/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/miniconda3/envs/pling/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'show-snps -TrH STRAIN1-v01-20250808132333-nanopore-autocycler-contigs-5~STRAIN2_AB-v01-20250822234754-nanopore-autocycler-contigs-4.1delta' returned non-zero exit status 1.

All the best,
Alex

[babayagaofficial]

Hi Alex,

yes please do send me the file! That would be very helpful -- if you could also send me the file for at least one of the other plasmids it failed with, that would be perfect. Feel free to use my email, daria@ebi.ac.uk.