Error in: xspect models train ncbi

[bheimbu]

Hi @aromberg

I've been already in contact with @Dominik0304 about XspecT, and he pointed me to this updated version of the software. I want to train a model for Wolbachia, however it doesn't work:

$ xspect models train ncbi
Model genus: wolbachia
Training wolbachia species and genus metagenome model.
2025-09-22 21:19:21.799 | INFO     | xspect.train:train_from_ncbi:215 - Getting NCBI metadata...
2025-09-22 21:19:23.784 | DEBUG    | xspect.ncbi:get_accessions:247 - Could not get reference accessions for taxon with ID: 244203. Skipping.
2025-09-22 21:19:24.217 | DEBUG    | xspect.ncbi:get_accessions:247 - Could not get complete_genome accessions for taxon with ID: 244203. Skipping.
2025-09-22 21:19:24.610 | DEBUG    | xspect.ncbi:get_accessions:247 - Could not get chromosome accessions for taxon with ID: 244203. Skipping.
2025-09-22 21:19:25.031 | DEBUG    | xspect.ncbi:get_accessions:247 - Could not get scaffold accessions for taxon with ID: 244203. Skipping.
2025-09-22 21:19:25.432 | DEBUG    | xspect.ncbi:get_accessions:247 - Could not get contig accessions for taxon with ID: 244203. Skipping.
2025-09-22 21:19:25.433 | WARNING  | xspect.train:train_from_ncbi:237 - No assemblies found for tax_id 244203. Skipping.
2025-09-22 21:19:28.018 | WARNING  | xspect.train:train_from_ncbi:237 - No assemblies found for tax_id 2640676. Skipping.
2025-09-22 21:19:30.146 | WARNING  | xspect.train:train_from_ncbi:237 - No assemblies found for tax_id 955. Skipping.
Error: No species with accessions found. Please check the genus name.

I've also tried with Cardinium but I always get the same error that accessions could not be found. Other commands such as xspect models download work.

Any help is highly appreciated!

Cheers Bastian

[aromberg]

Hey Bastian,

thanks for checking out XspecT!

During the training process, NCBI assemblies are filtered by XspecT (e. g. based on N50) to avoid data quality issues. In your case, Wolbachia assemblies were filtered out because the taxonomy check status is "inconclusive" on NCBI, while Cardinium assemblies were additionally filtered out due to "candidatus" being in their name.

To fix the issue, I have just updated XspecT: With 0.6.0, data quality filter settings are now exposed to the CLI.

Simply upgrade to 0.6.0:

pip install xspect==0.6.0

You can now see all data quality flags by running:

xspect models train ncbi --help

In your case add --allow-inconclusive for Wolbachia:

xspect models train ncbi -g Wolbachia --allow-inconclusive

For Cardinium, additionally add --allow-candidatus:

xspect models train ncbi -g Cardinium --allow-inconclusive --allow-candidatus

You should then be able to filter for Wolbachia/Cardinium using xspect filter genus.

Hope this helps! Please note that XspecT is under active development and still somewhat unstable, although we are happy to help.

Adrian

[bheimbu]

Many thanks @aromberg,

you make my day. May I also ask about the MLST functionality: Would it also be possible to create a MLST model to classify Wolbachia?

Cheers @bheimbu

ps: Also many thanks to Felix Langschied.

[Oemercetin06]

Hello Bastian,

first of all, thanks for pointing out that the documentation did not include the creation of other MLST models. I will add this information soon!

To answer your question: Yes it is possible to create other models than the default Acinetobacter baumannii ones, but the MLST scheme for the genus/species has to be featured in the PubMLST database (https://pubmlst.org/organisms). For your case, it seems that this is the case. In order to create one you have to run 'xspect models train mlst -c' (emphasis on the flag -c for choose scheme). The program will ask you to pick a genus or species out of the available ones and then a scheme (Search for your desired scheme or just use the "MLST" option, which is always the most popular scheme). After training you can run 'xspect classify mlst' and of course provide the path to the file of interest.

If you encounter any problems, please dont hesitate to ask right away, as it helps to improve the software.

Best regards,

Ömer

[bheimbu]

Dear @Oemercetin06 ,

many thanks for your help. I'll try it out asap.

Cheers Bastian

[bheimbu]

Hi @Oemercetin06 ,

I've tried as you suggested.

$ xspect models train mlst -c

I choosed 139 for Wolbachia and 1 for MLST, but when I run xspect classify mlst, I get

$ xspect classify mlst -i nanhermannia/ -o out -l
Classifying...
1:MLST
2:MLST (Oxford)

Pick a scheme for strain type prediction
Choose a scheme by selecting the corresponding number:1
Traceback (most recent call last):
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/bin/xspect", line 7, in <module>
    sys.exit(cli())
             ~~~^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 1383, in main
    rv = self.invoke(ctx)
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 1246, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/click/core.py", line 814, in invoke
    return callback(*args, **kwargs)
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/xspect/main.py", line 394, in classify_mlst
    classify.classify_mlst(Path(input_path), Path(output_path), limit)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/xspect/classify.py", line 106, in classify_mlst
    result = model.predict(scheme_path, current_path, step=1, limit=limit)
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/xspect/models/probabilistic_filter_mlst_model.py", line 340, in predict
    return ProbabilisticFilterMlstSchemeModel.predict(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<3 lines>...
        limit=limit,
        ^^^^^^^^^^^^
    )
    ^
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/xspect/models/probabilistic_filter_mlst_model.py", line 355, in predict
    individual_hits = self.calculate_hits(
        cobs_path, individual_seq.seq, step, limit
    )
  File "/mnt/vast-standard/home/bheimbu/u14105/miniforge3/envs/xspect2/lib/python3.13/site-packages/xspect/models/probabilistic_filter_mlst_model.py", line 263, in calculate_hits
    result_dict[scheme_path_list[counter]] = sorted_counts
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object does not support item assignment

No clue what is wrong here to be honest.

Cheers Bastian

[bheimbu]

Hi,

could you please get back to me? Just to let you know Wolbachia has now the number 141. But I still get the same error when I try to classify my genomes with xspect classify mlst.

Cheers Bastian

[bheimbu]

Hi,

I found a couple of bugs, which are related to #33 (comment), in probabilistic_filter_mlst_model.py -- with the help of ChatGPT, to be honest 🙈.

Here is a minimal patch summary (for a longer version see here):

Bug 1: In calculate_hits, long-sequence branch assigns a string to result_dict on if not sorted_counts:, but later uses result_dict[...] = ... → TypeError: 'str' object does not support item assignment.
Fix: keep result_dict a dict, store per-locus empties and any message separately.

Bug 2: In calculate_hits, short-sequence branch calls next(iter(result.items())) even when result can be {} → StopIteration.
Fix: guard against empty result and only call next(...) when non-empty.

Bug 3: ST lookup flattens highest_results assuming every per-locus dict is non-empty and keys are well-formed, which fails once empties are allowed.
Fix: skip empty dicts and be robust in parsing allele IDs.

Here is the patched version of probabilistic_filter_mlst_model.py. With this xspect classify mlst runs smoothly and produces the expected json files for each genome assembly.

Cheers Bastian

[Oemercetin06]

Hi Bastian,

First of all sorry for the late response! Due to other projects I was not able to work on XspecT for some time, but I will begin on improving the MLST feature from now on.

Some things that you have to know in advance:

• The MLST feature was originally meant to only support schemes of Acinetobacter baumannii. However, during the development phase it was decided to also create a foundation for other schemes of other species to make the feature more scalable. As a result, the MLST feature of XspecT may behave somewhat strange for other schemes, since it is still experimental and has not yet been thoroughly tested.
• Currently the MLST-feature only accepts complete genomes as input (e.g. whole genomes from NCBI). Inputs on Read-level, Contig-level, etc. are yet to be implemented.
• I will now focus on solving these main issues and make the feature more accessable.

Regarding your problem with Wolbachia:
Unfortunately I was not able to reproduce your error with the latest version of XspecT (0.7.2). I have used the reference genome of Wolbachia pipientis from NCBI (https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_007971685.1/) and other assemblies of lower quality as test cases. All of those tests produced the correct results.

Can you tell me exactly what your input consisted of (filetype, assembly-level, origin, etc.)? That information can help me to locate the issue.

[bheimbu]

Hi @Oemercetin06,

thanks for getting back to me. I was running xspect against unpublished PacBio assemblies of oribatid mites. This may explain why it failed in the first place ;)

Cheers Bastian