Uninformative run failures at polishing step

[mhibberd]

Hi!

We're running hybrid pacbio/illumina metagenomic assemblies through OPERA-MS, and repeatedly having the process fail at the pilon polishing step, even at supplied memory up to ~200G.

In the pilon phase of the assembly process, bwa mem runs fine, samtools can index, but the pilon run itself fails with "Error in. during pilon. Please see..." in the logfile. The "pilon.out" file has expected output, but the "pilon.err" file is empty. I did notice that the java call for pilon does not include an "xmx" type memory specification for pilon, and when I move the java pilon call outside my OPERA run and use an 'xmx' limit that's larger (100G+) pilon runs, albeit very, very slowly (14+ days and counting).

We're working at an illumina depth of ~225X.

Do you have recommendations on java memory handling within OPERA without running pilon externally, or for handling higher-depth short read data in a way that's more reasonable?

Thanks so much,

Matt Hibberd
Gordon Lab
WUSTL

[TongChengxuan]

Hi Matt,

A solution to this is that you can run pilon with --targets option to parallelize the run with multiple small fragments and combine them. This will improve the running time and reduce memory usage.

Best regards,
Chengxuan