ALEobserve crashes with tgamma overflow on large gene trees (~8000 nodes)

[linweiliarchaea]

Dear ALE developers @ssolo @Boussau @drenal @tbigot @maxemil @etannier ,

I encountered a numerical error when running ALEobserve (v1.0) on a large gene tree.

Specifically, when applying ALEobserve to a gene tree containing 8062 nodes (file: example_8062.treefile), the program terminates with the following error:

ALEobserve using ALE v1.0
1
- 8062 nodes loaded.
terminate called after throwing an instance of 'boost::wrapexcept<std::overflow_error>'
  what():  Error in function boost::math::tgamma<long double>(long double):
           Result of tgamma is too large to represent.
Aborted (core dumped)

I would like to ask whether this error indicates that the input tree exceeds the practical size limits that ALEobserve can handle. Is there a recommended upper bound on the number of nodes (or taxa) for gene trees used with ALE?

If so, would the recommended solution be to reduce the size of the gene tree (e.g. by pruning taxa), or are there specific parameters, alternative commands, or implementation details that allow ALE to handle trees at this scale more robustly?

Any guidance on best practices for reconciling very large gene trees with ALE would be greatly appreciated.

Best regards,
Lily

[drenal]

Dear Lily,

we would need more information to determine where the error has happened exactly (compiling ALE in Debug mode and provide a full traceback and/or sharing the example file).

Most probably the scalar_type that is defined as a long double inside ALE (and then passed to the boost library) is not enough to represent the double factorial of (2*8062-3) that would be somewhere in the order of 10^30000. If this is indeed the problem, fixing it would require substantial change in the algorithmic works of ALE.

The successor of ALE, AleRax, is better suited for reconciling both large families and many families with a species tree (and also avoids having to calculate the double factorial), so I would recommend you to give it a try instead of continuing with ALE when you're working with gene trees larger than 1000 taxa: https://github.com/BenoitMorel/AleRax .

Best,
Lenard

[linweiliarchaea]

Dear Lily,

we would need more information to determine where the error has happened exactly (compiling ALE in Debug mode and provide a full traceback and/or sharing the example file).

Most probably the scalar_type that is defined as a long double inside ALE (and then passed to the boost library) is not enough to represent the double factorial of (2*8062-3) that would be somewhere in the order of 10^30000. If this is indeed the problem, fixing it would require substantial change in the algorithmic works of ALE.

The successor of ALE, AleRax, is better suited for reconciling both large families and many families with a species tree (and also avoids having to calculate the double factorial), so I would recommend you to give it a try instead of continuing with ALE when you're working with gene trees larger than 1000 taxa: https://github.com/BenoitMorel/AleRax .

Best, Lenard

Dear Dr. Szánthó (@drenal),

Thank you very much for your prompt and helpful response.

Following your suggestion, I recompiled ALE in Debug mode and re-ran the command
ALEobserve example.txt using the minimal reproducible input file, which I have shared here:
example.txt.

The error is reproducible, and the full traceback obtained during execution is shown below:

ALEobserve using ALE v1.0
1
- 8062 nodes loaded.
terminate called after throwing an instance of 'boost::wrapexcept<std::overflow_error>'
  what():  Error in function boost::math::tgamma<long double>(long double): 
          Result of tgamma is too large to represent.
Aborted (core dumped)

This behavior appears consistent with your hypothesis that the internal scalar_type (defined as long double and passed to the Boost library) is insufficient to represent the extremely large values involved in computing the double factorial term, on the order of ~10^30000 for a tree of this size.

Please let me know if any additional information or diagnostics would be helpful.
I also appreciate your recommendation regarding AleRax and will explore it as an alternative for reconciling gene trees of this scale.

Best regards,
Lily

[linweiliarchaea]

Dear Dr. Szánthó (@drenal),

In this context, we would greatly appreciate your advice on designing an appropriate downsampling strategy for very large gene families when applying ALE.

Specifically, we would like to seek your guidance on the following points:

1. Upper bound on family size for reliable parameter estimation
Is there a generally accepted or empirically reasonable upper limit on the number of genes/proteins to retain during downsampling for robust DTL parameter estimation?
For instance, is there a commonly recommended target scale (e.g., several hundred versus approximately one thousand sequences), beyond which parameter optimization becomes either unreliable due to model underfitting or computationally prohibitive?

2. Clade-aware downsampling in highly unbalanced families
In cases where a gene family is massively expanded in eukaryotes but represented by a relatively moderate number of homologs in prokaryotes, would it be advisable to focus downsampling primarily on the eukaryotic clade while retaining most or all prokaryotic sequences?
If so, are there recommended sampling principles, such as maximizing phylogenetic diversity, stratified sampling across major lineages, or preserving deep-branching representatives?

3. Multiple downsampling replicates and result integration
Would you recommend performing multiple independent downsampling runs (e.g., random or diversity-aware subsampling) to assess the robustness of inferred DTL parameters and reconciliation patterns?
If multiple runs are performed, is there a principled way to compare or integrate ALE results across different sampled taxon sets (for example, by evaluating consistency in estimated rates or reconciliation scenarios)?

Overall, our aim is to strike a balance between computational feasibility and biological representativeness, while minimizing potential biases introduced by extreme gene copy number expansions in specific clades.

Thank you very much for your time and valuable insights.

Best regards,
Lily