HMM seed evaluation failed with "cannot draw more samples without replacement"

[spleonard1]

A CAMEOX run failed with the following output:

IMPORTANT: The random barcode on this run is: aWdI6d2C
INFO: Host: Parsed Codon Usage Table for taxid 562 (Escherichia coli)
INFO: std_setup:NST_full_general_main(): CAMEOX tensor built
INFO: std_setup:full_set_up(): Evaluating HMM seeds...
ERROR: LoadError: Cannot draw more samples without replacement.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] sample!(rng::Random._GLOBAL_RNG, a::Vector{Pair{Int64, Float64}}, x::Vector{Pair{Int64, Float64}}; replace::Bool, ordered::Bool)
    @ StatsBase ~/.julia/packages/StatsBase/Q76Ni/src/sampling.jl:465
  [3] #sample#193
    @ ~/.julia/packages/StatsBase/Q76Ni/src/sampling.jl:506 [inlined]
  [4] #sample#194
    @ ~/.julia/packages/StatsBase/Q76Ni/src/sampling.jl:508 [inlined]
  [5] full_set_up(out_path::SubString{String}, mark_name::SubString{String}, mark_hmm::SubString{String}, mark_grem::SubString{String}, deg_name::SubString{String}, deg_hmm::SubString{String}, deg_grem::SubString{String}, pop_size::Int64, num_samples::Int64, bad_threshold::Int64, rand_barcode::String, frame::SubString{String}, host_tid::Int64)
    @ Main.main.std_setup ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/std_setup.jl:565
  [6] set_up_and_optimize(log_io::IOStream, rand_barcode::String, out_path::SubString{String}, mark_name::SubString{String}, deg_name::SubString{String}, mark_grem::SubString{String}, deg_grem::SubString{String}, mark_hmm::SubString{String}, deg_hmm::SubString{String}, pop_size::Int64, frame::SubString{String}, rel_change_thr::Float64, host_tid::Int64, pll_weights::SubString{String}; X_range::Bool, Y_range::Bool, actually_mrf::Bool)
    @ Main.main ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:102
  [7] set_up_and_optimize(log_io::IOStream, rand_barcode::String, out_path::SubString{String}, mark_name::SubString{String}, deg_name::SubString{String}, mark_grem::SubString{String}, deg_grem::SubString{String}, mark_hmm::SubString{String}, deg_hmm::SubString{String}, pop_size::Int64, frame::SubString{String}, rel_change_thr::Float64, host_tid::Int64, pll_weights::SubString{String})
    @ Main.main ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:68
  [8] (::Main.main.var"#7#8"{String, IOStream, SubString{String}, SubString{String}, SubString{String}, SubString{String}, SubString{String}, SubString{String}, SubString{String}, SubString{String}, SubString{String}})()
    @ Main.main ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:514
  [9] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:491
 [10] with_logger
    @ ./logging.jl:603 [inlined]
 [11] run_file()
    @ Main.main ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:511
 [12] top-level scope
    @ ./timing.jl:210 [inlined]
 [13] top-level scope
    @ ~/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:0
in expression starting at /Users/leonard28/Desktop/Projects-LLNL/2021_Cameos/code_testing/CAMEOX/main/main.jl:5

I'm entangled phage proteins in this run, some of which have very orthologs in their MSAs with default parameters. Maybe this is somehow related to a not-diverse HMM that isn't generated enough samples....?

[khyox]

Long story short, what you mention seems to be the root problem! The algorithm is trying to sample (without replacement) a given top-scored fraction of the HMM-based seeds, and it fails because the number of samples is greater than the population. I am going to modify the code so that to catch that error and introduce a progressive relaxation on the sampling constraints with the hope that this approach will overcome the problem in many cases without compromising the quality of the seeds.

[spleonard1]

Ah I see, thank you!
Weirdly, if I run CAMEOX individually (so each pair gets a separate call to CAMEOX, not all 55 pairs in a single command file), they all run fine. Even this pair that previously gave an error... can you think of any reason why this would be the case??

[khyox]

Does that happen in a consistent way? I mean, always that you run the pairs in a single command file the same set of pairs fail, and always that you run them using a single command file they run fine.

[spleonard1]

I’ll experiment a little more and let you know!

…

On Tue, May 24, 2022 at 11:17 PM Jose Manuel Martí ***@***.***> wrote: Does that happen in a consistent way? I mean, always that you run the pairs in a single command file the same set of pairs fail, and always that you run them using a single command file they run fine. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABAKECGP7TZBB52YF7AFJPLVLXAXJANCNFSM5WWYQM3A> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[khyox]

Perfect! Thanks! :)