|
BPPARAM = BiocParallel::bpparam()){ |
Hi Olly,
Not sure what is going on but I have tried to explain the issue as best as I could.
Possibly CPU issues - possibly with assigning the correct number of cores when using default BPPARAM. On my previous laptop the bandle function would complete a run in ~40mins and appeared to run the default fine. When running bandle with the same parameters on my new laptop, with essentially the same/better specs, the code takes significantly longer (hours) and/or crashes. This seems to be resolved with the nu,ber of cores to use is explicitly specified in the function, as follows:
bandleres_12hr <- bandle(objectCond1 = control,
objectCond2 = treatment,
numIter = 10000, # usually 10,000
burnin = 5000, # usually 5,000
thin = 20, # usually 20
gpParams = gpParams,
pcPrior = pc_prior,
numChains = 4, # usually >=4
dirPrior = dirPrior,
BPPARAM = MulticoreParam(6L))
Without specifying the BBPARAM parameter, no progress bar shows and the CPU is working at 100% capacity. It does however appear to be using all cores - but these are all "maxed out", which causes R to crash or freeze. If I preserved potentially it may run okay but with my current laptop and a friend's laptop I borrowed this usually ended in the session crashing/freezing. When specifying as above the progress bar appears and the CPU works only at 50% capacity, uses all cores but these aren't all "maxed out".
The sessionInfo and bpparam for my old laptop is:
> bpparam()
class: SnowParam
bpisup: FALSE; bpnworkers: 6; bptasks: 0; bpjobname: BPJOB
bplog: FALSE; bpthreshold: INFO; bpstopOnError: TRUE
bpRNGseed: ; bptimeout: 2592000; bpprogressbar: FALSE
bpexportglobals: TRUE
bplogdir: NA
bpresultdir: NA
cluster type: SOCK
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)
Matrix products: default
locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C LC_TIME=English_United Kingdom.1252
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] bandle_1.0 pRoloc_1.32.0 BiocParallel_1.26.0 MLInterfaces_1.72.0 cluster_2.1.2 annotate_1.70.0 XML_3.99-0.6
[8] AnnotationDbi_1.54.0 IRanges_2.26.0 MSnbase_2.18.0 ProtGenerics_1.24.0 mzR_2.26.0 Rcpp_1.0.6 Biobase_2.52.0
[15] S4Vectors_0.30.0 BiocGenerics_0.38.0
loaded via a namespace (and not attached):
[1] BiocFileCache_2.1.1 plyr_1.8.6 splines_4.1.0 GenomeInfoDb_1.29.3 ggplot2_3.3.3 digest_0.6.27
[7] foreach_1.5.1 htmltools_0.5.1.1 viridis_0.6.1 fansi_0.5.0 magrittr_2.0.1 memoise_2.0.0
[13] doParallel_1.0.16 mixtools_1.2.0 limma_3.48.0 recipes_0.1.16 Biostrings_2.60.0 gower_0.2.2
[19] lpSolve_5.6.15 prettyunits_1.1.1 colorspace_2.0-1 blob_1.2.2 rappdirs_0.3.3 xfun_0.23
[25] dplyr_1.0.6 crayon_1.4.1 RCurl_1.98-1.3 hexbin_1.28.2 impute_1.66.0 survival_3.2-11
[31] iterators_1.0.13 glue_1.4.2 gtable_0.3.0 ipred_0.9-11 zlibbioc_1.38.0 XVector_0.32.0
[37] kernlab_0.9-29 scales_1.1.1 vsn_3.60.0 mvtnorm_1.1-1 DBI_1.1.1 viridisLite_0.4.0
[43] xtable_1.8-4 progress_1.2.2 clue_0.3-59 bit_4.0.4 proxy_0.4-25 mclust_5.4.7
[49] preprocessCore_1.54.0 lbfgs_1.2.1 MsCoreUtils_1.4.0 lava_1.6.9 prodlim_2019.11.13 sampling_2.9
[55] httr_1.4.2 FNN_1.1.3 RColorBrewer_1.1-2 ellipsis_0.3.2 pkgconfig_2.0.3 nnet_7.3-16
[61] dbplyr_2.1.1 utf8_1.2.1 caret_6.0-88 tidyselect_1.1.1 rlang_0.4.11 reshape2_1.4.4
[67] munsell_0.5.0 tools_4.1.0 LaplacesDemon_16.1.6 cachem_1.0.5 generics_0.1.0 RSQLite_2.2.7
[73] evaluate_0.14 stringr_1.4.0 fastmap_1.1.0 mzID_1.31.0 yaml_2.2.1 ModelMetrics_1.2.2.2
[79] knitr_1.33 bit64_4.0.5 purrr_0.3.4 randomForest_4.6-14 KEGGREST_1.33.0 dendextend_1.15.1
[85] ncdf4_1.17 nlme_3.1-152 xml2_1.3.2 biomaRt_2.49.2 compiler_4.1.0 rstudioapi_0.13
[91] filelock_1.0.2 curl_4.3.1 png_0.1-7 e1071_1.7-8 affyio_1.62.0 tibble_3.1.2
[97] stringi_1.6.2 lattice_0.20-44 Matrix_1.3-3 vctrs_0.3.8 pillar_1.6.2 lifecycle_1.0.0
[103] BiocManager_1.30.16 MALDIquant_1.19.3 data.table_1.14.0 bitops_1.0-7 R6_2.5.0 pcaMethods_1.84.0
[109] affy_1.70.0 gridExtra_2.3 codetools_0.2-18 MASS_7.3-54 gtools_3.8.2 assertthat_0.2.1
[115] rprojroot_2.0.2 withr_2.4.2 GenomeInfoDbData_1.2.6 hms_1.1.0 grid_4.1.0 rpart_4.1-15
[121] timeDate_3043.102 coda_0.19-4 class_7.3-19 rmarkdown_2.9 segmented_1.3-4 pROC_1.17.0.1
[127] lubridate_1.7.10
For my new laptop:
> bpparam()
class: SnowParam
bpisup: FALSE; bpnworkers: 6; bptasks: 0; bpjobname: BPJOB
bplog: FALSE; bpthreshold: INFO; bpstopOnError: TRUE
bpRNGseed: ; bptimeout: 2592000; bpprogressbar: FALSE
bpexportglobals: TRUE; bpforceGC: FALSE
bplogdir: NA
bpresultdir: NA
cluster type: SOCK
> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)
Matrix products: default
locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] bandle_1.0 pRoloc_1.34.0 BiocParallel_1.28.1 MLInterfaces_1.74.0 cluster_2.1.2 annotate_1.72.0 XML_3.99-0.8
[8] AnnotationDbi_1.56.2 IRanges_2.28.0 MSnbase_2.20.1 ProtGenerics_1.26.0 mzR_2.28.0 Rcpp_1.0.7 Biobase_2.54.0
[15] S4Vectors_0.32.3 BiocGenerics_0.40.0
loaded via a namespace (and not attached):
[1] snow_0.4-4 circlize_0.4.13 BiocFileCache_2.2.0 plyr_1.8.6 splines_4.1.2 listenv_0.8.0 GenomeInfoDb_1.30.0
[8] ggplot2_3.3.5 digest_0.6.28 foreach_1.5.1 htmltools_0.5.2 viridis_0.6.2 ggalluvial_0.12.3 fansi_0.5.0
[15] magrittr_2.0.1 memoise_2.0.0 doParallel_1.0.16 mixtools_1.2.0 limma_3.50.0 recipes_0.1.17 globals_0.14.0
[22] Biostrings_2.62.0 gower_0.2.2 lpSolve_5.6.15 prettyunits_1.1.1 colorspace_2.0-2 ggrepel_0.9.1 blob_1.2.2
[29] rappdirs_0.3.3 xfun_0.28 dplyr_1.0.7 crayon_1.4.2 RCurl_1.98-1.5 hexbin_1.28.2 impute_1.68.0
[36] survival_3.2-13 iterators_1.0.13 glue_1.5.0 gtable_0.3.0 ipred_0.9-12 zlibbioc_1.40.0 XVector_0.34.0
[43] kernlab_0.9-29 shape_1.4.6 future.apply_1.8.1 scales_1.1.1 vsn_3.62.0 mvtnorm_1.1-3 DBI_1.1.1
[50] viridisLite_0.4.0 xtable_1.8-4 progress_1.2.2 clue_0.3-60 bit_4.0.4 proxy_0.4-26 mclust_5.4.8
[57] preprocessCore_1.56.0 lbfgs_1.2.1 MsCoreUtils_1.6.0 lava_1.6.10 prodlim_2019.11.13 sampling_2.9 httr_1.4.2
[64] FNN_1.1.3 RColorBrewer_1.1-2 ellipsis_0.3.2 pkgconfig_2.0.3 nnet_7.3-16 dbplyr_2.1.1 utf8_1.2.2
[71] caret_6.0-90 tidyselect_1.1.1 rlang_0.4.12 reshape2_1.4.4 munsell_0.5.0 tools_4.1.2 LaplacesDemon_16.1.6
[78] cachem_1.0.6 generics_0.1.1 RSQLite_2.2.8 evaluate_0.14 stringr_1.4.0 fastmap_1.1.0 mzID_1.32.0
[85] yaml_2.2.1 ModelMetrics_1.2.2.2 knitr_1.36 bit64_4.0.5 randomForest_4.6-14 purrr_0.3.4 KEGGREST_1.34.0
[92] dendextend_1.15.2 ncdf4_1.17.1 future_1.23.0 nlme_3.1-153 xml2_1.3.2 biomaRt_2.50.1 compiler_4.1.2
[99] rstudioapi_0.13 filelock_1.0.2 curl_4.3.2 png_0.1-7 e1071_1.7-9 affyio_1.64.0 tibble_3.1.6
[106] stringi_1.7.5 lattice_0.20-45 Matrix_1.3-4 vctrs_0.3.8 pillar_1.6.4 lifecycle_1.0.1 BiocManager_1.30.16
[113] GlobalOptions_0.1.2 MALDIquant_1.20 data.table_1.14.2 bitops_1.0-7 R6_2.5.1 pcaMethods_1.86.0 affy_1.72.0
[120] gridExtra_2.3 parallelly_1.29.0 codetools_0.2-18 gtools_3.9.2 MASS_7.3-54 assertthat_0.2.1 rprojroot_2.0.2
[127] withr_2.4.2 GenomeInfoDbData_1.2.7 parallel_4.1.2 hms_1.1.1 grid_4.1.2 rpart_4.1-15 timeDate_3043.102
[134] tidyr_1.1.4 coda_0.19-4 class_7.3-19 rmarkdown_2.11 segmented_1.3-4 pROC_1.18.0 lubridate_1.8.0
Let me know if you have any questions or need more info.
bandle/R/bandle-function.R
Line 91 in 0fa56ad
Hi Olly,
Not sure what is going on but I have tried to explain the issue as best as I could.
Possibly CPU issues - possibly with assigning the correct number of cores when using default
BPPARAM. On my previous laptop thebandlefunction would complete a run in ~40mins and appeared to run the default fine. When runningbandlewith the same parameters on my new laptop, with essentially the same/better specs, the code takes significantly longer (hours) and/or crashes. This seems to be resolved with the nu,ber of cores to use is explicitly specified in the function, as follows:Without specifying the
BBPARAMparameter, no progress bar shows and the CPU is working at 100% capacity. It does however appear to be using all cores - but these are all "maxed out", which causes R to crash or freeze. If I preserved potentially it may run okay but with my current laptop and a friend's laptop I borrowed this usually ended in the session crashing/freezing. When specifying as above the progress bar appears and the CPU works only at 50% capacity, uses all cores but these aren't all "maxed out".The
sessionInfoandbpparamfor my old laptop is:For my new laptop:
Let me know if you have any questions or need more info.