nthreads starts at 1, ignoring settings

[adamaltmejd]

Took me ages to figure out why my analyses were running much slower all of a sudden. Turns out nthreads started defaulting to 1 because (I still do not know why) the environment variable "R_CHECK_COMPILATION_FLAGS_KNOWN" was set on my machine validating is_r_check() here

fixest/R/onLoad.R

Line 92 in 0a03d0e

if(is_r_check()){

. This also slows down all my datatable operations (why?). Is the lookup for is_r_check() too broad?

[adamaltmejd]

Seems to be default setting in Renviron.site on Debian, so should really not pass the check.

[lrberge]

Thanks for reporting. It makes sense to change this.

One issue is that finding when the package is run from a CRAN check isn't completely straightforward, that's why I used the _R_CHECK trick which I thought was working until now.

The problem is that specific "_R_CHECK*" environment variables have no guarantee to exist from one R version to the next (I remember a long discussion in a data.table issue when CRAN issue that _R_CHECK_LIMIT_CORES was no longer maintained).

I'll try to find a more robust solution that solves your case.

[adamaltmejd]

Thanks! If its mainly used during package compilation, could one perhaps just set an env var in the build directory?

[lrberge]

No, custom env vars are ignored on CRAN.

[waynelapierre]

A similar issue may be occurring in my setup, which does not involve the use of is_r_check() . On Linux Mint (an Ubuntu-based distribution), the default number of threads used by fixest is 1, rather than half of the available threads. By contrast, data.table correctly defaults to using half of the available threads. The session information is as follows:

R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 22.2

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8      
 [2] LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                 
 [9] LC_ADDRESS=C              
[10] LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C       

time zone: America/Chicago
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils    
[5] datasets  methods   base     

other attached packages:
[1] fixest_0.13.2     data.table_1.17.8

loaded via a namespace (and not attached):
 [1] zoo_1.8-15          compiler_4.3.3     
 [3] sandwich_3.1-1      tools_4.3.3        
 [5] dreamerr_1.5.0      Rcpp_1.1.0         
 [7] nlme_3.1-168        grid_4.3.3         
 [9] Formula_1.2-5       numDeriv_2016.8-1.1
[11] stringmagic_1.2.0   lattice_0.22-7

[lrberge]

@adamaltmejd: I'll check the presence of 5+ _R_CHECK env. variables, This will solve your issue and hopefully will be more robust. (It's unlikely that Renviron.site sets more than 5 of those as a default....)

@waynelapierre: the problem looks different, could you run fixest:::cpp_get_nb_threads()? My guess is that it will return 1, meaning that openMP is not detected.

[adamaltmejd]

Awesome! Thanks.

[waynelapierre]

library(fixest)
getFixest_nthreads() shows 1
fixest:::cpp_get_nb_threads() shows 16