Error making Beagle library with CUDA option in Kaggle #204
Description
I am facing the same kind of error while trying to install in Kaggle though the installation worked fine on Google Colab.
similar kind of error was found #169
[ 70%] Linking CXX shared library libhmsbeagle-cuda.so
/usr/bin/ld: cannot find -lcuda: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/build.make:162: libhmsbeagle/GPU/CMake_CUDA/libhmsbeagle-cuda.so.40.0.0] Error 1
make[1]: *** [CMakeFiles/Makefile2:293: libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
But that was for WSL_Ubuntu_20.04 and my Windows 11 PC having AMD Ryzen 7 5700U with Radeon Graphics running WSL2 but WSL2 have no support for AMD I guess! They had support for NVIDIA CUDA on WSL2 that's why I chose kaggle.
Following the Step2 making Beagle library from scratch with these commands
git clone --depth=1 https://github.com/beagle-dev/beagle-lib.git
cd beagle-lib
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr/local -DBUILD_OPENCL=OFF -DBUILD_CUDA=ON ..
sudo make install
cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr/local -DBUILD_OPENCL=OFF -DBUILD_CUDA=ON .. gave me:
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- BEAGLE_VERSION = 4.0.0
-- BEAGLE_PLUGIN_VERSION = 40
-- Found JNI: /usr/lib/jvm/java-11-openjdk-amd64/lib/libjawt.so
-- JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
-- JNI_INCLUDE_DIRS=/usr/lib/jvm/java-11-openjdk-amd64/include;/usr/lib/jvm/java-11-openjdk-amd64/include/linux;/usr/lib/jvm/java-11-openjdk-amd64/include
-- JNI_LIBRARIES=/usr/lib/jvm/java-11-openjdk-amd64/lib/libjawt.so;/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - Success
-- Not using libtools for plugins
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: /usr/local/cuda/lib64/libOpenCL.so (found version "2.2")
-- Found CUDA: /opt/conda (found version "12.1")
-- CUDA Includes: /opt/conda/include
-- Configuring done
-- Generating done
-- Build files have been written to: /kaggle/working/beagle-lib/build
sudo make install gave me:
[ 4%] Building CXX object libhmsbeagle/CMakeFiles/hmsbeagle.dir/beagle.cpp.o
[ 8%] Building CXX object libhmsbeagle/CMakeFiles/hmsbeagle.dir/benchmark/BeagleBenchmark.cpp.o
[ 12%] Building CXX object libhmsbeagle/CMakeFiles/hmsbeagle.dir/benchmark/linalg.cpp.o
[ 16%] Building CXX object libhmsbeagle/CMakeFiles/hmsbeagle.dir/plugin/Plugin.cpp.o
[ 20%] Building CXX object libhmsbeagle/CMakeFiles/hmsbeagle.dir/plugin/UnixSharedLibrary.cpp.o
[ 25%] Linking CXX shared library libhmsbeagle.so
[ 25%] Built target hmsbeagle
[ 29%] Building CXX object libhmsbeagle/CPU/CMakeFiles/hmsbeagle-cpu.dir/BeagleCPUPlugin.cpp.o
[ 33%] Linking CXX shared library libhmsbeagle-cpu.so
[ 33%] Built target hmsbeagle-cpu
[ 37%] Building CXX object libhmsbeagle/CPU/CMakeFiles/hmsbeagle-cpu-sse.dir/BeagleCPUSSEPlugin.cpp.o
In file included from /kaggle/working/beagle-lib/libhmsbeagle/CPU/BeagleCPU4StateSSEImpl.h:338,
from /kaggle/working/beagle-lib/libhmsbeagle/CPU/BeagleCPUSSEPlugin.cpp:9:
/kaggle/working/beagle-lib/libhmsbeagle/CPU/BeagleCPU4StateSSEImpl.hpp: In member function ‘virtual void beagle::cpu::BeagleCPU4StateSSEImpl<double, T_PAD, P_PAD>::calcCrossProductsPartials(const double*, const double*, const double*, const double*, double, double*, double*)’:
/kaggle/working/beagle-lib/libhmsbeagle/CPU/BeagleCPU4StateSSEImpl.hpp:563:25: warning: ignoring attributes on template argument ‘V_Real’ {aka ‘__m128d’} [�]8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wignored-attributes�-Wignored-attributes�]8;;�]
563 | std::array<V_Real, 8> vAcrossPatterns;
| ^
/kaggle/working/beagle-lib/libhmsbeagle/CPU/BeagleCPU4StateSSEImpl.hpp:566:25: warning: ignoring attributes on template argument ‘V_Real’ {aka ‘__m128d’} [�]8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wignored-attributes�-Wignored-attributes�]8;;�]
566 | std::array<V_Real, 8> vWithinPattern;
| ^
[ 41%] Linking CXX shared library libhmsbeagle-cpu-sse.so
[ 41%] Built target hmsbeagle-cpu-sse
[ 45%] Building CUDA kernels
NVCC=/opt/conda/bin/nvcc
NVCCFLAGS=-O3 -Wno-deprecated-gpu-targets -ccbin /usr/bin/c++ -m64 -D_POSIX_C_SOURCE -std=c++11
INCLUDE_DIRS=-I/kaggle/working/beagle-lib
Making CUDA SP state count = 16
Making CUDA SP state count = 32
Making CUDA SP state count = 48
Making CUDA SP state count = 64
Making CUDA SP state count = 80
Making CUDA SP state count = 128
Making CUDA SP state count = 192
Making CUDA SP state count = 256
Making CUDA DP state count = 16
Making CUDA DP state count = 32
Making CUDA DP state count = 48
Making CUDA DP state count = 64
Making CUDA DP state count = 80
Making CUDA DP state count = 128
Making CUDA DP state count = 192
Making CUDA DP state count = 256
[ 45%] Built target CudaKernels
[ 50%] Building CXX object libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/__/GPUImplHelper.cpp.o
[ 54%] Building CXX object libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/__/GPUInterfaceCUDA.cpp.o
[ 58%] Building CXX object libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/__/KernelLauncher.cpp.o
[ 62%] Building CXX object libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/__/KernelResource.cpp.o
[ 66%] Building CXX object libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/__/CUDAPlugin.cpp.o
[ 70%] Linking CXX shared library libhmsbeagle-cuda.so
/usr/bin/ld: cannot find -lcuda: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/build.make:162: libhmsbeagle/GPU/CMake_CUDA/libhmsbeagle-cuda.so.40.0.0] Error 1
make[1]: *** [CMakeFiles/Makefile2:293: libhmsbeagle/GPU/CMake_CUDA/CMakeFiles/hmsbeagle-cuda.dir/all] Error 2
As the library building was incomplete the !make test failed as expected:
Running tests...
Test project /kaggle/working/beagle-lib/build
Start 1: hmctest
Could not find executable hmctest
Looked in the following places:
hmctest
hmctest
Release/hmctest
Release/hmctest
Debug/hmctest
Debug/hmctest
MinSizeRel/hmctest
MinSizeRel/hmctest
RelWithDebInfo/hmctest
RelWithDebInfo/hmctest
Deployment/hmctest
Deployment/hmctest
Development/hmctest
Development/hmctest
Unable to find executable: hmctest
1/1 Test #1: hmctest ..........................***Not Run 0.00 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.00 sec
The following tests FAILED:
1 - hmctest (Not Run)
Errors while running CTest
Output from these tests are in: /kaggle/working/beagle-lib/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make: *** [Makefile:91: test] Error 8
!lsb_release -a output:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy
!lscpu output:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) CPU @ 2.00GHz
CPU family: 6
Model: 85
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
Stepping: 3
BogoMIPS: 4000.28
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mc
a cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscal
l nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopo
logy nonstop_tsc cpuid tsc_known_freq pni pclmulqdq sss
e3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes
xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefe
tch invpcid_single pti ssbd ibrs ibpb stibp fsgsbase ts
c_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx a
vx512f avx512dq rdseed adx smap clflushopt clwb avx512c
d avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat
md_clear arch_capabilities
Virtualization features:
Hypervisor vendor: KVM
Virtualization type: full
Caches (sum of all):
L1d: 32 KiB (1 instance)
L1i: 32 KiB (1 instance)
L2: 1 MiB (1 instance)
L3: 38.5 MiB (1 instance)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0,1
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Mitigation; PTE Inversion
Mds: Mitigation; Clear CPU buffers; SMT Host state unknown
Meltdown: Mitigation; PTI
Mmio stale data: Vulnerable: Clear CPU buffers attempted, no microcode;
SMT Host state unknown
Retbleed: Mitigation; IBRS
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
and seccomp
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer
sanitization
Spectre v2: Mitigation; IBRS, IBPB conditional, STIBP conditional,
RSB filling, PBRSB-eIBRS Not affected
Srbds: Not affected
Tsx async abort: Mitigation; Clear CPU buffers; SMT Host state unknown
!nvidia-smi output:
Fri Sep 1 13:35:09 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03 Driver Version: 470.161.03 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 30C P0 26W / 250W | 0MiB / 16280MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
I appreciate anyone guide me so that I can successfully install Beagle library with CUDA option in Kaggle. The intentiom is to run BEAST2 with beagle_GPU option.