UFS having problem using spack-stack@1.9.1 installed SCOTCH@7.0.7 #1608
Description
Describe the bug
When running UFS-WM cpld_debug_gfsv17 test, model is failing if using spack-stack installed SCOTCH@7.0.7 library.
If we run with manually installed SCOTCH, model is working OK.
To Reproduce
Steps to reproduce the behavior:
- Clone https://github.com/RatkoVasic-NOAA/ufs-weather-model.git, and checkout branch ss-191
- Make necessary change in WW3/model/src/PDLIB/yowpdlibmain.F90, line 629:
- call SCOTCH_ParMETIS_V3_PartGeomKway(vtxdist, xadj, adjncy, &
+ call SCOTCHFParMETIS_V3_PartGeomKway(vtxdist, xadj, adjncy, &
- change scotch version in modulefiles/ufs_common.lua from 7.0.4 to 7.0.7
- add scotch path in modulefiles/ufs_hera.intel.lua
prepend_path("MODULEPATH", "/contrib/spack-stack/modulefiles") - run test:
./rt.sh -a epic -r -k -c -l test-scotch.conf, where test-scotch.conf should be like:
COMPILE | s2swa_32bit_pdlib_debug | intel | -DAPP=S2SWA -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON -DDEBUG=ON | - noaacloud jet | fv3 |
RUN | cpld_debug_gfsv17 | - noaacloud jet | baseline |
This is running OK.
6. replace path to SCOTCH in modulefiles/ufs_hera.intel.lua with spack installed scotch:
prepend_path("MODULEPATH", "/contrib/spack-stack/spack-stack-1.9.1/envs/scotch-7.0.7/install/modulefiles/intel-oneapi-mpi/2021.13-sbi3u54/oneapi/2024.2.1")
- Repeat step 5.
This one is failing.
Expected behavior
Model should pass in both cases (steps 5. and 7. from above).
System:
Tested on Hera.
Additional context
scotch@7.0.7 was installed like this on Hera:
wget https://gitlab.inria.fr/scotch/scotch/-/archive/v7.0.7/scotch-v7.0.7.tar.gz
tar xvf scotch-v7.0.7.tar.gz
cd scotch-v7.0.7
mkdir build
cd build
module load intel/2024.2.1
module load impi/2024.2.1
cmake -DCMAKE_INSTALL_PREFIX=/contrib/spack-stack/installs/scotch/7.0.7 -DCMAKE_BUILD_TYPE=Release ..
make
make install