compilation error with Intel oneAPI 2022

[bm-pw]

I attempted to build the CEA with only FORTRAN and C bindings using Intel oneAPI 2022 on Red Hat Enterprise Linux 8. The build failed because of these errors:

[ 34%] Built target fbasics_core
[ 37%] Building Fortran object source/CMakeFiles/cea_core.dir/mixture.f90.o
/home/e10017898/dev/cea/source/mixture.f90(171): error #6303: The assignment operation or the binary expression operation is invalid for the data types of the two operands.   [THERMO]
                reactants = Mixture(thermo, reactant_names, ions=ions)
------------------------------------^
/home/e10017898/dev/cea/source/mixture.f90(171): error #8001: The type of the component in a structure-constructor differs from the type of the component in the derived-type-def.   [THERMO]
                reactants = Mixture(thermo, reactant_names, ions=ions)
------------------------------------^
/home/e10017898/dev/cea/source/mixture.f90(171): error #6303: The assignment operation or the binary expression operation is invalid for the data types of the two operands.   [REACTANT_NAMES]
                reactants = Mixture(thermo, reactant_names, ions=ions)
--------------------------------------------^
/home/e10017898/dev/cea/source/mixture.f90(171): error #8001: The type of the component in a structure-constructor differs from the type of the component in the derived-type-def.   [REACTANT_NAMES]
                reactants = Mixture(thermo, reactant_names, ions=ions)
--------------------------------------------^
/home/e10017898/dev/cea/source/mixture.f90(171): error #6594: The rank of a component in a structure constructor differs from the rank of the component of the derived type.   [REACTANT_NAMES]
                reactants = Mixture(thermo, reactant_names, ions=ions)
--------------------------------------------^
/home/e10017898/dev/cea/source/mixture.f90(171): error #6594: The rank of a component in a structure constructor differs from the rank of the component of the derived type.
                reactants = Mixture(thermo, reactant_names, ions=ions)
^
compilation aborted for /home/e10017898/dev/cea/source/mixture.f90 (code 1)

[markleader]

@bm-pw I'll try to get you squared away.

Can you please share:

• the command you used to compile right before this output
• the CMake configure output summary showing which Fortran compiler is being used
• the output from ifx --version

[bm-pw]

@markleader

I added a new preset to CMakePresets.json:

    {
      "name": "core-pw",
      "inherits": "intel",
      "displayName": "Fortran + C + Cpp)",
      "description": "Fortran, C, C++ build without Python bindings.",
      "binaryDir": "build-core-pw",
      "cacheVariables": {
        "CMAKE_BUILD_TYPE": "Release",
        "CEA_ENABLE_BIND_C": "TRUE",
        "CEA_ENABLE_BIND_CXX": "TRUE",
        "CEA_ENABLE_BIND_PYTHON": "FALSE",
        "CEA_ENABLE_BIND_MATLAB": "FALSE",
        "CEA_ENABLE_BIND_EXCEL": "FALSE",
        "CEA_BUILD_TESTING": "FALSE"
      }
    },

So I am using the classic Fortran compiler, ifort.

My build commands:
% cmake --preset core-pw
% cmake --build build-core-pw

Output from running preset:

Preset CMake variables:

  CEA_BUILD_TESTING="FALSE"
  CEA_ENABLE_BIND_C="TRUE"
  CEA_ENABLE_BIND_CXX="TRUE"
  CEA_ENABLE_BIND_EXCEL="FALSE"
  CEA_ENABLE_BIND_MATLAB="FALSE"
  CEA_ENABLE_BIND_PYTHON="FALSE"
  CMAKE_BUILD_TYPE="Release"
  CMAKE_CXX_COMPILER="icpc"
  CMAKE_C_COMPILER="icc"
  CMAKE_Fortran_COMPILER="ifort"
  CMAKE_Fortran_FLAGS="-std08"

-- The Fortran compiler identification is Intel 2021.5.0.20211109
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /pw/test/linux/intel/Compiler/2022.1/compiler/2022.0.2/linux/bin/intel64/ifort - skipped
-- Setting install path to '/home/e10017898/dev/cea/build-core-pw/install' since none specified.
-- The C compiler identification is Intel 2021.5.0.20211109
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /pw/test/linux/intel/Compiler/2022.1/compiler/2022.0.2/linux/bin/intel64/icc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done (6.5s)
-- Generating done (0.5s)
-- Build files have been written to: /home/e10017898/dev/cea/build-core-pw

Regarding the Fortran compilers,

hpclogin-nw31 e10017898 34% ifort --version
ifort (IFORT) 2021.5.0 20211109
Copyright (C) 1985-2021 Intel Corporation.  All rights reserved.

hpclogin-nw31 e10017898 35% ifx --version
ifx (IFORT) 2022.0.0 20211123
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

[markleader]

@bm-pw Try using your ifx compiler instead, using this build command:

cmake -S . -B build-core-intel \ -D CMAKE_Fortran_COMPILER=ifx \ -D CMAKE_C_COMPILER=icx \ -D CMAKE_CXX_COMPILER=icpx \ -D CEA_ENABLE_BIND_C=TRUE \ -D CEA_ENABLE_BIND_CXX=FALSE \ -D CEA_ENABLE_BIND_PYTHON=FALSE \ -D CEA_ENABLE_BIND_MATLAB=FALSE \ -D CEA_ENABLE_BIND_EXCEL=FALSE \ -D CEA_BUILD_TESTING=FALSE
cmake --build build-core-intel

[bm-pw]

@markleader I got the same compilation error with ifx.
Line#170 of Mixture.f90 reads:
reactants = Mixture(thermo, reactant_names, ions=ions)

Which constructor is supposed to be called? The default or the custom defined? If I change the code to call the custom defined constructor, the code compiles:
reactants = mixture_init(thermo=thermo, species_names=reactant_names, ions=ions)
But the executable seg-faults on the test cases.

[markleader]

@bm-pw It should call the custom constructor, but Intel is treating it as a structure constructor. Try compiling with these flags (but revert any changes you made to mixture.f90 first):

cmake -S . -B build-core-intel \ -D CMAKE_Fortran_COMPILER=ifx \ -D CMAKE_C_COMPILER=icx \ -D CMAKE_Fortran_FLAGS="-stand f08" \ -D CEA_ENABLE_BIND_C=TRUE \ -D CEA_ENABLE_BIND_CXX=FALSE \ -D CEA_ENABLE_BIND_PYTHON=FALSE \ -D CEA_ENABLE_BIND_MATLAB=FALSE \ -D CEA_ENABLE_BIND_EXCEL=FALSE \ -D CEA_BUILD_TESTING=FALSE
cmake --build build-core-intel

[bm-pw]

@markleader

'-standf08' is not a valid option. I changed it to '-std08'. I still got the same compilation error with ifx in oneAPI 2022.

But I got hold of oneAPI 2024. The code compiles ok with ifx. There were warnings about redundant ')' in main.f90, on line #2484 and #2497:

/home/e10017898/dev/cea/source/main.f90(2497): warning #8043: The extra characters in the format specification will be ignored.   ['(3(F13.5)))']
                                        write(ioout, '(3(F13.5)))') (solutions(i, j, k)%eq_soln(idx)%mole_fractions(ii), idx=x,y)
----------------------------------------------------------------^

The warnings went away after removing the extra ')'.

Unfortunately, cea still seg-faults:

hpclogin-nw31 e10017898 58% pwd
/home/e10017898/dev/cea/samples
hpclogin-nw31 e10017898 59% ../build-core-intel/install/bin/cea example1
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
libpthread-2.28.s  00007FCD91570990  Unknown               Unknown  Unknown
cea                000000000047D69C  Unknown               Unknown  Unknown
cea                000000000040A54A  Unknown               Unknown  Unknown
cea                0000000000407BB5  Unknown               Unknown  Unknown
cea                00000000004045CD  Unknown               Unknown  Unknown
libc-2.28.so       00007FCD911C17E5  __libc_start_main     Unknown  Unknown
cea                00000000004044EE  Unknown               Unknown  Unknown

[markleader]

@bm-pw Can you run that again with the -d flag (cea -d input_file)? That will help me see where the segfault is happening.
Also - can you try running one of the compiled c examples? They are located in <build>\source\bind\c. Just try any one of those examples and let me know if they segfault as well.

[bm-pw]

@markleader

Here is the debug run:

hpclogin-nw31 e10017898 61% ../build-core-intel/install/bin/cea -d example1
INFO: Reading input_file from first positional argument
INFO: CEA Version: 3.0.0
INFO: Input File:  example1
INFO: Thermo File: thermo.lib
INFO: Reading transport database file: trans.lib
INFO: Parsing input file: example1.inp
INFO: Parsing problem specification 1
DEBUG: Parsing dataset prob
DEBUG: Parsing prob literal case
DEBUG: Parsing prob literal tp
DEBUG: Parsing prob literal p(atm)
DEBUG: Parsing prob literal t(k)
DEBUG: Parsing prob literal r
DEBUG: Parsing dataset reac
DEBUG: Parsing parameters for reactant H2
DEBUG: Parsing parameters for reactant Air
DEBUG: Parsing dataset only
DEBUG: Parsed 20 species from only
DEBUG: Parsing dataset outp
DEBUG: Parsing prob literal calories
INFO: Parsing problem specification 2
INFO: No problem found. Reached EOF.
INFO: Parsed 1 problems from example1.inp
INFO: Reading thermo database file: thermo.lib
INFO: Reading 1262 gaseous products
INFO: Reading 991 condensed products
INFO: Reading 64 reactants
INFO: Building list of unique species
INFO: Found 2012 unique species
INFO: Building list of unique species
INFO: Found 62 unique species
INFO: Building list of unique elements
INFO: Found 56 unique elements
INFO: Executing problem 1: name=Example-1, type=tp
INFO: Solving equilibrium problem:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
libpthread-2.28.s  00007FFBB50C7990  Unknown               Unknown  Unknown
cea                000000000047D69C  Unknown               Unknown  Unknown
cea                000000000040A54A  Unknown               Unknown  Unknown
cea                0000000000407BB5  Unknown               Unknown  Unknown
cea                00000000004045CD  Unknown               Unknown  Unknown
libc-2.28.so       00007FFBB4D187E5  __libc_start_main     Unknown  Unknown
cea                00000000004044EE  Unknown               Unknown  Unknown

I don't see any executable in build-core-intel/source/bin/c:

hpclogin-nw31 e10017898 74% pwd
/home/e10017898/dev/cea/build-core-intel/source/bind/c
hpclogin-nw31 e10017898 75% ll
total 1772
drwxr-xr-x 4 e10017898 unit2000     105 Jan 13 16:35 ./
drwxr-xr-x 4 e10017898 unit2000      76 Jan 13 16:35 ../
drwxr-xr-x 3 e10017898 unit2000      88 Jan 13 16:35 CMakeFiles/
-rw-r--r-- 1 e10017898 unit2000    2773 Jan 13 16:35 cmake_install.cmake
-rwxr-xr-x 1 e10017898 unit2000 1799192 Jan 13 16:35 libcea_bindc.so*
-rw-r--r-- 1 e10017898 unit2000    8000 Jan 13 16:35 Makefile
drwxr-xr-x 2 e10017898 unit2000      27 Jan 13 16:35 modules/

[bm-pw]

@markleader
When I enabled CEA_BUILD_TESTING, I got this error:

CMake Error at CMakeLists.txt:50 (find_package):
  By not providing "FindPFUNIT.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "PFUNIT", but
  CMake did not find one.

  Could not find a package configuration file provided by "PFUNIT" with any
  of the following names:

    PFUNITConfig.cmake
    pfunit-config.cmake

  Add the installation prefix of "PFUNIT" to CMAKE_PREFIX_PATH or set
  "PFUNIT_DIR" to a directory containing one of the above files.  If "PFUNIT"
  provides a separate development package or SDK, be sure it has been
  installed.

Looks like I don't have pFUnit on my system.

[markleader]

@bm-pw This seems like an issue with the thermo.lib and trans.lib files. Can you try re-generating them so that they are consistent with your intel build?

# from the repo root (or any working dir)
../build-core-intel/install/bin/cea --compile-thermo data/thermo.inp
../build-core-intel/install/bin/cea --compile-trans data/trans.inp

# run using the newly generated libs in the current dir
../build-core-intel/install/bin/cea -d -t ./thermo.lib -r ./trans.lib example1

and make sure there isn't another thermo.lib/trans.lib in the working directory that could get picked up. If that doesn't work, please recompile with stronger runtime checks and full traceback:

cmake -S . -B build-core-intel-debug \
  -D CMAKE_BUILD_TYPE=Debug \
  -D CMAKE_Fortran_COMPILER=ifx \
  -D CMAKE_C_COMPILER=icx \
  -D CMAKE_Fortran_FLAGS="-g -O0 -traceback -check all -fpe0 -ftrapuv -warn all -debug all" \
  -D CEA_ENABLE_BIND_C=TRUE \
  -D CEA_ENABLE_BIND_CXX=FALSE \
  -D CEA_ENABLE_BIND_PYTHON=FALSE \
  -D CEA_ENABLE_BIND_MATLAB=FALSE \
  -D CEA_ENABLE_BIND_EXCEL=FALSE
cmake --build build-core-intel-debug

and then re-run in debug again and let me know what happens.

Thanks for your patience with this - feel free to email me directly if this doesn't work.

[bm-pw]

@markleader
It still crashes with regenerated thermo.lib and trans.lib.

With the debug build, cea crashes even earlier:

==2210447==ERROR: MemorySanitizer failed to allocate 0xf0000000000 (16492674416640) bytes at address 10000000000 (errno: 12)
FATAL: MemorySanitizer can not mmap the shadow memory.
FATAL: Make sure to compile with -fPIE and to link with -pie.
FATAL: Disabling ASLR is known to cause this error.
FATAL: If running under GDB, try 'set disable-randomization off'.
==2210447==Process memory map follows:
        0x000000400000-0x000001603000   /home/e10017898/dev/cea/build-dev-intel/install/bin/cea
        0x000001803000-0x000001805000   /home/e10017898/dev/cea/build-dev-intel/install/bin/cea
        0x000001805000-0x000001860000   /home/e10017898/dev/cea/build-dev-intel/install/bin/cea
        0x000001860000-0x000003244000
        0x7f22d8800000-0x7f22d8900000
        0x7f22d8a00000-0x7f22d8b00000
        0x7f22d8c00000-0x7f22d8d00000
        0x7f22d8e00000-0x7f22d8f00000
        0x7f22d9000000-0x7f22d9100000
        0x7f22d9199000-0x7f22d9527000
        0x7f22d9527000-0x7f22d953e000   /usr/lib64/libgcc_s-8-20210514.so.1
        0x7f22d953e000-0x7f22d973d000   /usr/lib64/libgcc_s-8-20210514.so.1
        0x7f22d973d000-0x7f22d973e000   /usr/lib64/libgcc_s-8-20210514.so.1
        0x7f22d973e000-0x7f22d973f000   /usr/lib64/libgcc_s-8-20210514.so.1
        0x7f22d973f000-0x7f22d990c000   /usr/lib64/libc-2.28.so
        0x7f22d990c000-0x7f22d9b0c000   /usr/lib64/libc-2.28.so
        0x7f22d9b0c000-0x7f22d9b10000   /usr/lib64/libc-2.28.so
        0x7f22d9b10000-0x7f22d9b12000   /usr/lib64/libc-2.28.so
        0x7f22d9b12000-0x7f22d9b16000
        0x7f22d9b16000-0x7f22d9b19000   /usr/lib64/libdl-2.28.so
        0x7f22d9b19000-0x7f22d9d18000   /usr/lib64/libdl-2.28.so
        0x7f22d9d18000-0x7f22d9d19000   /usr/lib64/libdl-2.28.so
        0x7f22d9d19000-0x7f22d9d1a000   /usr/lib64/libdl-2.28.so
        0x7f22d9d1a000-0x7f22d9d35000   /usr/lib64/libpthread-2.28.so
        0x7f22d9d35000-0x7f22d9f34000   /usr/lib64/libpthread-2.28.so
        0x7f22d9f34000-0x7f22d9f35000   /usr/lib64/libpthread-2.28.so
        0x7f22d9f35000-0x7f22d9f36000   /usr/lib64/libpthread-2.28.so
        0x7f22d9f36000-0x7f22d9f3a000
        0x7f22d9f3a000-0x7f22da0bb000   /usr/lib64/libm-2.28.so
        0x7f22da0bb000-0x7f22da2ba000   /usr/lib64/libm-2.28.so
        0x7f22da2ba000-0x7f22da2bb000   /usr/lib64/libm-2.28.so
        0x7f22da2bb000-0x7f22da2bc000   /usr/lib64/libm-2.28.so
        0x7f22da2bc000-0x7f22da2eb000   /usr/lib64/ld-2.28.so
        0x7f22da4c5000-0x7f22da4d6000
        0x7f22da4d6000-0x7f22da4eb000
        0x7f22da4eb000-0x7f22da4ec000   /usr/lib64/ld-2.28.so
        0x7f22da4ec000-0x7f22da4ee000   /usr/lib64/ld-2.28.so
        0x7ffe53f5e000-0x7ffe53f80000   [stack]
        0x7ffe53ff5000-0x7ffe53ff9000   [vvar]
        0x7ffe53ff9000-0x7ffe53ffb000   [vdso]
        0xffffffffff600000-0xffffffffff601000   [vsyscall]
==2210447==End of process memory map.

Maybe this is compiler and platform related. Which version of Intel compiler was used to build cea? On which Linux platform? I am on Red Hat Enterprise Linux 8.

[markleader]

This is definitely compiler/platform related. I am running on a Mac using GCC compilers. I was working with a co-worker this morning who uses oneAPI and he is getting similar errors, so it seems specific to intel.

That error looks like MemorySanitizer failing to start rather than a CEA error (yet) which is why the error is happening earlier now. From a clean build-dir, rebuild in debug mode without sanitizers so we can see the Fortran traceback:

cmake -S . -B build-dev-intel \
  -D CMAKE_BUILD_TYPE=Debug \
  -D CMAKE_Fortran_COMPILER=ifx \
  -D CMAKE_C_COMPILER=icx \
  -D CMAKE_Fortran_FLAGS_DEBUG="-g -O0 -traceback -check all -fpe0 -warn all"
cmake --build build-dev-intel

then re-run the example again in debug mode and it should give the line number where the segfault occurs.

[bm-pw]

@markleader
The MemorySanitizer must be triggered by the '-check all' option in debug build. I set the Fortran debug flags to: '-g -O0 -warn all -traceback -fpe0'. Now the output looks like:

hpclogin-nw31 e10017898 78% ../build-dev-intel/install/bin/cea -d example1
INFO: Reading input_file from first positional argument
INFO: CEA Version: 3.0.0
INFO: Input File:  example1
INFO: Thermo File: thermo.lib
INFO: Reading transport database file: trans.lib
INFO: Parsing input file: example1.inp
INFO: Parsing problem specification 1
DEBUG: Parsing dataset prob
DEBUG: Parsing prob literal case
DEBUG: Parsing prob literal tp
DEBUG: Parsing prob literal p(atm)
DEBUG: Parsing prob literal t(k)
DEBUG: Parsing prob literal r
DEBUG: Parsing dataset reac
DEBUG: Parsing parameters for reactant H2
DEBUG: Parsing parameters for reactant Air
DEBUG: Parsing dataset only
DEBUG: Parsed 20 species from only
DEBUG: Parsing dataset outp
DEBUG: Parsing prob literal calories
INFO: Parsing problem specification 2
INFO: No problem found. Reached EOF.
INFO: Parsed 1 problems from example1.inp
INFO: Reading thermo database file: thermo.lib
INFO: Reading 1262 gaseous products
INFO: Reading 991 condensed products
INFO: Reading 64 reactants
INFO: Building list of unique species
INFO: Found 2012 unique species
INFO: Building list of unique species
INFO: Found 62 unique species
INFO: Building list of unique elements
INFO: Found 56 unique elements
INFO: Executing problem 1: name=Example-1, type=tp
INFO: Solving equilibrium problem:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
libpthread-2.28.s  00007F8A782E1990  Unknown               Unknown  Unknown
cea                00000000004B715B  mixture_init              259  mixture.f90
cea                000000000040E03D  run_thermo_proble         245  main.f90
cea                00000000004092D9  cea                        92  main.f90
cea                00000000004045AD  Unknown               Unknown  Unknown
libc-2.28.so       00007F8A77D2E7E5  __libc_start_main     Unknown  Unknown
cea                00000000004044CE  Unknown               Unknown  Unknown

Looks like cea crashed when calling the recursive Mixture constructor. Maybe it gets into an infinite loop.

[markleader]

@bm-pw This is helpful output - thanks. I should have a fix for you to try tomorrow.