Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
6c022f9
add fix for hdf5 2.0.0
Yushan-Wang Mar 20, 2026
4e88161
Fix MPI condition for HDF5 parallel support
Yushan-Wang Apr 9, 2026
b342921
add test for precision conversion
Yushan-Wang Apr 14, 2026
28e3989
Add 'd2s_precision' to ALL_TEST_NAMES
Yushan-Wang Apr 14, 2026
3c39885
Remove parallel HDF5 version check
Yushan-Wang Apr 14, 2026
dfa8769
Fix HDF5 parallel check in FindNetCDF.cmake
Yushan-Wang Apr 14, 2026
94997ce
Refactor HDF5 version checks in CMakeLists
Yushan-Wang Apr 14, 2026
b980a9d
fix indent
Yushan-Wang Apr 14, 2026
bcdccf2
Merge branch 'precision-conversion-with-hdf5' of https://github.com/Y…
Yushan-Wang Apr 14, 2026
2774595
update decl_hdf5/CHANGELOG.md
Yushan-Wang Apr 14, 2026
9e51956
add cmake fix for version less than 4.3
Yushan-Wang Apr 14, 2026
071dff9
Add version checks for HDF5 in CMakeLists.txt
Yushan-Wang Apr 14, 2026
6c5ceaa
Typo [skip ci]
Yushan-Wang Apr 14, 2026
1c5c227
Merge branch 'patch-hdf5-2.0.0' of https://github.com/Yushan-Wang/pdi…
Yushan-Wang Apr 14, 2026
2c2c58b
add fix for hdf5 2.0.0
Yushan-Wang Mar 20, 2026
ba0aed4
Fix MPI condition for HDF5 parallel support
Yushan-Wang Apr 9, 2026
5150802
add cmake fix for version less than 4.3
Yushan-Wang Apr 14, 2026
1dcb3c2
Typo [skip ci]
Yushan-Wang Apr 14, 2026
b01bb1b
Add version checks for HDF5 in CMakeLists.txt
Yushan-Wang Apr 14, 2026
bd034c8
Remove redundant CMAKE_MODULE_PATH entry
Yushan-Wang Apr 14, 2026
3c5f4bc
Merge branch 'patch-hdf5-2.0.0' of https://github.com/Yushan-Wang/pdi…
Yushan-Wang Apr 15, 2026
106bfbf
remove 4.1 cmake patch
Yushan-Wang Apr 15, 2026
eeab5c1
Update CMakeLists.txt
Yushan-Wang Apr 15, 2026
6a33909
Update plugins/decl_hdf5/CMakeLists.txt
Yushan-Wang Apr 15, 2026
7ac0058
Update plugins/decl_netcdf/CMakeLists.txt
Yushan-Wang Apr 15, 2026
ff9f88b
Apply suggestions from code review
jbigot Apr 15, 2026
73cd0fc
Apply suggestion from @jbigot
jbigot Apr 15, 2026
8678b69
Apply suggestions from code review
jbigot Apr 15, 2026
7a863c4
Apply suggestions from code review
jbigot Apr 15, 2026
169bdad
Fix compatibility with cmake pre 3.29
jbigot Apr 15, 2026
f3d585a
Fix compatibility with cmake pre 3.25
jbigot Apr 15, 2026
6d26bee
Merge branch 'main' into patch-hdf5-2.0.0
Yushan-Wang Apr 15, 2026
c9817dd
update changelog
Yushan-Wang Apr 15, 2026
f7794cc
Merge branch 'main' into patch-hdf5-2.0.0
Yushan-Wang Apr 15, 2026
fe561e8
Merge branch 'main' into patch-hdf5-2.0.0
Yushan-Wang Apr 15, 2026
b43a1c7
update changelogs and spack.yaml
Yushan-Wang Apr 15, 2026
bf2cdc1
Merge branch 'patch-hdf5-2.0.0' of https://github.com/Yushan-Wang/pdi…
Yushan-Wang Apr 15, 2026
b842376
add fix for hdf5 2.0.0
Yushan-Wang Mar 20, 2026
fc361bb
Fix MPI condition for HDF5 parallel support
Yushan-Wang Apr 9, 2026
7b288e9
add test for precision conversion
Yushan-Wang Apr 14, 2026
97edfe9
fix indent
Yushan-Wang Apr 14, 2026
adc2e8d
Add 'd2s_precision' to ALL_TEST_NAMES
Yushan-Wang Apr 14, 2026
f78960d
Remove parallel HDF5 version check
Yushan-Wang Apr 14, 2026
6d24ef8
Fix HDF5 parallel check in FindNetCDF.cmake
Yushan-Wang Apr 14, 2026
21df9af
Refactor HDF5 version checks in CMakeLists
Yushan-Wang Apr 14, 2026
121fa7b
update decl_hdf5/CHANGELOG.md
Yushan-Wang Apr 14, 2026
8ecb3a5
Merge branch 'precision-conversion-with-hdf5' of https://github.com/Y…
Yushan-Wang Apr 15, 2026
595124d
Merge branch 'patch-hdf5-2.0.0' into precision-conversion-with-hdf5
Yushan-Wang Apr 15, 2026
4db4f55
update precision conversion test using new testing.h
Yushan-Wang Apr 16, 2026
a83ab3b
Merge branch 'main' into precision-conversion-with-hdf5
Yushan-Wang Apr 16, 2026
450cdf8
Update precision conversion test documentation and remove unused incl…
Yushan-Wang Apr 16, 2026
0e1f68c
Merge branch 'precision-conversion-with-hdf5' of https://github.com/Y…
Yushan-Wang Apr 16, 2026
6ab160c
Remove unused includes from decl_hdf5_precision_conversion_test.cxx
Yushan-Wang Apr 16, 2026
f12d601
Apply suggestions from code review
Yushan-Wang Apr 16, 2026
20a8ab1
compare the results using hdf5 read.
Yushan-Wang Apr 16, 2026
5e52643
Refactor precision conversion tests to use template functions for com…
Yushan-Wang Apr 17, 2026
dcfdf07
Merge branch 'main' into precision-conversion-with-hdf5
Yushan-Wang Apr 17, 2026
cec1507
Apply suggestions from code review
Yushan-Wang Apr 17, 2026
ddedac5
fix some typo and add gtest style d2d and f2d checking
Yushan-Wang Apr 17, 2026
4180b15
remain: int2double checking
Yushan-Wang Apr 17, 2026
d493808
indent
Yushan-Wang Apr 17, 2026
fd37e5a
add status checking for hdf5 read
Yushan-Wang Apr 17, 2026
5cdd046
one possible way to compare int and double
Yushan-Wang Apr 17, 2026
8a01504
fix: correct lambda parameter types in int to double comparison
Yushan-Wang Apr 17, 2026
d372511
attempt for fixing CI faileur with spack oldest
Yushan-Wang Apr 17, 2026
f60cb25
indent
Yushan-Wang Apr 17, 2026
7102952
refactor: update HDF5 precision conversion test in a more compact way
Yushan-Wang Apr 20, 2026
5dd30bb
skip precision conversion test when clang15
Yushan-Wang Apr 20, 2026
717cbc6
avoid compiling precision conversion test with clang15
Yushan-Wang Apr 20, 2026
4416895
skip only numerical comparaison with clang15
Yushan-Wang Apr 21, 2026
54ac0f0
empty line at eof
Yushan-Wang Apr 21, 2026
3dc95ad
Merge branch 'main' into precision-conversion-with-hdf5
Yushan-Wang Apr 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ and this project adheres to
### For users

#### Added
* Add tests for HDF5 precision conversion

#### Changed

Expand Down
4 changes: 4 additions & 0 deletions plugins/decl_hdf5/tests/compatibility_tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,7 @@ foreach(test_name ${ALL_TEST_NAMES})
endforeach(read_prefix)
endforeach(write_prefix)
endforeach(test_name)

add_executable(decl_hdf5_conversion_test decl_hdf5_precision_conversion_test.cxx)
target_link_libraries(decl_hdf5_conversion_test ${HDF5_DEPS} PDI::PDI_C GTest::gmock GTest::gmock_main GTest::gtest GTest::gtest_main)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do GTest::gtest_main and GTest::gmock_main overlink ? Two other files of the repo also use this

gtest_discover_tests(decl_hdf5_conversion_test)
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
/*******************************************************************************
* Copyright (C) 2026 Commissariat a l'energie atomique et aux energies alternatives (CEA)
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of CEA nor the names of its contributors may be used to
* endorse or promote products derived from this software without specific
* prior written permission.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
******************************************************************************/

#include <filesystem>
#include <ranges>

#include <hdf5.h>

#include <pdi/testing.h>

class DeclHdf5: public ::PDI::PdiTest
{};

/* Precision conversion with decl_hdf5
* data in double precision
* file dataset in double, float, and int
*/
TEST_F(DeclHdf5, PrecisionConversion)
{
InitPdi(PC_parse_string(R"==(
logging: trace
metadata:
N: int
data:
array: {type: array, size: [$N, $N], subtype: double}
plugins:
decl_hdf5:
- file: d2d_test.h5
datasets:
double_ds: {type: array, size: [$N, $N], subtype: double}
write:
array:
dataset: double_ds
- file: d2f_test.h5
datasets:
float_ds: {type: array, size: [$N, $N], subtype: float}
write:
array:
dataset: float_ds
- file: d2i_test.h5
datasets:
int_ds: {type: array, size: [$N, $N], subtype: int}
write:
array:
dataset: int_ds
)=="));

EXPECT_FALSE(std::filesystem::exists("d2d_test.h5"));
EXPECT_FALSE(std::filesystem::exists("d2f_test.h5"));
EXPECT_FALSE(std::filesystem::exists("d2i_test.h5"));
Comment on lines +69 to +71
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use std::filesystem::remove() before, to be sure ?


static constexpr size_t const N = 100;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

constexpr of a const

Suggested change
static constexpr size_t const N = 100;
static constexpr size_t N = 100;

PDI_expose("N", &N, PDI_OUT);

auto const test_array = make_a<std::array<std::array<double, N>, N>>();
PDI_expose("array", test_array.data(), PDI_OUT);

EXPECT_TRUE(std::filesystem::exists("d2d_test.h5"));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps, we should use assert her because if we don't have the file the comparison test
for double doesn't make any sense

Suggested change
EXPECT_TRUE(std::filesystem::exists("d2d_test.h5"));
ASSERT_TRUE(std::filesystem::exists("d2d_test.h5"));

EXPECT_TRUE(std::filesystem::exists("d2f_test.h5"));
EXPECT_TRUE(std::filesystem::exists("d2i_test.h5"));

// read double precision dataset and compare
hid_t file_id = H5Fopen("d2d_test.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
hid_t dataset_id = H5Dopen2(file_id, "/double_ds", H5P_DEFAULT);
hid_t type_id = H5Dget_type(dataset_id);
Comment on lines +84 to +86
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May also check for read failure ?

Suggested change
hid_t file_id = H5Fopen("d2d_test.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
hid_t dataset_id = H5Dopen2(file_id, "/double_ds", H5P_DEFAULT);
hid_t type_id = H5Dget_type(dataset_id);
hid_t file_id = H5Fopen("d2d_test.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
hid_t dataset_id = H5Dopen2(file_id, "/double_ds", H5P_DEFAULT);
hid_t type_id = H5Dget_type(dataset_id);
ASSERT_GE(file_id, 0);
ASSERT_GE(dataset_id, 0);
ASSERT_GE(type_id, 0);


EXPECT_TRUE(H5Tequal(type_id, H5T_IEEE_F64LE));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An error of value "-1" would pass this check ?

Suggested change
EXPECT_TRUE(H5Tequal(type_id, H5T_IEEE_F64LE));
EXPECT_GT(H5Tequal(type_id, H5T_IEEE_F64LE), 0);

std::array<std::array<double, N>, N> read_double_array;

herr_t status = H5Dread(dataset_id, H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_double_array.data());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointer is not only a double, as it includes the static constexpr size_t N

Suggested change
herr_t status = H5Dread(dataset_id, H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_double_array.data());
herr_t status = H5Dread(dataset_id, H5T_NATIVE_DOUBLE, H5S_ALL, H5S_ALL, H5P_DEFAULT, &read_double_array[0][0]);

ASSERT_GE(status, 0);

EXPECT_EQ(test_array, read_double_array);

H5Tclose(type_id);
H5Dclose(dataset_id);
H5Fclose(file_id);

// read simple precision dataset and compare
file_id = H5Fopen("d2f_test.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
dataset_id = H5Dopen2(file_id, "/float_ds", H5P_DEFAULT);
type_id = H5Dget_type(dataset_id);

EXPECT_TRUE(H5Tequal(type_id, H5T_IEEE_F32LE));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to EXPECT_TRUE issue of H5T_IEEE_F64LE

std::array< std::array<float, N>, N > read_float_array;

status = H5Dread(dataset_id, H5T_NATIVE_FLOAT, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_float_array.data());
ASSERT_GE(status, 0);

#if defined(__clang__) && (__clang_major__ == 15)
// Skipping the numerical comparaison because Clang 15 has known issues with ranges.
#else
EXPECT_THAT(read_float_array, testing::ElementsAreArray(test_array | std::views::transform([](std::array<double, N> const & aref) {
return testing::Pointwise(testing::FloatEq(), aref);
Comment thread
iole-bolognesi marked this conversation as resolved.
})));
#endif

H5Tclose(type_id);
H5Dclose(dataset_id);
H5Fclose(file_id);

// read integer dataset and compare
file_id = H5Fopen("d2i_test.h5", H5F_ACC_RDONLY, H5P_DEFAULT);
dataset_id = H5Dopen2(file_id, "/int_ds", H5P_DEFAULT);
type_id = H5Dget_type(dataset_id);

EXPECT_TRUE(H5Tequal(type_id, H5T_STD_I32LE));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to EXPECT_TRUE issue of H5T_IEEE_F64LE

std::array< std::array<int, N>, N > read_int_array;

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_int_array.data());
ASSERT_GE(status, 0);

#if defined(__clang__) && (__clang_major__ == 15)
// Skipping the numerical comparaison because Clang 15 has known issues with ranges.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps do something like that in case we have issue with ranges,

std::array<std::array<int, N>, N> expected_int_array;

for( int ii=0; ii<test_array.size(); ++ii) {
       std::transform(test_array[ii].cbegin(), test_array[ii].cend(), expected_int_array[ii].begin(), 
                       [](auto aa){ return static_cast<int>(aa); });
}

// comparison
EXPECT_EQ(expected_int_array, read_int_array);

#else
EXPECT_THAT(read_int_array, testing::ElementsAreArray(test_array | std::views::transform([](std::array<double, N> const & aref) {
return testing::ElementsAreArray(aref | std::views::transform([](double const & ref) {
return static_cast<int>(ref);
}));
})));
#endif

H5Tclose(type_id);
H5Dclose(dataset_id);
H5Fclose(file_id);
}
Loading