Skip to content

Fix shared file 4k read #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JulianKunkel wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Fix shared file 4k read #106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b3f187b
Bugfix for new 4k read write - allow for shared file.
JulianKGW Mar 14, 2026
74f3e76
Cherry pickign warning from #104. Thanks @jxiong
JulianKGW Mar 18, 2026
35ae48a
Support shifting one more node away for random read.
JulianKGW Apr 1, 2026
7a9ee46
rnd read. Moving away using -Q option.
JulianKGW Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion prepare.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ echo This script downloads the code for the benchmarks
echo It will also attempt to build the benchmarks
echo It will output OK at the end if builds succeed
echo
IOR_HASH=8ab8f69b32b919
IOR_HASH=4a34dfc
Copy link
Copy Markdown
Contributor

@adilger adilger Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A slightly longer hash would be preferred (10-12 chars). At only 7 chars this is likely to collide after only 16k commits in IOR. While not an issue today (about 1k commits in IOR), it is better to think about this for the future.

Copy link
Copy Markdown
Contributor Author

@JulianKunkel JulianKunkel Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I do use the hash proposed by GitHub to id the version.

PFIND_HASH=aaba722a178

INSTALL_DIR=$PWD
Expand Down
6 changes: 3 additions & 3 deletions src/phase_ior_easy.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,6 @@ void ior_easy_add_params(u_argv_t * argv, int addStdFlags){
u_argv_push(argv, "-O");
u_argv_push(argv, "allocateBufferOnGPU=1");
}
u_argv_push(argv, "-C"); /* reorder tasks in constant order for read */
u_argv_push(argv, "-Q"); /* task per node offset */
u_argv_push(argv, "1");
u_argv_push(argv, "-g"); /* barriers between open, read, write, close */
u_argv_push(argv, "-G"); /* use fixed timestamp signature */
int hash = u_phase_unique_random_number("ior-easy");
Expand All @@ -67,6 +64,9 @@ void ior_easy_add_params(u_argv_t * argv, int addStdFlags){
u_argv_push(argv, "-o"); /* filename for output file */
u_argv_push_printf(argv, "%s/ior-easy/ior_file_easy", opt.datadir);
if(addStdFlags){
u_argv_push(argv, "-C"); /* reorder tasks in constant order for read */
u_argv_push(argv, "-Q"); /* task per node offset */
u_argv_push(argv, "1");
u_argv_push(argv, "-O"); /* additional IOR options */
u_argv_push_printf(argv, "stoneWallingStatusFile=%s/ior-easy.stonewall", opt.resdir);
u_argv_push(argv, "-t"); /* transfer size */
Expand Down
8 changes: 7 additions & 1 deletion src/phase_ior_rnd_read4k-easywrite.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ static void validate(void){


static double run(void){
opt_ior_rnd_read d = o;
opt_ior_easy d = ior_easy_o;

u_argv_t * argv = u_argv_create();
ior_easy_add_params(argv, 0);
Expand All @@ -44,6 +44,12 @@ static double run(void){
u_argv_push(argv, "-s=10000000");
Copy link
Copy Markdown
Contributor

@adilger adilger Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As previously discussed, it would be good to add a separate patch that sets -s=(file_size / (10 * 4096)) (i.e. 10% of file size) so that the random reads don't have too many cache hits. The sqrt(file_size / 4096) is the chance of one collision with random reads, and we could work out the math so that the random read phase exits when the chance of repeated collisions is too high. 10% of the file size is just a random guess.

u_argv_push(argv, "-O");
u_argv_push(argv, "stoneWallingWearOut=1");

if (d.filePerProc){
u_argv_push(argv, "-C"); /* reorder tasks in constant order for read */
u_argv_push(argv, "-Q"); /* task per node offset */
u_argv_push(argv, "2");
}

o.command = u_flatten_argv(argv);

Expand Down
4 changes: 3 additions & 1 deletion src/phase_mdworkbench.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,9 @@ void mdworkbench_add_params(u_argv_t * argv, int is_create){
if(! f){
WARNING("Couldn't open mdworkbench-file: %s\n", file);
}else{
fread(& mdtest->rate, sizeof(mdtest->rate), 1, f);
if (fread(& mdtest->rate, sizeof(mdtest->rate), 1, f) != 1) {
WARNING("Failed to read mdworkbench-file: %s\n", file);
}
}
fclose(f);
}
Expand Down