Multi-node benchmarks with Legion

[ysfess22]

Hi,
I'm trying to benchmark Legion on a 2 nodes cluster. I would like to confirm if the output I'm getting is the expected behaviour.
To setup the the benchmarks I did the following:

USE_GASNET=1 USE_LEGION=1 ./get_deps.sh
USE_GASNET=1 CONDUIT=udp ./build_all.sh

I designated the two nodes (export SSH_SERVERS = node1,node2) and set up password-less authentication for the two nodes.
using ./task_bench -width 40 -steps 100 -type stencil_1d -kernel compute_bound -iter 1024
gives a GASNet related error where I have to mention the number of nodes.

So I'm using: ./task_bench 2 -width 40 -steps 100 -type stencil_1d -kernel compute_bound -iter 1024 instead.

The results I'm getting from doing this is: 8000 tasks launched, 4000 in node 1 and 4000 in node 2.
Once both nodes are done with their 4000 tasks, I get the results of the benchmark: with running time, FLOP/s etc...

I'm wondering if this is the result I should expect, or is there a way to make the 2 nodes split the 4000 tasks between them?

Thank you.

[elliottslaughter]

Can you do a run with -level announce=2 and let me know what you see in the logs?

This behavior makes it sound like GASNet is failing to connect properly, resulting in each job being run in isolation. The announce logging would confirm if that's the case or not.

[ysfess22]

My benchmark now runs as I expect it. The tasks are split between the nodes and no extra tasks are spawned. I think the problem was that I didn't pay attention to the warm-up run that the legion implementation does and that's why the number of tasks looked double.

[elliottslaughter]

Ok, cool. Let me know if you have any other questions.