Ebgeometry in AMReX MPI

[dalarlla]

I've been looking over the AMReX examples in the repository and it looks like if running in parallel over MPI all ranks will build the SDF. Is this correct? And if so are there any potential issues when running on a large number of ranks with a large STL file for example?

[rmrsk]

Yes, every rank needs the implicit function, so it has to be replicated. Distributed implicit functions is not a good idea, as you would need global broadcasts when calling the inplicit functions.

I'm not sure if one could use a different communicator or MPI+MPI pattern to offset the memory cost and avoid the hard replication.

And yes, replication eats memory.

If memory is an issue you can drop EBGeometry to single precision, or use fewer MPI ranks and compensate by using threading/tiling within AMReX.