Adds reading of omega HorzMesh from mesh iostream

components/omega/configs/Default.yml

@@ -82,6 +82,16 @@ Omega:
       Alpha: 5.0
       Exponent: 2.0
   IOStreams:
+    HorzMeshIn:
+      UsePointerFile: false
+      Filename: OmegaMesh.nc
+      Mode: read
+      Precision: double
+      Freq: 1
+      FreqUnits: OnStartup
+      UseStartEnd: false
+      Contents:
+        - HorzMeshIn
     InitialVertCoord:
       UsePointerFile: false
       Filename: OmegaMesh.nc

components/omega/doc/devGuide/Decomp.md

@@ -20,11 +20,14 @@ with the call:
  Decomp::init();
 ```
 This must be called very early in the init process, just after initializing
-the MachEnv, Config and IO.  Mesh information is first read using parallel
-IO into an equally-spaced linear decomposition, then partitioned by METIS or
-ParMETIS into a more optimal decomposition. The parallel ParMETIS
-implementation should be used to reduce the memory footprint for
-high-resolution configurations. Although the algorithmic method is similar,
+the MachEnv, Config and IO.  By default, the mesh information is read from
+the filename defined in the [HorzMeshIn IOStream](#omega-user-horz-mesh).
+However, an optional filename argument to the init function above can be used
+to override the input configuration (eg for unit testing). Mesh information is
+first read using parallel IO into an equally-spaced linear decomposition, then
+partitioned by METIS or ParMETIS into a more optimal decomposition. The
+parallel ParMETIS implementation should be used to reduce the memory footprint
+for high-resolution configurations. Although the algorithmic method is similar,
 ParMETIS leads to a somewhat different partitioning than the serial METIS
 version. The serial version should give the same partition as the serial
 offline gpmetis tool used by MPAS when the same version of the METIS library

components/omega/doc/devGuide/HorzMesh.md

@@ -7,17 +7,20 @@ Specification](https://mpas-dev.github.io/files/documents/MPAS-MeshSpec.pdf).
 A mesh object is created by the `init` method, which assumes that `Decomp` has
 already been initialized.
 ```c++
-Err = OMEGA::Decomp::init();
-Err = OMEGA::HorzMesh::init();
+OMEGA::Decomp::init();
+OMEGA::HorzMesh::init(ModelClock);
 ```
+The model clock should be the ocean model clock and is required by the IOStream
+routines that read the HorzMesh variables from the input stream
+[HorzMeshIn](#omega-user-horz-mesh).
 The constructor replicates the subdomain mesh cell/edge/vertex counts and
 connectivity information from Decomp so this information can be passed among the
 computational routines, alongside the other local mesh information.  It then
-creates several parallel I/O decompositions and reads in the remaining subdomain
-mesh information.  Finally, any mesh information needed on the device is copied
-from the host to a device Kokkos array. Arrays such as the coordinate variables,
-which are not involved in tendency calculations, are not transferred to the
-device. These tasks are organized into several private methods. Eventually,
+creates reads most of the remaining mesh variables (coordinates, lengths, areas)
+from the input HorzMeshIn IOStream. A few variables, like masks and operator
+scaling are computed internally. After reading, the init routine fills all
+halos and synchronizes the host and device copies of the mesh variables.
+These tasks are organized into several private methods. Eventually,
 dependent mesh variables will be computed from the minimum set of required mesh
 information.
 
@@ -45,11 +48,9 @@ OMEGA::parallelFor({HMesh->NCellsOwned,HMesh->MaxEdges},
 ```
 
 For member variables that are host arrays, variable names are appended with an
-`H`.  Array variable names not ending in `H` are device arrays.  The copy from
-host to device array is performed in the constructor via:
-```c++
-AreaCell = OMEGA::createDeviceMirrorCopy(AreaCellH);
-```
+`H`.  Array variable names not ending in `H` are device arrays.
 
 The device arrays are deallocated by the `HorzMesh::clear()` method, which is
-necessary before calling `Kokkos::finalize`.
+necessary before calling `Kokkos::finalize`. Because Field and IOStream also
+have references to HorzMesh variables, the `IOStream::clear()` and
+`Field::clear()` should also be called to ensure all device arrays are removed.

components/omega/doc/userGuide/Decomp.md

@@ -41,9 +41,10 @@ An input mesh file must be provided that contains at a minimum
     VerticesOnCell, CellsOnEdge, EdgesOnEdge, CellsOnVertex, EdgesOnVertex.
 Again, a full description of the mesh is given in the
 [Developer's Guide](#omega-dev-decomp).
-The mesh information is read via parallel IO into an initial linear domain
-decomposition and then is partitioned by METIS and rearranged into the
-final METIS parallel decomposition.
+The file name for this input file is extracted from the HorzMeshIn input
+mesh stream located in the IOStreams section of the input configuration.
+This value can be overridden by the driver routine if needed (eg for unit
+testing).
 
 Once the mesh is decomposed, all of the mesh index arrays are stored in
 a Decomp named Default which can be retrieved as described in the

components/omega/doc/userGuide/HorzMesh.md

@@ -16,7 +16,26 @@ additional mesh variables, which are not required for Decomp.
 
 Currently, the Mesh class reads in all variables from the MPAS mesh
 specification except those read by [Decomp](#omega-dev-decomp).
-This includes the following variables:
+These are all read using the input [IOStream](#omega-user-iostreams) HorzMeshIn
+```yaml
+  IOStreams:
+    HorzMeshIn:
+      UsePointerFile: false
+      Filename: OmegaMesh.nc
+      Mode: read
+      Precision: double
+      Freq: 1
+      FreqUnits: OnStartup
+      UseStartEnd: false
+      Contents:
+        - HorzMeshIn
+```
+Only the Filename should be changed by the user to point to the relevant input
+mesh file. The mesh Filename is sometimes overridden by the driver routine in
+the case of unit tests using an optional argument to the
+[decomposition](#omega-dev-decomp). The input contents are defined by the
+HorzMeshIn [FieldGroup](#omega-dev-field) that currently includes the following
+variables:
 
 | Variable Name | Description | Units |
 | ------------- | ----------- | ----- |

components/omega/src/base/Decomp.cpp

@@ -398,7 +398,7 @@ void readMesh(const int MeshFileID, // file ID for open mesh file
 // Initialize the decomposition and create the default decomposition with
 // (currently) one partition per MPI task using selected partition method
 
-void Decomp::init(const std::string &MeshFileName) {
+void Decomp::init(const std::string &InMeshFileName) {
 
    bool TimerFlag = Pacer::start("Decomp init", 0);
    Error Err; // default successful return code
@@ -421,6 +421,26 @@ void Decomp::init(const std::string &MeshFileName) {
 
    PartMethod Method = getPartMethodFromStr(DecompMethodStr);
 
+   // If no mesh filename supplied, get the filename from the input
+   // HorzMeshIn stream configuration
+   std::string MeshFileNameTmp = InMeshFileName;
+   if (InMeshFileName == "") {
+      Config StreamConfigTmp("IOStreams");
+      Err = OmegaConfig->get(StreamConfigTmp);
+      CHECK_ERROR_ABORT(Err, "Decomp: IOStreams not found in input Config");
+
+      Config HorzMeshInTmp("HorzMeshIn"); // retrieve HorzMeshIn stream info
+      Err = StreamConfigTmp.get(HorzMeshInTmp);
+      CHECK_ERROR_ABORT(Err, "Decomp: HorzMeshIn stream not found in Config");
+
+      Err = HorzMeshInTmp.get("Filename", MeshFileNameTmp);
+      CHECK_ERROR_ABORT(Err, "Decomp: Could not get mesh filename from Config");
+      // Testing - re-retrieve a fresh copy of the stream and mesh filename
+      Err = OmegaConfig->get(StreamConfigTmp);
+      Err = StreamConfigTmp.get(HorzMeshInTmp);
+      Err = HorzMeshInTmp.get("Filename", MeshFileNameTmp);
+   }
+
    // Retrieve the default machine environment
    MachEnv *DefEnv = MachEnv::getDefault();
 
@@ -429,7 +449,7 @@ void Decomp::init(const std::string &MeshFileName) {
 
    // Create the default decomposition and set pointer to it
    Decomp::DefaultDecomp = Decomp::create("Default", DefEnv, NParts, Method,
-                                          InHaloWidth, MeshFileName);
+                                          InHaloWidth, MeshFileNameTmp);
 
    TimerFlag = Pacer::stop("Decomp init", 0) && TimerFlag;
    if (!TimerFlag)

components/omega/src/base/Decomp.h

@@ -264,7 +264,7 @@ class Decomp {
    /// Initializes Omega decomposition info and creates the default
    /// decomposition based on the default MachEnv and configuration
    /// options.
-   static void init(const std::string &MeshFileName = "OmegaMesh.nc");
+   static void init(const std::string &InMeshFileName = "");
 
    // Creates a new decomposition using the constructor and puts it in the
    // AllDecomps map

components/omega/src/infra/IOStream.cpp

@@ -135,6 +135,16 @@ IOStream::get(const std::string &StreamName ///< [in] name of stream to retrieve
 
 } // End get stream
 
+//------------------------------------------------------------------------------
+// Changes the filename associated with a stream (eg during unit testing)
+void IOStream::changeFilename(
+    const std::string &StreamName, ///< [in] name of stream to modify
+    const std::string &NewFilename ///< [in] new filename for stream
+) {
+   auto StreamPtr      = get(StreamName);
+   StreamPtr->Filename = NewFilename;
+} // End changeFilename
+
 //------------------------------------------------------------------------------
 // Adds a field to the contents of a stream. Because streams may be created
 // before all Fields have been defined, we only store the name. Validity
@@ -2279,6 +2289,11 @@ Error IOStream::readStream(
    if (Mode != IO::ModeRead)
       ABORT_ERROR("IOStream read: cannot read stream defined as output stream");
 
+   // Validate the stream if not already validated (this also expands group
+   // names into field names).
+   if (!validate())
+      ABORT_ERROR("Unable to validate stream {}", Name);
+
    // If it is not time to read, return
    if (!ForceRead) {
       if (!MyAlarm.isRinging() and !OnStartup)

components/omega/src/infra/IOStream.h

@@ -294,6 +294,13 @@ class IOStream {
    get(const std::string &StreamName ///< [in] name of stream to retrieve
    );
 
+   //---------------------------------------------------------------------------
+   /// Changes the filename in a stream (typically during unit testing)
+   static void changeFilename(
+       const std::string &StreamName, ///< [in] name of stream to modify
+       const std::string &NewFilename ///< [in] new file name
+   );
+
    //---------------------------------------------------------------------------
    /// Adds a field to the contents of a stream. Because streams may be created
    /// before all Fields have been defined, we only store the name. Validity