[Bug] 2-stream mpas_atmosphere warnings

[jjguerrette]

Current behavior (describe the bug)

So far only tested with a 120km domain.

When using the 2-stream (static+input) initial state in an mpas_atmosphere forecast within mpas-bundle, the following warnings are issued:

WARNING: Variable meshScalingDel2 not in input file.
WARNING: Variable meshScalingDel4 not in input file.
WARNING: Variable meshScalingRegionalCell not in input file.
WARNING: Variable meshScalingRegionalEdge not in input file.
WARNING: Variable east not in input file.
WARNING: Variable north not in input file.
WARNING: Variable pin not in input file.
WARNING: Variable ozmixm not in input file.

The static and and input streams are as follows in streams.atmosphere:

<immutable_stream name="static"
                  type="input"
                  filename_template="static.nc"
                  io_type="pnetcdf,cdf5"
                  input_interval="initial_only" />

<immutable_stream name="input"
                  type="input"
                  filename_template="mpasin.$Y-$M-$D_$h.$m.$s.nc"
                  io_type="pnetcdf,cdf5"
                  input_interval="initial_only" />

I am using an x1.40962.init.DATE.nc file linked to static.nc and a da_state (mpasout) file as mpasin.$Y-$M-$D_$h.$m.$s.nc.

Looking at src/core_atmosphere/Registry.xml, the static stream is supposed to include east, north, pin, and ozmixm. However, the output stream in src/core_init_atmosphere/Registry.xml, which is used to produce x1.40962.init.DATE.nc files, does not include any of those four variables. So either we should not use init.nc files as static stream inputs or one of the two Registry.xml variable lists needs to be modified.

The source of the other warning (meshScalingDel2, meshScalingDel4, meshScalingRegionalCell, and meshScalingRegionalEdge) is not clear. These are only listed in the restart stream of src/core_atmosphere/Registry.xml, but I have config_do_restart = false set in namelist.atmosphere. Also, those appear to be dependent variables set in src/core_atmosphere/mpas_atm_core::atm_compute_mesh_scaling, not variables that need to be read in.

To Reproduce

What computer are you running on?
Cheyenne

What compilers/modules are you using?
jedi/gnu-openmpi/9.1.0-v0.4

Steps to reproduce the behavior

1. Run mpas_atmosphere forecast in 2-stream workflow
2. Look at log.atmosphere.0000.out

Expected behavior

The warnings should not be issued, either because the fields are present in the static.nc file or because they are not needed and are removed from the requested variables.

Additional information (optional)

[jjguerrette]

east and north are calculated in init_dirs_forphys. So long as that subroutine is called when config_do_DACycling is true, then those two variables do not need to be in the static stream.

EDIT: the relevant subroutine tree is as follows
atm_mpas_init_block --> physics_init --> init_dirs_forphys

There are no logical tests that avoid the tree from being followed. Therefore east and north can be removed from the static stream.

[liujake]

@junmeiban Can you look into this unwanted warning issue? Ideally, it will also be cleaner if we can really use a 'static' file instead of an init file for the static stream to avoid confusion.

@syha Your originally implemented this two-stream initialization. What is the reason you used an 'init' file for the static stream? Do you remember what fields are missing in the normal static file to be used in this two-stream initialization?

[junmeiban]

Sure. I can look into it.

[liujake]

From my recent regional test using the 2-stream option, I noticed that mpas-jedi seems to write out the analysis file with many variables (controlled by stream_list.atmosphere_output?). This behavior is different from earlier version of code only writing our very few analysis variables (we used ncks or overwrote the background file via yaml setting). I remember @jjguerrette or @byoung-joo changed to have a more flexible/controllable I/O with a PR in mpas-jedi. Can you point which PR it is (I had hard time to find it) or point to the code for analysis I/O in mpas-jedi? I think this WARNING issue is more likely a mpas-jedi issue, not a MPAS-Model issue, understanding current analysis I/O in MPAS-JEDI will be helpful to solve it.

[byoung-joo]

From my recent regional test using the 2-stream option, I noticed that mpas-jedi seems to write out the analysis file with many variables (controlled by stream_list.atmosphere_output?).

Currently, all of the read and write functions in mpas-jedi are controlled by output stream, and the variables for 'read' and 'write' are listed in stream_list.atmosphere_output.

I remember @jjguerrette or @byoung-joo changed to have a more flexible/controllable I/O with a PR in mpas-jedi.

We have an issue for that, but it is not done yet: https://github.com/JCSDA-internal/mpas-jedi/issues/355 .
This might be a right time to do this...

Can you point which PR it is (I had hard time to find it) or point to the code for analysis I/O in mpas-jedi?

read : https://github.com/JCSDA-internal/mpas-jedi/blob/1a8887d9d43c04bf7ffa10a4a9d63bf77def9725/src/mpasjedi/mpas_field_utils_mod.F90#L320
write : https://github.com/JCSDA-internal/mpas-jedi/blob/1a8887d9d43c04bf7ffa10a4a9d63bf77def9725/src/mpasjedi/mpas_field_utils_mod.F90#L432

[liujake]

My impression is that variables to 'read' is not only controlled by the '*_output' stream, yaml setting for background state variables also matters. For example, I remember I added t2m/q2m in yaml for making rttov operator work, but seems to no need to add them in 'output' stream file. That is another confusion part to me (i.e., variables I/O vs. background/state variables in use).

[byoung-joo]

If t2m/q2m is not added to stream_list.atmosphere_output, then the values for t2m/q2m variables must be assigned at the mpas_init stage, when the mpas initial fields are read from init or restart or 2stream mpasout files. Not through the read function.

@byoung-joo It is not very clear to me from your statement: is it a problem or not if I do not add t2m/q2m in stream_list.atmosphere_output? Sounds like all variables will be read anyway by 'mpas_init' no matter variables are included or not in stream_list.atmosphere_output. Then I guess it is Ok to not have t2m/q2m in stream_list.atmosphere_output. Now I recall that there are two Input/Read steps: one via model's mpas_init (for every thing, we have no much control), one through mpas-jedi's 'read' function (for variables defined in stream_list.atmosphere_output). We have added code to deallocate unused variables loaded by mpas_init. This worked fine so far, but be always confusing when doing configuration.

Sorry. I accidently deleted some message by using edit mode instead of Quote reply mode.

[syha]

Jake, I checked my MPAS codes w/ the 2-stream option, and found that I had defined three different stream names for the purpose of separating out the streams: "static" for static.nc, "init" for init.nc, and "da_state" for analysis variables only. I don't know how you guys adopted my codes, but I defined and used static and init streams separately. Let me attach my src/core_atmosphere/Registry.xml here.
Registry.xml.txt

[liujake]

@syha We did the same as you, just have a slightly different set of variables for 'da_state' stream. You may check back, linking init.nc to static.nc is in your documentation on how to use this option, we just followed. I believe it will not work if we just use static.nc for static stream. My question is that what is the reason you did this way (i.e., linking init.nc to static.nc)? Any suggestions how to make this cleaner, i.e., really use static.nc file for static stream.

[liujake]

@syha #5 is the PR in this repo for 2-stream modification.