Support UK radar data #1825
Support UK radar data #1825
Conversation
|
for info if the upgrade of you branch with main causes problems then I have created a branch of your branch and merged main in now including the addition of plotting transverse mercator. So you should be able to plot the radar data. |
|
#https://github.com/MetOffice/CSET/tree/1759_import_nimrod_merge_main adds nimrod radar switch into rose gui and enables nimrod switch in flow.cylc for the workflow. |
jfrost-mo
left a comment
jfrost-mo
left a comment
There was a problem hiding this comment.
src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py is the main file for this change, as the stuff in fetch_nimrod is largely unused.
What is our plan with this? Are we going to try and reuse the RES capability in cube_utils? Or do we not really need it given the existing capabilities of CSET?
The recipe and such looks broadly sensible, though I think we can get away with hard-coding some more for the initial simple recipe.
| [[FETCH_NIMROD]] | ||
| script = rose task-run -v --app-key=fetch_nimrod | ||
| execution time limit = PT1H | ||
| [[[environment]]] | ||
| ANALYSIS_LENGTH = {{ANALYSIS_LENGTH}} | ||
|
|
There was a problem hiding this comment.
We can make fetch_nimrod inherit from the FETCH_DATA family. Then we don't need a new family, don't need to add it to the graph above, and don't need a new dummy task.
See how the surface obs do it on line ~180.
| # for model, field, method in itertools.product( | ||
| for model, field in itertools.product( | ||
| models, conf.SURFACE_FIELDS, conf.SPATIAL_SURFACE_FIELD_METHOD | ||
| ): |
There was a problem hiding this comment.
I don't think need to loop here, as the field will remain constant for radar, and we don't need to worry about different models either.
So I think this can become something like:
if conf.NIMROD_FIELDS_COMP_HOUR: # Note: You might want to use a different setting.
yield RawRecipe(
recipe="generic_surface_spatial_plot_sequence_radar.yaml",
variables={ "VARNAME": conf.NIMROD_FIELDS_COMP_HOUR },
aggregation=False,
)| operator: constraints.combine_constraints | ||
| varname_constraint: | ||
| operator: constraints.generate_var_constraint | ||
| varname: $VARNAME |
There was a problem hiding this comment.
Could we get away with hardcoding VARNAME? Or do we need it to support different resolution data?
| # cell_methods_constraint: | ||
| # operator: constraints.generate_cell_methods_constraint | ||
| # cell_methods: [] | ||
| # varname: $VARNAME | ||
| # level_constraint: | ||
| # operator: constraints.generate_level_constraint | ||
| # coordinate: "pressure" | ||
| # levels: [] | ||
| # subarea_type: $SUBAREA_TYPE | ||
| # subarea_extent: $SUBAREA_EXTENT | ||
|
|
||
| # - operator: collapse.collapse | ||
| # coordinate: [time] | ||
| # method: $METHOD |
There was a problem hiding this comment.
| # cell_methods_constraint: | |
| # operator: constraints.generate_cell_methods_constraint | |
| # cell_methods: [] | |
| # varname: $VARNAME | |
| # level_constraint: | |
| # operator: constraints.generate_level_constraint | |
| # coordinate: "pressure" | |
| # levels: [] | |
| # subarea_type: $SUBAREA_TYPE | |
| # subarea_extent: $SUBAREA_EXTENT | |
| # - operator: collapse.collapse | |
| # coordinate: [time] | |
| # method: $METHOD |
| #title: $MODEL_NAME $VARNAME $METHOD | ||
| #description: Extracts and plots the $METHOD of $VARNAME from all times in $MODEL_NAME. | ||
| title: $VARNAME |
There was a problem hiding this comment.
| #title: $MODEL_NAME $VARNAME $METHOD | |
| #description: Extracts and plots the $METHOD of $VARNAME from all times in $MODEL_NAME. | |
| title: $VARNAME | |
| title: Radar plot of $VARNAME |
Add a little bit more to the title so it is useful.
There was a problem hiding this comment.
I'm a bit confused. Are we implementing the fetching of radar in fetch_fcst, or separately in fetch_nimrod?
| """Write the dictionary to retrieve Nimrod obs.""" | ||
| files = { | ||
| "rainaccum_comp_hour": { | ||
| "basedir": "/data/users/radar/UKnimrod", |
There was a problem hiding this comment.
Are these paths too Met Office specific? They might want to be pushed into a per-site file and then moved to the CSET restricted files repository.
| print(" Copying Nimrod files to directory: ", cycle_nimrod_dir) | ||
| for i in filepaths: | ||
| print(" Grabbing Nimrod file: ", i) | ||
| shutil.copy(i, cycle_nimrod_dir) |
There was a problem hiding this comment.
Since they are already on disk, can we just symlink to them to avoid copying data about?
There was a problem hiding this comment.
This file is unused. Is it only a reference for implementing src/CSET/cset_workflow/app/fetch_nimrod/bin/fetch-nimrod.py? (I presume it comes from the RES?)
There was a problem hiding this comment.
Ah, so this file just calls the fetch_nimrod function in src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py, and none of the other code is currently used for anything.
|
|
Fixes #1759
Contribution checklist
Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.