2158 add metplus ensemble stats capability#2159
Open
JorgeBornemann (JorgeBornemann) wants to merge 10 commits into
Open
2158 add metplus ensemble stats capability#2159JorgeBornemann (JorgeBornemann) wants to merge 10 commits into
JorgeBornemann (JorgeBornemann) wants to merge 10 commits into
Conversation
JorgeBornemann (JorgeBornemann)
added
enhancement
JorgeBornemann (JorgeBornemann)
linked an issue
Contributor
Total coverage: 91% (HTML report)Name Stmts Miss Branch BrPart Cover --------------------------------------------------------------------------------------------------- src/CSET/__init__.py 93 2 12 0 98% src/CSET/_common.py 149 0 52 0 100% src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py 115 27 24 0 79% src/CSET/cset_workflow/app/finish_website/bin/finish_website.py 70 0 4 0 100% src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py 29 0 8 0 100% src/CSET/cset_workflow/app/send_email/bin/send_email.py 25 0 4 0 100% src/CSET/cset_workflow/lib/python/jinja_utils.py 17 0 6 0 100% src/CSET/extract_workflow.py 47 0 16 0 100% src/CSET/graph.py 43 0 14 0 100% src/CSET/operators/__init__.py 89 0 26 0 100% src/CSET/operators/_atmospheric_constants.py 9 0 0 0 100% src/CSET/operators/_colormaps.py 229 4 62 4 97% src/CSET/operators/_stash_to_lfric.py 3 0 0 0 100% src/CSET/operators/_utils.py 183 8 74 6 95% src/CSET/operators/ageofair.py 141 7 64 5 94% src/CSET/operators/aggregate.py 76 1 22 1 98% src/CSET/operators/aviation.py 60 0 18 0 100% src/CSET/operators/collapse.py 154 12 72 5 91% src/CSET/operators/constraints.py 107 7 46 2 93% src/CSET/operators/convection.py 37 4 10 2 87% src/CSET/operators/ensembles.py 27 0 14 0 100% src/CSET/operators/feature.py 41 0 10 0 100% src/CSET/operators/filters.py 66 2 30 0 98% src/CSET/operators/humidity.py 139 0 56 0 100% src/CSET/operators/imageprocessing.py 56 0 16 0 100% src/CSET/operators/mesoscale.py 17 0 2 0 100% src/CSET/operators/misc.py 146 0 54 1 99% src/CSET/operators/plot.py 929 169 318 59 78% src/CSET/operators/power_spectrum.py 97 3 30 3 95% src/CSET/operators/precipitation.py 93 0 50 0 100% src/CSET/operators/pressure.py 41 0 12 0 100% src/CSET/operators/read.py 409 38 178 14 89% src/CSET/operators/regrid.py 122 18 64 1 83% src/CSET/operators/scoreswrappers.py 47 6 12 3 85% src/CSET/operators/temperature.py 121 0 32 0 100% src/CSET/operators/transect.py 62 0 24 0 100% src/CSET/operators/wind.py 45 3 10 2 91% src/CSET/operators/write.py 15 0 6 0 100% src/CSET/recipes/__init__.py 101 0 28 0 100% --------------------------------------------------------------------------------------------------- TOTAL 4250 311 1480 108 91% |
Remove commented out lines for table name and database name.
Removed unused variables TABLE_NAME and DB_NAME from METPLUS configuration.
Removed TABLE_NAME and DB_NAME assignments for point_stat.
JorgeBornemann (JorgeBornemann)
requested review from
Scott Wales (ScottWales) and
James Frost (jfrost-mo)
Contributor
Author
|
This PR covers the implementation of METPlus ensemble stats apps and expanding the VerPy apps to cover ensemble stat cases. It sets up the scaffolding, based on 2 variables and producing RMSE and spread in the same plot, upon which the rest of variables and metrics will be added. |
Scott Wales (ScottWales)
requested changes
Jul 7, 2026
Scott Wales (ScottWales)
left a comment
Scott Wales (ScottWales)
left a comment
Contributor
There was a problem hiding this comment.
Thanks JorgeBornemann (@JorgeBornemann) I've added comments below.
Thinking bigger picture now that there's the skeleton of a few different met tools in CSET, what is the general user story for the verification? Are we expecting Metplus and Verpy to be pre-set with some basic stats that users generally don't modify, or will users be selecting the fields and statistics they're plotting in the CSET configuration?
Contributor
There was a problem hiding this comment.
This file has quite a few issues
- incorrect comments referring to ascii2nc
- it's a config for PointStat rather than EnsembleStat
- input templates referring to NIWA local paths
- the variable configuration is probably going to be site specific
What's the goal of this file? It's not being used in the niwa opt and can't be used as a starting point for other sites as-is.
Comment on lines
+26
to
+27
| fcrs = [0, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000, 2100, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000, 3100, 3200, 3300, 3400, 3500, 3600] | ||
| times: [0, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000, 2100, 2200, 2300] |
Contributor
There was a problem hiding this comment.
It might be more flexible for other sites if these are left unset, so that verpy fills them in from what data is available?
Comment on lines
+34
to
+60
| [runtime] | ||
| [[metplus_prep_obs]] | ||
| inherit = METPLUS | ||
|
|
||
| {% for model in models %} | ||
| [[metplus_prep_fcst_m{{model["id"]}}]] | ||
| # Prepares model data for use by metplus | ||
| inherit = METPLUS | ||
| [[[environment]]] | ||
| ROSE_TASK_APP = metplus_prep_fcst | ||
| MODEL_NAME = {{model["name"]}} | ||
| MODEL_ID = m{{model["id"]}} | ||
| INPUT_PATHS = $CYLC_TASK_SHARE_CYCLE_DIR/data/{{model["id"]}} | ||
| {% if SELECT_SUBAREA %} | ||
| SUBAREA_TYPE = {{SUBAREA_TYPE}} | ||
| SUBAREA_EXTENT = {{SUBAREA_EXTENT}} | ||
| {% else %} | ||
| SUBAREA_TYPE = None | ||
| SUBAREA_EXTENT = None | ||
| {% endif %} | ||
| {% endfor %} | ||
|
|
||
| [[metplus_ascii2nc]] | ||
| # Runs METplus wrappers for ASCII to nc ingestion of obs. | ||
| inherit = METPLUS | ||
| [[[environment]]] | ||
| ROSE_APP_OPT_CONF_KEYS = {{SITE}}_ens |
Contributor
There was a problem hiding this comment.
Is this duplicating sections from point_stat that would be better off in metplus_common.cylc?
| # Runs METplus wrappers for ASCII to nc ingestion of obs. | ||
| inherit = METPLUS | ||
| [[[environment]]] | ||
| ROSE_APP_OPT_CONF_KEYS = {{SITE}}_ens |
Contributor
There was a problem hiding this comment.
There's not any opt files that match this - should it be made optional? with e.g.
Suggested change
| ROSE_APP_OPT_CONF_KEYS = {{SITE}}_ens | |
| ROSE_APP_OPT_CONF_KEYS = ({{SITE}}_ens) |
| inherit = METPLUS | ||
| [[[environment]]] | ||
| STAT = area | ||
| STAT_TYPE_LIST = "ecnt rhist rps pct" |
Contributor
There was a problem hiding this comment.
The example ensemble_stat config should output these stats
Comment on lines
+75
to
+78
| [[VERPY_METLOADER]] | ||
| # Family grouping for all VerPy metloader tasks | ||
| inherit = METPLUS | ||
| script = rose task-run -v --app-key=verpy_metloader |
Contributor
There was a problem hiding this comment.
Probably should be in metplus_common
Comment on lines
+92
to
+99
| MODEL_NAME={{model["name"]}} | ||
| ENSEMBLE_STAT_DIR=${CYLC_TASK_SHARE_CYCLE_DIR}/Ensemble_Stat_${MODEL_NAME} | ||
| mkdir -p ${CYLC_WORKFLOW_SHARE_DIR}/${MODEL_NAME}_ensemble_stat_${STAT}_tar | ||
| for STAT_TYPE in ${STAT_TYPE_LIST}; do | ||
| gzip ${ENSEMBLE_STAT_DIR}/*_${STAT_TYPE}.txt | ||
| files_list=$(ls ${ENSEMBLE_STAT_DIR}/*_${STAT_TYPE}.txt.gz | xargs -n 1 basename) | ||
| tar -rf ${CYLC_WORKFLOW_SHARE_DIR}/${MODEL_NAME}_ensemble_stat_${STAT}_tar/ensemble_stat_${CYLC_TASK_CYCLE_POINT}_${STAT_TYPE}.tar -C ${ENSEMBLE_STAT_DIR} $files_list | ||
| done |
Contributor
There was a problem hiding this comment.
Does making tar files of the output need to be done, or would it be simpler to just load the text files into the verpy database?
…om:MetOffice/CSET into 2158-add-metplus-ensemble_stats-capability
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces METPlus' EnsembleStat capability, with infrastructure based on PointStat. Fixes #2158
Contribution checklist
No AI has been used in this PR apart from web searches.
Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.
rose-suite.conf.examplehas been updated if new diagnostic added.