2215 ensemble verification#2235
Conversation
Total coverage: 91% (HTML report)Name Stmts Miss Branch BrPart Cover --------------------------------------------------------------------------------------------------- src/CSET/__init__.py 93 2 12 0 98% src/CSET/_common.py 149 0 52 0 100% src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py 115 27 24 0 79% src/CSET/cset_workflow/app/finish_website/bin/finish_website.py 70 0 4 0 100% src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py 29 0 8 0 100% src/CSET/cset_workflow/app/send_email/bin/send_email.py 25 0 4 0 100% src/CSET/cset_workflow/lib/python/jinja_utils.py 17 0 6 0 100% src/CSET/extract_workflow.py 47 0 16 0 100% src/CSET/graph.py 43 0 14 0 100% src/CSET/operators/__init__.py 89 0 26 0 100% src/CSET/operators/_atmospheric_constants.py 9 0 0 0 100% src/CSET/operators/_colormaps.py 229 4 62 4 97% src/CSET/operators/_stash_to_lfric.py 3 0 0 0 100% src/CSET/operators/_utils.py 183 8 74 6 95% src/CSET/operators/ageofair.py 141 7 64 5 94% src/CSET/operators/aggregate.py 76 1 22 1 98% src/CSET/operators/aviation.py 60 0 18 0 100% src/CSET/operators/collapse.py 154 12 72 5 91% src/CSET/operators/constraints.py 107 7 46 2 93% src/CSET/operators/convection.py 37 4 10 2 87% src/CSET/operators/ensembles.py 27 0 14 0 100% src/CSET/operators/feature.py 41 0 10 0 100% src/CSET/operators/filters.py 66 2 30 0 98% src/CSET/operators/humidity.py 139 0 56 0 100% src/CSET/operators/imageprocessing.py 56 0 16 0 100% src/CSET/operators/mesoscale.py 17 0 2 0 100% src/CSET/operators/misc.py 146 0 54 1 99% src/CSET/operators/plot.py 929 169 318 59 78% src/CSET/operators/power_spectrum.py 97 3 30 3 95% src/CSET/operators/precipitation.py 93 0 50 0 100% src/CSET/operators/pressure.py 41 0 12 0 100% src/CSET/operators/read.py 409 38 178 14 89% src/CSET/operators/regrid.py 122 18 64 1 83% src/CSET/operators/scoreswrappers.py 72 6 20 3 90% src/CSET/operators/temperature.py 121 0 32 0 100% src/CSET/operators/transect.py 62 0 24 0 100% src/CSET/operators/wind.py 45 3 10 2 91% src/CSET/operators/write.py 15 0 6 0 100% src/CSET/recipes/__init__.py 101 0 28 0 100% --------------------------------------------------------------------------------------------------- TOTAL 4275 311 1488 108 91% |
|
David Flack (@daflack) I needed the line "ctrl.data" in scores_crps_for_ensemble, I think something weird with dask. pre-commit doesn't like this though because it's treating it as redundant code |
David Flack (daflack)
left a comment
David Flack (daflack)
left a comment
There was a problem hiding this comment.
This PR is looking really good. I've left a couple of comments, including looking at the q you raised, I don't know if my suggestion will work or not though, athough note whilst I was typing this you have done that. I'll think of a few interpretation comments to add as well, whilst I'm reviewing some other PRs.
|
David Flack (@daflack) - Added some simple unit tests to this now. Not sure how how much you want to test, but just did some basic functionality. let me know if you want more. |
|
|
||
| assert isinstance(crps_cube, iris.cube.Cube) | ||
| assert feature_cube.coord("time").shape == crps_cube.coord("time").shape | ||
|
|
There was a problem hiding this comment.
would be also nice to have something like assert np.allclose(crps_cube.data,scores.ensemble.crps(control,ens).data,atol=1e-2,rtol=1e-6) as this will act to show that the calculation is doing what we expect as well - we can do it by calling scores directly as we are not testing the calculation (as that is tested in scores), instead here we are testing that the wrapper produces the same calculation as the original version of the code.
| def test_crps_one_time_coord(feature_cube): | ||
| """Test handling of only one time point in cube provided.""" | ||
| feature_cube_one_time = feature_cube[:, 0, :, :] | ||
| with pytest.raises(ValueError): |
There was a problem hiding this comment.
if you have a specific error message you could also add match='your message' too.
|
David Flack (@daflack) Made those changes to the tests |
|
Thanks, looks good. I'll let James Frost (@jfrost-mo) do a final review to make sure he is happy - he may think of some other relevant tests, and may be able to point out what is needed to fix the workflow tests. |
3 participants
Fixes #2215 . I have added a new operator that calculates CRPS for ensemble data. Added new recipe that uses this operator and a loader.
Contribution checklist
Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.
rose-suite.conf.examplehas been updated if new diagnostic added.