Forecast Verification Testing for Newly-released Code¶
Prior to each WRF code major release, simulations are run to verify that new and/or modified code produces reasonable and expected results. Forecasts include each of a selected combination of physics and dynamics options. Results are then used as follows:
Output from simulations with the new options is compared to observations and analyses (NCEP GFS surface and upper air), subjectively and objectively. Domain averaged BIAS and RMSE are calculated for conventional fields.
Output from simulations with the new options is compared to a suite of well-tested physics options.
A baseline test, using code that includes the new options, but may not be using them, is compared to the same baseline test using the previous version of WRF. Those baseline options are:
RRTMG long- and shortwave radiation
Tiedtke cumulus parameterization
YSU PBL scheme
Noah surface scheme
WSM5 microphysics
Below are the new or modified options tested for each release.
V4.5 Release¶
The test configuration is as follows:
Test dates : 1-28 February, 2017 and 1-28 May, 2017
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 415 x 325 horizontal grid points, 40 vertical levels, model top at 30 hPa
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis, 6-hourly intervals
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
26 cases are run for both the summer and winter dates
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Tests¶
The following options were tested for this release:
The new package to compute two-moment prognostics for graupel/hail in Thompson scheme
The new k-epsilon-theta PBL scheme
Updated P3 scheme
Scale-aware Tiedtke scheme
Updated Mynn-EDMF scheme
Tropical Suite
Conus Suite
Conus Suite but run with NoahMP LSM scheme
Options used for climate simulation
Comparisn of NoahMP-V4.4 and NoahMP-V4.5
V4.4 Release¶
The test configuration is as follows:
Test dates : 1-28 February, 2017 and 1-28 May, 2017
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 415 x 325 horizontal grid points, 40 vertical levels, model top at 30 hPa
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis, 6-hourly intervals
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
48 cases are run for both the summer and winter dates
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Tests¶
The following options were tested for this release:
updated Thompson microphysics and cloud fraction schemes
aerosol-aware Thompson scheme with new black carbon data
updated P3 1-ice category, 1-moment cloud water scheme
new rrtmg cloud overlap scheme
updated mynn sfclay scheme
WRF-solar EPS model
Run-time option of greenhouse gas
V4.3 Release¶
The test configuration is as follows:
Test dates : 1-28 February, 2017 and 1-28 May, 2017
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 415 x 325 horizontal grid points, 40 vertical levels, model top at 30 hPa
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis, 6-hourly intervals
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
48 cases are run for both the summer and winter dates
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Results from the CONUS suite tests are also verified against station observations and soundings archived in the GDAS historical dataset
Tests¶
The following options were tested for this release:
The turbulence kinetic energy (TKE) and TKE dissipation rate based 1.5-order closure (EEPS) PBL parameterization
The updated version of P3 microphsyics scheme that includes a one-ice category, 3-moment ice option
The NTU (National Taiwan University) scheme that applies double moments for the liquid-phase and triple moments for the ice-phase hydrometeors together with the consideration for ice crystal shape and density variations.
The enhanced GWD parameterization developed by NOAA
The multilayer BEP and BEP+BEM urban canopy models (UCMs) coupled with the Yonsei University (YSU) planetary boundary layer parameterization
The new dynamic irrigation scheme implemented in NoahMP
The Implicit-Explicit Vertical Advection (IEVA) scheme
V4.2 Release¶
The test configuration is as follows:
Test dates : 1-28 January, 2017 and 1-31 May, 2017
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 415 x 325 horizontal grid points, 40 vertical levels
Grid location : 39N, -101W
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis, 6-hourly intervals
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
31 and 28 cases are run for the spring and winter dates, respectively
Note
Two extra suites of 3km resolution runs are conducted for the period 1-31 May 2017 to test spectral bin microphysics and predicted particle property (P3) bulk microphysics.
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Results from the CONUS suite tests are also verified against station observations and soundings archived in the GDAS historical dataset
Precipitation from the 15 km and 3 km simulations is verified with StageIV observations.
Tests¶
Option |
Radiation |
Cumulus |
Microphysics |
Sfc Layer |
Surface |
PBL |
|---|---|---|---|---|---|---|
Conus 4.1.5 |
RRTMG |
Modified Tiedtke |
Thompson |
Janjic |
Noah |
MYJ |
Tropical 4.1.5 |
RRTMG |
New Tiedtke |
WSM6 |
Old MM5 |
Noah |
YSU |
MSKF |
RRTMG |
MSKF |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
BMJ |
RRTMG |
BMJ |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
MYNN |
RRTMG |
MSKF |
WSM5 |
MYNN |
Noah |
MYNN |
IRRI-3 |
RRTMG |
MSKF |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
3d TKE |
RRTMG |
Modified Tiedtke |
WSM5 |
Monin-Obukhov |
Noah |
Off |
FARMS |
RRTMG |
Modified Tiedtke |
Thompson |
Janjic |
Noah |
YSU |
FSBM (3km) |
RRTMG |
Off |
FSBM |
Monin-Obukhov |
Noah |
YSU |
P3 (3km) |
RRTMG |
Off |
P3 |
Monin-Obukhov |
Noah |
YSU |
V4.1 Release¶
The test configuration is as follows:
Test dates : 1-28 June, 2015, and 1-28 January, 2016
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 401 x 261 horizontal grid points, 40 vertical levels, model top at 30 hPa
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
48 cases are run for both the summer and winter dates.
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Results from the STD suite tests are also verified against station observations and soundings archived in the GDAS historical dataset.
Tests¶
Records missing. This will be updated if they are found.
V4.0 Release¶
The test configuration is as follows:
Test dates : 1-28 June, 2015, and 1-28 January, 2016
Domain configuration : CONUS domain with 15 km resolution
Grid structure : 401 x 261 horizontal grid points, 40 vertical levels, model top at 30 hPa
Map projection : Lambert-conformal
Initial and boundary conditions : NCEP GFS 0.25 degree analysis
Temporal information : each case is initialized at 00 UTC and runs for 48 hours
48 cases are run for both the summer and winter dates.
Verification¶
All results are compared to 0.25 degree GFS surface and upper air analyses.
Domain-averaged BIAS and RMSE are calcuted for conventional fields
Tests¶
Option |
HVC |
Moist |
Radiation |
Cumulus |
Microphysics |
Sfc Layer |
Surface |
PBL |
|---|---|---|---|---|---|---|---|---|
Stdhyb-v3.9.1.1 |
on |
on |
RRTMG |
Kfeta |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
Stdold-v3.9.1.1 |
off |
on |
RRTMG |
Kfeta |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
Stdv40 |
on |
on |
RRTMG |
Kfeta |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
Stdv40thm0 |
on |
off |
RRTMGK |
Kfeta |
WSM5 |
Monin-Obukhov |
Noah |
YSU |
Kiaps |
on |
on |
RRTMGK |
Ksas |
WSM6 |
Monin-Obukhov |
Noah |
YSU |
Aercu |
on |
on |
RRTMG |
MSKF |
Morrison TM Aero |
Monin-Obukhov |
Noah |
YSU |
Rap |
on |
on |
RRTMG |
Grell-Freitas |
Thompson Aero |
MYNN |
Ruc |
MYNN |
Rap2 |
on |
on |
RRTMG |
Grell-Freitas |
Thompson Aero |
MYNN |
Noah |
MYNN |
tropical |
on |
on |
RRTMG |
New Tiedtke |
WSM6 |
Old MM5 |
Noah |
YSU |
CONUS-v3.9.1.1 |
off |
off |
RRTMG |
Modified TKE |
Thompson |
Monin-Obukhov |
Noah |
MYJ |
CONUS-v4.0 |
on |
on |
RRTMG |
Modified TKE |
Thompson |
MYJ |
Noah |
MYJ |