WRF Features and Capabilities Testing Prior to Major Code Releases

For each of the tested options listed below, the following test is conducted.

  • Nested domain, usually with two nests (unless nesting is not applicable)

  • A standard WRF simulation is run

  • A restart is run from some point in the middle of the standard simulation

  • Output from both the standard simulation and the restart test are compared at the same ending time, for all domains, to ensure bit-for-bit output.

This testing handles all WRF lateral boundary condition options, and exercises many of the dynamics switches. All tests were run with distributed memory, with the exception of the 1D and 2D idealized cases, which were run serially.


Prior to a release, the following existing WRF capability options are tested to ensure they continue to work as expected.

  • adaptive time step

  • analysis nudging

  • diffusion options

  • digital filtering initialization (DFI)

  • idealized cases (restarts not tested)

  • NDOWN program

  • nest starts later than parent

  • observational nudging

  • obsgrid

  • one-way nest

  • restart

  • spectral nudging

  • sst update

  • vertical nesting

  • vortex following (moving nest)




Physics Options

The following physics options are tested for each release.



Suites

  • CONUS (testing began with v4.1)

  • tropical (testing began with v4.1)





Microphysics

  • CAM

  • Eta/Ferrier (testing began with v4.2)

  • Goddard 4-ice

  • Jensen (testing began with v4.1)

  • Kessler (testing began with v4.2)

  • Lin (testing began with v4.2)

  • Milbrandt

  • Morrison

  • Morrison/CESM aerosol (testing began with v4.2)

  • NSSL options

  • NTU (testing began with v4.3)

  • P3 options

  • SBM Fast/Full

  • SBU Ylin (testing began with v4.2)

  • Thompson (testing began with v4.2)

  • Thompson aerosol (testing began with v4.1)

  • Thompson GH (testing began with v4.5)

  • WSM3/5/6 (testing began with v4.2)

  • WDM5/6 (testing began with v4.2)

  • WDM7/WSM7 (testing began with v4.1)





PBL

  • BouLac (testing began with v4.2)

  • E-Epsilon (testing began with v4.3)

  • Grenier Bretherton

  • K Epsilon Theta^2 (testing began with v4.5)

  • MYJ (testing began with v4.2)

  • MYNN2/3 (testing began with v4.2)

  • QNSE (testing began with v4.2)

  • Shin Hong (testing began with v4.2)

  • TEMF

  • UW (testing began with v4.2)

  • YSU





Cumulus

  • BMJ (testing began with v4.2)

  • Grell-Freitas

  • Grell-3 (testing began with v4.1)

  • Grell-Devenyi (testing began with v4.2)

  • KF-Cup

  • Kain-Fritsch (testing began with v4.1)

  • Kain-Fritsch Scale-aware

  • Kiaps SAS (testing began with v4.2)

  • Old SAS (testing began with v4.2)

  • Tiedtke (testing began with v4.2)

  • New Tiedtke

  • Zhang McFarlane (testing began with v4.2)

  • Shallow Cumulus Deng (testing began with v4.1)

  • Shallow Cumulus Grims





Surface Layer

  • Eta/MYJ (testing began with v4.2)

  • MYNN (testing began with v4.2)

  • Pleim-Xu (testing began with v4.2)

  • QNSE (testing began with v4.2)

  • Revised MM5 (testing began with v4.2)

  • TEMF





LSM

  • CLM

  • Noah (testing began with v4.1)

  • NoahMP

  • NoahMP Dynamic Irrigation (opt_irr=1,2 and opt_irrm=1,2,3) (testing began with v4.3)

  • NoahMP Runoff and Groundwater (testing began with v4.4)

  • Noah Mosaic (testing began with v4.1)

  • RUC (testing began with v4.2)

  • Slab (testing began with v4.2)

  • SSIB (testing began with v4.2)





Radiation

  • CAM (testing began with v4.2)

  • FLG (testing began with v4.2)

  • GFDL (testing began with v4.2)

  • Goddard (Testing began with v4.1)

  • Goddard 4-ice (testing began with v4.1)

  • Goddard Aerosol (aer_opt)

  • RRTM/Dudhia (testing began with v4.2)

  • RRTMG/K/Fast (testing began with v4.2)

  • RRTMG Aerosol (with aer_opt=1)

  • Solar Eclipse (ra_sw_eclipse = 1) (testing began with v4.3)





Urban

  • BEM

  • BEM distributed drag, aerodynamic options (testing began with v4.6)

  • BEP

  • LCZ (wudapt) (testing began with v4.3)

  • UCM





Other Options

  • cold start clouds with Thompson MP (testing began with v4.3)

  • damp_opt=1,2 (testing began with v4.3)

  • icloud option 3

  • IEVA (testing began with v4.3)

  • km options 1, 2, 3 (testing began with v4.3)

  • Lake model

  • Lightning options 1, 2, 3, 11

  • MAD_WRF options 1, 2, with/without cloud initialization (testing began with v4.3)

  • Using MPAS input (testing began with v4.3)

  • ncd_nofill = .true. (testing began with v4.3)

  • Noah Subgrid Tiling

  • NWP diagnostics

  • Output Accumulated physics tendencies (testing began with v4.4)

  • Ozone options 0, 2

  • Phi Advect Z (testing began with v4.3)

  • sea ice

  • Shallow water roughness (testing began with v4.4)

  • SKEBS (backscatter scheme)

  • smooth options 1, 2

  • topo_wind option 2

  • trajectories

  • w-damping (testing began with v4.3)

  • windfarm superposition options (testing began with v4.6)

  • windfarm wake options (testing began with v4.6)

  • WRF-Solar EPS (testing began with v4.4)






Issues Found with Testing

Below are the issues found with each round of testing.



v4.6

New Issues lightning options 1, 2, and 3 do not give bit-for-bit results with a restart, when compared to a non-restart simulation.

Ongoing Issues

  • Eta/Ferrier Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Goddard 4-ice Radiation: WRF error interpolating ozone

  • E-epsilon PBL : restarts do not produce bit-for-bit results when compared to a non-restart simulation with GNU compiler; works okay with Intel

  • Old SAS Cumulus : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • ndown : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning option 11 : WRF segfault with Intel compiler; works okay with GNU

  • QNSE Surface Layer : WRF segfault with Intel compiler; works okay with GNU

  • SSIB : WRF segfault during restart

  • BEM Urban : restarts do not produce bit-for-bit results when compared to a non-restart simulation with GNU compiler; works okay with Intel

  • WRF-Solar EPS : restarts do not produce bit-for-bit results when compared to a non-restart simulation





v4.5

Ongoing Issues

  • Eta/Ferrier Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Goddard 4-ice Radiation: WRF error interpolating ozone

  • E-epsilon PBL : restarts do not produce bit-for-bit results when compared to a non-restart simulation with GNU compiler; works okay with Intel

  • Old SAS Cumulus : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • ndown : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning option 11 : WRF segfault with Intel compiler; works okay with GNU

  • QNSE Surface Layer : WRF segfault with Intel compiler; works okay with GNU

  • SSIB : WRF segfault during restart

  • BEM Urban : restarts do not produce bit-for-bit results when compared to a non-restart simulation with GNU compiler; works okay with Intel

  • WRF-Solar EPS : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • MAD_WRF : restarts do not produce bit-for-bit results when compared to a non-restart simulation





v4.4

New Issues

  • WRF-Solar EPS : restarts do not produce bit-for-bit results when compared to a non-restart simulation

Ongoing Issues

  • Eta/Ferrier Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Goddard 4-ice Radiation: WRF error interpolating ozone

  • Old SAS Cumulus : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning option 11 : WRF segfault with Intel compiler; works okay with GNU

  • Lightning option 3 : WRF error “corrupted double-linked list”

  • SSIB : WRF segfault during restart





v4.3

New Issues

  • LCZ (wudapt) with Urban option 3 : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • NSSL Microphysics option 22 : WRF fails

  • Observational Nuding : WRF fails

Ongoing Issues

  • SKEBS : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Adaptive Time-step : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Eta/Ferrier Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • SBM Full Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • GFDL Radiation : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning options 1 and 2 : restarts do not produce bit-for-bit results when compared to a non-restart simulation; works okay when cudt=0

  • Lightning option 3 : WRF fails when using a nest

  • SSIB LSM : restarts do not produce bit-for-bit results when compared to a non-restart simulation





v4.2

New Issues

  • Eta/Ferrier Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • GFDL Radiation : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • SSIB LSM : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • SBM Full Microphysics : restarts do not produce bit-for-bit results when compared to a non-restart simulation

Ongoing Issues

  • Adaptive Time-step : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning options 1, 2, and 11 : restarts do not produce bit-for-bit results when compared to a non-restart simulation; works okay when cudt=0

  • Lightning option 3 : WRF fails when using a nest

  • Goddard 4-ice Radiation : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • SKEBS : restarts do not produce bit-for-bit results when compared to a non-restart simulation





v4.1

New Issues

  • Adaptive Time-step : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning options 1, 2, and 11 : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Lightning option 3 : WRF fails when using a nest

  • Goddard 4-ice Radiation : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • Urban options 1, 2, and 3 : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • ndown : restarts do not produce bit-for-bit results when compared to a non-restart simulation

  • SBM Fast Microphysics : WRF fails

Ongoing Issues

  • SKEBS : restarts do not produce bit-for-bit results when compared to a non-restart simulation





v4.0

  • SKEBS : restarts do not produce bit-for-bit results when compared to a non-restart simulation