WRF Feature, Capabilities, Infrastructure Testing

For the release, a number of existing capability options in the WRF code need to be explicitly tested to insure the continued correct working of the model for users.  Among these options tested are:
       adaptive time step
       analysis nudging
       observational nudging
       DFI
       diffusion approach
       program NDOWN
       program OBSGRID
       sst update
       Global Model
       Vortex Following

While the testing of some of the new physics options is done for 28 48-h time periods, all of the new physics options are run through a couple of canned cases, and verified with subjective analysis.  The level of success is easier to achieve, generally producing a complete forecast that gives reasonable results.  The specific new additions to the model are scrutinized much more closely.  An important consideration for this level of testing is the interworking of schemes, from source-code incompatibility to the consequences of unphysical results.  Testing for these new features includes:
       aerosols (Goddard, Thompson, RRTMG
       CLM
       Grell Freitas cumulus
       Grenier Bretherton PBL
       GRIMS shallow cumulus scheme
       HUJI SBM (regular and fast modes)
       icloud option 3 (new)
       lightning options 1, 2, 3, 11
       Milbrandt 2-moment
       Morrison Gettleman microphysics
       NoahMP land surface
       Noah subgrid tiling 
       NSSL microphysics options 17, 19, 21
       NWP diagnostics
       ozone options 0, 2
       sea ice
       scale aware cumulus scheme (new)
       SKEBS (backscatter scheme)
       TEMF PBL Scheme 
       Tiedke Cumulus Scheme (new)
       topo_wind option 2
       trajectories
       YSU PBL scheme (new-modified)
       
In addition to modifications to the WRF code directly, the land surface community has been  working to augment the available source for higher-resolution input data.  These data sets are now available through the WPS system (specifically the static data for geogrid).  
       nlcd 2006 (9s)
       nlcd 2011 can (9s)
       nlcd 2011 imp (9s)
       nlcd 2011 (9s)

For each of the above tests (types of land data, various physics options, feature upgrades, previous capability re-assurances), a fixed set of permutations were conducted:
       single domain case
       nested case, one-way-nest, and nest-starts-later cases 
       restart run
       compared output from original run to restart run to ensure they were identical

As part of the assurance that the WRF code is not suffering from accidental side-effects, all of the idealized cases have been run, comparing the previous release and the current WRF 3.7 release.  The cases all build, run, and produce similar results.  As some of the cases use physics schemes that have been upgraded, it is not really possible to get identical results in most of these tests. This portion of the testing handles all of the lateral boundary condition options in WRF, and exercises many of the dynamics switches. A new idealized case, convective radiation (em_convrad) is introduced with version 3.7, and has also been tested.

The feature testing was conducted on a Linux platform. All tests were run with MPI for both Intel V12.1.5 and GNU V4.8.2. 


Things that do not work:
       Global Run does not give bit for bit output when running a restart
       HUJI SBM can only run on a very small grid (less than 10km) and core dumps with the GNU compiler
       TEMF PBL scheme does not give bit for bit output when running a restart 
       NSSL option 17 needs a small time-step (4xDX)