WRF Feature, Capabilities, Infrastructure Testing

For the release, a number of existing capability options in the WRF code need to be explicitly tested to insure the continued correct working of the model for users.  Among these options tested are:
       adaptive time step
       analysis nudging
       observational nudging
       DFI
       program NDOWN
       program OBSGRID
       sst update
       TC Bogus
       Global Model
       Vortex Following

While the testing of some of the new physics options is done for 28 48-h time periods, all of the new physics options are run through a couple of canned cases, and verified with subjective analysis.  The level of success is easier to achieve, generally producing a complete forecast that gives reasonable results.  The specific new additions to the model are scrutinized much more closely.  An important consideration for this level of testing is the interworking of schemes, from source-code incompatibility to the consequences of unphysical results.  Testing for these new features includes:
       aerosols (new - Goddard and RRTMG schemes have been modified)
       CLM
       Grell Freitas cumulus
       Grenier Bretherton PBL
       GRIMS shallow cumulus scheme
       HUJI SBM (new)
       Lake Model
       lightning options 1, 2, 11
       Milbrandt 2-moment
       Morrison Gettleman microphysics
       NoahMP land surface
       Noah subgrid tiling (new)
       NSSL microphysics options 17 (new), 19, 21
       NWP diagnostics
       ozone options 0, 2
       sea ice
       SKEBS (backscatter scheme)
       TEMB PBL Scheme 
       Thompson Scheme (new - aerosol climatology added)
       topo_wind option 2
       trajectories
       
In addition to modifications to the WRF code directly, the land surface community has been  working to augment the available source for higher-resolution input data.  These data sets are now available through the WPS system (specifically the static data for geogrid).  
       MODIS LAI (10m, 30s)
       Landuse MODIS (15s)
       VARSSO (2m, 5m, 10m)
       Lake Depth

For each of the above tests (types of land data, various physics options, feature upgrades, previous capability re-assurances), a fixed set of permutations were conducted:
       single domain case
       nested case, one-way-nest, and nest-starts-later cases 
       restart run
       compared output from original run to restart run to ensure they were identical

As part of the assurance that the WRF code is not suffering from accidental side-effects, all of the idealized cases have been run, comparing the previous release and the current WRF 3.6 release.  The cases all build, run, and produce similar results.  As some of the cases use physics schemes that have been upgraded, it is not really possible to get identical results in most of these tests. This portion of the testing handles all of the lateral boundary condition options in WRF, and exercises many of the dynamics switches.

The feature testing was conducted on a Linux platform. All tests were run with MPI for both Intel V12.1.5 and PGI 12.5, while a few were also tested with GNU V4.8.2. 


Things that do not work:
	   Global Run does not give bit for bit output when running a restart
       HUJI SBM can only run on a very small grid (less than 10km)
       NSSL option 19 does not give bit for bit output when running a restart with a nest
       NSSL option 21 same as above
       NWP Diagnostics core dumps unless using only 1 processor (PGI only)
       SKEBS does not have a restart capability at this time
       TEMF PBL scheme does not give bit for bit output when running a restart (PGI and GNU only)
       Thompson aerosol scheme does not give bit for bit output when running a restart
       new geography data with RUC input
       new geography data with noahmp
       new geography data with CLM