WRF Features, Capabilities, and infrastructure Testing

For the release, a number of existing capability options in the WRF code need to 
be explicitly tested to ensure the model continues to correctly work for users.  
Among these tested options are:
       adaptive time step
       analysis nudging (with and without surface nudging)
       observational nudging
       digital filtering initialization
       diffusion options (diff_opt=1,2; km_opt=1,2,3,4; damp_opt=1,3;  w_damping=1)
       using MPAS input
       nest starts later than parent
       one-way nest
       program NDOWN
       restart
       sst update
       vertical nesting
       vortex following (moving nest - standard, and with highres input option)
       All idealized cases (3D cases tested in serial/parallel, 2D/1D cases only tested in serial)

While the testing of some of the new physics options is done for 28 48-h time periods, 
all of the new physics options are run through a couple of canned cases, and verified 
with subjective analysis.  The level of success is easier to achieve, generally producing 
a complete forecast that gives reasonable results.  The specific new additions to the 
model are scrutinized much more closely.  An important consideration for this level of 
testing is the interworking of schemes, from source-code incompatibility to the 
consequences of unphysical results.  Testing for these new features includes:
       Aerosols: 
            RRTMG (aer_opt = 1)
	    Goddard
            Thompson
       Microphysics:
            CAM
            Goddard (4ice)
            HUJI SMB, regular and fast options
            Jensen
	    Milbrandt
            Morrison Gettleman
            NSSL (options 17, 19, 21)
	    P3 (options 50, 51)
            Thompson aerosol-aware
            WDM7/WSM7
       PBL Physics:
            Grenier-Bretherton-McCaa
            TEMF
            YSU
       Cumulus Physics:
            Grell 3
            Grell-Freitas 
            Kain-Fritsch (kfcup option, scale-aware)
            New Tiedtke
       Shallow Cumulus Physics:
            Deng
            Grims
       Radiation Physics:
            Goddard
            Ozone (options 0, 2)
       Surface Physics:
            CLM
            Noah Mosaid subgrid tiling
       Physics Suites (CONUS and Tropical)
       icloud option 3
       lightning options 1, 2, 3, 11
       NWP diagnostics
       sea ice
       SKEBS (backscatter scheme)
	   smooth options 1, 2
       topo_wind option 1, 2
       trajectories
       urban physics 1, 2, 3
       
For each of the above tests (types of land data, various physics options, feature 
upgrades, previous capability re-assurances), a fixed set of permutations were conducted:
       single domain case, concurrent nesting, restart, nested restart
The restart runs compared output from an original end-to-end run to the restarted simulation 
to ensure the two data sets were bit-wise identical.

The feature testing was conducted on a Linux platform. All tests were run with MPI (with
the exception of the idealized cases, which were run serially) with GNU V8.1.0, and some with Intel V17.0.1.132.


The following are known to NOT give bit for bit identical answers with a restart:
       Global Run
       SKEBS
       Adaptive Time Step (with a nested domain)
       lightning options 1, 2, 11
       Goddard Radiation/Microphysics 4-ice
       urban options 1, 2, 3       
       ndown

The following have had problems completing a short simulation:
       lightning option 3 (with a nest)
       HUJI SMB (fast option) microphysics

NSSL option 17 needs a small time-step (4xDX).