WRF Feature, Capabilities, Infrastructure Testing

For the release, a number of existing WRF capability options must 
be explicitly tested to ensure they continue to work as expected.  
Among these tested options are:
       adaptive time step
       analysis nudging
       diffusion options
       digital filtering initialization (DFI)
       idealized cases 
       NDOWN program
       nest starts later than parent
       observational nudging
       obsgrid
       one-way nest
       restart
       spectral nudging
       sst update
       vertical nesting
       vortex following (moving nest)

While the testing of some of the new physics options is done for 28 48-h time periods, 
all of the physics options are run through a couple of canned cases, and verified 
with subjective analysis. New additions to the model are scrutinized much more closely. 
An important consideration for this level of testing is the interworking of schemes, 
from source-code incompatibility to the consequences of unphysical results.  
Testing for these features includes:

Physics Suites
CONUS
tropical

Microphysics
CAM (option 11)
Eta/Ferrier (option 5)
Goddard 4-ice (option 7)
Jensen (option 55)
Kessler (option 1)
Lin (option 2)
Milbrandt (option 9)
Morrison (option 10)
Morrison/CESM aerosol (option 40)
NSSL (options 17, 18, 19, 21, 22)
NTU (option 56)
P3 (options 50, 51, 52, 53)
SBM Fast/Full (options 30, 32)
SBU Ylin (option 13)
Thompson (options 8, 28)
WSM3/5/6/7 (options 3, 4, 6, 24)
WDM5/6/7 (options 14, 16, 26)

PBL   
BouLac (option 8)
E-Epsilon (option 16)
Grenier Bretherton (option 12)
MYJ (option 2)
MYNN2/3 (options 5, 6)
QNSE (option 4)
Shin Hong (option 11)
TEMF (option 10)
UW (option 9)
YSU (option 1)

Cumulus
BMJ (option 2)
Grell/Freitas/3/Devenyi (options 3, 5, 93)
Kain-Fritsch/CUP/scale-aware (options 1, 10, 11)
Kiaps SAS (option 14)
Old SAS (option 4)
Tiedtke/new (option 6, 16)
Zhang McFarlane (option 7)

Shallow Cumulus Deng (cu_shcu_physics=5) 
Shallow Cumulus Grims (cu_shcu_physics=3) 

Surface Layer
Eta/MYJ (option 2)
MYNN (option 5)
Pleim-Xu (option 7)
QNSE (option 4)
Revised MM5 (option 1)
TEMF (option 10)

LSM
CLM4 (option 5)
Noah/MP (options 2, 4)
NoahMP Dynamic Irrigation (opt_irr=1,2 and opt_irrm=1,2,3)
NoahMP Runoff and Groundwater (opt_run = 1,2,3,4,5i)(=6,7,8 *new*)
Noah Mosaic
RUC (option 3)
Slab (option 1)
SSIB (option 8)

Radiation
CAM (option 3)
FLG (option 7)
GFDL (option 99)
Goddard (option 5)
Goddard Aerosol (aer_opt)
RRTM/Dudhia (option 1)
RRTMG/K/Fast (options 4, 14, 24)
Solar Eclipse (ra_sw_eclipse = 1)

Urban
BEM (option 3)
BEP (option 2)
LCZ(wudapt) (use_wudapt_lcz=1)
UCM (option 1)

Other Options
cold start clouds with Thompson MP
damp_opt=1,2
icloud option 3
IEVA 
km options 1, 2, 3
Lake model
Lightning options 1, 2, 3, 11
MAD_WRF options 1, 2, with/without cloud initialization 
Using MPAS input
ncd_nofill = .true.
Noah Subgrid Tiling
NWP diagnostics
Output Accumulated physics tendencies (*new*)
Ozone options 0, 2
Phi Advect Z 
sea ice
Shallow water roughness (*new*)
SKEBS (backscatter scheme)
smooth options 1, 2
topo_wind options 1, 2
trajectories
w-damping
WRF-Solar EPS (*new*)
       
For each of the above tests (types of land data, various physics options, feature 
upgrades, previous capability re-assurances), a fixed set of permutations were conducted:
concurrent nesting and restart. The restart runs compared output from an original end-to-end run to the restarted simulation 
to ensure the two data sets were bit-wise identical.

As part of the assurance that the WRF code is not suffering from accidental side-effects, 
all of the idealized cases have been built and run.
This portion of the testing handles all of the lateral boundary condition options in WRF, 
and exercises many of the dynamics switches. 

The feature testing was conducted on a Linux platform with the following compilers.
Ifort V19.1.1
GNU V10.1.0
 
All tests were run with distributed memory, with the exception of the 1D and 2D idealized
cases, which were run serially.

The following are known to NOT give bit for bit identical answers with a restart:
        Lightning option 11	
	WRF-Solar EPS
       
The following are known to be problematic and may not run to completion:
	Lightning Option 3
	SSiB LSM (when using restart option)
	
NSSL option 17 needs a small time-step (4xDX).