WRF Feature, Capabilities, Infrastructure Testing

For the release, a number of existing capability options in the WRF code need to 
be explicitly tested to ensure the continued correct working of the model for users.  
Among these tested options are:
       adaptive time step
       analysis nudging
       observational nudging
       spectral nudging
       obsgrid
       digital filtering initialization
       diffusion options
       nest starts later than parent
       one-way nest
       program NDOWN
       restart
       sst update
       vertical nesting
       vortex following (moving nest)
       All idealized cases 

While the testing of some of the new physics options is done for 28 48-h time periods, 
all of the physics options are run through a couple of canned cases, and verified 
with subjective analysis.  The level of success is easier to achieve, generally producing 
a complete forecast that gives reasonable results.  The specific new additions to the 
model are scrutinized much more closely.  An important consideration for this level of 
testing is the interworking of schemes, from source-code incompatibility to the 
consequences of unphysical results.  Testing for these features includes:

Microphysics
Kessler (option 1)
Lin (option 2)
WSM3/5/6/7 (options 3, 4, 6, 24)
Eta/Ferrier (option 5)
Goddard 4-ice (option 7)
Milbrandt (option 9)
Morrison (option 10)
CAM (option 11)
SBU Ylin (option 13)
WDM5/6/7 (options 14, 16, 26)
NSSL (options 17, 18, 19, 21, 22)
Thompson (options 8, 28)
SBM Fast/Full (options 30, 32)
Morrison/CESM aerosol (option 40)
P3 (options 50, 51, 52) (option 53 - *new*)
Jensen (option 55)
NTU (option 56) (*new*)

PBL   
YSU (option 1)
MYJ (option 2)
QNSE (option 4)
MYNN2/3 (options 5, 6)
BouLac (option 8)
UW (option 9)
TEMF (option 10)
Shin Hong (option 11)
Grenier Bretherton (option 12)
E-Epsilon (*new*) (option 16)

Cumulus
Kain-Fritsch/CUP/scale-aware (options 1, 10, 11)
BMJ (option 2)
Grell/Freitas/3/Devenyi (options 3, 5, 93)
Old SAS (option 4)
Tiedtke/new (option 6, 16)
Zhang McFarlane (option 7)
Kiaps SAS (option 14)

Shallow Cumulus Grims (cu_shcu_physics=3) 
Shallow Cumulus Deng (cu_shcu_physics=5) 

Surface Layer
Revised MM5 (option 1)
Eta/MYJ (option 2)
QNSE (option 4)
MYNN (option 5)
Pleim-Xu (option 7)
TEMF (option 10)

LSM
Slab (option 1)
Noah/MP (options 2, 4)
RUC (option 3)
CLM4 (option 5)
SSIB (option 8)
NoahMP Dynamic Irrigation (*new*)
Noah Mosaic

Radiation
RRTM/Dudhia (option 1)
CAM (option 3)
RRTMG/K/Fast (options 4, 14, 24)
Goddard (option 5)
FLG (option 7)
GFDL (option 99)
Goddard Aerosol (aer_opt)
Solar Eclipse (ra_sw_eclipse = 1) (*new*)

Urban
UCM (option 1)
BEP (option 2)
BEM (option 3)
LCZ(wudapt) (use_wudapt_lcz=1) (*new*)

Other Options
icloud option 3
Lake model
Lightning options 1, 2, 3, 11
NWP diagnostics
Ozone options 0, 2
sea ice
SKEBS (backscatter scheme)
smooth options 1, 2
topo_wind options 1, 2
trajectories
Noah Subgrid Tiling
Boundary Control
damp_opt=1,2
w-damping
IEVA (*new*)
km options 1, 2, 3
MAD_WRF options 1, 2, with/without cloud initialization (*new*)
Using MPAS input
cold start clouds with Thompson MP (*new*)
ncd_nofill = .true.
Phi Advect Z (*new*)
       
For each of the above tests (types of land data, various physics options, feature 
upgrades, previous capability re-assurances), a fixed set of permutations were conducted:
       single domain case, concurrent nesting, restart, nested restart
The restart runs compared output from an original end-to-end run to the restarted simulation 
to ensure the two data sets were bit-wise identical.

As part of the assurance that the WRF code is not suffering from accidental side-effects, 
all of the idealized cases have been built and run.
This portion of the testing handles all of the lateral boundary condition options in WRF, 
and exercises many of the dynamics switches. 

The feature testing was conducted on a Linux platform. All ideal tests were run serially
with Intel V19.0.5.281.

The following are known to NOT give bit for bit identical answers with a restart:
        SKEBS
	adaptive time-stepping (when using nested feedback)
	Eta-Ferrier microphysics
	SBM Full microphysics
	GFDL Radiation
	Lightning Options 1 and 2
        LCZ Wudapt with Urban option 3
	
       
The following are known to be problematic and may not run to completion:
	Lightning Option 3
	SSiB LSM (when using restart option)
        NSSL option 22 microphysics 
	Obs Nudging
	 
	
NSSL option 17 needs a small time-step (4xDX).