WRF Software Verification Testing Prior to Major Code Releases¶
Testing is conducted prior to each major WRF code release to insure bit-for-bit behavior on differing processor counts. Testing is conducted on hundreds of short forecasts (about 10 time steps) to activate numerous physics options. If single processor versus multiple processor results differ, there is a strong likelihood of improper initialization of variables, missing communications, or race conditions. Physics options that exhibit clean bit-wise reproducible results are more likely to be robust.
This testing is conducted automatically when modifications or new additions are proposed via the WRF GitHub repository.
WRF Test Framework¶
The WRF Testing Framework (WTF) is designed to build, test, and analyze test results for one or more versions of the WRF model, using the NSF NCAR HPC. A complete WRF test (compiling WRF, running tests, and analyzing results) takes about three hours. WRF is tested regularly for the following compilers, parallel processing configurations, compile-time variations, and run-time variations of the WRF software:
Compilers¶
GNU fortran
Parallel Build Configurations¶
Serial => S (single-processor) build
OpenMP => O (multithreaded, shared memory) build
MPI => M (multiprocessor, distributed memory) build
WRF Compile-time Variations¶
Build configurations are shown in parentheses beside each variation. See above section for key
ARW (SOM)
ARW real*8 (SOM)
ARW Nested (SOM)
ARW Moving Nest (M)
CHEM (S M)
CHEM with KPP (S M)
Idealized Fire (SOM)
Idealized Super Cell (SOM)
Idealized Super Cell real*8 (SOM)
Idealized Baroclinc Wave (SOM)
Idealized 2D Hill (S)
Idealized Single Column Model (S)
WRFPlus
WRFDA-4DVAR
ARW em_real Run-time Variations¶
Adaptive Time Stepping
Digital Filtering
FDDA: Spectral nudging, obs nudging, surface and upper air nudging
Stochastic Forcing
Nesting
Vertical Nesting
Hybrid and Terrain Following Vertical Coordinates
Dry and Moist Potential Temperature
restart
Physics Options Applied in WRF Tests¶
The following Tables and associated table keys summarize the combinations of physics options that are tested.
Each row in the table represents a specific test, and each column a specific physics option.
Each physics combination listed can be considered a “safe” combination that should provide successful short-term forecasts with bit-for-bit results when comparing single-processor output against multi-processor output.
Note
While the choice of one physics option should not influence the choice of another physics option (e.g., the choice of a microphysics scheme should be independent of the cumulus scheme), in practice certain options are developed and tested for a small subset of other physics option combinations. Therefore, the following table is a useful guide for physics combinations known to provide bit-for-bit results between serial and MPI versions of WRF.
Key 1 : Column Labels (Tables 1-7)¶
NL => Test Namelist Identifier
PBL => Planetary Boundary Layer Scheme
CU => Cumulus Scheme
MP => Microphysics Scheme
LW => Longwave Radiation Scheme
SW => Shortwave Radiation Scheme
SFC => Surface Physics Scheme
LAND => Land Surface Scheme
URB => Urban Physics Scheme
SHCU => Shallow Cumulus Scheme
TOPO => Topography-Following Wind Scheme
Key 2: Test Namelist Codes (Tables 1-7)¶
AD => Adaptive Time Stepping
BN => Binary WRF Output
DF => Digital Filtering
FD => FDDA
GR => Grib 1 WRF Output
NE => Basic Nesting
QT => Quilting
VN => Vertical Nesting
Table 1¶
This table lists WRF tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs.
Namelist |
PBL |
CU |
MP |
LW |
SW |
SFC |
PBL |
URB |
SHCU |
TOPO |
|---|---|---|---|---|---|---|---|---|---|---|
cmt |
1 |
11 |
6 |
4 |
4 |
1 |
2 |
0 |
0 |
0 |
conus |
C |
C |
C |
C |
C |
C |
C |
0 |
0 |
0 |
kiaps1NE |
11 |
14 |
16 |
14 |
14 |
1 |
4 |
0 |
4 |
0 |
kiaps2 |
1 |
3 |
16 |
14 |
14 |
91 |
4 |
1 |
4 |
0 |
rap |
5 |
1 |
28 |
4 |
4 |
5 |
3 |
0 |
0 |
0 |
solaraNE |
5 |
16 |
8 |
4 |
4 |
5 |
2 |
0 |
0 |
0 |
tropical |
T |
T |
T |
T |
T |
T |
T |
0 |
0 |
0 |
urb3bNE |
8 |
3 |
16 |
14 |
14 |
2 |
4 |
3 |
0 |
0 |
03 |
4 |
3 |
3 |
24 |
24 |
4 |
1 |
0 |
0 |
0 |
03DF |
4 |
6 |
3 |
4 |
4 |
4 |
1 |
0 |
0 |
0 |
03FD |
4 |
14 |
3 |
4 |
4 |
4 |
1 |
0 |
0 |
0 |
3dtke |
0 |
C |
C |
C |
C |
1 |
C |
0 |
0 |
0 |
06 |
8 |
2 |
6 |
24 |
24 |
2 |
1 |
0 |
0 |
0 |
07NE |
8 |
6 |
8 |
5 |
5 |
1 |
2 |
2 |
0 |
0 |
10 |
4 |
14 |
10 |
1 |
2 |
4 |
7 |
0 |
0 |
0 |
11 |
4 |
2 |
10 |
1 |
2 |
4 |
7 |
0 |
0 |
0 |
14 |
4 |
2 |
3 |
3 |
3 |
4 |
3 |
0 |
0 |
0 |
16 |
9 |
6 |
8 |
5 |
5 |
2 |
7 |
0 |
0 |
0 |
17 |
2 |
1 |
4 |
3 |
3 |
2 |
2 |
0 |
0 |
0 |
17AD |
2 |
1 |
4 |
3 |
3 |
2 |
2 |
0 |
0 |
0 |
18 |
10 |
14 |
8 |
5 |
5 |
10 |
7 |
0 |
0 |
0 |
20 |
12 |
3 |
4 |
1 |
2 |
1 |
2 |
0 |
0 |
0 |
20NE |
12 |
1 |
4 |
1 |
2 |
1 |
2 |
0 |
0 |
0 |
38 |
2 |
1 |
2 |
4 |
4 |
2 |
7 |
0 |
0 |
0 |
48 |
4 |
1 |
3 |
24 |
24 |
4 |
1 |
0 |
0 |
0 |
49 |
1 |
3 |
3 |
24 |
24 |
91 |
2 |
0 |
0 |
0 |
50 |
1 |
3 |
3 |
24 |
24 |
91 |
4 |
0 |
0 |
0 |
51 |
1 |
3 |
3 |
24 |
24 |
91 |
4 |
0 |
0 |
0 |
52 |
4 |
11 |
17 |
24 |
24 |
4 |
1 |
0 |
0 |
0 |
52DF |
4 |
11 |
17 |
4 |
4 |
4 |
1 |
0 |
0 |
0 |
52FD |
4 |
7 |
17 |
4 |
4 |
4 |
1 |
0 |
0 |
0 |
60 |
1 |
1 |
6 |
24 |
24 |
1 |
4 |
0 |
0 |
0 |
60NE |
1 |
1 |
6 |
4 |
4 |
1 |
4 |
0 |
0 |
0 |
65DF |
9 |
1 |
28 |
4 |
4 |
2 |
3 |
0 |
0 |
0 |
66FD |
4 |
14 |
3 |
4 |
4 |
4 |
1 |
0 |
0 |
0 |
71 |
1 |
1 |
8 |
4 |
4 |
1 |
2 |
0 |
0 |
0 |
78 |
1 |
1 |
52 |
4 |
4 |
1 |
2 |
0 |
0 |
0 |
79 |
5 |
14 |
2 |
4 |
4 |
2 |
7 |
0 |
0 |
0 |
Table 2¶
This table lists WRF tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs for real*8 compilations.
Namelist |
PBL |
CU |
MP |
LW |
SW |
SFC |
PBL |
URB |
SHCU |
TOPO |
|---|---|---|---|---|---|---|---|---|---|---|
14 |
4 |
6 |
3 |
3 |
3 |
4 |
3 |
0 |
0 |
0 |
17AD |
2 |
2 |
4 |
3 |
3 |
2 |
2 |
0 |
0 |
0 |
Table 3¶
This table lists WRF idealized supercell tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs.
Namelist |
PBL |
CU |
MP |
LW |
SW |
SFC |
|---|---|---|---|---|---|---|
02NE |
0 |
0 |
1 |
0 |
0 |
1 |
03 |
0 |
0 |
1 |
0 |
0 |
1 |
03NE |
0 |
0 |
1 |
0 |
0 |
1 |
04 |
0 |
0 |
2 |
0 |
0 |
1 |
Table 4¶
This table lists WRF idealized supercell tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs for real*8 compilations.
Namelist |
PBL |
CU |
MP |
LW |
SW |
SFC |
|---|---|---|---|---|---|---|
02NE |
0 |
0 |
1 |
0 |
0 |
1 |
03 |
0 |
0 |
1 |
0 |
0 |
1 |
03NE |
0 |
0 |
1 |
0 |
0 |
1 |
04 |
0 |
0 |
2 |
0 |
0 |
1 |
Table 5¶
This table lists WRF idealized b-wave tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs.
Namelist |
PBL |
CU |
MP |
LW |
SW |
SFC |
|---|---|---|---|---|---|---|
1NE |
0 |
0 |
1 |
0 |
0 |
0 |
2 |
0 |
0 |
1 |
0 |
0 |
0 |
2NE |
0 |
0 |
1 |
0 |
0 |
0 |
3 |
0 |
0 |
2 |
0 |
0 |
0 |