General Info

About wrfhelp

wrf-news list

WRF Workshop

WRF Tutorial

 

 

 

WRF Software Testing

The testing conducted on the WRF code to ensure bit-for-bit behavior on differing processor counts runs through hundreds of short forecasts (only about 10 time-steps) in about an hour. The purpose is to activate as many possible physics options. If single processor vs. multiple processor results differ, then there is a strong likelihood that improper initialization of variables, missing communications, or race conditions exist. While tracking down the root cause of the problem is extremely time-consuming, physics options that exhibit clean bit-wise reproducible results are more likely to be robust. This testing is handled entirely with a mechanism designed to run on small desktops, as well as on batch systems.

 

     WRF Test Framework

1. Overview

The WRF Testing Framework is designed to build, test, and analyze test results for one or more versions of the WRF model. With the advent of NCAR's flagship mainframe, Yellowstone, in early 2013, the testing framework for WRF was rewritten to accommodate Yellowstone's additional flexibility. For example, a large number of Fortran compilers are available now, instead of just a single compiler that was available on NCAR's Bluefire machine. Yellowstone's faster speed and greater job throughput has also allowed the number of tests performed for WRF to expand. Despite all of the additional variations available on Yellowstone, a complete WRF test run (compiling WRF, running tests, and analyzing results) often takes less than one hour on Yellowstone. WRF is now tested regularly for the following compilers, parallel processing configurations, compile-time variations, and run-time variations of the WRF software:
COMPILERS

GNU Fortran version 4.7.2 and 4.8.2
PGI Fortran version 12.5, 12.10, 13.3, and 13.9
Intel Fortran version 12.1.5 and 14.0.2
PARALLEL BUILD CONFIGURATIONS

Serial (single-processor) build
OpenMP (multithreaded, shared memory) build
MPI (multiprocessor, distributed memory) build
WRF COMPILE-TIME VARIATIONS

ARW
NMM
NMM Nested
NMM HWRF
ARW CHEM
ARW CHEM with KPP
Idealized Super Cell
Idealized Baroclinic Wave
ARW em_real RUN-TIME VARIATIONS

Adaptive Time-Stepping
Digital Filtering
FDDA
Grib 1 WRF Output
Binary WRF Output
Nesting
Quilting
Global Domain
2. Physics Options Applied in WRF Tests

The following table and associated table key summarize the combinations of physics options that are tested for WRF. It is important to note that while the choice of one physics option should not influence the choice of another physics option (i.e., the choice of a microphysics scheme should be independent of the cumulus scheme), in practice, certain options are developed and tested for a small subset of other physics option combinations; therefore, the following table is useful as a guide for combinations of WRF physics options that are known to provide bit-for-bit results between serial and MPI compilations of WRF. Each row in the table represents a specific test, and each column represents a specific physics option. All of the following tests are exercised using all three compilers on Yellowstone. Each of the physics combinations listed in the tables can be considered "safe" combinations that will provide successful short-term forecasts with bit-for-bit results when comparing single-processor output against distributed-memory multi-processor (MPI, for example) output.


KEY 1: Column Labels (Tables 1-4) KEY 2: Test Namelist Codes (Tables 1-4)
NL   =>  Test Namelist Identifier
PBL   =>  Planetary Boundary Layer Scheme
CU   =>  Cumulus Scheme
MP   =>  Microphysics Scheme
LW   =>  Longwave Radiation Scheme
SW   =>  Shortwave Radiation Scheme
SFC   =>  Surface Physics Scheme
LAND   =>  Land Surface Scheme
URB   =>  Urban Physics Scheme
SHCU   =>  Shallow Cumulus Scheme
TOPO   =>  Topography-Following Wind Scheme
AD   =>  Adaptive Time Stepping
BN   =>  Binary WRF Output
DF   =>  Digital Filtering
FD   =>  FDDA
GR   =>  Grib 1 WRF Output
NE   =>  Basic Nesting
QT   =>  Quilting




TABLE 1:
WRF ARW Tests, Providing Successful 10-Timestep Forecasts,
and Bit-for-Bit Results, on Serial vs. MPI runs

NLPBLCUMPLWSWSFCLANDURBSHCUTOPO
global1131111000
011111111001
021243314000
02GR1243314000
034334441000
03DF4334441000
03FD4334441000
04FD4334441000
057555577000
05AD7555577000
05FD7555577000
068664421000
06BN8664421000
0781477712200
07NE81477712200
089785523000
095091253000
09QT5091253000
1042101247000
1283164412300
12GR83164412300
1397131123020
144633343000
1551425517000
15AD51425517000
1691445527000
16BN91445527000
16DF91445527000
172243322000
17AD2243322000
18106455107000
18BN106455107000
191141215000
2012141212000
20NE12141212000
2511111911001
262111131000
299341215020
29QT9341215020
302341121020
3172143371000
31AD72143371000
3297113315020
3397114415020
3497114412020
3597113312020
3683164412000
36GR83164412000
3797114422020
3851425527000
38AD51425527000
3951425557000
39AD51425557000
4072143371000
412243322000
4242101247000
4329341121000
4597285523000
4642101247000
4797285523000
484334441000
4911344912000
5011344912000
5111344912000
5243174441000
52DF43174441000
52FD43174441000
5311304412001
5411324412001
55FD43174441000




TABLE 2:
WRF ARW Tests, Providing Successful 10-Timestep Forecasts,
and Bit-for-Bit Results, on Serial vs. OpenMP runs

NLPBLCUMPLWSWSFCLANDURBSHCUTOPO
global1131111000
034334441000
03DF4334441000
03FD4334441000
068664421000
06BN8664421000
0781477712200
07NE81477712200
095091253000
09QT5091253000
1042101247000
144633343000
1691445527000
16BN91445527000
16DF91445527000
172243322000
17AD2243322000
18106455107000
18BN106455107000
2012141212000
20NE12141212000
3172143371000
31AD72143371000
3851425527000
4242101247000
484334441000
4911344912000
5011344912000
5111344912000
5243174441000
52DF43174441000
52FD43174441000




TABLE 3:
WRF Idealized Supercell Tests, Providing Successful 10-Timestep Forecasts,
and Bit-for-Bit Results, on Serial vs. Non-serial runs (OpenMP and MPI)

NLPBLCUMPLWSWSFCLANDURBSHCUTOPO
010010000000
01NE0010000000
020010010000
02NE0010010000
030010010000
03NE0010010000
040020010000
04NE0020010000
050020010000
05NE0020010000
0600180010000
06NE00180010000
0700170000000
0800180010000
0900190010000
1000210010000
11NE0010000000
12NE0010000000
13NE0010000000
14NE0010000000




TABLE 4:
WRF Idealized B-wave Tests, Providing Successful 10-Timestep Forecasts,
and Bit-for-Bit Results, on Serial vs. Non-serial runs (OpenMP and MPI)

NLPBLCUMPLWSWSFCLANDURBSHCUTOPO
10010000000
1NE0010000000
20010000000
2NE0010000000
30020000000
3NE0020000000
40020000000
4NE0020000000
50000000000
5NE0000000000

 

    

 

 



 
Home -- Model System -- User Support -- Doc / Pub -- Links -- Download -- WRF Real-time Forecast