Testing for WRF Version 3.0
The testing for the WRF version 3.0 was conducted by numerous individuals, internally and externally of NCAR, including staff at DTC, NOAA GSD, DoD AFWA, and NOAA NCEP. A large group of university, government lab, and commercial developers provided feedback during a month-long friendly-user period for the pre-release of WRF. Locally, the overall testing consisted of a broad spectrum of investigations: architecture ports, compiler version tests, bit-for-bit comparisons, restart tests, test-case 1-way and 2-way nesting forecasts, physics interoperability between NMM and ARW, 4-km convective forecasts, high-resolution hurricane tracks with automated moving nests, real-time runs (20 km, and 36/12 km) at NCAR/MMM, cycling tests with variational assimilation, along with tests conducted by developers prior to introducing new features (global domains, digital filtering, chemistry, adaptive time stepping, and a number of new physics packages). Following is a brief description of the testing for the version 3.0 release of the WRF modeling system.
Automated Tests
A large amount of the weekly testing of the WRF code is handled through suites of very short forecasts that determine if serial, OpenMP, and MPI outputs are bitwise identical. These automated regression tests are conducted locally at NCAR on three architectures: IBM/AIX/xlf, MAC/Darwin/g95, and Intel32/FC/pgf90. For each of these machines, short forecasts are conducted with several physics options, with real-data and idealized cases, for ARW and NMM cores. Subsets of these tests are re-run for nesting, fully optimized, RSL_LITE, 8-byte reals, netcdf/grib/binary output, analysis and observation nudging, global domains, and chemistry.
The WRF system has a small, fully-optimized test that runs through several domains (CONUS, southern Europe, and New Zealand). For each of these domains, several input Grib files are selected (eight for CONUS, one for Europe, and two for New Zealand). Each of these initializations produces a 6-h, 30-km forecast. These are non-cycled, end-to-end tests that confirm the interoperability of the WPS components, the real pre-processor, the WRF model, and a graphical post-processor RIP. For domains with multiple initial sources, visual comparisons of the forecasts are conducted.
Smoke Tests
On the IBM, DEC-HP, and Intel32 systems, several
nesting 24-h forecasts are conducted with the distributed-memory,
fully-optimized code (all 30/10 km). These include a 1-way nesting
forecast using ndown, a 2-way nesting forecast with a single coarse-grid
input file, a 2-way nesting forecast with both a coarse and a
fine grid input, and a 2-way nest with a specified moving inner
domain.
The WPS package has a small test that runs through
several domains (CONUS, southern Europe, and New Zealand). For
each of these domains, several input Grib files are selected (eight
for CONUS, one for Europe, and two for New Zealand). Each of these
initializations produces a 6-h, 30-km forecast. For domains with
multiple initial sources, visual comparisons of the forecasts
are conducted.
Architectures with Regression Tests
The primary testing system at NCAR has been the two IBM machines, bluevista and blueice. During the last month prior to the release, the regression testing has included a multi-processor MAC/Intel running the g95 compiler. Just prior to the code release, the testing has included an Intel IA32 with PGI.
bluevista AIX
xlfrte 11.1.0.2
blueice AIX
xlfrte 10.1.0.4
bay-mmm Linux 2.6.18-53.1.13.el5 #1 SMP
pgf90 6.2-5 32-bit target on x86 Linux
stink Darwin Kernel Version 8.11.1
g95 gcc version 4.0.3 (g95 0.91!)
