Testing for WRF Version 3.8
The WRF version 3.8 has undergone fairly extensive testing. However, ignoring the deprecated physics options, the combinatorial space for possible major physics suites totals more than 2 million options. Once you start adding in the capabilities in dynamics, urban effects, shallow convection, and impacts of climatologically sensitive constituents, then the task becomes even more challenging. With recent access to our new NCAR supercomputer, yellowstone, we have been able to more thoroughly test on compilers traditionally available to users: Intel, PGI, and GNU. Admittedly, we are unable to exhaustively test the WRF code. The following describes our testing practices over the past few months.
There are three main areas on which we focus our efforts. There is a month-long winter period (January 2016) and a similar length case for summer (June 2015) that covers the contiguous US at 20-km resolution. A total of 28 48-h forecasts were conducted over these time periods for each of five selected suites of physics options. The forecast results from these tests were then compared to NCEP GFS surface and upper-air analyses, subjectively and objectively. The purpose of this testing is to insure the validity of the forecast performance of the new schemes. Click here for details.
The second type of testing involves WRF capabilities, features, and infrastructure. Users rely on the availability of restarts that provide identical restarts, inclusion and use of extra data (such as with SST updates and data assimilation), digital filtering, adaptive time-stepping, and re-use of the WRF system data (through such programs as NDOWN and OBSGRID). These tests incorporate a spectrum of options, with some affording a definitive correct answer (i.e. yes, these results are indeed identical), while others are more subjectively based (as in there seems to be lightning in the areas expected). The feature testing assures users that when the option for diagnostics is selected, or when trajectories are requested, or output of vertical profiles is desired, then the correct data is available. Click here for details.
The final type of testing is entirely software oriented, and primarily is oriented towards covering as many physics options as possible. This testing assumes that the WRF code that runs on a single processor is correct, and that the WRF parallel infrastructure needs to be able to repeat those values using differing numbers of processors. This testing is heavily automated since the correctness of the solution is determined by successfully reproducing the exact bit-for-bit results with the different core counts. Click here to see details.
Architectures with Regression Tests and Other Tests
The primary testing system at NCAR has been the Linux-like machine, yellowstone, which supports multiple compilers such as GNU, Intel and PGI. The model code has been tested extensively using the following versions of the compilers:
- gfortran 4.9.2
- ifort 15.0.1, 16.0.2
- pgi 15.1
