Testing for WRF Version 4.3
The WRF version 4.3 has undergone fairly extensive testing. However, ignoring the deprecated physics options, the combinatorial space for possible major physics suites totals more than 2 million options. Once you start adding in the capabilities in dynamics, urban effects, shallow convection, and impacts of climatologically sensitive constituents, then the task becomes even more challenging.
Admittedly, we are unable to exhaustively test the WRF code. The following describes our testing practices.
There are three main areas on which we focus our efforts. There is a month-long winter period (January 2016) and a similar length case for summer (June 2015) that covers the contiguous US at 15-km resolution. A total of 28 48-h forecasts were conducted over these time periods for each of selected combinations of physics/dynamics options. The forecast results from these tests were then compared to observations and analyses (NCEP GFS surface and upper-air), subjectively and objectively. The purpose of this testing is to ensure the validity of the forecast performance of the new schemes. Click here for details.
The second type of testing involves WRF capabilities, features, and infrastructure. Users rely on the availability of restarts that provide identical restarts, inclusion and use of extra data (such as with SST updates and data assimilation), digital filtering, adaptive time-stepping, and re-use of the WRF system data (through such programs as NDOWN and OBSGRID). These tests incorporate a spectrum of options, with some affording a definitive correct answer (i.e., yes, these results are indeed identical), while others are more subjectively based (as in there seems to be lightning in the areas expected). The feature testing assures users that when the option for diagnostics is selected, or when trajectories are requested, or output of vertical profiles is desired, then the correct data is available. Click here for details.
The final type of testing is entirely software oriented, and primarily is oriented towards covering as many physics options as possible. This testing assumes that the WRF code that runs on a single processor is correct, and that the WRF parallel infrastructure needs to be able to repeat those values using differing numbers of processors. This testing is heavily automated since the correctness of the solution is determined by reproducing the bit-for-bit results with the different core counts. An advantage for the user is that this testing happens for each proposed source code modification (a pull request on github) to the WRF repository. Click here to see details.
Architectures with Regression Tests and Other Tests
The primary testing system at NCAR for case and feature testing has been the Linux-based machine, cheyenne, which supports multiple compilers such as GNU and Intel. The model code has been tested extensively using the following versions of the compilers:
- gfortran 9.1.0
- ifort 19.0.5
The automated testing is conducted inside of docker containers, with Linux OS, with compilers:
- gfortran 8.3.0 (for NMM)
- gfortran 9.3.1 (for ARW)
