Testing for WRF Version 4.4
WRF version 4.4 has undergone extensive testing; however, the combination of possible physics suites totals more than 2 million options. That number increases with the addition of dynamics options, urban effects, shallow convection, and impacts of climatologically sensitive constituents. Therefore we are unable to exhaustively test the WRF code. The following describes the three areas on which we focus of testing.
28 48-hr forecasts are conducted over a month-long winter period (February 2017) and late spring period (May 2017). The domain covers the contiguous US at 15-km resolution. Forecasts include each of a selected combination of physics/dynamics options. Forecast results from these tests are then compared to observations and analyses (NCEP GFS surface and upper-air), subjectively and objectively. This testing validates forecast performance of new schemes. (details)
The second type of testing involves WRF capabilities, features, and infrastructure. It is expected that restarts provide identical results as if the same simulation was run from beginning to end, without restarting in the middle. These tests are run with both as a single simulation, and then again, starting from a restart time to ensure bit-for-bit results. This ensures the capabilities work as expected, and work with a restart. The feature testing also assures users that when the option for diagnostics is selected, or when trajectories are requested, or output of vertical profiles is desired, then the correct data is available. (details)
The final type of testing is entirely software-oriented, and primarily oriented towards covering as many physics options as possible. This testing assumes that the WRF code that runs on a single processor is correct, and that the WRF parallel infrastructure is able to repeat those values using differing numbers of processors. This testing is heavily automated since the correctness of the solution is determined by reproducing bit-for-bit results with different core counts. An advantage for the user is that this testing happens for each proposed source code modification (a pull request on github) to the WRF repository. (details)
Architectures with Regression Tests and Other Tests
The primary testing system at NCAR for case and feature testing has been the Linux-based machine, cheyenne, which supports multiple compilers such as GNU and Intel. The model code has been tested extensively using the following versions of the compilers:
- gfortran 10.1.0
- ifort 19.1.1
The automated testing is conducted inside of docker containers, with Linux OS, with compilers:
- gfortran 9.3.1
