============================================================== WRF Software Verification Testing Prior to Major Code Releases ============================================================== Testing is conducted prior to each major WRF code release to insure bit-for-bit behavior on differing processor counts. Testing is conducted on hundreds of short forecasts (about 10 time steps) to activate numerous physics options. If single processor versus multiple processor results differ, there is a strong likelihood of improper initialization of variables, missing communications, or race conditions. Physics options that exhibit clean bit-wise reproducible results are more likely to be robust. This testing is conducted automatically when modifications or new additions are proposed via the `WRF GitHub repository `_. | | | WRF Test Framework ================== The WRF Testing Framework (WTF) is designed to build, test, and analyze test results for one or more versions of the WRF model, using the NSF NCAR HPC. A complete WRF test (compiling WRF, running tests, and analyzing results) takes about three hours. WRF is tested regularly for the following compilers, parallel processing configurations, compile-time variations, and run-time variations of the WRF software: | | | Compilers --------- GNU fortran | | | | Parallel Build Configurations ----------------------------- Serial => S (single-processor) build |br| OpenMP => O (multithreaded, shared memory) build |br| MPI => M (multiprocessor, distributed memory) build | | | | WRF Compile-time Variations --------------------------- *Build configurations are shown in parentheses beside each variation. See above section for key* ARW (SOM) |br| ARW real*8 (SOM) |br| ARW Nested (SOM) |br| ARW Moving Nest (M) |br| CHEM (S M) |br| CHEM with KPP (S M) |br| Idealized Fire (SOM) |br| Idealized Super Cell (SOM) |br| Idealized Super Cell real*8 (SOM) |br| Idealized Baroclinc Wave (SOM) |br| Idealized 2D Hill (S) |br| Idealized Single Column Model (S) |br| WRFPlus |br| WRFDA-4DVAR | | | | ARW em_real Run-time Variations ------------------------------- Adaptive Time Stepping |br| Digital Filtering |br| FDDA: Spectral nudging, obs nudging, surface and upper air nudging |br| Stochastic Forcing |br| Nesting |br| Vertical Nesting |br| Hybrid and Terrain Following Vertical Coordinates |br| Dry and Moist Potential Temperature |br| restart | | | | | Physics Options Applied in WRF Tests ==================================== The following Tables and associated table keys summarize the combinations of physics options that are tested. * Each row in the table represents a specific test, and each column a specific physics option. * Each physics combination listed can be considered a "safe" combination that should provide successful short-term forecasts with bit-for-bit results when comparing single-processor output against multi-processor output. | .. note:: While the choice of one physics option should not influence the choice of another physics option (e.g., the choice of a microphysics scheme should be independent of the cumulus scheme), in practice certain options are developed and tested for a small subset of other physics option combinations. Therefore, the following table is a useful guide for physics combinations known to provide bit-for-bit results between serial and MPI versions of WRF. | | | Key 1 : Column Labels (Tables 1-7) ---------------------------------- NL => Test Namelist Identifier |br| PBL => Planetary Boundary Layer Scheme |br| CU => Cumulus Scheme |br| MP => Microphysics Scheme |br| LW => Longwave Radiation Scheme |br| SW => Shortwave Radiation Scheme |br| SFC => Surface Physics Scheme |br| LAND => Land Surface Scheme |br| URB => Urban Physics Scheme |br| SHCU => Shallow Cumulus Scheme |br| TOPO => Topography-Following Wind Scheme | | | | Key 2: Test Namelist Codes (Tables 1-7) ---------------------------------------- AD => Adaptive Time Stepping |br| BN => Binary WRF Output |br| DF => Digital Filtering |br| FD => FDDA |br| GR => Grib 1 WRF Output |br| NE => Basic Nesting |br| QT => Quilting |br| VN => Vertical Nesting | | | | Table 1 ------- This table lists WRF tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs. | .. csv-table:: :escape: \ :header: Namelist, PBL, CU, MP, LW, SW, SFC, PBL, URB, SHCU, TOPO cmt, 1, 11, 6, 4, 4, 1, 2, 0, 0, 0 conus, C, C, C, C, C, C, C, 0, 0, 0 kiaps1NE, 11, 14, 16, 14, 14, 1, 4, 0, 4, 0 kiaps2, 1, 3, 16, 14, 14, 91, 4, 1, 4, 0 rap, 5, 1, 28, 4, 4, 5, 3, 0, 0, 0 solaraNE, 5, 16, 8, 4, 4, 5, 2, 0, 0, 0 tropical, T, T, T, T, T, T, T, 0, 0, 0 urb3bNE, 8, 3, 16, 14, 14, 2, 4, 3, 0, 0 03, 4, 3, 3, 24, 24, 4, 1, 0, 0, 0 03DF, 4, 6, 3, 4, 4, 4, 1, 0, 0, 0 03FD, 4, 14, 3, 4, 4, 4, 1, 0, 0, 0 3dtke, 0, C, C, C, C, 1, C, 0, 0, 0 06, 8, 2, 6, 24, 24, 2, 1, 0, 0, 0 07NE, 8, 6, 8, 5, 5, 1, 2, 2, 0, 0 10, 4, 14, 10, 1, 2, 4, 7, 0, 0, 0 11, 4, 2, 10, 1, 2, 4, 7, 0, 0, 0 14, 4, 2, 3, 3, 3, 4, 3, 0, 0, 0 16, 9, 6, 8, 5, 5, 2, 7, 0, 0, 0 17, 2, 1, 4, 3, 3, 2, 2, 0, 0, 0 17AD, 2, 1, 4, 3, 3, 2, 2, 0, 0, 0 18, 10, 14, 8, 5, 5, 10, 7, 0, 0, 0 20, 12, 3, 4, 1, 2, 1, 2, 0, 0, 0 20NE, 12, 1, 4, 1, 2, 1, 2, 0, 0, 0 38, 2, 1, 2, 4, 4, 2, 7, 0, 0, 0 48, 4, 1, 3, 24, 24, 4, 1, 0, 0, 0 49, 1, 3, 3, 24, 24, 91, 2, 0, 0, 0 50, 1, 3, 3, 24, 24, 91, 4, 0, 0, 0 51, 1, 3, 3, 24, 24, 91, 4, 0, 0, 0 52, 4, 11, 17, 24, 24, 4, 1, 0, 0, 0 52DF, 4, 11, 17, 4, 4, 4, 1, 0, 0, 0 52FD, 4, 7, 17, 4, 4, 4, 1, 0, 0, 0 60, 1, 1, 6, 24, 24, 1, 4, 0, 0, 0 60NE, 1, 1, 6, 4, 4, 1, 4, 0, 0, 0 65DF, 9, 1, 28, 4, 4, 2, 3, 0, 0, 0 66FD, 4, 14, 3, 4, 4, 4, 1, 0, 0, 0 71, 1, 1, 8, 4, 4, 1, 2, 0, 0, 0 78, 1, 1, 52, 4, 4, 1, 2, 0, 0, 0 79, 5, 14, 2, 4, 4, 2, 7, 0, 0, 0 | | | | Table 2 ------- This table lists WRF tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs for real*8 compilations. | .. csv-table:: :escape: \ :header: Namelist, PBL, CU, MP, LW, SW, SFC, PBL, URB, SHCU, TOPO 14, 4, 6, 3, 3, 3, 4, 3, 0, 0, 0 17AD, 2, 2, 4, 3, 3, 2, 2, 0, 0, 0 | | | | Table 3 ------- This table lists WRF idealized supercell tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs. | .. csv-table:: :escape: \ :header: Namelist, PBL, CU, MP, LW, SW, SFC 02NE, 0, 0, 1, 0, 0, 1 03, 0, 0, 1, 0, 0, 1 03NE, 0, 0, 1, 0, 0, 1 04, 0, 0, 2, 0, 0, 1 | | | | Table 4 ------- This table lists WRF idealized supercell tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs for real*8 compilations. | .. csv-table:: :escape: \ :header: Namelist, PBL, CU, MP, LW, SW, SFC 02NE, 0, 0, 1, 0, 0, 1 03, 0, 0, 1, 0, 0, 1 03NE, 0, 0, 1, 0, 0, 1 04, 0, 0, 2, 0, 0, 1 | | | | Table 5 ------- This table lists WRF idealized b-wave tests that provide successful 10-time-step forecasts, and bit-for-bit results for serial versus MPI runs and serial versus OpenMP runs. | .. csv-table:: :escape: \ :header: Namelist, PBL, CU, MP, LW, SW, SFC 1NE, 0,0,1,0,0,0 2,0,0,1,0,0,0 2NE,0,0,1,0,0,0 3,0,0,2,0,0,0 | | | | |