General Info

About wrfhelp

wrf-news list

wrf-users list

Becoming a Registered User

WRF Workshop

WRF Tutorial

 

 

 

The WRF Physics Review Process

Background

 

In recent years, there has been considerable discussion about the large number of physics options in the WRF Model and growing sentiment that it would be beneficial to maintain fewer physics options and to focus on the development, performance, and understanding of a limited set of physics suites.  Changes in the approach to WRF physics to address these ideas were discussed at the 2015 WRF Users' Workshop and received strong support from the community.  The discussions also endorsed the formation of a physics review panel that, with solid community representation and expertise, would screen and make recommendations on the addition of proposed new physics for WRF.  At the ensuing WRF Research Applications Board (RAB) meeting, the RAB approved the proposed changes and suggested scientists from the community with extensive experience with WRF physics to initially oversee the revised process.  Through further discussions, the procedures for the review and approval of new physics packages for WRF were refined, and the WRF Physics Review Panel (PRP) (e-mail: wprp@ucar.edu) was created.  The WRF physics review process and the role of the PRP are described below.

 

For new physics packages submitted for inclusion in WRF, the developers will be required to demonstrate that the package provides an improvement or feature not available in other WRF physics and is of broad potential interest to users.  In addition, sufficient testing of the package must be provided to demonstrate its beneficial features, along with documentation that clearly explains the scientific basis for the scheme, its intended areas of application, and its proper use and options.

 

 

The Panel and Review Process

 

The WRF PRP consists of a board of three scientists with a history of involvement with WRF and its physics, and who appreciate the issues of both maintaining and advancing and the system for the community.  The panel oversees the review of new physics packages proposed for WRF by developers.  Similar to a journal editorial committee, the panel communicates with prospective contributors and taps relevant experts from the scientific community to review the submissions anonymously.  The panel receives the reviewers' input and makes a recommendation on whether the package should be accepted for inclusion in the WRF repository.  The PRP then communicates this decision to the WRF Developers' Committee (DC), which oversees code additions to, and maintenance of, the WRF repository.  If the DC confirms a PRP recommendation of addition, the submission must then meet the further testing and other usual prerequisites for a commit to the repository trunk.  Note that the PRP is only engaged for submissions of new physics packages, not for other updates such as bugfixes, non-fundamental modifications to existing physics schemes, or developments/modifications of the WRF dynamics, numerics, or software framework.  Additions for WRF-DA and WRF-Chem are also not considered by the panel.

 

The panel begins the review process by considering a letter of intent (LOI) submitted by the developer.  The LOI can be submitted prior to running any additional cases deemed necessary for testing purposes, as the cases appropriate for testing and evaluation may vary according to the physics being proposed.  The LOI is thus a way for developers to avoid testing of physics prior to an initial panel statement of interest.  If there is interest, the panel will then request the more extensive review materials (described below) and engage appropriate reviewers, communicating with the developer as needed.  Based on the reviews, the panel then makes a recommendation to the DC on whether the code should be added to the WRF repository.

 

The panel's recommendations to the Developers' Committee are not mandates, and the DC has the final say on repository integration of submitted code.  Thus, there may be situations in which additions are denied despite a recommendation.  This is because, ultimately, the DC is responsible for the maintenance of the WRF repository, and NCAR/MMM provides support for the WRF system with limited resources.  For any recommendations not adopted, the DC will provide a written explanation to the PRP and the RAB.  For code that is adopted, the package developer will work with a designated point of contact on the DC on technical aspects of implementation of the code into the current version of the repository.

 

Code that is not recommended by the PRP or otherwise accepted in the WRF repository may be added to a "contributed" codeset.  This will be a collection of codes not in the official release, but made available to the community.  These codes will be supported by their specific developers or contributors instead of NCAR, which only supports the released version of the code.

 

The PRP selects reviewers based on their recognized scientific expertise in areas relevant to the submitted physics.  Reviewers agree to review the submission materials in a timely manner (e.g., within three weeks).  The process aims to avoid unduly burdening the reviewers or hampering the addition of worthwhile physics to WRF. 

 

The reviews of the materials are akin to reviews of proposals, which result in a recommendation on the merits.  They are not, however, like reviews of journal articles, where iterations on questions and revisions routinely occur.  Any lack of consensus among the reviewers is resolved by the PRP.

 

 

Review Process and Materials Required

 

The process for new physics review has two stages: (1) letter of intent and (2) review of materials.  After these, the PRP provides its recommendation to the Developers' Committee on whether the new package merits inclusion in the WRF repository.  The PRP may be contacted via wprp@ucar.edu .

 

The following are the procedures and requirements for the submission and evaluation of new physics packages for WRF.  While the avoidance of undue burden for all involved is desired, the overarching goal is to enable the panel and reviewers to make determinations for the ultimate betterment of WRF and its future maintenance and support.

 

Requirements

 

Letter of Intent

 

Developers seeking to submit physics packages must first submit a letter of intent to the PRP.  This allows the panel, first, to determine whether the submission should proceed and, second, to tailor for relevance the testing sought.  The letter can be short (e.g., a half-page or so), but must provide statements on: what is being proposed and what is new about it, and the testing performed or to be performed.  Merit criteria at this stage are (i) scientific soundness, (ii) novelty compared to WRF’s current capabilities, (iii) potential for interest from the community. Based on the letter, the panel will direct the developer on whether to proceed with the submission and, if so, suggested testing.  The developer can then prepare and submit the required materials for testing and documentation.

 

Testing

 

The developer must conduct tests with WRF simulations to demonstrate clearly the scientific improvement from, or advantages of, the package and provide the results of these tests to the PRP.  At this stage, however, testing is not required to address other issues (e.g., coding standards, computational issues) that are important, but that will eventually be handled by the DC prior to any commit.  The developer must also address any interactions or dependencies of the proposed package with other WRF schemes/code. 

 

To ensure the relevance of the results and to assist in their review and interpretation, the testing must be done with at least one model configuration using recommended/commonly-used WRF packages.  To this end, there will be designated subsets of 2–3 acceptable namelist options for each of the main WRF physical process areas (e.g., microphysics, PBL, etc.) for use in the test simulations.  In addition, for the submission's targeted physical process, the testing must include comparisons with one or more (at the discretion of the panel) of the other recommended WRF options for that process.  Thus, if the developer is proposing the addition of a new PBL scheme, the testing should compare the proposed PBL scheme with at least one of the currently-available WRF PBL schemes that are among those recommended.  Prospective contributors should outline their specific testing plans in their letter of intent to reach agreement with the PRP on the appropriate schemes for comparison before testing begins.

 

The PRP will communicate to the developer the testing that will be required, which may involve the simulation of cases as described in "Information for Code Contributors" (www2.mmm.ucar.edu/wrf/users/contrib_info.php) or other applications that the PRP deems appropriate for the area addressed, based on the nature of the package.  For testing analysis, verification against observations and statistical evaluation of the results are highly desired.  At the discretion of the PRP, published results relevant to the proposed technique(s) may also be considered in meeting the testing requirements.

 

The testing should demonstrate improvement or benefits from the new package.  The standard for assessment by the reviewers and the PRP is that the proposed physics package or code should be comparable to a publishable development.  Here, this means that it: (a) is sufficiently novel (not just incremental) and (b) demonstrates potential for advancement over what exists in WRF. 

 

Documentation

 

Documentation is essential to the understanding and evaluation of the proposed code by the reviewers, the panel, and subsequent users.  It should explain what the code is and does, what the advancement is, and what the intended areas of application are.  In addition, the documentation within the code itself must be sufficient so that future users and WRF support can understand it.

 

The documentation provided with submissions of new physics must be sufficient to explain:

 

(i) The scheme, its motivation, and the principles/assumptions of its development;

(ii) Why the approach is an improvement over existing schemes/code in WRF;

(iii) How the scheme works and whether/how it is connected to other physics packages; and  

(iv) All of its adjustable parameters.

 

E-mail: All communications for the WRF Physics Review Panel, including letters of intent and queries, may be directed to: wprp@ucar.edu .

 

 



 
Home -- Model System -- User Support -- Doc / Pub -- Links -- Download -- WRF Real-time Forecast