In recent years, there has been
considerable discussion about the large number of physics options in the WRF
Model and growing sentiment that it would be beneficial to maintain fewer
physics options and to focus on the development, performance, and understanding
of a limited set of physics suites. Changes
in the approach to WRF physics to address these ideas were discussed at the
2015 WRF Users' Workshop and received strong support from the community. The discussions also endorsed the formation of
a physics review panel that, with solid community representation and expertise,
would screen and make recommendations on the addition of proposed new physics for
WRF. At the ensuing WRF Research
Applications Board (RAB) meeting, the RAB approved the proposed changes and suggested
scientists from the community with extensive experience with WRF physics to initially
oversee the revised process. Through
further discussions, the procedures for the review and approval of new physics
packages for WRF were refined, and the WRF Physics Review Panel (PRP) (e-mail: firstname.lastname@example.org) was created. The WRF physics review process and the role of the PRP are described below.
For new physics packages submitted for
inclusion in WRF, the developers will be required to demonstrate that the package
provides an improvement or feature not available in other WRF physics and is of
broad potential interest to users. In
addition, sufficient testing of the package must be provided to demonstrate its
beneficial features, along with documentation that clearly explains the
scientific basis for the scheme, its intended areas of application, and its proper
use and options.
Panel and Review Process
The WRF PRP consists of a board of three
scientists with a history of involvement with WRF and its physics, and who
appreciate the issues of both maintaining and advancing and the system for the
community. The panel oversees the review
of new physics packages proposed for WRF by developers. Similar to a journal editorial committee, the
panel communicates with prospective contributors and taps relevant experts from
the scientific community to review the submissions anonymously. The panel receives the reviewers' input and
makes a recommendation on whether the package should be accepted for inclusion
in the WRF repository. The PRP then communicates
this decision to the WRF Developers' Committee (DC), which oversees code
additions to, and maintenance of, the WRF repository. If the DC confirms a PRP recommendation of addition,
the submission must then meet the further testing and other usual prerequisites
for a commit to the repository trunk.
Note that the PRP is only engaged for submissions of new physics
packages, not for other updates such
as bugfixes, non-fundamental modifications to
existing physics schemes, or developments/modifications of the WRF dynamics, numerics, or software framework. Additions for WRF-DA and WRF-Chem are also not considered by the panel.
The panel begins the review process by
considering a letter of intent (LOI) submitted by the developer. The LOI can be submitted prior to running any
additional cases deemed necessary for testing purposes, as the cases appropriate
for testing and evaluation may vary according to the physics being
proposed. The LOI is thus a way for
developers to avoid testing of physics prior to an initial panel statement of
interest. If there is interest, the
panel will then request the more extensive review materials (described below) and
engage appropriate reviewers, communicating with the developer as needed. Based on the reviews, the panel then makes a
recommendation to the DC on whether the code should be added to the WRF
The panel's recommendations to the
Developers' Committee are not mandates, and the DC has the final say on repository
integration of submitted code. Thus, there
may be situations in which additions are denied despite a recommendation. This is because, ultimately, the DC is
responsible for the maintenance of the WRF repository, and NCAR/MMM provides
support for the WRF system with limited resources. For any recommendations not adopted, the DC
will provide a written explanation to the PRP and the RAB. For code that is adopted, the package
developer will work with a designated point of contact on the DC on technical
aspects of implementation of the code into the current version of the
Code that is not recommended by the PRP
or otherwise accepted in the WRF repository may be added to a "contributed"
will be a collection of codes not in the official release, but made available
to the community. These codes will be
supported by their specific developers or contributors instead of NCAR, which only
supports the released version of the code.
The PRP selects reviewers based on their
recognized scientific expertise in areas relevant to the submitted physics. Reviewers agree to review the submission materials
in a timely manner (e.g., within three weeks).
The process aims to avoid unduly burdening the reviewers or hampering
the addition of worthwhile physics to WRF.
The reviews of the materials are akin to
reviews of proposals, which result in a recommendation on the merits. They are not, however, like reviews of journal
articles, where iterations on questions and revisions routinely occur. Any lack of consensus among the reviewers is
resolved by the PRP.
Review Process and Materials Required
The process for
new physics review has two stages: (1) letter of intent and (2) review of
materials. After these, the PRP provides
its recommendation to the Developers' Committee on whether the new package merits
inclusion in the WRF repository. The PRP
may be contacted via email@example.com .
are the procedures and requirements for the submission and evaluation of new
physics packages for WRF. While the avoidance
of undue burden for all involved is desired, the overarching goal is to enable
the panel and reviewers to make determinations for the ultimate betterment of
WRF and its future maintenance and support.
Letter of Intent
seeking to submit physics packages must first submit a letter of intent to the
PRP. This allows the panel, first, to
determine whether the submission should proceed and, second, to tailor for
relevance the testing sought. The letter
can be short (e.g., a half-page or so), but must provide statements on: what is
being proposed and what is new about it, and the testing performed or to be
performed. Merit criteria at this stage
are (i) scientific soundness, (ii) novelty compared
to WRF’s current capabilities, (iii) potential for interest from the community.
Based on the letter, the panel will direct the developer on whether to proceed with
the submission and, if so, suggested testing.
The developer can then prepare and submit the required materials for
testing and documentation.
must conduct tests with WRF simulations to demonstrate clearly the scientific improvement
from, or advantages of, the package and provide the results of these tests to
the PRP. At this stage, however, testing
is not required to address other issues (e.g., coding standards, computational
are important, but that will eventually be handled by the DC prior to any commit. The developer must also address any
interactions or dependencies of the proposed package with other WRF schemes/code.
To ensure the
relevance of the results and to assist in their review and interpretation, the
testing must be done with at least one model configuration using
recommended/commonly-used WRF packages.
To this end, there will be designated subsets of 2–3 acceptable namelist options for each of the main WRF physical process
areas (e.g., microphysics, PBL, etc.) for use in the test simulations. In addition, for the submission's targeted physical
process, the testing must include comparisons with one or more (at the
discretion of the panel) of the other recommended WRF options for that process. Thus, if the developer is proposing the
addition of a new PBL scheme, the testing should compare the proposed PBL
scheme with at least one of the currently-available WRF PBL schemes that are
among those recommended. Prospective
contributors should outline their specific testing plans in their letter of
intent to reach agreement with the PRP on the appropriate schemes for
comparison before testing begins.
The PRP will
communicate to the developer the testing that will be required, which may involve
the simulation of cases as described in "Information for Code
Contributors" (www2.mmm.ucar.edu/wrf/users/contrib_info.php) or other
applications that the PRP deems appropriate for the area addressed, based on the
nature of the package. For testing analysis,
verification against observations and statistical evaluation of the results are
highly desired. At the discretion of the
PRP, published results relevant to the proposed technique(s) may also be considered
in meeting the testing requirements.
should demonstrate improvement or benefits from the new package. The standard for assessment by the reviewers
and the PRP is that the proposed physics package or code should be comparable
to a publishable development. Here, this
means that it: (a) is sufficiently novel (not just incremental) and (b)
demonstrates potential for advancement over what exists in WRF.
essential to the understanding and evaluation of the proposed code by the reviewers,
the panel, and subsequent users. It
should explain what the code is and does, what the advancement is, and what the
intended areas of application are. In
addition, the documentation within the code itself must be sufficient so that future
users and WRF support can understand it.
documentation provided with submissions of new physics must be sufficient to
(i) The scheme,
its motivation, and the principles/assumptions of its development;
(ii) Why the approach is an improvement
over existing schemes/code in WRF;
(iii) How the scheme works and whether/how
it is connected to other physics packages; and
(iv) All of its adjustable
E-mail: All communications for the WRF Physics
Review Panel, including letters of intent and queries, may be directed to: firstname.lastname@example.org .