Information for WRF
is a community modeling system, reflecting a wide range of contributions from
scientists and software engineers. It is
through these contributions that the system improves and advances. To facilitate the process, this document
describes the procedures and standards for adding code. Prospective contributors should familiarize
themselves with this information.
administration of WRF-related code, NCAR is primarily responsible for the
support of the main WRF system (WRF, WRFDA, and WPS), while the DTC supports
HWRF, and NOAA supports WRF-Chem. NCAR
is responsible for the oversight of WRF releases and management of the WRF repository,
including the addition of new code. To
guide those activities, “WRF Code Repository and Release Administration” (http://www2.mmm.ucar.edu/wrf/users/code_admin.html)
describes the policies and procedures.
WRF Code Contribution—
WRF Developers’ Committee (DC) manages the WRF repository, while the WRF
Release Committee (RC) manages WRF major releases. These committees are described in detail in “WRF
Code Repository and Release Administration”.
Developers’ Committee oversees the management and maintenance of the repository
of WRF system software. The DC keeps the
WRF code in order and in a state of readiness through testing and through review
of proposed modifications to the code trunk.
As described below, code proposed to be contributed must first be tested
by its developer. Code changes that
modify scientific results must be verified by the developer and be available
DC has responsibility for the timely consideration and processing of code for
addition to the repository. It meets on a
regular basis to consider proposals for code commits. Such proposals are submitted to the DC prior
to its meetings and contain required information on the code.
of Prospective Contributors
preparing their code and integrating it into the WRF repository, prospective
contributors will be assigned a DC liaison, and they should contact the wrfhelp user support service (email@example.com) for this. Prior to code submission, however,
prospective contributors must perform the testing and meet the requirements
the contribution of new physics packages, there is a special process and a review
by the WRF Physics Review Panel (WPRP) (detailed in "The WRF Physics
Review Process" at: http://www2.mmm.ucar.edu/wrf/users/physics_review.php). The WPRP oversees
the review of proposed new physics packages, communicating with the prospective
contributors and having relevant experts from the community review the
submissions anonymously. After receiving
the reviewers' input, the panel makes a recommendation to the WRF Developers'
Committee (DC) on whether the package should be considered for commit. If the DC confirms the panel's recommendation
of addition, the submission must then meet the regular testing and other
prerequisites for a code commit. Note
that the panel is not engaged for submissions other than new physics packages,
such as (i) bugfixes or non-fundamental modifications to existing physics
schemes or (ii) developments in WRF dynamics, numerics, or software framework.
is a proposing developer’s responsibility to perform all required testing and
provide all required documentation prior to proposing a commit or to submitting
code. The proposing developer must make reasonable
efforts to ensure that the proposed change is correct and that its impact on
other parts of the model, if any, are described in adequate detail to the DC
and to any other developers who might be affected. Any impacted developer may request a code
review and may work with the originating developer to run tests to verify that
the proposed change does no harm. Disagreements
about changes to scientific or forecast results must be resolved before code or
code changes can be committed. Developers
have effective veto power over changes that affect aspects of the WRF system
for which they have primary responsibility.
each Developers' Committee meeting, the DC liaison will have already run the
regression tests on his/her working WRF copy, using code updated to the top of
the repository trunk. During the
meeting, proposals will be reviewed, and those that are approved will be placed
in the commit queue.
WRF Code Contribution—
for physics schemes submitted for consideration in WRF should adhere to the
following rules and coding practices.
– Use a single Fortran file, preferably as a single module; multiple
modules, however, are
– Use standard Fortran free format (“!” for comments, “&”
at the end for continuation lines).
– Indent code, and do not exceed line lengths of 132
– If modifying an existing module, follow the stylistic
coding conventions already in place.
– In the top level physics routine of the new module: (i)
list each variable and (ii) provide a brief
description of it, with its units shown.
– Include references from publications where possible.
– Place comments around all non-trivial blocks of code.
– Use IMPLICIT NONE and declare all variables.
– Use INTENT IN/OUT/INOUT, at least at the top level. While these statements are not always
heeded by compilers, they provide
valuable information for other researchers.
– Do not include any PRINT statements or diagnostic WRITE
statements: instead, use “call
wrf_message”. Debugging statements that are contained
within special IF tests are acceptable, however.
– Do not include READ statements after model initialization. However, input of “look-up table”
values on a single processor that
is then broadcast is acceptable.
Initializations such as
these should not be handled when
the scheme is called from the solve
– Do not use STOP or PAUSE statements: instead, use “call
– It is preferred that no EQUIVALENCE statements be used, although
there are large existing
packages which use them extensively. There are issues with optimization and
that the user has to consider when
using EQUIVALENCE statements.
– Do not use COMMON blocks that are outside of a
self-contained module: that is, no COMMON
blocks should appear above the
CONTAINS statement). It is recommended that
COMMON blocks be used within a
– Do not use MPI calls or do communication within the
physics layer. Do not introduce OpenMP
directives in physics schemes.
– Use the top of a module for module-wide parameters (e.g., for
look-up table values and flags set
– No physics routines should do computations that involve
horizontal differencing, as the
assumption for all physics modules
is that the computations are in the vertical direction only. If horizontal differencing is required in a
routine, please contact wrfhelp so
that assistance may be provided and the DC can be made aware of the code.
– When adding new state variables to an existing physics
package, or when adding a new physics
package, use the “package” option
in the Registry files.
– Any known restrictions to physics package should be
expressly coded into the namelist tests
such as grid distance assumptions, mandatory microphysics variables, and model
– Write code modularly where possible. Repeated loops over different variables
(e.g., u, v, T, etc.)
should be abstracted into a
subroutine with repeated calls.
– Pass variable arrays, flags, and physical constants via
argument lists, not through USE
– Include an initialization routine to do
This routine should be
called from the physics
initialization during the model start, not from the solver and not
based on a check of “Is this the
first time step?” That sort of logic
introduces OpenMP race conditions.
– The init routine is to be called from the relevant routine
of physics_init (CASE SELECT).
– The main run-time routine is to be called from the relevant
driver (CASE SELECT).
– Use the physical constants (e.g., gravity, etc.) that are already
defined in the model, if possible.
Note that share/module_model_constants.F
is where to define new, required constants or to obtain existing constants.
– Use the correct indexing inside of the physics routines:
loops should always, and only, run
through the tile index extents, and
2-D/3-D arrays defined as state variables in the Registry
should always, and only, be memory-dimensioned. Local arrays should be tile-dimensioned.
– Distinguish indices explicitly when passing values into
the top-level physics scheme.
dimension— its, ite, etc: for loop limits and local array dimensions
dimension— ims, ime, etc: for dimensions of WRF (registry) arrays
dimension— ids, ide, etc: only for domain-boundary tests
WRF Code Contribution—
offered to the WRF repository must be tested by the submitting developer. The areas of testing required are: (a) software
testing and (b) case testing. These
areas are summarized below and are detailed on the WRF code testing information
page at: http://www2.wrf-model.org/users/testing.php. Materials that can be used for this testing
are also provided through that page. For
the developer, the DC points of contact can provide information and advice on
(a) Software Testing
Code or modifications can be written for either the ARW or
NMM core. If a developer only seeks to
add something for one of the cores (e.g., the ARW), then it is not necessary to
make it available to the other core.
However, the testing must confirm that compilation and operation with
the other core are not harmed. Developers
choosing only to support their code in a single core must still test that the
WRF code will build and run successfully when the other core is chosen. Known restrictions, such as suitability for a
particular dynamical core, should be coded into the WRF routine that checks the
consistency of the namelist run-time settings (viz., check_a_mundo).
The testing by developers must be conducted on a machine,
OS, and compiler that are reasonable (i.e., not exotic or irrelevant in the
opinion of the DC). New schemes must
provide identical results for unoptimized code for the serial and shared-memory
and distributed-memory tests. The
results of the tests must be provided to the DC for its review. The namelist.input files used for the
software testing must also be provided.
The DC may deem these satisfactory in lieu of case testing, but the DC
must communicate that to the contributor.
Before code is accepted, developers must run their codes
through the WRF Test Framework (WTF). This
is to ensure that the new code compiles, runs, and generates bit-for-bit
comparison results. The WTF has basic
tests, as well as tests of advanced features in WRF. These are short tests that do not have large
memory or time requirements. The areas
to be checked through use of the WTF are: (i) compilation, (ii) parallel
operation, (iii) nesting, (iv) operation with other physics, and (v) restarts. Information on the WTF may be found at: http://www2.mmm.ucar.edu/wrf/users/wrfv3.8/Test3.html
The WTF has configurations for WRF that allow a developer to
activate a number of compiler switches: uninitialized stack variables, floating
point exceptions, uninitialized pointers, bounds checking, etc. No new warnings are to be introduced (in the
physics module) from the new code. Note
that the default value of the TRAP_ERRORS option in WTF is set to “false”, which
avoids a higher level of code checking. Users
setting this to “true” may see reported errors in the standard WRF code even
though the standard code yields bit-for-bit reproducibility.
If an option is introduced in the code, there should be no
changes to the behavior of WRF when the option is not activated. This is to ensure that the option can be cleanly
bypassed with no effect on the existing WRF code.
When submitting the code, the amount of memory and CPU time used
are important concerns. The WRF code
identifies the amount of memory allocated in a line in the standard
output. There should be no increase in
memory with the inclusion of the new code when the run-time option is not selected. For comparative purposes, the developer/contributor
must supply the cost of the new or modified scheme, relative to several other
similar schemes. For example, when
providing a new cumulus scheme, the contributor should provide the elapsed
time-steps for the new scheme and the other cumulus schemes.
(b) Case Testing
simulations are to be done to assess the code's robustness and
reasonableness. These do not need to involve
large domains or long forecasts, however, they should employ parallel
processing. For those without data and
setups already, information and data for the case testing is provided under
this page: http://www2.mmm.ucar.edu/wrf/users/wrfv3.8/Test1.html . Contributors may use their own test cases, as
long as they meet the requirements below.
testing requires at least one 24-hour case simulation in each of two seasons,
summer and winter (i.e., a minimum of two, 24-hr cases). To pass these tests, the following are
required: (a) the simulations must run to completion; (b) the output must be
checked for both a lack of NaN's ("not a number" designations) and a
lack of fatal CFL violations (i.e., CFL
warnings/violations that terminate a run); and (c) the forecast output must be checked
to ensure that the results are not unphysical.
the case materials provided by NCAR, some standard option output and plots of
certain forecast fields has been provided for developers to compare their
output to. The goals are to see general
consistency of results from the modified WRF with the known results and to see
if there are any gross discrepancies from a trusted simulation. NCL scripts are provided for the tester to
make some basic plots, which can be compared to the provided plots.
the time of code proposal, contributors must provide information on the commit
to the DC. The DC has a form for this
that calls for information on the type of modification, modification keywords,
source and purpose of the modification, description of changes, and list of
modified files. The form includes the following.
Information on Proposed Commit
TYPE (release, bug fix, enhancement, new
feature, new option, or new scheme):
KEYWORDS (5 to 10 words related to commit,
white space/commas acceptable):
SOURCE (developer's name &
PURPOSE (short, usually one sentence):
DESCRIPTION OF CHANGES
(paragraph describing problem, solution,
and required changes):
LIST OF MODIFIED FILES (annotated, if not
TESTS CONDUCTED (state the mandatory,
voluntary, and assigned tests performed):
of the testing described below is required.
The DC decides whether to proceed with the commit process based on
code has been accepted, contributors must provide documentation or, if
available, reference papers for their code or feature contributions. The documentation may also be in the form of
a web page or adequate (as deemed by the DC) README files or inline documentation. The documentation is to allow users to
understand the code better and to allow those supporting WRF to the user
community to understand the code's structure, function, and behavior. The documentation may be provided up to the
targeted release date.
WRF Code Contribution—
processes for code contributions are described here in terms of the annual cycle
of WRF major releases. “WRF Code
Repository and Release Administration” provides information on release
procedures and timetables, as well as on the responsibilities of contributors.
noted above, for contributors seeking to submit new physics packages to WRF, the
document "The WRF Physics Review Panel and Process" (http://www2.mmm.ucar.edu/wrf/users/physics_review.php)
describes the specific procedures required, and prospective contributors of new
physics should become familiar with them.
The process for new physics review has two stages: (1) letter of intent
and (2) submission and review of materials.
A letter of intent must first be sent to the WRF Physics Review Panel. If the panel directs the developer to proceed
with the submission, the developer then prepares and submits review materials. These consist of testing data and
documentation, and the panel will discuss with the prospective contributor the
specific case testing required.
should coordinate with Developers’ Committee members to make sure their code
intentions for future releases are known.
It is emphasized that there is no guarantee that any code, whenever it
is submitted, will get into a given release, although usually the DC can gauge
whether developments will be ready for inclusion in the next release.
developments, existing code improvements, and contributions of bugfixes may be
made to the trunk on an ongoing basis. As
noted above, however, there is no guarantee that new code will get into a
particular release, regardless of when submitted.
items are developed either by DC members or by contributors who provide them to
a DC member. In either case, it is up to
such DC members to confirm that the provided code works with the top of the
repository, has gone through the required testing, and passes the basic regression
tests. Code contributors are responsible
the required code testing;
that the code has been submitted with the required commit information and that
the code conforms to the WRF coding standards;
the Developers’ Committee about any limiting underlying assumptions or possible
with Developers’ Committee members to develop the necessary tests to verify
that future modifications do not adversely impact their code and incorporating
these tests into the WRF regression test suite; and
proposed modification to the repository is circulated among the DC via email,
with a description in the commit form described above. The proposing DC member classifies the
modification (e.g., bug fix, enhancement, new feature), provides a motivation
for the modification, describes the changes to the code required, and lists the
touched files. If the proposed change
modifies scientific results, the proposer must show how results are changed,
and additional testing may be required to obtain this information. Other members of the DC may ask for a hold to
be placed on some or all of a set of proposed commits to allow for further
of contributions is coordinated with the regular meetings of the DC. After each meeting’s accumulated
modifications have been committed to the repository, a large regression test is
conducted on the primary supercomputers at NCAR. The purpose of the large regression test is to
identify software errors (e.g., bit-for-bit differences) and failures to
compile or run. These tests do not
attempt to detect or analyze impacts on forecast skill.
the large regression test has been conducted, any unexpected failures are investigated
to identify which of the previous commits introduced a fail condition in the
testing package. The contributor is
informed of the testing status. If a fix
is easily integrated into the code, it is, and the fixed code is re-tested. If no simple fix is available, or if the
supplied fix cannot return the testing suite to the baseline passing status,
then the modification is backed out of the repository. A final large regression test is always
required on the current top of the repository to validate the code.
begins approximately 4‒5 mos. before a projected release. Prospective contributors of code for the next
release who are not on the DC are assigned a DC member to coordinate testing of
their components (if they do not already have a DC liaison), and they should work
with their DC liaisons to get the source code committed to the repository before
the branch cutoff date. The contributors
are responsible for following all contribution and coding policies, testing
requirements, and DC requests.
is a generic major release schedule showing the basic timeframe. The Release Committee determines the key
dates for each major release, and they are posted on a release information web page.
(i) Release minus 6
Release Committee meets for the first time for the new cycle. It determines the major release timetable and
compiles an initial list of candidate features.
It provides information to wrfhelp
to be communicated to the WRF user community on the upcoming release, including
dates and information for code contributors.
Wrfhelp notifies users of the
should use the latest released version of the code as the basis for their
development and submitted code. They may also use the latest version of the WRF
repository code, which they may obtain through their POC. Developers' new code must conform to the
standards described above (“WRF Code Contribution— Coding Standards”). In this period or at any time through the
year, presentation of code to be committed does not guarantee its inclusion in
any given release.
addition, as described above, developers must provide the proposed commit
information to the DC and must put their offered codes through software and
case testing. This testing is described above
and in: http://wrf-model.org/users/testing.php.
(iii) Release minus 3–4 months (R–3–4m)
this period all new code and features sought for the release branch should have
been submitted. The release branch is
cut off and frozen, except for those changes that are necessary for the release’s
development (e.g., bugfixes). Limited modifications,
such as for bugs found in the new features or for difficulties with feature interaction,
are permitted within this period. Large
code changes unrelated to the main elements or goals of the release may be
proposed for commit to the trunk, however, which remains open for additions.
the 3-month period a first friendly-user release of WRF is typically made
available to contributors. This is a
current version of the tentative release code that permits testing of the
system primarily by developers, after their contributions have been integrated,
to allow them to check for correctness of the code and to verify its
(iv) Release minus 2 months (R–2m)
release branch code remains frozen except to modifications that are related to bugfixes
and to deficient features slated for the release. Within this period a second friendly-user
release may be made available to contributors and other interested users. This is an updated version of the branch code
that permits testing by a broader group of users who can provide feedback prior
to the main release.
(v) Release minus 1 month (R–1m)
only acceptable changes to the branch are those pertaining to testing of the
frozen code. Modifications or bugfixes related
to testing issues and results, but only with approval of the DC. Documentation updates are allowed into the
WRF repository is the store of code constituting the WRF modeling system and
software infrastructure (code and meta-code, build scripts, testing mechanisms
and datasets, documentation, etc.), maintained under a software management
system. The repository is managed and
maintained by the WRF Developers’ Committee such that it always contains the
most current, working, and theoretically-releasable revision of the WRF model,
plus a fully-recoverable history of past revisions and developer notations.
directory structure contains the trunk, which is the repository itself; tags,
which are a series of development snapshots of the trunk; plus a number of
branches. At some point before a planned
major release date a branch is created for the code to be released. From the release branch cutoff date, the
branch will only accept bugfixes and corrections for the code to be released.
to the repository is by agreement of the Developers’ Committee. Write access to the repository is limited to
the members of the DC. Read-only access
to the repository may be available upon request and DC approval, with the
understanding that repository versions of the code are not releases and are not
supported. Any unsanctioned use of the
repository code (such as accessing another scientist's branch, releasing new
versions for which a user is not the author, etc.), without the express,
written consent of the DC is prohibited and could result in access to the repository
being revoked by the DC.