General Info

About wrfhelp

wrf-news list

wrf-users list

Becoming a Registered User

WRF Workshop

WRF Tutorial

 

 

 

Information for WRF Code Contributors

Info for Contributors

Information for WRF Code Contributors

 

WRF is a community modeling system, reflecting a wide range of contributions from scientists and software engineers.  It is through these contributions that the system improves and advances.  To facilitate the process, this document describes the procedures and standards for adding code.  Prospective contributors should familiarize themselves with this information. 

 

Regarding administration of WRF-related code, NCAR is primarily responsible for the support of the main WRF system (WRF, WRFDA, and WPS), while the DTC supports HWRF, and NOAA supports WRF-Chem.  NCAR is responsible for the oversight of WRF releases and management of the WRF repository, including the addition of new code.  To guide those activities, “WRF Code Repository and Release Administration” (http://www2.mmm.ucar.edu/wrf/users/code_admin.html) describes the policies and procedures.

 

 

WRF Code Contribution— Background Information

 

Developers’ Committee Oversight

 

The WRF Developers’ Committee (DC) manages the WRF repository, while the WRF Release Committee (RC) manages WRF major releases.  These committees are described in detail in “WRF Code Repository and Release Administration”.

 

The Developers’ Committee oversees the management and maintenance of the repository of WRF system software.  The DC keeps the WRF code in order and in a state of readiness through testing and through review of proposed modifications to the code trunk.  As described below, code proposed to be contributed must first be tested by its developer.  Code changes that modify scientific results must be verified by the developer and be available for review.  

 

The DC has responsibility for the timely consideration and processing of code for addition to the repository.  It meets on a regular basis to consider proposals for code commits.  Such proposals are submitted to the DC prior to its meetings and contain required information on the code.

 

 

Responsibilities of Prospective Contributors  

 

For preparing their code and integrating it into the WRF repository, prospective contributors will be assigned a DC liaison, and they should contact the wrfhelp user support service (wrfhelp@ucar.edu) for this.  Prior to code submission, however, prospective contributors must perform the testing and meet the requirements described below.

 

For the contribution of new physics packages, there is a special process and a review by the WRF Physics Review Panel (WPRP) (detailed in "The WRF Physics Review Process" at: http://www2.mmm.ucar.edu/wrf/users/physics_review.php). The WPRP oversees the review of proposed new physics packages, communicating with the prospective contributors and having relevant experts from the community review the submissions anonymously.  After receiving the reviewers' input, the panel makes a recommendation to the WRF Developers' Committee (DC) on whether the package should be considered for commit.  If the DC confirms the panel's recommendation of addition, the submission must then meet the regular testing and other prerequisites for a code commit.  Note that the panel is not engaged for submissions other than new physics packages, such as (i) bugfixes or non-fundamental modifications to existing physics schemes or (ii) developments in WRF dynamics, numerics, or software framework.

 

It is a proposing developer’s responsibility to perform all required testing and provide all required documentation prior to proposing a commit or to submitting code.  The proposing developer must make reasonable efforts to ensure that the proposed change is correct and that its impact on other parts of the model, if any, are described in adequate detail to the DC and to any other developers who might be affected.  Any impacted developer may request a code review and may work with the originating developer to run tests to verify that the proposed change does no harm.  Disagreements about changes to scientific or forecast results must be resolved before code or code changes can be committed.  Developers have effective veto power over changes that affect aspects of the WRF system for which they have primary responsibility.

 

Before each Developers' Committee meeting, the DC liaison will have already run the regression tests on his/her working WRF copy, using code updated to the top of the repository trunk.  During the meeting, proposals will be reviewed, and those that are approved will be placed in the commit queue.

 

 

WRF Code Contribution— Coding Standards

 

Code for physics schemes submitted for consideration in WRF should adhere to the following rules and coding practices.

 

 

– Use a single Fortran file, preferably as a single module; multiple modules, however, are

acceptable.

 

– Use standard Fortran free format (“!” for comments, “&” at the end for continuation lines).

 

– Indent code, and do not exceed line lengths of 132 characters.

 

– If modifying an existing module, follow the stylistic coding conventions already in place.

 

– In the top level physics routine of the new module: (i) list each variable and (ii) provide a brief

description of it, with its units shown.

 

– Include references from publications where possible.

 

– Place comments around all non-trivial blocks of code.

 

– Use IMPLICIT NONE and declare all variables.

 

– Use INTENT IN/OUT/INOUT, at least at the top level.  While these statements are not always

heeded by compilers, they provide valuable information for other researchers.

 

– Do not include any PRINT statements or diagnostic WRITE statements: instead, use “call

wrf_message”.  Debugging statements that are contained within special IF tests are acceptable, however.

 

– Do not include READ statements after model initialization.  However, input of “look-up table”

values on a single processor that is then broadcast is acceptable.  Initializations such as

these should not be handled when the scheme is called from the solve subroutine.

 

– Do not use STOP or PAUSE statements: instead, use “call wrf_error_fatal”.

 

– It is preferred that no EQUIVALENCE statements be used, although there are large existing

packages which use them extensively.  There are issues with optimization and parallelism

that the user has to consider when using EQUIVALENCE statements.

 

– Do not use COMMON blocks that are outside of a self-contained module: that is, no COMMON

blocks should appear above the CONTAINS statement).  It is recommended that no

COMMON blocks be used within a module.

 

– Do not use MPI calls or do communication within the physics layer.  Do not introduce OpenMP

directives in physics schemes.

 

– Use the top of a module for module-wide parameters (e.g., for look-up table values and flags set

at initialization).

 

– No physics routines should do computations that involve horizontal differencing, as the

assumption for all physics modules is that the computations are in the vertical direction only.  If horizontal differencing is required in a routine, please contact wrfhelp so that assistance may be provided and the DC can be made aware of the code.

 

– When adding new state variables to an existing physics package, or when adding a new physics

package, use the “package” option in the Registry files.

 

– Any known restrictions to physics package should be expressly coded into the namelist tests

(share/module_check_a_mundo.F), such as grid distance assumptions, mandatory microphysics variables, and model top.

 

– Write code modularly where possible.  Repeated loops over different variables (e.g., u, v, T, etc.)

should be abstracted into a subroutine with repeated calls.

 

– Pass variable arrays, flags, and physical constants via argument lists, not through USE

association.

 

– Include an initialization routine to do start-of-simulation initializations.  This routine should be

called from the physics initialization during the model start, not from the solver and not

based on a check of “Is this the first time step?”  That sort of logic introduces OpenMP race conditions.

 

– The init routine is to be called from the relevant routine of physics_init (CASE SELECT).

 

– The main run-time routine is to be called from the relevant driver (CASE SELECT).

 

– Use the physical constants (e.g., gravity, etc.) that are already defined in the model, if possible. 

Note that share/module_model_constants.F is where to define new, required constants or to obtain existing constants.

 

– Use the correct indexing inside of the physics routines: loops should always, and only, run

through the tile index extents, and 2-D/3-D arrays defined as state variables in the Registry

should always, and only, be memory-dimensioned.  Local arrays should be tile-dimensioned.

 

– Distinguish indices explicitly when passing values into the top-level physics scheme.

     (i) Tile dimension— its, ite, etc: for loop limits and local array dimensions

     (ii) Memory dimension— ims, ime, etc: for dimensions of WRF (registry) arrays

     (iii) Domain dimension— ids, ide, etc: only for domain-boundary tests

 

 

WRF Code Contribution— Code Testing

 

Code offered to the WRF repository must be tested by the submitting developer.  The areas of testing required are: (a) software testing and (b) case testing.  These areas are summarized below and are detailed on the WRF code testing information page at: http://www2.wrf-model.org/users/testing.php.  Materials that can be used for this testing are also provided through that page.  For the developer, the DC points of contact can provide information and advice on the process.

 

(a) Software Testing

 

Code or modifications can be written for either the ARW or NMM core.  If a developer only seeks to add something for one of the cores (e.g., the ARW), then it is not necessary to make it available to the other core.  However, the testing must confirm that compilation and operation with the other core are not harmed.  Developers choosing only to support their code in a single core must still test that the WRF code will build and run successfully when the other core is chosen.  Known restrictions, such as suitability for a particular dynamical core, should be coded into the WRF routine that checks the consistency of the namelist run-time settings (viz., check_a_mundo).

 

The testing by developers must be conducted on a machine, OS, and compiler that are reasonable (i.e., not exotic or irrelevant in the opinion of the DC).  New schemes must provide identical results for unoptimized code for the serial and shared-memory and distributed-memory tests.  The results of the tests must be provided to the DC for its review.  The namelist.input files used for the software testing must also be provided.  The DC may deem these satisfactory in lieu of case testing, but the DC must communicate that to the contributor.

 

Before code is accepted, developers must run their codes through the WRF Test Framework (WTF).  This is to ensure that the new code compiles, runs, and generates bit-for-bit comparison results.  The WTF has basic tests, as well as tests of advanced features in WRF.  These are short tests that do not have large memory or time requirements.  The areas to be checked through use of the WTF are: (i) compilation, (ii) parallel operation, (iii) nesting, (iv) operation with other physics, and (v) restarts.  Information on the WTF may be found at: http://www2.mmm.ucar.edu/wrf/users/wrfv3.8/Test3.html .

 

The WTF has configurations for WRF that allow a developer to activate a number of compiler switches: uninitialized stack variables, floating point exceptions, uninitialized pointers, bounds checking, etc.  No new warnings are to be introduced (in the physics module) from the new code.  Note that the default value of the TRAP_ERRORS option in WTF is set to “false”, which avoids a higher level of code checking.  Users setting this to “true” may see reported errors in the standard WRF code even though the standard code yields bit-for-bit reproducibility.

 

If an option is introduced in the code, there should be no changes to the behavior of WRF when the option is not activated.  This is to ensure that the option can be cleanly bypassed with no effect on the existing WRF code.

 

When submitting the code, the amount of memory and CPU time used are important concerns.  The WRF code identifies the amount of memory allocated in a line in the standard output.  There should be no increase in memory with the inclusion of the new code when the run-time option is not selected.  For comparative purposes, the developer/contributor must supply the cost of the new or modified scheme, relative to several other similar schemes.  For example, when providing a new cumulus scheme, the contributor should provide the elapsed time-steps for the new scheme and the other cumulus schemes.

 

(b) Case Testing

 

Case simulations are to be done to assess the code's robustness and reasonableness.  These do not need to involve large domains or long forecasts, however, they should employ parallel processing.  For those without data and setups already, information and data for the case testing is provided under this page: http://www2.mmm.ucar.edu/wrf/users/wrfv3.8/Test1.html .  Contributors may use their own test cases, as long as they meet the requirements below.

 

This testing requires at least one 24-hour case simulation in each of two seasons, summer and winter (i.e., a minimum of two, 24-hr cases).  To pass these tests, the following are required: (a) the simulations must run to completion; (b) the output must be checked for both a lack of NaN's ("not a number" designations) and a lack of fatal CFL violations (i.e., CFL warnings/violations that terminate a run); and (c) the forecast output must be checked to ensure that the results are not unphysical. 

 

For the case materials provided by NCAR, some standard option output and plots of certain forecast fields has been provided for developers to compare their output to.  The goals are to see general consistency of results from the modified WRF with the known results and to see if there are any gross discrepancies from a trusted simulation.  NCL scripts are provided for the tester to make some basic plots, which can be compared to the provided plots.

 

 

WRF Code Contribution— Documentation

 

At the time of code proposal, contributors must provide information on the commit to the DC.  The DC has a form for this that calls for information on the type of modification, modification keywords, source and purpose of the modification, description of changes, and list of modified files.  The form includes the following.

 

Information on Proposed Commit

 

TYPE (release, bug fix, enhancement, new feature, new option, or new scheme):

 

KEYWORDS (5 to 10 words related to commit, white space/commas acceptable):

 

SOURCE (developer's name & affiliation):

 

PURPOSE (short, usually one sentence):

 

DESCRIPTION OF CHANGES

(paragraph describing problem, solution, and required changes):

 

LIST OF MODIFIED FILES (annotated, if not obvious):

 

TESTS CONDUCTED (state the mandatory, voluntary, and assigned tests performed):

 

Evidence of the testing described below is required.  The DC decides whether to proceed with the commit process based on this. 

 

After code has been accepted, contributors must provide documentation or, if available, reference papers for their code or feature contributions.  The documentation may also be in the form of a web page or adequate (as deemed by the DC) README files or inline documentation.  The documentation is to allow users to understand the code better and to allow those supporting WRF to the user community to understand the code's structure, function, and behavior.  The documentation may be provided up to the targeted release date. 

 

 

WRF Code Contribution— Procedures

 

The processes for code contributions are described here in terms of the annual cycle of WRF major releases.  “WRF Code Repository and Release Administration” provides information on release procedures and timetables, as well as on the responsibilities of contributors.  

 

As noted above, for contributors seeking to submit new physics packages to WRF, the document "The WRF Physics Review Panel and Process" (http://www2.mmm.ucar.edu/wrf/users/physics_review.php) describes the specific procedures required, and prospective contributors of new physics should become familiar with them.  The process for new physics review has two stages: (1) letter of intent and (2) submission and review of materials.  A letter of intent must first be sent to the WRF Physics Review Panel.  If the panel directs the developer to proceed with the submission, the developer then prepares and submits review materials.  These consist of testing data and documentation, and the panel will discuss with the prospective contributor the specific case testing required.

 

Contributors should coordinate with Developers’ Committee members to make sure their code intentions for future releases are known.  It is emphasized that there is no guarantee that any code, whenever it is submitted, will get into a given release, although usually the DC can gauge whether developments will be ready for inclusion in the next release. 

 

Year-Round

 

New developments, existing code improvements, and contributions of bugfixes may be made to the trunk on an ongoing basis.  As noted above, however, there is no guarantee that new code will get into a particular release, regardless of when submitted. 

 

New items are developed either by DC members or by contributors who provide them to a DC member.  In either case, it is up to such DC members to confirm that the provided code works with the top of the repository, has gone through the required testing, and passes the basic regression tests.  Code contributors are responsible for:

 

(1)   Performing the required code testing;

(2)   Ensuring that the code has been submitted with the required commit information and that the code conforms to the WRF coding standards;

(3)   Warning the Developers’ Committee about any limiting underlying assumptions or possible code conflicts;

(4)   Working with Developers’ Committee members to develop the necessary tests to verify that future modifications do not adversely impact their code and incorporating these tests into the WRF regression test suite; and

(5)   Documentation.

 

A proposed modification to the repository is circulated among the DC via email, with a description in the commit form described above.  The proposing DC member classifies the modification (e.g., bug fix, enhancement, new feature), provides a motivation for the modification, describes the changes to the code required, and lists the touched files.  If the proposed change modifies scientific results, the proposer must show how results are changed, and additional testing may be required to obtain this information.  Other members of the DC may ask for a hold to be placed on some or all of a set of proposed commits to allow for further review.

 

Testing of contributions is coordinated with the regular meetings of the DC.  After each meeting’s accumulated modifications have been committed to the repository, a large regression test is conducted on the primary supercomputers at NCAR.  The purpose of the large regression test is to identify software errors (e.g., bit-for-bit differences) and failures to compile or run.  These tests do not attempt to detect or analyze impacts on forecast skill.

 

Once the large regression test has been conducted, any unexpected failures are investigated to identify which of the previous commits introduced a fail condition in the testing package.  The contributor is informed of the testing status.  If a fix is easily integrated into the code, it is, and the fixed code is re-tested.  If no simple fix is available, or if the supplied fix cannot return the testing suite to the baseline passing status, then the modification is backed out of the repository.  A final large regression test is always required on the current top of the repository to validate the code.

 

            Pre-Release Period

 

This begins approximately 4‒5 mos. before a projected release.  Prospective contributors of code for the next release who are not on the DC are assigned a DC member to coordinate testing of their components (if they do not already have a DC liaison), and they should work with their DC liaisons to get the source code committed to the repository before the branch cutoff date.  The contributors are responsible for following all contribution and coding policies, testing requirements, and DC requests. 

 

Below is a generic major release schedule showing the basic timeframe.  The Release Committee determines the key dates for each major release, and they are posted on a release information web page.

 

                        (i) Release minus 6 months (R–6m)

 

The Release Committee meets for the first time for the new cycle.  It determines the major release timetable and compiles an initial list of candidate features.  It provides information to wrfhelp to be communicated to the WRF user community on the upcoming release, including dates and information for code contributors.  Wrfhelp notifies users of the planned release.

 

Developers should use the latest released version of the code as the basis for their development and submitted code. They may also use the latest version of the WRF repository code, which they may obtain through their POC.  Developers' new code must conform to the standards described above (“WRF Code Contribution— Coding Standards”).  In this period or at any time through the year, presentation of code to be committed does not guarantee its inclusion in any given release.

 

In addition, as described above, developers must provide the proposed commit information to the DC and must put their offered codes through software and case testing.  This testing is described above and in: http://wrf-model.org/users/testing.php.     

 

(iii) Release minus 3–4 months (R–3–4m)

 

In this period all new code and features sought for the release branch should have been submitted.  The release branch is cut off and frozen, except for those changes that are necessary for the release’s development (e.g., bugfixes).  Limited modifications, such as for bugs found in the new features or for difficulties with feature interaction, are permitted within this period.  Large code changes unrelated to the main elements or goals of the release may be proposed for commit to the trunk, however, which remains open for additions.

 

Within the 3-month period a first friendly-user release of WRF is typically made available to contributors.  This is a current version of the tentative release code that permits testing of the system primarily by developers, after their contributions have been integrated, to allow them to check for correctness of the code and to verify its results.  

 

(iv) Release minus 2 months (R–2m)

 

The release branch code remains frozen except to modifications that are related to bugfixes and to deficient features slated for the release.  Within this period a second friendly-user release may be made available to contributors and other interested users.  This is an updated version of the branch code that permits testing by a broader group of users who can provide feedback prior to the main release.

 

(v) Release minus 1 month (R–1m)

 

The only acceptable changes to the branch are those pertaining to testing of the frozen code.  Modifications or bugfixes related to testing issues and results, but only with approval of the DC.  Documentation updates are allowed into the release branch.

 

 

Repository Information

 

The WRF repository is the store of code constituting the WRF modeling system and software infrastructure (code and meta-code, build scripts, testing mechanisms and datasets, documentation, etc.), maintained under a software management system.  The repository is managed and maintained by the WRF Developers’ Committee such that it always contains the most current, working, and theoretically-releasable revision of the WRF model, plus a fully-recoverable history of past revisions and developer notations.

 

The directory structure contains the trunk, which is the repository itself; tags, which are a series of development snapshots of the trunk; plus a number of branches.  At some point before a planned major release date a branch is created for the code to be released.  From the release branch cutoff date, the branch will only accept bugfixes and corrections for the code to be released.

 

Access to the repository is by agreement of the Developers’ Committee.  Write access to the repository is limited to the members of the DC.  Read-only access to the repository may be available upon request and DC approval, with the understanding that repository versions of the code are not releases and are not supported.  Any unsanctioned use of the repository code (such as accessing another scientist's branch, releasing new versions for which a user is not the author, etc.), without the express, written consent of the DC is prohibited and could result in access to the repository being revoked by the DC.

 

 

11/16

 

 

 

 



 
Home -- Model System -- User Support -- Doc / Pub -- Links -- Download -- WRF Real-time Forecast