General Info

About wrfhelp

wrf-news list

wrf-users list

Becoming a Registered User

WRF Workshop

WRF Tutorial

 

 

 

WRF Code Repository and Release Administration

Code Management Activity Levels

WRF Code Repository and Release Administration

 

This document describes the policies and procedures for maintaining the WRF code repository and for overseeing releases.  Those seeking details on submitting code to WRF may consult the document “Information for Code Contributors” on the policies and procedures for contributing to WRF system.  The oversight, maintenance, and support of the WRF code is handled primarily by NCAR, but the DTC oversees and support HWRF and NOAA oversees and supports WRF-Chem.

 

 

A. Committees

 

            1) Overview

 

WRF repository and release management is handled by two committees: the Developers’ Committee and the Release Committee.  The Developers’ Committee oversees additions to, and maintenance of, the repository, while the Release Committee oversees the new releases to the user community.

 

The Developers’ Committee has responsibility for maintaining the WRF system software by implementing procedures for, and overseeing the process of, code contributions, code testing, and repository upkeep.  It determines whether code modifications are acceptable for inclusion, and it manages the repository.  The Release Committee oversees the preparation and issuance of WRF major releases.  It also provides release information to the user community.

 

 

            2) Structures and Functions

 

                        a) Developers’ Committee (DC)

 

The Developers’ Committee keeps the WRF system code in order through testing and review of proposed modifications and contributions to the repository.  The Developers’ Committee has responsibility for the timely testing and inclusion of code into the repository.  Contributors of code are required to perform testing and provide information to the DC on their proposed code commits.  All changes are checked via an automated set of tests on code modifications and additions that are run independently of the contributor.  In addition, changes that modify scientific results must be verified by the contributor and be available for review.

 

Members of the Developers’ Committee are those active in developing or maintaining the WRF system.  Committee membership requirements have been informal, requiring that: one participate in the regular meetings; one have some code either already in, or proposed to go in, WRF; and that one agrees to abide by the rules for making commits (changes) to the WRF repository.  In addition, membership on the DC carries the responsibility of continually shepherding new code into the repository by being a liaison for contributors.

 

                        b) Release Committee (RC)

 

The Release Committee oversees the scheduling, preparation, and issuance of updated WRF system code in the form of major releases.  It compiles a prospective list (the “release picture”) of additions or modifications (“candidate features”) offered by developers as contributions to a release.  Committee members serve as the points of contact with specific developers offering code for the release.  The committee monitors testing, progress, and issues with candidate features.  As the release cycle proceeds, the committee assesses which features can meet the release schedule given the status of their implementation and testing, as well as their expected progress.

 

The Release Committee sets the specific release schedule, including the timetable for required notifications, code submissions, and testing.  The Release Committee provides information on releases to the WRF user community though the WRF model web page and at the annual WRF workshop.

 

The Release Committee has members from various groups involved in WRF code development and user support.  The committee members serve as points of contact for the major areas of the WRF system, such as software, physics, data assimilation, and chemistry.  Some of its members sit on the Developers’ Committee and ensure coordination with that committee.  The Release Committee is chaired by a scientist in NCAR’s Mesoscale and Microscale Meteorology (MMM) Laboratory (appointed by the MMM director), as MMM is responsible for support of the primary WRF system to the community.  Committee decisions are made by consensus, with the chairperson arbitrating decisions if necessary.

 

 

B. Repository and Release Management

 

            1) Repository

 

a) Definition, structure, and access

 

The WRF repository is the store of code constituting the WRF modeling system and software infrastructure (code and meta-code, build scripts, testing mechanisms and datasets, documentation, etc.), maintained under a software management system.  The repository is managed and maintained by the WRF Developers’ Committee such that it always contains the most current, working, and theoretically-releasable revision of the WRF model, plus a fully-recoverable history of past revisions and developer notations.

 

The directory structure contains the trunk, which is the repository itself; tags, which are a series of development snapshots of the trunk; plus a number of branches.  At some point before a planned major release date a branch is created for the code to be released.  From the release branch cutoff date, the branch will only accept bugfixes and corrections for the code to be released.

 

Access to the repository is by agreement of the Developers’ Committee.  Write access to the repository is limited to the members of the DC.  Read-only access to the repository may be available upon request and DC approval, with the understanding that repository versions of the code are not releases and are not supported.  Any unsanctioned use of the repository code (such as accessing another scientist's branch, releasing new versions for which a user is not the author, etc.), without the express, written consent of the National Football League and the DC is prohibited and could result in access to the repository being revoked by the DC.

 

 

b) Responsibilities of the Developers’ Committee  

 

The Developers’ Committee oversees the management and maintenance of the WRF repository.

The purposes and principles of the Developers’ Committee are as follows.

 

·         Shepherd new development

The Developers’ Committee provides an interface to outside developers and facilitates incorporation of new or enhanced features and functionalities in WRF, subject to the constraints of appropriateness, of meeting coding standards, of impact on existing features and functionalities, and of available resources for implementation and testing.

 

·         Quality assurance and investment preservation

The Developers’ Committee ensures that the current revision of WRF— the top of the repository— is in order and in principle releasable, subject to the procedures described here.  New or enhanced functionality should have minimal impact on existing features and functionalities.

 

·         Process management

The Developers’ Committee is responsible for establishing and following processes for maintaining the WRF repository and for evaluating, incorporating, and assuring the quality of new features and functionalities.  The DC reviews proposed commits as well and the testing results and documentation required for the code submissions.

 

·         Release recommendation

The Developers’ Committee, after adequate testing has been performed, reports to the Release Committee to certify the identified release codeset as suitable for release.

 

·         WRF Physics Review Panel interaction

The Developers' Committee communicates with the WRF Physics Review Panel when the panel has reviewed a proposed new physics package.  The panel provides a recommendation to the DC on whether to accept the package, and the DC makes the ultimate decision on whether to allow the new code in.

 

The Developers’ Committee meets on a regular basis (weekly to biweekly) to consider proposals for additions to the WRF repository.  Such proposals are submitted in an agreed-upon time period prior to the meeting, and they are distributed to the committee members.

 

It is a proposing developer’s responsibility to perform all required testing and provide all required documentation prior to proposing a commit or to submitting code.  The proposing developer must make reasonable efforts to ensure that the proposed change is correct and that its impact on other parts of the model, if any, are described in adequate detail to the DC and to any other developers who might be affected.  Any impacted developer may request a code review and may work with the originating developer to run tests to verify that the proposed change does no harm.  Disagreements about changes to scientific or forecast results must be resolved before code or code changes can be committed.  Developers have effective veto power over changes that affect aspects of the WRF system for which they have primary responsibility.

 

Before each Developers' Committee meeting, the DC liaison will have already run the regression tests on his/her working WRF copy, using code updated to the top of the repository trunk.  During the meeting, proposals will be reviewed, and those that are approved will be placed in the commit queue.

 

 

2) Releases

 

a) Responsibilities of the Release Committee

 

The WRF Release Committee oversees the process of major releases.  This includes scheduling of major releases; review of information on submitted or desired features; review of the status of new code or features contributed or proposed; making final determinations on items for major releases; and coordination of release preparation, testing, and communication.  The Release Committee may determine that a major release is warranted based on the annual release cycle or on the needs of the user community.  In addition, the WRF Developers’ Committee may make a recommendation for a major release to the Release Committee, reflecting significant accumulated changes or major updates.

 

The Release Committee sets the major release timeline.  It also communicates release information to the WRF community, typically though announcements at the WRF Users’ Workshop and through a notification to the users six months prior to the tentative release date.  Information for contributors, on release procedures, and on the release timetable is posted under the general WRF model web page (http://www.wrf-model.org) and made available through the WRF users' page (http://www2.mmm.ucar.edu/wrf/users).

 

 

                        b) Types

 

(i) Major

 

Major releases are made on an approximately annual basis.  The period may vary slightly depending on the status and testing of code and of external factors that can affect the release preparation process (e.g., hardware and system issues).  Major releases generally reflect the addition of new and improved capabilities and, occasionally, significant structural changes warranting a new model version number.  With a major release, run-time options can change and default behavior can be modified.  For a major release, the Release Committee makes a plan and schedule and posts release information and a timetable on the web.

 

                                    (ii) Minor

 

Minor releases primarily address bug fixes.  Minor releases are determined and overseen by the director of wrfhelp (the WRF users' assistance service at NCAR) in coordination with the Developers' Committee, and their preparation and support are the responsibility of wrfhelp.  Minor release material usually consists of the modified routines or files and accompanying documentation or information posted on the “Known Problems” web page under the WRF users' site.  Most modifications to the source code in a minor release should fix software errors, provide code clean-up, or address a problem that justifies an immediate fix.  Information on bug fixes is posted on the web.  The director of wrfhelp informs the Release Committee of the decision to make a minor release.

 

 

c) Nomenclature

 

Releases are generally numbered as Version a.b or a.b.c, and, in emergency situations, a.b.c.d.  The first digit, or changes to it, reflects major restructuring, development, or upgrades.  As an example, a new version of WRF with a different leading digit might not ingest or correctly process data from a version of the code with a different leading digit.  The second digit reflects new capabilities and backward compatibility with older input files, and is usually associated with a major release.  Changes to the second digit will typically mean that the new version will provide access to new features.  The new version may also modify the meaning of run-time options.  The third digit reflects bug fixes/minor releases.  On rare occasions, an emergency minor release might be issued to quickly correct a significant bug discovered.  This would be designated by a fourth digit.

 

 

3) Activity status and procedures

 

The levels of code maintenance and testing activity vary with the calendar for a major release.  The two basic status designations are normal and pre-release. 

 

The normal status period, that after a major release and before the ramp-up to the next, has a baseline level of activity.  This level of activity includes the reviewing of proposed commits, integration of contributions, and software testing.  The testing is performed with an automated system and is done to ensure accuracy and confidence in parallel results. 

 

The pre-release status period begins approximately five months prior to a scheduled major release, when the release committee has begun meeting regularly.  In this period the major release schedule is being refined, an initial release picture is compiled, and the release branch cutoff date is set.  Code testing covers additional architectures, case studies, regression testing, and pre- and post-processors.

 

a)      Baseline activity— Status: Normal

 

 (i) Post-release

 

After a WRF major release testing is tied to the regular meetings of the Developers’ Committee.  Every proposed commit undergoes testing conducted by the contributor and summarized and documented for the Developers' Committee.  After accumulated modifications have been committed to the repository, a larger regression test is conducted on the primary supercomputers at NCAR.  The purpose of this regression testing is to identify software errors (e.g., bit-for-bit differences) and failures to compile or run.  These tests are short and do not attempt to detect or analyze differences in the forecast skill.

 

A proposed modification to the repository is circulated among the Developers’ Committee via email.  The proposing contributor must first describe and explain the modification (e.g., bug fix, enhancement, new feature) on a form provided by the DC (provided under the information for WRF Code Contributors: http://www2.mmm.ucar.edu/wrf/doc/contrib_info.pdf).  The contributor must also describe the changes to the code required and list the touched files.  If the proposed change modifies scientific results, the contributor must show how results are changed, and additional testing may be required to obtain this information.  Other members of the Developers’ Committee may ask for a hold to be placed on some or all of a set of commits to allow for further review.

 

Once the large regression test has been conducted, any unexpected failures are investigated to identify which of the previous commits introduced the fail condition.  The contributor is informed of the testing status.  If a fix is easily integrated into the code, it is, and the fixed code is re-tested.  If no simple fix is available, or if the supplied fix is not able to rectify the testing suite to the baseline passing status, then the modification is backed out of the repository.  A final large regression test is always required on the current top of the repository to validate the code.

 

 (ii) Release minus 6 months (R-6m)

 

At about this point the Release Committee (RC) has a first meeting to begin discussion of the next major release.  The RC identifies a major release as a combination of candidate features and a proposed date.  Through an announcement made through wrfhelp, the Release Committee provides the preliminary information on the release. 

 

The Release Committee considers for the release those capabilities that have been proposed and that can be integrated into the repository prior to the branch cutoff.  The release picture is the list of features that is being considered for the release as well as anything that the RC considers mandatory for the release.  The RC compiles this release picture and posts the release schedule on a web page.

 

Prospective code contributors who are not on the Developers’ Committee are assigned a DC member to coordinate testing of their components and to support the commit process.  The contributors work with these liaisons to get the source code into the repository.  The contributors are responsible for following all testing and documentation requirements and DC requests and for providing any necessary data.  The Developers’ Committee updates the Release Committee on the status of the code testing and the commit progress of targeted new features.

 

 

b) Release activity— Status: Pre-release

 

(i) Release minus 4–5 months (R-4–5m)

 

If not done previously, prospective contributors should submit to the Release Committee a notice of intent to submit code that they would like to have considered for the release.  This provides the committee with information on what submissions to expect and assists with the overall planning.  However, at no point is there any guarantee that any code proposed will get into a given release.

 

(ii) Release minus 3–4 months (R-3-4 m)

 

In this period developers intending to contribute code must have provided their code to their points of contact (POCs) on the Release Committee or Developers' Committee if they want it considered for inclusion in the release branch.  While it is emphasized that there is no guarantee that any code submitted will get into a given release, knowing that no further submissions will be considered assists in the preparation of the release branch, which is cut off at approximately R‒3 mos.  The code must conform to the WRF coding standards described in “Information for WRF Code Contributors" (http://www2.mmm.ucar.edu/wrf/docs/contrib_info.pdf).

 

Developers must put their offered codes through software and case testing.  This testing is described in “Information for WRF Code Contributors" (http://www2.mmm.ucar.edu/wrf/docs/contrib_info.pdf) and in "Testing Information for Contributors" (http://wrf-model.org/users/testing.php).  Developers should consult with a member of the Release Committee or Developers’ Committee to be set up with a point of contact, and developers can contact wrfhelp if they need assistance in this.

 

At approximately the 3-month point (R-3m), the release branch is cut off.  All new code and features to be included in the release must be in the repository at this time in order to be available for the branch.  After the branch is formed, the changes allowed to its code are only those necessary for the release’s development, such as bug fixes or modifications for difficulties with new feature interaction.  Tests for new features in the release are included in the regression suite. 

 

Even with the release branch created, the trunk remains open to commits.  Code proposed for commit after the branch cutoff is directed to the trunk.  Commits to the trunk follow the standard procedures.

 

Within this time frame, meetings of the Release Committee are held approximately every 2 weeks.

Checks on new features to be included in the release are discussed by the Release Committee and by the Developers’ Committee.  The decision to either (i) drop a feature which will not be ready within the schedule or (ii) change the release date so that a crucial feature is included in the release is made by the Release Committee. 

 

 

(iii) Release minus 2 months (R-2m)

 

Members of the Release Committee may decide upon non-automated tests to review the forecast skill of the new release code.  These may include case study analyses or linking to other software packages.  As tests are run, the results are discussed at the Release Committee meetings.

 

During this period the regression test data sets are finalized.  Links to the pre- and post-processors are tested with the release branch (e.g., naming conventions, directory structure).  The statuses of the links to the model forecast component are discussed in the Release Committee meetings.  Needs for documentation of new features or modified code are identified, and the contributing developers are requested to provide this, as well as recommendations to users.

 

Within this period, the Developers’ Committee at its discretion may make beta-versions of the code available to friendly users and to vendors and for pre-release testing.  Also within this period, the Developers’ Committee may implement moratoria on changes/commits to the branch to facilitate the resolution of problems or testing by various users.  Such moratoria and their timing are at the discretion of the Developers’ Committee. 

 

(iv) Release minus 1 month (R-1m)

 

Except for the accommodation of bug fixes, in this period the branch remains frozen: only corrections for problems identified during testing on the code to be released may be introduced.  Typically, errors introduced into previously existing (and working) code are allowed to be fixed, while larger-scale restructuring (even for code logic which may arguably be a better way to run) may be redirected to the trunk, which remains open to both continuing and new development.  Changes to documentation files and inline documentation, however, are always freely accepted. 

 

Changes to the branch are announced, and a designated member of the Developers’ Committee updates the branch.  The Release Committee and the Developers’ Committee each meets weekly, where the testing status and issues are the focus.

 

The full set of regression tests is run on machines at NCAR.  The pre-release version of the code may be used in real-time applications in additional configuration tests.

 

(v) Release minus 3 weeks (R-3w)

 

Software-related issues that have been uncovered in testing are assigned to members of the Developers’ Committee, who are to return with a solution.  The committee may require the developer to resolve or assist in resolving the problem.  If the problem cannot be fixed, then the Release Committee is given one of three recommendations: (i) accept the code as is, (ii) remove the offending code from the release, or (iii) delay the release date.  The Release Committee makes the decision with respect to the release.  The final round of large testing begins.

 

(vii) Release minus 1 week (R-1w)

 

The Developers’ Committee determines the certification, or final approval, of the code.  The branch is turned over to the control of the WRF user support group.  The final modifications for code identification, documentation, and release readiness are committed.  The code is packaged and staged for final release.  The countdown list of WRF user support includes tar file generation, web page updates, documentation updates, and user notification.

 

 

c) Release procedures

 

(i) Publication of release plans

 

Once developed, the plans and schedule for the next major release are posted on the web.  This typically follows the initial Release Committee meeting in the R‒6 mos. timeframe.  The release information includes a list of the significant candidate items (if they are known) and a timetable.

 

At the annual WRF Users’ Workshop, the previous release features are covered in detail in a presentation.  Any significant known plans for the next major release may be included as well.

 

(ii) Coordination with contributors and developers

 

Contributors should coordinate with Developers’ Committee members to make sure their code intentions for future releases are known.  While there is no guarantee that any code will get into a given release, usually the DC members can gauge whether developments will be ready for inclusion in the next release.

 

(iii) Publication of procedures and responsibilities

 

The Release Committee maintains a web page providing information on the timetable and known candidate features of pending major releases.  The page includes links to the pages with responsibilities of contributors and the procedures for contribution.  Information on the Release Committee is also provided.

 

 

4) Code Contribution

 

a) Status Normal

 

New developments, code improvements, and contributions of bugfixes continue for the trunk between releases.  It is up to the Developers’ Committee to confirm that all code provided works as advertised with the top of the repository and passes the basic regression tests.  Code contributors are responsible for:

 

(1)   Performing the required code testing;

(2)   Ensuring that the code has been submitted with the required commit information and that the code conforms to the WRF coding standards;

(3)   Warning the Developers’ Committee about any limiting underlying assumptions or possible code conflicts;

(4)   Working with Developers’ Committee members to develop the necessary tests to verify that future modifications do not adversely impact their code and incorporating these tests into the WRF regression test suite; and

(5)   Documentation.

 

Contributors are responsible for supplying documentation on the code they provide, which may be in the form of a web page or adequate (as deemed by the Developers’ Committee) README files or inline documentation.

 

The Developer’s Committee determines the procedures for repository access.  As described under the "WRF Physics Review Process" page (http://www2.mmm.ucar.ed/wr/users/physics_review.php), for proposed new physics packages the WRF Physics Review Panel first reviews the submitted materials.  It then makes a recommendation to the DC on whether to accept the new package.  This is simply a recommendation, and the Developers' Committee has the final say.  Denial of any code's addition to the repository is a decision by the DC.  The procedure reflects a denial by veto rather than a majority rule decision.  Upon any member’s veto of the change, the proposed code modification will not occur.

 

 

                        b) Pre-Release Period

 

(i) Release minus 3–4 months (R–3-4m)

 

By this period, all code sought to be considered for the release by developers should be provided by them to the release committee point(s) of contact.  While there is no guarantee that any code will get into a given release, this assists in the preparation of the release branch, which will be cut off at approximately R3 mos.  The code must have been put through the testing procedures described in “Information for WRF Code Contributors" (http://www2.mmm.ucar.edu/wrf/docs/contrib_info.pdf) and in "Testing Information for Contributors" (http://wrf-model.org/users/testing.php).  It is the responsibility of the developers to be communicating with the POCs prior to this point to make sure that the deadline is met.

 

By the release branch cutoff date (approximately R‒3 mos.), all of the proposed new features for the release must be available for the branch.  The Release Committee and the Developers’ Committee work with contributors to make sure that the release schedule does not slip to accommodate laggard development efforts.  As the release gets nearer, general bug fixes or modifications to address issues uncovered in testing are still eligible for inclusion.  While large code changes unrelated to the main elements or goals of the release are not permitted in the branch, they remain eligible for the trunk.  Also, new items arriving after the branch cutoff date may be proposed for commit to the trunk.

 

(iii) Release minus 1 month (R–1m)

 

In this stage the only acceptable changes are those pertaining to testing of the frozen code.  Improvements to the frozen code or bugfixes uncovered in testing are allowed into the branch, but only with approval of the Developers’ Committee.  The automated regression tests and the branch updates are handled by a designated member of the Developers’ Committee.

 

The Release Committee posts information on any changes in the status of the major release.   During this last month of testing, final results for additional architectures, case studies, and timing studies are received.  The Release Committee reviews the information to ascertain that the release is on schedule or that remedial action is required.

 

 

11/16

 



 
Home -- Model System -- User Support -- Doc / Pub -- Links -- Download -- WRF Real-time Forecast