MM5 Model Version 2

WHAT'S NEW IN MM5 VERSION 2?

Jimy Dudhia, Dan Hansen, Wei Wang

1. INTRODUCTION

MM5 Version 1 was released early in 1994, and since then the average model user's computing environment has undergone significant change. Version 1 was based on the assumption that most users use the NCAR Cray for running MM5 and all the pre-processors, but nowadays this is no longer the case as many users use workstations at their own institutions to run the model. While the pre-processors are still quite tied to NCAR's Cray by their use of NCAR's data archives, the model itself does not need to be, and being the most costly component of the modeling system, it makes sense to allow users flexibility in where to use their resources. Consequently a workstation version of MM5 was also released in 1994, it was maintained but not supported by NCAR's mesouser support due to the machine-specific nature of user problems. A mailing list was set-up as a self-support group by Jim Steenburgh (then at University of Washington). Version 2 has arisen from a need to maintain a single code for use on the Cray and most workstations.

Another driving factor in Version 2 is the continually growing number of physics options becoming available. To incorporate all of these in a single code would noticeably lengthen the compile time unless measures were taken to selectively compile only those options that were needed. Consequently a large effort has been devoted to developing a Unix framework to allow this to be done in as portable a way as possible. Considering that the number of routines in Version 2 exceeds 200 it was also decided to subdivide the code into a multi-level directory structure.

Furthermore, since "nupdate" will no longer be supported under UNICOS 9.0, and is generally not available on non-Cray platforms, code maintenance has to be modified from the standard methods used up till now for all the modeling system decks. Internally at NCAR, Version 2 has been developed and will be maintained using CVS (built upon RCS). This is a commonly used and easy-to-learn software management system that allows us to keep track of all changes to given files and to retrieve old versions. It is freely and widely available if users would also like to install it on their own systems. However the methods of passing to users code changes or bugfixes will have to change from the current "mod decks" methods with which users are familiar.

2. SPECIFIC CHANGES

a. Portability

Version 2 is based on the workstation Version 1. The code is self-contained, not relying on libraries or any Cray-specific functions. It does not depend upon 64-bit words, using an 8-digit integer to represent the date instead of a 10-digit one. Certain pointer-length integer-length inconsistencies are accounted for on SGI's and DEC's. The Unix commands are as general as possible. The code multi-tasks on both Cray and SGI through cmic and c$ directives respectively.

b. Selective compilation

The Unix "make" utility is the basis for compiling the code. The user edits one file with compile, domain and certain physics options, and the "make" uses this information to select the routines for compilation. A C-preprocessor is applied for two purposes. (i) It adds all the include files. This used to be done with common decks in nupdate or with Fortran includes in the workstation version. Now the C #include is used. (ii) For a certain small number of routines, such as SOLVE3 #define and #ifdef are used to remove calls to routines that are not selected for compilation so that "unsatisfied externals" are not encountered by the compiler. The cpp (C-preprocessor) acts on the .F files in the Version 2 directories to create standard Fortran files (.i or .f) for compilation.

c. Directory tree structure

The Version 2 code has more than 200 subroutines and these are grouped according to function or physics option. For example, a single directory may contain routines that are relevant to the Kain-Fritsch cumulus parameterization, while another would have the Grell scheme's. This allows the Makefiles in each directory to be short and simple, and complements the selective compilation method by allowing branching in the Makefiles' calls to those in lower directories. For convenience in browsing the code there is a capability for putting all the selected Fortran code in a single directory or even in a single file.

d. Use of "make"

As stated above Unix "make" command generates the executable from the .F files. The internal "targets" are set according to the user-edited it configure.user file. The primary advantage of "make" for compilation is that when code is modified, only the modified routines will be recompiled. It can also detect dependencies between Fortran routines and include files such that changes to an include file will necessitate recompilation of its dependent routines. Other options are "make clean" to clean up the directories, "make code" to put the code in a single directory, and "make mm5.deck" will generate a machine-specific deck to link i/o files and run the code.

e. New physics options

Having the selective compilation capability allows us to increase physics options more easily without having to increase the size of the compilation task, so that in principle there is no limit to the number of options we can make available. Version 2 is being released with three more cumulus parameterizations (Kain-Fritsch, Fritsch-Chappell and Betts-Miller). There is also a new planetary boundary layer scheme (Burk-Thompson), radiation scheme (CCM2), two microphysics schemes with graupel (Reisner-2 and Goddard) and a multi-layer soil temperature prediction scheme.

f. Changes to existing schemes

There is an option to use horizontal diffusion of only perturbation temperature rather than full temperature in the nonhydrostatic option (possibly more accurate over terrain). Background vertical diffusion has been reduced from its previous default value and now depends on vertical grid size. The Grell cumulus parameterization scheme has been modified to allow more convective clouds. Solar clear-air scattering in the cloud-radiative scheme has been reduced to agree more with observations. A small curvature effect has been added to vertical acceleration to account for the primary earth-curvature term. Microphysics options can now be run with different options on different domains in a two-way nested run, and ice sedimentation effects are included in the simple-ice and Reisner schemes.

g. Efficiency changes

Some microphysics options now have look-up tables replacing exponential calculations as options. Many divisions, particularly by "pstar", have been replaced with multiplies by inverses. Statement functions have been replaced by 3D arrays to save CPU time at the expense of a slight memory increase. As a consequence the Version 2 code is slightly more efficient (5-10%) than Version 1.

h. FDDA memory changes

The memory for FDDA has been separated from the rest of the code so that restarts may be done without using the added FDDA memory, even if the first part included FDDA. This is primarily of use in dynamically initialized forecasts using an FDDA pre-forecast period.

i. Switch changes

Many Fortran parameters used for memory assignment based on physics options no longer need to be set directly as they are set automatically when the physics options are chosen prior to compilation. This particularly will avoid confusion with combinations of switches that were required to select microphysics options in Version 1, by using just one new switch (IMPHYS=1 to 7) to select all options from dry to full graupel schemes. Old switches like IICE, IEXICE no longer need to be set by the user. There is a new switch called ISKIP which allows users to initialize the model from a time other than the first time on the model-input file. It takes care of the time difference in the boundary-condition file. The fake dry (no latent heat) switch, IFDRY, has had its use extended to all microphysics options, and ICLOUD now works with the cloud-radiation and simple radiation options.

j. Code maintenance by CVS

CVS will be used to maintain the code at NCAR. The line-identifiers will be kept similar to those used by nupdate so that passing changes to users is done more easily. This means that our modifications to the code will now be done by direct editing, saving the step of having to find line identifiers. When a change is "committed", CVS allows a memo to be attached and it can log all the changes to a file by date and memo. Periodically, after a group of changes, a new version can be labeled with a "tag", and it is these tagged versions that will be released to the users similarly to the current periodic bugfix mod decks. We are hoping to develop an automatic method of naming changed and added lines that follows the nupdate system.

k. Code appearance

The code has been automatically indented to allow do-loops and if-blocks to be seen more easily. The Fortran code is all upper case to distinguish it from non-Fortran statements in lower case. These latter include C-preprocessor commands beginning with # and multi-tasking directives for Cray and SGI.

3. ON-GOING AND FURTHER WORK

The Version 2 code is still being tested on various platforms and certain physics options do not run on certain platforms yet but these problems are being addressed. The majority of the options work on all platforms tested which include Cray, SGI, Sun, DEC and IBM.

The development of Version 2 of the model is only the first step in making each modeling-system component work on workstations. As stated, the other components will have the added difficulty of being tied to NCAR data archives, and to be truly portable need to be made more generic with respect to their inputs of terrain, gridded analyses and observations.

ACKNOWLEDGMENTS

We are grateful for the help of Sue Chen in getting this effort going, and to John Michalakes for providing us with his code to indent the V2 code.