Chapter 5: REGRID

5.1 Purpose

 

The purpose of REGRID is to read archived gridded meteorological analyses and forecasts on pressure levels and interpolate those analyses from some native grid and map projection to the horizontal grid and map projection as defined by the MM5 preprocessor program TERRAIN. REGRID handles pressure-level and surface analyses. Two-dimensional interpolation is performed on these levels. Other types of levels, such as constant height surfaces, isentropic levels or model sigma or eta levels, are not handled.

 

REGRID is the second step in the flow diagram of the MM5 modeling system (Fig.1.1) . It expects input from the TERRAIN program, and creates files ready for RAWINS, LITTLE_R, or INTERPF. These files are generally used as the first guess to an objective analysis (RAWINS or LITTLE_R), or as analyses which are to be directly interpolated to the MM5 model levels for initial and boundary conditions for MM5 (INTERPF).

 

An additional capability of REGRID is the capability to insert, or "bogus in" a tropical cyclone into the analysis. This is a fairly specialized usage of REGRID, and will not be discussed in any detail here. For details on this tropical cyclone bogussing method, see:

http://www2.mmm.ucar.edu/mm5/mm5v3/tc-report.pdf (pdf format)

http://www2.mmm.ucar.edu/mm5/mm5v3/tc-report.doc (word format)

 

 

5.2 Structure

 

REGRID is not a single program, but a suite of programs to handle various tasks of the REGRID package. The tasks are split up into two main components:

1) data input (i.e., reading the original meteorological analyses)

2) interpolation to MM5 grid.

The data input task is handled by the programs collectively known as "pregrid", and the interpolation to the MM5 grid is handled by the program "regridder". Communication between these programs is accomplished via intermediate files written in a pretty simple format. The pregrid tasks are further subdivided into programs which read some specific data formats, while the regridder tasks are managed in a single program. The intent is that individual users can easily write their own data input programs (i.e., their own pregrid programs) if necessary, thus introducing their own data into the MM5 Modeling System.

 

This division separates the fairly messy and very dataset-specific task of reading data from the more general task of interpolation. By this division, REGRID can be easily expanded to ingest more data sets, and users can more easily ingest their own data sets for use in the MM5 system.

 

5.3 A schematic

 

 

 

5.4 Input to pregrid

 

The pregrid program expects to find files of gridded meteorological analyses. Currently, pregrid can read many data sets formatted in GRIB Edition 1 (hereinafter referred to as GRIB), as well as several GRIB and non-GRIB data sets that have traditionally been available to MM5 users.

 

Most of the individual pregrid programs, particularly those dealing with GRIB datasets, also expect to find tables which tell the pregrid program what fields to access from the input files. These are referred to as "Vtables" and are discussed in greater detail below.

 

A Fortran namelist file passes user-specified options to pregrid. For pregrid, this is mostly date information.

 

 

5.5 Input to regridder

 

The regridder program expects to find in the files from pregrid the fields of temperature, horizontal wind components, relative humidity, height of pressure levels, sea-level pressure, sea-surface temperature, and snow-cover data. Other fields may be used as well, interpolated and passed on to the rest of the modeling system. When you set up and run the pregrid programs, you should verify that the files you pass to regridder contain the necessary fields. One way to verify this is to run regridder and see what error messages it gives you.

 

From the TERRAIN files, regridder finds terrain, land-use, and map data.

 

A Fortran namelist file passes user-specified options to regridder.

 

 

5.6 Output from regridder

 

The regridder program creates a file called "REGRID_DOMAIN#". This file contains the data at every time period for a single domain. This file is in MM5v3 format which is discussed in greater detail in Chapter 13, "I/O Format"

 

 

5.7 Intermediate Data Format

 

Key to the REGRID package is the data format used for passing data from pregrid to regridder. Data are passed from the pregrid programs to regridder via files written in the format described in this section.

 

5.8 Pregrid VTables

Pregrid is intended to read a wide variety of data sets. Since many data sets are archived and distributed in GRIB format, and the non-GRIB data sets we read use many of the same ideas for describing given fields, it is convenient to use the GRIB data sets as an example. 
 

The GRIB format describes each field by several code numbers. However, we cannot include the code tables in the program code itself, because these code numbers are not consistent from one data set to another. Also, pregrid must have the capability to ingest analyses that we have not anticipated. Therefore, we have to supply coded information to the program through some sort of input file. The pregrid VTables are the means we have chosen to do this. These tables are essentially a conversion from the GRIB method of referring to fields to the MM5-System method of referring to fields.
 

The body of the VTables consists of one or more lines describing the fields we want to extract from the analysis files. A couple of examples are in order:

 

GRIB  | Level | Level | Level | REGRID    | REGRID    | REGRID                |

Code  | Code  | 1     | 2     | Name      | Units     | Description           |

------+-------+-------+-------+-----------+-----------+-----------------------+

11    | 100   | *     |       | T         | K         | Temperature           |

11    | 105   | 2     |       | T         | K         | Temperature at 2 m    |

------+-------+-------+-------+-----------+-----------+-----------------------+
 

The first four columns of the Vtable represent the GRIB method of identifying fields. The last three columns represent the MM5 method of identifying fields. 
 

The GRIB Code is the code number identifying the variable to access. For example, in NCEP GRIB files, temperature is generally coded as 11.
 

The Level Code is the code number identifying the type of level on which the variable is expected. For example, GRIB Level Code 100 refers to pressure levels, and GRIB Level Code 105 refers to a fixed height (in meters) above the ground.
 

Level 1 is the GRIB code for the value of the level. An asterisk (*) means to get data from every level of the type defined by the Level Code. This (*) wild-card is effective only for the pressure-levels (level-code 100).
 

Level 2 is often needed for types of levels (such as averages or sums over a depth) which are defined by two values.
 

REGRID Name is the character string identifying the field to the rest of the modeling system.
 

REGRID Units are the units used for this field in the rest of the modeling system. This is simply a descriptive text to remind the user what the units are. Do not attempt to change the units in which a field is output by changing the Units string. It will not work, and you will wind up confusing yourself later.
 

REGRID Description is a text description of the field.
 

There are a few subtleties to the VTables. A situation that sometimes occurs is that we want a field that is not included in the source files, but may be derived from fields which are in those files. One example is relative humidity. Some data sets may archive specific humidity instead. Yet we can derive RH from specific humidity, temperature, and pressure. We want to write out RH, but not write out specific humidity. Since we need specific humidity to compute relative humidity, we need to ask for specific humidity in the Vtables. The signal in the VTables that a certain field is not to be written out is a blank REGRID Description. Since we want to write out relative humidity, we include the relative humidity in the VTables in the usual way (with no GRIB Code since it wouldn't be found anyway). There is coded into the program the conversion from specific humidity to relative humidity, so pregrid will create the relative humidity field.
 

GRIB | Level | Level | Level | REGRID   | REGRID    | REGRID                |

Code | Code  | 1     | 2     | Name     | Units     | Description           |

-----+-------+-------+-------+----------+-----------+-----------------------+

11   | 100   | *     |       | T        | K         | Temperature           |

51   | 100   | *     |       | SPECHUMD | kg kg{-1} |                       |

     | 100   | *     |       | RH       | %         | Relative Humidity     |

-----+-------+-------+-------+----------+-----------+-----------------------+
 

Those conversions already coded into pregrid are:

- Relative humidity, from specific humidity (SPECHUMD), pressure, and temperature.

- Relative humidity, from dewpoint (DEWPT), pressure, and temperature

- Relative humidity, from dewpoint depression (DEPR), pressure, and temperature

- Relative humidity, from vapor pressure (VAPP), pressure, and temperature.

- Height, from geopotential (GEOPT)

This list may grow as users encounter various situations in which a computed field is necessary.

 

There are several VTables already set up for certain data sets that we have commonly accessed. Most of these are found in the directory pregrid/grib.misc. If you want to access a different GRIB-formatted data set, you must first determine which variables are included in that data set, and find the appropriate code numbers that are used by that data set. If you want to extract additional variables from a data set, you are responsible for finding the appropriate GRIB Code and Level Code numbers. You may find NCEP Office Note 388, a description of the GRIB Edition 1 format, useful. This document can be found in many places on the internet, including:

http://www.nco.ncep.noaa.gov/pmb/docs/on388.
 

 

5.9 Pregrid program functioning

 

The pregrid programs first read the namelist to determine the starting and ending times of the period of interest, and to find the desired time-interval of data. It then reads the VTable to determine which variables to extract from the source files. Then for each source file, the program scans through the data, pulling out all analyses which fall between the starting and ending times, and which are listed in the VTable. These analyses are written to preliminary files (named by time and written in the intermediate format). Once a record with a time greater than the user-specified ending time has been read, processing on that analysis file stops and the next file is opened (i.e., records in the source file are assumed to be sorted by time; this assumption can be turned off in the namelist). This cycle repeats until all the source files have been scanned.

 

Once that cycle is finished, the preliminary files are reread and derived fields are computed. Temporal interpolation is performed as necessary to fill in missing time periods. The final intermediate files are written.

 

5.10 Handy pregrid utility programs
 

We have created a handful of handy programs you should be aware of:

gribprint [- v | -V] file
Scans through a GRIB-formatted file, printing out a few details of each GRIB record.
With the -v option, prints more details of the GRIB header.
With the -V option, prints way too much of the actual data.
This program is made automatically when you issue the top-level make. It is found in
the pregrid/util directory.

 

plotfmt file
Makes plots of each field in the file which has been created by pregrid. This program
requires NCAR Graphics. To make this program, go to the util directory, and execute
"make plotfmt", or compile plotfmt.F using NCAR-Graphics and loading library
libpgu.a.

 

get_ncep.csh
to download archives from the GRIB-formatted NCEP GDAS analyses.

 

get_on84.csh
to download archives from the ON84-formatted NCEP GDAS analyses.

 

get_fnl.csh
to download archives from the ON84-formatted NCEP GDAS analyses.

 

get_nnrp.csh
to download archives from the NCEP/NCAR Reanalysis project.

 

get_era.csh
to download archives from the ECMWF Reanalysis project.

 

get_awip.csh
to download archives from the NCEP Eta model output (GRIB212).

 

get_toga.csh
to download archives from the ECMWF Reanalysis project.

 

All the get_*.csh scripts are available from ~mesouser/MM5V3/Util.
These scripts can be run on NCAR IBM computers to download analysis data from the mass store.
Be sure to first check the DSS catalogs for missing analyses.

 

5.11 How to run REGRID

 

1) Get the source code. The current REGRID release resides on NCAR's anonymous FTP site, ftp://mesouser/MM5V3/REGRID.TAR.gz. There may be a regrid.tar file available elsewhere for the tutorial. Uncompress ("gunzip regrid.tar.gz") and untar ("tar -xvf regrid.tar") the file. This creates a top-level REGRID directory called, strangely enough, REGRID.

 

2) Make the executables. To do this, go into the REGRID directory, and type "make". The makefiles we've set up attempt to recognize the type of system you are using, and select the appropriate compile and load options. Watch while your machine builds all the executables. If this doesn't work, you may find yourself having to go into the Makefiles yourself, and tuning some of the compiler and load options.

 

3) Get the analysis files. It may be convenient to put these files in a directory of their own. For users of NCAR's machines interested in historical cases, investigate the get_*** programs mentioned in "Handy utility programs".

 

4) Set up to run pregrid. The "pregrid.csh" shell in the pregrid subdirectory is handy. This is discussed below. If you need to create your own Vtables, do it now.

 

5) Make sure the pregrid.csh script is executable: "chmod u+x pregrid.csh"

 

6) Run pregrid.csh: "pregrid.csh"

 

7) Check your output: Make sure that pregrid created files for every time between your starting and ending dates. Check the printout to see which fields are available at which times.

 

8) Set up to run regridder: Get your terrain output file. Go to the regridder subdirectory, and edit the namelist for your specific case.

 

9) Run regridder. "regridder". This creates a file "REGRID_DOMAIN#".

 

 

5.12 pregrid.csh

 

A shell script has been created, called pregrid.csh, as a higher-level user interface for the pregrid programs. The top part of the pregrid.csh script looks something like this (variables the user may have to set are noted with a vertical bar to the left).

 

#############################################################################

#!/bin/csh -f

#

# set echo

#

# Put your input files for pregrid into the directory you specify as DataDir:

#

set DataDir = /usr/tmp/username/REGRID

#

# Specify the source of 3-d analyses

#

set SRC3D = ON84 # Old ON84-formatted NCEP GDAS analyses

# set SRC3D = NCEP # Newer GRIB-formatted NCEP GDAS analyses

# set SRC3D = GRIB # Many GRIB-format datasets

 

# InFiles: Tell the program where you have put the analysis files,

# and what you have called them. If SRC3D has the value "GRIB",

# then the Vtables you specify below in the script variable VT3D will

# be used to interpret the files you specify in the ${InFiles} variable.

 

set InFiles = ( ${DataDir}/NCEP* )

#

# Specify the source of SST analyses

#

 

# set SRCSST = ON84

# set SRCSST = NCEP

# set SRCSST = NAVY

set SRCSST = $SRC3D

 

#

# InSST: Tell the program where the files with SST analyses are. Do

# this only if SST analyses are coming from files not named above in

# InFiles. If SRCSST has the value "GRIB", then the Vtables you

# specify below in the script variable VTSST will be used to interpret

# the files you specify in the ${InSST} variable.

#

 

set InSST = ( )

 

#

# Select the source of snow-cover analyses (entirely optional)

#

set SRCSNOW = $SRC3D

# set SRCSNOW = ON84

# set SRCSNOW = GRIB

 

# InSnow: Set InSnow only if the snow-cover analyses are from files

# not listed in InFiles. If SRCSNOW has the value "GRIB", then the

# Vtables you specify below in the script variable VTSNOW will be used

# to interpret the files you specify in the ${InSnow} variable.

 

set InSnow = ()

 

#

# Select the source of soil model analyses (entirely optional)

#

# set SRCSOIL = $SRC3D

 

# InSoil: Set InSoil only if the soil analyses are from files

# not listed in InFiles. If SRCSOIL has the value "GRIB", then the

# Vtables you specify below in the script variable VTSOIL will be

# used to interpret the files you specify in the ${InSoil} variable.

 

# set InSoil = ()

 

#

# Build the Namelist

#

if ( -e ./pregrid.namelist ) then

rm ./pregrid.namelist

endif

cat << End_Of_Namelist | sed -e `s/#.*//; s/ *$//' > ./pregrid.namelist

&record1

#

# Set the starting date of the time period you want to process:

#

START_YEAR = 1993 # Year (Four digits)

START_MONTH = 03 # Month ( 01 - 12 )

START_DAY = 13 # Day ( 01 - 31 )

START_HOUR = 00 # Hour ( 00 - 23 )

 

END_YEAR = 1993 # Year (Four digits)

END_MONTH = 03 # Month ( 01 - 12 )

END_DAY = 14 # Day ( 01 - 31 )

END_HOUR = 00 # Hour ( 00 - 23 )

#

# Define the time interval to process.

#

INTERVAL = 43200 # Time interval (seconds) to process.

# This is most sanely the same as the time interval for

# which the analyses were archived, but you can really

# set this to just about anything, and pregrid will

# interpolate in time and/or skip over time periods for

# your regridding pleasure.

 

/

End_Of_Namelist

 

#

# Tell the pregrid programs which Vtables to use. Do this only

# if you have selected GRIB-formatted input using SRC___ = GRIB above.

# The directories referenced here are relative to REGRID/pregrid/.

#

# The Vtable files specified in VT3D will be applied to the files

# specified in the InFiles variable. Similarly, the Vtable files

# specified in VTSST, VTSNOW, and VTSOIL will be applied to the files

# listed above in InSST, InSNOW, and InSoil, respectively.

#

set VT3D = ( grib.misc/Vtable.NNRP3D )

set VTSST = ( grib.misc/Vtable.NNRPSST )

set VTSNOW = ( grib.misc/Vtable.xxxxSNOW )

set VTSOIL = ( grib.misc/Vtable.xxxxSOIL )

 

########################################################################

########################################################################

######                                                            ######

######                  END USER MODIFICATION                     ######

######                                                            ######

########################################################################

########################################################################

 

The rest of the shell performs some file shuffling and linking to put files in places that the pregrid programs expect. The shell links the source files to files of specific names which the pregrid programs expect. The shell builds a file called "Vtable" from the individual Vtables named by the user in the shell. The shell then executes the program, and moves the final output files to the pregrid directory.

 

 

5.13 The regridder Namelist options

 

The regridder program is run entirely through the namelist file. The regridder namelist is separated into four namelist records.

 

5.14 REGRID tar File
 

The regrid.tar file contains the following files and directories:

 

REGRID/README	general information about REGRID
REGRID/CHANGES	description of changes since earlier releases
REGRID/configure.rules	Rules for make
REGRID/pregrid	pregrid directory
REGRID/regridder	regridder directory
REGRID/Makefile	Makefile to create REGRID executables
REGRID/regrid.deck	batch job deck for NCAR's Cray

 

I would further direct your attention to the directory REGRID/pregrid/Doc/html, which contains documentation on REGRID in html format. Direct your web browser to REGRID/pregrid/Doc/html/Documentation_Home.html. The contents of the html documentation is approximately what is in these tutorial notes, though the organization and details differ.

 

 

5.15 Data
 

Users with NCAR computing accounts have ready access to a variety of gridded meteorological analyses and forecast products.

Some of the more commonly-used archived analyses include:

NCEP GDAS

NCEP/NCAR Reanalysis

NCEP EDAS

ECMWF Reanalysis

MRF/AVN "final analysis"

 

Users who do not have NCAR computing accounts must find their own ways to get gridded analyses or forecasts. Various real-time analyses and forecasts may be available via anonymous ftp, for example.