WRF Utilities and Tools

This chapter contains descriptions about various data types, and discusses a number of simple utilities that read, plot, and manipulate WRF-ARW data. Verification and a few other miscellaneous tools are also described.

GRIB Data

GRIB Data

GRidded Binary WMO data format, which is output by many operational weather centers, and can be used as input for the WPS program.

GRIB data is available in either “Edition 1” or “Edition 2.” Documentation for both GRIB1 and GRIB2 can be found on the NSF NCAR/CISL RDA WMO Grib Data Format web page.

It is useful to be able to to interpret which fields are available in input data set. For instance, NCEP uses the GRIB1 code 33 for the U-component of the wind, and 34 for the V-component. Other centers may use different codes, so always obtain the GRIB codes from the center where the data originate.

GRIB2 uses 3 codes for each field - product, category and parameter. product 0 refers to meteorological products. Category refers to the type of field; e.g., category 0 is temperature, category 1 is moisture and category 2 is momentum. Parameter is the field number. So whereas GRIB1 only uses code 33 for the U-component of the wind, GRIB2 will use 0,2,2, for the U-component, and 0,2,3 for the V-component.

Reading GRIB Data

RDA Decoders

GRIB data decoders and various other tools are available from NSF NCAR/CISL RDA WMO Grib Data Format.

g1print & g2print

g1print.exe and g2print.exe are utilities that print a listing of fields, levels, and dates in GRIB Edition 1 and GRIB Edition 2 files. These utilities are built along with an installation of WPS, and are found in the WPS/util/ directory. Find use instructions in the WPS chapter.

wgrib & wgrib2

The wgrib and wgrib2 tools print information found in GRIB Edition 1/2 files. Both tools are available from NSF NCAR/CISL RDA WMO Grib Data Format.

Displaying GRIB Data

GRIB data can be displayed with GrADS with the use of the grib2ctl.pl script, and with Panoply.

Manipulating GRIB Data

To convert GRIB data files to netCDF formatted files, use the ncl_convert2nc tool.

Intermediate File Format

Intermediate file format is one that is created by the WPS/ungrib program, which decodes GRIB datasets into a simple format that is required by the metgrid program. File output during ungrib.exe are in intermediate format. See Writing Meteorological Data to the Intermediate Format for details.

Reading Intermediate Data

The rd_intermediate.exe utility reads and prints information about the fields contained in an ungrib-produced intermediate file. The tool is built along with a WPS installation and is found in the WPS/util/ directory. To run this program, from the WPS directory, type

./util/rd_intermediate.exe PREFIX:YYYY-MM-DD_hh

where PREFIX:YYYY-MM-DD_hh must be replaced with the intermediate file name.

Displaying Intermediate Data

The plotfmt.ncl NCL script can be used to display intermediate file content created by WPS/ungrib.exe. This utility is automatically built along with an installation of WPS, and is found in the WPS/util/ directory.

This script reads intermediate files and outputs graphics in a variety of formats (on the screen, pdf, ps, ncgm). The script requires NCL version 6.2.0 or newer. The following is an example to run this script:

ncl plotfmt.ncl 'filename="PREFIX:YYYY-MM-DD_hh"'

where PREFIX:YYYY-MM-DD_hh must be replaced with the specific intermediate file name.

Manipulating Intermediate Data

The int2nc.exe utility can be used to convert intermediate files created by WPS/ungrib.exe into netCDF formatted files. This tool is automatically built with a WPS installation, and is available in the WPS/util/ directory.

To use this tool, from the WPS/ directory, issue:

./util/int2nc.exe PREFIX:YYYY-MM-DD_hh

where PREFIX:YYYY-MM-DD_hh must be replaced with the specific intermediate file name. The resulting file will be called PREFIX:YYYY-MM-DD_hh.nc. To plot variables contained in the resulting file, use the plotfmt.ncl NCL script discussed in the above section.

NetCDF Data

NetCDF stands for Network Common Data Form. Most tools mentioned in this section can be used for WRF netCDF data, as well as other netCDF data sets. NetCDF is one of the supported data formats chosen for WRF I/O API. The advantages of using netCDF data are:

• Most graphical packages support netCDF file formats

• NetCDF files are platform-independent (big-endian / little-endian)

• A plethora of software exists that can be used to process/manipulate netCDF data

See the following documentation for additional details about the netCDF format:

• Unidata General NetCDF Information

• NetCDF-Fortran User Guide

Reading NetCDF Data

The following tools can be used to read netCDF data formatted files.

ncdump

The ncdump command-line utility converts netCDF data to human-readable text form. It can be useful for viewing content within a netCDF file. This tool is available with the installation of netCDF libraries. Some examples uses are:

• ncdump -h file (print header information)

• ncdump -v VAR file (print header information and the full field VAR)

• ncdump -v Times file (prints the times available in a WRF output file)

read_wrf_nc

The read_wrf_nc utility provides a method for taking a quick look at WRF netCDF files. This utility differs from the netCDF ncdump tool in that read_wrf_nc:

• Includes a large number of options, allowing users to look at a specific part of a netCDF file

• Is written in Fortran 90, which allows users to add options

read_wrf_nc can be used for geogrid, metgrid and wrf input/output files. Only three basic diagnostics are available: pressure, height, and tk, which can be activated with the -diag command (these diagnostics are only available for wrfout files).

Install read_wrf_nc

1. Obtain read_wrf_nc.

   • The code should run on any machine with a netCDF library.

   
   

2. To compile the code, use the compiler-specific flags listed near the top of the read_wrf_nc.f file. For e.g., for a Linux machine, issue the following command (all one line):

   pgf90  read_wrf_nc.f  -L/usr/local/netcdf/lib
   -lnetcdf  -lm  -I/usr/local/netcdf/include
   -Mfree  -o read_wrf_nc
   

   
   

   If successful, the read_wrf_nc executable will be created.

Run read_wrf_nc

To run this program, use the following command:

./read_wrf_nc  wrf_data_file_name  [-options]

options : [-h / help] [-att] [-m] [-M z] [-s]
[-S x y z] [-v VAR] [-V VAR] [-w VAR]
[-t t1 [t2]] [-times]
[-ts xy X  Y  VAR VAR ....]
[-ts ll  lat lon VAR VAR ....]
[-lev z] [-rot] [-diag]
[-EditData VAR]

See a description of the options below:

Note

Options [-att], [-t], and [-diag] can be used with other options. Default options are [-att -s].

-h / help	Prints help information
-att	Prints global attributes
-m	Prints a list of fields available for each time, plus the minimum and maximum values for each field
-M z	Prints a list of fields available for each time, plus the minimum and maximum values for each field; the min and max values of 3d fields will be for the z level of the field
-s	Prints a list of fields available for each time, plus a sample value for each field; sample value is taken from the middle of the model domain
-S x y z	Prints a list of fields available for each time, plus a sample value for each field; Sample value is at point x,y,z in the model domain
-t t1 [t2]	Applies options only to times t1 to t2 (t2 is optional); if not set, options will only apply to t1
-times	Prints only the times in the file
-ts	Generates time series output; a full vertical profile for each variable will be created -ts xy X Y VAR VAR … will generate time series output for all variables (VAR) at location x,y -ts ll lat lon VAR VAR … will generate time series output for all variables at the x,y location nearest to lat/lon
-lev z	Creates a time series for level z (only works with option -ts)
-rot	Rotates winds to Earth coordinates (only works with option -ts)
-diag	Add this option to see output for the diagnostics temperature (K), full model pressure and model height (tk, pressure, and height)
-v VAR	Prints basic information about field VAR
-V VAR	Prints basic information about field VAR, and dumps the full field out to the screen
-w VAR	Writes the full field out to a file VAR.out

SPECIAL Option: -EditData

The option -EditData VAR (where VAR is replaced with the variable name) provides the option to read a WRF output netCDF file, change a specific field, and write it back to the output file.

• Take care when using this option, as it changes the current WRF output file!

• Only one field at a time can be changed; therefore, if, for e.g., three fields need to be changed, the program will need to be run three times - each with a different VAR.

• If there are multiple times in the WRF netCDF file, by default, all times for the variable VAR will be changed. To only change one time period, use the -t option.

To use this option:

Note

Make a copy of the WRF netCDF file before using this option!

1. Edit the subroutine USER_CODE

   • Add an ‘IF-statement’ block for the variable that will be changed. This prevents a variable from being overwritten by mistake.

   • For REAL data arrays, work with the array data_real and for INTEGER data arrays, work with the array data_int.

   
   

   Example 1
   To change all (all time periods) values of U to a constant 10.0 m/s, add the following IF-statement:

   else if ( var == 'U') then
         data_real = 10.0
   

   
   

   Example 2
   To change a section of the LANDMASK data to water points:

   else if ( var == 'LANDMASK') then
         data_real(10:15,20:25,1) = 0
   

   
   

   Example 3
   Change all ISLTYP category 3 values to category 7 values (Note this is an INTEGER field):

   else if ( var == 'ISLTYP') then
         where (data_int == 3 )
             data_int = 7
         end where
   

   
   

2. Compile and run the program. When prompted, reply “yes” to confirm these changes should be made.

diffwrf

The diffwrf utility allows for the comparison of one netCDF-formatted WRF output file, with another one. This can be useful to determine differences in output files. diffwrf is automatically built with a WRF installation, and is found in the WRF/external/io_netcdf/ directory. The following example shows the syntax for using this tool:

First, move the tool to the current running directory (for e.g., em_real):

mv ../../external/io_netcdf/diffwrf .

Then issue something like:

./diffwrf wrfout_file1 wrfout_file2

where wrfout_file1 and wrfout_file2 are the two files that are compared.

Displaying NetCDF Data

The following tools can be used to plot netCDF formatted data files.

ncview

ncview is a graphical interface that provides interactive graphical displays for a quick look at netCDF file content. It displays all variables found within a file, and comes with various display options - for e.g., time-series or plotting a vertical cross section.

Download ncview

ncBrowse

ncBrowse is a graphical interface (similar to ncview) that provides interactive graphical displays of data and attributes from netCDF data files.

Download ncBrowse

plotfmt_nc.ncl

plotfmt_nc.ncl is an NCL script that plots netCDF output files created by int2nc.exe (see the Manipulating Intermediate Data section above for details). This script is built automatically with an installation of WPS, and is found in the WPS/util/ directory. plotfmt_nc.ncl must be run in the same directory where the netCDF files reside. The file to be plotted should be entered on the command line. For e.g.,

ncl plotfmt_nc.ncl 'inputFILE="PREFIX:YYYY-MM-DD_hh.nc"'

where PREFIX:YYYY-MM-DD_hh.nc must be replaced by the specific name of the file(s) created by int2nc.exe.

Manipulating NetCDF Data

The following tools are options for converting or modifying netCDF data files.

iowrf

iowrf is a utility that allows for basic manipulation of WRF output netCDF files - for e.g., thinning the data, de-staggering the data, or extracting data from a section of the data file.

Install iowrf

1. Obtain iowrf.

   • The code should run on any machine with a netCDF library.

   
   

2. To compile the code, use the compile flags at the top of the utility. For e.g., for a Linux machine, issue the command (all one line):

   pgf90 iowrf.f  -L/usr/local/netcdf/lib
   -lnetcdf  -lm  -I/usr/local/netcdf/include
   -Mfree  -o iowrf
   

   
   

   If successful, this will create the executable iowrf.

Run iowrf

To run iowrf, issue the following command:

./iowrf  wrf_data_file_name  [-options]

options : [-h / help] [-thina X] [-thin X] [-box {}] [-A] [-64bit]

See a description of the options below.

-thina X	Thins the data with a ratio of 1:X; data will be averaged before being fed back
-thin X	Thins the data with a ratio of 1:X; no averaging will be done
-box {}	Extracts a box from the data file; x,y,z can be controlled independently; for e.g. -box x 10 30 y 10 30 z 5 15 -box x 10 30 z 5 15 -box y 10 30 -box z 5 15
-A	De-staggers the data; no thinning will take place
-64bit	Allows large files (> 2GB) to have read / write access

wrf_interp

Note

wrf_interp is the newer version of the older utility, p_interp. wrf_interp is the recommended utility, but it is still possible to download and use p_interp.

wrf_interp is a program to interpolate WRF output files to pressure, height-agl, height-msl, potential temperature, and equivalent potential temperature. It also has the capability to perform underground extrapolation, and it calculates relative humidity, temperature (K), geopotential height (m), and potential vorticity on the user-requested interpolation levels.

** KKW COME BACK TO THIS - NEED TO FIGURE OUT HOW TO COMPILE THIS ON DERECHO **

v_interp

v_interp is a utility program to add vertical levels in the WRF input and boundary files.

Note

If running a nested domain, this utility can only be used with one-way nesting (i.e., when feedback=0).

** KKW COME BACK TO THIS - NEED TO FIGURE OUT HOW TO COMPILE THIS ON DERECHO **

NetCDF Operators (NCO)

NetCDF operators (NCO) is a suite of stand-alone programs that can be used to manipulate data (by performing grid point averaging / file differencing / file appending); a few available programs, with examples, are listed below, see the NCO site for a list of all available programs.

• ncdiff : Difference between two files; e.g.,

  ncdiff  input1.nc input2.nc output.nc
  

  
  

• ncrcat : Writes specified variables / times to a new file; e.g.

  ncrcat -v RAINNC wrfout* RAINNC.nc
  ncrcat -d Time,0,231 -v  RAINNC  wrfout* RAINNC.nc
  

  
  

• ncra : Averages variables and writes to a new file; e.g.

  ncra -v OLR  wrfout* OLR.nc
  

  
  

• ncks (nc kitchen sink) : Combination of NCO tools all in one (handy: one tool for multiple operations). An especially handy use of this tool is to split large files into smaller files, e.g.

  ncks -A -F -d Time,1,1 wrfout* -o wrfout_time1.nc
  

Domain Configuration

plotgrids.ncl (or plotgrids_new.ncl) is an NCL script that displays the domain configuration from the namelist.wps file. The script is located in the WPS/util/ directory. See Visualizing the Domain from the WPS chapter of this guide for additional details.

Model Verification

The Model Evaluation Tools (MET) program is a highly configurable, state-of-the-art suite of verification tools. It was developed using output from the Weather Research and Forecasting (WRF) modeling system, but may be applied to output from other modeling systems, as well. MET provides a variety of verification techniques, including:

• Standard verification scores, comparing gridded model data to point-based observations

• Standard verification scores, comparing gridded model data to gridded observations

• Object-based verification method, comparing gridded model data to gridded observations

Data Conversion

ImageMagick is a software suite that creates, edits, and composes bitmap images. It can read, convert, and write images in a variety of formats (over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD, PNG, Postscript, SVG, and TIFF. Use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and B_zier curves. See the ImageMagick website to download and install the software package, free of charge.

Examples of converting data with ImageMagick software:

convert  file.pdf     file.png
convert  file.png     file.bmp
convert  file.pdf     file.gif
convert  file.ras     file.png

Note

ImageMagick cannot convert ncgm (NCAR Graphics) file format to other file formats.

Converting ncgm (NCAR Graphics) File Format

NCAR Graphics contains tools to convert ncgm files to raster file formats. Once files are in raster file format, ImageMagick can be used to translate the files into other formats. For ncgm files containing a single frame, use ctrans:

ctrans  -d sun  file.ncgm  file.ras

For ncgm files containing multiple frames, first use med (metafile frame editor) and then ctrans. med will create multiple single frame files called medxxx.ncgm.

med -e '1,$ split $' file.ncgm
ctrans  -d sun_ med001.ncgm > med001.ras

Processing 3D HYCOM Ocean Data

The proc_oml.f tool is a utility for processing 3D HYCOM ocean model temperature data in netCDF format to produce initial ocean mixed layer depth field (H0ML) for use in a WRF simulation that uses the simple ocean mixed layer model option (omlcall=1, and oml_hml0<0).

proc_oml.f estimates two fields from the HYCOM data:

1. Effective mixed layer depth, based on the idea of ocean heat content (H0ML)

2. Mean ocean temperature in the top 200 m depth (TMOML). This is used as the lower limit for cooling SSTs in the wake of a hurricane.

Install proc_oml.f

1. Download the ‘proc_oml.f’ utility

2. To compile the code, use the compile flags shown at the top of the utility program. For example, for a Linux machine and pgf90 compiler, enter the command:

   pgf90 proc_oml.f  -L/usr/local/netcdf/lib -lnetcdf \
   -I/usr/local/netcdf/include -Mfree  -o proc_oml.f
   

If successful, this will create the executable proc_oml.

Run proc_oml.f

To run proc_oml.f, issue the command:

> ./proc_oml ocean-data-file.nc yyyymmddhh

where ocean-data-file.nc is the HYCOM ocean data file, and yyyymmddhh is the 10-digit date when the data is valid for (e.g. 2005082700). Successfully running the program produces an output file, MLD, which is in intermediate format - the format produced by the WPS/ungrib program.

To use this field in WPS/metgrid, add the constants_name parameter to the &metgrid namelist record in namelist.wps. For e.g.,:

&metgrid
constants_name = 'MLD',

WPS/metgrid has the additional fields in METGRID.TBL for proper horizontal interpolation. See AHW (WRF-ARW): Moving Nest and Ocean Initialization for additional information.

Objective Analysis - OBSGRID

The OBSGRID utility is an objective analysis tool, used specifically with WRF, for improving meteorological analysis on the mesoscale grid by incorporating observations. This is done without having to use three- or four-dimensional variational techniques (3DVAR or 4DVAR). See OBSGRID Documentation for details, including program installation and running options.