======================= WRF Utilities and Tools ======================= .. role:: underline :class: underline .. role:: nlnote :class: nlnote .. role:: nlheader :class: nlheader This chapter contains descriptions about various data types, and discusses a number of simple utilities that read, plot, and manipulate WRF-ARW data. Verification and a few other miscellaneous tools are also described. | | GRIB Data ========= .. container:: row m-0 p-0 .. container:: col-md-12 pl-0 pr-3 py-3 m-0 .. container:: card px-0 h-100 .. rst-class:: card-header-def .. rubric:: GRIB Data .. container:: card-body-def GRidded Binary WMO data format, which is output by many operational weather centers, and can be used as input for the WPS program. | | GRIB data is available in either "Edition 1" or "Edition 2." Documentation for both GRIB1 and GRIB2 can be found on the `NSF NCAR/CISL RDA WMO Grib Data Format `_ web page. | It is useful to be able to to interpret which fields are available in input data set. For instance, NCEP uses the GRIB1 code 33 for the U-component of the wind, and 34 for the V-component. *Other centers may use different codes, so always obtain the GRIB codes from the center where the data originate.* GRIB2 uses 3 codes for each field - *product*, *category* and *parameter*. *product* 0 refers to meteorological products. *Category* refers to the type of field; e.g., category 0 is temperature, category 1 is moisture and category 2 is momentum. *Parameter* is the field number. So whereas GRIB1 only uses code 33 for the U-component of the wind, GRIB2 will use 0,2,2, for the U-component, and 0,2,3 for the V-component. | | Reading GRIB Data ----------------- | RDA Decoders ++++++++++++ GRIB data decoders and various other tools are available from `NSF NCAR/CISL RDA WMO Grib Data Format `_. | | g1print & g2print +++++++++++++++++ *g1print.exe* and *g2print.exe* are utilities that print a listing of fields, levels, and dates in GRIB Edition 1 and GRIB Edition 2 files. These utilities are built along with an installation of WPS, and are found in the *WPS/util/* directory. `Find use instructions in the WPS chapter `_. | | wgrib & wgrib2 ++++++++++++++ The *wgrib* and *wgrib2* tools print information found in GRIB Edition 1/2 files. Both tools are available from `NSF NCAR/CISL RDA WMO Grib Data Format `_. | | | | Displaying GRIB Data -------------------- GRIB data can be displayed with GrADS with the use of the `grib2ctl.pl `_ script, and with `Panoply `_. | | | | Manipulating GRIB Data ---------------------- To convert GRIB data files to netCDF formatted files, use the `ncl_convert2nc tool `_. | | | | | Intermediate File Format ======================== Intermediate file format is one that is created by the WPS/ungrib program, which decodes GRIB datasets into a simple format that is required by the metgrid program. File output during *ungrib.exe* are in intermediate format. `See Writing Meteorological Data to the Intermediate Format `_ for details. | | Reading Intermediate Data ------------------------- The *rd_intermediate.exe* utility reads and prints information about the fields contained in an ungrib-produced intermediate file. The tool is built along with a WPS installation and is found in the *WPS/util/* directory. To run this program, from the WPS directory, type .. code-block:: ./util/rd_intermediate.exe PREFIX:YYYY-MM-DD_hh | where *PREFIX:YYYY-MM-DD_hh* must be replaced with the intermediate file name. | | | | Displaying Intermediate Data ---------------------------- The *plotfmt.ncl* NCL script can be used to display intermediate file content created by *WPS/ungrib.exe*. This utility is automatically built along with an installation of WPS, and is found in the *WPS/util/* directory. This script reads intermediate files and outputs graphics in a variety of formats (on the screen, pdf, ps, ncgm). The script requires NCL version 6.2.0 or newer. The following is an example to run this script: .. code-block:: ncl plotfmt.ncl 'filename="PREFIX:YYYY-MM-DD_hh"' | where *PREFIX:YYYY-MM-DD_hh* must be replaced with the specific intermediate file name. | | | | .. _Manipulating Intermediate Data: Manipulating Intermediate Data ------------------------------ The *int2nc.exe* utility can be used to convert intermediate files created by *WPS/ungrib.exe* into netCDF formatted files. This tool is automatically built with a WPS installation, and is available in the *WPS/util/* directory. To use this tool, from the *WPS/* directory, issue: .. code-block:: ./util/int2nc.exe PREFIX:YYYY-MM-DD_hh | where *PREFIX:YYYY-MM-DD_hh* must be replaced with the specific intermediate file name. The resulting file will be called *PREFIX:YYYY-MM-DD_hh.nc*. To plot variables contained in the resulting file, use the *plotfmt.ncl* NCL script discussed in the above section. | | | | | NetCDF Data =========== NetCDF stands for Network Common Data Form. Most tools mentioned in this section can be used for WRF netCDF data, as well as other netCDF data sets. NetCDF is one of the supported data formats chosen for WRF I/O API. The advantages of using netCDF data are: * Most graphical packages support netCDF file formats * NetCDF files are platform-independent (big-endian / little-endian) * A plethora of software exists that can be used to process/manipulate netCDF data | See the following documentation for additional details about the netCDF format: * `Unidata General NetCDF Information `_ * `NetCDF-Fortran User Guide `_ | | Reading NetCDF Data ------------------- The following tools can be used to read netCDF data formatted files. | ncdump ++++++ The `ncdump `_ command-line utility converts netCDF data to human-readable text form. It can be useful for viewing content within a netCDF file. This tool is available with the installation of netCDF libraries. Some examples uses are: * ``ncdump -h file`` (print header information) * ``ncdump -v VAR file`` (print header information and the full field VAR) * ``ncdump -v Times file`` (prints the times available in a WRF output file) | | read_wrf_nc +++++++++++ The *read_wrf_nc* utility provides a method for taking a quick look at WRF netCDF files. This utility differs from the netCDF *ncdump* tool in that *read_wrf_nc*: * Includes a large number of options, allowing users to look at a specific part of a netCDF file * Is written in Fortran 90, which allows users to add options | *read_wrf_nc* can be used for geogrid, metgrid and wrf input/output files. Only three basic diagnostics are available: *pressure*, *height*, and *tk*, which can be activated with the ``-diag`` command (these diagnostics are only available for *wrfout* files). | | **Install read_wrf_nc** #. `Obtain read_wrf_nc `_. * The code should run on any machine with a netCDF library. | #. To compile the code, use the compiler-specific flags listed near the top of the *read_wrf_nc.f* file. For e.g., for a Linux machine, issue the following command (all one line): .. code-block:: pgf90 read_wrf_nc.f -L/usr/local/netcdf/lib -lnetcdf -lm -I/usr/local/netcdf/include -Mfree -o read_wrf_nc | If successful, the *read_wrf_nc* executable will be created. | | **Run read_wrf_nc** To run this program, use the following command: .. code-block:: ./read_wrf_nc wrf_data_file_name [-options] options : [-h / help] [-att] [-m] [-M z] [-s] [-S x y z] [-v VAR] [-V VAR] [-w VAR] [-t t1 [t2]] [-times] [-ts xy X Y VAR VAR ....] [-ts ll lat lon VAR VAR ....] [-lev z] [-rot] [-diag] [-EditData VAR] | See a description of the options below: .. note:: Options [-att]\, [-t]\, and [-diag] can be used with other options. Default options are [-att -s]. | .. csv-table:: :widths: 25, 75 :width: 100% :escape: \ **-h / help**, Prints help information **-att**, Prints global attributes **-m**, Prints a list of fields available for each time\, plus the minimum and maximum values for each field **-M z**, Prints a list of fields available for each time\, plus the minimum and maximum values for each field; the min and max values of 3d fields will be for the *z* level of the field **-s**, Prints a list of fields available for each time\, plus a sample value for each field; sample value is taken from the middle of the model domain **-S x y z**, Prints a list of fields available for each time\, plus a sample value for each field; Sample value is at point *x\,y\,z* in the model domain **-t t1 [t2]**, Applies options only to times t1 to t2 (t2 is optional); if not set\, options will only apply to t1 **-times**, Prints only the times in the file **-ts**, Generates time series output; a full vertical profile for each variable will be created |br| |br| *-ts xy X Y VAR VAR ...* will generate time series output for all variables (*VAR*) at location *x\,y* |br| |br| *-ts ll lat lon VAR VAR ...* will generate time series output for all variables at the *x\,y* location nearest to lat/lon **-lev z**, Creates a time series for level *z* (only works with option *-ts*) **-rot**, Rotates winds to Earth coordinates (only works with option *-ts*) **-diag**, Add this option to see output for the diagnostics temperature (K)\, full model pressure and model height (tk\, pressure\, and height) **-v VAR**, Prints basic information about field *VAR* **-V VAR**, Prints basic information about field *VAR*\, and dumps the full field out to the screen **-w VAR**, Writes the full field out to a file *VAR.out* | | **SPECIAL Option: -EditData** The option ``-EditData VAR`` (where *VAR* is replaced with the variable name) provides the option to read a WRF output netCDF file, change a specific field, and write it back to the output file. * **Take care when using this option**, as it changes the current WRF output file! * Only one field at a time can be changed; therefore, if, for e.g., three fields need to be changed, the program will need to be run three times - each with a different *VAR*. * If there are multiple times in the WRF netCDF file, by default, all times for the variable *VAR* will be changed. To only change one time period, use the ``-t`` option. | | **To use this option:** .. note:: Make a copy of the WRF netCDF file before using this option! | #. Edit the subroutine *USER_CODE* * Add an 'IF-statement' block for the variable that will be changed. This prevents a variable from being overwritten by mistake. * For *REAL* data arrays, work with the array *data_real* and for *INTEGER* data arrays, work with the array *data_int*. | **Example 1** |br| To change all (all time periods) values of *U* to a constant 10.0 m/s, add the following IF-statement: .. code-block:: else if ( var == 'U') then data_real = 10.0 | **Example 2** |br| To change a section of the *LANDMASK* data to water points: .. code-block:: else if ( var == 'LANDMASK') then data_real(10:15,20:25,1) = 0 | **Example 3** |br| Change all *ISLTYP* category 3 values to category 7 values (*Note this is an INTEGER field*): .. code-block:: else if ( var == 'ISLTYP') then where (data_int == 3 ) data_int = 7 end where | #. Compile and run the program. When prompted, reply "yes" to confirm these changes should be made. | | | diffwrf +++++++ The *diffwrf* utility allows for the comparison of one netCDF-formatted WRF output file, with another one. This can be useful to determine differences in output files. *diffwrf* is automatically built with a WRF installation, and is found in the *WRF/external/io_netcdf/* directory. The following example shows the syntax for using this tool: First, move the tool to the current running directory (for e.g., *em_real*): .. code-block:: mv ../../external/io_netcdf/diffwrf . | Then issue something like: .. code-block:: ./diffwrf wrfout_file1 wrfout_file2 | where *wrfout_file1* and *wrfout_file2* are the two files that are compared. | | | | | Displaying NetCDF Data ---------------------- The following tools can be used to plot netCDF formatted data files. | | ncview ++++++ *ncview* is a graphical interface that provides interactive graphical displays for a quick look at netCDF file content. It displays all variables found within a file, and comes with various display options - for e.g., time-series or plotting a vertical cross section. | `Download ncview `_ | | | ncBrowse ++++++++ *ncBrowse* is a graphical interface (similar to *ncview*) that provides interactive graphical displays of data and attributes from netCDF data files. | `Download ncBrowse `_ | | | plotfmt_nc.ncl ++++++++++++++ *plotfmt_nc.ncl* is an NCL script that plots netCDF output files created by *int2nc.exe* (see the :ref:`Manipulating Intermediate Data` section above for details). This script is built automatically with an installation of WPS, and is found in the *WPS/util/* directory. *plotfmt_nc.ncl* must be run in the same directory where the netCDF files reside. The file to be plotted should be entered on the command line. For e.g., .. code-block:: ncl plotfmt_nc.ncl 'inputFILE="PREFIX:YYYY-MM-DD_hh.nc"' | where *PREFIX:YYYY-MM-DD_hh.nc* must be replaced by the specific name of the file(s) created by *int2nc.exe*. | | | | Manipulating NetCDF Data ------------------------ The following tools are options for converting or modifying netCDF data files. | | iowrf +++++ *iowrf* is a utility that allows for basic manipulation of WRF output netCDF files - for e.g., thinning the data, de-staggering the data, or extracting data from a section of the data file. | | **Install iowrf** #. `Obtain iowrf `_. * The code should run on any machine with a netCDF library. | #. To compile the code, use the compile flags at the top of the utility. For e.g., for a Linux machine, issue the command (all one line): .. code-block:: pgf90 iowrf.f -L/usr/local/netcdf/lib -lnetcdf -lm -I/usr/local/netcdf/include -Mfree -o iowrf | If successful, this will create the executable *iowrf*. | | **Run iowrf** To run *iowrf*, issue the following command: .. code-block:: ./iowrf wrf_data_file_name [-options] options : [-h / help] [-thina X] [-thin X] [-box {}] [-A] [-64bit] | | See a description of the options below. .. csv-table:: :widths: 25, 75 :width: 100% :escape: \ **-thina X**, Thins the data with a ratio of 1:X; data will be averaged before being fed back **-thin X**, Thins the data with a ratio of 1:X; no averaging will be done **-box {}**, Extracts a box from the data file; x\,y\,z can be controlled independently; for e.g. |br| -box x 10 30 y 10 30 z 5 15 |br| -box x 10 30 z 5 15 |br| -box y 10 30 |br| -box z 5 15 **-A**, De-staggers the data; no thinning will take place **-64bit**, Allows large files (> 2GB) to have read / write access | | | | wrf_interp ---------- .. note:: *wrf_interp* is the newer version of the older utility, *p_interp*. *wrf_interp* is the recommended utility, but it is still possible to `download and use p_interp `_. | *wrf_interp* is a program to interpolate WRF output files to pressure, height-agl, height-msl, potential temperature, and equivalent potential temperature. It also has the capability to perform underground extrapolation, and it calculates relative humidity, temperature (K), geopotential height (m), and potential vorticity on the user-requested interpolation levels. ** KKW COME BACK TO THIS - NEED TO FIGURE OUT HOW TO COMPILE THIS ON DERECHO ** | | | | v_interp -------- *v_interp* is a utility program to add vertical levels in the WRF input and boundary files. | .. note:: If running a nested domain, this utility can only be used with one-way nesting (i.e., when *feedback=0*). ** KKW COME BACK TO THIS - NEED TO FIGURE OUT HOW TO COMPILE THIS ON DERECHO ** | | | | NetCDF Operators (NCO) ---------------------- NetCDF operators (NCO) is a suite of stand-alone programs that can be used to manipulate data (by performing grid point averaging / file differencing / file appending); a few available programs, with examples, are listed below, `see the NCO site `_ for a list of all available programs. | * *ncdiff* : Difference between two files; e.g., .. code-block:: ncdiff input1.nc input2.nc output.nc | * *ncrcat* : Writes specified variables / times to a new file; e.g. .. code-block:: ncrcat -v RAINNC wrfout* RAINNC.nc ncrcat -d Time,0,231 -v RAINNC wrfout* RAINNC.nc | * *ncra* : Averages variables and writes to a new file; e.g. .. code-block:: ncra -v OLR wrfout* OLR.nc | * *ncks* (nc kitchen sink) : Combination of NCO tools all in one (handy: one tool for multiple operations). An especially handy use of this tool is to split large files into smaller files, e.g. .. code-block:: ncks -A -F -d Time,1,1 wrfout* -o wrfout_time1.nc | | | | | Domain Configuration ==================== *plotgrids.ncl* (or *plotgrids_new.ncl*) is an NCL script that displays the domain configuration from the *namelist.wps* file. The script is located in the *WPS/util/* directory. `See Visualizing the Domain `_ from the WPS chapter of this guide for additional details. | | | | | Model Verification ================== The `Model Evaluation Tools (MET) `_ program is a highly configurable, state-of-the-art suite of verification tools. It was developed using output from the Weather Research and Forecasting (WRF) modeling system, but may be applied to output from other modeling systems, as well. MET provides a variety of verification techniques, including: * Standard verification scores, comparing gridded model data to point-based observations * Standard verification scores, comparing gridded model data to gridded observations * Object-based verification method, comparing gridded model data to gridded observations | | | | | Data Conversion =============== ImageMagick is a software suite that creates, edits, and composes bitmap images. It can read, convert, and write images in a variety of formats (over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD, PNG, Postscript, SVG, and TIFF. Use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and B_zier curves. `See the ImageMagick website `_ to download and install the software package, free of charge. | Examples of converting data with ImageMagick software: .. code-block:: convert file.pdf file.png convert file.png file.bmp convert file.pdf file.gif convert file.ras file.png | .. note:: ImageMagick cannot convert ncgm (NCAR Graphics) file format to other file formats. | | Converting ncgm (NCAR Graphics) File Format ------------------------------------------- NCAR Graphics contains tools to convert ncgm files to raster file formats. Once files are in raster file format, ImageMagick can be used to translate the files into other formats. For ncgm files containing a single frame, use *ctrans*: .. code-block:: ctrans -d sun file.ncgm file.ras | For ncgm files containing multiple frames, first use *med* (metafile frame editor) and then *ctrans*. *med* will create multiple single frame files called *medxxx.ncgm*. .. code-block:: med -e '1,$ split $' file.ncgm ctrans -d sun_ med001.ncgm > med001.ras | | | | | Processing 3D HYCOM Ocean Data ============================== The *proc_oml.f* tool is a utility for processing `3D HYCOM `_ ocean model temperature data in netCDF format to produce initial ocean mixed layer depth field (*H0ML*) for use in a WRF simulation that uses the simple ocean mixed layer model option (*omlcall=1*, and *oml_hml0<0*). *proc_oml.f* estimates two fields from the HYCOM data: #. Effective mixed layer depth, based on the idea of ocean heat content (*H0ML*) #. Mean ocean temperature in the top 200 m depth (TMOML). This is used as the lower limit for cooling SSTs in the wake of a hurricane. | | | Install proc_oml.f ------------------ #. `Download the 'proc_oml.f' utility `_ #. To compile the code, use the compile flags shown at the top of the utility program. For example, for a Linux machine and pgf90 compiler, enter the command: .. code-block:: pgf90 proc_oml.f -L/usr/local/netcdf/lib -lnetcdf \ -I/usr/local/netcdf/include -Mfree -o proc_oml.f | If successful, this will create the executable *proc_oml*. | | | Run proc_oml.f -------------- To run *proc_oml.f*, issue the command: .. code-block:: > ./proc_oml ocean-data-file.nc yyyymmddhh | where *ocean-data-file.nc* is the HYCOM ocean data file, and *yyyymmddhh* is the 10-digit date when the data is valid for (e.g. *2005082700*). Successfully running the program produces an output file, *MLD*, which is in intermediate format - the format produced by the WPS/ungrib program. To use this field in WPS/metgrid, add the *constants_name* parameter to the *&metgrid* namelist record in namelist.wps. For e.g.,: .. code-block:: &metgrid constants_name = 'MLD', | WPS/metgrid has the additional fields in *METGRID.TBL* for proper horizontal interpolation. See `AHW (WRF-ARW): Moving Nest and Ocean Initialization `_ for additional information. | | | | | Objective Analysis - OBSGRID ============================ The OBSGRID utility is an objective analysis tool, used specifically with WRF, for improving meteorological analysis on the mesoscale grid by incorporating observations. This is done without having to use three- or four-dimensional variational techniques (3DVAR or 4DVAR). `See OBSGRID Documentation <./obsgrid.html>`_ for details, including program installation and running options. | | | | |