~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is the README file describing the MADIS2LITTLER converter, including installation and running instructions.


This is Version 1.1; updated and released January 2014

Release history:
V0.0 (unofficial release): March 24, 2011
V1.0:                      Oct 2013
V1.1:                      Feb 2014       Mike Kavulich:   Fixed an incorrect location for -lmadis in compile script

Limited support is available; if you have questions specifically related to the MADIS2LITTLER converter or LITTLE_R format, 
contact wrfhelp@ucar.edu. For questions related to MADIS data or MADIS API, contact madis-support@noaa.gov


PUBLIC DOMAIN NOTICE:

This converter package (MADIS2LITTLER) was developed at the National Center for Atmospheric Research (NCAR), which is 
operated by the University Corporation for Atmospheric Research (UCAR). NCAR and UCAR make no proprietary claims, either 
statutory or otherwise, to this version and release of MADIS2LITTLER, and consider MADIS2LITTLER to be in the public domain 
for use by any person or entity for any purpose without any fee or charge. UCAR requests that any user include this notice 
on any partial or full copies of MADIS2LITTLER. This converter is provided on an "AS IS" basis and any warranties, either 
express or implied, including but not limited to implied warranties of non-infringement, originality, merchantability and 
fitness for a particular purpose, are disclaimed. In no event shall UCAR be liable for any damages, whatsoever, whether 
direct, indirect, consequential or special, that arise out of or in connection with the access, use or performance of 
MADIS2LITTLER, including infringement actions. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Supported MADIS data
The following MADIS data can be converted to little_r format with this converter:
 
  - Surface datasets

     - ASOS METAR reports (NWS & FAA)
     - Non-ASOS METAR reports
     - SAO reports
     - Buoys, ships, C-MAN stations
     - Modernized NWS Cooperative Observer stations
     - Mesonets
    
   - Radiosondes

   - NOAA Profiler Network (NPN) hourly winds

   - Automated aircraft reports (including ACARS, AMDARS, TAMDARS)

   - Multi-Agency Profilers (MAP)

   - Satellite wind datasets

     - GOES satellites, operational winds
     - GOES satellites, experimental winds



2. Converter components

The following files are included in this package:

  README:                      The file that you are currently reading.
  namelist.madis:              The namelist file for the converter; here the user can specify various options for the 
                                   conversion from MADIS to LITTLE_R format 
  metar_provider:              Set up metar provider
  compile.ksh:                 Compiles the source code for the converter into an executable file (madis_to_little_r.exe)
  run_madis_to_little_r.ksh:   Sets the appropriate variables (as indicated by the user) and runs the converter package to
                                   convert the indicated MADIS observations to LITTLE_R format.
  madis_to_little_r.f90:       Source code to read MADIS data and write to LITTLE_R format.
  module_output.F:             Contains several necessary subroutines called by madis_to_little_r.f90.
  da_advance_time.f90          When compiled by compile.ksh, is used by madis_2_little_r.exe to manipulate time variables.

3. Required libraries

MADIS API and NETCDF libraries are required to compile this converter. If you do not have them, you can download them
from the following sites:

  - MADIS API: http://madis.noaa.gov/madis_api.html
  - NetCDF C/C++/Fortran Stable Releases: http://www.unidata.ucar.edu/downloads/netcdf/index.jsp


4. MADIS and LITTLE_R observation data structure

The MADIS API library assumes a specific directory structure where the MADIS observation files are kept. $MADIS_DATA is 
the MADIS observation root directory; it is defined in "run_madis_to_little_r.ksh". The necessary directory structure is as 
follows:

  METAR:              $MADIS_DATA/point/metar/netcdf/20070101_2300
  MESONET             $MADIS_DATA/LDAD/mesonet/netCDF/20070101_2300
  SAO:                $MADIS_DATA/point/sao/netcdf/20070101_2300
  COOP:               $MADIS_DATA/LDAD/coop/netCDF/20070101_2300
  MARINE:             $MADIS_DATA/point/maritime/netcdf/20070101_2300
  ACARS:              $MADIS_DATA/point/acars/netcdf/20070101_2300
  RadioSonde:         $MADIS_DATA/point/raob/netcdf/20070101_2300
  NPN:                $MADIS_DATA/point/profiler/netcdf/20070101_2300
  MAP:                $MADIS_DATA/LDAD/profiler/netCDF/20070101_2300
  SATWIND:            $MADIS_DATA/point/HDW/netcdf/20070101_2300
                      $MADIS_DATA/point/HDW1h/netcdf/20070101_2300

After running the converter, the resulting output in LITTLE_R format will be in the $MADIS2LITTLE_R_DIR root directory,
as defined in run_madis_to_little_r.ksh. This output will be in a different directory structure:

  METAR,SAO,MESO,COOP      $MADIS2LITTLE_R_DIR/2007010123/metar/METAR_LITTLE_R_2007-01-01_23
  MARINE:                  $MADIS2LITTLE_R_DIR/2007010123/marine/SHIP_LITTLE_R_2007-01-01_23
  ACARS:                   $MADIS2LITTLE_R_DIR/2007010123/acars/ACARS_LITTLE_R_2007-01-01_23
  RadioSonde:              $MADIS2LITTLE_R_DIR/2007010123/raob/RAOB_LITTLE_R_2007-01-01_23
  NPN:                     $MADIS2LITTLE_R_DIR/2007010123/npn/NPN_LITTLE_R_2007-01-01_23
  MAP:                     $MADIS2LITTLE_R_DIR/2007010123/map/MAP_LITTLE_R_2007-01-01_23
  SATWIND:                 $MADIS2LITTLE_R_DIR/2007010123/HDW/SATOB_LITTLE_R_2007-01-01_23


5. How to compile

A script (compile.ksh) is provided that will compile the converter executable (madis_to_little_r.exe) and a WRFDA 
executable needed for time manipulation (da_advance_time.exe). You must compile the required libraries (MADIS API and 
netCDF; see section 3 of this README for further details) prior to running the compile script. The required libraries MUST 
be compiled with the same Fortran compiler you plan to use for MADIS2LITTLER, otherwise you will likely encounter errors.

To compile, run the script with your fortran compiler name as a command line argument. Using gfortran as an example:

  ./compile.ksh gfortran

This will compile the necessary executables.

6. How to run

Before running, edit the values in the script file run_madis_to_little_r.ksh that correspond to your machine and dataset:

  - In the first section, make sure that MADIS API library and observation dataset are linked properly (MADIS_DATA and 
    MADIS_STATIC) 

  - The second section contains a list of variables as follows:
      export METAR=TRUE
      export MARINE=TRUE
      export GPSPW=TRUE
      export ACARS=TRUE
      export RAOB=TRUE
      export NPN=TRUE
      export MAP=TRUE
      export SATWND=TRUE

    These variables turn on (TRUE) or turn off (FALSE) the option to convert the corresponding MADIS dataset. Turn on the 
    appropriate variables only for those datasets which are available.

    In the SATWND=TRUE case, if both $MADIS_DATA/point/HDW/netcdf/YYYYMMDD_HHNN and 
    $MADIS_DATA/point/HDW1h/netcdf/YYYYMMDD_HHNN are present, both of them are read, and the output is written to 
    $MADIS2LITTLE_R_DIR/YYYYMMDDHH/HDW/SATOB_LITTLE_R_YYYY-MM-DD_HH
    If only one of them are available, that one dataset will be converted.

  - The third section contains time info. Set the start and end date, plus the interval (in hours) between files.

  - The fourth section contains more directory info. CODE_DIR is the directory where the code was compiled, and 
    MADIS2LITTLE_R_DIR should be set as described above in section 3.

You should also modify any desired values in the namelist file (namelist.madis):

  - madis_debug = 0 (default); currently this option has no effect.

  - "madis_*_minbck" and "madis_*_minfwd" set the time, in minutes, before and after the nominal time that observations 
    will be read and converted by MADIS2LITTLER. Note that the maximum acceptable value for the total span is 180 minutes.

  - The "madis_*_recwin" variables control how multiple reports from a single station will be converted from MADIS to 
    LITTLE_R files:
      0 - Converts all records in the given file (regardless of the indicated "minbck" and "minfwd" values)
      1 - Returns one record per fixed station, closest to nominal time
      2 - Returns one record per fixed station, closest to start of time window
      3 - Returns one record per fixed station, closest to end of time window
      4 - Returns all records within the given time window (indicated by the above "minbck" and "minfwd" values)

  - The "madis_*_db" variables indicate which database the MADIS observations have been retrieved from; acceptable values
    are "FSL", "AWIPS", and "AWIPS2". More information can be found here: http://madis.noaa.gov/acarsp_variable_list.html

  - If madis_metar_all_providers = .true., it converts all surface datasets (METAR, MESONET, SAO, COOP) to little_r format 
    and writes to $MADIS2LITTLE_R_DIR/YYYYMMDDHH/metar/METAR_LITTLE_R_YYYY-MM-DD_HH (YYYY=year, MM=month, DD=day, HH=hour)

  - If madis_metar_all_providers = .false., the file "metar_provider" is used to select specific surface datasets to 
    be converted:
      ALL-MTR:  read $MADIS_DATA/point/metar/netcdf/YYYYMMDD_HHNN  
                write to $MADIS2LITTLE_R_DIR/YYYYMMDDHH/metar/METAR_LITTLE_R_YYYY-MM-DD_HH
      ALL-MESO: read $MADIS_DATA/LDAD/mesonet/netCDF/YYYYMMDD_HHNN 
                write to $MADIS2LITTLE_R_DIR/YYYYMMDDHH/metar/METAR_LITTLE_R_YYYY-MM-DD_HH
      COOP:     read $MADIS_DATA//LDAD/coop/netCDF/YYYYMMDD_HHNN   
                write to $MADIS2LITTLE_R_DIR/YYYYMMDDHH/metar/METAR_LITTLE_R_YYYY-MM-DD_HH
      SAO:      read $MADIS_DATA/point/sao/netcdf/YYYYMMDD_HHNN    
                write to $MADIS2LITTLE_R_DIR/YYYYMMDDHH/metar/METAR_LITTLE_R_YYYY-MM-DD_HH

  - For the "SATWND=TRUE" case, only use the variables madis_geoamv_wv and madis_geoamv_ir; in this version of the converter 
    the variables madis_geoamv_vis, madis_geoamv_s10, madis_geoamv_s11 do not return any converted data.

When you are ready, run the run_madis_to_little_r.ksh script to convert the requested MADIS observations to LITTLE_R format


7. Tested systems

1) Platform:     Linux x86_64
   Compilers:    ifort, pgf90, gfortran

Old versions of the tested compilers may not work, as this converter uses some components of the Fortran 2003 standard.

Other platforms/compilers may work, but have not been tested.

