page header
 
Hybrid data assimilation

Hybrid data assimilation is a combination of variational and ensemble methods of data assimilation. This lesson has exercises for 3DENVAR.

This case utilizes a domain over the eastern United States and western atlantic ocean, 111x111 gridpoints, 41 vertical levels, and 24km resolution (it can be found at /home/ec2-user/test_data/hybrid). The exact area of the domain is shown at right.

It features a major blizzard which produced a record amount of snowfall in some cities from January 22-24, 2016.

 Domain of this test case

Reference: Download the tutorial presentation

3DENVAR

Source code

Get the pre-compiled code, if you have not done so.

WRFDA/var/build/gen_be_ensmean.exe
WRFDA/var/build/gen_be_ep2.exe
WRFDA/var/build/gen_be_vertloc.exe
WRFDA/var/build/da_wrfvar.exe
are the four executables that will be used in this session.

Choice of your working directory

We recommend running each session in a separate directory, so that it will be easier to check for the necessary input files and look for what output files are created after a successful run.

mkdir /home/ec2-user/workdir/hybrid
cd /home/ec2-user/workdir/hybrid

Input data

The procedure for hybrid 3DENVAR assimilation is much the same as Running WRFDA for 3DVAR, except for some more input files and namelist.input settings.

In addition to the basic input files (LANDUSE.TBL, ob.ascii, be.dat) that you should have been familiar with by now, an ensemble mean (which will be the fg for the hybrid application) and ensemble perturbations are the extra required input files.

  • ensemble mean

    Usually, the first step of hybrid data assimilation is to prepare a set of ensembles. However, this step is time-consuming, so a set of 10 ensemble forecasts is provided under the /home/ec2-user/test_data/hybrid/fc/2016012300 directory.

    ls -al /home/ec2-user/test_data/hybrid/fc/2016012300/
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e001
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e002
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e003
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e004
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e005
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e006
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e007
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e008
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e009
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:52 wrfout_d01_2016-01-23_00:00:00.e010
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:55 wrfout_d01_2016-01-23_00:00:00.mean
    -rw-r--r-- 1 19974 instructor 41525452 Jul 21 16:55 wrfout_d01_2016-01-23_00:00:00.vari

    wrfout_d01_2016-01-23_00:00:00.e* are a set of 12-hour WRF forecasts (valid at 2016012300, initialized at 2016012212).

    wrfout_d01_2016-01-23_00:00:00.mean and wrfout_d01_2016-01-23_00:00:00.vari are two template files that will be overwritten by a program that calculates ensemble mean from ensemble forecasts.

    • Copy ensemble forecasts and template files to your working directory.

    cp -r /home/ec2-user/test_data/hybrid/fc/2016012300/ .

    • Edit gen_be_ensmean_nl.nl (or copy it from /home/ec2-user/test_data/hybrid/gen_be_ensmean_nl.nl)

    vi gen_be_ensmean_nl.nl

    &gen_be_ensmean_nl
    directory = './2016012300'
    filename = 'wrfout_d01_2016-01-23_00:00:00'
    num_members = 10
    nv = 7
    cv = 'U', 'V', 'W', 'PH', 'T', 'MU', 'QVAPOR'
    /

    • Run gen_be_ensmean.exe

    /home/ec2-user/compiled_code/WRFDA-v4.1.2/var/build/gen_be_ensmean.exe

    • Check output

    2016012300/wrfout_d01_2016-01-23_00:00:00.mean is the ensemble mean

    2016012300/wrfout_d01_2016-01-23_00:00:00.vari is the ensemble variance. This file is for diagnostic purposes only, and will not be used in assimilation.

You may use the NCL script /home/ec2-user/WRFDA_TOOLS/graphics/ncl/hybrid/fig2.ncl to make some plots.

This script displays the mean and standard deviation for the ensemble at a single level for a single variable.

cp /home/ec2-user/WRFDA_TOOLS/graphics/ncl/hybrid/fig2.ncl .
cp /home/ec2-user/WRFDA_TOOLS/graphics/ncl/WRF_contributed.ncl.test .

Edit fig2.ncl to set the proper date and path path
load "WRF_contributed.ncl.test"

dir = "/home/ec2-user/workdir/hybrid/" + date + "/"

filename1 = file_type + "_" + year + "-" + month + "-" + day + "_" + hour + ":00:00.mean"
...
d1 = sqrt(f2)

ncl fig2.ncl
display fig2.pdf

You can change the level and variable by changing these lines a bit further down in the script:

var = "U"
kl = 24

U, V, W, P, and QVAPOR are some other interesting fields to look at. The plot at right uses var = "U" and kl = 32.

 Output from fig2.ncl
  • ensemble perturbations
    • This step will extract the ensemble perturbations from the existing ensemble wrfout files provided for you. Create another sub-directory where you will be working for this step.

    cd /home/ec2-user/workdir/hybrid

    mkdir -p 2016012300/ep

    cd 2016012300/ep

  • Run gen_be_ep2.exe

gen_be_ep2.exe requires 4 command arguments (DATE, NUM_MEMBER, DIRECTORY, FILENAME) as defined below:

/home/ec2-user/compiled_code/WRFDA-v4.1.2/var/build/gen_be_ep2.exe 2016012300 10 ../../2016012300 wrfout_d01_2016-01-23_00:00:00

  • Check the output files generated by gen_be_ep2.exe

A list of binary files will be created under 2016012300/ep directory. Among them, tmp.e* are temporary scratch files that can be removed.

You may use the NCL script /home/ec2-user/WRFDA_TOOLS/graphics/ncl/hybrid/fig1.ncl to make some plots.

cp /home/ec2-user/WRFDA_TOOLS/graphics/ncl/hybrid/fig1.ncl .
cp /home/ec2-user/WRFDA_TOOLS/graphics/ncl/WRF_contributed.ncl.test .

Edit fig1.ncl to set the proper date and path.

load "WRF_contributed.ncl.test"

directory = "/home/ec2-user/workdir/hybrid/" + date

filename = file_type + "_" + year + "-" + month + "-" + day + "_" + hour + ":00:00"

file0 = addfile(dir + filename+ ".mean", "r")

file1 = addfile(dir + filename+ ".e001", "r")

...

file10= addfile(dir+ filename+ ".e010", "r")

ncl fig1.ncl
display fig1.pdf

Again, you can modify the script to look at different vertical levels and variables by changing these settings:

var = "U"
kl = 32

Depending on what variable you display, you may have to change the color contour settings "cont_min", "cont_max", and "cont_spacing" as well to get a nice-looking plot.

Run gen_be_vertloc.exe

gen_be_vertloc.exe generates the input file for vertical localization. This program requires one command-line argument: the number of vertical levels of the model configuration (same value as e_vert in the namelist; for the tutorial example, this should be 41

cd /home/ec2-user/workdir/hybrid

/home/ec2-user/compiled_code/WRFDA-v4.1.2/var/build/gen_be_vertloc.exe 41

This generates a file be.vertloc.dat which will be used by WRFDA.

Run WRFDA in hybrid mode

cd /home/ec2-user/workdir/hybrid

ln -fs 2016012300/ep ./ep # ensemble perturbation files should be under the ep subdirectory
ln -fs 2016012300/wrfout_d01_2016-01-23_00:00:00.mean ./fg # first guess is an ensemble forecast mean
ln -fs /home/ec2-user/compiled_code/WRFDA-v4.1.2/run/LANDUSE.TBL .
ln -sf /home/ec2-user/test_data/hybrid/ob/2016012300/ob.ascii ob.ascii
# we created the observation file during the OBSPROC exercise, if you did not finish you can find a copy here: /home/ec2-user/test_data/hybrid/ob/2016012300/ob.ascii
ln -fs /home/ec2-user/test_data/hybrid/be/be.dat ./be.dat# 3DENVAR uses a combination of static and ensemble-based background error. The static portion, contained in be.dat, is the same kind used in other assimilation types we have already demonstrated.

Edit namelist.input (a sample file is provided as /home/ec2-user/test_data/hybrid/namelist.input)

cp /home/ec2-user/test_data/hybrid/namelist.input .

vi namelist.input

Pay special attention to the following hybrid-related settings:

&wrfvar7
je_factor = 2.0
/
&wrfvar16
ensdim_alpha = 10 # hybrid mode is activated when ensdim_alpha is larger than zero.
alphacv_method = 2
alpha_corr_scale = 500.0
/

mpirun -np 8 /home/ec2-user/compiled_code/WRFDA-v4.1.2/var/build/da_wrfvar.exe

View the different output files to see how they differ from a 3DVAR assimilation run. For example, in cost_fn and grad_fn, you will see the ensemble terms of the cost function and gradient, respectively.

Other practice

Do a pseudo-single obs test (PSOT) like we learned in a previous exercise. You can run this test with the 3DENVAR case. Edit the following namelist options to run the test:

&wrfvar15
num_pseudo = 1,
pseudo_x = 50.0,
pseudo_y = 50.0,
pseudo_z = 10.0,
pseudo_err = 1.0,
pseudo_val = 1.0,
/
. . .
&wrfvar19
pseudo_var = 'u'
/

mpirun -np 1 /home/ec2-user/compiled_code/WRFDA-v4.1.2/var/build/da_wrfvar.exe

To the right you can see what the lowest level pressure perturbation should look like. Notice the differences from the basic 3dvar case due to the BE contribution from the ensemble; instead of a featureless blob, the increment from a single observation can be quite complex!

 Pressure perturbation from PSOT with the above settings.

 

Try different settings like the ones below, then run a PSOT to see how they influence the background error statistics.

Ensemble covariance weighting factor (je): try
je_factor = 10.0 (jb = je_factor/(je_factor - 1), so jb = 1.11 in this example)
je_factor = 1.25 (jb = 5)
je_factor = 1.1 (jb = 11)

Hybrid covariance localization scale (alpha_corr_scale): try
alpha_corr_scale = 200.0
alpha_corr_scale = 1500.0

Try setting ensdim_alpha = 0, and compare the results.

You have now completed the WRFDA hybrid assimilation tutorial. You can now move on to the next exercise.

 
back to top
 
practice_page_header