Compute a New Diagnostic Variable


This exercise demonstrates adding a new diagnostic variable (maximum 10 m winds) to the WRF code by modifying wrf/dyn_em/solve_em.F. You may use this as a template for future additions.




Add Subroutine “USE” Information

First, make a copy of solve_em.F - if you make mistakes, the original file is still available.


cd /glade/derecho/scratch/$USER/practice_exercises/wrf/dyn_em

cp solve_em.F solve_em.F.save


  1. Open solve_em.F and look for the following code (around line 20).

    Note

    If possible, widen your window to view text without it wrapping, which is easier to read - alternatively if using “vi” issue :set nowrap once inside the file.



    #ifdef DM_PARALLEL
   USE module_dm, ONLY : &
         local_communicator, mytask, ntasks, ntasks_x, ntasks_y &
         ,local_communicator_periodic, wrf_dm_maxval
   USE module_comm_dm, ONLY : &
         halo_em_a_sub,halo_em_b_sub,halo_em_c2_sub,halo_em_chem_e_3_sub &
        ,halo_em_chem_e_5_sub,halo_em_chem_e_7_sub,halo_em_chem_old_e_5_sub &
        ,halo_em_chem_old_e_7_sub,halo_em_c_sub,halo_em_d2_3_sub &
        ,halo_em_d2_5_sub,halo_em_d3_3_sub,halo_em_d3_5_sub,halo_em_d_sub &
        ,halo_em_e_3_sub,halo_em_e_5_sub,halo_em_hydro_uv_sub &
        ,halo_em_moist_e_3_sub,halo_em_moist_e_5_sub,halo_em_moist_e_7_sub &
        ,halo_em_moist_old_e_5_sub,halo_em_moist_old_e_7_sub &
        ,halo_em_scalar_e_3_sub,halo_em_scalar_e_5_sub,halo_em_scalar_e_7_sub &
        ,halo_em_scalar_old_e_5_sub,halo_em_scalar_old_e_7_sub,halo_em_tke_3_sub &
        ,halo_em_tke_5_sub,halo_em_tke_7_sub,halo_em_tke_advect_3_sub &
        ,halo_em_tke_advect_5_sub,halo_em_tke_old_e_5_sub &
        ,halo_em_tke_old_e_7_sub,halo_em_tracer_e_3_sub,halo_em_tracer_e_5_sub &
        ,halo_em_tracer_e_7_sub,halo_em_tracer_old_e_5_sub &
        ,halo_em_tracer_old_e_7_sub,period_bdy_em_a_sub &
        ,period_bdy_em_b3_sub,period_bdy_em_b_sub,period_bdy_em_chem2_sub &
        ,period_bdy_em_chem_old_sub,period_bdy_em_chem_sub,period_bdy_em_d3_sub &
        ,period_bdy_em_d_sub,period_bdy_em_e_sub,period_bdy_em_moist2_sub &
        ,period_bdy_em_moist_old_sub,period_bdy_em_moist_sub &
        ,period_bdy_em_scalar2_sub,period_bdy_em_scalar_old_sub &
        ,period_bdy_em_scalar_sub,period_bdy_em_tke_old_sub &
        ,period_bdy_em_tracer2_sub,period_bdy_em_tracer_old_sub &
        ,period_bdy_em_tracer_sub,period_em_da_sub,period_em_hydro_uv_sub &
        ,halo_em_f_sub,halo_em_init_4_sub
#endif



  2. Add the highlighted line below. Everything should line up as is shown in the example.

    Important

    Don’t forget to add an “&” to the end of the line above!



         #ifdef DM_PARALLEL
        USE module_dm, ONLY : &
              local_communicator, mytask, ntasks, ntasks_x, ntasks_y &
              ,local_communicator_periodic, wrf_dm_maxval &
              ,wrf_dm_sum_real, wrf_dm_max_real, wrf_dm_maxval_real
        USE module_comm_dm, ONLY : &
              halo_em_a_sub,halo_em_b_sub,halo_em_c2_sub,halo_em_chem_e_3_sub &
             ,halo_em_chem_e_5_sub,halo_em_chem_e_7_sub,halo_em_chem_old_e_5_sub &
             ,halo_em_chem_old_e_7_sub,halo_em_c_sub,halo_em_d2_3_sub &
             ,halo_em_d2_5_sub,halo_em_d3_3_sub,halo_em_d3_5_sub,halo_em_d_sub &
             ,halo_em_e_3_sub,halo_em_e_5_sub,halo_em_hydro_uv_sub &
             ,halo_em_moist_e_3_sub,halo_em_moist_e_5_sub,halo_em_moist_e_7_sub &
             ,halo_em_moist_old_e_5_sub,halo_em_moist_old_e_7_sub &
             ,halo_em_scalar_e_3_sub,halo_em_scalar_e_5_sub,halo_em_scalar_e_7_sub &
             ,halo_em_scalar_old_e_5_sub,halo_em_scalar_old_e_7_sub,halo_em_tke_3_sub &
             ,halo_em_tke_5_sub,halo_em_tke_7_sub,halo_em_tke_advect_3_sub &
             ,halo_em_tke_advect_5_sub,halo_em_tke_old_e_5_sub &
             ,halo_em_tke_old_e_7_sub,halo_em_tracer_e_3_sub,halo_em_tracer_e_5_sub &
             ,halo_em_tracer_e_7_sub,halo_em_tracer_old_e_5_sub &
             ,halo_em_tracer_old_e_7_sub,period_bdy_em_a_sub &
             ,period_bdy_em_b3_sub,period_bdy_em_b_sub,period_bdy_em_chem2_sub &
             ,period_bdy_em_chem_old_sub,period_bdy_em_chem_sub,period_bdy_em_d3_sub &
             ,period_bdy_em_d_sub,period_bdy_em_e_sub,period_bdy_em_moist2_sub &
             ,period_bdy_em_moist_old_sub,period_bdy_em_moist_sub &
             ,period_bdy_em_scalar2_sub,period_bdy_em_scalar_old_sub &
             ,period_bdy_em_scalar_sub,period_bdy_em_tke_old_sub &
             ,period_bdy_em_tracer2_sub,period_bdy_em_tracer_old_sub &
             ,period_bdy_em_tracer_sub,period_em_da_sub,period_em_hydro_uv_sub &
             ,halo_em_f_sub,halo_em_init_4_sub
     #endif




Declare New Variables

Fortran requires the new output variables to be “declared.”


  1. Find the following code (around new line 144):


    logical, save :: firstime = .true.,   &  ! logical variable indicating first time
       feedback_is_ready,   &  ! logical variable indicating feedback process can proceed
       feedback_restart,    &  ! logical variable indicating feedback information is available
       direct_sw_feedback      ! logical variable indicating direct aerosol sw feedback is on or not
! end WRF-CMAQ twoway coupled model block


  2. Add the following code just below that section. Line everything up as is shown in the example.


         logical, save :: firstime = .true.,   &  ! logical variable indicating first time
            feedback_is_ready,   &  ! logical variable indicating feedback process can proceed
            feedback_restart,    &  ! logical variable indicating feedback information is available
            direct_sw_feedback      ! logical variable indicating direct aerosol sw feedback is on or not
     ! end WRF-CMAQ twoway coupled model block

     REAL :: sum_ws, max_ws, glat, glon, wind_vel
     LOGICAL, EXTERNAL :: wrf_dm_on_monitor
     CHARACTER*256 :: outstring




Add the Computations

  1. Find the following code (around new line 4906):


    !  Are we about to read the lateral boundary file?  This is a domain one action only.
   IF ( grid%id .EQ. 1 ) grid%just_read_boundary = Is_alarm_tstep(grid%domain_clock, grid%alarms(BOUNDARY_ALARM))


  2. Just below those lines, add the following code (copy/paste this into the code, instead of typing it all out). Line everything up as is shown in the example.


         !  Are we about to read the lateral boundary file?  This is a domain one action only.
        IF ( grid%id .EQ. 1 ) grid%just_read_boundary = Is_alarm_tstep(grid%domain_clock, grid%alarms(BOUNDARY_ALARM))

     ! Compute local maximum and sum of 10m wind-speed
     sum_ws = 0.
     max_ws = 0.
     DO j = jps, jpe
       DO i = ips, ipe
         wind_vel = sqrt( grid%u10(i,j)*grid%u10(i,j) + grid%v10(i,j)*grid%v10(i,j))
         IF ( wind_vel .GT. max_ws ) THEN
           max_ws = wind_vel
           idex = i
           jdex = j
         ENDIF
         sum_ws = sum_ws + wind_vel
      ENDDO
     ENDDO

     ! Compute global sum
     sum_ws = wrf_dm_sum_real ( sum_ws )

     ! Compute global maximum and associated i,j point
     CALL wrf_dm_maxval_real ( max_ws, idex, jdex )

     ! Determine if i,j point of maximum is on this process
     ! and if so, set the lat and lon of that point, otherwise
     ! set to an absolute minimum
     IF ( ips .LE. idex .AND. idex .LE. ipe .AND. &
           jps .LE. jdex .AND. jdex .LE. jpe ) THEN
       glat = grid%xlat(idex,jdex)
       glon = grid%xlong(idex,jdex)
     ELSE
       glat = -99999.
       glon = -99999.
     ENDIF

     ! Compute global maximum to find glat and glon
     glat = wrf_dm_max_real ( glat )
     glon = wrf_dm_max_real ( glon )

     ! Print out the result on the monitor process
     IF ( wrf_dm_on_monitor() ) THEN
       WRITE(outstring,*)'Avg. ',sum_ws/((ide-ids+1)*(jde-jds+1))
       CALL wrf_message ( TRIM(outstring) )
       WRITE(outstring,*)'Max. ',max_ws,' Lat. ',glat,' Lon. ',glon
       CALL wrf_message ( TRIM(outstring) )
     ENDIF


  3. Save and then exit the file.





Compile the Modified Code

Because no modifications were made to any “Registry” file (those in wrf/Registry), the code doesn’t need to be cleaned or reconfigured before recompiling. You only need to recompile the routines needed for the change, so you can simply recompile the code, which is much faster.


  1. Move to the /glade/derecho/scratch/$USER/practice_exercises/wrf directory.


  2. Compile the code.


    Note

    When compiling on Derecho specifically, to avoid saturating the login nodes with compilation processes and negatively impacting other users, the compilation command must run in an interactive session, which launches the specified command on a batch node and returns when the command has finished.


    • Issue the following command to enter an interactive session (where UMMM0013 is the project code for this tutorial class). This may take a minute or two to complete.

      qinteractive -A UMMM0013

    • To set the interactive environment correctly, source the .bashrc script that resides in your home directory:

      source /glade/u/home/$user/.bashrc

    • Issue the following command to compile WRF:

      ./compile em_real -j 6 >& compile.log


  3. Check whether the compile was successful:


    tail log.compile

    If successful, you should see something like:


    ---> Executables successfully built <---

-rwxr-xr-x 1 class103 cbet 37955312 July 20 14:22 main/ndown.exe
-rwxr-xr-x 1 class103 cbet 37828240 July 20 14:22 main/real.exe
-rwxr-xr-x 1 class103 cbet 37472288 July 20 14:22 main/tc.exe
-rwxr-xr-x 1 class103 cbet 41615480 July 20 14:22 main/wrf.exe

==========================================================================




Run WRF Using the Modified Code

Run real and wrf, per usual. Look in the rsl.out* files to see extra print-outs similar to the following:


Avg. 5.881783
Max. 14.73283    Lat. 29.00684    Lon. 126.9531





Return to the Practice Exercise home to page to run another exercise.