In this exercise, you will practice adding a new diagnostic variable to the WRF code, so that it can be computed. This will give you an idea of the process of modifying code for specific needs.

**Click on the links to jump to the desired section.**

Add Subroutine "USE" Info.
Declare New Variables
Add Computations
Compile and Run Modified Code

Add Subroutine "USE" Information

You are going to make modifications to the subroutine solve_em.F to allow the model to compute "maximum 10m winds."

1. First, it's best to make a copy of solve_em.F (name it something else) so that if you mess up the modifications, you will still have the original file to go back to.

   cd /glade/derecho/scratch/$USER/practice_exercises/wrf/dyn_em
   cp solve_em.F solve_em.F.save



2. Open the solve_em.F file and look for the following code (around line 20). NOTE that if you're able to widen your window, you can see the text without it wrapping, which will make it easier to read.





#ifdef DM_PARALLEL    USE module_dm, ONLY : &          local_communicator, mytask, ntasks, ntasks_x, ntasks_y &         ,local_communicator_periodic, wrf_dm_maxval    USE module_comm_dm, ONLY : &         halo_em_a_sub,halo_em_b_sub,halo_em_c2_sub,halo_em_chem_e_3_sub &         ,halo_em_chem_e_5_sub,halo_em_chem_e_7_sub,halo_em_chem_old_e_5_sub &         ,halo_em_chem_old_e_7_sub,halo_em_c_sub,halo_em_d2_3_sub &         ,halo_em_d2_5_sub,halo_em_d3_3_sub,halo_em_d3_5_sub,halo_em_d_sub &         ,halo_em_e_3_sub,halo_em_e_5_sub,halo_em_hydro_uv_sub &         ,halo_em_moist_e_3_sub,halo_em_moist_e_5_sub,halo_em_moist_e_7_sub &         ,halo_em_moist_old_e_5_sub,halo_em_moist_old_e_7_sub &         ,halo_em_scalar_e_3_sub,halo_em_scalar_e_5_sub,halo_em_scalar_e_7_sub &         ,halo_em_scalar_old_e_5_sub,halo_em_scalar_old_e_7_sub,halo_em_tke_3_sub &         ,halo_em_tke_5_sub,halo_em_tke_7_sub,halo_em_tke_advect_3_sub &         ,halo_em_tke_advect_5_sub,halo_em_tke_old_e_5_sub &         ,halo_em_tke_old_e_7_sub,halo_em_tracer_e_3_sub,halo_em_tracer_e_5_sub &         ,halo_em_tracer_e_7_sub,halo_em_tracer_old_e_5_sub &         ,halo_em_tracer_old_e_7_sub,period_bdy_em_a_sub &         ,period_bdy_em_b3_sub,period_bdy_em_b_sub,period_bdy_em_chem2_sub &         ,period_bdy_em_chem_old_sub,period_bdy_em_chem_sub,period_bdy_em_d3_sub &         ,period_bdy_em_d_sub,period_bdy_em_e_sub,period_bdy_em_moist2_sub &         ,period_bdy_em_moist_old_sub,period_bdy_em_moist_sub &         ,period_bdy_em_scalar2_sub,period_bdy_em_scalar_old_sub &         ,period_bdy_em_scalar_sub,period_bdy_em_tke_old_sub &         ,period_bdy_em_tracer2_sub,period_bdy_em_tracer_old_sub &         ,period_bdy_em_tracer_sub,period_em_da_sub,period_em_hydro_uv_sub &         ,halo_em_f_sub,halo_em_init_4_sub #endif




3. Add the highlighted bits below (make sure everything lines up with the existing code):




#ifdef DM_PARALLEL    USE module_dm, ONLY : &          local_communicator, mytask, ntasks, ntasks_x, ntasks_y &         ,local_communicator_periodic, wrf_dm_maxval, wrf_dm_sum_real &         ,wrf_dm_max_real, wrf_dm_maxval_real    USE module_comm_dm, ONLY : &         halo_em_a_sub,halo_em_b_sub,halo_em_c2_sub,halo_em_chem_e_3_sub &         ,halo_em_chem_e_5_sub,halo_em_chem_e_7_sub,halo_em_chem_old_e_5_sub &         ,halo_em_chem_old_e_7_sub,halo_em_c_sub,halo_em_d2_3_sub &         ,halo_em_d2_5_sub,halo_em_d3_3_sub,halo_em_d3_5_sub,halo_em_d_sub &         ,halo_em_e_3_sub,halo_em_e_5_sub,halo_em_hydro_uv_sub &         ,halo_em_moist_e_3_sub,halo_em_moist_e_5_sub,halo_em_moist_e_7_sub &         ,halo_em_moist_old_e_5_sub,halo_em_moist_old_e_7_sub &         ,halo_em_scalar_e_3_sub,halo_em_scalar_e_5_sub,halo_em_scalar_e_7_sub &         ,halo_em_scalar_old_e_5_sub,halo_em_scalar_old_e_7_sub,halo_em_tke_3_sub &         ,halo_em_tke_5_sub,halo_em_tke_7_sub,halo_em_tke_advect_3_sub &         ,halo_em_tke_advect_5_sub,halo_em_tke_old_e_5_sub &         ,halo_em_tke_old_e_7_sub,halo_em_tracer_e_3_sub,halo_em_tracer_e_5_sub &         ,halo_em_tracer_e_7_sub,halo_em_tracer_old_e_5_sub &         ,halo_em_tracer_old_e_7_sub,period_bdy_em_a_sub &         ,period_bdy_em_b3_sub,period_bdy_em_b_sub,period_bdy_em_chem2_sub &         ,period_bdy_em_chem_old_sub,period_bdy_em_chem_sub,period_bdy_em_d3_sub &         ,period_bdy_em_d_sub,period_bdy_em_e_sub,period_bdy_em_moist2_sub &         ,period_bdy_em_moist_old_sub,period_bdy_em_moist_sub &         ,period_bdy_em_scalar2_sub,period_bdy_em_scalar_old_sub &         ,period_bdy_em_scalar_sub,period_bdy_em_tke_old_sub &         ,period_bdy_em_tracer2_sub,period_bdy_em_tracer_old_sub &         ,period_bdy_em_tracer_sub,period_em_da_sub,period_em_hydro_uv_sub &         ,halo_em_f_sub,halo_em_init_4_sub #endif

Go back to top

Declare New Variables

Now you need to add declarations for the new variables you are wanting to output.

1. Look for the following code (it will be around new line 180):



! Changes in tendency at this timestep real ,DIMENSION(grid%sm31:grid%em31,grid%sm32:grid%em32,grid%sm33:grid%em33) :: h_tendency, & z_tendency



2. Just above these lines, add the following code:




REAL :: sum_ws, max_ws, glat, glon, wind_vel LOGICAL, EXTERNAL :: wrf_dm_on_monitor CHARACTER*256 :: outstring

Go back to top

Add Computations

1. Look for the following code (it will be around new line 4930):




! Finish timers if compiled with -DBENCH. #include "bench_solve_em_end.h"





2. Just above those lines, add the following code (it is advised to copy/paste this into the code, instead of typing it all out):



! Compute local maximum and sum of 10m wind-speed sum_ws = 0. max_ws = 0. DO j = jps, jpe   DO i = ips, ipe     wind_vel = sqrt( grid%u10(i,j)*grid%u10(i,j) + grid%v10(i,j)*grid%v10(i,j))     IF ( wind_vel .GT. max_ws ) THEN       max_ws = wind_vel       idex = i       jdex = j     ENDIF     sum_ws = sum_ws + wind_vel  ENDDO ENDDO   ! Compute global sum sum_ws = wrf_dm_sum_real ( sum_ws )   ! Compute global maximum and associated i,j point CALL wrf_dm_maxval_real ( max_ws, idex, jdex )   ! Determine if i,j point of maximum is on this process ! and if so, set the lat and lon of that point, otherwise ! set to an absolute minimum IF ( ips .LE. idex .AND. idex .LE. ipe .AND. &       jps .LE. jdex .AND. jdex .LE. jpe ) THEN   glat = grid%xlat(idex,jdex)   glon = grid%xlong(idex,jdex) ELSE   glat = -99999.   glon = -99999. ENDIF   ! Compute global maximum to find glat and glon glat = wrf_dm_max_real ( glat ) glon = wrf_dm_max_real ( glon )   ! Print out the result on the monitor process IF ( wrf_dm_on_monitor() ) THEN   WRITE(outstring,*)'Avg. ',sum_ws/((ide-ids+1)*(jde-jds+1))   CALL wrf_message ( TRIM(outstring) )   WRITE(outstring,*)'Max. ',max_ws,' Lat. ',glat,' Lon. ',glon   CALL wrf_message ( TRIM(outstring) ) ENDIF





3. Now save and exit the file.

Go back to top

Compile and run the modified code

Because you have not made any modifications to the Registry file, it is not be necessary to clean the code and reconfigure before recompiling. You only need to recompile the few routines needed for the change, so you can simply recompile the code and it will be much faster than it would be if you had to start fresh.

1. Make sure you are back in the /glade/derecho/scratch/$USER/practice_exercises/wrf directory.


2. Now compile the code.
   
   
   

   IMPORTANT NOTE!! When compiling on Derecho specifically, to avoid saturating the Derecho login nodes with compilation processes and negatively impacting other users, we will run our compilation command on a batch node with the special qcmd command, which launches the specified command on a batch node and returns when the command has finished. So, although you would normally not use "qcmd" when compiling WRF on other systems, for this tutorial we will prefix the usual compilation command with qcmd -A UMMM0007 -- (where UMMM0007 is the computing project we're working under).

   
   

   qcmd -A UMMM0007 -- ./compile em_real -j 6 >& log.compile



3. Issue the following command to determine whether the compilation was successful:
   

   tail log.compile

   If successful, you should see something like:

   ---> Executables successfully built <---
   
   -rwxr-xr-x 1 class103 cbet 37955312 July 20 14:22 main/ndown.exe
   -rwxr-xr-x 1 class103 cbet 37828240 July 20 14:22 main/real.exe
   -rwxr-xr-x 1 class103 cbet 37472288 July 20 14:22 main/tc.exe
   -rwxr-xr-x 1 class103 cbet 41615480 July 20 14:22 main/wrf.exe
   
   ==========================================================================

    
    
4. Run the model as before

   Look in the rsl.out* files, you will see extra print-outs that look like this:

   Avg. 5.881783
   Max. 14.73283    Lat. 29.00684    Lon. 126.9531