|
Using WRF and WPS on NSF NCAR's Derecho HPC
Page Contents
Logging-in to Derecho
Important Notes
Compiling WRF with an Intel Compiler
Compiling WRF with a GNU Compiler
Compiling WPS with an Intel Compiler
Compiling WPS with a GNU Compiler
Running WRF on Derecho
Other Notes
Logging-in to Derecho
- Issue the following command in your terminal window environment.
ssh -X username@derecho.hpc.ucar.edu
Note that when using a Mac, replace the above "X" with a "Y."
- You will be asked to enter your "ncar-two-factor" password.
- Once your password has been entered, a notification should be sent to the DUO app on your phone. Click "Approve."
Important Notes
- It is not recommend that you implement your own .login, .cshrc, .bashrc, etc. files until you are very familiar with the Derecho system. Derecho provides several modules that allow you to change some of your settings. See CISL's Derecho Supercomputer page for additional details.
- If you do not plan to make modifications to the WRF and/or WPS code, versions 4.0+ are precompiled and available from ~wrfhelp/derecho_pre_compiled_code directory. The precompiled WRF is built with an "INTEL (ftn/icc): Cray XC" compiler (option 2 below) and is configured for distributed-memory (dmpar) processing and for a basic nesting option (option #1). WPS is compiled for use of both GRIB1 and GRIB2 input data.
- To compile your own code on Derecho, see the information below that discusses the options for compiling with either an Intel or GNU compiler, and then follow instructions in the Compiling chapter of the WRF Users Guide.
- Static input data for running geogrid.exe are available in ~wrfhelp/WPS_GEOG. This path should be specified in namelist.wps.
Compiling WRF with an Intel Compiler
- Using intel-classic [INTEL (ifort/icc) ]: options 13-16 (Note this compiler may not be updated in the future)
To use this option, the intel-classic module must be loaded. Issue the command:
module load intel-classic
- This compiler is the closest to using Intel on Cheyenne, but not identical, meaning it may not be possible to obtain the exact same results as were produced on Cheyenne.
- This version of Intel should work for all versions of WRF.
- Compilation time is about 37 minutes (dmpar), which is faster than on Cheyenne.
- Using the Cray wrapped ifort compiler [INTEL (ftn/icc): Cray XC]: options 48-51
To use this compiler, the following module must be loaded:
module load intel-classic
- This ifort compiler works for all versions of WRF.
- Compilation time is about 37 minutes (dmpar).
- Using Intel oneAPI ifx/icx [INTEL (ifx/icx) : oneAPI LLVM]: options 76-79 (NOTE: This option is only available for WRF v4.5.2+. WRF-Chem and WRFDA may require additional changes to compile.)
To use this option, the following module must be loaded:
module load intel-oneapi
- To compile with this option, either the queue or the following command must be used:
qcmd -A Project Number -- ./compile em_test-case -j 2 >& compile.log &
- Compilation time is about 15 minutes (dmpar).
Compiling WRF with a GNU Compiler: [GNU (gfortran/gcc)]: options 32-35
Upon logging-in to Derecho, the default compiler is Intel. To compile with a GNU compiler, the following module should be loaded.
module load gcc
After this, WRF can be compiled following instructions in the Compiling chapter of the WRF Users Guide.
Compiling WPS with an Intel Compiler: [Linux x86_64, Intel compiler]: options 17-20
To compile WPS with an Intel compiler, WRF must have been successfully built with the same Intel compiler. Prior to configuring WPS, the following must be set:
export WRF_DIR=path-to-installed-wrf
After this, WPS can be compiled following instructions in the Compiling chapter of the WRF Users Guide.
Compiling WPS with a GNU Compiler: [Linux x86_64, gfortran]: options 1-4
To compile WPS with a GNU compiler, WRF must have been successfully built with the same GNU compiler. Prior to configuring WPS, the following must be set:
module load gcc
export WRF_DIR=path-to-installed-wrf
After this, WPS can be compiled following instructions in the Compiling chapter of the WRF Users Guide.
Running WRF on Derecho
If WRF code is built for distributed-memory (dmpar) or shared-memory (smpar) processing, the job must be submitted to a queue, using a batch script. Example batch scripts are available in the ~wrfhelp/BATCH_SCRIPTS directory. To use these, copy them to your running directory and then make modifications specific to your simulation. Make sure to set the project number to your specific project.
To submit a job to the queue, issue the following command:
qsub my-script.csh
If WRF code is built to be run serially, it can be run in the command line with a command like:
./wrf.exe >& wrf.log
Other Notes
- If you plan to use NCL, it can be loaded with the following command:
module load ncl
- If you plan to use ncview, it can be loaded with the following command:
module load ncview
|
