General Questions

NCAR IBM Related Questions

Terrain

Regrid

Datagrid

Rawins

MM5

Graph

Conventions used in MM5

Trouble shooting and clues on various computers

Using OpenMP on a PC running Linux/pgf90

This note mostly concern newer PC's running pgf90 3.2 and higher

All the tests where done on a multi-processor computer running:

Red Hat Linux release 7.2
Portland Group Compiler pgf90, version 3.3-2

Compiling for OpenMP

1) Set the "-mp" and "-nosgimp" compiler flags

2) If you are using ISOIL=2 and run into compiler errors:

    Edit the following files:

    in directory:
     MM5/physics/pbl_sfc/osusfc,
    and make the change below to all of these files.
    (ie, place the omp directives BELOW the COMMON)

    So in all the above files, where the following statement appears,

cdir$ TASKCOMMON RITE
#ifndef rs6000
c$omp threadprivate (/rite/)
#else
cibmp threradlocal /ritr/
#endif
       COMMON/RITE/BETA,DRIP,EC,EDIR,ETT,FLX1,FLX2,FLX3,RHO,RUNOFF,
     &             DEW,RIB
 
 

    Change it to look like this:

       COMMON/RITE/BETA,DRIP,EC,EDIR,ETT,FLX1,FLX2,FLX3,RHO,RUNOFF,
     &             DEW,RIB

cdir$ TASKCOMMON RITE
#ifndef rs6000
c$omp threadprivate (/rite/)
#else
cibmp threradlocal /ritr/
#endif
 
 

Running the code:

Type : unlimit    (This may not be necessary on all computers)

Set the following environmental variables.

    setenv OMP_NUM_THREADS X    (where X is the number of threads you would like to use)

    setenv MPSTKZ X    (where X is the stack size value, for example 256M)

If after this you get a segmentation fault when running MM5, either increase OR descrease the MPSTKZ value. MM5 needs to have MPSTKZ reasonable large to be able to run, BUT if you oversubscribe MKSTKZ, for your specific computer, you will also get a segmentation fault. If you oversubscribe MPSTKZ by a small amount (ie, 256M is the largest value your computer can use, and you set MPSTKZ to 257M), your MM5
run may well hang.
On most computers, 256M will be the largest value you can set MPSTKZ to.
 
 
 
 

Some extra information on this subject send in by a user:

In this case the model was compiled a Portland Group compiler, version 4.0.2
More information on this new release can be found at:

    http://www.pgroup.com/faq/new_rel.htm

There is a library "libpgthread" that is still under construction at the Portland Group.
To avoid errors in the compilation of MM5, change the name of libpgthread.so
(e.g. to libpgthreat.so.old). MM5 using OpenMP compiles fine without this library.

Thanks to Sharan Majumdar from the University of Miami, for sending in this tip.