In recent months, MCSR has upgraded PGI compilers several times, preparing for next month's upgrade from G98 to G03 on mimosa. At any given time, there may be several versions of PGI installed, and so users should know what the default version is, how to invoke the current default version, and how to override the default to use a specific version, if needed. At the time of this article, the default version is 5.1, but versions 4.1 and 5.2 are also operational, and we hope to add 5.1.6 in the next few weeks. By "default version" of PGI, we mean the versions of the important PGI files that you will access without setting any environment variables yourself, either interactively or in one of your login scripts, and without using absolute pathnames. There are four files of interest:

1. the compiler (e.g., pgcc, pgCC, pgf77, pgf90 etc.)
2. mpirun (the executable invoked to run your program on a specified # of CPUs)
3. mpirun.args (the file of default arguments to mpirun )
4. license.dat (the license file authorizing your use of PGI)

The default versions of these four are determined, and may be overridden, as follows:

1. compiler - from the PATH environment variable: the shell will invoke whichever version of the compiler it finds first when searching the pathnames in the PATH variable. The default PATH setting will come from /etc/profile, for sh/ksh/bash shell users, but may be permanently overridden in the user's .profile, .bashrc, .bash_profile, or .kshrc files in his home directory; or, for, csh/tcsh users, the default PGI setting will come from /etc/csh.login, but may be permanently overridden in the user's .login, .cshrc, or .tcshrc files in his home directory. Or, the user may temporarily override an environment variable setting (for the duration of the login session only) from the command-line. For more information on updating environment variables, see the Managing Your Environment section of the online Online Unix Help for Users Guide
2. mpirun - also from the PATH variable, with the same default originations and override options as above. Each mpirun program is found in the same directory as the compilers corresponding to the same release version. However, there are several additional versions of mpirun on the system, which correspond to MPICH implementations downloaded and installed separately from the PGI MPI distributions.
3. mpirun.args - determined by the PGI environment variable, which should be set to the root installation directory of the version of MPI that you need. (For instance, /usr/local/apps/pgi-4.1, /usr/local/apps/pgi-5.1, etc.) The default PGI setting will come from /etc/profile, for sh/ksh/bash shell users, but may be overridden in the user's .profile, .bashrc, .bash_profile, or .kshrc files in his home directory; or, for, csh/tcsh users, the default PGI setting will come from /etc/csh.login, but may be overridden in the user's .login, .cshrc, or .tcshrc files in his home directory. If you invoke a version of mpirun with an incompatible version mpirun.args, as indicated by the following error message, you can address the problem by changing the PGI ennvironment variable.
   ...mpirun.args: No such file or directory
4. license.dat - determined by the LM_LICENSE_FILE environment variable, as per the default settings in /etc/profile or /etc/csh.login, unless overridden elsewhere. A license file is version specific, but will sometimes work ok with previous versions as well (but not later versions.). If you attempt to run a newer version of mpirun with an old license file, you may see an error like this: Unable to open machines file /etc/machines at /usr/bin/mpirun line

To find out the version of PGI compilers that are your current defaults, according to your PATH variable you can type this command:

pgf90 –V

Since it is important to run your executable with the same version of mpirun as the version of the compiler you compiled with, you can find out what your default mpirun version is by typing:

which mpirun

As mentioned above, you don't have to accept the defaults. You can update your PATH environment variable, which tells the shell where to look for mpirun and the associated compilers (pgf77, pgf90, pgcc, pgCC, etc.), and your PGI variable, which tells mpirun where its mpirun.args file is; or you can specify the full pathnames of the compiler and mpirun. For instance, if you are using the bash shell, and your current defaults are the PGI 5.1 files, but you want to compile and execute your MPI C program with PGI version 5.2, using absolute pathnames:

export PGI=/usr/local/apps/pgi-5.2/
/usr/local/apps/pgi-5.2/linux86/5.2/bin/pgcc myprog.c -lmpich
/usr/local/apps/pgi-5.2/linux86/5.2/bin/mpirun a.out

Alternatively, assuming your defaults are again PGI version 5.2, and you want to run an executable previously compiled with the 4.1 version of PGI, the following lines illustrate setting the PATH and PGI environment variables so that when you enter the mpirun exectuable, without specifying the absolute pathnames, you get the version (PGI 4.1, in this case) of PGI that you intend.

For csh/tcsh family of shells:
setenv PGI /usr/local/apps/pgi-4.1
setenv PATH "/usr/local/apps/pgi-4.1/linux86/bin:${PATH}"
mpirun -np 2 a.out

For sh/ksh/bash family of shells:
export PGI=/usr/local/apps/pgi-4.1
export PATH=/usr/local/apps/pgi-4.1/linux86/bin:$PATH
mpirun -np 2 a.out

One final note. Since you should be using PBS for all computations on mimosa, if you intend to override default PGI settings by setting the PGI or PATH environment variables, you should do so in your PBS script. See PBS at MCSR for more instructions about using PBS.

For up-to-date information on the versions of PGI and other MPI implementations on MCSR systems, see http://www.mcsr.olemiss.edu/computing/mpi.html.