Using OpenMPI

Grid users interested in MPI are encouraged to use OpenMPI because it has a native, tight integration with GridEngine.

Hello, World

Here is a simple "Hello, World" MPI application:

   main(int argc, char **argv)
   {
      int node;
   
      int i, j;
      float f;
      
      MPI_Init(&argc,&argv);
      MPI_Comm_rank(MPI_COMM_WORLD, &node);
        
      printf("Hello World from Node %d.\n", node);
      for (j=0; j<=100000; j++)
         for(i=0; i<=100000; i++)
             f=i*2.718281828*i+i+i*3.141592654;
   
      MPI_Finalize();
   }

OpenMPI provides a compiler wrapper script for compiling MPI applications. Be sure to compile on a 64-bit linux machine if you intend to use the grid.

   % mpicc -o mpihello mpihello.c

GridEngine requires that you run this from a script:

   % cat hello
   mpirun ~/myproject/mpihello

Let's run four instances:

   % qsub -pe orte 4 hello
   Your job 102 ("hello") has been submitted

Qstat will only show you one job when you use a parallel environment. Give it the -t option to see all the slave tasks:

   % qstat -t
   job-ID prior   name  user state submit/start at     queue          master
   -------------------------------------------------------------------------
      102 0.56000 hello jsb  r     12/09/2010 10:47:16 short.q@ang23  MASTER
                                                       short.q@ang23  SLAVE
      102 0.56000 hello jsb  r     12/09/2010 10:47:16 short.q@ang39  SLAVE
      102 0.56000 hello jsb  r     12/09/2010 10:47:16 short.q@ang22  SLAVE
      102 0.56000 hello jsb  r     12/09/2010 10:47:16 short.q@ang24  SLAVE

The standard output and error from all slave tasks is accumulated in the job output files:

   % cat hello.po102
   Hello World from Node 0.
   Hello World from Node 3.
   Hello World from Node 2.
   Hello World from Node 1.