Instrumenting the application code

Now that the reference execution was successful, it is time to prepare an instrumented executable using Score-P to perform an initial measurement. By default, Score-P leverages the compiler to automatically instrument every function entry and exit. This is usually the best first approach, when you don't have detailed knowledge about the application and need to identify the hotspots in your code. For BT, using Score-P for instrumentation is simply accomplished by prefixing the compile and link commands specified in the config/make.def Makefile snippet by the Score-P instrumenter command scorep:

  MPIF77     = scorep mpif77

Note that the linker specification variable FLINK in config/make.def defaults to the value of MPIF77, i.e., no further modifications are necessary in this case.

Recompilation of the BT source code in the top-level directory now creates an instrumented executable, overwriting the uninstrumented binary (for archiving purposes, one may consider renaming it before recompiling):

  % cd ..
  % make clean
  rm -f core
  rm -f *~ */core */*~ */*.o */npbparams.h */*.obj */*.exe
  rm -f MPI_dummy/test MPI_dummy/libmpi.a
  rm -f sys/setparams sys/makesuite sys/setparams.h
  rm -f btio.*.out*
  % make bt NPROCS=64 CLASS=D
     =========================================
     =      NAS Parallel Benchmarks 3.3      =
     =      MPI/F77/C                        =
     =========================================

  cd BT; make NPROCS=64 CLASS=D SUBTYPE= VERSION=
  make[1]: Entering directory `/tmp/NPB3.3-MPI/BT'
  make[2]: Entering directory `/tmp/NPB3.3-MPI/sys'
  cc -g  -o setparams setparams.c
  make[2]: Leaving directory `/tmp/NPB3.3-MPI/sys'
  ../sys/setparams bt 64 D
  make[2]: Entering directory `/tmp/NPB3.3-MPI/BT'
  scorep mpif77 -c  -O2 bt.f
  scorep mpif77 -c  -O2 make_set.f
  scorep mpif77 -c  -O2 initialize.f
  scorep mpif77 -c  -O2 exact_solution.f
  scorep mpif77 -c  -O2 exact_rhs.f
  scorep mpif77 -c  -O2 set_constants.f
  scorep mpif77 -c  -O2 adi.f
  scorep mpif77 -c  -O2 define.f
  scorep mpif77 -c  -O2 copy_faces.f
  scorep mpif77 -c  -O2 rhs.f
  scorep mpif77 -c  -O2 solve_subs.f
  scorep mpif77 -c  -O2 x_solve.f
  scorep mpif77 -c  -O2 y_solve.f
  scorep mpif77 -c  -O2 z_solve.f
  scorep mpif77 -c  -O2 add.f
  scorep mpif77 -c  -O2 error.f
  scorep mpif77 -c  -O2 verify.f
  scorep mpif77 -c  -O2 setup_mpi.f
  cd ../common; scorep mpif77 -c  -O2 print_results.f
  cd ../common; scorep mpif77 -c  -O2 timers.f
  make[3]: Entering directory `/tmp/NPB3.3-MPI/BT'
  scorep mpif77 -c  -O2 btio.f
  scorep mpif77 -O2 -o ../bin/bt.D.64 bt.o make_set.o initialize.o exact_solution.o
  exact_rhs.o set_constants.o adi.o define.o copy_faces.o rhs.o solve_subs.o
  x_solve.o y_solve.o z_solve.o add.o error.o verify.o setup_mpi.o
  ../common/print_results.o ../common/timers.o btio.o
  make[3]: Leaving directory `/tmp/NPB3.3-MPI/BT'
  make[2]: Leaving directory `/tmp/NPB3.3-MPI/BT'
  make[1]: Leaving directory `/tmp/NPB3.3-MPI/BT'