Using PETSc With Jetstream

Edit 0 9 …
Click here to return to top level Jetstream page.

Support has been added to solve the mesh movement and mesh adjoint equations in parallel with CG preconditioned with Additive-Schwarz using PETSc. If you want to use that feature you'll need to download, compile and install PETSc yourself.

Follow the steps below to install PETSc for use with Jetstream:

1. Get version 3.8.4
cd
tar -xvf petsc-3.8.4.tar.gz
 

2. Configure PETSc to compile on your system
  • For Graham, execute the following commands: NOTE: On Graham, you should be using the Petsc module instead!
cd $HOME/petsc-3.8.4
./configure --with-debugging=0 --with-c2html=0 --with-blas-lapack-dir=$MKLROOT
  • For Niagara, execute the following
cd $HOME/petsc-3.8.4
./configure --with-debugging=0 --with-blas-lapack-dir=$MKLROOT


4. Compile PETSc library
  • After the configuration from the previous step is complete, you will see output similar to below. Execute the first make command.
xxx=========================================================================xxx
 
 Configure stage complete. Now build PETSc libraries with (cmake build):
 
   make PETSC_DIR=/home/z/zingg/<your_username>/petsc-3.8.4/ PETSC_ARCH=arch-linux2-c-opt all
 
 or (experimental with python):
 
   PETSC_DIR=/home/z/zingg/<your_username>/petsc-3.8.4/ PETSC_ARCH=arch-linux2-c-opt ./config/builder.py
 
xxx=========================================================================xxx

5. Compile Jetstream with PETSc
  • In the jetstream Makefile set "petsc = on", then make Jetstream
cd $HOME/jetstream
make clean
make

6. Set PETSc flag in input.param file
  • To use PETSc for parallel mesh move and mesh adjoint calculations when running an optimization case in Jetstream, in input.param file, in the &MESH namelist:
    • GRID% PETSC = .true.
  • Otherwise, for serial mesh move and mesh adjoint calculations:
    • GRID% PETSC = .false.