Installation & Execution
************************


Installation with AMSpackages
=============================

Use the AMSpackages package manager to install Quantum ESPRESSO. This
will install a version of Quantum ESPRESSO 7.1 that has been modified
to support communicating with the AMS driver via the AMSpipe
interface. **This is the recommended way to install Quantum
ESPRESSO.**

* Open the package manager from the SCM → Packages menu in the
  graphical user interface, select Quantum ESPRESSO and click Install,
  *or*

* Directly install Quantum ESPRESSO from the command-line:
  ""$AMSBIN/amspackages" install qe"

Note that this will download Quantum ESPRESSO from SCM servers on the
internet. For offline installation, see the package manager
documentation.

Alternatively, you can compile your own version from source code.


How to run Quantum ESPRESSO via AMS driver
==========================================

Quantum ESPRESSO should be installed through the package manager as
explained above. That version contains some modifications by SCM that
makes it compatible with the AMS driver.

You *must* run "ams" in **serial**. This will still let you run "pw.x"
(Quantum ESPRESSO) in parallel.

To run "ams" in serial we recommend setting the environment variable
"SCM_DISABLE_MPI=1", as this will make sure that the AMS driver is
**not** launched through "mpirun" (or a batch system specific
equivalent launcher such as "srun" on SLURM).

   export SCM_DISABLE_MPI=1

If you launch AMS through PLAMS, this environment variable is
automatically set in the runscript when the Quantum ESPRESSO engine is
used.

See also: Examples

The Quantum ESPRESSO binaries are executed by the AMS driver using the
"$AMSBIN/startqe" script. This script sets up the necessary executable
and library search paths and configures the parallel environment for
Quantum ESPRESSO programs. By default, "startqe" will launch Quantum
ESPRESSO programs as a SLURM job step using "srun" when running in a
SLURM job, or as MPI programs using "mpiexec" when available. If these
defaults are not appropriate (especially when using a manually
compiled Quantum ESPRESSO), they can be customized through a number of
environment variables:

* **SCM_QESTARTCMD**: Command called to launch Quantum ESPRESSO
  binaries (like "pw.x", "projwfc.x", ...). This might need to be set
  when a custom launcher is needed to spawn the binaries. For example,
  when using a locally compiled version of Quantum ESPRESSO with MPI
  support, you might set "export SCM_QESTARTCMD=mpirun".

* **SCM_QESTARTCMD_OPTIONS**: Options to be passed to the QE start
  command defined by "SCM_QESTARTCMD". This variable is ignored when
  "SCM_QESTARTCMD" is not set.

* (*expert option*) **SCM_QERC**: Can be set to the (absolute) path to
  a shell script that will be sourced into "startqe" right before a
  Quantum ESPRESSO binary is executed. (Environment setup that might
  be needed to run QE binaries can then be put into that script.)

* (*expert option*) **SCM_USE_LOCAL_QE**: When set, instead of using
  the AMSpackage version of Quantum ESPRESSO, use a local version of
  QE that must have been compiled to support the AMS interface. The
  Quantum ESPRESSO binaries (like "pw.x", "projwfc.x", ...) of the
  local QE version should then be accessible on "$PATH".

The distribution of Quantum ESPRESSO provided by SCM is parallelized
as follows:

* **Linux**: Primary parallelization using Intel MPI, secondary
  parallelization using multiple OpenMP threads in each MPI process

* **macOS**: Single process using multiple OpenMP threads (no MPI
  support)

* **Windows**: Parallelization using Microsoft MPI (no OpenMP support)

By default, "startqe" will set the number of CPU cores used by Quantum
ESPRESSO according to the following logic:

* When running under SLURM, the number of MPI ranks and OpenMP threads
  per MPI rank is controlled by the batch system (using its
  "-n/--ntasks" and "-c/--cpus-per-task" options, respectively)

* Otherwise, the number of MPI ranks is given by the
  "NSCM_AMSEXTERNAL" or "NSCM" environment variables (the former takes
  precedence) or the "-n" command line argument to "ams".

* OpenMP threading will be disabled by default if MPI is being used.
  This can be overridden by setting "OMP_NUM_THREADS" to the desired
  number of threads per MPI rank.

* If MPI support is not available, "NSCM_AMSEXTERNAL"/"NSCM"/"-n" will
  control the number of OpenMP threads instead.

Note:

  MPI support is automatically disabled by AMS for Quantum ESPRESSO
  programs other than "pw.x" because these programs can fail when too
  many MPI ranks are used. These programs can still run with OpenMP
  parallelization as described above.

Warning:

  Setting "SCM_QESTARTCMD" disables the autoconfiguration of MPI and
  OpenMP and automatic usage of "srun". Make sure to set
  "SCM_QESTARTCMD_OPTIONS" appropriately to use the desired number of
  cores.


Manual compilation and installation
===================================

Alternatively you can compile the modified version of Quantum ESPRESSO
yourself. It is available as a fork in Gitlab:

gitlab.com/scm-bv/q-e/-/tree/amspipe

Instructions on how to build Quantum ESPRESSO can be found on the
official Wiki:

gitlab.com/QEF/q-e/-/wikis/Developers/CMake-build-system

You will have to build it using CMake and set "QE_ENABLE_AMSPIPE=ON"
to enable the integration with the AMS driver. The following script
may be a good starting point for compiling the modified Quantum
ESPRESSO version on your machine.

   git clone https://gitlab.com/scm-bv/q-e.git
   cd q-e
   git checkout amspipe
   mkdir build
   cd build
   cmake -DQE_ENABLE_AMSPIPE=ON ..
   cmake --build .

By default the AMS driver will always try to use the AMSpackages
version of QuantumESPRESSO. If you compiled your own binaries and want
to use them, make sure that your "pw.x" is available on "$PATH" and
set the "SCM_USE_LOCAL_QE" environment variable:

   export PATH="/path/to/my/q-e/build/bin:$PATH"
   export SCM_USE_LOCAL_QE=1


Compile Quantum ESPRESSO for a CUDA GPU and use with AMS Driver
---------------------------------------------------------------

Note:

  These are just example instructions that can be taken as a starting
  point. They are provided for information purposes only. You likely
  need to adapt them.Not all Quantum ESPRESSO features may work on the
  GPU.

**Step 1**: Download and install NVHPC SDK from
https://developer.nvidia.com/hpc-sdk-downloads.

**Step 2**: Source appropriate environment variables to have "nvc" and
"nvfortran" in your PATH. See the NVIDIA documentation. Example (but
this will likely depend on the installation location, specific
version, etc.):

   NVARCH=`uname -s`_`uname -m`; export NVARCH
   NVCOMPILERS=/opt/nvidia/hpc_sdk; export NVCOMPILERS
   MANPATH=$MANPATH:$NVCOMPILERS/$NVARCH/24.5/compilers/man; export MANPATH
   PATH=$NVCOMPILERS/$NVARCH/24.5/compilers/bin:$PATH; export PATH
   export PATH=$NVCOMPILERS/$NVARCH/24.5/comm_libs/mpi/bin:$PATH
   export MANPATH=$MANPATH:$NVCOMPILERS/$NVARCH/24.5/comm_libs/mpi/man

**Step 3**: Clone and build Quantum ESPRESSO to support the AMSpipe
interface

   git clone https://gitlab.com/scm-bv/q-e.git
   cd q-e
   git checkout amspipe
   mkdir build
   cd build
   cmake -DQE_ENABLE_AMSPIPE=ON -DQE_ENABLE_CUDA=ON -DCMAKE_C_COMPILER=nvc -DCMAKE_Fortran_COMPILER=nvfortran ..
   cmake --build .

**Step 4**: Run a simple calculation. Note that it is still
recommended to have installed the AMS Quantum ESPRESSO package since
that includes the pseudopotential files.

   #!/bin/sh

   ### !!! modify the below path !!! ###
   export PATH="/path/to/my/q-e/build/bin:$PATH"
   export SCM_USE_LOCAL_QE=1
   export NO_STOP_MESSAGE=yes  # prevent "ieee_inexact is signaling" warning when stopping

   export SCM_DISABLE_MPI=1

   rm -rf ams.results
   NSCM=1 OMP_NUM_THREADS=1 "$AMSBIN/ams" <<EOF
   Task GeometryOptimization
   System
     Atoms
       H 0. 0. 0.
       H 1. 0. 0.
     End
     Lattice
       6. 0. 0.
       0. 6. 0.
       0. 0. 6.
     End
   End
   Engine QuantumEspresso
        K_Points gamma
        End
   EndEngine
   EOF

Note:

  If you receive an error like "pw.x: error while loading shared
  libraries: libmkl_intel_lp64.so.2: cannot open shared object file:
  No such file or directory", in order you run your custom Quantum
  ESPRESSO you may need to set any other environment variables (in
  particular LD_LIBRARY_PATH) that you had set up for the compilation.
