6.2.6. Parallelization options¶

The ParallelLevels block provides fine control over how ParAMS parallelizes the optimization at various levels. The most important option is ParallelLevels%Optimizations which allows you to run multiple optimizers in parallel.

The total number of cores required to satisfy a ParallelLevels configuration is the product of all the levels. For example, the unrealistic setup below implicitly expects a \(6 \times 5 \times 4 \times 3 \times 2 = 720\) core system:

ParallelLevels
    Optimizations 6
    ParameterVectors 5
    Jobs 4
    Processes 3
    Threads 2
End

This configuration will run 6 optimizers in parallel. Each optimizer will evaluate 5 parameter sets in parallel. For each evaluation 4 jobs will be evaluated simultaneously. For each job, 3 processes will be started. For each process, 2 threads will be used.

Important

Configure the parallelization options with care. In general we recommend to only setting ParallelLevels%Optimizations as appropriate defaults are automatically determined for the other levels. Many optimizers do not support parallel function evaluations, and tuning lower levels is very dependent on the size and properties of your JobCollection.

ParallelLevels
   Optimizations integer
   ParameterVectors integer
   Jobs integer
   Processes integer
   Threads integer
End

ParallelLevels

Type: Block
GUI name: Parallelization distribution:
Description: Distribution of threads/processes between the parallelization levels.

Optimizations

Type: Integer
Default value: 1
GUI name: Number of parallel optimizers
Description: Number of independent optimizers to run in parallel.

ParameterVectors

Type: Integer
Default value: 0
GUI name: Loss function evaluations (per optimizer)
Description: Number of parameter vectors to try in parallel for each optimizer iteration. This level of parallelism can only be used with optimizers that support parallel optimization! Default (0) will set this value to the number of cores on the system divided by the number of optimizers run in parallel, i.e., each optimizer will be given an equal share of the resources.

Jobs

Type: Integer
Default value: 0
GUI name: Jobs (per loss function evaluation)
Description: Number of JobCollection jobs to run in parallel for each loss function evaluation.

Processes

Type: Integer
Default value: 1
GUI name: Processes (per Job)
Description: Number of processes (MPI ranks) to spawn for each JobCollection job. This effectively sets the NSCM environment variable for each job. A value of -1 will disable explicit setting of related variables. We recommend a value of 1 in almost all cases. A value greater than 1 would only be useful if you parametrize DFTB with a serial optimizer and have very few jobs in the job collection.

Threads

Type: Integer
Default value: 1
GUI name: Threads (per Process)
Description: Number of threads to use for each of the processes. This effectively set the OMP_NUM_THREADS environment variable. Note that the DFTB engine does not use threads, so the value of this variable would not have any effect. We recommend always leaving it at the default value of 1. Please consult the manual of the engine you are parameterizing. A value of -1 will disable explicit setting of related variables.