SCF

The SCF procedure is regulated with keys that set the maximum number of iterations, the convergence criterion, and various items that control the iterative update method. Molecules may display wildly different SCF-iteration behavior, ranging from easy and rapid convergence to troublesome oscillations. We expect that the default settings take care of most cases, but one should realize that this is a difficult and tricky subject. The user has a few (main) options to adapt the procedure to the situation at hand: simple damping or the DIIS procedure (Direct Inversion in the Iterative Subspace). Either of them can be combined with Level-Shifting.

See also

In case of SCF convergence problems, see SCF troubleshooting

At each cycle the density is computed as a sum of occupied orbitals squared; the new density defines the potential from which the orbitals are re-computed. The cycle is repeated until convergence is reached. To speed-up convergence and to avoid non-convergent oscillatory behavior the values at the next iteration are constructed as a mixture of the computed new data and those used at the cycles before. This may involve only the previous cycle and is then called damping. Alternatively some sort of DIIS or LIST procedure can be invoked, which is a generalization of damping to include more previous iterations.

Starting from ADF2016, the SCF module has been rewritten from scratch. The old SCF is still available and it can be turned on using the OldSCF keyword. The description below refers to the new SCF code only. User documentation for the OldSCF module is available in the ADF2014 User’s Guide.

There are various SCF acceleration techniques implemented in ADF. By default the mixed ADIIS+DIIS method by Hu and Wang [291] is used, unless the number of DIIS expansion vectors is set to zero (see the DIIS N key below) or NoADIIS is specified. Other available methods are LISTi, LISTb, LISTf, and fDIIS. The ARH and Energy-DIIS methods are available with the OldSCF method only.

Main options

Subkeys in the data block of the master key SCF control the aspects mentioned above. Each subkey is optional. Omission means the application of default values. Omission of the SCF key altogether implies defaults for all subkeys.

SCF
 Iterations Niter
 Converge SCFcnv { sconv2 }
 ADIIS
   {THRESH1 a1}
   {THRESH2 a2}
 End
 NoADIIS
 DIIS
   {N n}
   {OK ok}
   {cyc cyc}
 End
 Mixing mix
 LISTi
 LISTb
 LISTf
 fDIIS
 OldSCF
End
Iterations Niter
Niter
The maximum number of SCF cycles allowed. In case of Geometry Optimizations it applies separately to each of the SCF procedures that are executed. Default is 300. The program executes at least one cycle, even if you specify a non-positive number.
Converge SCFcnv { sconv2 }
SCFcnv
The criterion to stop the SCF updates. The tested error is the commutator of the Fock matrix and the P-matrix (=density matrix in the representation of the basis functions) from which the F-matrix was obtained. This commutator is zero when absolute self-consistency is reached. Convergence is considered reached when the maximum element falls below SCFcnv and the norm of the matrix below 10*SCFcnv. The default is 1e-6 (in Create mode: 1e-8).
sconv2
A second criterion which plays a role when the SCF procedure has difficulty converging. When in any SCF procedure the currently applicable criterion does not seem to be achievable, the program stops the SCF. When the secondary criterion (sconv2) has been met, only a warning is issued and the program continues normally. If the secondary criterion was not met either, the program terminates any further geometry optimizations, frequency steps, etc. You can prevent the program from terminating in such a case with the key ALLOW. The default for sconv2 is 1e-3.
ADIIS
THRESH1 and THRESH2 correspond to values of the maximum element of the [F,P] commutator matrix, ErrMax. If ErrMax \(\geq\) a1 then only A-DIIS coefficients are used to determine the next Fock matrix. If ErrMax \(\leq\) a2 then only SDIIS coefficients are used. For ErrMax between a2 and a1 the total DIIS coefficients are calculated from SDIIS and A-DIIS values weighted proportionally according to the ErrMax value. Thus, the weight of A-DIIS coefficients decreases with the ErrMax value. The default values for a1 and a2 are 0.01 and 0.0001, respectively. In difficult SCF cases, especially when the Pulay DIIS appears to be unstable, it may be worth trying to decrease the thresholds and let A-DIIS approach the final solution on its own.
NoADIIS
Disables A-DIIS and switches SCF to a damping+DIIS scheme. In this scheme, SCF starts with simple damping (a.k.a. mixing) and continues until the maximum element value of the [F,P] commutator drops below the DIIS OK parameter but for no more than DIIS Cyc iterations. After this, the pure Pulay DIIS scheme becomes effective till the end of the SCF procedure.
DIIS

The DIIS sub-block specifications can be given to control the SCF acceleration procedure. All these specifications are optional. Normally, ADIIS+SDIIS is activated starting from the 2nd iteration regardless of the OK and Cyc settings below. The two settings become effective only when NoADIIS is specified.

N
The number of expansion vectors used for accelerating the SCF. The number of previous cycles taken into the linear combination is then n-1 (the new computed potential is also involved in the linear combination). By default n=10. An input value smaller than 2 disables the DIIS. Note that this number applies not only to Pulay DIIS scheme but also to other DIIS-like methods, such as A-DIIS and LIST methods. It should be noted that methods from the LIST family are quite sensitive to the number of expansion vectors used. For this reason, these methods have a built-in limit on the number depending on the iteration number and the degree of convergence. The number of vectors in LIST increases when approaching convergence but it is always limited by DIIS N, which is a hard limit. It may be useful to set DIIS N to a value larger than 10 when using a LIST method with a difficult to converge system. A value between 12 and 16 can sometimes get the job done.
OK
The Pulay DIIS starting criterion, when A-DIIS is disabled, by default 0.5 a.u..
Cyc
When A-DIIS is disabled, the Pulay DIIS will start at this iteration (by default 5) irrespective of the DIIS OK value above.
LISTi LISTb LISTf
The LIST family method has been implemented following publications by the group of Y.A. Wang [432]. Even though the keyword does not take any arguments, the number of vectors to store specified by DIIS N also applies to the LIST methods. This number is a very important parameter and it is worthwhile increasing or decreasing it in case of SCF convergence problems. A word of caution: do not just blindly increase the number for every system. Testing showed that a large number breaks convergence for some, mainly small, systems.
Mixing mix
mix
When none of the SCF acceleration methods is active, the next Fock matrix is determined as \(F = mix F_{n} + (1-mix) F_{n-1}\). The default value is 0.2.
OldSCF
Enforce the old SCF procedure. It will also be used automatically when one of the following is true: [*] Occupations Steep is specified; [*] Level-shifting is enabled (see the Lshift key below); [*] The augmented Roothaan-Hall method is enabled (see the ARH key below); [*] The Energy-DIIS method is enabled (see the EDIIS key below); [*] The SIC-OEP method is used; [*] The RESTOCC key is specified.
Lshift vshift {Err=shift_err} {Cyc=shift_cyc}

Level shifting is currently not implemented in the new SCF, therefore specifying Lshift switches on OldSCF automatically.

VShift
The level shifting parameter. The diagonal elements of the Fock matrix, in the representation of the orbitals of the previous iteration, are raised by vshift hartree energy units for the virtual orbitals. This may help to solve convergence problems when during the SCF iterations charge is sloshing back and forth between different orbitals that are close in energy and all located around the Fermi level. Level shifting is not supported in the case of Spin-Orbit coupling. At the moment properties that use virtuals, like excitation energies, response properties, NMR calculations, will give incorrect results if level shifting is applied.
Shift_err
Specifies that level shifting will be turned off by the program as soon as the SCF error drops below a threshold; default value: 0.
Shift_cyc
Specifies that level shifting is not turned on before the given SCF cycle number (for the start-up geometry); default value: 1.

Note 1: very strong damping, i.e. a very small value of mix such as 1e-3, may not combine very well with the DIIS procedure. The reason is that with strong damping successive SCF cycles tend to be very similar and the vectors building up the DIIS space become linearly dependent. We recommend in difficult cases either to use a not too strong damping (mix=0.03) or to use strong damping while the DIIS is disabled (by setting n=0 for the DIIS subkey and include the NOADIIS subkey of SCF) during a limited number of SCF iterations, and then restart with DIIS activated and less stringent damping.

Note 2: Another feature, electron smearing, may be used to overcome convergence difficulties. The idea is to distribute electron occupations fractionally over a few states around the Fermi level, by a pseudo-thermal distribution function. This aspect is controlled with the Smear option to the Occupations key. One should be aware that the applied distribution of occupations is not really an approximation to the finite-temperature case. In fact, the results are unphysical and one should not use the results as a meaningful outcome. The smearing trick is only to be used to overcome convergence difficulties. Having reached convergence with it, one should typically do a follow-up restart calculation without smearing, using the converged outcomes to hopefully get the thing to converge properly. A typical ‘allowed’ application is the usage of smearing during geometry optimizations, because the intermediate geometries are not relevant anyway and only a step towards the final results. By default, the program does not apply any smearing unless during a geometry optimization. See the Occupations key for more details.

Note 3: A-DIIS is not compatible with enforced non-aufbau electronic configurations and it is disabled in such a case automatically. A non-aufbau electronic configuration may be enforced using a block form of the Occupations key, but it may also result from the KeepOrbitals (a.k.a. orbital tracking) feature.

Energy-DIIS

SCF
 EDIIS
 ...
End

Energy-DIIS is implemented following the paper by Kudin, Scuseria, and Cances [289]. The method is invoked by specifying an EDIIS keyword in the SCF block. Please note that similar to ARH and unlike the standard SCF procedure in ADF this method requires energy evaluation at each SCF cycle, which makes it significantly slower compared to energy-free SCF. You might need a higher integration accuracy to get an accurate total energy. The same restrictions apply as for the key TOTALENERGY. The EDIIS method will start at the 2nd SCF cycle, and the size of the DIIS space will be the same as for the normal DIIS. This subkey EDIIS can be used in addition to the other subkeys of the block key SCF. This method has been superceeded by A-DIIS and the LIST family methods

Augmented Roothaan-Hall (ARH)

The Augmented Roothaan-Hall method has been developed by T. Helgaker and coworkers and is extensively discussed in Ref. [271]. The basic idea of the method is that the density matrix is optimized directly to minimize total energy. At each step, the new density matrix is parametrized in terms of matrix exponent:

Pnew = exp(-X) Pold exp(X),

here, X is an anti-symmetric step matrix subject to the following conditions:

X = argmin{E(P(X))} - X minimizes the energy

|X| < h - length of X is smaller than or equal to some trust radius

The optimal X is found using a Conjugate Gradient method, possibly with pre-conditioning. The trust radius is updated based on how well the energy change is predicted.

ARH Input

The ARH procedure is invoked by including the ARH block in the SCF input block. This subkey ARH can be used in addition to the other subkeys of the block key SCF.

SCF
  ARH
    {CONV conv}
    {ITER iter}
    {NSAVED nsaved}
    {START start}
    {FINAL} ...
  End
  ...
End
SYMMETRY NOSYM

All parameters in the ARH keyword are optional. The following arguments determine the main parameters of the ARH procedure.

CONV conv
ARH convergence criterion. When the RMS gradient and its maximum components are both lower than the criterion, the ARH procedure will be considered converged. The default value is 10-4 .
ITER iter
Maximum number of ARH iteration to perform. Please note that in difficult cases a huge number of iterations may be required for complete SCF convergence. The default value is 500.
FINAL
Determines whether SCF is continued after ARH has completed. If this option is set, one Fock matrix diagonalization will be performed to get orbitals and the SCF procedure will be halted. By default this option is OFF.
START start
Sets the SCF cycle number on which the ARH method is invoked. The default value is 2. Using a larger value may provide a better starting guess for the ARH minimization.
NSAVED nsaved
Sets the number of saved density and Fock matrices used for augmentation of the electronic Hessian. The default value is 8. A larger nsaved value should be used in difficult cases when the number of orbitals very close to the Fermi level is large.

The default minimization method is Untransformed Pre-conditioned Conjugate Gradient. The following two parameters may be used to change this.

NOPRECOND
Disables pre-conditioning during the CG minimization. This option should not be used if atoms heavier than the second-row elements are present.
TRANSPCG
Specifying this option will enable the use of the Transformed Pre-conditioned CG method, which may result in better SCF convergence in some cases.

At each SCF step, the procedures begins by performing usual CG minimization keeping track of the total step length. If at some micro-iteration the step length exceeds the trust radius, the procedure switches to trust-radius optimization in the reduced space, which, in turn, is halted as soon as the level-shift parameter mu has converged. The final step is then calculated as a Newton step in the reduced space of all the trial vectors generated during CG minimization. The following options may be used to modify this behavior.

NOSWITCHING
Setting this option turns OFF the switching from the normal CG to a trust-radius minimization in reduced space. Using this option helps to reduce the total number of SCF cycles is some cases.
SHIFTED
Setting this option will turn ON the trust-radius optimization from the first micro- iteration.
CGITER=cgiter
Sets the maximum number of micro-iterations.

The next two options determine the trust radius.

TRUSTR=trustr
Initial value for the trust radius. Default: 0.5
MAXTRUSTR=maxtrustr
The maximum trust radius value. This is set to 0.5 by default and should never be changed.

ARH Notes and Recommendations

Restriction: The method currently works for symmetry NOSYM calculations only. The NOSYM requirement comes from the fact that during direct optimization of the density matrix it may have a symmetry lower than that of the molecule.

ARH can not be used in combination with explicit occupation numbers.

The method requires the total energy to be calculated at each step, which makes it much more expensive compared to the standard SCF procedure that does not need or use the energy. Therefore, the method should only be used when the standard SCF procedure fails. Another complication caused by the use of the total energy is that somewhat higher integration accuracy may be required to get stable SCF convergence, and that the method may not be applicable in all cases. It is also recommended to use the ADDDIFFUSEFIT keyword to increase accuracy of the total energy and, thus, improve convergence. Please refer to the TOTALENERGY keyword for more information.