SCF¶

SCF convergence problems can have various reasons. Thus, finding the reasons for a particular SCF behavior is a half-way to solving the problem. You’ll be surprised but the majority of SCF convergence problems are caused by a non-physical calculation setup, such as mistakes in the geometry or a too large negative charge. Thus, the first thing to do is to check if the geometry is really what it is meant to be. Check for too short inter-atomic distances, make sure the coordinates are specified in the right units. By default ADF expects atomic coordinates in Angstrom so check that the coordinates are provided in these units. Also check that no atoms got “lost” when importing coordinates.

So, your calculation is set up correctly, but the SCF still does not converge. Before discussing other options, let’s look at how the SCF process in ADF is organized. In a nutshell, it consists of the following steps:

A Fock matrix is constructed from the current density and the potential.
The Fock matrix is used in a DIIS procedure where it is mixed with some previous Fock matrices to construct a new one. By default ADF uses a mixed ADIIS+DIIS scheme.
This new Fock matrix is diagonalized to obtain molecular orbitals (MOs).
MOs are populated by electrons following the aufbau principle or, if the KeepOrbitals feature is ON, by overlap with a previous density matrix.
A new density matrix is constructed from occupied MOs and the steps 1-5 are repeated.

Problems on any of the steps 1, 2, and 4 above can cause problems in the whole SCF process. Usually one can identify which step causes the problems by looking in the logfile and in the output file. In the logfile, two values are printed for each SCF cycle: ErrMat and MaxEl. Both values are related to commutator of the current Fock and density matrices, [F,P]. ErrMax is a sum of squares of the commutator matrix elements while MaxEl is its largest (by absolute value) element. Below, different SCF patterns will be discussed with suggestions on how to solve them.

By far the most common reason for non-converging SCF is a very small or absent HOMO-LUMO gap. This problem is most frequently observed for compounds containing d- and f-elements (transition and rare-earth metals). This causes different MOs to be populated in subsequent cycles at the step 4 above, which, in turn, leads to large changes in the density and Fock matrices between cycles. In the logfile, the problem manifests itself by the ErrMat and MaxEl values remaining rather large (in the order of 0.1 to a few tens) sometimes going down to smaller values but then jumping back up. By looking at the MO population numbers in the output file one can sometimes see that the HOMO changes from cycle to cycle. There are a few ways to get the SCF converged in such a situation.

For open-shell electronic configurations, it is possible that a spin-unrestricted calculation will converge better than a spin-restricted one. Thus, if the molecule is not going to be used as a fragment (in which case it must be spin-restricted) then it is recommended to perform a spin-unrestricted calculation in a high-spin configuration. This is particularly useful for molecules with multiple radical centers, such bi- or multi-nuclear transition metal complexes. After a high-spin calculation of the complex has converged one can perform a broken-symmetry low-spin calculation using the high-spin results as a restart and a SpinFlip feature. See also a tutorial on spin coupling of an iron compound for an example of this approach.

If a spin-unrestricted calculation is not desirable or if it also has SCF convergence problems one may consider trying different DIIS methods in the step 2 above or try a completely different SCF method, preferably in the order listed below:

ADIIS is incompatible with enforced non-aufbau electronic configurations so it is disabled in such a case and a pure DIIS is used. A non-aufbau electronic configuration may be enforced using a block form of the Occupations key, but it may also result from the KeepOrbitals (a.k.a. orbital tracking) feature. In both cases A-DIIS should not be used.
The LIST family of methods are developed by Y.A. Wang and coworkers. The convergence when using LISTi or LISTb is often as good as that of A-DIIS, but sometimes it may be better or worse. Like A-DIIS, these methods also replace the DIIS step only leaving other steps unchanged. In the LIST methods, the number of the expansion vectors (also known as the size of the DIIS space) is an important factor which may have to be adjusted to achieve convergence.
Energy-DIIS by Scuseria and Kudin is a powerful method. It requires evaluation of the total energy, which is its strong point and a weakness at the same time. It is a strong point because it lets the algorithm converge the SCF to a configuration with the lowest energy and it is a weakness because energy evaluation is a computationally expensive procedure in ADF. To use Energy-DIIS just add an EDIIS keyword to the SCF input block.
The Augmented Roothaan-Hall (ARH) method is an alternative SCF method that combines steps 2-4 above into a single step. Essentially, ARH performs a direct minimization of the energy as a function of the density matrix combining a preconditioned conjugate-gradient method with a trust-radius approach. This is probably the most powerful SCF method to date because it can converge even the most difficult cases. However, it also has its limitations and drawbacks discussed in the corresponding section of the ADF User’s Guide. The most important drawback is that, like Energy-DIIS, it also requires evaluation of the total energy. Besides, for the method to work reliably, the energy must be accurate, which means that an accurate fit and high integration accuracy should be used.

As mentioned above, A-DIIS, LIST and Energy-DIIS affect only the DIIS step of the whole SCF process. Thus, it is possible that the SCF still has trouble converging even with the best DIIS method, because different MOs are occupied in different SCF cycles, which induces large changes in the density. ADF has a built-in feature called KeepOrbitals that assigns electrons to MOs based on their overlap with occupied MOs from the previous SCF cycle. KeepOrbitals is usually switched on after half of the allowed SCF cycles has passed (150 by default). However, if there are SCF problems, switching on KeepOrbitals may not have the desired effect. For example, the system may be trapped in an excited state due to it. Changing the SCF cycle at which KeepOrbitals kicks in may affect the final electronic configuration. Thus, playing with KeepOrbitals and trying different values for its parameter (the SCF cycle number) is encouraged.

For the DIIS-like methods listed above (A-DIIS and LISTi), the number of expansion vectors is an important parameter. We found that in some cases, for example, transition metal clusters, increasing this number to 20 or 25 solves most of the problems. Thus, the first thing to try when there are SCF convergence problems is to use the following input parameters:

SCF
  ADIIS
  DIIS N=20
  Iterations 300
END
Occupations KeepOrbitals=300

The input above turns A-DIIS on from the first SCF cycle, increases the number of expansion vectors to 20 and disables KeepOrbitals during the whole SCF process. With these settings, the previously non-converging transition metal clusters have converged in under 100 cycles. You can also try inserting any of the LIST methods instead of ADIIS.

Another trick that may help sometimes is allowing more cycles with simple damping before switching to DIIS. It is also recommended to use a smaller damping factor in this case. The following SCF settings are then recommended:

SCF
! The default value of 0.2 may be too high for some systems
! so we change it to 0.1
  NoDIIS
  Mixing 0.1
! The default value DIIS ok=0.5 might make DIIS kick in too soon
! thus making it unstable. The cycle starting from which DIIS is
! enforced may also be shifted forward.
  DIIS ok=0.01 cyc=20
! Set the max number of SCF cycles to 100 but you can use an
! even larger value.
  Iterations 100
END

Numerical noise in the exchange-correlation potential may also contribute to SCF convergence problems. If this is the case, the SCF process starts off converging quite well until some point after which the ErrMat and MaxEl values remain relatively small but do not decrease further. This behavior is typically observed for systems where weak (Van der Waals or hydrogen bonding) interactions are present. It is caused by the density fit not being good enough in the chemically relevant region between weakly bonded fragments.

This problem can be resolved by adding the EXACTDENSITY keyword to the input file. When using ADFinput, the corresponding option called “Density used in XC-potential” found on the Accuracy tab should be set to “Exact” or “Exact MO-based”, “Exact” being preferred.