6.2.4.2. Optimizer Stoppers
***************************

A "Stopper" will cause an optimizer to stop and a new one to start
(according to the OptimizerSelector). Stoppers can be used to define
convergence criteria, or to stop optimizers that are performing poorly
compared to the best optimizer.

"Stoppers" also follow the general pattern.

Tip:

  "ExitConditions" are different from "Stoppers". To understand the
  difference between them, see the Difference Between Stoppers and
  ExitConditions.

By default, the current best optimizer is never stopped even if it
satisfies the "Stopper" conditions, but this can be allowed with the
"ApplyStoppersToBestOptimizer" option:

"ApplyStoppersToBestOptimizer"
   Type:
      Bool

   Default value:
      No

   Description:
      By default the stoppers are not applied to the best optimizer
      (the one who has seen the best value thus far). This is because
      many stoppers are based on comparisons to the best optimizer,
      and in most scenarios one would like to keep a well-performing
      optimizer alive. For some stopper configurations this paradigm
      does not make sense and we would prefer to apply the stoppers
      equally to all optimizers.

"Stoppers" are evaluated after every loss function call by default,
but this interval can be adjusted with the "CheckStopperInterval" key:

"CheckStopperInterval"
   Type:
      Integer

   Default value:
      1

   Description:
      Number of loss function evaluations between evaluations of the
      stopper conditions.

You can specify multiple stopper conditions, and combine them using
logical operators together with the "StopperBooleanCombination"
option.

"StopperBooleanCombination"
   Type:
      String

   GUI name:
      Combine stoppers

   Description:
      If multiple Stoppers are used this is required to indicate how
      their evaluations relate to one another.                   Use
      an integer to refer to a stopper (defined by order in input
      file).                   Recognizes the symbols: ( ) & |
      E.g. (1 & 2) | 3.                   Defaults to an OR
      combination of all selected stoppers.

For example, to stop optimizers that have used more than 10000
function calls, or that have not improved their best value in a long
time:

   Stopper
       Type MaxFunctionCalls
       MaxFunctionCalls 10000
   End
   Stopper
       Type BestFunctionValueUnmoving
       BestFunctionValueUnmoving
           Tolerance 0.1
       End
   End
   StopperBooleanCombination 1 | 2

Available types of stoppers:

BestFunctionValueUnmoving
   Stop optimizers that haven't improved their best point in a long
   time.

CurrentFunctionValueUnmoving
   Stop optimizers which have not changed their currently explored
   loss function value a lot in a long time.

MaxFunctionCalls
   Stop optimizers when they have evaluated the loss function a
   certain number of times.

MaxInteroptimizerDistance
   Stop optimizers too close together in parameter space.

MaxSequentialInvalidPoints
   Stop optimizers that are stuck in areas of parameter space which
   return non-finite loss function values.

MinStepSize
   Stop optimizers that are taking very small steps between
   iterations.

OptimizerType
   For use with other conditions. Stop optimizers of a specific type.

TimeAnnealing
   For use with other conditions. Keep an optimizer alive for longer
   if it's relatively new.

ValidationWorsening
   Stop optimizers which are evaluating points where the validation
   set loss value is worse than the average of several previously seen
   ones.

ValueAnnealing
   For use with other conditions. Keep an optimizer alive if it looks
   likely to overtake the best optimizer.

Below we provide a technical, non-technical, and graphical description
for each of the available "Stoppers" so that you can understand their
function.


6.2.4.2.1. BestFunctionValueUnmoving
====================================

This "Stopper" is designed to stop optimizers which are 'lost' and
unable to improve the best loss value they have found. For example,
CMA-ES will occasionally not show any degree of convergence over many
evaluations when handling tough ReaxFF reparameterizations (see
example below).

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop optimizers which have not improved by more   | Stopper     Type BestFunctionValueUnmoving         |
| than 10% in 900 function evaluations."             | BestFunctionValueUnmoving                          |
|                                                    | NumberOfFunctionCalls 900         Tolerance 0.1    |
|                                                    | End End                                            |
+----------------------------------------------------+----------------------------------------------------+

   [image]Optimizer running loss after 1000 function evaluations at
   which point the Stopper is evaluated. Best points shown in red.
   "Best @ x" means the best loss value seen after x function
   evaluations.

"Stopper"
   "BestFunctionValueUnmoving"
      Type:
         Block

      Description:
         Return True if an optimizer's best value has not improved
         significantly in a given amount of time.

      "NumberOfFunctionCalls"
         Type:
            Integer

         Description:
            Number of function evaluations between comparison points.

      "Tolerance"
         Type:
            Float

         Default value:
            0.0

         Description:
            Function tolerance fraction between 0 and 1.


6.2.4.2.2. CurrentFunctionValueUnmoving
=======================================

This "Stopper" stops optimizers which are fluctuating near a minimum
but are unable to converge with their own internal conditions. This is
a very common problem with ReaxFF reparameterizations; optimizers are
unable to converge due to numerical instability near minima.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop optimizers which do not change the function  | Stopper     Type CurrentFunctionValueUnmoving      |
| values they are exploring by more than 10% in 400  | CurrentFunctionValueUnmoving                       |
| function evaluations."                             | NumberOfFunctionCalls 400         Tolerance 0.1    |
|                                                    | End End                                            |
+----------------------------------------------------+----------------------------------------------------+

   [image]Optimizer running loss after 1600 function evaluations at
   which point the Stopper is evaluated. Average and standard
   deviation shown in red.

"Stopper"
   "CurrentFunctionValueUnmoving"
      Type:
         Block

      GUI name:
         Current function value unmoving:

      Description:
         Return True if an optimizer's current function value has not
         improved significantly in a given amount of time.

      "NumberOfFunctionCalls"
         Type:
            Integer

         Description:
            Number of function evaluations between comparison points.

      "Tolerance"
         Type:
            Float

         Default value:
            0.0

         Description:
            Function tolerance fraction between 0 and 1.


6.2.4.2.3. MaxFunctionCalls
===========================

Simple "Stopper" to limit the number of function evaluations an
optimizer is allowed. This is distinct from the similar
"ExitCondition" MaxTotalFunctionCalls.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop an optimizer after it has used 500 function  | Stopper     Type MaxFunctionCalls                  |
| evaluations."                                      | MaxFunctionCalls 500 End                           |
+----------------------------------------------------+----------------------------------------------------+

   [image]Two parallel optimizers, only optimizer 1 would be stopped
   by the MaxFunctionCalls Stopper. Evaluation numbers in ParAMS
   graphs are global, hence in this example the Stopper is only
   triggered after global evaluation 900, which corresponds to 500
   calls for optimizer 1 and 400 calls for optimizer 2.

"Stopper"
   "MaxFunctionCalls"
      Type:
         Integer

      Description:
         Returns True after an optimizer has evaluated at least n
         points.                      Use in an AND combination with
         other stoppers to keep optimizers alive for at least some
         period of time, or alone or in an OR combination to shut them
         down after some period of time.


6.2.4.2.4. MaxInteroptimizerDistance
====================================

Having two optimizers explore the same minimum is a waste of
resources, particularly if you would like to identify multiple minima.
This "Stopper" shuts down optimizers which are very close to one
another in parameter space.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop an optimizer if it is within 5% of another   | Stopper     Type MaxInteroptimizerDistance         |
| one in parameter space."                           | MaxInteroptimizerDistance                          |
|                                                    | MaxRelativeDistance 0.05                           |
|                                                    | CompareAllOptimizers No     End End                |
+----------------------------------------------------+----------------------------------------------------+

In this example, we plot two optimizers' trajectories in the 2D
parameter space they are exploring. They both start at the same point,
diverge in their initial exploration and then begin to move towards
the same minimum. Out of the two, the optimizer that is stopped is the
one that has the higher best loss function value.

In order to make the "MaxInteroptimizerDistance" easier to set up, the
cutoff threshold is made relative to the longest distance in the
space. This depends on the dimensionality of the problem, in most
cases the parameters are scaled between 0 and 1 (as in the example
below), and the longest distance in the space is \sqrt{n} where n is
the number of optimized parameters.

"CompareAllOptimizers" determines which optimizers are compared by the
"Stopper". If "No" then the best optimizer (the one that has seen the
lowest value) is compared to all other living optimizers. If "Yes",
then all optimizer combinations are tested.

   [image]Two parallel optimizers, approaching the same minimum.

"Stopper"
   "MaxInteroptimizerDistance"
      Type:
         Block

      Description:
         Return True if two optimizers are evaluating points too close
         to one another in parameter space.

      "CompareAllOptimizers"
         Type:
            Bool

         Default value:
            No

         Description:
            If Yes the distance between all optimizer combinations are
            compared, otherwise worse optimizers are only compared to
            the best one.

      "MaxRelativeDistance"
         Type:
            Float

         Description:
            Fraction (between 0 and 1) of the maximum distance in the
            space (from the point at all lower bounds to the point at
            all upper bounds) below which the optimizers are deemed
            too close and the victim will be stopped.


6.2.4.2.5. MaxSequentialInvalidPoints
=====================================

Loss functions can return non-finite results if jobs are unable to
converge or constraints are violated. Some optimizers may struggle to
deal with non-finite values, or be unable to escape from that region
of parameter space.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop optimizers which are stuck in a non-finite   | Stopper     Type MaxSequentialInvalidPoints        |
| region of parameter space for more than 50         | MaxSequentialInvalidPoints 50 End                  |
| function evaluations."                             |                                                    |
+----------------------------------------------------+----------------------------------------------------+

   [image]Optimizer evaluating many non-finite points (indicated by
   gaps in the graph). Stopper returns True at evaluation 150, the
   first time 50 sequential non-finite values occur. In ParAMS graphs
   only sequential finite points are connected, so there may be finite
   evaluations within the gaps.

"Stopper"
   "MaxSequentialInvalidPoints"
      Type:
         Integer

      Default value:
         1

      Description:
         Return True if the optimizer has failed to find a finite
         function value in the last n iterations. Usually indicates a
         lost optimizer with poor starting point.


6.2.4.2.6. MinStepSize
======================

Optimizers which are averaging very small step sizes should also be
stopped. This is sometimes needed, instead of an internal optimizer
step size conditions which typically look at single step sizes.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop an optimizer if it is averaging steps of     | Stopper     Type MinStepSize     MinStepSize       |
| less than 5% over the last 30 function             | NumberOfFunctionCalls 30         Tolerance 0.05    |
| evaluations."                                      | End End                                            |
+----------------------------------------------------+----------------------------------------------------+

   [image]Optimizer trajectory in parameter space.

   [image]Optimizer step size taken as a function of function
   evaluation.

"Stopper"
   "MinStepSize"
      Type:
         Block

      Description:
         Return True if an optimizer's step size between evaluations
         is too small.

      "NumberOfFunctionCalls"
         Type:
            Integer

         Description:
            Number of function evaluations between comparison points.

      "Tolerance"
         Type:
            Float

         Default value:
            0.0

         Description:
            Fraction (between 0 and 1) of the maximum distance in the
            space (from the point at all lower bounds to the point at
            all upper bounds)


6.2.4.2.7. OptimizerType
========================

This is an advanced "Stopper" intended for use in combination with
others. Will be triggered if an optimizer is of a specific type. The
intended use is to configure different stopping criteria for different
optimizers if multiple types are used.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop CMA Optimizers."                             | Stopper     Type OptimizerType     OptimizerType   |
|                                                    | CMAES End                                          |
+----------------------------------------------------+----------------------------------------------------+

"Stopper"
   "OptimizerType"
      Type:
         Multiple Choice

      Options:
         [AdaptiveRateMonteCarlo, CMAES, Nevergrad, Scipy,
         RandomSampling, GridSampling]

      Description:
         Intended for use with other stoppers to allow for specific
         hunting conditions based on the type of optimizer.


6.2.4.2.8. TimeAnnealing
========================

This is an advanced "Stopper" intended for use in combination with
others. Optimizers are stopped randomly with some probability based on
the time they have been alive. By using this "Stopper" with other
conditions one can ensure optimizers are only stopped after they have
been given some time to explore.

"TimeAnnealing" is resolved as follows:

   \frac{n_b}{n_t} < u

where:

* n_\text{b} is the number of function evaluations used by the best
  optimizer;

* n_\text{t} is the number of function evaluations used by the
  optimizer we is considering stopping;

* u is a uniformly randomly drawn test statistic (u ~ \mathscr{U}[0,
  u_\text{crit}));

* u_\text{crit} is the critical ratio of function evaluations below
  which a tested optimizer will not be stopped.

If the test is less than this number the optimizer is stopped.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Keep an optimizer alive at least until it has     | Stopper     Type TimeAnnealing     TimeAnnealing   |
| evaluated half as many times as the best           | CriticalRatio 0.5     End End                      |
| optimizer."                                        |                                                    |
+----------------------------------------------------+----------------------------------------------------+

The image below shows the probability of the "Stopper" being triggered
as a function of the number of function evaluations used by the tested
optimizer. In this example, the best optimizer (to which the tested
optimizer is compared) has used 1000 function evaluations.

   [image]Probability that the "TimeAnnealing" stopper will be
   triggered as a function of the number of function evaluations used
   by the tested optimizer. The best optimizer has used 1000 function
   evaluations.

"Stopper"
   "TimeAnnealing"
      Type:
         Block

      Description:
         Keeps optimizers alive based on how long they have been
         alive. Randomly keeps optimizers alive based on how long (in
         function evaluations) they have been active. The newer an
         optimizer is, the more likely it will pass the test and be
         kept alive. Used to temper very strict termination
         conditions.

      "CriticalRatio"
         Type:
            Float

         Default value:
            1.0

         Description:
            Critical ratio of function calls between stopper and
            victim above which the victim is guaranteed to survive.
            Values lower than one are looser and allow the victim to
            survive even if it has been in operation longer than the
            stopper. Values higher than one are stricter and may stop
            the victim even if it has iterated fewer times than the
            stopper.


6.2.4.2.9. ValidationWorsening
==============================

If a validation set is being used, test its loss function value and
stop the optimizer if this value is getting worse. This is useful to
guard against overfitting, a common problem during reparameterization.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Stop an optimizer if the validation set is        | Stopper     Type ValidationUnmoving                |
| getting worse over the last 1000 evaluations.      | ValidationUnmoving         NumberOfFunctionCalls   |
| Allow for fluctuations of 1%."                     | 1000         Tolerance 0.01     End End            |
+----------------------------------------------------+----------------------------------------------------+

   [image]Optimizer validation set running loss. Red line shows
   averaging.

"Stopper"
   "ValidationWorsening"
      Type:
         Block

      Description:
         Return True if the loss value of the validation set is
         increasing.

      "NumberOfFunctionCalls"
         Type:
            Integer

         Default value:
            1

         Description:
            Number of function evaluations between comparison points.

      "Tolerance"
         Type:
            Float

         Default value:
            0.0

         Description:
            The loss value (f) is modified by: f * (1 + *tol*). This
            can also be used to ignore the effects of mild
            fluctuations.


6.2.4.2.10. ValueAnnealing
==========================

This is an advanced "Stopper" intended for use in combination with
others. Optimizers are stopped randomly with some probability based on
the best loss value they have seen. The intended use is to keep
optimizers alive which otherwise meet the stopping conditions but are
close to being the best optimizer, and thus have the potential to
overtake it.

"ValueAnnealing" is resolved as follows:

   \exp \left( s \left| \frac{f_b - f_t}{f_b} \right| \right) < u

where:

* f_b is the lowest function value seen by the best optimizer;

* f_t is the lowest loss function value seen by the optimizer the
  Stopper is considering stopping;

* s is the critical stop chance defined as the probability of stopping
  the tested optimizer which is twice as large as the best optimizer
  in absolute value;

* u is a uniformly randomly drawn test statistic such that u ~
  \mathscr{U}[0, 1).

If the test is smaller than u, the optimizer is stopped.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Keep an optimizer alive if it has a reasonable    | Stopper     Type ValueAnnealing     ValueAnnealing |
| chance of overtaking the best optimizer."          | CriticalStopChance 0.5     End End                 |
+----------------------------------------------------+----------------------------------------------------+

   [image]Probability that the "ValueAnnealing" stopper will be
   triggered as a function of the loss value of the tested optimizer.
   The best optimizer's best function value is 10^4 in this example.

"Stopper"
   "ValueAnnealing"
      Type:
         Block

      Description:
         Intended for use with other stoppers. This condition is
         unlikely to stop a victim which has a loss function value
         very near the best seen thus far, but the probability of
         stopping increases with the difference between them.

      "CriticalStopChance"
         Type:
            Float

         Default value:
            0.5

         Description:
            Value is the probability of stopping a victim which is
            twice as large as the stopper in absolute value. The
            default is 50%.
