6.2.4.3. Exit Conditions
************************

Exit conditions will stop all optimizers and cause ParAMS to exit when
they are triggered.

Exit conditions follow the general pattern.

Tip:

  "ExitConditions" are different from "Stoppers". To understand the
  difference between them, see the Difference Between Stoppers and
  ExitConditions.

You can include multiple ExitCondition blocks, and combine them using
logical operators with the "ExitConditionBooleanCombination" key:

"ExitConditionBooleanCombination"
   Type:
      String

   GUI name:
      Combine exit conditions

   Description:
      If multiple ExitConditions are used, this key indicates how
      their evaluations relate to one another.                   Use
      an integer to refer to a exit condition (defined by order in
      input file).                   Recognizes the symbols: ( ) & |
      E.g. (1 & 2) | 3.                   Defaults to an OR
      combination of all selected exit conditions.

For example, to run the optimization for at least one hour, and end
when 20 optimizers have been stopped or 1 has converged:

   ExitCondition
       Type TimeLimit
       TimeLimit 3600
   End
   ExitCondition
       Type MaxOptimizersStopped
       MaxOptimizersStopped 20
   End
   ExitCondition
       Type MaxOptimizersConverged
       MaxOptimizersConverged 1
   End
   ExitConditionBooleanCombination 1 & (2 | 3)

The following exit conditions are available:

MaxOptimizersConverged
   Shut down when optimizers have converged via their own internal
   conditions.

MaxOptimizersStarted
   Shut down after a certain number of optimizers have been spawned.

MaxOptimizersStopped
   Shut down when the Optimizer Stoppers configuration has resulted in
   a number of optimizers being stopped.

MaxTotalFunctionCalls
   Stop the optimization after a certain number of loss function
   evaluations.

StopsAfterConvergence
   Stop when some optimizers have converged and others have been
   stopped.

TargetFunctionValue
   Shut down when a small enough loss function value is found.

TimeLimit
   Limit the optimization wall time.

TimeLimitThroughRestarts
   Limit the total cumulative wall time across multiple restarts of
   the optimization.


6.2.4.3.1. MaxOptimizersConverged
=================================

Exit after a certain number of optimizers have converged due to their
own internal convergence conditions. In other words, optimizers which
have been stopped by a "Stopper" are *not* counted.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit when we have found ten good solutions."      | ExitCondition     Type MaxOptimizersConverged      |
|                                                    | MaxOptimizersConverged 10 End                      |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "MaxOptimizersConverged"
      Type:
         Integer

      Description:
         Return True after n optimizers have converged normally.
         An optimizer that has 'converged' is distinct from an
         optimizer that was shutdown via the Stopper settings (i.e.
         'stopped').


6.2.4.3.2. MaxOptimizersStarted
===============================

Exit after a certain number of optimizers have been spawned.

Warning:

  This condition is mainly intended for use with other conditions. It
  will trigger as soon as the nth optimizer is *started*. In other
  words, if this condition is used alone then the new optimizer will
  not be given time to evaluate at all.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit after starting at least four optimizers."    | ExitCondition     Type MaxOptimizersStarted        |
|                                                    | MaxOptimizersStarted 4 End                         |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "MaxOptimizersStarted"
      Type:
         Integer

      Description:
         Return True after n optimizers have been started.
         Note, this is best used in combination with other conditions
         because it will stop the optimization as soon as the correct
         number have been started and not allow newly spawned
         optimizer to iterate at all.


6.2.4.3.3. MaxOptimizersStopped
===============================

Exit after a certain number of optimizers have been stopped by the
"Stopper". In other words, optimizers which have converged naturally
are *not* counted.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit when five optimizers have been stopped."     | ExitCondition     Type MaxOptimizersStopped        |
|                                                    | MaxOptimizersStopped 5 End                         |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "MaxOptimizersStopped"
      Type:
         Integer

      Description:
         Return True after n optimizers have been stopped.
         An optimizer that has been 'stopped' is distinct from an
         optimizer that stopped due to its internal convergence
         conditions (i.e., 'converged').


6.2.4.3.4. MaxTotalFunctionCalls
================================

Exit the optimization after all optimizers combined have evaluated the
loss function a certain number of times. This is distinct from the
similar "Stopper" MaxFunctionCalls. ParAMS Graphs always show the
*global* evaluation number, so the number seen there will match the
number used in this condition.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit after evaluating the loss function 10000     | ExitCondition     Type MaxTotalFunctionCalls       |
| times."                                            | MaxTotalFunctionCalls 10000 End                    |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "MaxTotalFunctionCalls"
      Type:
         Integer

      Description:
         Return True after n function calls have been executed.


6.2.4.3.5. StopsAfterConvergence
================================

First waits for a certain number of optimizers to converge, and then
waits for a number of optimizers to be stopped *after* this. In other
words, this is *not* the same as combining the MaxOptimizersConverged
and MaxOptimizersStopped conditions, since this condition counts
events sequentially.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit when we have found at least one good         | ExitCondition     Type StopsAfterConvergence       |
| solution, and five other optimizers are unable to  | StopsAfterConvergence         OptimizersConverged  |
| improve on it."                                    | 1         OptimizersStopped 5     End End          |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "StopsAfterConvergence"
      Type:
         Block

      GUI name:
         Max stops after n converged:

      Description:
         Returns True when at least n_a optimizers have been stopped
         *after* n_b optimizers have converged.

      "OptimizersConverged"
         Type:
            Integer

         Default value:
            1

         GUI name:
            Min # converged:

         Description:
            Minimum number of converged optimizers.

      "OptimizersStopped"
         Type:
            Integer

         Default value:
            0

         GUI name:
            Max # stopped:

         Description:
            Maximum number of stopped optimizers.


6.2.4.3.6. TargetFunctionValue
==============================

Exit when any optimizer finds a loss function value below the target
value. Useful if you have an understanding of the problem and do not
want optimizers spending a long time overfitting.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit when an optimizer decreases the loss past    | ExitCondition     Type TargetFunctionValue         |
| 100."                                              | TargetFunctionValue 100 End                        |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "TargetFunctionValue"
      Type:
         Float

      Description:
         Return True after an optimizer finds a function value less
         than or equal to n.


6.2.4.3.7. TimeLimit
====================

Exit after the optimization has been running for a certain amount of
time.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit after 1 hour."                               | ExitCondition     Type TimeLimit     TimeLimit     |
|                                                    | 3600  # Expects seconds End                        |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "TimeLimit"
      Type:
         Float

      Unit:
         s

      Description:
         Return True after the entire optimization has been running
         for n seconds.                      Note, this is NOT the
         time for any particular optimizer.                      See
         also: Time Limit Through Restarts


6.2.4.3.8. TimeLimitThroughRestarts
===================================

For almost all scenarios TimeLimit is the appropriate exit condition
to use to end optimizations based on the amount of time used. In some
cases, if checkpointing is being used and an optimization has been
restarted, you may want to limit the *cumulative* amount of time used.
In other words, the amount of time saved in the checkpoint and the
time used in the resumed optimization. The "TimeLimitThroughRestarts"
condition exists for this purpose.

The "TimeLimit" exit condition measures only the uninterrupted time
for a single continuous optimization session.

+----------------------------------------------------+----------------------------------------------------+
| Intention                                          | Input Block                                        |
|====================================================|====================================================|
| "Exit after 1 hour, including the time used in the | ExitCondition     Type TimeLimitThroughRestarts    |
| checkpoint I am loading from."                     | TimeLimitThroughRestarts 3600  # Expects seconds   |
|                                                    | End                                                |
+----------------------------------------------------+----------------------------------------------------+

"ExitCondition"
   "TimeLimitThroughRestarts"
      Type:
         Float

      Unit:
         s

      Description:
         Return True after the sum-total of all optimization runs
         (i.e., through all restarts) has been running for n seconds.
         Note, this is NOT the time for any particular optimizer.
         See also: Time Limit
