6.2.5. Generator, Selector, Spawning
************************************


6.2.5.1. Initial parameters generator
=====================================

When a new optimizer is started, the initial parameters can be
controlled with the "Generator" block which follows the general
pattern:

   Generator
      ExploreExploit
         Focus float
         MaxFunctionCalls integer
      End
      Perturbation
         StandardDeviation float
      End
      Type [Incumbent | ExploreExploit | Perturbation | Random | SinglePoint]
   End

"Generator"
   Type:
      Block

   GUI name:
      Starting point generator:

   Description:
      A Generator used to produce x0 starting points for the
      optimizers.

   "ExploreExploit"
      Type:
         Block

      Description:
         Blends a randomly generated point with the location of an
         existing optimizer, based on the time progress through the
         optimization. The optimizer is chosen based on a weighted
         roulette selection based on their function value. Early in
         the optimization optimizers are started randomly, and later
         they are started near previously found good minima.

      "Focus"
         Type:
            Float

         Default value:
            1.0

         Description:
            The blend parameter between random point and incumbent
            points. Used as follows: p=(f_calls / max_f_calls) **
            focus

      "MaxFunctionCalls"
         Type:
            Integer

         Description:
            Maximum function calls allowed for the optimization, at
            and beyond this point there is a 100% chance that a
            previously evaluated point will be returned by the
            generator. If the optimization is not limited by the
            number of function calls, provide an estimate.

   "Perturbation"
      Type:
         Block

      Description:
         Randomly generates parameter vectors from a multivariate
         normal distribution around the starting parameters.

      "StandardDeviation"
         Type:
            Float

         Default value:
            0.2

         Description:
            Standard deviation of the multivariate normal
            distribution. Used here to control how wide the generator
            should explore around the starting parameters.

   "Type"
      Type:
         Multiple Choice

      Default value:
         SinglePoint

      Options:
         [Incumbent, ExploreExploit, Perturbation, Random,
         SinglePoint]

      GUI name:
         Starting points generator

      Description:
         Algorithm used to pick starting points for the optimizers.
         Available options:                      • Incumbent:
         Optimizers will be started at the best point seen thus far by
         any optimizer. First point is random.                      •
         ExploreExploit: Early starting points are random, but later
         points are closer to good minima.                      •
         Perturbation: Each starting point is a drawn from a
         multivariate Gaussian distribution centred at the initial
         parameter set.                      • Random: Optimizers are
         started at random locations in parameter space.
         • SinglePoint: All optimizers are started at the initial
         parameter values.


6.2.5.2. Optimizer selector
===========================

If you have multiple "Optimizer" blocks, then when a new Optimizer is
started it will be selected according to the rule defined by the
"OptimizerSelector" which follows the general pattern:

   OptimizerSelector
      Chain
         Thresholds integer_list
      End
      Type [Cycle | Random | Chain]
   End

"OptimizerSelector"
   Type:
      Block

   GUI name:
      Selection between optimizer types:

   Description:
      If multiple Optimizers are included, then this block must be
      included and configures the Selector which will choose between
      them.

   "Chain"
      Type:
         Block

      Description:
         Start different optimizers at different points in time based
         on the number of function evaluations used.

      "Thresholds"
         Type:
            Integer List

         Description:
            List of loss function evaluation thresholds which switch
            to the next optimizer in the list. If there are n
            optimizers available, then this list should have n-1
            thresholds.

   "Type"
      Type:
         Multiple Choice

      Default value:
         Cycle

      Options:
         [Cycle, Random, Chain]

      GUI name:
         Select optimizer by

      Description:
         Name of the algorithm selecting between optimizers.
         Available options:                      • Cycle (Default):
         Sequential loop through available optimizers.
         • Random: Optimizers are selected randomly from the available
         options.                      • Chain: Time-based progression
         through the list of available optimizers. The next option
         will be started once a certain number of loss function
         evaluations have been used.


6.2.5.3. Control optimizer spawning
===================================

   ControlOptimizerSpawning
      MaxEvaluations integer
      MaxOptimizers integer
   End

"ControlOptimizerSpawning"
   Type:
      Block

   Description:
      Control the spawning of optimizers. Note, this is different from
      ExitConditions. These simply stop new optimizers from spawning,
      but leave existing ones untouched. ExitConditions shutdown
      active optimizers and stop the optimization.

   "MaxEvaluations"
      Type:
         Integer

      GUI name:
         – n loss function evaluations

      Description:
         No new optimizers will be started after this number of
         function evaluations has been used.
         Note, this is different from the equivalent exit condition
         which would terminate existing optimizers rather than simply
         not allowing new ones to spawn.

   "MaxOptimizers"
      Type:
         Integer

      GUI name:
         – n optimizers started

      Description:
         No new optimizers will be started after this number has been
         spawned.                      Note, this is different from
         the equivalent exit condition which would terminate existing
         optimizers rather than simply not allowing new ones to spawn.
