Set Up ParAMSJob Settings
*************************

ParAMS Technical Settings

A "ParAMSJob" has a "settings" attribute which contains the settings
for the job. The "settings" will be converted to the "params.in" file
before the job is run.

The first example below shows how to manually set all the input
settings, but this can be quite tedious. Further down some helpful
functions are shown, which make it easier to set up the settings.

**Requires:** AMS2026 or later

-[ Related examples ]-

* Run a ParAMSJob for Lennard-Jones

* Import ParAMS Training Data with ResultsImporter

* Convert Between ParAMS and ASE Formats

-[ Related tutorials ]-

* Getting Started: Lennard-Jones Potential for Argon

* Python Getting Started tutorial

-[ Related documentation ]-

* ParAMSJob API

* ParAMS input reference

* PLAMS Settings


Overview
========

This example shows how to configure a "ParAMSJob" directly in Python,
load an existing "params.in" file, and initialize a job from stored
ParAMS YAML files.

Call the "get_input()" function to see what the "params.in" file will
look like.

Note:

* All paths must be absolute paths. This is needed to be able to run
  the job through PLAMS

* "DataSet" and "Optimizer" are *recurring blocks*, so they are
  initialized as lists. The "DataSet[0]" syntax means that the
  settings are set for the *first* dataset, etc.

   from scm.plams import *
   from scm.params import *
   import os

   job = ParAMSJob()
   job.settings.input.Task = "Optimization"
   job.settings.input.JobCollection = "/path/job_collection.yaml"  # absolute path
   job.settings.input.DataSet = [Settings(), Settings()]  # DataSet is a recurring block
   job.settings.input.DataSet[0].Name = "training_set"
   job.settings.input.DataSet[0].Path = "/path/training_set.yaml"  # absolute path
   job.settings.input.DataSet[1].Name = "validation_set"
   job.settings.input.DataSet[1].Path = "/path/validation_set.yaml"  # absolute path
   job.settings.input.LoggingInterval.General = 10
   job.settings.input.SkipX0 = "No"  # Booleans are specified as strings "Yes" or "No"
   job.settings.input.Optimizer = [Settings()]  # Optimizer is a recurring block
   job.settings.input.Optimizer[0].Type = "CMAES"
   job.settings.input.Optimizer[0].CMAES.Sigma0 = 0.01
   job.settings.input.Optimizer[0].CMAES.Popsize = 8
   job.settings.input.ParallelLevels.Optimizations = 1  # ParallelLevels is NOT a recurring block

   print(job.get_input())

   Task Optimization

   DataSet
     Name training_set
     Path /path/training_set.yaml
   end
   DataSet
     Name validation_set
     Path /path/validation_set.yaml
   end
   ... output trimmed ....
     Type CMAES
   end

   ParallelLevels
     Optimizations 1
   end

   SkipX0 No


Load a job from a params.in file
--------------------------------

If you already have a "params.in" file (for example created by the GUI
or by hand), you can simply load it into a "ParAMSJob" using
"from_inputfile()".

Note that any paths in the "params.in" file get converted to absolute
paths.

   params_in_file = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar/params.in")
   job = ParAMSJob.from_inputfile(params_in_file)
   print(job.get_input())

   task Optimization

   parameterinterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

   dataset
     name training_set
     path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
   end

   jobcollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml
   ... output trimmed ....
   end

   parallellevels
     jobs 1
     optimizations 1
     parametervectors 1
     processes 1
   end


Create a ParAMSJob from a directory with .yaml files
----------------------------------------------------

In the input file you need to specify many paths to different .yaml
files. This can be tedious to set up manually. If you have a directory
with .yaml files (e.g. the "jobname.params" directory created by the
GUI), you can initialize a ParAMSJob to read those yaml files using
"from_yaml()". The files need to have the default names:

* job_collection.yaml

* training_set.yaml

* validation_set.yaml

* job_collection_engines.yaml or engine_collection.yaml

* parameter_interface.yaml or parameters.yaml

Note 1: When you run a ParAMSJob, any .yaml files in the current
working directory will be used if they have the default names and if
the corresponding settings are unset. In this way, you do not need to
specify the paths in the "settings" if you have the .yaml files in the
same directory as the script ".py" file that runs the job.

Note 2: "from_yaml()" only sets the settings for the yaml files and
leaves all other settings empty. "from_inputfile()" reads all the
settings from the params.in file.

   job = ParAMSJob.from_yaml(os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar"))
   print(job.get_input())

   ParameterInterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

   DataSet
     Name training_set
     Path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
   end

   JobCollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml


Input validation
----------------

The allowed settings blocks, keys, and values are described in the
documentation. If you make a mistake in the block or key names,
"get_input()" will raise an error:

   job.settings.input.NonExistingKey = 3.14159
   try:
       print(job.get_input())
   except Exception as e:
       print(e)

   Input error: unrecognized entry "nonexistingkey" found in line 10

If you want to print the input anyway, use
"get_input(validate=False)":

   print(job.get_input(validate=False))  # print the input anyway

   ParameterInterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

   DataSet
     Name training_set
     Path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
   end

   JobCollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml

   NonExistingKey 3.14159


Delete a block or key
---------------------

To delete an entry from the Settings, use "del":

   del job.settings.input.NonExistingKey


Attributes for easier setup of .yaml files
------------------------------------------

"ParAMSJob" has some special attributes which makes it easier to set
up the settings.

   job = ParAMSJob()
   job.job_collection = "my_job_collection.yaml"  # will be converted to absolute path if it exists
   job.training_set = "my_training_set.yaml"  # will be converted to absolute path if it exists
   print(job.get_input())

   DataSet
     Name training_set
     path my_training_set.yaml
   end

   JobCollection my_job_collection.yaml

Note that "job.training_set" and "job.validation_set" are quite
special: when you assign a string to them as above, it will set the
corresponding "path" in the "settings". But when you read them you
will get the corresponding Settings block:

   print(job.training_set)
   print(type(job.training_set))

   Name:   training_set
   path:   my_training_set.yaml

   <class 'scm.plams.core.settings.Settings'>

   print(job.training_set.path)
   print(type(job.training_set.path))

   my_training_set.yaml
   <class 'str'>

Assigning to "job.validation_set" will create another item in the
"job.settings.input.DataSet" list:

   job.validation_set = "validation_set.yaml"
   print(job.get_input())

   DataSet
     Name training_set
     path my_training_set.yaml
   end
   DataSet
     Name validation_set
     path validation_set.yaml
   end

   JobCollection my_job_collection.yaml

To set other settings for the training set or validation set, use the
standard dot-notation:

   job.validation_set.EvaluateEvery = 100
   job.settings.input.LoggingInterval.General = 100
   print(job.get_input())

   DataSet
     Name training_set
     path my_training_set.yaml
   end
   DataSet
     EvaluateEvery 100
     Name validation_set
     path validation_set.yaml
   end

   JobCollection my_job_collection.yaml

   LoggingInterval
     General 100
   end

You can also use the "job.parameter_interface" and
"job.engine_collection" in the same way as "job.job_collection":

   job = ParAMSJob()
   job.parameter_interface = (
       "my_parameter_interface.yaml"  # will be converted to absolute path if it exists
   )
   job.engine_collection = (
       "my_engine_collection.yaml"  # will be converted to absolute path if it exists
   )
   print(job.get_input())
   # note: job.training_set is always defined, this is why a DataSet block is printed below

   ParameterInterface my_parameter_interface.yaml

   DataSet
     Name training_set
   end

   EngineCollection my_engine_collection.yaml


Functions for recurring blocks: Optimizers, Stoppers, ExitConditions
--------------------------------------------------------------------

Use the below functions to easily add optimizers, stoppers, or exit
conditions:

   job = ParAMSJob()

   job.add_exit_condition("MaxTotalFunctionCalls", 100000)
   job.add_exit_condition("TimeLimit", 24 * 60 * 60)
   job.add_exit_condition("StopsAfterConvergence", {"OptimizersConverged": 3, "OptimizersStopped": 1})

   job.add_optimizer("CMAES", {"Sigma0": 0.01, "PopSize": 8})
   job.add_optimizer("Scipy")

   job.add_stopper("BestFunctionValueUnmoving", {"Tolerance": 0.1})
   job.add_stopper("MaxFunctionCalls", 1000)

   print(job.get_input())

   DataSet
     Name training_set
   end

   ExitCondition
     MaxTotalFunctionCalls 100000
     Type MaxTotalFunctionCalls
   end
   ExitCondition
     TimeLimit 86400
   ... output trimmed ....
       Tolerance 0.1
     End
     Type BestFunctionValueUnmoving
   end
   Stopper
     MaxFunctionCalls 1000
     Type MaxFunctionCalls
   end

To delete an added recurring block, use "pop" together with zero-based
indices:

   job.settings.input.ExitCondition.pop(1)  # 2nd exit condition
   job.settings.input.Optimizer.pop(1)  # 2nd optimizer
   job.settings.input.Stopper.pop(0)  # first stopper
   print(job.get_input())

   DataSet
     Name training_set
   end

   ExitCondition
     MaxTotalFunctionCalls 100000
     Type MaxTotalFunctionCalls
   end
   ExitCondition
     StopsAfterConvergence
   ... output trimmed ....
     End
     Type CMAES
   end

   Stopper
     MaxFunctionCalls 1000
     Type MaxFunctionCalls
   end

Note: the "ExitConditionBooleanCombination" and
"StopperBooleanCombination" work with indices starting with 1.


See also
========


Related examples
----------------

* Run a ParAMSJob for Lennard-Jones

* Import ParAMS Training Data with ResultsImporter

* Convert Between ParAMS and ASE Formats


Related tutorials
-----------------

* Getting Started: Lennard-Jones Potential for Argon

* Python Getting Started tutorial


Related documentation
---------------------

* ParAMSJob API

* ParAMS input reference

* PLAMS Settings


Python Script
=============

   #!/usr/bin/env python
   # coding: utf-8

   # ## Overview
   #
   # This example shows how to configure a ``ParAMSJob`` directly in Python, load an existing
   # ``params.in`` file, and initialize a job from stored ParAMS YAML files.

   # Call the ``get_input()`` function to see what the ``params.in`` file will look like.
   #
   # Note:
   #
   # * All paths must be absolute paths. This is needed to be able to run the job through PLAMS
   # * ``DataSet`` and ``Optimizer`` are *recurring blocks*, so they are initialized as lists. The ``DataSet[0]`` syntax means that the settings are set for the *first* dataset, etc.

   from scm.plams import *
   from scm.params import *
   import os

   job = ParAMSJob()
   job.settings.input.Task = "Optimization"
   job.settings.input.JobCollection = "/path/job_collection.yaml"  # absolute path
   job.settings.input.DataSet = [Settings(), Settings()]  # DataSet is a recurring block
   job.settings.input.DataSet[0].Name = "training_set"
   job.settings.input.DataSet[0].Path = "/path/training_set.yaml"  # absolute path
   job.settings.input.DataSet[1].Name = "validation_set"
   job.settings.input.DataSet[1].Path = "/path/validation_set.yaml"  # absolute path
   job.settings.input.LoggingInterval.General = 10
   job.settings.input.SkipX0 = "No"  # Booleans are specified as strings "Yes" or "No"
   job.settings.input.Optimizer = [Settings()]  # Optimizer is a recurring block
   job.settings.input.Optimizer[0].Type = "CMAES"
   job.settings.input.Optimizer[0].CMAES.Sigma0 = 0.01
   job.settings.input.Optimizer[0].CMAES.Popsize = 8
   job.settings.input.ParallelLevels.Optimizations = 1  # ParallelLevels is NOT a recurring block

   print(job.get_input())


   # ### Load a job from a params.in file
   #
   # If you already have a ``params.in`` file (for example created by the GUI or by hand), you can simply load it into a ``ParAMSJob`` using ``from_inputfile()``.
   #
   # Note that any paths in the ``params.in`` file get converted to absolute paths.

   params_in_file = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar/params.in")
   job = ParAMSJob.from_inputfile(params_in_file)
   print(job.get_input())


   # ### Create a ParAMSJob from a directory with .yaml files
   #
   # In the input file you need to specify many paths to different .yaml files. This can be tedious to set up manually. If you have a directory with .yaml files (e.g. the ``jobname.params`` directory created by the GUI), you can initialize a ParAMSJob to read those yaml files using ``from_yaml()``. The files need to have the default names:
   #
   #
   #
   # * job_collection.yaml
   # * training_set.yaml
   # * validation_set.yaml
   # * job_collection_engines.yaml or engine_collection.yaml
   # * parameter_interface.yaml or parameters.yaml
   #
   # Note 1: When you run a ParAMSJob, any .yaml files in the current working directory will be used if they have the default names and if the corresponding settings are unset. In this way, you do not need to specify the paths in the ``settings`` if you have the .yaml files in the same directory as the script ``.py`` file that runs the job.
   #
   # Note 2: ``from_yaml()`` only sets the settings for the yaml files and leaves all other settings empty. ``from_inputfile()`` reads all the settings from the params.in file.

   job = ParAMSJob.from_yaml(os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar"))
   print(job.get_input())


   # ### Input validation

   # The allowed settings blocks, keys, and values are described in the documentation. If you make a mistake in the block or key names, ``get_input()`` will raise an error:

   job.settings.input.NonExistingKey = 3.14159
   try:
       print(job.get_input())
   except Exception as e:
       print(e)


   # If you want to print the input anyway, use ``get_input(validate=False)``:

   print(job.get_input(validate=False))  # print the input anyway


   # ### Delete a block or key

   # To delete an entry from the Settings, use ``del``:

   del job.settings.input.NonExistingKey


   # ### Attributes for easier setup of .yaml files

   # ``ParAMSJob`` has some special attributes which makes it easier to set up the settings.

   job = ParAMSJob()
   job.job_collection = "my_job_collection.yaml"  # will be converted to absolute path if it exists
   job.training_set = "my_training_set.yaml"  # will be converted to absolute path if it exists
   print(job.get_input())


   # Note that ``job.training_set`` and ``job.validation_set`` are quite special: when you assign a string to them as above, it will set the corresponding ``path`` in the ``settings``. But when you read them you will get the corresponding Settings block:

   print(job.training_set)
   print(type(job.training_set))


   print(job.training_set.path)
   print(type(job.training_set.path))


   # Assigning to ``job.validation_set`` will create another item in the ``job.settings.input.DataSet`` list:

   job.validation_set = "validation_set.yaml"
   print(job.get_input())


   # To set other settings for the training set or validation set, use the standard dot-notation:

   job.validation_set.EvaluateEvery = 100
   job.settings.input.LoggingInterval.General = 100
   print(job.get_input())


   # You can also use the ``job.parameter_interface`` and ``job.engine_collection`` in the same way as ``job.job_collection``:

   job = ParAMSJob()
   job.parameter_interface = "my_parameter_interface.yaml"  # will be converted to absolute path if it exists
   job.engine_collection = "my_engine_collection.yaml"  # will be converted to absolute path if it exists
   print(job.get_input())
   # note: job.training_set is always defined, this is why a DataSet block is printed below


   # ### Functions for recurring blocks: Optimizers, Stoppers, ExitConditions
   #
   # Use the below functions to easily add optimizers, stoppers, or exit conditions:

   job = ParAMSJob()

   job.add_exit_condition("MaxTotalFunctionCalls", 100000)
   job.add_exit_condition("TimeLimit", 24 * 60 * 60)
   job.add_exit_condition("StopsAfterConvergence", {"OptimizersConverged": 3, "OptimizersStopped": 1})

   job.add_optimizer("CMAES", {"Sigma0": 0.01, "PopSize": 8})
   job.add_optimizer("Scipy")

   job.add_stopper("BestFunctionValueUnmoving", {"Tolerance": 0.1})
   job.add_stopper("MaxFunctionCalls", 1000)

   print(job.get_input())


   # To delete an added recurring block, use ``pop`` together with zero-based indices:

   job.settings.input.ExitCondition.pop(1)  # 2nd exit condition
   job.settings.input.Optimizer.pop(1)  # 2nd optimizer
   job.settings.input.Stopper.pop(0)  # first stopper
   print(job.get_input())


   # Note: the ``ExitConditionBooleanCombination`` and ``StopperBooleanCombination`` work with indices starting with 1.
