Set Up ParAMSJob Settings¶

A ParAMSJob has a settings attribute which contains the settings for the job. The settings will be converted to the params.in file before the job is run.

The first example below shows how to manually set all the input settings, but this can be quite tedious. Further down some helpful functions are shown, which make it easier to set up the settings.

Downloads: Notebook | Script ?

Requires: AMS2026 or later

Overview¶

This example shows how to configure a ParAMSJob directly in Python, load an existing params.in file, and initialize a job from stored ParAMS YAML files.

Call the get_input() function to see what the params.in file will look like.

Note:

All paths must be absolute paths. This is needed to be able to run the job through PLAMS
DataSet and Optimizer are recurring blocks, so they are initialized as lists. The DataSet[0] syntax means that the settings are set for the first dataset, etc.

from scm.plams import *
from scm.params import *
import os

job = ParAMSJob()
job.settings.input.Task = "Optimization"
job.settings.input.JobCollection = "/path/job_collection.yaml"  # absolute path
job.settings.input.DataSet = [Settings(), Settings()]  # DataSet is a recurring block
job.settings.input.DataSet[0].Name = "training_set"
job.settings.input.DataSet[0].Path = "/path/training_set.yaml"  # absolute path
job.settings.input.DataSet[1].Name = "validation_set"
job.settings.input.DataSet[1].Path = "/path/validation_set.yaml"  # absolute path
job.settings.input.LoggingInterval.General = 10
job.settings.input.SkipX0 = "No"  # Booleans are specified as strings "Yes" or "No"
job.settings.input.Optimizer = [Settings()]  # Optimizer is a recurring block
job.settings.input.Optimizer[0].Type = "CMAES"
job.settings.input.Optimizer[0].CMAES.Sigma0 = 0.01
job.settings.input.Optimizer[0].CMAES.Popsize = 8
job.settings.input.ParallelLevels.Optimizations = 1  # ParallelLevels is NOT a recurring block

print(job.get_input())

Task Optimization

DataSet
  Name training_set
  Path /path/training_set.yaml
end
DataSet
  Name validation_set
  Path /path/validation_set.yaml
end
... output trimmed ....
  Type CMAES
end

ParallelLevels
  Optimizations 1
end

SkipX0 No

Load a job from a params.in file¶

If you already have a params.in file (for example created by the GUI or by hand), you can simply load it into a ParAMSJob using from_inputfile().

Note that any paths in the params.in file get converted to absolute paths.

params_in_file = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar/params.in")
job = ParAMSJob.from_inputfile(params_in_file)
print(job.get_input())

task Optimization

parameterinterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

dataset
  name training_set
  path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
end

jobcollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml
... output trimmed ....
end

parallellevels
  jobs 1
  optimizations 1
  parametervectors 1
  processes 1
end

Create a ParAMSJob from a directory with .yaml files¶

In the input file you need to specify many paths to different .yaml files. This can be tedious to set up manually. If you have a directory with .yaml files (e.g. the jobname.params directory created by the GUI), you can initialize a ParAMSJob to read those yaml files using from_yaml(). The files need to have the default names:

job_collection.yaml
training_set.yaml
validation_set.yaml
job_collection_engines.yaml or engine_collection.yaml
parameter_interface.yaml or parameters.yaml

Note 1: When you run a ParAMSJob, any .yaml files in the current working directory will be used if they have the default names and if the corresponding settings are unset. In this way, you do not need to specify the paths in the settings if you have the .yaml files in the same directory as the script .py file that runs the job.

Note 2: from_yaml() only sets the settings for the yaml files and leaves all other settings empty. from_inputfile() reads all the settings from the params.in file.

job = ParAMSJob.from_yaml(os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar"))
print(job.get_input())

ParameterInterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

DataSet
  Name training_set
  Path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
end

JobCollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml

Input validation¶

The allowed settings blocks, keys, and values are described in the documentation. If you make a mistake in the block or key names, get_input() will raise an error:

job.settings.input.NonExistingKey = 3.14159
try:
    print(job.get_input())
except Exception as e:
    print(e)

Input error: unrecognized entry "nonexistingkey" found in line 10

If you want to print the input anyway, use get_input(validate=False):

print(job.get_input(validate=False))  # print the input anyway

ParameterInterface /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/parameter_interface.yaml

DataSet
  Name training_set
  Path /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/training_set.yaml
end

JobCollection /home/hellstrom/adfhome/scripting/scm/params/examples/LJ_Ar/job_collection.yaml

NonExistingKey 3.14159

Delete a block or key¶

To delete an entry from the Settings, use del:

del job.settings.input.NonExistingKey

Attributes for easier setup of .yaml files¶

ParAMSJob has some special attributes which makes it easier to set up the settings.

job = ParAMSJob()
job.job_collection = "my_job_collection.yaml"  # will be converted to absolute path if it exists
job.training_set = "my_training_set.yaml"  # will be converted to absolute path if it exists
print(job.get_input())

DataSet
  Name training_set
  path my_training_set.yaml
end

JobCollection my_job_collection.yaml

Note that job.training_set and job.validation_set are quite special: when you assign a string to them as above, it will set the corresponding path in the settings. But when you read them you will get the corresponding Settings block:

print(job.training_set)
print(type(job.training_set))

Name:   training_set
path:   my_training_set.yaml

<class 'scm.plams.core.settings.Settings'>

print(job.training_set.path)
print(type(job.training_set.path))

my_training_set.yaml
<class 'str'>

Assigning to job.validation_set will create another item in the job.settings.input.DataSet list:

job.validation_set = "validation_set.yaml"
print(job.get_input())

DataSet
  Name training_set
  path my_training_set.yaml
end
DataSet
  Name validation_set
  path validation_set.yaml
end

JobCollection my_job_collection.yaml

To set other settings for the training set or validation set, use the standard dot-notation:

job.validation_set.EvaluateEvery = 100
job.settings.input.LoggingInterval.General = 100
print(job.get_input())

DataSet
  Name training_set
  path my_training_set.yaml
end
DataSet
  EvaluateEvery 100
  Name validation_set
  path validation_set.yaml
end

JobCollection my_job_collection.yaml

LoggingInterval
  General 100
end

You can also use the job.parameter_interface and job.engine_collection in the same way as job.job_collection:

job = ParAMSJob()
job.parameter_interface = (
    "my_parameter_interface.yaml"  # will be converted to absolute path if it exists
)
job.engine_collection = (
    "my_engine_collection.yaml"  # will be converted to absolute path if it exists
)
print(job.get_input())
# note: job.training_set is always defined, this is why a DataSet block is printed below

ParameterInterface my_parameter_interface.yaml

DataSet
  Name training_set
end

EngineCollection my_engine_collection.yaml

Functions for recurring blocks: Optimizers, Stoppers, ExitConditions¶

Use the below functions to easily add optimizers, stoppers, or exit conditions:

job = ParAMSJob()

job.add_exit_condition("MaxTotalFunctionCalls", 100000)
job.add_exit_condition("TimeLimit", 24 * 60 * 60)
job.add_exit_condition("StopsAfterConvergence", {"OptimizersConverged": 3, "OptimizersStopped": 1})

job.add_optimizer("CMAES", {"Sigma0": 0.01, "PopSize": 8})
job.add_optimizer("Scipy")

job.add_stopper("BestFunctionValueUnmoving", {"Tolerance": 0.1})
job.add_stopper("MaxFunctionCalls", 1000)

print(job.get_input())

DataSet
  Name training_set
end

ExitCondition
  MaxTotalFunctionCalls 100000
  Type MaxTotalFunctionCalls
end
ExitCondition
  TimeLimit 86400
... output trimmed ....
    Tolerance 0.1
  End
  Type BestFunctionValueUnmoving
end
Stopper
  MaxFunctionCalls 1000
  Type MaxFunctionCalls
end

To delete an added recurring block, use pop together with zero-based indices:

job.settings.input.ExitCondition.pop(1)  # 2nd exit condition
job.settings.input.Optimizer.pop(1)  # 2nd optimizer
job.settings.input.Stopper.pop(0)  # first stopper
print(job.get_input())

DataSet
  Name training_set
end

ExitCondition
  MaxTotalFunctionCalls 100000
  Type MaxTotalFunctionCalls
end
ExitCondition
  StopsAfterConvergence
... output trimmed ....
  End
  Type CMAES
end

Stopper
  MaxFunctionCalls 1000
  Type MaxFunctionCalls
end

Note: the ExitConditionBooleanCombination and StopperBooleanCombination work with indices starting with 1.

Python Script¶

#!/usr/bin/env python
# coding: utf-8

# ## Overview
#
# This example shows how to configure a ``ParAMSJob`` directly in Python, load an existing
# ``params.in`` file, and initialize a job from stored ParAMS YAML files.

# Call the ``get_input()`` function to see what the ``params.in`` file will look like.
#
# Note:
#
# * All paths must be absolute paths. This is needed to be able to run the job through PLAMS
# * ``DataSet`` and ``Optimizer`` are *recurring blocks*, so they are initialized as lists. The ``DataSet[0]`` syntax means that the settings are set for the *first* dataset, etc.

from scm.plams import *
from scm.params import *
import os

job = ParAMSJob()
job.settings.input.Task = "Optimization"
job.settings.input.JobCollection = "/path/job_collection.yaml"  # absolute path
job.settings.input.DataSet = [Settings(), Settings()]  # DataSet is a recurring block
job.settings.input.DataSet[0].Name = "training_set"
job.settings.input.DataSet[0].Path = "/path/training_set.yaml"  # absolute path
job.settings.input.DataSet[1].Name = "validation_set"
job.settings.input.DataSet[1].Path = "/path/validation_set.yaml"  # absolute path
job.settings.input.LoggingInterval.General = 10
job.settings.input.SkipX0 = "No"  # Booleans are specified as strings "Yes" or "No"
job.settings.input.Optimizer = [Settings()]  # Optimizer is a recurring block
job.settings.input.Optimizer[0].Type = "CMAES"
job.settings.input.Optimizer[0].CMAES.Sigma0 = 0.01
job.settings.input.Optimizer[0].CMAES.Popsize = 8
job.settings.input.ParallelLevels.Optimizations = 1  # ParallelLevels is NOT a recurring block

print(job.get_input())


# ### Load a job from a params.in file
#
# If you already have a ``params.in`` file (for example created by the GUI or by hand), you can simply load it into a ``ParAMSJob`` using ``from_inputfile()``.
#
# Note that any paths in the ``params.in`` file get converted to absolute paths.

params_in_file = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar/params.in")
job = ParAMSJob.from_inputfile(params_in_file)
print(job.get_input())


# ### Create a ParAMSJob from a directory with .yaml files
#
# In the input file you need to specify many paths to different .yaml files. This can be tedious to set up manually. If you have a directory with .yaml files (e.g. the ``jobname.params`` directory created by the GUI), you can initialize a ParAMSJob to read those yaml files using ``from_yaml()``. The files need to have the default names:
#
#
#
# * job_collection.yaml
# * training_set.yaml
# * validation_set.yaml
# * job_collection_engines.yaml or engine_collection.yaml
# * parameter_interface.yaml or parameters.yaml
#
# Note 1: When you run a ParAMSJob, any .yaml files in the current working directory will be used if they have the default names and if the corresponding settings are unset. In this way, you do not need to specify the paths in the ``settings`` if you have the .yaml files in the same directory as the script ``.py`` file that runs the job.
#
# Note 2: ``from_yaml()`` only sets the settings for the yaml files and leaves all other settings empty. ``from_inputfile()`` reads all the settings from the params.in file.

job = ParAMSJob.from_yaml(os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar"))
print(job.get_input())


# ### Input validation

# The allowed settings blocks, keys, and values are described in the documentation. If you make a mistake in the block or key names, ``get_input()`` will raise an error:

job.settings.input.NonExistingKey = 3.14159
try:
    print(job.get_input())
except Exception as e:
    print(e)


# If you want to print the input anyway, use ``get_input(validate=False)``:

print(job.get_input(validate=False))  # print the input anyway


# ### Delete a block or key

# To delete an entry from the Settings, use ``del``:

del job.settings.input.NonExistingKey


# ### Attributes for easier setup of .yaml files

# ``ParAMSJob`` has some special attributes which makes it easier to set up the settings.

job = ParAMSJob()
job.job_collection = "my_job_collection.yaml"  # will be converted to absolute path if it exists
job.training_set = "my_training_set.yaml"  # will be converted to absolute path if it exists
print(job.get_input())


# Note that ``job.training_set`` and ``job.validation_set`` are quite special: when you assign a string to them as above, it will set the corresponding ``path`` in the ``settings``. But when you read them you will get the corresponding Settings block:

print(job.training_set)
print(type(job.training_set))


print(job.training_set.path)
print(type(job.training_set.path))


# Assigning to ``job.validation_set`` will create another item in the ``job.settings.input.DataSet`` list:

job.validation_set = "validation_set.yaml"
print(job.get_input())


# To set other settings for the training set or validation set, use the standard dot-notation:

job.validation_set.EvaluateEvery = 100
job.settings.input.LoggingInterval.General = 100
print(job.get_input())


# You can also use the ``job.parameter_interface`` and ``job.engine_collection`` in the same way as ``job.job_collection``:

job = ParAMSJob()
job.parameter_interface = "my_parameter_interface.yaml"  # will be converted to absolute path if it exists
job.engine_collection = "my_engine_collection.yaml"  # will be converted to absolute path if it exists
print(job.get_input())
# note: job.training_set is always defined, this is why a DataSet block is printed below


# ### Functions for recurring blocks: Optimizers, Stoppers, ExitConditions
#
# Use the below functions to easily add optimizers, stoppers, or exit conditions:

job = ParAMSJob()

job.add_exit_condition("MaxTotalFunctionCalls", 100000)
job.add_exit_condition("TimeLimit", 24 * 60 * 60)
job.add_exit_condition("StopsAfterConvergence", {"OptimizersConverged": 3, "OptimizersStopped": 1})

job.add_optimizer("CMAES", {"Sigma0": 0.01, "PopSize": 8})
job.add_optimizer("Scipy")

job.add_stopper("BestFunctionValueUnmoving", {"Tolerance": 0.1})
job.add_stopper("MaxFunctionCalls", 1000)

print(job.get_input())


# To delete an added recurring block, use ``pop`` together with zero-based indices:

job.settings.input.ExitCondition.pop(1)  # 2nd exit condition
job.settings.input.Optimizer.pop(1)  # 2nd optimizer
job.settings.input.Stopper.pop(0)  # first stopper
print(job.get_input())


# Note: the ``ExitConditionBooleanCombination`` and ``StopperBooleanCombination`` work with indices starting with 1.

Set Up ParAMSJob Settings¶

Overview¶

Load a job from a params.in file¶

Create a ParAMSJob from a directory with .yaml files¶

Input validation¶

Delete a block or key¶

Attributes for easier setup of .yaml files¶

Functions for recurring blocks: Optimizers, Stoppers, ExitConditions¶

See also¶

Related examples¶

Related tutorials¶

Related documentation¶

Python Script¶