5.3. Generate reference values

In the previous tutorials, reference data was either:

  • pre-calculated and provided as part of the tutorial,

  • or, we generated the training data by setting up a job in AMSinput and importing the results in ParAMS.

You can also use ParAMS to calculate the reference values as part of the parametrization. An example is illustrated in this tutorial.

This can be useful if you, for example:

  • want to re-evaluate (parts of) the training set with a different reference method,

  • have added a reaction energy without a reference value, or

  • only have a few reference jobs that are relatively quick to calculate.

../../_images/LJ_Ar_no_reference_data_toc.png

Note

In most situations, it is better to first run the reference calculations and then import them into ParAMS (i.e., by following the Import training data (GUI) or Import training data (Python) tutorials). That way you can:

  • run the reference calculations on multiple nodes,

  • store the reference calculations and results in arbitrary places on disk,

  • inspect the reference results and correct any errors before the parametrization, and

  • use the results importer for PESScans which generate plots.

Reference value generation is ParAMS is primarily intended for re-parametrization of existing models, or when using small datasets / fast reference methods.

5.3.1. Prerequisites

  • Go through the Getting Started: Lennard-Jones tutorial.

  • Make a copy of the example directory $AMSHOME/scripting/scm/params/examples/LJ_Ar_no_reference_data.

5.3.2. The input files

5.3.2.1. Training set without reference values

Open the ParAMS GUI: SCM → ParAMS
File → Open the job_collection.yaml file in the example directory
Switch to the Training Set panel
Note that the Values column is empty: There are no reference values!
../../_images/noref_initial_view.png

The reference values will be calculated when you start the parametrization.

Unlike the training_set.yaml file in the previous tutorial, this example’s training_set.yaml does not contain any reference values:

---
dtype: DataSet
version: '2022.101'
---
Expression: energy('Ar32_frame001')-energy('Ar32_frame002')
Weight: 1.0
Sigma: 0.054422772491975996
Unit: eV, 27.211386245988
---
Expression: energy('Ar32_frame003')-energy('Ar32_frame002')
Weight: 1.0
Sigma: 0.054422772491975996
Unit: eV, 27.211386245988
---
Expression: forces('Ar32_frame001')
Weight: 1.0
Sigma: 0.15426620242897765
Unit: eV/angstrom, 51.422067476325886
---
Expression: forces('Ar32_frame002')
Weight: 1.0
Sigma: 0.15426620242897765
Unit: eV/angstrom, 51.422067476325886
---
Expression: forces('Ar32_frame003')
Weight: 1.0
Sigma: 0.15426620242897765
Unit: eV/angstrom, 51.422067476325886
...

This tutorial shows you how you can calculate the reference values (in units of Unit) with params.

5.3.2.2. Jobs and Engines

All Jobs that are mentioned in training set entries without a reference value, will be calculated before the parametrization starts with the help of a Reference Engine.

Switch to the Jobs panel
The Reference Engine column contains entries with dftb;;kspace;;quality;GammaOnly;model;GFN1-xTB;.
../../_images/noref_initial_jobs.png

This rather cryptic ID is a reference engine id.

Switch to the Engines panel
../../_images/noref_initial_engines.png
Double-click in the Detail column where it says Engine dftb   kspace .......
../../_images/noref_engine_settings.png

You then see the Engine settings in the top box. If you are familiar with this type of input, you can directly edit it in the text box.

Below, you see the Engine ID. Change the ID to something that is easier to read, for example GFN1xTB_GammaPoint.
../../_images/noref_rename_engine.png
Click OK
This updates the Engine ID in the table
../../_images/noref_renamed_engine_engines_panel.png

Modifications to the engine will affect all jobs with that particular reference engine.

To change the engine that is used for a job:

Switch back to the Jobs panel
../../_images/noref_renamed_engine_jobs_panel.png
The column with reference engines has been updated to contain the ID you chose.
Double-click on the Ar32_frame002 job in the Detail column (where it says SinglePoint + gradients)
../../_images/noref_job_details_before_change.png
Click the AMSinput button next to Reference Engine.
This brings up an AMSinput window on the DFTBPanel panel (if you do not have a DFTB license, choose an engine for which you have a valid license).
The engine settings that will be used for the job are shown. For example, KSpace is set to GammaOnly.
You can change the DFTB settings. For example, set KSpace to Basic and Occupation to Fermi
../../_images/noref_amsinput_settings.png
Close the AMSinput window
In the dialog “Pass to ParAMS?” click Yes
A window informs that you that related reference values will be cleared. In this tutorial there are no reference values yet anyway. Click OK
This creates a new reference engine GFN1xTB;2526569033. The numbers are a hash of the settings.
../../_images/noref_added_engine.png
The Ar32_frame002 job has the new reference engine. You can rename it on the Engines panel if you prefer.

In this way, you can create an arbitrary number of engines, or choose already-created engines.

Switch to the Jobs panel
Double-click on the Ar32_frame002 job in the Detail column (where it says SinglePoint + gradients)
../../_images/noref_added_engine_job_details.png
In the Reference Engine drop-down, choose the original reference engine (GFN1xTB_GammaPoint).
../../_images/noref_reset_engine.png
Click OK
Verify that all jobs have the GFN1xTB_GammaPoint engine

Unlike the previous tutorial, here each entry in the job_collection.yaml has a ReferenceEngineID. For example, the first entry is

 1---
 2Engines: job_collection_engines.yaml
 3dtype: JobCollection
 4version: '2022.101'
 5---
 6ID: 'Ar32_frame001'
 7ReferenceEngineID: dftb;;kspace;;quality;GammaOnly;model;GFN1-xTB;
 8AMSInput: |
 9   properties
10     gradients Yes
11   End
12   system
13     Atoms
14                Ar       5.1883477539      -0.4887475488       7.9660568076 
15                Ar       5.7991822399       0.4024595652       2.5103286966 
16                Ar       6.1338265157       5.5335946219       7.0874208384 
17                Ar       4.6137188191       5.9644505949       3.0942810122 
18                Ar       8.4186778390       7.6292969115       8.0729664423 
19                Ar       8.3937816110       8.6402371806       2.6057806799 
20                Ar       7.5320205143       1.7666481606       7.7525889818 
21                Ar       8.5630139885       2.0472039529       2.6380554086 
22                Ar       2.6892353632       7.8435284207       7.7883054306 
23                Ar       2.4061636915       7.5716025415       2.4535180075 
24                Ar       2.2485171283       2.9764130946       7.8589298904 
25                Ar       3.0711058946       1.8500587164       2.5620921469 
26                Ar       7.6655637500      -0.4865893003       0.0018797080 
27                Ar       7.7550067215      -0.0222821825       4.8528637785 
28                Ar       7.7157262425       4.6625079517      -0.3861722152 
29                Ar       7.7434900996       5.2619590353       4.2602386226 
30                Ar       3.4302237084      -0.2708640738       0.6280466620 
31                Ar       2.8648051689       0.6106220610       6.1208342905 
32                Ar       3.2529823775       5.7151788324      -0.2024448179 
33                Ar       2.0046357208       4.9353027402       5.4968740217 
34                Ar       0.9326855213       8.0600564695      -0.3181225099 
35                Ar      -0.5654205469       8.5703446434       5.8930973456 
36                Ar      -0.9561293133       2.1098403312      -0.0052667919 
37                Ar      -0.8081417664       3.2747992855       5.5295389610 
38                Ar       5.5571960244       7.5645919074       0.1312355350 
39                Ar       4.4530832384       7.6170633330       5.4810860433 
40                Ar       5.1235367625       2.7983577675      -0.3161069611 
41                Ar       5.2048439076       2.9885672135       4.5193274119 
42                Ar      -0.2535891591       0.0134355189       8.3061692970 
43                Ar       0.5614183785      -0.1927751317       3.2355155467 
44                Ar      -0.0234943080       5.0313863031       8.0451075074 
45                Ar      -0.4760138873       6.2617510830       2.5759742219 
46     End
47     Lattice
48           10.5200000000     0.0000000000     0.0000000000
49            0.0000000000    10.5200000000     0.0000000000
50            0.0000000000     0.0000000000    10.5200000000

The ReferenceEngineID refers to an engine in job_collection_engines.yaml, which is an Engine Collection. Each entry has a unique ID, and an AMSInput block containing calculation settings for the AMS engine.

---
Jobs: job_collection.yaml
dtype: EngineCollection
version: '2022.101'
---
ID: 'dftb;;kspace;;quality;GammaOnly;model;GFN1-xTB;'
AMSInput: |
   Engine dftb
     kspace
       quality GammaOnly
     End
     model GFN1-xTB
   EndEngine
...

In this example, there is only one reference engine. It has the ID dftb;;kspace;;quality;GammaOnly;model;GFN1-xTB;. The ID could be any string. It does not affect the results, but should describe the reference engine. Each job in the job collection with this ReferenceEngineID will be evaluated with this reference engine.

The AMSInput affects the calculation. In this example, it sets up a GFN1-xTB engine with Γ-point sampling. The AMSInput will be added verbatim to the input to the reference job.

Note

If you do not have a DFTB license, change the Engine block in job_collection_engines.yaml to

Engine ForceField
    Type UFF
EndEngine

to instead use a UFF force field as the reference method.

5.3.3. Calculate the reference values

Select the Generate Reference task:

Click OptimizationPanel to open the input panel
Click OptimizationPanel again to open the dropdown of available tasks
Select Generate Reference

Save and run the task:

File → Save As
Save with the name calc_ref_values.params
File → Run. This brings up AMSjobs
../../_images/noref_running.png

Once complete the GUI will automatically load the reference values:

Switch back to the ParAMS window
On the Training Set panel, the reference values have been added
../../_images/noref_added_refvalues.png

Since the reference data has been automatically loaded, you may now immediately continue to set up an optimization. You can switch the task back to Optimization, change any settings you like (for details see Getting Started: Lennard-Jones), and then save the job. The reference values will be saved within the new job folder as normal.

To run the reference jobs and generate the reference data, change to the example directory and run:

"$AMSBIN/params"

This runs the reference calculations, adds the reference data to the training set, and saves it in results/training_set.ref.yaml

To use the results in an optimization simply set one up as described in Getting Started: Lennard-Jones and ensure that the DataSet%Path key points to results/training_set.ref.yaml or a copy thereof.

5.3.4. Output files for reference calculations and data

In the results folder, you will find:

  • the individual job results saved in results/reference_jobs

  • the task input saved in results/settings_and_initial_data

  • a file training_set.ref.yaml containing the reference values.

5.3.4.1. The reference_jobs folder

For example, the file reference_jobs/Ar32_frame001/Ar32_frame001.in contains the input to the Ar32_frame001 job, which combines input from the job collection and engine collection:

properties
  gradients yes
End

system
  Atoms
             Ar       5.1883477539      -0.4887475488       7.9660568076 
             Ar       5.7991822399       0.4024595652       2.5103286966 
             Ar       6.1338265157       5.5335946219       7.0874208384 
             Ar       4.6137188191       5.9644505949       3.0942810122 
             Ar       8.4186778390       7.6292969115       8.0729664423 
             Ar       8.3937816110       8.6402371806       2.6057806799 
             Ar       7.5320205143       1.7666481606       7.7525889818 
             Ar       8.5630139885       2.0472039529       2.6380554086 
             Ar       2.6892353632       7.8435284207       7.7883054306 
             Ar       2.4061636915       7.5716025415       2.4535180075 
             Ar       2.2485171283       2.9764130946       7.8589298904 
             Ar       3.0711058946       1.8500587164       2.5620921469 
             Ar       7.6655637500      -0.4865893003       0.0018797080 
             Ar       7.7550067215      -0.0222821825       4.8528637785 
             Ar       7.7157262425       4.6625079517      -0.3861722152 
             Ar       7.7434900996       5.2619590353       4.2602386226 
             Ar       3.4302237084      -0.2708640738       0.6280466620 
             Ar       2.8648051689       0.6106220610       6.1208342905 
             Ar       3.2529823775       5.7151788324      -0.2024448179 
             Ar       2.0046357208       4.9353027402       5.4968740217 
             Ar       0.9326855213       8.0600564695      -0.3181225099 
             Ar      -0.5654205469       8.5703446434       5.8930973456 
             Ar      -0.9561293133       2.1098403312      -0.0052667919 
             Ar      -0.8081417664       3.2747992855       5.5295389610 
             Ar       5.5571960244       7.5645919074       0.1312355350 
             Ar       4.4530832384       7.6170633330       5.4810860433 
             Ar       5.1235367625       2.7983577675      -0.3161069611 
             Ar       5.2048439076       2.9885672135       4.5193274119 
             Ar      -0.2535891591       0.0134355189       8.3061692970 
             Ar       0.5614183785      -0.1927751317       3.2355155467 
             Ar      -0.0234943080       5.0313863031       8.0451075074 
             Ar      -0.4760138873       6.2617510830       2.5759742219 
  End
  Lattice
        10.5200000000     0.0000000000     0.0000000000
         0.0000000000    10.5200000000     0.0000000000
         0.0000000000     0.0000000000    10.5200000000
  End
End

task singlepoint

Engine dftb
  kspace
    quality GammaOnly
  End
  model GFN1-xTB
EndEngine

The normal AMS output can be found in the same folder: the logfile, standard outputfile, and the binary ams.rkf and dftb.rkf files.

5.3.4.2. The training_set.ref.yaml file

The training_set.ref.yaml file contains the calculated reference values. For example, it starts with

---
dtype: DataSet
version: '2022.101'
---
Expression: energy('Ar32_frame001')-energy('Ar32_frame002')
Weight: 1.0
Sigma: 0.054422772491975996
ReferenceValue: 0.20395942701637979
Unit: eV, 27.211386245988
---
Expression: energy('Ar32_frame003')-energy('Ar32_frame002')
Weight: 1.0
Sigma: 0.054422772491975996
ReferenceValue: 0.22060005303998803
Unit: eV, 27.211386245988
---
Expression: forces('Ar32_frame001')
Weight: 1.0
Sigma: 0.15426620242897765
ReferenceValue: |

These are the same reference values that were used in the Getting Started: Lennard-Jones tutorial.

5.3.5. Generate reference values in Python

See run.py, where first the GenerateReference job is run, and then the resulting reference values are used to run an optimization. See also ParAMSJob and ParAMSResults.

#!/usr/bin/env amspython
from scm.plams import *
from scm.params import *
import os


def main():
    init()

    inputfile = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar_no_reference_data/params.in")
    job = ParAMSJob.from_inputfile(inputfile, name="genref")
    job.run()

    # use all settings from the LJ_Ar example except for the path to the training set
    inputfile = os.path.expandvars("$AMSHOME/scripting/scm/params/examples/LJ_Ar/params.in")
    opt_job = ParAMSJob.from_inputfile(inputfile, name="optimization")
    opt_job.training_set = job.results.get_training_set_ref_path()
    opt_job.run()

    loss = opt_job.results.get_loss(source="latest")
    print(f"Final loss: {loss}")

    finish()


if __name__ == "__main__":
    main()