Conformers: Active learning with CREST metadynamics and custom addition of data points¶
Trained model: M3GNet, starting from the Universal Potential (UP)
Reference method: GFN-1xTB (DFTB engine)
System: Organic molecule with SMILES code OC(CC1c2ccccc2Sc2ccccc21)CN1CCCC1
Problem: M3GNet-UP-2022 is not reliable for conformer stability prediction:
Solution: Retraining the model gives better agreement:
Expected duration: This notebook takes about 24 hours to run on the CPU. This includes both the active learning and the production simulations to generate new conformers.
To follow along, either
Download
conformers_training.py(run as$AMSBIN/amsipython conformers_training.py).Download
conformers_training.ipynb(see also: how to install Jupyterlab in AMS)
When running active learning it’s usually a good idea to start off “simple” and make the system/structures gradually more complicated.
For getting a model which predicts conformers accurately, we may take the following approach:
first train a potential at slightly above room temperature with NVT MD
continue training a second potential using CREST metadynamics
generate conformers with the previous model and also train to those
Initial imports¶
import scm.plams as plams
import scm.params as params
from scm.simple_active_learning import SimpleActiveLearningJob
import matplotlib.pyplot as plt
from scm.external_engines.core import interface_is_installed
import os
import numpy as np
from typing import List
from scm.conformers import ConformersJob
from scm.conformers.plams.plot import plot_conformers
assert interface_is_installed(
"m3gnet"
), "You must install the m3gnet backend before following this tutorial!"
plams.init()
PLAMS working folder: /path/plams_workdir
Create the initial structure¶
molecule = plams.from_smiles("OC(CC1c2ccccc2Sc2ccccc21)CN1CCCC1", forcefield="uff")
molecule.delete_all_bonds()
for at in molecule:
at.properties = plams.Settings()
plams.plot_molecule(molecule)
starting_structure = molecule
Reference engine settings¶
fast_ref_s = plams.Settings()
fast_ref_s.input.DFTB.Model = "GFN1-xTB"
slow_ref_s = plams.Settings()
slow_ref_s.input.ADF.Basis.Type = "TZP"
slow_ref_s.input.ADF.Basis.Core = "None"
slow_ref_s.input.ADF.XC.Hybrid = "B3LYP"
slow_ref_s.input.ADF.XC.DISPERSION = "GRIMME3 BJDAMP"
Change to slow_ref_s to train to B3LYP-D3(BJ) instead:
ref_s = fast_ref_s.copy()
# ref_s = slow_ref_s.copy()
perform_expensive_tests = ref_s == fast_ref_s
Problem statement: Generate a few conformers with M3GNet-UP-2022 and Score with reference method¶
m3gnet_up_s = plams.Settings()
m3gnet_up_s.input.MLPotential.Model = "M3GNet-UP-2022"
generate_conformers_m3gnet_up_job = ConformersJob(
name="generate_conformers_m3gnet_up", molecule=starting_structure
)
generate_conformers_m3gnet_up_job.settings.input.ams.Task = "Generate"
generate_conformers_m3gnet_up_job.settings.input.ams.Generator.Method = "RDKit"
generate_conformers_m3gnet_up_job.settings.input.ams.Generator.RDKit.InitialNConformers = 40
# generate_conformers_m3gnet_up_job.settings += crest_al_job.results.get_production_engine_settings()
generate_conformers_m3gnet_up_job.settings += m3gnet_up_s
# generate_conformers_m3gnet_up_job.settings.runscript.nproc = 1 # run in serial, useful if you run out of memory when running M3GNet on the GPU
generate_conformers_m3gnet_up_job.run();
[13.03|09:15:30] JOB generate_conformers_m3gnet_up STARTED
[13.03|09:15:30] JOB generate_conformers_m3gnet_up RUNNING
[13.03|09:17:10] JOB generate_conformers_m3gnet_up FINISHED
[13.03|09:17:10] JOB generate_conformers_m3gnet_up SUCCESSFUL
Show the 4 most stable conformers with M3GNet-UP-2022:
plot_conformers(generate_conformers_m3gnet_up_job, 4, unit="eV");
Score the generated conformers with the reference method:
score_conformers_ref_job = ConformersJob(name="score_conformers_ref")
score_conformers_ref_job.settings.input.ams.Task = "Score"
score_conformers_ref_job.settings.input.ams.InputConformersSet = (
generate_conformers_m3gnet_up_job.results.rkfpath()
)
score_conformers_ref_job.settings.input += ref_s.input
score_conformers_ref_job.run();
[13.03|09:17:10] JOB score_conformers_ref STARTED
[13.03|09:17:10] JOB score_conformers_ref RUNNING
[13.03|09:17:16] JOB score_conformers_ref FINISHED
[13.03|09:17:16] JOB score_conformers_ref SUCCESSFUL
plot_conformers(score_conformers_ref_job, 4, unit="eV");
Here we can see that the ordering of conformers at the reference level is completely different compared to M3GNet-UP-2022.
The Score job reorders the conformers. To compare the energies more precisely, we can use the minimum pairwise RMSD (which should be 0) to identify how the order of conformers change:
def get_pairwise_rmsds(
molecules_ref: List[plams.Molecule], molecules_pred: List[plams.Molecule]
) -> np.ndarray:
"""Returns a len(molecules_ref)*len(molecules_pred) numpy array with pairwise rmsds (in angstrom) between the structures"""
rmsds = np.zeros((len(molecules_ref), len(molecules_pred)))
for i, ref_mol in enumerate(molecules_ref):
for j, pred_mol in enumerate(molecules_pred):
rmsds[i, j] = plams.Molecule.rmsd(ref_mol, pred_mol)
return rmsds
def get_reordering(rmsds: np.ndarray) -> np.ndarray:
"""
rmsds: numpy array with shape len(molecules_ref)*len(molecules_pred)
Returns a len(molecules_ref) integer numpy array.
The first element is the index (0-based) in molecules_pred corresponding to the first reference molecule, etc.
"""
rmsds = get_pairwise_rmsds(molecules_ref, molecules_pred)
reordering = np.argmin(rmsds, axis=1)
return reordering
def print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred, ax=None):
"""This functions prints the reordering table including relative energies. It also plots the predicted relative energies versus the reference relative energies"""
print(f"Ref_i Pred_i RMSD Ref_dE Pred_dE")
x, y = [], []
rmsds = get_pairwise_rmsds(molecules_ref, molecules_pred)
reordering = get_reordering(rmsds)
rmsd_threshold = 0.7 # angstrom, for printing/plotting points
for ref_i, pred_i in enumerate(reordering):
rmsd = rmsds[ref_i, pred_i]
ref_relative_e = energies_ref[ref_i]
pred_relative_e = energies_pred[pred_i] - energies_pred[reordering[0]]
if rmsd <= rmsd_threshold:
print(
f"{ref_i:6d} {pred_i:6d} {rmsd:4.1f} {ref_relative_e:7.2f} {pred_relative_e:7.2f}"
)
x.append(ref_relative_e)
y.append(pred_relative_e)
else:
print(f"{ref_i:6d} {pred_i:6d} rmsd > {rmsd_threshold:.1f} ang.")
if ax is None:
_, ax = plt.subplots()
m, M = np.min([np.min(x), np.min(y)]), np.max([np.max(x), np.max(y)])
ax.plot(x, y, ".")
ax.plot([m, M], [m, M], "-")
ax.set_xlabel("ref. deltaE (eV)")
ax.set_ylabel("pred. deltaE (eV)")
return ax
In the below table the energies are given with respect to the structure that had the lowest reference energy. This is thus different compared to the first image above, where the predicted (m3gnet-up-2022) relative energies were given to predicted lowest energy conformer.
molecules_ref = score_conformers_ref_job.results.get_conformers()
energies_ref = score_conformers_ref_job.results.get_relative_energies(unit="eV")
molecules_pred = generate_conformers_m3gnet_up_job.results.get_conformers()
energies_pred = generate_conformers_m3gnet_up_job.results.get_relative_energies(unit="ev")
ax = print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred)
ax.set_title("M3GNet-UP-2022 relative to reference method");
Ref_i Pred_i RMSD Ref_dE Pred_dE
0 5 0.0 0.00 0.00
1 3 0.0 0.08 -0.01
2 19 0.0 0.14 0.07
3 20 0.0 0.15 0.08
4 7 0.0 0.18 0.02
5 2 0.0 0.21 -0.04
6 6 0.0 0.22 0.01
7 18 0.0 0.26 0.07
8 15 0.0 0.30 0.05
9 0 0.0 0.31 -0.07
10 17 0.0 0.31 0.07
11 14 0.0 0.35 0.04
12 23 0.0 0.36 0.12
13 21 0.0 0.37 0.08
14 22 0.0 0.37 0.09
15 4 0.0 0.39 -0.01
16 10 0.0 0.39 0.02
17 11 0.0 0.42 0.03
18 9 0.0 0.48 0.02
19 16 0.0 0.50 0.06
20 12 0.0 0.56 0.04
21 24 0.0 0.60 0.17
22 8 0.0 0.68 0.02
23 13 0.0 0.75 0.04
24 1 0.0 0.97 -0.05
Here we see that there is no correlation between the relative stabilities calculated with M3GNet-UP-2022 and the reference method.
Example: The most stable conformer with the reference method (Ref_i=0) corresponds to the tenth most stable conformer with M3GNet-UP-2022 (Pred_i=10)
Example 2: The second most stable conformer with the reference method (Ref_i=1) should be 0.08 eV less stable than the ref_i=0 conformer, but with M3GNet-UP-2022 method it is actually 0.01 eV more stable!
Optimize a few conformers for initial reference data¶
Conformers are local minima on the potential energy surface. The Active Learning MD will sample off-equilibrium structures. So let’s make sure that there are at least some local minima in the training set by optimizing some of the generated conformers.
Here, we loop over the conformers and run GeometryOptimization jobs. This produces output files in the normal AMS format that can easily be imported into ParAMS
max_N = min(6, len(molecules_pred)) # at most 6 optimizations
opt_s = plams.Settings()
opt_s.input.ams.Task = "GeometryOptimization"
opt_s.input.ams.GeometryOptimization.Convergence.Quality = "Basic"
opt_jobs = [
plams.AMSJob(settings=opt_s + ref_s, name=f"opt_{i}", molecule=mol)
for i, mol in enumerate(molecules_pred[:max_N])
]
for opt_job in opt_jobs:
opt_job.run()
[13.03|09:17:18] JOB opt_0 STARTED
[13.03|09:17:18] JOB opt_0 RUNNING
[13.03|09:17:28] JOB opt_0 FINISHED
[13.03|09:17:28] JOB opt_0 SUCCESSFUL
[13.03|09:17:28] JOB opt_1 STARTED
[13.03|09:17:28] JOB opt_1 RUNNING
[13.03|09:17:41] JOB opt_1 FINISHED
[13.03|09:17:41] JOB opt_1 SUCCESSFUL
[13.03|09:17:41] JOB opt_2 STARTED
[13.03|09:17:41] JOB opt_2 RUNNING
[13.03|09:17:47] JOB opt_2 FINISHED
[13.03|09:17:47] JOB opt_2 SUCCESSFUL
[13.03|09:17:47] JOB opt_3 STARTED
[13.03|09:17:47] JOB opt_3 RUNNING
[13.03|09:17:54] JOB opt_3 FINISHED
[13.03|09:17:55] JOB opt_3 SUCCESSFUL
[13.03|09:17:55] JOB opt_4 STARTED
[13.03|09:17:55] JOB opt_4 RUNNING
[13.03|09:18:08] JOB opt_4 FINISHED
[13.03|09:18:08] JOB opt_4 SUCCESSFUL
[13.03|09:18:08] JOB opt_5 STARTED
[13.03|09:18:08] JOB opt_5 RUNNING
[13.03|09:18:14] JOB opt_5 FINISHED
[13.03|09:18:14] JOB opt_5 SUCCESSFUL
Now import the data into a ParAMS results importer and store in the
directory my_initial_reference_data:
yaml_dir = os.path.abspath("my_initial_reference_data")
ri = params.ResultsImporter(settings={"units": {"energy": "eV", "forces": "eV/angstrom"}})
for opt_job in opt_jobs:
ri.add_singlejob(opt_job, properties=["energy", "forces"], task="SinglePoint")
ri.store(yaml_dir, backup=False)
['/home/user/testnb/conformers/my_initial_reference_data/job_collection.yaml',
'/home/user/testnb/conformers/my_initial_reference_data/results_importer_settings.yaml',
'/home/user/testnb/conformers/my_initial_reference_data/training_set.yaml']
Simple Active Learning setup¶
Molecular dynamics settings (temperature ramp)¶
nvt_md_s = plams.AMSNVTJob(
nsteps=20000,
timestep=0.5,
temperature=(270, 350, 350),
tau=100,
thermostat="Berendsen",
).settings
ParAMS machine learning settings¶
ml_s = plams.Settings()
ml_s.input.ams.MachineLearning.Backend = "M3GNet"
ml_s.input.ams.MachineLearning.CommitteeSize = 1
ml_s.input.ams.MachineLearning.M3GNet.Model = "UniversalPotential"
ml_s.input.ams.MachineLearning.MaxEpochs = 200
Active learning settings¶
Conformer search of a single molecule is quite simple, so we can expect the ML method to perform quite well. We therefore decrease the success criteria thresholds a bit compared to the default values, to ensure that we get accurate results.
Since we will immediately continue with another active learning loop, we disable the “RerunSimulation” as we are not interested in the MD simulation per se.
al_s = plams.Settings()
al_s.input.ams.ActiveLearning.Steps.Type = "Geometric"
al_s.input.ams.ActiveLearning.Steps.Geometric.Start = 10
al_s.input.ams.ActiveLearning.Steps.Geometric.NumSteps = 8
al_s.input.ams.ActiveLearning.InitialReferenceData.Load.Directory = yaml_dir
al_s.input.ams.ActiveLearning.InitialReferenceData.Generate.M3GNetShortMD.Enabled = "Yes"
al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Relative = 0.002
al_s.input.ams.ActiveLearning.SuccessCriteria.Forces.MaxDeviationForZeroForce = 0.30
al_s.input.ams.ActiveLearning.AtEnd.RerunSimulation = "No"
Initial structure¶
Let’s start with the lowest-energy conformer according to the reference method:
starting_structure_al = molecules_ref[0]
# The bonds are not used, so delete them to make the input file less confusing:
starting_structure_al.delete_all_bonds()
Complete job¶
settings = ref_s + nvt_md_s + ml_s + al_s
job = SimpleActiveLearningJob(settings=settings, molecule=starting_structure_al, name="sal")
print(job.get_input())
ActiveLearning
AtEnd
RerunSimulation False
End
InitialReferenceData
Generate
M3GNetShortMD
Enabled True
End
End
Load
Directory /home/user/testnb/conformers/my_initial_reference_data
End
End
Steps
Geometric
NumSteps 8
Start 10
End
Type Geometric
End
SuccessCriteria
Energy
Relative 0.002
End
Forces
MaxDeviationForZeroForce 0.3
End
End
End
MachineLearning
Backend M3GNet
CommitteeSize 1
M3GNet
Model UniversalPotential
End
MaxEpochs 200
End
MolecularDynamics
BinLog
DipoleMoment False
PressureTensor False
Time False
End
CalcPressure False
Checkpoint
Frequency 1000000
End
InitialVelocities
Temperature 270.0
Type Random
End
NSteps 20000
Thermostat
Duration 10000 10000
Tau 100.0
Temperature 270.0 350.0 350.0
Type Berendsen
End
TimeStep 0.5
Trajectory
SamplingFreq 100
WriteBonds True
WriteCharges True
WriteEngineGradients False
WriteMolecules True
WriteVelocities True
End
End
Task MolecularDynamics
Engine DFTB
Model GFN1-xTB
EndEngine
System
Atoms
O 1.4391780112 1.2169129235 -1.7054069200
C 1.5317092689 1.4035863111 -0.2938413718
C 0.1268560563 1.5694033539 0.2784574216
C -1.0118917223 0.8423026126 -0.4698246606
C -0.8595600851 -0.6658991850 -0.4667308109
C -0.2377365209 -1.3589913541 -1.5140191569
C -0.0600750557 -2.7425928290 -1.4776768275
C -0.4597592917 -3.4730429030 -0.3538527735
C -1.0904714542 -2.8195492532 0.6980164273
C -1.3258901173 -1.4413002818 0.6124763877
S -2.1263245435 -0.7063910252 1.9678587611
C -2.9671793611 0.5717690643 1.1430257653
C -4.2196660834 0.9342256470 1.6549843220
C -4.8874241675 2.0284840067 1.1177318046
C -4.2799714238 2.7745677848 0.1034338312
C -3.0261225271 2.4041998424 -0.3849495690
C -2.3576383467 1.2649839383 0.0830777170
C 2.3379956154 0.2559815681 0.3637145979
N 3.5335877858 -0.0348493319 -0.3803421395
C 4.5276364405 0.9770646676 -0.7067783761
C 5.8039959976 0.2245804369 -1.0606706500
C 5.5160036103 -1.2576104771 -0.8851251216
C 4.0862710617 -1.3773039698 -0.3681850712
H 2.2480451477 0.7255622089 -1.9424793652
H 2.1022898536 2.3275368122 -0.1065487649
H 0.1414799097 1.2581805099 1.3365407102
H -0.0919678354 2.6491225081 0.2950015300
H -0.9576926619 1.1682381587 -1.5255106842
H 0.1420308060 -0.7848172199 -2.3630630488
H 0.3961348030 -3.2525380401 -2.3295748047
H -0.2848438820 -4.5496996295 -0.2996394873
H -1.4160325939 -3.3529364566 1.5899014786
H -4.6473491024 0.3418629744 2.4625500438
H -5.8766900291 2.3003086826 1.4921385558
H -4.7831313928 3.6509218791 -0.3112339451
H -2.5490456322 3.0178426103 -1.1569355816
H 1.7230872667 -0.6569638072 0.4236796419
H 2.5727552504 0.5413965040 1.4124509436
H 4.1762166705 1.5873527730 -1.5520160700
H 4.6790314500 1.6618285869 0.1450036528
H 6.6295996825 0.5480905465 -0.4090915009
H 6.1038361613 0.4476381022 -2.0952671283
H 5.6230261611 -1.8000335231 -1.8359927555
H 6.2159448427 -1.7154693665 -0.1710768313
H 3.4863964364 -2.0479923169 -1.0050050526
H 4.0606739540 -1.7926476557 0.6541872002
End
End
Run the simple active learning job¶
job.run(watch=True);
[13.03|09:18:15] JOB sal STARTED
[13.03|09:18:15] JOB sal RUNNING
[13.03|09:18:17] Simple Active Learning 2024.101, Nodes: 1, Procs: 64
[13.03|09:18:21] Composition of main system: C20H23NOS
[13.03|09:18:21] All REFERENCE calculations will be performed with the following DFTB engine:
[13.03|09:18:21]
Engine dftb
model GFN1-xTB
EndEngine
[13.03|09:18:21] The following are the settings for the to-be-trained MACHINE LEARNING model:
[13.03|09:18:21]
MachineLearning
Backend M3GNet
CommitteeSize 1
M3GNet
Model UniversalPotential
End
MaxEpochs 200
End
ParallelLevels
End
[13.03|09:18:21] A single model will be trained (no committee).
[13.03|09:18:21] The ACTIVE LEARNING loop will contain 8 steps, using the following schema:
[13.03|09:18:21] Active Learning Step 1: 10 MD Steps (cumulative: 10)
[13.03|09:18:21] Active Learning Step 2: 19 MD Steps (cumulative: 29)
[13.03|09:18:21] Active Learning Step 3: 58 MD Steps (cumulative: 87)
[13.03|09:18:21] Active Learning Step 4: 172 MD Steps (cumulative: 259)
[13.03|09:18:21] Active Learning Step 5: 510 MD Steps (cumulative: 769)
[13.03|09:18:21] Active Learning Step 6: 1510 MD Steps (cumulative: 2279)
[13.03|09:18:21] Active Learning Step 7: 4473 MD Steps (cumulative: 6752)
[13.03|09:18:21] Active Learning Step 8: 13248 MD Steps (cumulative: 20000)
[13.03|09:18:21] Total number of MD Steps: 20000
[13.03|09:18:21] Max attempts per active learning step: 15
[13.03|09:18:21]
[13.03|09:18:21] The directory /home/user/testnb/conformers/my_initial_reference_data is of type RestartDirectoryType.YAML_DIR
[13.03|09:18:21] Loading yaml files (data sets) from /home/user/testnb/conformers/my_initial_reference_data
[13.03|09:18:22] Running M3GNet-UP-2022 structure samplers...
[13.03|09:18:22] Running initial_sampler_enhanced_md
[13.03|09:18:47] Running initial_reference_enhanced_md
[13.03|09:18:52] Done!
[13.03|09:18:52] Initial reference data has been created. Storing data in folder: initial_reference_data
[13.03|09:18:52] Running initial ParAMS training in folder: initial_training
[13.03|09:18:57] JOB m3gnet STARTED
[13.03|09:18:57] Starting m3gnet.prerun()
[13.03|09:18:57] m3gnet.prerun() finished
[13.03|09:18:57] JOB m3gnet RUNNING
[13.03|09:18:57] Executing m3gnet.run
[13.03|09:19:45] training_set Optimizer: 001 Epoch: 0 Loss: 0.092347
[13.03|09:19:45] validation_set Optimizer: 001 Epoch: 0 Loss: 0.157095
[13.03|09:19:49] training_set Optimizer: 001 Epoch: 10 Loss: 0.005975
[13.03|09:19:49] validation_set Optimizer: 001 Epoch: 10 Loss: 0.027354
[13.03|09:19:54] training_set Optimizer: 001 Epoch: 20 Loss: 0.003755
[13.03|09:19:54] validation_set Optimizer: 001 Epoch: 20 Loss: 0.019911
[13.03|09:19:59] training_set Optimizer: 001 Epoch: 30 Loss: 0.002318
[13.03|09:19:59] validation_set Optimizer: 001 Epoch: 30 Loss: 0.025397
[13.03|09:20:03] training_set Optimizer: 001 Epoch: 40 Loss: 0.001810
[13.03|09:20:03] validation_set Optimizer: 001 Epoch: 40 Loss: 0.017050
[13.03|09:20:08] training_set Optimizer: 001 Epoch: 50 Loss: 0.001529
[13.03|09:20:08] validation_set Optimizer: 001 Epoch: 50 Loss: 0.019239
[13.03|09:20:13] training_set Optimizer: 001 Epoch: 60 Loss: 0.001342
[13.03|09:20:13] validation_set Optimizer: 001 Epoch: 60 Loss: 0.018329
[13.03|09:20:17] training_set Optimizer: 001 Epoch: 70 Loss: 0.001218
[13.03|09:20:17] validation_set Optimizer: 001 Epoch: 70 Loss: 0.016792
[13.03|09:20:22] training_set Optimizer: 001 Epoch: 80 Loss: 0.001107
[13.03|09:20:22] validation_set Optimizer: 001 Epoch: 80 Loss: 0.013660
[13.03|09:20:27] training_set Optimizer: 001 Epoch: 90 Loss: 0.001018
[13.03|09:20:27] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013199
[13.03|09:20:32] training_set Optimizer: 001 Epoch: 100 Loss: 0.000950
[13.03|09:20:32] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013209
[13.03|09:20:37] training_set Optimizer: 001 Epoch: 110 Loss: 0.000886
[13.03|09:20:37] validation_set Optimizer: 001 Epoch: 110 Loss: 0.012001
[13.03|09:20:41] training_set Optimizer: 001 Epoch: 120 Loss: 0.000838
[13.03|09:20:41] validation_set Optimizer: 001 Epoch: 120 Loss: 0.011997
[13.03|09:20:46] training_set Optimizer: 001 Epoch: 130 Loss: 0.000785
[13.03|09:20:46] validation_set Optimizer: 001 Epoch: 130 Loss: 0.008306
[13.03|09:20:50] training_set Optimizer: 001 Epoch: 140 Loss: 0.000742
[13.03|09:20:50] validation_set Optimizer: 001 Epoch: 140 Loss: 0.008596
[13.03|09:20:55] training_set Optimizer: 001 Epoch: 150 Loss: 0.000699
[13.03|09:20:55] validation_set Optimizer: 001 Epoch: 150 Loss: 0.008151
[13.03|09:21:00] training_set Optimizer: 001 Epoch: 160 Loss: 0.000662
[13.03|09:21:00] validation_set Optimizer: 001 Epoch: 160 Loss: 0.008286
[13.03|09:21:05] training_set Optimizer: 001 Epoch: 170 Loss: 0.000638
[13.03|09:21:05] validation_set Optimizer: 001 Epoch: 170 Loss: 0.008540
[13.03|09:21:09] training_set Optimizer: 001 Epoch: 180 Loss: 0.000595
[13.03|09:21:09] validation_set Optimizer: 001 Epoch: 180 Loss: 0.007890
[13.03|09:21:14] training_set Optimizer: 001 Epoch: 190 Loss: 0.000560
[13.03|09:21:14] validation_set Optimizer: 001 Epoch: 190 Loss: 0.008513
[13.03|09:21:20] Execution of m3gnet.run finished with returncode 0
[13.03|09:21:20] JOB m3gnet FINISHED
[13.03|09:21:20] Starting m3gnet.postrun()
[13.03|09:21:20] m3gnet.postrun() finished
[13.03|09:21:20] JOB m3gnet SUCCESSFUL
[13.03|09:21:36] Running all jobs through AMS....
[13.03|09:21:36] Storing results/optimization/training_set_results/best
[13.03|09:21:36] Storing results/optimization/validation_set_results/best
[13.03|09:21:36] PLAMS environment cleaned up successfully
[13.03|09:21:36] PLAMS run finished. Goodbye
[13.03|09:21:37] Initial model has been trained!
[13.03|09:21:37] ParAMSResults training_set validation_set
[13.03|09:21:37] energy MAE 0.1190 0.0609 eV
[13.03|09:21:37] forces MAE 0.0497 0.0807 eV/angstrom
[13.03|09:21:37]
[13.03|09:21:37] Starting active learning loop...
[13.03|09:21:37] ##########################
[13.03|09:21:37] ### Step 1 / Attempt 1 ###
[13.03|09:21:37] ##########################
[13.03|09:21:37] MD Steps: 10 (cumulative: 10)
[13.03|09:21:37] Current engine settings:
[13.03|09:21:37]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/initial_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:21:37] Running step1_attempt1_simulation...
[13.03|09:21:54] Job step1_attempt1_simulation finished
[13.03|09:21:54] Deleting files that are no longer needed...
[13.03|09:21:54] Launching reference calculation
[13.03|09:21:57] Reference calculation finished!
[13.03|09:21:57] Checking success for step1_attempt1
[13.03|09:21:57] CheckEnergy: Checking energy for MDStep10, n_atoms = 46
[13.03|09:21:57] CheckEnergy: normalization coefficient = 46
[13.03|09:21:57] CheckEnergy: Actual Threshold
[13.03|09:21:57] CheckEnergy: dE/46 0.0033 0.2000 OK!
[13.03|09:21:57]
[13.03|09:21:57] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:21:57] CheckForces: ------------
[13.03|09:21:57] CheckForces: Reference job from step1_attempt1_reference_calc1
[13.03|09:21:57] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt1_simulation
[13.03|09:21:57] CheckForces: ------------
[13.03|09:21:57] CheckForces: Histogram of forces
[13.03|09:21:57] CheckForces: eV/Ang Ref Pred
[13.03|09:21:57] CheckForces: -4 1 1
[13.03|09:21:57] CheckForces: -3 2 2
[13.03|09:21:57] CheckForces: -2 6 6
[13.03|09:21:57] CheckForces: -1 55 58
[13.03|09:21:57] CheckForces: 0 66 63
[13.03|09:21:57] CheckForces: 1 7 7
[13.03|09:21:57] CheckForces: 2 1 1
[13.03|09:21:57] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:21:57] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:21:57] CheckForces: Ref Pred Delta Threshold
[13.03|09:21:57] CheckForces: 0.54 0.93 0.39 0.33
[13.03|09:21:57] CheckForces: 0.66 1.01 0.35 0.33
[13.03|09:21:57] CheckForces: Maximum deviation: 0.389 eV/angstrom
[13.03|09:21:57] CheckForces: Actual Threshold
[13.03|09:21:57] CheckForces: # > thr. 2 0 Not OK!
[13.03|09:21:57] CheckForces: MAE 0.069 0.30 OK!
[13.03|09:21:57] CheckForces: R^2 0.982 0.80 OK!
[13.03|09:21:57] CheckForces: --------------------
[13.03|09:21:57]
[13.03|09:21:57] Adding results from step1_attempt1_reference_calc1 to training set
[13.03|09:21:57] Current # training set entries: 11
[13.03|09:21:57] Current # validation set entries: 1
[13.03|09:21:57] Storing data in step1_attempt1_reference_data
[13.03|09:21:57] Deleting initial_reference_data
[13.03|09:21:57] Deleting step1_attempt1_reference_calc1
[13.03|09:21:57]
[13.03|09:21:57] Current (cumulative) timings:
[13.03|09:21:57] Time (s) Fraction
[13.03|09:21:57] Ref. calcs 33.12 0.154
[13.03|09:21:57] ML training 165.15 0.768
[13.03|09:21:57] Simulations 16.67 0.078
[13.03|09:21:57]
[13.03|09:21:57]
[13.03|09:21:57]
[13.03|09:21:57] --- Begin summary ---
[13.03|09:21:57] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:21:57] 1 1 FAILED Inaccurate 0.3894
[13.03|09:21:57] --- End summary ---
[13.03|09:21:57]
[13.03|09:21:57] Running more reference calculations....
[13.03|09:21:57] Running reference calculations on frames [6] from step1_attempt1_simulation/ams.rkf
[13.03|09:21:57] Calculating 1 frames in total
[13.03|09:21:57] Running step1_attempt1_reference_calc2
[13.03|09:22:00] Reference calculations finished!
[13.03|09:22:00] Adding results from step1_attempt1_reference_calc2 to validation set
[13.03|09:22:00] Current # training set entries: 11
[13.03|09:22:00] Current # validation set entries: 2
[13.03|09:22:00] Storing data in step1_attempt1_reference_data
[13.03|09:22:00] Deleting step1_attempt1_reference_calc2
[13.03|09:22:00] Launching reparametrization job: step1_attempt1_training
[13.03|09:22:06] JOB m3gnet STARTED
[13.03|09:22:06] Starting m3gnet.prerun()
[13.03|09:22:06] m3gnet.prerun() finished
[13.03|09:22:06] JOB m3gnet RUNNING
[13.03|09:22:06] Executing m3gnet.run
[13.03|09:23:12] training_set Optimizer: 001 Epoch: 0 Loss: 0.004532
[13.03|09:23:12] validation_set Optimizer: 001 Epoch: 0 Loss: 0.083063
[13.03|09:23:18] training_set Optimizer: 001 Epoch: 10 Loss: 0.000675
[13.03|09:23:18] validation_set Optimizer: 001 Epoch: 10 Loss: 0.019418
[13.03|09:23:24] training_set Optimizer: 001 Epoch: 20 Loss: 0.000440
[13.03|09:23:24] validation_set Optimizer: 001 Epoch: 20 Loss: 0.010321
[13.03|09:23:30] training_set Optimizer: 001 Epoch: 30 Loss: 0.000422
[13.03|09:23:30] validation_set Optimizer: 001 Epoch: 30 Loss: 0.007838
[13.03|09:23:36] training_set Optimizer: 001 Epoch: 40 Loss: 0.000571
[13.03|09:23:36] validation_set Optimizer: 001 Epoch: 40 Loss: 0.009253
[13.03|09:23:42] training_set Optimizer: 001 Epoch: 50 Loss: 0.000368
[13.03|09:23:42] validation_set Optimizer: 001 Epoch: 50 Loss: 0.008492
[13.03|09:23:48] training_set Optimizer: 001 Epoch: 60 Loss: 0.000366
[13.03|09:23:48] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007814
[13.03|09:23:54] training_set Optimizer: 001 Epoch: 70 Loss: 0.000331
[13.03|09:23:54] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006949
[13.03|09:24:00] training_set Optimizer: 001 Epoch: 80 Loss: 0.000450
[13.03|09:24:00] validation_set Optimizer: 001 Epoch: 80 Loss: 0.008353
[13.03|09:24:06] training_set Optimizer: 001 Epoch: 90 Loss: 0.000383
[13.03|09:24:06] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006949
[13.03|09:24:12] training_set Optimizer: 001 Epoch: 100 Loss: 0.000413
[13.03|09:24:12] validation_set Optimizer: 001 Epoch: 100 Loss: 0.008888
[13.03|09:24:18] training_set Optimizer: 001 Epoch: 110 Loss: 0.000227
[13.03|09:24:18] validation_set Optimizer: 001 Epoch: 110 Loss: 0.007933
[13.03|09:24:24] training_set Optimizer: 001 Epoch: 120 Loss: 0.000236
[13.03|09:24:24] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005755
[13.03|09:24:30] training_set Optimizer: 001 Epoch: 130 Loss: 0.000223
[13.03|09:24:30] validation_set Optimizer: 001 Epoch: 130 Loss: 0.010382
[13.03|09:24:35] training_set Optimizer: 001 Epoch: 140 Loss: 0.000204
[13.03|09:24:35] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006623
[13.03|09:24:42] training_set Optimizer: 001 Epoch: 150 Loss: 0.000187
[13.03|09:24:42] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005228
[13.03|09:24:47] training_set Optimizer: 001 Epoch: 160 Loss: 0.000199
[13.03|09:24:47] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006984
[13.03|09:24:53] training_set Optimizer: 001 Epoch: 170 Loss: 0.000181
[13.03|09:24:53] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006308
[13.03|09:24:59] training_set Optimizer: 001 Epoch: 180 Loss: 0.000247
[13.03|09:24:59] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005386
[13.03|09:25:05] training_set Optimizer: 001 Epoch: 190 Loss: 0.000221
[13.03|09:25:05] validation_set Optimizer: 001 Epoch: 190 Loss: 0.007270
[13.03|09:25:12] Execution of m3gnet.run finished with returncode 0
[13.03|09:25:12] JOB m3gnet FINISHED
[13.03|09:25:12] Starting m3gnet.postrun()
[13.03|09:25:12] m3gnet.postrun() finished
[13.03|09:25:13] JOB m3gnet SUCCESSFUL
[13.03|09:25:28] Running all jobs through AMS....
[13.03|09:25:28] Storing results/optimization/training_set_results/best
[13.03|09:25:28] Storing results/optimization/validation_set_results/best
[13.03|09:25:28] PLAMS environment cleaned up successfully
[13.03|09:25:28] PLAMS run finished. Goodbye
[13.03|09:25:29] ParAMSResults training_set validation_set
[13.03|09:25:29] energy MAE 0.0365 0.0689 eV
[13.03|09:25:29] forces MAE 0.0301 0.0548 eV/angstrom
[13.03|09:25:29] Newly created parameter file/dir: step1_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:25:29] Done!
[13.03|09:25:29] Deleting initial_training
[13.03|09:25:29] ##########################
[13.03|09:25:29] ### Step 1 / Attempt 2 ###
[13.03|09:25:29] ##########################
[13.03|09:25:29] MD Steps: 10 (cumulative: 10)
[13.03|09:25:29] Current engine settings:
[13.03|09:25:29]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step1_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:25:29] Running step1_attempt2_simulation...
[13.03|09:25:46] Job step1_attempt2_simulation finished
[13.03|09:25:46] Deleting files that are no longer needed...
[13.03|09:25:46] Launching reference calculation
[13.03|09:25:49] Reference calculation finished!
[13.03|09:25:49] Checking success for step1_attempt2
[13.03|09:26:00] CheckEnergy: Checking energy for MDStep10, n_atoms = 46
[13.03|09:26:00] CheckEnergy: normalization coefficient = 46
[13.03|09:26:00] CheckEnergy: Actual Threshold
[13.03|09:26:00] CheckEnergy: dE/46 0.0012 0.2000 OK!
[13.03|09:26:00] CheckEnergy: ddE/46 0.0000 0.0020 OK! (relative to step1_attempt1_simulation:MDStep10)
[13.03|09:26:00]
[13.03|09:26:00] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:26:00] CheckForces: ------------
[13.03|09:26:00] CheckForces: Reference job from step1_attempt2_reference_calc1
[13.03|09:26:00] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt2_simulation
[13.03|09:26:00] CheckForces: ------------
[13.03|09:26:00] CheckForces: Histogram of forces
[13.03|09:26:00] CheckForces: eV/Ang Ref Pred
[13.03|09:26:00] CheckForces: -3 1 1
[13.03|09:26:00] CheckForces: -2 4 5
[13.03|09:26:00] CheckForces: -1 63 59
[13.03|09:26:00] CheckForces: 0 65 68
[13.03|09:26:00] CheckForces: 1 5 5
[13.03|09:26:00] CheckForces: 2 0 0
[13.03|09:26:00] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:26:00] CheckForces: All force components are within the acceptable error!
[13.03|09:26:00] CheckForces: Maximum deviation: 0.197 eV/angstrom
[13.03|09:26:00] CheckForces: Actual Threshold
[13.03|09:26:00] CheckForces: # > thr. 0 0 OK!
[13.03|09:26:00] CheckForces: MAE 0.049 0.30 OK!
[13.03|09:26:00] CheckForces: R^2 0.986 0.80 OK!
[13.03|09:26:00] CheckForces: --------------------
[13.03|09:26:00]
[13.03|09:26:00] Adding results from step1_attempt2_reference_calc1 to validation set
[13.03|09:26:00] Current # training set entries: 11
[13.03|09:26:00] Current # validation set entries: 3
[13.03|09:26:00] Storing data in step1_attempt2_reference_data
[13.03|09:26:00] Deleting step1_attempt1_reference_data
[13.03|09:26:00] Deleting step1_attempt2_reference_calc1
[13.03|09:26:00]
[13.03|09:26:00] Current (cumulative) timings:
[13.03|09:26:00] Time (s) Fraction
[13.03|09:26:00] Ref. calcs 39.25 0.088
[13.03|09:26:00] ML training 374.17 0.838
[13.03|09:26:00] Simulations 33.21 0.074
[13.03|09:26:00]
[13.03|09:26:00]
[13.03|09:26:00] Step 1 finished successfully!
[13.03|09:26:00]
[13.03|09:26:00] --- Begin summary ---
[13.03|09:26:00] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:26:00] 1 1 FAILED Inaccurate 0.3894
[13.03|09:26:00] 1 2 SUCCESS Accurate 0.1974
[13.03|09:26:00] --- End summary ---
[13.03|09:26:00]
[13.03|09:26:00] ##########################
[13.03|09:26:00] ### Step 2 / Attempt 1 ###
[13.03|09:26:00] ##########################
[13.03|09:26:00] MD Steps: 19 (cumulative: 29)
[13.03|09:26:00] Current engine settings:
[13.03|09:26:00]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step1_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:26:00] Running step2_attempt1_simulation...
[13.03|09:26:14] Job step2_attempt1_simulation finished
[13.03|09:26:14] Deleting files that are no longer needed...
[13.03|09:26:14] Deleting step1_attempt1_simulation
[13.03|09:26:14] Launching reference calculation
[13.03|09:26:17] Reference calculation finished!
[13.03|09:26:17] Checking success for step2_attempt1
[13.03|09:26:28] CheckEnergy: Checking energy for MDStep29, n_atoms = 46
[13.03|09:26:28] CheckEnergy: normalization coefficient = 46
[13.03|09:26:28] CheckEnergy: Actual Threshold
[13.03|09:26:28] CheckEnergy: dE/46 0.0006 0.2000 OK!
[13.03|09:26:28] CheckEnergy: ddE/46 -0.0006 0.0020 OK! (relative to step1_attempt2_simulation:MDStep10)
[13.03|09:26:28]
[13.03|09:26:28] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:26:28] CheckForces: ------------
[13.03|09:26:28] CheckForces: Reference job from step2_attempt1_reference_calc1
[13.03|09:26:28] CheckForces: Prediction job from final frame (MDStep29) of step2_attempt1_simulation
[13.03|09:26:28] CheckForces: ------------
[13.03|09:26:28] CheckForces: Histogram of forces
[13.03|09:26:28] CheckForces: eV/Ang Ref Pred
[13.03|09:26:28] CheckForces: -3 0 0
[13.03|09:26:28] CheckForces: -2 13 11
[13.03|09:26:28] CheckForces: -1 56 59
[13.03|09:26:28] CheckForces: 0 61 60
[13.03|09:26:28] CheckForces: 1 8 8
[13.03|09:26:28] CheckForces: 2 0 0
[13.03|09:26:28] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:26:28] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:26:28] CheckForces: Ref Pred Delta Threshold
[13.03|09:26:28] CheckForces: -0.51 -0.15 0.36 0.32
[13.03|09:26:28] CheckForces: 0.02 -0.33 0.34 0.30
[13.03|09:26:28] CheckForces: Maximum deviation: 0.363 eV/angstrom
[13.03|09:26:28] CheckForces: Actual Threshold
[13.03|09:26:28] CheckForces: # > thr. 2 0 Not OK!
[13.03|09:26:28] CheckForces: MAE 0.073 0.30 OK!
[13.03|09:26:28] CheckForces: R^2 0.977 0.80 OK!
[13.03|09:26:28] CheckForces: --------------------
[13.03|09:26:28]
[13.03|09:26:28] Adding results from step2_attempt1_reference_calc1 to training set
[13.03|09:26:28] Current # training set entries: 12
[13.03|09:26:28] Current # validation set entries: 3
[13.03|09:26:28] Storing data in step2_attempt1_reference_data
[13.03|09:26:29] Deleting step1_attempt2_reference_data
[13.03|09:26:29] Deleting step2_attempt1_reference_calc1
[13.03|09:26:29]
[13.03|09:26:29] Current (cumulative) timings:
[13.03|09:26:29] Time (s) Fraction
[13.03|09:26:29] Ref. calcs 42.26 0.091
[13.03|09:26:29] ML training 374.17 0.807
[13.03|09:26:29] Simulations 47.16 0.102
[13.03|09:26:29]
[13.03|09:26:29]
[13.03|09:26:29]
[13.03|09:26:29] --- Begin summary ---
[13.03|09:26:29] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:26:29] 1 1 FAILED Inaccurate 0.3894
[13.03|09:26:29] 1 2 SUCCESS Accurate 0.1974
[13.03|09:26:29] 2 1 FAILED Inaccurate 0.3629
[13.03|09:26:29] --- End summary ---
[13.03|09:26:29]
[13.03|09:26:29] Running more reference calculations....
[13.03|09:26:29] Running reference calculations on frames [21] from step2_attempt1_simulation/ams.rkf
[13.03|09:26:29] Calculating 1 frames in total
[13.03|09:26:29] Running step2_attempt1_reference_calc2
[13.03|09:26:32] Reference calculations finished!
[13.03|09:26:32] Adding results from step2_attempt1_reference_calc2 to validation set
[13.03|09:26:32] Current # training set entries: 12
[13.03|09:26:32] Current # validation set entries: 4
[13.03|09:26:32] Storing data in step2_attempt1_reference_data
[13.03|09:26:32] Deleting step2_attempt1_reference_calc2
[13.03|09:26:32] Launching reparametrization job: step2_attempt1_training
[13.03|09:26:37] JOB m3gnet STARTED
[13.03|09:26:37] Starting m3gnet.prerun()
[13.03|09:26:37] m3gnet.prerun() finished
[13.03|09:26:37] JOB m3gnet RUNNING
[13.03|09:26:37] Executing m3gnet.run
[13.03|09:27:44] training_set Optimizer: 001 Epoch: 0 Loss: 0.002080
[13.03|09:27:44] validation_set Optimizer: 001 Epoch: 0 Loss: 0.031911
[13.03|09:27:51] training_set Optimizer: 001 Epoch: 10 Loss: 0.000372
[13.03|09:27:51] validation_set Optimizer: 001 Epoch: 10 Loss: 0.007403
[13.03|09:27:57] training_set Optimizer: 001 Epoch: 20 Loss: 0.000209
[13.03|09:27:57] validation_set Optimizer: 001 Epoch: 20 Loss: 0.007535
[13.03|09:28:04] training_set Optimizer: 001 Epoch: 30 Loss: 0.000226
[13.03|09:28:04] validation_set Optimizer: 001 Epoch: 30 Loss: 0.008321
[13.03|09:28:10] training_set Optimizer: 001 Epoch: 40 Loss: 0.000200
[13.03|09:28:10] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006091
[13.03|09:28:17] training_set Optimizer: 001 Epoch: 50 Loss: 0.000197
[13.03|09:28:17] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005056
[13.03|09:28:23] training_set Optimizer: 001 Epoch: 60 Loss: 0.000176
[13.03|09:28:23] validation_set Optimizer: 001 Epoch: 60 Loss: 0.009636
[13.03|09:28:30] training_set Optimizer: 001 Epoch: 70 Loss: 0.000192
[13.03|09:28:30] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005940
[13.03|09:28:36] training_set Optimizer: 001 Epoch: 80 Loss: 0.000159
[13.03|09:28:36] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005043
[13.03|09:28:42] training_set Optimizer: 001 Epoch: 90 Loss: 0.000166
[13.03|09:28:42] validation_set Optimizer: 001 Epoch: 90 Loss: 0.012280
[13.03|09:28:49] training_set Optimizer: 001 Epoch: 100 Loss: 0.000147
[13.03|09:28:49] validation_set Optimizer: 001 Epoch: 100 Loss: 0.004983
[13.03|09:28:55] training_set Optimizer: 001 Epoch: 110 Loss: 0.000147
[13.03|09:28:55] validation_set Optimizer: 001 Epoch: 110 Loss: 0.004930
[13.03|09:28:58] Execution of m3gnet.run finished with returncode 0
[13.03|09:28:58] JOB m3gnet FINISHED
[13.03|09:28:58] Starting m3gnet.postrun()
[13.03|09:28:58] m3gnet.postrun() finished
[13.03|09:28:58] JOB m3gnet SUCCESSFUL
[13.03|09:29:15] Running all jobs through AMS....
[13.03|09:29:15] Storing results/optimization/training_set_results/best
[13.03|09:29:15] Storing results/optimization/validation_set_results/best
[13.03|09:29:15] PLAMS environment cleaned up successfully
[13.03|09:29:15] PLAMS run finished. Goodbye
[13.03|09:29:16] ParAMSResults training_set validation_set
[13.03|09:29:16] energy MAE 0.0342 0.0071 eV
[13.03|09:29:16] forces MAE 0.0289 0.0499 eV/angstrom
[13.03|09:29:16] Newly created parameter file/dir: step2_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:29:16] Done!
[13.03|09:29:16] Deleting step1_attempt1_training
[13.03|09:29:16] ##########################
[13.03|09:29:16] ### Step 2 / Attempt 2 ###
[13.03|09:29:16] ##########################
[13.03|09:29:16] MD Steps: 19 (cumulative: 29)
[13.03|09:29:16] Current engine settings:
[13.03|09:29:16]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step2_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:29:16] Running step2_attempt2_simulation...
[13.03|09:29:30] Job step2_attempt2_simulation finished
[13.03|09:29:30] Deleting files that are no longer needed...
[13.03|09:29:30] Launching reference calculation
[13.03|09:29:33] Reference calculation finished!
[13.03|09:29:33] Checking success for step2_attempt2
[13.03|09:29:44] CheckEnergy: Checking energy for MDStep29, n_atoms = 46
[13.03|09:29:44] CheckEnergy: normalization coefficient = 46
[13.03|09:29:44] CheckEnergy: Actual Threshold
[13.03|09:29:44] CheckEnergy: dE/46 -0.0001 0.2000 OK!
[13.03|09:29:44] CheckEnergy: ddE/46 0.0002 0.0020 OK! (relative to step2_attempt1_simulation:MDStep29)
[13.03|09:29:44]
[13.03|09:29:44] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:29:44] CheckForces: ------------
[13.03|09:29:44] CheckForces: Reference job from step2_attempt2_reference_calc1
[13.03|09:29:44] CheckForces: Prediction job from final frame (MDStep29) of step2_attempt2_simulation
[13.03|09:29:44] CheckForces: ------------
[13.03|09:29:44] CheckForces: Histogram of forces
[13.03|09:29:44] CheckForces: eV/Ang Ref Pred
[13.03|09:29:44] CheckForces: -3 0 0
[13.03|09:29:44] CheckForces: -2 12 12
[13.03|09:29:44] CheckForces: -1 56 57
[13.03|09:29:44] CheckForces: 0 62 61
[13.03|09:29:44] CheckForces: 1 8 8
[13.03|09:29:44] CheckForces: 2 0 0
[13.03|09:29:44] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:29:44] CheckForces: All force components are within the acceptable error!
[13.03|09:29:44] CheckForces: Maximum deviation: 0.185 eV/angstrom
[13.03|09:29:44] CheckForces: Actual Threshold
[13.03|09:29:44] CheckForces: # > thr. 0 0 OK!
[13.03|09:29:44] CheckForces: MAE 0.040 0.30 OK!
[13.03|09:29:44] CheckForces: R^2 0.994 0.80 OK!
[13.03|09:29:44] CheckForces: --------------------
[13.03|09:29:44]
[13.03|09:29:44] Adding results from step2_attempt2_reference_calc1 to validation set
[13.03|09:29:44] Current # training set entries: 12
[13.03|09:29:44] Current # validation set entries: 5
[13.03|09:29:44] Storing data in step2_attempt2_reference_data
[13.03|09:29:44] Deleting step2_attempt1_reference_data
[13.03|09:29:44] Deleting step2_attempt2_reference_calc1
[13.03|09:29:44]
[13.03|09:29:44] Current (cumulative) timings:
[13.03|09:29:44] Time (s) Fraction
[13.03|09:29:44] Ref. calcs 48.39 0.075
[13.03|09:29:44] ML training 538.08 0.831
[13.03|09:29:44] Simulations 61.09 0.094
[13.03|09:29:44]
[13.03|09:29:44]
[13.03|09:29:44] Step 2 finished successfully!
[13.03|09:29:44]
[13.03|09:29:44] --- Begin summary ---
[13.03|09:29:44] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:29:44] 1 1 FAILED Inaccurate 0.3894
[13.03|09:29:44] 1 2 SUCCESS Accurate 0.1974
[13.03|09:29:44] 2 1 FAILED Inaccurate 0.3629
[13.03|09:29:44] 2 2 SUCCESS Accurate 0.1848
[13.03|09:29:44] --- End summary ---
[13.03|09:29:44]
[13.03|09:29:44] ##########################
[13.03|09:29:44] ### Step 3 / Attempt 1 ###
[13.03|09:29:44] ##########################
[13.03|09:29:44] MD Steps: 58 (cumulative: 87)
[13.03|09:29:44] Current engine settings:
[13.03|09:29:44]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step2_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:29:44] Running step3_attempt1_simulation...
[13.03|09:30:03] Job step3_attempt1_simulation finished
[13.03|09:30:03] Deleting files that are no longer needed...
[13.03|09:30:03] Deleting step1_attempt2_simulation
[13.03|09:30:03] Deleting step2_attempt1_simulation
[13.03|09:30:03] Launching reference calculation
[13.03|09:30:06] Reference calculation finished!
[13.03|09:30:06] Checking success for step3_attempt1
[13.03|09:30:17] CheckEnergy: Checking energy for MDStep87, n_atoms = 46
[13.03|09:30:17] CheckEnergy: normalization coefficient = 46
[13.03|09:30:17] CheckEnergy: Actual Threshold
[13.03|09:30:17] CheckEnergy: dE/46 -0.0018 0.2000 OK!
[13.03|09:30:17] CheckEnergy: ddE/46 -0.0016 0.0020 OK! (relative to step2_attempt2_simulation:MDStep29)
[13.03|09:30:17]
[13.03|09:30:17] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:30:17] CheckForces: ------------
[13.03|09:30:17] CheckForces: Reference job from step3_attempt1_reference_calc1
[13.03|09:30:17] CheckForces: Prediction job from final frame (MDStep87) of step3_attempt1_simulation
[13.03|09:30:17] CheckForces: ------------
[13.03|09:30:17] CheckForces: Histogram of forces
[13.03|09:30:17] CheckForces: eV/Ang Ref Pred
[13.03|09:30:17] CheckForces: -3 2 2
[13.03|09:30:17] CheckForces: -2 2 1
[13.03|09:30:17] CheckForces: -1 63 68
[13.03|09:30:17] CheckForces: 0 63 62
[13.03|09:30:17] CheckForces: 1 8 5
[13.03|09:30:17] CheckForces: 2 0 0
[13.03|09:30:17] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:30:17] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:30:17] CheckForces: Ref Pred Delta Threshold
[13.03|09:30:17] CheckForces: -0.51 -0.16 0.36 0.32
[13.03|09:30:17] CheckForces: Maximum deviation: 0.372 eV/angstrom
[13.03|09:30:17] CheckForces: Actual Threshold
[13.03|09:30:17] CheckForces: # > thr. 1 0 Not OK!
[13.03|09:30:17] CheckForces: MAE 0.091 0.30 OK!
[13.03|09:30:17] CheckForces: R^2 0.954 0.80 OK!
[13.03|09:30:17] CheckForces: --------------------
[13.03|09:30:17]
[13.03|09:30:17] Adding results from step3_attempt1_reference_calc1 to training set
[13.03|09:30:17] Current # training set entries: 13
[13.03|09:30:17] Current # validation set entries: 5
[13.03|09:30:17] Storing data in step3_attempt1_reference_data
[13.03|09:30:17] Deleting step2_attempt2_reference_data
[13.03|09:30:17] Deleting step3_attempt1_reference_calc1
[13.03|09:30:17]
[13.03|09:30:17] Current (cumulative) timings:
[13.03|09:30:17] Time (s) Fraction
[13.03|09:30:17] Ref. calcs 51.39 0.077
[13.03|09:30:17] ML training 538.08 0.804
[13.03|09:30:17] Simulations 79.62 0.119
[13.03|09:30:17]
[13.03|09:30:17]
[13.03|09:30:17]
[13.03|09:30:17] --- Begin summary ---
[13.03|09:30:17] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:30:17] 1 1 FAILED Inaccurate 0.3894
[13.03|09:30:17] 1 2 SUCCESS Accurate 0.1974
[13.03|09:30:17] 2 1 FAILED Inaccurate 0.3629
[13.03|09:30:17] 2 2 SUCCESS Accurate 0.1848
[13.03|09:30:17] 3 1 FAILED Inaccurate 0.3716
[13.03|09:30:17] --- End summary ---
[13.03|09:30:17]
[13.03|09:30:17] Running more reference calculations....
[13.03|09:30:17] Running reference calculations on frames [34, 38] from step3_attempt1_simulation/ams.rkf
[13.03|09:30:17] Calculating 2 frames in total
[13.03|09:30:17] Running step3_attempt1_reference_calc2
[13.03|09:30:20] Running step3_attempt1_reference_calc3
[13.03|09:30:23] Reference calculations finished!
[13.03|09:30:23] Adding results from step3_attempt1_reference_calc2 to validation set
[13.03|09:30:23] Adding results from step3_attempt1_reference_calc3 to training set
[13.03|09:30:23] Current # training set entries: 14
[13.03|09:30:23] Current # validation set entries: 6
[13.03|09:30:23] Storing data in step3_attempt1_reference_data
[13.03|09:30:23] Deleting step3_attempt1_reference_calc2
[13.03|09:30:23] Deleting step3_attempt1_reference_calc3
[13.03|09:30:23] Launching reparametrization job: step3_attempt1_training
[13.03|09:30:29] JOB m3gnet STARTED
[13.03|09:30:29] Starting m3gnet.prerun()
[13.03|09:30:29] m3gnet.prerun() finished
[13.03|09:30:29] JOB m3gnet RUNNING
[13.03|09:30:29] Executing m3gnet.run
[13.03|09:31:36] training_set Optimizer: 001 Epoch: 0 Loss: 0.003469
[13.03|09:31:36] validation_set Optimizer: 001 Epoch: 0 Loss: 0.074424
[13.03|09:31:46] Execution of m3gnet.run finished with returncode 0
[13.03|09:31:46] JOB m3gnet FINISHED
[13.03|09:31:46] Starting m3gnet.postrun()
[13.03|09:31:46] m3gnet.postrun() finished
[13.03|09:31:46] JOB m3gnet SUCCESSFUL
[13.03|09:32:01] Running all jobs through AMS....
[13.03|09:32:01] Storing results/optimization/training_set_results/best
[13.03|09:32:01] Storing results/optimization/validation_set_results/best
[13.03|09:32:01] PLAMS environment cleaned up successfully
[13.03|09:32:01] PLAMS run finished. Goodbye
[13.03|09:32:02] ParAMSResults training_set validation_set
[13.03|09:32:02] energy MAE 0.1847 0.2191 eV
[13.03|09:32:02] forces MAE 0.0398 0.0512 eV/angstrom
[13.03|09:32:02] Newly created parameter file/dir: step3_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:32:02] Done!
[13.03|09:32:02] Deleting step2_attempt1_training
[13.03|09:32:02] ##########################
[13.03|09:32:02] ### Step 3 / Attempt 2 ###
[13.03|09:32:02] ##########################
[13.03|09:32:02] MD Steps: 58 (cumulative: 87)
[13.03|09:32:02] Current engine settings:
[13.03|09:32:02]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step3_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:32:02] Running step3_attempt2_simulation...
[13.03|09:32:21] Job step3_attempt2_simulation finished
[13.03|09:32:21] Deleting files that are no longer needed...
[13.03|09:32:21] Launching reference calculation
[13.03|09:32:24] Reference calculation finished!
[13.03|09:32:24] Checking success for step3_attempt2
[13.03|09:32:35] CheckEnergy: Checking energy for MDStep87, n_atoms = 46
[13.03|09:32:35] CheckEnergy: normalization coefficient = 46
[13.03|09:32:35] CheckEnergy: Actual Threshold
[13.03|09:32:35] CheckEnergy: dE/46 0.0034 0.2000 OK!
[13.03|09:32:35] CheckEnergy: ddE/46 0.0001 0.0020 OK! (relative to step3_attempt1_simulation:MDStep87)
[13.03|09:32:35]
[13.03|09:32:35] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:32:35] CheckForces: ------------
[13.03|09:32:35] CheckForces: Reference job from step3_attempt2_reference_calc1
[13.03|09:32:35] CheckForces: Prediction job from final frame (MDStep87) of step3_attempt2_simulation
[13.03|09:32:35] CheckForces: ------------
[13.03|09:32:35] CheckForces: Histogram of forces
[13.03|09:32:35] CheckForces: eV/Ang Ref Pred
[13.03|09:32:35] CheckForces: -3 2 2
[13.03|09:32:35] CheckForces: -2 2 1
[13.03|09:32:35] CheckForces: -1 64 66
[13.03|09:32:35] CheckForces: 0 63 64
[13.03|09:32:35] CheckForces: 1 7 5
[13.03|09:32:35] CheckForces: 2 0 0
[13.03|09:32:35] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:32:35] CheckForces: All force components are within the acceptable error!
[13.03|09:32:35] CheckForces: Maximum deviation: 0.304 eV/angstrom
[13.03|09:32:35] CheckForces: Actual Threshold
[13.03|09:32:35] CheckForces: # > thr. 0 0 OK!
[13.03|09:32:35] CheckForces: MAE 0.074 0.30 OK!
[13.03|09:32:35] CheckForces: R^2 0.968 0.80 OK!
[13.03|09:32:35] CheckForces: --------------------
[13.03|09:32:35]
[13.03|09:32:35] Adding results from step3_attempt2_reference_calc1 to validation set
[13.03|09:32:35] Current # training set entries: 14
[13.03|09:32:35] Current # validation set entries: 7
[13.03|09:32:35] Storing data in step3_attempt2_reference_data
[13.03|09:32:35] Deleting step3_attempt1_reference_data
[13.03|09:32:35] Deleting step3_attempt2_reference_calc1
[13.03|09:32:35]
[13.03|09:32:35] Current (cumulative) timings:
[13.03|09:32:35] Time (s) Fraction
[13.03|09:32:35] Ref. calcs 60.44 0.076
[13.03|09:32:35] ML training 637.06 0.801
[13.03|09:32:35] Simulations 98.16 0.123
[13.03|09:32:35]
[13.03|09:32:35]
[13.03|09:32:35] Step 3 finished successfully!
[13.03|09:32:35]
[13.03|09:32:35] --- Begin summary ---
[13.03|09:32:35] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:32:35] 1 1 FAILED Inaccurate 0.3894
[13.03|09:32:35] 1 2 SUCCESS Accurate 0.1974
[13.03|09:32:35] 2 1 FAILED Inaccurate 0.3629
[13.03|09:32:35] 2 2 SUCCESS Accurate 0.1848
[13.03|09:32:35] 3 1 FAILED Inaccurate 0.3716
[13.03|09:32:35] 3 2 SUCCESS Accurate 0.3045
[13.03|09:32:35] --- End summary ---
[13.03|09:32:35]
[13.03|09:32:35] ##########################
[13.03|09:32:35] ### Step 4 / Attempt 1 ###
[13.03|09:32:35] ##########################
[13.03|09:32:35] MD Steps: 172 (cumulative: 259)
[13.03|09:32:35] Current engine settings:
[13.03|09:32:35]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step3_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:32:35] Running step4_attempt1_simulation...
[13.03|09:32:56] Job step4_attempt1_simulation finished
[13.03|09:32:56] Deleting files that are no longer needed...
[13.03|09:32:56] Deleting step2_attempt2_simulation
[13.03|09:32:56] Deleting step3_attempt1_simulation
[13.03|09:32:56] Launching reference calculation
[13.03|09:32:59] Reference calculation finished!
[13.03|09:32:59] Checking success for step4_attempt1
[13.03|09:33:10] CheckEnergy: Checking energy for MDStep259, n_atoms = 46
[13.03|09:33:10] CheckEnergy: normalization coefficient = 46
[13.03|09:33:10] CheckEnergy: Actual Threshold
[13.03|09:33:10] CheckEnergy: dE/46 0.0046 0.2000 OK!
[13.03|09:33:10] CheckEnergy: ddE/46 0.0011 0.0020 OK! (relative to step3_attempt2_simulation:MDStep87)
[13.03|09:33:10]
[13.03|09:33:10] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:33:10] CheckForces: ------------
[13.03|09:33:10] CheckForces: Reference job from step4_attempt1_reference_calc1
[13.03|09:33:10] CheckForces: Prediction job from final frame (MDStep259) of step4_attempt1_simulation
[13.03|09:33:10] CheckForces: ------------
[13.03|09:33:10] CheckForces: Histogram of forces
[13.03|09:33:10] CheckForces: eV/Ang Ref Pred
[13.03|09:33:10] CheckForces: -3 0 0
[13.03|09:33:10] CheckForces: -2 10 9
[13.03|09:33:10] CheckForces: -1 61 61
[13.03|09:33:10] CheckForces: 0 57 57
[13.03|09:33:10] CheckForces: 1 10 11
[13.03|09:33:10] CheckForces: 2 0 0
[13.03|09:33:10] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:33:10] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:33:10] CheckForces: Ref Pred Delta Threshold
[13.03|09:33:10] CheckForces: -0.86 -0.47 0.39 0.34
[13.03|09:33:10] CheckForces: Maximum deviation: 0.389 eV/angstrom
[13.03|09:33:10] CheckForces: Actual Threshold
[13.03|09:33:10] CheckForces: # > thr. 1 0 Not OK!
[13.03|09:33:10] CheckForces: MAE 0.075 0.30 OK!
[13.03|09:33:10] CheckForces: R^2 0.977 0.80 OK!
[13.03|09:33:10] CheckForces: --------------------
[13.03|09:33:10]
[13.03|09:33:10] Adding results from step4_attempt1_reference_calc1 to training set
[13.03|09:33:10] Current # training set entries: 15
[13.03|09:33:10] Current # validation set entries: 7
[13.03|09:33:10] Storing data in step4_attempt1_reference_data
[13.03|09:33:10] Deleting step3_attempt2_reference_data
[13.03|09:33:10] Deleting step4_attempt1_reference_calc1
[13.03|09:33:10]
[13.03|09:33:10] Current (cumulative) timings:
[13.03|09:33:10] Time (s) Fraction
[13.03|09:33:10] Ref. calcs 63.37 0.077
[13.03|09:33:10] ML training 637.06 0.778
[13.03|09:33:10] Simulations 118.48 0.145
[13.03|09:33:10]
[13.03|09:33:10]
[13.03|09:33:10]
[13.03|09:33:10] --- Begin summary ---
[13.03|09:33:10] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:33:10] 1 1 FAILED Inaccurate 0.3894
[13.03|09:33:10] 1 2 SUCCESS Accurate 0.1974
[13.03|09:33:10] 2 1 FAILED Inaccurate 0.3629
[13.03|09:33:10] 2 2 SUCCESS Accurate 0.1848
[13.03|09:33:10] 3 1 FAILED Inaccurate 0.3716
[13.03|09:33:10] 3 2 SUCCESS Accurate 0.3045
[13.03|09:33:10] 4 1 FAILED Inaccurate 0.3887
[13.03|09:33:10] --- End summary ---
[13.03|09:33:10]
[13.03|09:33:10] Running more reference calculations....
[13.03|09:33:11] Running reference calculations on frames [47, 49, 52] from step4_attempt1_simulation/ams.rkf
[13.03|09:33:11] Calculating 3 frames in total
[13.03|09:33:11] Running step4_attempt1_reference_calc2
[13.03|09:33:14] Running step4_attempt1_reference_calc3
[13.03|09:33:17] Running step4_attempt1_reference_calc4
[13.03|09:33:20] Reference calculations finished!
[13.03|09:33:20] Adding results from step4_attempt1_reference_calc2 to validation set
[13.03|09:33:20] Adding results from step4_attempt1_reference_calc3 to training set
[13.03|09:33:20] Adding results from step4_attempt1_reference_calc4 to training set
[13.03|09:33:20] Current # training set entries: 17
[13.03|09:33:20] Current # validation set entries: 8
[13.03|09:33:20] Storing data in step4_attempt1_reference_data
[13.03|09:33:20] Deleting step4_attempt1_reference_calc2
[13.03|09:33:20] Deleting step4_attempt1_reference_calc3
[13.03|09:33:20] Deleting step4_attempt1_reference_calc4
[13.03|09:33:20] Launching reparametrization job: step4_attempt1_training
[13.03|09:33:25] JOB m3gnet STARTED
[13.03|09:33:25] Starting m3gnet.prerun()
[13.03|09:33:25] m3gnet.prerun() finished
[13.03|09:33:25] JOB m3gnet RUNNING
[13.03|09:33:25] Executing m3gnet.run
[13.03|09:34:33] training_set Optimizer: 001 Epoch: 0 Loss: 0.003225
[13.03|09:34:33] validation_set Optimizer: 001 Epoch: 0 Loss: 0.049225
[13.03|09:34:42] training_set Optimizer: 001 Epoch: 10 Loss: 0.000361
[13.03|09:34:42] validation_set Optimizer: 001 Epoch: 10 Loss: 0.008489
[13.03|09:34:51] training_set Optimizer: 001 Epoch: 20 Loss: 0.000264
[13.03|09:34:51] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005222
[13.03|09:35:00] training_set Optimizer: 001 Epoch: 30 Loss: 0.000263
[13.03|09:35:00] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005071
[13.03|09:35:09] training_set Optimizer: 001 Epoch: 40 Loss: 0.000225
[13.03|09:35:09] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005161
[13.03|09:35:15] Execution of m3gnet.run finished with returncode 0
[13.03|09:35:15] JOB m3gnet FINISHED
[13.03|09:35:15] Starting m3gnet.postrun()
[13.03|09:35:15] m3gnet.postrun() finished
[13.03|09:35:15] JOB m3gnet SUCCESSFUL
[13.03|09:35:32] Running all jobs through AMS....
[13.03|09:35:32] Storing results/optimization/training_set_results/best
[13.03|09:35:32] Storing results/optimization/validation_set_results/best
[13.03|09:35:32] PLAMS environment cleaned up successfully
[13.03|09:35:32] PLAMS run finished. Goodbye
[13.03|09:35:33] ParAMSResults training_set validation_set
[13.03|09:35:33] energy MAE 0.1630 0.1331 eV
[13.03|09:35:33] forces MAE 0.0365 0.0485 eV/angstrom
[13.03|09:35:33] Newly created parameter file/dir: step4_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:35:33] Done!
[13.03|09:35:34] Deleting step3_attempt1_training
[13.03|09:35:34] ##########################
[13.03|09:35:34] ### Step 4 / Attempt 2 ###
[13.03|09:35:34] ##########################
[13.03|09:35:34] MD Steps: 172 (cumulative: 259)
[13.03|09:35:34] Current engine settings:
[13.03|09:35:34]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step4_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:35:34] Running step4_attempt2_simulation...
[13.03|09:35:54] Job step4_attempt2_simulation finished
[13.03|09:35:54] Deleting files that are no longer needed...
[13.03|09:35:54] Launching reference calculation
[13.03|09:35:57] Reference calculation finished!
[13.03|09:35:57] Checking success for step4_attempt2
[13.03|09:36:08] CheckEnergy: Checking energy for MDStep259, n_atoms = 46
[13.03|09:36:08] CheckEnergy: normalization coefficient = 46
[13.03|09:36:08] CheckEnergy: Actual Threshold
[13.03|09:36:08] CheckEnergy: dE/46 -0.0025 0.2000 OK!
[13.03|09:36:08] CheckEnergy: ddE/46 0.0006 0.0020 OK! (relative to step4_attempt1_simulation:MDStep259)
[13.03|09:36:08]
[13.03|09:36:08] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:36:08] CheckForces: ------------
[13.03|09:36:08] CheckForces: Reference job from step4_attempt2_reference_calc1
[13.03|09:36:08] CheckForces: Prediction job from final frame (MDStep259) of step4_attempt2_simulation
[13.03|09:36:08] CheckForces: ------------
[13.03|09:36:08] CheckForces: Histogram of forces
[13.03|09:36:08] CheckForces: eV/Ang Ref Pred
[13.03|09:36:08] CheckForces: -3 0 0
[13.03|09:36:08] CheckForces: -2 9 9
[13.03|09:36:08] CheckForces: -1 61 64
[13.03|09:36:08] CheckForces: 0 56 53
[13.03|09:36:08] CheckForces: 1 12 12
[13.03|09:36:08] CheckForces: 2 0 0
[13.03|09:36:08] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:36:08] CheckForces: All force components are within the acceptable error!
[13.03|09:36:08] CheckForces: Maximum deviation: 0.220 eV/angstrom
[13.03|09:36:08] CheckForces: Actual Threshold
[13.03|09:36:08] CheckForces: # > thr. 0 0 OK!
[13.03|09:36:08] CheckForces: MAE 0.049 0.30 OK!
[13.03|09:36:08] CheckForces: R^2 0.990 0.80 OK!
[13.03|09:36:08] CheckForces: --------------------
[13.03|09:36:08]
[13.03|09:36:08] Adding results from step4_attempt2_reference_calc1 to validation set
[13.03|09:36:08] Current # training set entries: 17
[13.03|09:36:08] Current # validation set entries: 9
[13.03|09:36:08] Storing data in step4_attempt2_reference_data
[13.03|09:36:08] Deleting step4_attempt1_reference_data
[13.03|09:36:08] Deleting step4_attempt2_reference_calc1
[13.03|09:36:08]
[13.03|09:36:08] Current (cumulative) timings:
[13.03|09:36:08] Time (s) Fraction
[13.03|09:36:08] Ref. calcs 75.48 0.077
[13.03|09:36:08] ML training 770.76 0.782
[13.03|09:36:08] Simulations 138.86 0.141
[13.03|09:36:08]
[13.03|09:36:08]
[13.03|09:36:08] Step 4 finished successfully!
[13.03|09:36:08]
[13.03|09:36:08] --- Begin summary ---
[13.03|09:36:08] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:36:08] 1 1 FAILED Inaccurate 0.3894
[13.03|09:36:08] 1 2 SUCCESS Accurate 0.1974
[13.03|09:36:08] 2 1 FAILED Inaccurate 0.3629
[13.03|09:36:08] 2 2 SUCCESS Accurate 0.1848
[13.03|09:36:08] 3 1 FAILED Inaccurate 0.3716
[13.03|09:36:08] 3 2 SUCCESS Accurate 0.3045
[13.03|09:36:08] 4 1 FAILED Inaccurate 0.3887
[13.03|09:36:08] 4 2 SUCCESS Accurate 0.2201
[13.03|09:36:08] --- End summary ---
[13.03|09:36:08]
[13.03|09:36:08] ##########################
[13.03|09:36:08] ### Step 5 / Attempt 1 ###
[13.03|09:36:08] ##########################
[13.03|09:36:08] MD Steps: 510 (cumulative: 769)
[13.03|09:36:08] Current engine settings:
[13.03|09:36:08]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step4_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:36:08] Running step5_attempt1_simulation...
[13.03|09:36:46] Job step5_attempt1_simulation finished
[13.03|09:36:46] Deleting files that are no longer needed...
[13.03|09:36:46] Deleting step3_attempt2_simulation
[13.03|09:36:46] Deleting step4_attempt1_simulation
[13.03|09:36:46] Launching reference calculation
[13.03|09:36:50] Reference calculation finished!
[13.03|09:36:50] Checking success for step5_attempt1
[13.03|09:37:00] CheckEnergy: Checking energy for MDStep769, n_atoms = 46
[13.03|09:37:00] CheckEnergy: normalization coefficient = 46
[13.03|09:37:01] CheckEnergy: Actual Threshold
[13.03|09:37:01] CheckEnergy: dE/46 -0.0021 0.2000 OK!
[13.03|09:37:01] CheckEnergy: ddE/46 0.0004 0.0020 OK! (relative to step4_attempt2_simulation:MDStep259)
[13.03|09:37:01]
[13.03|09:37:01] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:37:01] CheckForces: ------------
[13.03|09:37:01] CheckForces: Reference job from step5_attempt1_reference_calc1
[13.03|09:37:01] CheckForces: Prediction job from final frame (MDStep769) of step5_attempt1_simulation
[13.03|09:37:01] CheckForces: ------------
[13.03|09:37:01] CheckForces: Histogram of forces
[13.03|09:37:01] CheckForces: eV/Ang Ref Pred
[13.03|09:37:01] CheckForces: -3 0 0
[13.03|09:37:01] CheckForces: -2 10 12
[13.03|09:37:01] CheckForces: -1 55 52
[13.03|09:37:01] CheckForces: 0 67 68
[13.03|09:37:01] CheckForces: 1 4 4
[13.03|09:37:01] CheckForces: 2 1 2
[13.03|09:37:01] CheckForces: 3 1 0
[13.03|09:37:01] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:37:01] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:37:01] CheckForces: Ref Pred Delta Threshold
[13.03|09:37:01] CheckForces: 0.88 0.46 0.43 0.35
[13.03|09:37:01] CheckForces: Maximum deviation: 0.426 eV/angstrom
[13.03|09:37:01] CheckForces: Actual Threshold
[13.03|09:37:01] CheckForces: # > thr. 1 0 Not OK!
[13.03|09:37:01] CheckForces: MAE 0.079 0.30 OK!
[13.03|09:37:01] CheckForces: R^2 0.977 0.80 OK!
[13.03|09:37:01] CheckForces: --------------------
[13.03|09:37:01]
[13.03|09:37:01] Adding results from step5_attempt1_reference_calc1 to training set
[13.03|09:37:01] Current # training set entries: 18
[13.03|09:37:01] Current # validation set entries: 9
[13.03|09:37:01] Storing data in step5_attempt1_reference_data
[13.03|09:37:01] Deleting step4_attempt2_reference_data
[13.03|09:37:01] Deleting step5_attempt1_reference_calc1
[13.03|09:37:01]
[13.03|09:37:01] Current (cumulative) timings:
[13.03|09:37:01] Time (s) Fraction
[13.03|09:37:01] Ref. calcs 78.52 0.077
[13.03|09:37:01] ML training 770.76 0.751
[13.03|09:37:01] Simulations 176.74 0.172
[13.03|09:37:01]
[13.03|09:37:01]
[13.03|09:37:01]
[13.03|09:37:01] --- Begin summary ---
[13.03|09:37:01] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:37:01] 1 1 FAILED Inaccurate 0.3894
[13.03|09:37:01] 1 2 SUCCESS Accurate 0.1974
[13.03|09:37:01] 2 1 FAILED Inaccurate 0.3629
[13.03|09:37:01] 2 2 SUCCESS Accurate 0.1848
[13.03|09:37:01] 3 1 FAILED Inaccurate 0.3716
[13.03|09:37:01] 3 2 SUCCESS Accurate 0.3045
[13.03|09:37:01] 4 1 FAILED Inaccurate 0.3887
[13.03|09:37:01] 4 2 SUCCESS Accurate 0.2201
[13.03|09:37:01] 5 1 FAILED Inaccurate 0.4263
[13.03|09:37:01] --- End summary ---
[13.03|09:37:01]
[13.03|09:37:01] Running more reference calculations....
[13.03|09:37:01] Running reference calculations on frames [57, 59, 62] from step5_attempt1_simulation/ams.rkf
[13.03|09:37:01] Calculating 3 frames in total
[13.03|09:37:01] Running step5_attempt1_reference_calc2
[13.03|09:37:04] Running step5_attempt1_reference_calc3
[13.03|09:37:07] Running step5_attempt1_reference_calc4
[13.03|09:37:10] Reference calculations finished!
[13.03|09:37:10] Adding results from step5_attempt1_reference_calc2 to validation set
[13.03|09:37:10] Adding results from step5_attempt1_reference_calc3 to training set
[13.03|09:37:10] Adding results from step5_attempt1_reference_calc4 to training set
[13.03|09:37:10] Current # training set entries: 20
[13.03|09:37:10] Current # validation set entries: 10
[13.03|09:37:10] Storing data in step5_attempt1_reference_data
[13.03|09:37:11] Deleting step5_attempt1_reference_calc2
[13.03|09:37:11] Deleting step5_attempt1_reference_calc3
[13.03|09:37:11] Deleting step5_attempt1_reference_calc4
[13.03|09:37:11] Launching reparametrization job: step5_attempt1_training
[13.03|09:37:16] JOB m3gnet STARTED
[13.03|09:37:16] Starting m3gnet.prerun()
[13.03|09:37:16] m3gnet.prerun() finished
[13.03|09:37:16] JOB m3gnet RUNNING
[13.03|09:37:16] Executing m3gnet.run
[13.03|09:38:02] training_set Optimizer: 001 Epoch: 0 Loss: 0.003187
[13.03|09:38:02] validation_set Optimizer: 001 Epoch: 0 Loss: 0.063171
[13.03|09:38:12] training_set Optimizer: 001 Epoch: 10 Loss: 0.000369
[13.03|09:38:12] validation_set Optimizer: 001 Epoch: 10 Loss: 0.008751
[13.03|09:38:21] training_set Optimizer: 001 Epoch: 20 Loss: 0.000302
[13.03|09:38:21] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006110
[13.03|09:38:31] training_set Optimizer: 001 Epoch: 30 Loss: 0.000284
[13.03|09:38:31] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005460
[13.03|09:38:41] training_set Optimizer: 001 Epoch: 40 Loss: 0.000271
[13.03|09:38:41] validation_set Optimizer: 001 Epoch: 40 Loss: 0.007775
[13.03|09:38:50] Execution of m3gnet.run finished with returncode 0
[13.03|09:38:50] JOB m3gnet FINISHED
[13.03|09:38:50] Starting m3gnet.postrun()
[13.03|09:38:50] m3gnet.postrun() finished
[13.03|09:38:51] JOB m3gnet SUCCESSFUL
[13.03|09:39:06] Running all jobs through AMS....
[13.03|09:39:07] Storing results/optimization/training_set_results/best
[13.03|09:39:07] Storing results/optimization/validation_set_results/best
[13.03|09:39:07] PLAMS environment cleaned up successfully
[13.03|09:39:07] PLAMS run finished. Goodbye
[13.03|09:39:08] ParAMSResults training_set validation_set
[13.03|09:39:08] energy MAE 0.0717 0.0438 eV
[13.03|09:39:08] forces MAE 0.0382 0.0499 eV/angstrom
[13.03|09:39:08] Newly created parameter file/dir: step5_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:39:08] Done!
[13.03|09:39:08] Deleting step4_attempt1_training
[13.03|09:39:08] ##########################
[13.03|09:39:08] ### Step 5 / Attempt 2 ###
[13.03|09:39:08] ##########################
[13.03|09:39:08] MD Steps: 510 (cumulative: 769)
[13.03|09:39:08] Current engine settings:
[13.03|09:39:08]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step5_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:39:08] Running step5_attempt2_simulation...
[13.03|09:39:46] Job step5_attempt2_simulation finished
[13.03|09:39:46] Deleting files that are no longer needed...
[13.03|09:39:46] Launching reference calculation
[13.03|09:39:49] Reference calculation finished!
[13.03|09:39:49] Checking success for step5_attempt2
[13.03|09:40:00] CheckEnergy: Checking energy for MDStep769, n_atoms = 46
[13.03|09:40:00] CheckEnergy: normalization coefficient = 46
[13.03|09:40:00] CheckEnergy: Actual Threshold
[13.03|09:40:00] CheckEnergy: dE/46 0.0002 0.2000 OK!
[13.03|09:40:00] CheckEnergy: ddE/46 0.0004 0.0020 OK! (relative to step5_attempt1_simulation:MDStep769)
[13.03|09:40:00]
[13.03|09:40:00] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:40:00] CheckForces: ------------
[13.03|09:40:00] CheckForces: Reference job from step5_attempt2_reference_calc1
[13.03|09:40:00] CheckForces: Prediction job from final frame (MDStep769) of step5_attempt2_simulation
[13.03|09:40:00] CheckForces: ------------
[13.03|09:40:00] CheckForces: Histogram of forces
[13.03|09:40:00] CheckForces: eV/Ang Ref Pred
[13.03|09:40:00] CheckForces: -3 0 0
[13.03|09:40:00] CheckForces: -2 12 12
[13.03|09:40:00] CheckForces: -1 52 47
[13.03|09:40:00] CheckForces: 0 68 73
[13.03|09:40:00] CheckForces: 1 4 4
[13.03|09:40:00] CheckForces: 2 2 2
[13.03|09:40:00] CheckForces: 3 0 0
[13.03|09:40:00] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:40:00] CheckForces: All force components are within the acceptable error!
[13.03|09:40:00] CheckForces: Maximum deviation: 0.193 eV/angstrom
[13.03|09:40:00] CheckForces: Actual Threshold
[13.03|09:40:00] CheckForces: # > thr. 0 0 OK!
[13.03|09:40:00] CheckForces: MAE 0.057 0.30 OK!
[13.03|09:40:00] CheckForces: R^2 0.988 0.80 OK!
[13.03|09:40:00] CheckForces: --------------------
[13.03|09:40:00]
[13.03|09:40:00] Adding results from step5_attempt2_reference_calc1 to training set
[13.03|09:40:00] Current # training set entries: 21
[13.03|09:40:00] Current # validation set entries: 10
[13.03|09:40:00] Storing data in step5_attempt2_reference_data
[13.03|09:40:00] Deleting step5_attempt1_reference_data
[13.03|09:40:00] Deleting step5_attempt2_reference_calc1
[13.03|09:40:00]
[13.03|09:40:00] Current (cumulative) timings:
[13.03|09:40:00] Time (s) Fraction
[13.03|09:40:00] Ref. calcs 90.55 0.076
[13.03|09:40:00] ML training 887.91 0.744
[13.03|09:40:00] Simulations 214.62 0.180
[13.03|09:40:00]
[13.03|09:40:00]
[13.03|09:40:00] Step 5 finished successfully!
[13.03|09:40:00]
[13.03|09:40:00] --- Begin summary ---
[13.03|09:40:00] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:40:00] 1 1 FAILED Inaccurate 0.3894
[13.03|09:40:00] 1 2 SUCCESS Accurate 0.1974
[13.03|09:40:00] 2 1 FAILED Inaccurate 0.3629
[13.03|09:40:00] 2 2 SUCCESS Accurate 0.1848
[13.03|09:40:00] 3 1 FAILED Inaccurate 0.3716
[13.03|09:40:00] 3 2 SUCCESS Accurate 0.3045
[13.03|09:40:00] 4 1 FAILED Inaccurate 0.3887
[13.03|09:40:00] 4 2 SUCCESS Accurate 0.2201
[13.03|09:40:00] 5 1 FAILED Inaccurate 0.4263
[13.03|09:40:00] 5 2 SUCCESS Accurate 0.1931
[13.03|09:40:00] --- End summary ---
[13.03|09:40:00]
[13.03|09:40:00] ##########################
[13.03|09:40:00] ### Step 6 / Attempt 1 ###
[13.03|09:40:00] ##########################
[13.03|09:40:00] MD Steps: 1510 (cumulative: 2279)
[13.03|09:40:00] Current engine settings:
[13.03|09:40:00]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step5_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:40:00] Running step6_attempt1_simulation...
[13.03|09:41:19] Job step6_attempt1_simulation finished
[13.03|09:41:19] Deleting files that are no longer needed...
[13.03|09:41:19] Deleting step4_attempt2_simulation
[13.03|09:41:19] Deleting step5_attempt1_simulation
[13.03|09:41:19] Launching reference calculation
[13.03|09:41:22] Reference calculation finished!
[13.03|09:41:22] Checking success for step6_attempt1
[13.03|09:41:33] CheckEnergy: Checking energy for MDStep2279, n_atoms = 46
[13.03|09:41:33] CheckEnergy: normalization coefficient = 46
[13.03|09:41:33] CheckEnergy: Actual Threshold
[13.03|09:41:33] CheckEnergy: dE/46 0.0010 0.2000 OK!
[13.03|09:41:33] CheckEnergy: ddE/46 0.0008 0.0020 OK! (relative to step5_attempt2_simulation:MDStep769)
[13.03|09:41:33]
[13.03|09:41:33] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:41:33] CheckForces: ------------
[13.03|09:41:33] CheckForces: Reference job from step6_attempt1_reference_calc1
[13.03|09:41:33] CheckForces: Prediction job from final frame (MDStep2279) of step6_attempt1_simulation
[13.03|09:41:33] CheckForces: ------------
[13.03|09:41:33] CheckForces: Histogram of forces
[13.03|09:41:33] CheckForces: eV/Ang Ref Pred
[13.03|09:41:33] CheckForces: -3 1 0
[13.03|09:41:33] CheckForces: -2 10 12
[13.03|09:41:33] CheckForces: -1 63 63
[13.03|09:41:33] CheckForces: 0 51 51
[13.03|09:41:33] CheckForces: 1 12 11
[13.03|09:41:33] CheckForces: 2 1 1
[13.03|09:41:33] CheckForces: 3 0 0
[13.03|09:41:33] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:41:33] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:41:33] CheckForces: Ref Pred Delta Threshold
[13.03|09:41:33] CheckForces: -0.91 -1.27 0.35 0.35
[13.03|09:41:33] CheckForces: Maximum deviation: 0.354 eV/angstrom
[13.03|09:41:33] CheckForces: Actual Threshold
[13.03|09:41:33] CheckForces: # > thr. 1 0 Not OK!
[13.03|09:41:33] CheckForces: MAE 0.074 0.30 OK!
[13.03|09:41:33] CheckForces: R^2 0.983 0.80 OK!
[13.03|09:41:33] CheckForces: --------------------
[13.03|09:41:33]
[13.03|09:41:33] Adding results from step6_attempt1_reference_calc1 to training set
[13.03|09:41:33] Current # training set entries: 22
[13.03|09:41:33] Current # validation set entries: 10
[13.03|09:41:34] Storing data in step6_attempt1_reference_data
[13.03|09:41:34] Deleting step5_attempt2_reference_data
[13.03|09:41:34] Deleting step6_attempt1_reference_calc1
[13.03|09:41:34]
[13.03|09:41:34] Current (cumulative) timings:
[13.03|09:41:34] Time (s) Fraction
[13.03|09:41:34] Ref. calcs 93.62 0.073
[13.03|09:41:34] ML training 887.91 0.697
[13.03|09:41:34] Simulations 293.15 0.230
[13.03|09:41:34]
[13.03|09:41:34]
[13.03|09:41:34]
[13.03|09:41:34] --- Begin summary ---
[13.03|09:41:34] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:41:34] 1 1 FAILED Inaccurate 0.3894
[13.03|09:41:34] 1 2 SUCCESS Accurate 0.1974
[13.03|09:41:34] 2 1 FAILED Inaccurate 0.3629
[13.03|09:41:34] 2 2 SUCCESS Accurate 0.1848
[13.03|09:41:34] 3 1 FAILED Inaccurate 0.3716
[13.03|09:41:34] 3 2 SUCCESS Accurate 0.3045
[13.03|09:41:34] 4 1 FAILED Inaccurate 0.3887
[13.03|09:41:34] 4 2 SUCCESS Accurate 0.2201
[13.03|09:41:34] 5 1 FAILED Inaccurate 0.4263
[13.03|09:41:34] 5 2 SUCCESS Accurate 0.1931
[13.03|09:41:34] 6 1 FAILED Inaccurate 0.3540
[13.03|09:41:34] --- End summary ---
[13.03|09:41:34]
[13.03|09:41:34] Running more reference calculations....
[13.03|09:41:34] Running reference calculations on frames [70, 73, 77] from step6_attempt1_simulation/ams.rkf
[13.03|09:41:34] Calculating 3 frames in total
[13.03|09:41:34] Running step6_attempt1_reference_calc2
[13.03|09:41:42] Running step6_attempt1_reference_calc3
[13.03|09:41:45] Running step6_attempt1_reference_calc4
[13.03|09:41:48] Reference calculations finished!
[13.03|09:41:48] Adding results from step6_attempt1_reference_calc2 to validation set
[13.03|09:41:48] Adding results from step6_attempt1_reference_calc3 to training set
[13.03|09:41:49] Adding results from step6_attempt1_reference_calc4 to training set
[13.03|09:41:49] Current # training set entries: 24
[13.03|09:41:49] Current # validation set entries: 11
[13.03|09:41:49] Storing data in step6_attempt1_reference_data
[13.03|09:41:49] Deleting step6_attempt1_reference_calc2
[13.03|09:41:49] Deleting step6_attempt1_reference_calc3
[13.03|09:41:49] Deleting step6_attempt1_reference_calc4
[13.03|09:41:49] Launching reparametrization job: step6_attempt1_training
[13.03|09:41:54] JOB m3gnet STARTED
[13.03|09:41:54] Starting m3gnet.prerun()
[13.03|09:41:54] m3gnet.prerun() finished
[13.03|09:41:54] JOB m3gnet RUNNING
[13.03|09:41:54] Executing m3gnet.run
[13.03|09:43:03] training_set Optimizer: 001 Epoch: 0 Loss: 0.001590
[13.03|09:43:03] validation_set Optimizer: 001 Epoch: 0 Loss: 0.031535
[13.03|09:43:15] training_set Optimizer: 001 Epoch: 10 Loss: 0.000335
[13.03|09:43:15] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006747
[13.03|09:43:28] training_set Optimizer: 001 Epoch: 20 Loss: 0.000316
[13.03|09:43:28] validation_set Optimizer: 001 Epoch: 20 Loss: 0.008348
[13.03|09:43:40] training_set Optimizer: 001 Epoch: 30 Loss: 0.000297
[13.03|09:43:40] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006324
[13.03|09:43:52] training_set Optimizer: 001 Epoch: 40 Loss: 0.000281
[13.03|09:43:52] validation_set Optimizer: 001 Epoch: 40 Loss: 0.007726
[13.03|09:44:04] training_set Optimizer: 001 Epoch: 50 Loss: 0.000275
[13.03|09:44:04] validation_set Optimizer: 001 Epoch: 50 Loss: 0.007796
[13.03|09:44:16] training_set Optimizer: 001 Epoch: 60 Loss: 0.000252
[13.03|09:44:16] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006528
[13.03|09:44:28] training_set Optimizer: 001 Epoch: 70 Loss: 0.000236
[13.03|09:44:28] validation_set Optimizer: 001 Epoch: 70 Loss: 0.007615
[13.03|09:44:41] training_set Optimizer: 001 Epoch: 80 Loss: 0.000229
[13.03|09:44:41] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006153
[13.03|09:44:53] training_set Optimizer: 001 Epoch: 90 Loss: 0.000220
[13.03|09:44:53] validation_set Optimizer: 001 Epoch: 90 Loss: 0.009211
[13.03|09:45:05] training_set Optimizer: 001 Epoch: 100 Loss: 0.000208
[13.03|09:45:05] validation_set Optimizer: 001 Epoch: 100 Loss: 0.006159
[13.03|09:45:17] training_set Optimizer: 001 Epoch: 110 Loss: 0.000209
[13.03|09:45:17] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011036
[13.03|09:45:29] training_set Optimizer: 001 Epoch: 120 Loss: 0.000187
[13.03|09:45:29] validation_set Optimizer: 001 Epoch: 120 Loss: 0.006845
[13.03|09:45:41] training_set Optimizer: 001 Epoch: 130 Loss: 0.000189
[13.03|09:45:41] validation_set Optimizer: 001 Epoch: 130 Loss: 0.008792
[13.03|09:45:53] training_set Optimizer: 001 Epoch: 140 Loss: 0.000187
[13.03|09:45:53] validation_set Optimizer: 001 Epoch: 140 Loss: 0.009989
[13.03|09:46:05] training_set Optimizer: 001 Epoch: 150 Loss: 0.000168
[13.03|09:46:05] validation_set Optimizer: 001 Epoch: 150 Loss: 0.006005
[13.03|09:46:17] training_set Optimizer: 001 Epoch: 160 Loss: 0.000200
[13.03|09:46:17] validation_set Optimizer: 001 Epoch: 160 Loss: 0.011610
[13.03|09:46:29] training_set Optimizer: 001 Epoch: 170 Loss: 0.000171
[13.03|09:46:29] validation_set Optimizer: 001 Epoch: 170 Loss: 0.008292
[13.03|09:46:42] training_set Optimizer: 001 Epoch: 180 Loss: 0.000154
[13.03|09:46:42] validation_set Optimizer: 001 Epoch: 180 Loss: 0.006353
[13.03|09:46:54] training_set Optimizer: 001 Epoch: 190 Loss: 0.000172
[13.03|09:46:54] validation_set Optimizer: 001 Epoch: 190 Loss: 0.008893
[13.03|09:47:07] Execution of m3gnet.run finished with returncode 0
[13.03|09:47:07] JOB m3gnet FINISHED
[13.03|09:47:07] Starting m3gnet.postrun()
[13.03|09:47:07] m3gnet.postrun() finished
[13.03|09:47:07] JOB m3gnet SUCCESSFUL
[13.03|09:47:25] Running all jobs through AMS....
[13.03|09:47:25] Storing results/optimization/training_set_results/best
[13.03|09:47:25] Storing results/optimization/validation_set_results/best
[13.03|09:47:25] PLAMS environment cleaned up successfully
[13.03|09:47:25] PLAMS run finished. Goodbye
[13.03|09:47:26] ParAMSResults training_set validation_set
[13.03|09:47:26] energy MAE 0.0301 0.0335 eV
[13.03|09:47:26] forces MAE 0.0285 0.0445 eV/angstrom
[13.03|09:47:26] Newly created parameter file/dir: step6_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|09:47:26] Done!
[13.03|09:47:26] Deleting step5_attempt1_training
[13.03|09:47:26] ##########################
[13.03|09:47:26] ### Step 6 / Attempt 2 ###
[13.03|09:47:26] ##########################
[13.03|09:47:26] MD Steps: 1510 (cumulative: 2279)
[13.03|09:47:26] Current engine settings:
[13.03|09:47:26]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step6_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:47:26] Running step6_attempt2_simulation...
[13.03|09:48:45] Job step6_attempt2_simulation finished
[13.03|09:48:45] Deleting files that are no longer needed...
[13.03|09:48:46] Launching reference calculation
[13.03|09:48:49] Reference calculation finished!
[13.03|09:48:49] Checking success for step6_attempt2
[13.03|09:49:00] CheckEnergy: Checking energy for MDStep2279, n_atoms = 46
[13.03|09:49:00] CheckEnergy: normalization coefficient = 46
[13.03|09:49:00] CheckEnergy: Actual Threshold
[13.03|09:49:00] CheckEnergy: dE/46 0.0004 0.2000 OK!
[13.03|09:49:00] CheckEnergy: ddE/46 -0.0006 0.0020 OK! (relative to step6_attempt1_simulation:MDStep2279)
[13.03|09:49:00]
[13.03|09:49:00] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:49:00] CheckForces: ------------
[13.03|09:49:00] CheckForces: Reference job from step6_attempt2_reference_calc1
[13.03|09:49:00] CheckForces: Prediction job from final frame (MDStep2279) of step6_attempt2_simulation
[13.03|09:49:00] CheckForces: ------------
[13.03|09:49:00] CheckForces: Histogram of forces
[13.03|09:49:00] CheckForces: eV/Ang Ref Pred
[13.03|09:49:00] CheckForces: -4 0 0
[13.03|09:49:00] CheckForces: -3 3 2
[13.03|09:49:00] CheckForces: -2 10 12
[13.03|09:49:00] CheckForces: -1 57 57
[13.03|09:49:00] CheckForces: 0 56 58
[13.03|09:49:00] CheckForces: 1 9 7
[13.03|09:49:00] CheckForces: 2 2 1
[13.03|09:49:00] CheckForces: 3 1 1
[13.03|09:49:00] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:49:00] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:49:00] CheckForces: Ref Pred Delta Threshold
[13.03|09:49:00] CheckForces: -0.96 -0.46 0.50 0.35
[13.03|09:49:00] CheckForces: 1.33 1.77 0.44 0.38
[13.03|09:49:00] CheckForces: 1.01 1.38 0.37 0.36
[13.03|09:49:00] CheckForces: -0.12 -0.46 0.34 0.31
[13.03|09:49:00] CheckForces: Maximum deviation: 0.499 eV/angstrom
[13.03|09:49:00] CheckForces: Actual Threshold
[13.03|09:49:00] CheckForces: # > thr. 4 0 Not OK!
[13.03|09:49:00] CheckForces: MAE 0.070 0.30 OK!
[13.03|09:49:00] CheckForces: R^2 0.983 0.80 OK!
[13.03|09:49:00] CheckForces: --------------------
[13.03|09:49:00]
[13.03|09:49:00] Adding results from step6_attempt2_reference_calc1 to training set
[13.03|09:49:00] Current # training set entries: 25
[13.03|09:49:00] Current # validation set entries: 11
[13.03|09:49:00] Storing data in step6_attempt2_reference_data
[13.03|09:49:00] Deleting step6_attempt1_reference_data
[13.03|09:49:00] Deleting step6_attempt2_reference_calc1
[13.03|09:49:00]
[13.03|09:49:00] Current (cumulative) timings:
[13.03|09:49:00] Time (s) Fraction
[13.03|09:49:00] Ref. calcs 110.80 0.065
[13.03|09:49:00] ML training 1225.22 0.717
[13.03|09:49:00] Simulations 372.42 0.218
[13.03|09:49:00]
[13.03|09:49:00]
[13.03|09:49:00]
[13.03|09:49:00] --- Begin summary ---
[13.03|09:49:00] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:49:00] 1 1 FAILED Inaccurate 0.3894
[13.03|09:49:00] 1 2 SUCCESS Accurate 0.1974
[13.03|09:49:00] 2 1 FAILED Inaccurate 0.3629
[13.03|09:49:00] 2 2 SUCCESS Accurate 0.1848
[13.03|09:49:00] 3 1 FAILED Inaccurate 0.3716
[13.03|09:49:00] 3 2 SUCCESS Accurate 0.3045
[13.03|09:49:00] 4 1 FAILED Inaccurate 0.3887
[13.03|09:49:00] 4 2 SUCCESS Accurate 0.2201
[13.03|09:49:00] 5 1 FAILED Inaccurate 0.4263
[13.03|09:49:00] 5 2 SUCCESS Accurate 0.1931
[13.03|09:49:00] 6 1 FAILED Inaccurate 0.3540
[13.03|09:49:00] 6 2 FAILED Inaccurate 0.4992
[13.03|09:49:00] --- End summary ---
[13.03|09:49:00]
[13.03|09:49:00] Running more reference calculations....
[13.03|09:49:00] Running reference calculations on frames [70, 73, 77] from step6_attempt2_simulation/ams.rkf
[13.03|09:49:00] Calculating 3 frames in total
[13.03|09:49:00] Running step6_attempt2_reference_calc2
[13.03|09:49:03] Running step6_attempt2_reference_calc3
[13.03|09:49:06] Running step6_attempt2_reference_calc4
[13.03|09:49:09] Reference calculations finished!
[13.03|09:49:09] Adding results from step6_attempt2_reference_calc2 to validation set
[13.03|09:49:10] Adding results from step6_attempt2_reference_calc3 to training set
[13.03|09:49:10] Adding results from step6_attempt2_reference_calc4 to training set
[13.03|09:49:10] Current # training set entries: 27
[13.03|09:49:10] Current # validation set entries: 12
[13.03|09:49:10] Storing data in step6_attempt2_reference_data
[13.03|09:49:10] Deleting step6_attempt2_reference_calc2
[13.03|09:49:10] Deleting step6_attempt2_reference_calc3
[13.03|09:49:10] Deleting step6_attempt2_reference_calc4
[13.03|09:49:10] Launching reparametrization job: step6_attempt2_training
[13.03|09:49:15] JOB m3gnet STARTED
[13.03|09:49:15] Starting m3gnet.prerun()
[13.03|09:49:15] m3gnet.prerun() finished
[13.03|09:49:15] JOB m3gnet RUNNING
[13.03|09:49:15] Executing m3gnet.run
[13.03|09:50:24] training_set Optimizer: 001 Epoch: 0 Loss: 0.002102
[13.03|09:50:24] validation_set Optimizer: 001 Epoch: 0 Loss: 0.045540
[13.03|09:50:38] training_set Optimizer: 001 Epoch: 10 Loss: 0.000224
[13.03|09:50:38] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006848
[13.03|09:50:53] training_set Optimizer: 001 Epoch: 20 Loss: 0.000208
[13.03|09:50:53] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006351
[13.03|09:51:07] training_set Optimizer: 001 Epoch: 30 Loss: 0.000190
[13.03|09:51:07] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006347
[13.03|09:51:20] training_set Optimizer: 001 Epoch: 40 Loss: 0.000179
[13.03|09:51:20] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006575
[13.03|09:51:35] training_set Optimizer: 001 Epoch: 50 Loss: 0.000185
[13.03|09:51:35] validation_set Optimizer: 001 Epoch: 50 Loss: 0.006361
[13.03|09:51:48] training_set Optimizer: 001 Epoch: 60 Loss: 0.000171
[13.03|09:51:48] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007117
[13.03|09:52:02] training_set Optimizer: 001 Epoch: 70 Loss: 0.000155
[13.03|09:52:02] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006618
[13.03|09:52:16] training_set Optimizer: 001 Epoch: 80 Loss: 0.000160
[13.03|09:52:16] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006604
[13.03|09:52:30] training_set Optimizer: 001 Epoch: 90 Loss: 0.000153
[13.03|09:52:30] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006586
[13.03|09:52:43] training_set Optimizer: 001 Epoch: 100 Loss: 0.000145
[13.03|09:52:43] validation_set Optimizer: 001 Epoch: 100 Loss: 0.006977
[13.03|09:52:58] training_set Optimizer: 001 Epoch: 110 Loss: 0.000152
[13.03|09:52:58] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006393
[13.03|09:53:11] training_set Optimizer: 001 Epoch: 120 Loss: 0.000140
[13.03|09:53:11] validation_set Optimizer: 001 Epoch: 120 Loss: 0.006292
[13.03|09:53:25] training_set Optimizer: 001 Epoch: 130 Loss: 0.000134
[13.03|09:53:25] validation_set Optimizer: 001 Epoch: 130 Loss: 0.006681
[13.03|09:53:39] training_set Optimizer: 001 Epoch: 140 Loss: 0.000129
[13.03|09:53:39] validation_set Optimizer: 001 Epoch: 140 Loss: 0.008101
[13.03|09:53:53] training_set Optimizer: 001 Epoch: 150 Loss: 0.000137
[13.03|09:53:53] validation_set Optimizer: 001 Epoch: 150 Loss: 0.010731
[13.03|09:54:07] training_set Optimizer: 001 Epoch: 160 Loss: 0.000135
[13.03|09:54:07] validation_set Optimizer: 001 Epoch: 160 Loss: 0.007180
[13.03|09:54:21] training_set Optimizer: 001 Epoch: 170 Loss: 0.000132
[13.03|09:54:21] validation_set Optimizer: 001 Epoch: 170 Loss: 0.011654
[13.03|09:54:35] training_set Optimizer: 001 Epoch: 180 Loss: 0.000140
[13.03|09:54:35] validation_set Optimizer: 001 Epoch: 180 Loss: 0.010158
[13.03|09:54:49] training_set Optimizer: 001 Epoch: 190 Loss: 0.000124
[13.03|09:54:49] validation_set Optimizer: 001 Epoch: 190 Loss: 0.010093
[13.03|09:55:04] Execution of m3gnet.run finished with returncode 0
[13.03|09:55:04] JOB m3gnet FINISHED
[13.03|09:55:04] Starting m3gnet.postrun()
[13.03|09:55:04] m3gnet.postrun() finished
[13.03|09:55:04] JOB m3gnet SUCCESSFUL
[13.03|09:55:24] Running all jobs through AMS....
[13.03|09:55:24] Storing results/optimization/training_set_results/best
[13.03|09:55:24] Storing results/optimization/validation_set_results/best
[13.03|09:55:24] PLAMS environment cleaned up successfully
[13.03|09:55:24] PLAMS run finished. Goodbye
[13.03|09:55:26] ParAMSResults training_set validation_set
[13.03|09:55:26] energy MAE 0.0981 0.0877 eV
[13.03|09:55:26] forces MAE 0.0257 0.0441 eV/angstrom
[13.03|09:55:26] Newly created parameter file/dir: step6_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|09:55:26] Done!
[13.03|09:55:26] Deleting step6_attempt1_training
[13.03|09:55:26] ##########################
[13.03|09:55:26] ### Step 6 / Attempt 3 ###
[13.03|09:55:26] ##########################
[13.03|09:55:26] MD Steps: 1510 (cumulative: 2279)
[13.03|09:55:26] Current engine settings:
[13.03|09:55:26]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step6_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|09:55:26] Running step6_attempt3_simulation...
[13.03|09:56:46] Job step6_attempt3_simulation finished
[13.03|09:56:46] Deleting files that are no longer needed...
[13.03|09:56:47] Launching reference calculation
[13.03|09:56:50] Reference calculation finished!
[13.03|09:56:50] Checking success for step6_attempt3
[13.03|09:57:01] CheckEnergy: Checking energy for MDStep2279, n_atoms = 46
[13.03|09:57:01] CheckEnergy: normalization coefficient = 46
[13.03|09:57:01] CheckEnergy: Actual Threshold
[13.03|09:57:01] CheckEnergy: dE/46 -0.0014 0.2000 OK!
[13.03|09:57:01] CheckEnergy: ddE/46 -0.0039 0.0020 Not OK! (relative to step6_attempt2_simulation:MDStep2279)
[13.03|09:57:01]
[13.03|09:57:01] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|09:57:01] CheckForces: ------------
[13.03|09:57:01] CheckForces: Reference job from step6_attempt3_reference_calc1
[13.03|09:57:01] CheckForces: Prediction job from final frame (MDStep2279) of step6_attempt3_simulation
[13.03|09:57:01] CheckForces: ------------
[13.03|09:57:01] CheckForces: Histogram of forces
[13.03|09:57:01] CheckForces: eV/Ang Ref Pred
[13.03|09:57:01] CheckForces: -3 0 0
[13.03|09:57:01] CheckForces: -2 16 13
[13.03|09:57:01] CheckForces: -1 55 57
[13.03|09:57:01] CheckForces: 0 55 57
[13.03|09:57:01] CheckForces: 1 9 8
[13.03|09:57:01] CheckForces: 2 2 2
[13.03|09:57:01] CheckForces: 3 1 1
[13.03|09:57:01] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|09:57:01] CheckForces: Force components with an error exceeding the threshold:
[13.03|09:57:01] CheckForces: Ref Pred Delta Threshold
[13.03|09:57:01] CheckForces: 0.60 0.25 0.35 0.33
[13.03|09:57:01] CheckForces: -1.34 -0.94 0.40 0.38
[13.03|09:57:01] CheckForces: Maximum deviation: 0.399 eV/angstrom
[13.03|09:57:01] CheckForces: Actual Threshold
[13.03|09:57:01] CheckForces: # > thr. 2 0 Not OK!
[13.03|09:57:01] CheckForces: MAE 0.070 0.30 OK!
[13.03|09:57:01] CheckForces: R^2 0.983 0.80 OK!
[13.03|09:57:01] CheckForces: --------------------
[13.03|09:57:01]
[13.03|09:57:01] Adding results from step6_attempt3_reference_calc1 to training set
[13.03|09:57:01] Current # training set entries: 28
[13.03|09:57:01] Current # validation set entries: 12
[13.03|09:57:01] Storing data in step6_attempt3_reference_data
[13.03|09:57:01] Deleting step6_attempt2_reference_data
[13.03|09:57:01] Deleting step6_attempt3_reference_calc1
[13.03|09:57:01]
[13.03|09:57:01] Current (cumulative) timings:
[13.03|09:57:01] Time (s) Fraction
[13.03|09:57:01] Ref. calcs 122.61 0.056
[13.03|09:57:01] ML training 1600.96 0.735
[13.03|09:57:01] Simulations 453.27 0.208
[13.03|09:57:01]
[13.03|09:57:01]
[13.03|09:57:02]
[13.03|09:57:02] --- Begin summary ---
[13.03|09:57:02] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|09:57:02] 1 1 FAILED Inaccurate 0.3894
[13.03|09:57:02] 1 2 SUCCESS Accurate 0.1974
[13.03|09:57:02] 2 1 FAILED Inaccurate 0.3629
[13.03|09:57:02] 2 2 SUCCESS Accurate 0.1848
[13.03|09:57:02] 3 1 FAILED Inaccurate 0.3716
[13.03|09:57:02] 3 2 SUCCESS Accurate 0.3045
[13.03|09:57:02] 4 1 FAILED Inaccurate 0.3887
[13.03|09:57:02] 4 2 SUCCESS Accurate 0.2201
[13.03|09:57:02] 5 1 FAILED Inaccurate 0.4263
[13.03|09:57:02] 5 2 SUCCESS Accurate 0.1931
[13.03|09:57:02] 6 1 FAILED Inaccurate 0.3540
[13.03|09:57:02] 6 2 FAILED Inaccurate 0.4992
[13.03|09:57:02] 6 3 FAILED Inaccurate 0.3989
[13.03|09:57:02] --- End summary ---
[13.03|09:57:02]
[13.03|09:57:02] Running more reference calculations....
[13.03|09:57:02] Running reference calculations on frames [70, 73, 77] from step6_attempt3_simulation/ams.rkf
[13.03|09:57:02] Calculating 3 frames in total
[13.03|09:57:02] Running step6_attempt3_reference_calc2
[13.03|09:57:05] Running step6_attempt3_reference_calc3
[13.03|09:57:08] Running step6_attempt3_reference_calc4
[13.03|09:57:11] Reference calculations finished!
[13.03|09:57:11] Adding results from step6_attempt3_reference_calc2 to validation set
[13.03|09:57:11] Adding results from step6_attempt3_reference_calc3 to training set
[13.03|09:57:11] Adding results from step6_attempt3_reference_calc4 to training set
[13.03|09:57:11] Current # training set entries: 30
[13.03|09:57:11] Current # validation set entries: 13
[13.03|09:57:11] Storing data in step6_attempt3_reference_data
[13.03|09:57:11] Deleting step6_attempt3_reference_calc2
[13.03|09:57:11] Deleting step6_attempt3_reference_calc3
[13.03|09:57:11] Deleting step6_attempt3_reference_calc4
[13.03|09:57:11] Launching reparametrization job: step6_attempt3_training
[13.03|09:57:17] JOB m3gnet STARTED
[13.03|09:57:17] Starting m3gnet.prerun()
[13.03|09:57:17] m3gnet.prerun() finished
[13.03|09:57:17] JOB m3gnet RUNNING
[13.03|09:57:17] Executing m3gnet.run
[13.03|09:58:07] training_set Optimizer: 001 Epoch: 0 Loss: 0.002553
[13.03|09:58:07] validation_set Optimizer: 001 Epoch: 0 Loss: 0.040008
[13.03|09:58:21] training_set Optimizer: 001 Epoch: 10 Loss: 0.000175
[13.03|09:58:21] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006190
[13.03|09:58:36] training_set Optimizer: 001 Epoch: 20 Loss: 0.000156
[13.03|09:58:36] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006863
[13.03|09:58:50] training_set Optimizer: 001 Epoch: 30 Loss: 0.000151
[13.03|09:58:50] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006305
[13.03|09:59:05] training_set Optimizer: 001 Epoch: 40 Loss: 0.000142
[13.03|09:59:05] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006219
[13.03|09:59:20] training_set Optimizer: 001 Epoch: 50 Loss: 0.000141
[13.03|09:59:20] validation_set Optimizer: 001 Epoch: 50 Loss: 0.006132
[13.03|09:59:34] training_set Optimizer: 001 Epoch: 60 Loss: 0.000141
[13.03|09:59:34] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006583
[13.03|09:59:49] training_set Optimizer: 001 Epoch: 70 Loss: 0.000129
[13.03|09:59:49] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006178
[13.03|10:00:03] training_set Optimizer: 001 Epoch: 80 Loss: 0.000128
[13.03|10:00:03] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006936
[13.03|10:00:18] training_set Optimizer: 001 Epoch: 90 Loss: 0.000125
[13.03|10:00:18] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006457
[13.03|10:00:21] Execution of m3gnet.run finished with returncode 0
[13.03|10:00:21] JOB m3gnet FINISHED
[13.03|10:00:21] Starting m3gnet.postrun()
[13.03|10:00:21] m3gnet.postrun() finished
[13.03|10:00:22] JOB m3gnet SUCCESSFUL
[13.03|10:00:39] Running all jobs through AMS....
[13.03|10:00:39] Storing results/optimization/training_set_results/best
[13.03|10:00:39] Storing results/optimization/validation_set_results/best
[13.03|10:00:39] PLAMS environment cleaned up successfully
[13.03|10:00:39] PLAMS run finished. Goodbye
[13.03|10:00:41] ParAMSResults training_set validation_set
[13.03|10:00:41] energy MAE 0.0309 0.0509 eV
[13.03|10:00:41] forces MAE 0.0270 0.0461 eV/angstrom
[13.03|10:00:41] Newly created parameter file/dir: step6_attempt3_training/results/optimization/m3gnet/m3gnet
[13.03|10:00:41] Done!
[13.03|10:00:41] Deleting step6_attempt2_training
[13.03|10:00:41] ##########################
[13.03|10:00:41] ### Step 6 / Attempt 4 ###
[13.03|10:00:41] ##########################
[13.03|10:00:41] MD Steps: 1510 (cumulative: 2279)
[13.03|10:00:41] Current engine settings:
[13.03|10:00:41]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step6_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:00:41] Running step6_attempt4_simulation...
[13.03|10:02:01] Job step6_attempt4_simulation finished
[13.03|10:02:01] Deleting files that are no longer needed...
[13.03|10:02:01] Launching reference calculation
[13.03|10:02:04] Reference calculation finished!
[13.03|10:02:04] Checking success for step6_attempt4
[13.03|10:02:15] CheckEnergy: Checking energy for MDStep2279, n_atoms = 46
[13.03|10:02:15] CheckEnergy: normalization coefficient = 46
[13.03|10:02:15] CheckEnergy: Actual Threshold
[13.03|10:02:15] CheckEnergy: dE/46 -0.0012 0.2000 OK!
[13.03|10:02:15] CheckEnergy: ddE/46 0.0007 0.0020 OK! (relative to step6_attempt3_simulation:MDStep2279)
[13.03|10:02:15]
[13.03|10:02:15] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:02:15] CheckForces: ------------
[13.03|10:02:15] CheckForces: Reference job from step6_attempt4_reference_calc1
[13.03|10:02:15] CheckForces: Prediction job from final frame (MDStep2279) of step6_attempt4_simulation
[13.03|10:02:15] CheckForces: ------------
[13.03|10:02:15] CheckForces: Histogram of forces
[13.03|10:02:15] CheckForces: eV/Ang Ref Pred
[13.03|10:02:15] CheckForces: -3 1 1
[13.03|10:02:15] CheckForces: -2 11 11
[13.03|10:02:15] CheckForces: -1 57 57
[13.03|10:02:15] CheckForces: 0 60 59
[13.03|10:02:15] CheckForces: 1 7 8
[13.03|10:02:15] CheckForces: 2 2 2
[13.03|10:02:15] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:02:15] CheckForces: All force components are within the acceptable error!
[13.03|10:02:15] CheckForces: Maximum deviation: 0.300 eV/angstrom
[13.03|10:02:15] CheckForces: Actual Threshold
[13.03|10:02:15] CheckForces: # > thr. 0 0 OK!
[13.03|10:02:15] CheckForces: MAE 0.053 0.30 OK!
[13.03|10:02:15] CheckForces: R^2 0.989 0.80 OK!
[13.03|10:02:15] CheckForces: --------------------
[13.03|10:02:15]
[13.03|10:02:16] Adding results from step6_attempt4_reference_calc1 to validation set
[13.03|10:02:16] Current # training set entries: 30
[13.03|10:02:16] Current # validation set entries: 14
[13.03|10:02:16] Storing data in step6_attempt4_reference_data
[13.03|10:02:16] Deleting step6_attempt3_reference_data
[13.03|10:02:16] Deleting step6_attempt4_reference_calc1
[13.03|10:02:16]
[13.03|10:02:16] Current (cumulative) timings:
[13.03|10:02:16] Time (s) Fraction
[13.03|10:02:16] Ref. calcs 134.69 0.054
[13.03|10:02:16] ML training 1810.19 0.730
[13.03|10:02:16] Simulations 533.30 0.215
[13.03|10:02:16]
[13.03|10:02:16]
[13.03|10:02:16] Step 6 finished successfully!
[13.03|10:02:16]
[13.03|10:02:16] --- Begin summary ---
[13.03|10:02:16] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:02:16] 1 1 FAILED Inaccurate 0.3894
[13.03|10:02:16] 1 2 SUCCESS Accurate 0.1974
[13.03|10:02:16] 2 1 FAILED Inaccurate 0.3629
[13.03|10:02:16] 2 2 SUCCESS Accurate 0.1848
[13.03|10:02:16] 3 1 FAILED Inaccurate 0.3716
[13.03|10:02:16] 3 2 SUCCESS Accurate 0.3045
[13.03|10:02:16] 4 1 FAILED Inaccurate 0.3887
[13.03|10:02:16] 4 2 SUCCESS Accurate 0.2201
[13.03|10:02:16] 5 1 FAILED Inaccurate 0.4263
[13.03|10:02:16] 5 2 SUCCESS Accurate 0.1931
[13.03|10:02:16] 6 1 FAILED Inaccurate 0.3540
[13.03|10:02:16] 6 2 FAILED Inaccurate 0.4992
[13.03|10:02:16] 6 3 FAILED Inaccurate 0.3989
[13.03|10:02:16] 6 4 SUCCESS Accurate 0.3000
[13.03|10:02:16] --- End summary ---
[13.03|10:02:16]
[13.03|10:02:16] ##########################
[13.03|10:02:16] ### Step 7 / Attempt 1 ###
[13.03|10:02:16] ##########################
[13.03|10:02:16] MD Steps: 4473 (cumulative: 6752)
[13.03|10:02:16] Current engine settings:
[13.03|10:02:16]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step6_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:02:16] Running step7_attempt1_simulation...
[13.03|10:05:38] Job step7_attempt1_simulation finished
[13.03|10:05:38] Deleting files that are no longer needed...
[13.03|10:05:38] Deleting step5_attempt2_simulation
[13.03|10:05:38] Deleting step6_attempt1_simulation
[13.03|10:05:38] Deleting step6_attempt2_simulation
[13.03|10:05:38] Deleting step6_attempt3_simulation
[13.03|10:05:38] Launching reference calculation
[13.03|10:05:41] Reference calculation finished!
[13.03|10:05:41] Checking success for step7_attempt1
[13.03|10:05:53] CheckEnergy: Checking energy for MDStep6752, n_atoms = 46
[13.03|10:05:53] CheckEnergy: normalization coefficient = 46
[13.03|10:05:53] CheckEnergy: Actual Threshold
[13.03|10:05:53] CheckEnergy: dE/46 0.0001 0.2000 OK!
[13.03|10:05:53] CheckEnergy: ddE/46 0.0013 0.0020 OK! (relative to step6_attempt4_simulation:MDStep2279)
[13.03|10:05:53]
[13.03|10:05:53] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:05:53] CheckForces: ------------
[13.03|10:05:53] CheckForces: Reference job from step7_attempt1_reference_calc1
[13.03|10:05:53] CheckForces: Prediction job from final frame (MDStep6752) of step7_attempt1_simulation
[13.03|10:05:53] CheckForces: ------------
[13.03|10:05:53] CheckForces: Histogram of forces
[13.03|10:05:53] CheckForces: eV/Ang Ref Pred
[13.03|10:05:53] CheckForces: -3 0 0
[13.03|10:05:53] CheckForces: -2 15 16
[13.03|10:05:53] CheckForces: -1 56 56
[13.03|10:05:53] CheckForces: 0 47 47
[13.03|10:05:53] CheckForces: 1 20 19
[13.03|10:05:53] CheckForces: 2 0 0
[13.03|10:05:53] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:05:53] CheckForces: Force components with an error exceeding the threshold:
[13.03|10:05:53] CheckForces: Ref Pred Delta Threshold
[13.03|10:05:53] CheckForces: -1.42 -1.03 0.39 0.39
[13.03|10:05:53] CheckForces: -0.50 -0.88 0.39 0.32
[13.03|10:05:53] CheckForces: 0.72 0.18 0.54 0.34
[13.03|10:05:53] CheckForces: Maximum deviation: 0.540 eV/angstrom
[13.03|10:05:53] CheckForces: Actual Threshold
[13.03|10:05:53] CheckForces: # > thr. 3 0 Not OK!
[13.03|10:05:53] CheckForces: MAE 0.068 0.30 OK!
[13.03|10:05:53] CheckForces: R^2 0.984 0.80 OK!
[13.03|10:05:53] CheckForces: --------------------
[13.03|10:05:53]
[13.03|10:05:53] Adding results from step7_attempt1_reference_calc1 to training set
[13.03|10:05:53] Current # training set entries: 31
[13.03|10:05:53] Current # validation set entries: 14
[13.03|10:05:53] Storing data in step7_attempt1_reference_data
[13.03|10:05:54] Deleting step6_attempt4_reference_data
[13.03|10:05:54] Deleting step7_attempt1_reference_calc1
[13.03|10:05:54]
[13.03|10:05:54] Current (cumulative) timings:
[13.03|10:05:54] Time (s) Fraction
[13.03|10:05:54] Ref. calcs 137.64 0.051
[13.03|10:05:54] ML training 1810.19 0.675
[13.03|10:05:54] Simulations 734.76 0.274
[13.03|10:05:54]
[13.03|10:05:54]
[13.03|10:05:54]
[13.03|10:05:54] --- Begin summary ---
[13.03|10:05:54] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:05:54] 1 1 FAILED Inaccurate 0.3894
[13.03|10:05:54] 1 2 SUCCESS Accurate 0.1974
[13.03|10:05:54] 2 1 FAILED Inaccurate 0.3629
[13.03|10:05:54] 2 2 SUCCESS Accurate 0.1848
[13.03|10:05:54] 3 1 FAILED Inaccurate 0.3716
[13.03|10:05:54] 3 2 SUCCESS Accurate 0.3045
[13.03|10:05:54] 4 1 FAILED Inaccurate 0.3887
[13.03|10:05:54] 4 2 SUCCESS Accurate 0.2201
[13.03|10:05:54] 5 1 FAILED Inaccurate 0.4263
[13.03|10:05:54] 5 2 SUCCESS Accurate 0.1931
[13.03|10:05:54] 6 1 FAILED Inaccurate 0.3540
[13.03|10:05:54] 6 2 FAILED Inaccurate 0.4992
[13.03|10:05:54] 6 3 FAILED Inaccurate 0.3989
[13.03|10:05:54] 6 4 SUCCESS Accurate 0.3000
[13.03|10:05:54] 7 1 FAILED Inaccurate 0.5401
[13.03|10:05:54] --- End summary ---
[13.03|10:05:54]
[13.03|10:05:54] Running more reference calculations....
[13.03|10:05:54] Running reference calculations on frames [100, 111, 122] from step7_attempt1_simulation/ams.rkf
[13.03|10:05:54] Calculating 3 frames in total
[13.03|10:05:54] Running step7_attempt1_reference_calc2
[13.03|10:05:57] Running step7_attempt1_reference_calc3
[13.03|10:06:00] Running step7_attempt1_reference_calc4
[13.03|10:06:03] Reference calculations finished!
[13.03|10:06:03] Adding results from step7_attempt1_reference_calc2 to validation set
[13.03|10:06:03] Adding results from step7_attempt1_reference_calc3 to training set
[13.03|10:06:04] Adding results from step7_attempt1_reference_calc4 to training set
[13.03|10:06:04] Current # training set entries: 33
[13.03|10:06:04] Current # validation set entries: 15
[13.03|10:06:04] Storing data in step7_attempt1_reference_data
[13.03|10:06:04] Deleting step7_attempt1_reference_calc2
[13.03|10:06:04] Deleting step7_attempt1_reference_calc3
[13.03|10:06:04] Deleting step7_attempt1_reference_calc4
[13.03|10:06:04] Launching reparametrization job: step7_attempt1_training
[13.03|10:06:10] JOB m3gnet STARTED
[13.03|10:06:10] Starting m3gnet.prerun()
[13.03|10:06:10] m3gnet.prerun() finished
[13.03|10:06:10] JOB m3gnet RUNNING
[13.03|10:06:10] Executing m3gnet.run
[13.03|10:07:17] training_set Optimizer: 001 Epoch: 0 Loss: 0.001434
[13.03|10:07:17] validation_set Optimizer: 001 Epoch: 0 Loss: 0.016553
[13.03|10:07:28] Execution of m3gnet.run finished with returncode 0
[13.03|10:07:28] JOB m3gnet FINISHED
[13.03|10:07:28] Starting m3gnet.postrun()
[13.03|10:07:28] m3gnet.postrun() finished
[13.03|10:07:29] JOB m3gnet SUCCESSFUL
[13.03|10:07:47] Running all jobs through AMS....
[13.03|10:07:47] Storing results/optimization/training_set_results/best
[13.03|10:07:47] Storing results/optimization/validation_set_results/best
[13.03|10:07:47] PLAMS environment cleaned up successfully
[13.03|10:07:47] PLAMS run finished. Goodbye
[13.03|10:07:49] ParAMSResults training_set validation_set
[13.03|10:07:49] energy MAE 0.2058 0.2292 eV
[13.03|10:07:49] forces MAE 0.0315 0.0487 eV/angstrom
[13.03|10:07:49] Newly created parameter file/dir: step7_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|10:07:49] Done!
[13.03|10:07:49] Deleting step6_attempt3_training
[13.03|10:07:49] ##########################
[13.03|10:07:49] ### Step 7 / Attempt 2 ###
[13.03|10:07:49] ##########################
[13.03|10:07:49] MD Steps: 4473 (cumulative: 6752)
[13.03|10:07:49] Current engine settings:
[13.03|10:07:49]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step7_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:07:49] Running step7_attempt2_simulation...
[13.03|10:11:16] Job step7_attempt2_simulation finished
[13.03|10:11:16] Deleting files that are no longer needed...
[13.03|10:11:17] Launching reference calculation
[13.03|10:11:19] Reference calculation finished!
[13.03|10:11:19] Checking success for step7_attempt2
[13.03|10:11:33] CheckEnergy: Checking energy for MDStep6752, n_atoms = 46
[13.03|10:11:33] CheckEnergy: normalization coefficient = 46
[13.03|10:11:33] CheckEnergy: Actual Threshold
[13.03|10:11:33] CheckEnergy: dE/46 -0.0058 0.2000 OK!
[13.03|10:11:33] CheckEnergy: ddE/46 -0.0017 0.0020 OK! (relative to step7_attempt1_simulation:MDStep6752)
[13.03|10:11:33]
[13.03|10:11:33] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:11:33] CheckForces: ------------
[13.03|10:11:33] CheckForces: Reference job from step7_attempt2_reference_calc1
[13.03|10:11:33] CheckForces: Prediction job from final frame (MDStep6752) of step7_attempt2_simulation
[13.03|10:11:33] CheckForces: ------------
[13.03|10:11:33] CheckForces: Histogram of forces
[13.03|10:11:33] CheckForces: eV/Ang Ref Pred
[13.03|10:11:33] CheckForces: -4 0 0
[13.03|10:11:33] CheckForces: -3 3 3
[13.03|10:11:33] CheckForces: -2 8 8
[13.03|10:11:33] CheckForces: -1 54 56
[13.03|10:11:33] CheckForces: 0 63 62
[13.03|10:11:33] CheckForces: 1 9 8
[13.03|10:11:33] CheckForces: 2 0 1
[13.03|10:11:33] CheckForces: 3 1 0
[13.03|10:11:33] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:11:33] CheckForces: Force components with an error exceeding the threshold:
[13.03|10:11:33] CheckForces: Ref Pred Delta Threshold
[13.03|10:11:33] CheckForces: 0.06 0.45 0.39 0.30
[13.03|10:11:33] CheckForces: -0.01 -0.36 0.35 0.30
[13.03|10:11:33] CheckForces: 1.56 1.10 0.45 0.40
[13.03|10:11:33] CheckForces: Maximum deviation: 0.453 eV/angstrom
[13.03|10:11:33] CheckForces: Actual Threshold
[13.03|10:11:33] CheckForces: # > thr. 3 0 Not OK!
[13.03|10:11:33] CheckForces: MAE 0.076 0.30 OK!
[13.03|10:11:33] CheckForces: R^2 0.980 0.80 OK!
[13.03|10:11:33] CheckForces: --------------------
[13.03|10:11:33]
[13.03|10:11:33] Adding results from step7_attempt2_reference_calc1 to training set
[13.03|10:11:33] Current # training set entries: 34
[13.03|10:11:33] Current # validation set entries: 15
[13.03|10:11:33] Storing data in step7_attempt2_reference_data
[13.03|10:11:33] Deleting step7_attempt1_reference_data
[13.03|10:11:33] Deleting step7_attempt2_reference_calc1
[13.03|10:11:33]
[13.03|10:11:33] Current (cumulative) timings:
[13.03|10:11:33] Time (s) Fraction
[13.03|10:11:33] Ref. calcs 149.78 0.050
[13.03|10:11:33] ML training 1915.05 0.637
[13.03|10:11:33] Simulations 942.06 0.313
[13.03|10:11:33]
[13.03|10:11:33]
[13.03|10:11:33]
[13.03|10:11:33] --- Begin summary ---
[13.03|10:11:33] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:11:33] 1 1 FAILED Inaccurate 0.3894
[13.03|10:11:33] 1 2 SUCCESS Accurate 0.1974
[13.03|10:11:33] 2 1 FAILED Inaccurate 0.3629
[13.03|10:11:33] 2 2 SUCCESS Accurate 0.1848
[13.03|10:11:33] 3 1 FAILED Inaccurate 0.3716
[13.03|10:11:33] 3 2 SUCCESS Accurate 0.3045
[13.03|10:11:33] 4 1 FAILED Inaccurate 0.3887
[13.03|10:11:33] 4 2 SUCCESS Accurate 0.2201
[13.03|10:11:33] 5 1 FAILED Inaccurate 0.4263
[13.03|10:11:33] 5 2 SUCCESS Accurate 0.1931
[13.03|10:11:33] 6 1 FAILED Inaccurate 0.3540
[13.03|10:11:33] 6 2 FAILED Inaccurate 0.4992
[13.03|10:11:33] 6 3 FAILED Inaccurate 0.3989
[13.03|10:11:33] 6 4 SUCCESS Accurate 0.3000
[13.03|10:11:33] 7 1 FAILED Inaccurate 0.5401
[13.03|10:11:33] 7 2 FAILED Inaccurate 0.4528
[13.03|10:11:33] --- End summary ---
[13.03|10:11:33]
[13.03|10:11:33] Running more reference calculations....
[13.03|10:11:34] Running reference calculations on frames [100, 111, 122] from step7_attempt2_simulation/ams.rkf
[13.03|10:11:34] Calculating 3 frames in total
[13.03|10:11:34] Running step7_attempt2_reference_calc2
[13.03|10:11:37] Running step7_attempt2_reference_calc3
[13.03|10:11:40] Running step7_attempt2_reference_calc4
[13.03|10:11:43] Reference calculations finished!
[13.03|10:11:43] Adding results from step7_attempt2_reference_calc2 to validation set
[13.03|10:11:43] Adding results from step7_attempt2_reference_calc3 to training set
[13.03|10:11:43] Adding results from step7_attempt2_reference_calc4 to training set
[13.03|10:11:43] Current # training set entries: 36
[13.03|10:11:43] Current # validation set entries: 16
[13.03|10:11:43] Storing data in step7_attempt2_reference_data
[13.03|10:11:43] Deleting step7_attempt2_reference_calc2
[13.03|10:11:43] Deleting step7_attempt2_reference_calc3
[13.03|10:11:43] Deleting step7_attempt2_reference_calc4
[13.03|10:11:43] Launching reparametrization job: step7_attempt2_training
[13.03|10:11:49] JOB m3gnet STARTED
[13.03|10:11:49] Starting m3gnet.prerun()
[13.03|10:11:49] m3gnet.prerun() finished
[13.03|10:11:49] JOB m3gnet RUNNING
[13.03|10:11:49] Executing m3gnet.run
[13.03|10:12:59] training_set Optimizer: 001 Epoch: 0 Loss: 0.003233
[13.03|10:12:59] validation_set Optimizer: 001 Epoch: 0 Loss: 0.051391
[13.03|10:13:18] training_set Optimizer: 001 Epoch: 10 Loss: 0.000213
[13.03|10:13:18] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006257
[13.03|10:13:36] training_set Optimizer: 001 Epoch: 20 Loss: 0.000185
[13.03|10:13:36] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006053
[13.03|10:13:54] training_set Optimizer: 001 Epoch: 30 Loss: 0.000193
[13.03|10:13:54] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005714
[13.03|10:14:13] training_set Optimizer: 001 Epoch: 40 Loss: 0.000206
[13.03|10:14:13] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006220
[13.03|10:14:31] training_set Optimizer: 001 Epoch: 50 Loss: 0.000157
[13.03|10:14:31] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005841
[13.03|10:14:50] training_set Optimizer: 001 Epoch: 60 Loss: 0.000165
[13.03|10:14:50] validation_set Optimizer: 001 Epoch: 60 Loss: 0.009516
[13.03|10:15:08] training_set Optimizer: 001 Epoch: 70 Loss: 0.000178
[13.03|10:15:08] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005641
[13.03|10:15:26] training_set Optimizer: 001 Epoch: 80 Loss: 0.000190
[13.03|10:15:26] validation_set Optimizer: 001 Epoch: 80 Loss: 0.009026
[13.03|10:15:45] training_set Optimizer: 001 Epoch: 90 Loss: 0.000189
[13.03|10:15:45] validation_set Optimizer: 001 Epoch: 90 Loss: 0.008874
[13.03|10:16:03] training_set Optimizer: 001 Epoch: 100 Loss: 0.000152
[13.03|10:16:03] validation_set Optimizer: 001 Epoch: 100 Loss: 0.007563
[13.03|10:16:22] training_set Optimizer: 001 Epoch: 110 Loss: 0.000151
[13.03|10:16:22] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006172
[13.03|10:16:41] training_set Optimizer: 001 Epoch: 120 Loss: 0.000138
[13.03|10:16:41] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005523
[13.03|10:16:59] training_set Optimizer: 001 Epoch: 130 Loss: 0.000134
[13.03|10:16:59] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005780
[13.03|10:17:18] training_set Optimizer: 001 Epoch: 140 Loss: 0.000134
[13.03|10:17:18] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006071
[13.03|10:17:36] training_set Optimizer: 001 Epoch: 150 Loss: 0.000134
[13.03|10:17:36] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005455
[13.03|10:17:55] training_set Optimizer: 001 Epoch: 160 Loss: 0.000135
[13.03|10:17:55] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006901
[13.03|10:18:13] training_set Optimizer: 001 Epoch: 170 Loss: 0.000122
[13.03|10:18:13] validation_set Optimizer: 001 Epoch: 170 Loss: 0.009568
[13.03|10:18:31] training_set Optimizer: 001 Epoch: 180 Loss: 0.000129
[13.03|10:18:31] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005979
[13.03|10:18:49] training_set Optimizer: 001 Epoch: 190 Loss: 0.000118
[13.03|10:18:49] validation_set Optimizer: 001 Epoch: 190 Loss: 0.006718
[13.03|10:19:09] Execution of m3gnet.run finished with returncode 0
[13.03|10:19:09] JOB m3gnet FINISHED
[13.03|10:19:09] Starting m3gnet.postrun()
[13.03|10:19:09] m3gnet.postrun() finished
[13.03|10:19:09] JOB m3gnet SUCCESSFUL
[13.03|10:19:28] Running all jobs through AMS....
[13.03|10:19:28] Storing results/optimization/training_set_results/best
[13.03|10:19:28] Storing results/optimization/validation_set_results/best
[13.03|10:19:28] PLAMS environment cleaned up successfully
[13.03|10:19:28] PLAMS run finished. Goodbye
[13.03|10:19:30] ParAMSResults training_set validation_set
[13.03|10:19:30] energy MAE 0.1479 0.1244 eV
[13.03|10:19:30] forces MAE 0.0269 0.0486 eV/angstrom
[13.03|10:19:30] Newly created parameter file/dir: step7_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|10:19:30] Done!
[13.03|10:19:30] Deleting step7_attempt1_training
[13.03|10:19:30] ##########################
[13.03|10:19:30] ### Step 7 / Attempt 3 ###
[13.03|10:19:30] ##########################
[13.03|10:19:30] MD Steps: 4473 (cumulative: 6752)
[13.03|10:19:30] Current engine settings:
[13.03|10:19:30]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:19:30] Running step7_attempt3_simulation...
[13.03|10:22:49] Job step7_attempt3_simulation finished
[13.03|10:22:49] Deleting files that are no longer needed...
[13.03|10:22:50] Launching reference calculation
[13.03|10:22:53] Reference calculation finished!
[13.03|10:22:53] Checking success for step7_attempt3
[13.03|10:23:05] CheckEnergy: Checking energy for MDStep6752, n_atoms = 46
[13.03|10:23:05] CheckEnergy: normalization coefficient = 46
[13.03|10:23:06] CheckEnergy: Actual Threshold
[13.03|10:23:06] CheckEnergy: dE/46 0.0029 0.2000 OK!
[13.03|10:23:06] CheckEnergy: ddE/46 -0.0005 0.0020 OK! (relative to step7_attempt2_simulation:MDStep6752)
[13.03|10:23:06]
[13.03|10:23:06] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:23:06] CheckForces: ------------
[13.03|10:23:06] CheckForces: Reference job from step7_attempt3_reference_calc1
[13.03|10:23:06] CheckForces: Prediction job from final frame (MDStep6752) of step7_attempt3_simulation
[13.03|10:23:06] CheckForces: ------------
[13.03|10:23:06] CheckForces: Histogram of forces
[13.03|10:23:06] CheckForces: eV/Ang Ref Pred
[13.03|10:23:06] CheckForces: -3 1 1
[13.03|10:23:06] CheckForces: -2 14 14
[13.03|10:23:06] CheckForces: -1 53 51
[13.03|10:23:06] CheckForces: 0 61 63
[13.03|10:23:06] CheckForces: 1 7 7
[13.03|10:23:06] CheckForces: 2 1 1
[13.03|10:23:06] CheckForces: 3 1 1
[13.03|10:23:06] CheckForces: 4 0 0
[13.03|10:23:06] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:23:06] CheckForces: All force components are within the acceptable error!
[13.03|10:23:06] CheckForces: Maximum deviation: 0.292 eV/angstrom
[13.03|10:23:06] CheckForces: Actual Threshold
[13.03|10:23:06] CheckForces: # > thr. 0 0 OK!
[13.03|10:23:06] CheckForces: MAE 0.064 0.30 OK!
[13.03|10:23:06] CheckForces: R^2 0.988 0.80 OK!
[13.03|10:23:06] CheckForces: --------------------
[13.03|10:23:06]
[13.03|10:23:06] Adding results from step7_attempt3_reference_calc1 to training set
[13.03|10:23:06] Current # training set entries: 37
[13.03|10:23:06] Current # validation set entries: 16
[13.03|10:23:06] Storing data in step7_attempt3_reference_data
[13.03|10:23:06] Deleting step7_attempt2_reference_data
[13.03|10:23:06] Deleting step7_attempt3_reference_calc1
[13.03|10:23:06]
[13.03|10:23:06] Current (cumulative) timings:
[13.03|10:23:06] Time (s) Fraction
[13.03|10:23:06] Ref. calcs 161.90 0.044
[13.03|10:23:06] ML training 2381.28 0.646
[13.03|10:23:06] Simulations 1141.86 0.310
[13.03|10:23:06]
[13.03|10:23:06]
[13.03|10:23:06] Step 7 finished successfully!
[13.03|10:23:06]
[13.03|10:23:06] --- Begin summary ---
[13.03|10:23:06] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:23:06] 1 1 FAILED Inaccurate 0.3894
[13.03|10:23:06] 1 2 SUCCESS Accurate 0.1974
[13.03|10:23:06] 2 1 FAILED Inaccurate 0.3629
[13.03|10:23:06] 2 2 SUCCESS Accurate 0.1848
[13.03|10:23:06] 3 1 FAILED Inaccurate 0.3716
[13.03|10:23:06] 3 2 SUCCESS Accurate 0.3045
[13.03|10:23:06] 4 1 FAILED Inaccurate 0.3887
[13.03|10:23:06] 4 2 SUCCESS Accurate 0.2201
[13.03|10:23:06] 5 1 FAILED Inaccurate 0.4263
[13.03|10:23:06] 5 2 SUCCESS Accurate 0.1931
[13.03|10:23:06] 6 1 FAILED Inaccurate 0.3540
[13.03|10:23:06] 6 2 FAILED Inaccurate 0.4992
[13.03|10:23:06] 6 3 FAILED Inaccurate 0.3989
[13.03|10:23:06] 6 4 SUCCESS Accurate 0.3000
[13.03|10:23:06] 7 1 FAILED Inaccurate 0.5401
[13.03|10:23:06] 7 2 FAILED Inaccurate 0.4528
[13.03|10:23:06] 7 3 SUCCESS Accurate 0.2915
[13.03|10:23:06] --- End summary ---
[13.03|10:23:06]
[13.03|10:23:06] ##########################
[13.03|10:23:06] ### Step 8 / Attempt 1 ###
[13.03|10:23:06] ##########################
[13.03|10:23:06] MD Steps: 13248 (cumulative: 20000)
[13.03|10:23:06] Current engine settings:
[13.03|10:23:06]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:23:06] Running step8_attempt1_simulation...
[13.03|10:32:55] Job step8_attempt1_simulation finished
[13.03|10:32:55] Deleting files that are no longer needed...
[13.03|10:32:55] Deleting step6_attempt4_simulation
[13.03|10:32:55] Deleting step7_attempt1_simulation
[13.03|10:32:55] Deleting step7_attempt2_simulation
[13.03|10:32:56] Launching reference calculation
[13.03|10:32:59] Reference calculation finished!
[13.03|10:32:59] Checking success for step8_attempt1
[13.03|10:33:11] CheckEnergy: Checking energy for MDStep20000, n_atoms = 46
[13.03|10:33:11] CheckEnergy: normalization coefficient = 46
[13.03|10:33:11] CheckEnergy: Actual Threshold
[13.03|10:33:11] CheckEnergy: dE/46 0.0025 0.2000 OK!
[13.03|10:33:11] CheckEnergy: ddE/46 -0.0004 0.0020 OK! (relative to step7_attempt3_simulation:MDStep6752)
[13.03|10:33:11]
[13.03|10:33:11] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:33:11] CheckForces: ------------
[13.03|10:33:11] CheckForces: Reference job from step8_attempt1_reference_calc1
[13.03|10:33:11] CheckForces: Prediction job from final frame (MDStep20000) of step8_attempt1_simulation
[13.03|10:33:11] CheckForces: ------------
[13.03|10:33:11] CheckForces: Histogram of forces
[13.03|10:33:11] CheckForces: eV/Ang Ref Pred
[13.03|10:33:11] CheckForces: -4 0 0
[13.03|10:33:11] CheckForces: -3 3 2
[13.03|10:33:11] CheckForces: -2 12 13
[13.03|10:33:11] CheckForces: -1 56 58
[13.03|10:33:11] CheckForces: 0 52 49
[13.03|10:33:11] CheckForces: 1 13 14
[13.03|10:33:11] CheckForces: 2 1 2
[13.03|10:33:11] CheckForces: 3 1 0
[13.03|10:33:11] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:33:11] CheckForces: All force components are within the acceptable error!
[13.03|10:33:11] CheckForces: Maximum deviation: 0.302 eV/angstrom
[13.03|10:33:11] CheckForces: Actual Threshold
[13.03|10:33:11] CheckForces: # > thr. 0 0 OK!
[13.03|10:33:11] CheckForces: MAE 0.062 0.30 OK!
[13.03|10:33:11] CheckForces: R^2 0.991 0.80 OK!
[13.03|10:33:11] CheckForces: --------------------
[13.03|10:33:11]
[13.03|10:33:11] Adding results from step8_attempt1_reference_calc1 to training set
[13.03|10:33:11] Current # training set entries: 38
[13.03|10:33:11] Current # validation set entries: 16
[13.03|10:33:11] Storing data in step8_attempt1_reference_data
[13.03|10:33:12] Deleting step7_attempt3_reference_data
[13.03|10:33:12] Deleting step8_attempt1_reference_calc1
[13.03|10:33:12]
[13.03|10:33:12] Current (cumulative) timings:
[13.03|10:33:12] Time (s) Fraction
[13.03|10:33:12] Ref. calcs 164.83 0.039
[13.03|10:33:12] ML training 2381.28 0.557
[13.03|10:33:12] Simulations 1730.73 0.405
[13.03|10:33:12]
[13.03|10:33:12]
[13.03|10:33:12] Step 8 finished successfully!
[13.03|10:33:12]
[13.03|10:33:12] --- Begin summary ---
[13.03|10:33:12] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:33:12] 1 1 FAILED Inaccurate 0.3894
[13.03|10:33:12] 1 2 SUCCESS Accurate 0.1974
[13.03|10:33:12] 2 1 FAILED Inaccurate 0.3629
[13.03|10:33:12] 2 2 SUCCESS Accurate 0.1848
[13.03|10:33:12] 3 1 FAILED Inaccurate 0.3716
[13.03|10:33:12] 3 2 SUCCESS Accurate 0.3045
[13.03|10:33:12] 4 1 FAILED Inaccurate 0.3887
[13.03|10:33:12] 4 2 SUCCESS Accurate 0.2201
[13.03|10:33:12] 5 1 FAILED Inaccurate 0.4263
[13.03|10:33:12] 5 2 SUCCESS Accurate 0.1931
[13.03|10:33:12] 6 1 FAILED Inaccurate 0.3540
[13.03|10:33:12] 6 2 FAILED Inaccurate 0.4992
[13.03|10:33:12] 6 3 FAILED Inaccurate 0.3989
[13.03|10:33:12] 6 4 SUCCESS Accurate 0.3000
[13.03|10:33:12] 7 1 FAILED Inaccurate 0.5401
[13.03|10:33:12] 7 2 FAILED Inaccurate 0.4528
[13.03|10:33:12] 7 3 SUCCESS Accurate 0.2915
[13.03|10:33:12] 8 1 SUCCESS Accurate 0.3019
[13.03|10:33:12] --- End summary ---
[13.03|10:33:12]
[13.03|10:33:12] The engine settings for the final trained ML engine are:
[13.03|10:33:12]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/sal/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:33:12] Active learning finished!
[13.03|10:33:12] Goodbye!
[13.03|10:33:13] JOB sal FINISHED
[13.03|10:33:13] JOB sal SUCCESSFUL
Final structure from MD simulation¶
new_structure = job.results.get_main_molecule()
plams.plot_molecule(new_structure, rotation="-5x,5y,0z")
Second active learning job: CREST metadynamics¶
Here we set Steps.Type = “Linear” to run reference calculations every 2000 MD steps.
nsteps = 20000
crest_md_s = plams.Settings()
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.Height = 0.138
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.NGaussiansMax = 50
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.NSteps = 200
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.Width = 0.62
crest_complete_md_s = plams.AMSMDJob(
molecule=new_structure,
nsteps=nsteps,
settings=crest_md_s,
tau=10, # small time constant
thermostat="NHC",
temperature=300,
timestep=0.5,
samplingfreq=20,
).settings
# job = SimpleActiveLearningJob.load_external(plams.config.default_jobmanager.workdir + "/sal.002")
crest_ml_s = ml_s.copy()
crest_ml_s.input.ams.MachineLearning.LoadModel = job.results.get_params_results_directory()
crest_ml_s.input.ams.MachineLearning.Target.Forces.MAE = 0.04
crest_ml_s.input.ams.MachineLearning.MaxEpochs = 200
crest_al_s = plams.Settings()
crest_al_s.input.ams.ActiveLearning.Steps.Type = "Linear"
crest_al_s.input.ams.ActiveLearning.Steps.Linear.Start = 500
crest_al_s.input.ams.ActiveLearning.Steps.Linear.StepSize = 2000
crest_al_s.input.ams.ActiveLearning.InitialReferenceData.Load.FromPreviousModel = "Yes"
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Relative = 0.002
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Total = 0.010
# because we do not set Normalization, the above Energy criteria are energies per atom
# crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Normalization =
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Forces.MaxDeviationForZeroForce = 0.30
crest_al_s.input.ams.ActiveLearning.AtEnd.RerunSimulation = "No"
crest_al_s.input.ams.ActiveLearning.MaxReferenceCalculationsPerAttempt = 3
crest_al_job = SimpleActiveLearningJob(
name="crest_al",
settings=ref_s + crest_complete_md_s + crest_ml_s + crest_al_s,
molecule=new_structure,
)
crest_al_job.run(watch=True);
[13.03|10:33:13] JOB crest_al STARTED
[13.03|10:33:13] JOB crest_al RUNNING
[13.03|10:33:16] Simple Active Learning 2024.101, Nodes: 1, Procs: 64
[13.03|10:33:19] Composition of main system: C20H23NOS
[13.03|10:33:19] All REFERENCE calculations will be performed with the following DFTB engine:
[13.03|10:33:19]
Engine dftb
model GFN1-xTB
EndEngine
[13.03|10:33:19] The following are the settings for the to-be-trained MACHINE LEARNING model:
[13.03|10:33:19]
MachineLearning
Backend M3GNet
CommitteeSize 1
LoadModel /path/plams_workdir/sal/step7_attempt2_training/results
M3GNet
Model UniversalPotential
End
MaxEpochs 200
Target
Forces
MAE 0.04
End
End
End
ParallelLevels
End
[13.03|10:33:19] A single model will be trained (no committee).
[13.03|10:33:19] The ACTIVE LEARNING loop will contain 11 steps, using the following schema:
[13.03|10:33:19] Active Learning Step 1: 500 MD Steps (cumulative: 500)
[13.03|10:33:19] Active Learning Step 2: 2000 MD Steps (cumulative: 2500)
[13.03|10:33:19] Active Learning Step 3: 2000 MD Steps (cumulative: 4500)
[13.03|10:33:19] Active Learning Step 4: 2000 MD Steps (cumulative: 6500)
[13.03|10:33:19] Active Learning Step 5: 2000 MD Steps (cumulative: 8500)
[13.03|10:33:19] Active Learning Step 6: 2000 MD Steps (cumulative: 10500)
[13.03|10:33:19] Active Learning Step 7: 2000 MD Steps (cumulative: 12500)
[13.03|10:33:19] Active Learning Step 8: 2000 MD Steps (cumulative: 14500)
[13.03|10:33:19] Active Learning Step 9: 2000 MD Steps (cumulative: 16500)
[13.03|10:33:19] Active Learning Step 10: 2000 MD Steps (cumulative: 18500)
[13.03|10:33:19] Active Learning Step 11: 1500 MD Steps (cumulative: 20000)
[13.03|10:33:19] Total number of MD Steps: 20000
[13.03|10:33:19] Max attempts per active learning step: 15
[13.03|10:33:19]
[13.03|10:33:19] The directory /path/plams_workdir/sal/step7_attempt2_training/results is of type RestartDirectoryType.PARAMS_RESULTS
[13.03|10:33:19] Successfully loaded previous ParAMS Job: from /path/plams_workdir/sal/step7_attempt2_training/results
[13.03|10:33:20] Starting active learning loop...
[13.03|10:33:20] ##########################
[13.03|10:33:20] ### Step 1 / Attempt 1 ###
[13.03|10:33:20] ##########################
[13.03|10:33:20] MD Steps: 500 (cumulative: 500)
[13.03|10:33:20] Current engine settings:
[13.03|10:33:20]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/loaded_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:33:20] Running step1_attempt1_simulation...
[13.03|10:33:55] Job step1_attempt1_simulation finished
[13.03|10:33:55] Deleting files that are no longer needed...
[13.03|10:33:55] Launching reference calculation
[13.03|10:33:58] Reference calculation finished!
[13.03|10:33:58] Checking success for step1_attempt1
[13.03|10:33:58] CheckEnergy: Checking energy for MDStep500, n_atoms = 46
[13.03|10:33:58] CheckEnergy: normalization coefficient = 46
[13.03|10:33:58] CheckEnergy: Actual Threshold
[13.03|10:33:58] CheckEnergy: dE/46 -0.0012 0.0100 OK!
[13.03|10:33:58]
[13.03|10:33:58] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:33:58] CheckForces: ------------
[13.03|10:33:58] CheckForces: Reference job from step1_attempt1_reference_calc1
[13.03|10:33:58] CheckForces: Prediction job from final frame (MDStep500) of step1_attempt1_simulation
[13.03|10:33:58] CheckForces: ------------
[13.03|10:33:58] CheckForces: Histogram of forces
[13.03|10:33:58] CheckForces: eV/Ang Ref Pred
[13.03|10:33:58] CheckForces: -2 3 2
[13.03|10:33:58] CheckForces: -1 73 73
[13.03|10:33:58] CheckForces: 0 55 57
[13.03|10:33:58] CheckForces: 1 4 3
[13.03|10:33:58] CheckForces: 2 3 3
[13.03|10:33:58] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:33:58] CheckForces: Force components with an error exceeding the threshold:
[13.03|10:33:58] CheckForces: Ref Pred Delta Threshold
[13.03|10:33:58] CheckForces: -0.73 -0.27 0.46 0.34
[13.03|10:33:58] CheckForces: 0.27 0.75 0.48 0.31
[13.03|10:33:58] CheckForces: -0.52 -0.13 0.39 0.33
[13.03|10:33:58] CheckForces: Maximum deviation: 0.479 eV/angstrom
[13.03|10:33:58] CheckForces: Actual Threshold
[13.03|10:33:58] CheckForces: # > thr. 3 0 Not OK!
[13.03|10:33:58] CheckForces: MAE 0.074 0.30 OK!
[13.03|10:33:58] CheckForces: R^2 0.965 0.80 OK!
[13.03|10:33:58] CheckForces: --------------------
[13.03|10:33:58]
[13.03|10:33:59] Adding results from step1_attempt1_reference_calc1 to training set
[13.03|10:33:59] Current # training set entries: 37
[13.03|10:33:59] Current # validation set entries: 16
[13.03|10:33:59] Storing data in step1_attempt1_reference_data
[13.03|10:33:59] Deleting initial_reference_data
[13.03|10:33:59] Deleting step1_attempt1_reference_calc1
[13.03|10:33:59]
[13.03|10:33:59] Current (cumulative) timings:
[13.03|10:33:59] Time (s) Fraction
[13.03|10:33:59] Ref. calcs 2.93 0.078
[13.03|10:33:59] ML training 0.00 0.000
[13.03|10:33:59] Simulations 34.85 0.922
[13.03|10:33:59]
[13.03|10:33:59]
[13.03|10:33:59]
[13.03|10:33:59] --- Begin summary ---
[13.03|10:33:59] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:33:59] 1 1 FAILED Inaccurate 0.4793
[13.03|10:33:59] --- End summary ---
[13.03|10:33:59]
[13.03|10:33:59] Running more reference calculations....
[13.03|10:33:59] Running reference calculations on frames [9, 17] from step1_attempt1_simulation/ams.rkf
[13.03|10:33:59] Calculating 2 frames in total
[13.03|10:33:59] Running step1_attempt1_reference_calc2
[13.03|10:34:02] Running step1_attempt1_reference_calc3
[13.03|10:34:05] Reference calculations finished!
[13.03|10:34:06] Adding results from step1_attempt1_reference_calc2 to validation set
[13.03|10:34:06] Adding results from step1_attempt1_reference_calc3 to training set
[13.03|10:34:06] Current # training set entries: 38
[13.03|10:34:06] Current # validation set entries: 17
[13.03|10:34:06] Storing data in step1_attempt1_reference_data
[13.03|10:34:06] Deleting step1_attempt1_reference_calc2
[13.03|10:34:06] Deleting step1_attempt1_reference_calc3
[13.03|10:34:06] Launching reparametrization job: step1_attempt1_training
[13.03|10:34:12] JOB m3gnet STARTED
[13.03|10:34:12] Starting m3gnet.prerun()
[13.03|10:34:12] m3gnet.prerun() finished
[13.03|10:34:12] JOB m3gnet RUNNING
[13.03|10:34:12] Executing m3gnet.run
[13.03|10:35:21] training_set Optimizer: 001 Epoch: 0 Loss: 0.001681
[13.03|10:35:21] validation_set Optimizer: 001 Epoch: 0 Loss: 0.019892
[13.03|10:35:40] training_set Optimizer: 001 Epoch: 10 Loss: 0.000219
[13.03|10:35:40] validation_set Optimizer: 001 Epoch: 10 Loss: 0.008011
[13.03|10:35:59] training_set Optimizer: 001 Epoch: 20 Loss: 0.000161
[13.03|10:35:59] validation_set Optimizer: 001 Epoch: 20 Loss: 0.007962
[13.03|10:36:19] training_set Optimizer: 001 Epoch: 30 Loss: 0.000146
[13.03|10:36:19] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005683
[13.03|10:36:38] training_set Optimizer: 001 Epoch: 40 Loss: 0.000137
[13.03|10:36:38] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005717
[13.03|10:36:56] training_set Optimizer: 001 Epoch: 50 Loss: 0.000139
[13.03|10:36:56] validation_set Optimizer: 001 Epoch: 50 Loss: 0.008602
[13.03|10:37:16] training_set Optimizer: 001 Epoch: 60 Loss: 0.000148
[13.03|10:37:16] validation_set Optimizer: 001 Epoch: 60 Loss: 0.011474
[13.03|10:37:35] training_set Optimizer: 001 Epoch: 70 Loss: 0.000153
[13.03|10:37:35] validation_set Optimizer: 001 Epoch: 70 Loss: 0.008153
[13.03|10:37:54] training_set Optimizer: 001 Epoch: 80 Loss: 0.000124
[13.03|10:37:54] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006172
[13.03|10:38:13] training_set Optimizer: 001 Epoch: 90 Loss: 0.000128
[13.03|10:38:13] validation_set Optimizer: 001 Epoch: 90 Loss: 0.012830
[13.03|10:38:32] training_set Optimizer: 001 Epoch: 100 Loss: 0.000113
[13.03|10:38:32] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005578
[13.03|10:38:51] training_set Optimizer: 001 Epoch: 110 Loss: 0.000113
[13.03|10:38:51] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006181
[13.03|10:39:10] training_set Optimizer: 001 Epoch: 120 Loss: 0.000111
[13.03|10:39:10] validation_set Optimizer: 001 Epoch: 120 Loss: 0.007777
[13.03|10:39:29] training_set Optimizer: 001 Epoch: 130 Loss: 0.000108
[13.03|10:39:29] validation_set Optimizer: 001 Epoch: 130 Loss: 0.006564
[13.03|10:39:48] training_set Optimizer: 001 Epoch: 140 Loss: 0.000101
[13.03|10:39:48] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005861
[13.03|10:40:08] training_set Optimizer: 001 Epoch: 150 Loss: 0.000100
[13.03|10:40:08] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005999
[13.03|10:40:27] training_set Optimizer: 001 Epoch: 160 Loss: 0.000105
[13.03|10:40:27] validation_set Optimizer: 001 Epoch: 160 Loss: 0.008489
[13.03|10:40:46] training_set Optimizer: 001 Epoch: 170 Loss: 0.000097
[13.03|10:40:46] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006224
[13.03|10:41:05] training_set Optimizer: 001 Epoch: 180 Loss: 0.000101
[13.03|10:41:05] validation_set Optimizer: 001 Epoch: 180 Loss: 0.010573
[13.03|10:41:24] training_set Optimizer: 001 Epoch: 190 Loss: 0.000109
[13.03|10:41:24] validation_set Optimizer: 001 Epoch: 190 Loss: 0.009400
[13.03|10:41:44] Execution of m3gnet.run finished with returncode 0
[13.03|10:41:44] JOB m3gnet FINISHED
[13.03|10:41:44] Starting m3gnet.postrun()
[13.03|10:41:44] m3gnet.postrun() finished
[13.03|10:41:45] JOB m3gnet SUCCESSFUL
[13.03|10:42:06] Running all jobs through AMS....
[13.03|10:42:06] Storing results/optimization/training_set_results/best
[13.03|10:42:06] Storing results/optimization/validation_set_results/best
[13.03|10:42:06] PLAMS environment cleaned up successfully
[13.03|10:42:06] PLAMS run finished. Goodbye
[13.03|10:42:08] ParAMSResults training_set validation_set
[13.03|10:42:08] energy MAE 0.0717 0.0640 eV
[13.03|10:42:08] forces MAE 0.0237 0.0497 eV/angstrom
[13.03|10:42:08] Newly created parameter file/dir: step1_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|10:42:08] Done!
[13.03|10:42:08] Deleting loaded_training
[13.03|10:42:08] ##########################
[13.03|10:42:08] ### Step 1 / Attempt 2 ###
[13.03|10:42:08] ##########################
[13.03|10:42:08] MD Steps: 500 (cumulative: 500)
[13.03|10:42:08] Current engine settings:
[13.03|10:42:08]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step1_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:42:08] Running step1_attempt2_simulation...
[13.03|10:42:42] Job step1_attempt2_simulation finished
[13.03|10:42:42] Deleting files that are no longer needed...
[13.03|10:42:42] Launching reference calculation
[13.03|10:42:45] Reference calculation finished!
[13.03|10:42:45] Checking success for step1_attempt2
[13.03|10:42:55] CheckEnergy: Checking energy for MDStep500, n_atoms = 46
[13.03|10:42:55] CheckEnergy: normalization coefficient = 46
[13.03|10:42:56] CheckEnergy: Actual Threshold
[13.03|10:42:56] CheckEnergy: dE/46 0.0003 0.0100 OK!
[13.03|10:42:56] CheckEnergy: ddE/46 0.0001 0.0020 OK! (relative to step1_attempt1_simulation:MDStep500)
[13.03|10:42:56]
[13.03|10:42:56] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:42:56] CheckForces: ------------
[13.03|10:42:56] CheckForces: Reference job from step1_attempt2_reference_calc1
[13.03|10:42:56] CheckForces: Prediction job from final frame (MDStep500) of step1_attempt2_simulation
[13.03|10:42:56] CheckForces: ------------
[13.03|10:42:56] CheckForces: Histogram of forces
[13.03|10:42:56] CheckForces: eV/Ang Ref Pred
[13.03|10:42:56] CheckForces: -3 0 0
[13.03|10:42:56] CheckForces: -2 2 3
[13.03|10:42:56] CheckForces: -1 58 57
[13.03|10:42:56] CheckForces: 0 77 78
[13.03|10:42:56] CheckForces: 1 1 0
[13.03|10:42:56] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:42:56] CheckForces: Force components with an error exceeding the threshold:
[13.03|10:42:56] CheckForces: Ref Pred Delta Threshold
[13.03|10:42:56] CheckForces: 0.46 0.06 0.41 0.32
[13.03|10:42:56] CheckForces: Maximum deviation: 0.406 eV/angstrom
[13.03|10:42:56] CheckForces: Actual Threshold
[13.03|10:42:56] CheckForces: # > thr. 1 0 Not OK!
[13.03|10:42:56] CheckForces: MAE 0.057 0.30 OK!
[13.03|10:42:56] CheckForces: R^2 0.961 0.80 OK!
[13.03|10:42:56] CheckForces: --------------------
[13.03|10:42:56]
[13.03|10:42:56] Adding results from step1_attempt2_reference_calc1 to training set
[13.03|10:42:56] Current # training set entries: 39
[13.03|10:42:56] Current # validation set entries: 17
[13.03|10:42:56] Storing data in step1_attempt2_reference_data
[13.03|10:42:56] Deleting step1_attempt1_reference_data
[13.03|10:42:56] Deleting step1_attempt2_reference_calc1
[13.03|10:42:56]
[13.03|10:42:56] Current (cumulative) timings:
[13.03|10:42:56] Time (s) Fraction
[13.03|10:42:56] Ref. calcs 12.10 0.022
[13.03|10:42:56] ML training 481.68 0.856
[13.03|10:42:56] Simulations 68.62 0.122
[13.03|10:42:56]
[13.03|10:42:56]
[13.03|10:42:56]
[13.03|10:42:56] --- Begin summary ---
[13.03|10:42:56] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:42:56] 1 1 FAILED Inaccurate 0.4793
[13.03|10:42:56] 1 2 FAILED Inaccurate 0.4059
[13.03|10:42:56] --- End summary ---
[13.03|10:42:56]
[13.03|10:42:56] Running more reference calculations....
[13.03|10:42:57] Running reference calculations on frames [9, 17] from step1_attempt2_simulation/ams.rkf
[13.03|10:42:57] Calculating 2 frames in total
[13.03|10:42:57] Running step1_attempt2_reference_calc2
[13.03|10:43:00] Running step1_attempt2_reference_calc3
[13.03|10:43:03] Reference calculations finished!
[13.03|10:43:03] Adding results from step1_attempt2_reference_calc2 to validation set
[13.03|10:43:03] Adding results from step1_attempt2_reference_calc3 to training set
[13.03|10:43:03] Current # training set entries: 40
[13.03|10:43:03] Current # validation set entries: 18
[13.03|10:43:03] Storing data in step1_attempt2_reference_data
[13.03|10:43:03] Deleting step1_attempt2_reference_calc2
[13.03|10:43:03] Deleting step1_attempt2_reference_calc3
[13.03|10:43:03] Launching reparametrization job: step1_attempt2_training
[13.03|10:43:09] JOB m3gnet STARTED
[13.03|10:43:09] Starting m3gnet.prerun()
[13.03|10:43:09] m3gnet.prerun() finished
[13.03|10:43:09] JOB m3gnet RUNNING
[13.03|10:43:09] Executing m3gnet.run
[13.03|10:43:59] training_set Optimizer: 001 Epoch: 0 Loss: 0.001907
[13.03|10:43:59] validation_set Optimizer: 001 Epoch: 0 Loss: 0.008827
[13.03|10:44:19] training_set Optimizer: 001 Epoch: 10 Loss: 0.000112
[13.03|10:44:19] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005153
[13.03|10:44:38] training_set Optimizer: 001 Epoch: 20 Loss: 0.000109
[13.03|10:44:38] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005266
[13.03|10:44:58] training_set Optimizer: 001 Epoch: 30 Loss: 0.000106
[13.03|10:44:58] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005233
[13.03|10:45:17] training_set Optimizer: 001 Epoch: 40 Loss: 0.000101
[13.03|10:45:17] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005834
[13.03|10:45:37] training_set Optimizer: 001 Epoch: 50 Loss: 0.000098
[13.03|10:45:37] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005229
[13.03|10:45:56] training_set Optimizer: 001 Epoch: 60 Loss: 0.000098
[13.03|10:45:56] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007160
[13.03|10:46:15] training_set Optimizer: 001 Epoch: 70 Loss: 0.000092
[13.03|10:46:15] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005266
[13.03|10:46:35] training_set Optimizer: 001 Epoch: 80 Loss: 0.000108
[13.03|10:46:35] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006220
[13.03|10:46:54] training_set Optimizer: 001 Epoch: 90 Loss: 0.000112
[13.03|10:46:54] validation_set Optimizer: 001 Epoch: 90 Loss: 0.007439
[13.03|10:47:14] training_set Optimizer: 001 Epoch: 100 Loss: 0.000096
[13.03|10:47:14] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005406
[13.03|10:47:34] training_set Optimizer: 001 Epoch: 110 Loss: 0.000094
[13.03|10:47:34] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005948
[13.03|10:47:53] training_set Optimizer: 001 Epoch: 120 Loss: 0.000095
[13.03|10:47:53] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005141
[13.03|10:48:12] training_set Optimizer: 001 Epoch: 130 Loss: 0.000085
[13.03|10:48:12] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005286
[13.03|10:48:32] training_set Optimizer: 001 Epoch: 140 Loss: 0.000090
[13.03|10:48:32] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005527
[13.03|10:48:51] training_set Optimizer: 001 Epoch: 150 Loss: 0.000085
[13.03|10:48:51] validation_set Optimizer: 001 Epoch: 150 Loss: 0.007423
[13.03|10:49:10] training_set Optimizer: 001 Epoch: 160 Loss: 0.000086
[13.03|10:49:10] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005132
[13.03|10:49:30] training_set Optimizer: 001 Epoch: 170 Loss: 0.000083
[13.03|10:49:30] validation_set Optimizer: 001 Epoch: 170 Loss: 0.005542
[13.03|10:49:49] training_set Optimizer: 001 Epoch: 180 Loss: 0.000107
[13.03|10:49:49] validation_set Optimizer: 001 Epoch: 180 Loss: 0.006242
[13.03|10:50:09] training_set Optimizer: 001 Epoch: 190 Loss: 0.000090
[13.03|10:50:09] validation_set Optimizer: 001 Epoch: 190 Loss: 0.005768
[13.03|10:50:27] Execution of m3gnet.run finished with returncode 0
[13.03|10:50:28] JOB m3gnet FINISHED
[13.03|10:50:28] Starting m3gnet.postrun()
[13.03|10:50:28] m3gnet.postrun() finished
[13.03|10:50:28] JOB m3gnet SUCCESSFUL
[13.03|10:50:47] Running all jobs through AMS....
[13.03|10:50:47] Storing results/optimization/training_set_results/best
[13.03|10:50:47] Storing results/optimization/validation_set_results/best
[13.03|10:50:47] PLAMS environment cleaned up successfully
[13.03|10:50:47] PLAMS run finished. Goodbye
[13.03|10:50:49] ParAMSResults training_set validation_set
[13.03|10:50:49] energy MAE 0.0250 0.0427 eV
[13.03|10:50:49] forces MAE 0.0213 0.0469 eV/angstrom
[13.03|10:50:49] Newly created parameter file/dir: step1_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|10:50:49] Done!
[13.03|10:50:49] Deleting step1_attempt1_training
[13.03|10:50:49] ##########################
[13.03|10:50:49] ### Step 1 / Attempt 3 ###
[13.03|10:50:49] ##########################
[13.03|10:50:49] MD Steps: 500 (cumulative: 500)
[13.03|10:50:49] Current engine settings:
[13.03|10:50:49]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step1_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|10:50:49] Running step1_attempt3_simulation...
[13.03|10:51:26] Job step1_attempt3_simulation finished
[13.03|10:51:26] Deleting files that are no longer needed...
[13.03|10:51:27] Launching reference calculation
[13.03|10:51:30] Reference calculation finished!
[13.03|10:51:30] Checking success for step1_attempt3
[13.03|10:51:41] CheckEnergy: Checking energy for MDStep500, n_atoms = 46
[13.03|10:51:41] CheckEnergy: normalization coefficient = 46
[13.03|10:51:41] CheckEnergy: Actual Threshold
[13.03|10:51:41] CheckEnergy: dE/46 0.0005 0.0100 OK!
[13.03|10:51:41] CheckEnergy: ddE/46 0.0011 0.0020 OK! (relative to step1_attempt2_simulation:MDStep500)
[13.03|10:51:41]
[13.03|10:51:41] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|10:51:41] CheckForces: ------------
[13.03|10:51:41] CheckForces: Reference job from step1_attempt3_reference_calc1
[13.03|10:51:41] CheckForces: Prediction job from final frame (MDStep500) of step1_attempt3_simulation
[13.03|10:51:41] CheckForces: ------------
[13.03|10:51:41] CheckForces: Histogram of forces
[13.03|10:51:41] CheckForces: eV/Ang Ref Pred
[13.03|10:51:41] CheckForces: -2 3 2
[13.03|10:51:41] CheckForces: -1 66 66
[13.03|10:51:41] CheckForces: 0 69 69
[13.03|10:51:41] CheckForces: 1 0 1
[13.03|10:51:41] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|10:51:41] CheckForces: Force components with an error exceeding the threshold:
[13.03|10:51:41] CheckForces: Ref Pred Delta Threshold
[13.03|10:51:41] CheckForces: 0.56 0.08 0.48 0.33
[13.03|10:51:41] CheckForces: -0.35 -0.71 0.37 0.32
[13.03|10:51:41] CheckForces: -1.26 -0.83 0.43 0.37
[13.03|10:51:41] CheckForces: -0.27 0.06 0.33 0.31
[13.03|10:51:41] CheckForces: 0.73 0.36 0.37 0.34
[13.03|10:51:41] CheckForces: Maximum deviation: 0.485 eV/angstrom
[13.03|10:51:41] CheckForces: Actual Threshold
[13.03|10:51:41] CheckForces: # > thr. 5 0 Not OK!
[13.03|10:51:41] CheckForces: MAE 0.077 0.30 OK!
[13.03|10:51:41] CheckForces: R^2 0.927 0.80 OK!
[13.03|10:51:41] CheckForces: --------------------
[13.03|10:51:41]
[13.03|10:51:41] Adding results from step1_attempt3_reference_calc1 to training set
[13.03|10:51:41] Current # training set entries: 41
[13.03|10:51:41] Current # validation set entries: 18
[13.03|10:51:41] Storing data in step1_attempt3_reference_data
[13.03|10:51:41] Deleting step1_attempt2_reference_data
[13.03|10:51:41] Deleting step1_attempt3_reference_calc1
[13.03|10:51:41]
[13.03|10:51:41] Current (cumulative) timings:
[13.03|10:51:41] Time (s) Fraction
[13.03|10:51:41] Ref. calcs 21.13 0.020
[13.03|10:51:41] ML training 946.96 0.881
[13.03|10:51:41] Simulations 106.35 0.099
[13.03|10:51:41]
[13.03|10:51:41]
[13.03|10:51:42]
[13.03|10:51:42] --- Begin summary ---
[13.03|10:51:42] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|10:51:42] 1 1 FAILED Inaccurate 0.4793
[13.03|10:51:42] 1 2 FAILED Inaccurate 0.4059
[13.03|10:51:42] 1 3 FAILED Inaccurate 0.4848
[13.03|10:51:42] --- End summary ---
[13.03|10:51:42]
[13.03|10:51:42] Running more reference calculations....
[13.03|10:51:42] Running reference calculations on frames [9, 17] from step1_attempt3_simulation/ams.rkf
[13.03|10:51:42] Calculating 2 frames in total
[13.03|10:51:42] Running step1_attempt3_reference_calc2
[13.03|10:51:45] Running step1_attempt3_reference_calc3
[13.03|10:51:48] Reference calculations finished!
[13.03|10:51:48] Adding results from step1_attempt3_reference_calc2 to validation set
[13.03|10:51:48] Adding results from step1_attempt3_reference_calc3 to training set
[13.03|10:51:48] Current # training set entries: 42
[13.03|10:51:48] Current # validation set entries: 19
[13.03|10:51:48] Storing data in step1_attempt3_reference_data
[13.03|10:51:48] Deleting step1_attempt3_reference_calc2
[13.03|10:51:48] Deleting step1_attempt3_reference_calc3
[13.03|10:51:48] Launching reparametrization job: step1_attempt3_training
[13.03|10:51:54] JOB m3gnet STARTED
[13.03|10:51:54] Starting m3gnet.prerun()
[13.03|10:51:54] m3gnet.prerun() finished
[13.03|10:51:54] JOB m3gnet RUNNING
[13.03|10:51:54] Executing m3gnet.run
[13.03|10:53:03] training_set Optimizer: 001 Epoch: 0 Loss: 0.001236
[13.03|10:53:03] validation_set Optimizer: 001 Epoch: 0 Loss: 0.017854
[13.03|10:53:24] training_set Optimizer: 001 Epoch: 10 Loss: 0.000105
[13.03|10:53:24] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005300
[13.03|10:53:46] training_set Optimizer: 001 Epoch: 20 Loss: 0.000100
[13.03|10:53:46] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005531
[13.03|10:54:07] training_set Optimizer: 001 Epoch: 30 Loss: 0.000091
[13.03|10:54:07] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006807
[13.03|10:54:28] training_set Optimizer: 001 Epoch: 40 Loss: 0.000093
[13.03|10:54:28] validation_set Optimizer: 001 Epoch: 40 Loss: 0.009450
[13.03|10:54:49] training_set Optimizer: 001 Epoch: 50 Loss: 0.000094
[13.03|10:54:49] validation_set Optimizer: 001 Epoch: 50 Loss: 0.006547
[13.03|10:55:10] training_set Optimizer: 001 Epoch: 60 Loss: 0.000094
[13.03|10:55:10] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005200
[13.03|10:55:31] training_set Optimizer: 001 Epoch: 70 Loss: 0.000095
[13.03|10:55:31] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006634
[13.03|10:55:52] training_set Optimizer: 001 Epoch: 80 Loss: 0.000108
[13.03|10:55:52] validation_set Optimizer: 001 Epoch: 80 Loss: 0.012065
[13.03|10:56:13] training_set Optimizer: 001 Epoch: 90 Loss: 0.000083
[13.03|10:56:13] validation_set Optimizer: 001 Epoch: 90 Loss: 0.008832
[13.03|10:56:34] training_set Optimizer: 001 Epoch: 100 Loss: 0.000083
[13.03|10:56:34] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005152
[13.03|10:56:55] training_set Optimizer: 001 Epoch: 110 Loss: 0.000087
[13.03|10:56:55] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005627
[13.03|10:57:16] training_set Optimizer: 001 Epoch: 120 Loss: 0.000084
[13.03|10:57:16] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005469
[13.03|10:57:36] training_set Optimizer: 001 Epoch: 130 Loss: 0.000079
[13.03|10:57:36] validation_set Optimizer: 001 Epoch: 130 Loss: 0.006000
[13.03|10:57:58] training_set Optimizer: 001 Epoch: 140 Loss: 0.000077
[13.03|10:57:58] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006378
[13.03|10:58:18] training_set Optimizer: 001 Epoch: 150 Loss: 0.000084
[13.03|10:58:18] validation_set Optimizer: 001 Epoch: 150 Loss: 0.006519
[13.03|10:58:39] training_set Optimizer: 001 Epoch: 160 Loss: 0.000107
[13.03|10:58:39] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006168
[13.03|10:59:00] training_set Optimizer: 001 Epoch: 170 Loss: 0.000094
[13.03|10:59:00] validation_set Optimizer: 001 Epoch: 170 Loss: 0.010230
[13.03|10:59:20] training_set Optimizer: 001 Epoch: 180 Loss: 0.000071
[13.03|10:59:20] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005667
[13.03|10:59:41] training_set Optimizer: 001 Epoch: 190 Loss: 0.000103
[13.03|10:59:41] validation_set Optimizer: 001 Epoch: 190 Loss: 0.009091
[13.03|11:00:03] Execution of m3gnet.run finished with returncode 0
[13.03|11:00:03] JOB m3gnet FINISHED
[13.03|11:00:03] Starting m3gnet.postrun()
[13.03|11:00:03] m3gnet.postrun() finished
[13.03|11:00:03] JOB m3gnet SUCCESSFUL
[13.03|11:00:22] Running all jobs through AMS....
[13.03|11:00:23] Storing results/optimization/training_set_results/best
[13.03|11:00:23] Storing results/optimization/validation_set_results/best
[13.03|11:00:23] PLAMS environment cleaned up successfully
[13.03|11:00:23] PLAMS run finished. Goodbye
[13.03|11:00:24] ParAMSResults training_set validation_set
[13.03|11:00:24] energy MAE 0.2119 0.1873 eV
[13.03|11:00:24] forces MAE 0.0202 0.0473 eV/angstrom
[13.03|11:00:24] Newly created parameter file/dir: step1_attempt3_training/results/optimization/m3gnet/m3gnet
[13.03|11:00:24] Done!
[13.03|11:00:24] Deleting step1_attempt2_training
[13.03|11:00:24] ##########################
[13.03|11:00:24] ### Step 1 / Attempt 4 ###
[13.03|11:00:24] ##########################
[13.03|11:00:24] MD Steps: 500 (cumulative: 500)
[13.03|11:00:24] Current engine settings:
[13.03|11:00:24]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step1_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:00:24] Running step1_attempt4_simulation...
[13.03|11:01:03] Job step1_attempt4_simulation finished
[13.03|11:01:03] Deleting files that are no longer needed...
[13.03|11:01:03] Launching reference calculation
[13.03|11:01:06] Reference calculation finished!
[13.03|11:01:06] Checking success for step1_attempt4
[13.03|11:01:17] CheckEnergy: Checking energy for MDStep500, n_atoms = 46
[13.03|11:01:17] CheckEnergy: normalization coefficient = 46
[13.03|11:01:17] CheckEnergy: Actual Threshold
[13.03|11:01:17] CheckEnergy: dE/46 0.0062 0.0100 OK!
[13.03|11:01:17] CheckEnergy: ddE/46 0.0015 0.0020 OK! (relative to step1_attempt3_simulation:MDStep500)
[13.03|11:01:17]
[13.03|11:01:17] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:01:17] CheckForces: ------------
[13.03|11:01:17] CheckForces: Reference job from step1_attempt4_reference_calc1
[13.03|11:01:17] CheckForces: Prediction job from final frame (MDStep500) of step1_attempt4_simulation
[13.03|11:01:17] CheckForces: ------------
[13.03|11:01:17] CheckForces: Histogram of forces
[13.03|11:01:17] CheckForces: eV/Ang Ref Pred
[13.03|11:01:17] CheckForces: -3 0 0
[13.03|11:01:17] CheckForces: -2 3 3
[13.03|11:01:17] CheckForces: -1 63 62
[13.03|11:01:17] CheckForces: 0 71 73
[13.03|11:01:17] CheckForces: 1 1 0
[13.03|11:01:17] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:01:17] CheckForces: Force components with an error exceeding the threshold:
[13.03|11:01:17] CheckForces: Ref Pred Delta Threshold
[13.03|11:01:17] CheckForces: -0.12 0.22 0.35 0.31
[13.03|11:01:17] CheckForces: -0.77 -0.43 0.34 0.34
[13.03|11:01:17] CheckForces: -0.30 -0.64 0.34 0.31
[13.03|11:01:17] CheckForces: -0.27 0.42 0.70 0.31
[13.03|11:01:17] CheckForces: 0.62 0.20 0.42 0.33
[13.03|11:01:17] CheckForces: Maximum deviation: 0.695 eV/angstrom
[13.03|11:01:17] CheckForces: Actual Threshold
[13.03|11:01:17] CheckForces: # > thr. 5 0 Not OK!
[13.03|11:01:17] CheckForces: MAE 0.087 0.30 OK!
[13.03|11:01:17] CheckForces: R^2 0.902 0.80 OK!
[13.03|11:01:17] CheckForces: --------------------
[13.03|11:01:17]
[13.03|11:01:18] Adding results from step1_attempt4_reference_calc1 to training set
[13.03|11:01:18] Current # training set entries: 43
[13.03|11:01:18] Current # validation set entries: 19
[13.03|11:01:18] Storing data in step1_attempt4_reference_data
[13.03|11:01:18] Deleting step1_attempt3_reference_data
[13.03|11:01:18] Deleting step1_attempt4_reference_calc1
[13.03|11:01:18]
[13.03|11:01:18] Current (cumulative) timings:
[13.03|11:01:18] Time (s) Fraction
[13.03|11:01:18] Ref. calcs 30.29 0.018
[13.03|11:01:18] ML training 1462.72 0.893
[13.03|11:01:18] Simulations 144.97 0.089
[13.03|11:01:18]
[13.03|11:01:18]
[13.03|11:01:18]
[13.03|11:01:18] --- Begin summary ---
[13.03|11:01:18] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:01:18] 1 1 FAILED Inaccurate 0.4793
[13.03|11:01:18] 1 2 FAILED Inaccurate 0.4059
[13.03|11:01:18] 1 3 FAILED Inaccurate 0.4848
[13.03|11:01:18] 1 4 FAILED Inaccurate 0.6954
[13.03|11:01:18] --- End summary ---
[13.03|11:01:18]
[13.03|11:01:18] Running more reference calculations....
[13.03|11:01:18] Running reference calculations on frames [9, 17] from step1_attempt4_simulation/ams.rkf
[13.03|11:01:18] Calculating 2 frames in total
[13.03|11:01:18] Running step1_attempt4_reference_calc2
[13.03|11:01:21] Running step1_attempt4_reference_calc3
[13.03|11:01:24] Reference calculations finished!
[13.03|11:01:25] Adding results from step1_attempt4_reference_calc2 to validation set
[13.03|11:01:25] Adding results from step1_attempt4_reference_calc3 to training set
[13.03|11:01:25] Current # training set entries: 44
[13.03|11:01:25] Current # validation set entries: 20
[13.03|11:01:25] Storing data in step1_attempt4_reference_data
[13.03|11:01:25] Deleting step1_attempt4_reference_calc2
[13.03|11:01:25] Deleting step1_attempt4_reference_calc3
[13.03|11:01:25] Launching reparametrization job: step1_attempt4_training
[13.03|11:01:31] JOB m3gnet STARTED
[13.03|11:01:31] Starting m3gnet.prerun()
[13.03|11:01:31] m3gnet.prerun() finished
[13.03|11:01:31] JOB m3gnet RUNNING
[13.03|11:01:31] Executing m3gnet.run
[13.03|11:02:40] training_set Optimizer: 001 Epoch: 0 Loss: 0.002785
[13.03|11:02:40] validation_set Optimizer: 001 Epoch: 0 Loss: 0.049659
[13.03|11:03:01] training_set Optimizer: 001 Epoch: 10 Loss: 0.000095
[13.03|11:03:01] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005348
[13.03|11:03:22] training_set Optimizer: 001 Epoch: 20 Loss: 0.000085
[13.03|11:03:22] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005347
[13.03|11:03:44] training_set Optimizer: 001 Epoch: 30 Loss: 0.000085
[13.03|11:03:44] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005038
[13.03|11:04:05] training_set Optimizer: 001 Epoch: 40 Loss: 0.000080
[13.03|11:04:05] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006006
[13.03|11:04:27] training_set Optimizer: 001 Epoch: 50 Loss: 0.000076
[13.03|11:04:27] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005273
[13.03|11:04:48] training_set Optimizer: 001 Epoch: 60 Loss: 0.000074
[13.03|11:04:48] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005426
[13.03|11:05:10] training_set Optimizer: 001 Epoch: 70 Loss: 0.000081
[13.03|11:05:10] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006694
[13.03|11:05:31] training_set Optimizer: 001 Epoch: 80 Loss: 0.000076
[13.03|11:05:31] validation_set Optimizer: 001 Epoch: 80 Loss: 0.004972
[13.03|11:05:53] training_set Optimizer: 001 Epoch: 90 Loss: 0.000071
[13.03|11:05:53] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005010
[13.03|11:06:14] training_set Optimizer: 001 Epoch: 100 Loss: 0.000072
[13.03|11:06:14] validation_set Optimizer: 001 Epoch: 100 Loss: 0.006714
[13.03|11:06:36] training_set Optimizer: 001 Epoch: 110 Loss: 0.000068
[13.03|11:06:36] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005842
[13.03|11:06:57] training_set Optimizer: 001 Epoch: 120 Loss: 0.000076
[13.03|11:06:57] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005358
[13.03|11:07:18] training_set Optimizer: 001 Epoch: 130 Loss: 0.000083
[13.03|11:07:18] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005154
[13.03|11:07:40] training_set Optimizer: 001 Epoch: 140 Loss: 0.000072
[13.03|11:07:40] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005658
[13.03|11:08:02] training_set Optimizer: 001 Epoch: 150 Loss: 0.000069
[13.03|11:08:02] validation_set Optimizer: 001 Epoch: 150 Loss: 0.007076
[13.03|11:08:23] training_set Optimizer: 001 Epoch: 160 Loss: 0.000078
[13.03|11:08:23] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005080
[13.03|11:08:45] training_set Optimizer: 001 Epoch: 170 Loss: 0.000071
[13.03|11:08:45] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006236
[13.03|11:09:06] training_set Optimizer: 001 Epoch: 180 Loss: 0.000074
[13.03|11:09:06] validation_set Optimizer: 001 Epoch: 180 Loss: 0.006607
[13.03|11:09:27] training_set Optimizer: 001 Epoch: 190 Loss: 0.000068
[13.03|11:09:27] validation_set Optimizer: 001 Epoch: 190 Loss: 0.005340
[13.03|11:09:49] Execution of m3gnet.run finished with returncode 0
[13.03|11:09:50] JOB m3gnet FINISHED
[13.03|11:09:50] Starting m3gnet.postrun()
[13.03|11:09:50] m3gnet.postrun() finished
[13.03|11:09:50] JOB m3gnet SUCCESSFUL
[13.03|11:10:09] Running all jobs through AMS....
[13.03|11:10:09] Storing results/optimization/training_set_results/best
[13.03|11:10:09] Storing results/optimization/validation_set_results/best
[13.03|11:10:09] PLAMS environment cleaned up successfully
[13.03|11:10:09] PLAMS run finished. Goodbye
[13.03|11:10:11] ParAMSResults training_set validation_set
[13.03|11:10:11] energy MAE 0.0896 0.0685 eV
[13.03|11:10:11] forces MAE 0.0194 0.0468 eV/angstrom
[13.03|11:10:11] Newly created parameter file/dir: step1_attempt4_training/results/optimization/m3gnet/m3gnet
[13.03|11:10:11] Done!
[13.03|11:10:11] Deleting step1_attempt3_training
[13.03|11:10:11] ##########################
[13.03|11:10:11] ### Step 1 / Attempt 5 ###
[13.03|11:10:11] ##########################
[13.03|11:10:11] MD Steps: 500 (cumulative: 500)
[13.03|11:10:11] Current engine settings:
[13.03|11:10:11]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step1_attempt4_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:10:11] Running step1_attempt5_simulation...
[13.03|11:10:49] Job step1_attempt5_simulation finished
[13.03|11:10:49] Deleting files that are no longer needed...
[13.03|11:10:49] Launching reference calculation
[13.03|11:10:52] Reference calculation finished!
[13.03|11:10:52] Checking success for step1_attempt5
[13.03|11:11:03] CheckEnergy: Checking energy for MDStep500, n_atoms = 46
[13.03|11:11:03] CheckEnergy: normalization coefficient = 46
[13.03|11:11:03] CheckEnergy: Actual Threshold
[13.03|11:11:03] CheckEnergy: dE/46 0.0006 0.0100 OK!
[13.03|11:11:03] CheckEnergy: ddE/46 -0.0015 0.0020 OK! (relative to step1_attempt4_simulation:MDStep500)
[13.03|11:11:03]
[13.03|11:11:03] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:11:03] CheckForces: ------------
[13.03|11:11:03] CheckForces: Reference job from step1_attempt5_reference_calc1
[13.03|11:11:03] CheckForces: Prediction job from final frame (MDStep500) of step1_attempt5_simulation
[13.03|11:11:03] CheckForces: ------------
[13.03|11:11:03] CheckForces: Histogram of forces
[13.03|11:11:03] CheckForces: eV/Ang Ref Pred
[13.03|11:11:03] CheckForces: -2 3 2
[13.03|11:11:03] CheckForces: -1 73 70
[13.03|11:11:03] CheckForces: 0 59 63
[13.03|11:11:03] CheckForces: 1 3 3
[13.03|11:11:03] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:11:03] CheckForces: All force components are within the acceptable error!
[13.03|11:11:03] CheckForces: Maximum deviation: 0.230 eV/angstrom
[13.03|11:11:03] CheckForces: Actual Threshold
[13.03|11:11:03] CheckForces: # > thr. 0 0 OK!
[13.03|11:11:03] CheckForces: MAE 0.055 0.30 OK!
[13.03|11:11:03] CheckForces: R^2 0.971 0.80 OK!
[13.03|11:11:03] CheckForces: --------------------
[13.03|11:11:03]
[13.03|11:11:04] Adding results from step1_attempt5_reference_calc1 to training set
[13.03|11:11:04] Current # training set entries: 45
[13.03|11:11:04] Current # validation set entries: 20
[13.03|11:11:04] Storing data in step1_attempt5_reference_data
[13.03|11:11:04] Deleting step1_attempt4_reference_data
[13.03|11:11:04] Deleting step1_attempt5_reference_calc1
[13.03|11:11:04]
[13.03|11:11:04] Current (cumulative) timings:
[13.03|11:11:04] Time (s) Fraction
[13.03|11:11:04] Ref. calcs 39.34 0.018
[13.03|11:11:04] ML training 1988.46 0.900
[13.03|11:11:04] Simulations 182.80 0.083
[13.03|11:11:04]
[13.03|11:11:04]
[13.03|11:11:04] Step 1 finished successfully!
[13.03|11:11:04]
[13.03|11:11:04] --- Begin summary ---
[13.03|11:11:04] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:11:04] 1 1 FAILED Inaccurate 0.4793
[13.03|11:11:04] 1 2 FAILED Inaccurate 0.4059
[13.03|11:11:04] 1 3 FAILED Inaccurate 0.4848
[13.03|11:11:04] 1 4 FAILED Inaccurate 0.6954
[13.03|11:11:04] 1 5 SUCCESS Accurate 0.2303
[13.03|11:11:04] --- End summary ---
[13.03|11:11:04]
[13.03|11:11:04] ##########################
[13.03|11:11:04] ### Step 2 / Attempt 1 ###
[13.03|11:11:04] ##########################
[13.03|11:11:04] MD Steps: 2000 (cumulative: 2500)
[13.03|11:11:04] Current engine settings:
[13.03|11:11:04]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step1_attempt4_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:11:04] Running step2_attempt1_simulation...
[13.03|11:12:44] Job step2_attempt1_simulation finished
[13.03|11:12:44] Deleting files that are no longer needed...
[13.03|11:12:45] Deleting step1_attempt1_simulation
[13.03|11:12:45] Deleting step1_attempt2_simulation
[13.03|11:12:45] Deleting step1_attempt3_simulation
[13.03|11:12:45] Deleting step1_attempt4_simulation
[13.03|11:12:45] Launching reference calculation
[13.03|11:12:48] Reference calculation finished!
[13.03|11:12:48] Checking success for step2_attempt1
[13.03|11:13:00] CheckEnergy: Checking energy for MDStep2500, n_atoms = 46
[13.03|11:13:00] CheckEnergy: normalization coefficient = 46
[13.03|11:13:00] CheckEnergy: Actual Threshold
[13.03|11:13:00] CheckEnergy: dE/46 -0.0004 0.0100 OK!
[13.03|11:13:00] CheckEnergy: ddE/46 -0.0010 0.0020 OK! (relative to step1_attempt5_simulation:MDStep500)
[13.03|11:13:00]
[13.03|11:13:00] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:13:00] CheckForces: ------------
[13.03|11:13:00] CheckForces: Reference job from step2_attempt1_reference_calc1
[13.03|11:13:00] CheckForces: Prediction job from final frame (MDStep2500) of step2_attempt1_simulation
[13.03|11:13:00] CheckForces: ------------
[13.03|11:13:00] CheckForces: Histogram of forces
[13.03|11:13:00] CheckForces: eV/Ang Ref Pred
[13.03|11:13:00] CheckForces: -3 0 0
[13.03|11:13:00] CheckForces: -2 5 6
[13.03|11:13:00] CheckForces: -1 53 56
[13.03|11:13:00] CheckForces: 0 76 72
[13.03|11:13:00] CheckForces: 1 3 3
[13.03|11:13:00] CheckForces: 2 1 1
[13.03|11:13:00] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:13:00] CheckForces: Force components with an error exceeding the threshold:
[13.03|11:13:00] CheckForces: Ref Pred Delta Threshold
[13.03|11:13:00] CheckForces: -1.15 -1.69 0.54 0.36
[13.03|11:13:00] CheckForces: 0.28 -0.11 0.39 0.31
[13.03|11:13:00] CheckForces: -0.78 -1.39 0.61 0.34
[13.03|11:13:00] CheckForces: 0.02 0.42 0.40 0.30
[13.03|11:13:00] CheckForces: -1.01 -0.53 0.48 0.36
[13.03|11:13:00] CheckForces: -0.75 -0.37 0.39 0.34
[13.03|11:13:00] CheckForces: ... and 13 more.
[13.03|11:13:00] CheckForces: Maximum deviation: 0.608 eV/angstrom
[13.03|11:13:00] CheckForces: Actual Threshold
[13.03|11:13:00] CheckForces: # > thr. 19 0 Not OK!
[13.03|11:13:00] CheckForces: MAE 0.145 0.30 OK!
[13.03|11:13:00] CheckForces: R^2 0.830 0.80 OK!
[13.03|11:13:00] CheckForces: --------------------
[13.03|11:13:00]
[13.03|11:13:00] Adding results from step2_attempt1_reference_calc1 to training set
[13.03|11:13:00] Current # training set entries: 46
[13.03|11:13:00] Current # validation set entries: 20
[13.03|11:13:00] Storing data in step2_attempt1_reference_data
[13.03|11:13:00] Deleting step1_attempt5_reference_data
[13.03|11:13:00] Deleting step2_attempt1_reference_calc1
[13.03|11:13:00]
[13.03|11:13:00] Current (cumulative) timings:
[13.03|11:13:00] Time (s) Fraction
[13.03|11:13:00] Ref. calcs 42.25 0.018
[13.03|11:13:00] ML training 1988.46 0.859
[13.03|11:13:00] Simulations 283.07 0.122
[13.03|11:13:00]
[13.03|11:13:00]
[13.03|11:13:01]
[13.03|11:13:01] --- Begin summary ---
[13.03|11:13:01] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:13:01] 1 1 FAILED Inaccurate 0.4793
[13.03|11:13:01] 1 2 FAILED Inaccurate 0.4059
[13.03|11:13:01] 1 3 FAILED Inaccurate 0.4848
[13.03|11:13:01] 1 4 FAILED Inaccurate 0.6954
[13.03|11:13:01] 1 5 SUCCESS Accurate 0.2303
[13.03|11:13:01] 2 1 FAILED Inaccurate 0.6081
[13.03|11:13:01] --- End summary ---
[13.03|11:13:01]
[13.03|11:13:01] Running more reference calculations....
[13.03|11:13:01] Running reference calculations on frames [60, 93] from step2_attempt1_simulation/ams.rkf
[13.03|11:13:01] Calculating 2 frames in total
[13.03|11:13:01] Running step2_attempt1_reference_calc2
[13.03|11:13:04] Running step2_attempt1_reference_calc3
[13.03|11:13:07] Reference calculations finished!
[13.03|11:13:07] Adding results from step2_attempt1_reference_calc2 to validation set
[13.03|11:13:07] Adding results from step2_attempt1_reference_calc3 to training set
[13.03|11:13:07] Current # training set entries: 47
[13.03|11:13:07] Current # validation set entries: 21
[13.03|11:13:07] Storing data in step2_attempt1_reference_data
[13.03|11:13:08] Deleting step2_attempt1_reference_calc2
[13.03|11:13:08] Deleting step2_attempt1_reference_calc3
[13.03|11:13:08] Launching reparametrization job: step2_attempt1_training
[13.03|11:13:13] JOB m3gnet STARTED
[13.03|11:13:13] Starting m3gnet.prerun()
[13.03|11:13:13] m3gnet.prerun() finished
[13.03|11:13:14] JOB m3gnet RUNNING
[13.03|11:13:14] Executing m3gnet.run
[13.03|11:14:24] training_set Optimizer: 001 Epoch: 0 Loss: 0.002633
[13.03|11:14:24] validation_set Optimizer: 001 Epoch: 0 Loss: 0.023347
[13.03|11:14:47] training_set Optimizer: 001 Epoch: 10 Loss: 0.000124
[13.03|11:14:47] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006000
[13.03|11:15:11] training_set Optimizer: 001 Epoch: 20 Loss: 0.000105
[13.03|11:15:11] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005653
[13.03|11:15:34] training_set Optimizer: 001 Epoch: 30 Loss: 0.000097
[13.03|11:15:34] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006269
[13.03|11:15:58] training_set Optimizer: 001 Epoch: 40 Loss: 0.000098
[13.03|11:15:58] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005515
[13.03|11:16:22] training_set Optimizer: 001 Epoch: 50 Loss: 0.000087
[13.03|11:16:22] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005672
[13.03|11:16:45] training_set Optimizer: 001 Epoch: 60 Loss: 0.000084
[13.03|11:16:45] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006779
[13.03|11:17:09] training_set Optimizer: 001 Epoch: 70 Loss: 0.000080
[13.03|11:17:09] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005426
[13.03|11:17:32] training_set Optimizer: 001 Epoch: 80 Loss: 0.000079
[13.03|11:17:32] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005286
[13.03|11:17:56] training_set Optimizer: 001 Epoch: 90 Loss: 0.000089
[13.03|11:17:56] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006796
[13.03|11:18:20] training_set Optimizer: 001 Epoch: 100 Loss: 0.000079
[13.03|11:18:20] validation_set Optimizer: 001 Epoch: 100 Loss: 0.006138
[13.03|11:18:43] training_set Optimizer: 001 Epoch: 110 Loss: 0.000083
[13.03|11:18:43] validation_set Optimizer: 001 Epoch: 110 Loss: 0.008457
[13.03|11:19:07] training_set Optimizer: 001 Epoch: 120 Loss: 0.000085
[13.03|11:19:07] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005965
[13.03|11:19:30] training_set Optimizer: 001 Epoch: 130 Loss: 0.000086
[13.03|11:19:30] validation_set Optimizer: 001 Epoch: 130 Loss: 0.009134
[13.03|11:19:54] training_set Optimizer: 001 Epoch: 140 Loss: 0.000078
[13.03|11:19:54] validation_set Optimizer: 001 Epoch: 140 Loss: 0.007163
[13.03|11:20:17] training_set Optimizer: 001 Epoch: 150 Loss: 0.000076
[13.03|11:20:17] validation_set Optimizer: 001 Epoch: 150 Loss: 0.006783
[13.03|11:20:40] training_set Optimizer: 001 Epoch: 160 Loss: 0.000066
[13.03|11:20:40] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006149
[13.03|11:21:04] training_set Optimizer: 001 Epoch: 170 Loss: 0.000081
[13.03|11:21:04] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006311
[13.03|11:21:28] training_set Optimizer: 001 Epoch: 180 Loss: 0.000078
[13.03|11:21:28] validation_set Optimizer: 001 Epoch: 180 Loss: 0.013387
[13.03|11:21:51] training_set Optimizer: 001 Epoch: 190 Loss: 0.000066
[13.03|11:21:51] validation_set Optimizer: 001 Epoch: 190 Loss: 0.005298
[13.03|11:22:14] Execution of m3gnet.run finished with returncode 0
[13.03|11:22:14] JOB m3gnet FINISHED
[13.03|11:22:14] Starting m3gnet.postrun()
[13.03|11:22:14] m3gnet.postrun() finished
[13.03|11:22:15] JOB m3gnet SUCCESSFUL
[13.03|11:22:33] Running all jobs through AMS....
[13.03|11:22:34] Storing results/optimization/training_set_results/best
[13.03|11:22:34] Storing results/optimization/validation_set_results/best
[13.03|11:22:34] PLAMS environment cleaned up successfully
[13.03|11:22:34] PLAMS run finished. Goodbye
[13.03|11:22:35] ParAMSResults training_set validation_set
[13.03|11:22:35] energy MAE 0.0344 0.0581 eV
[13.03|11:22:35] forces MAE 0.0199 0.0487 eV/angstrom
[13.03|11:22:35] Newly created parameter file/dir: step2_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|11:22:35] Done!
[13.03|11:22:35] Deleting step1_attempt4_training
[13.03|11:22:35] ##########################
[13.03|11:22:35] ### Step 2 / Attempt 2 ###
[13.03|11:22:35] ##########################
[13.03|11:22:35] MD Steps: 2000 (cumulative: 2500)
[13.03|11:22:35] Current engine settings:
[13.03|11:22:35]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step2_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:22:35] Running step2_attempt2_simulation...
[13.03|11:24:15] Job step2_attempt2_simulation finished
[13.03|11:24:15] Deleting files that are no longer needed...
[13.03|11:24:15] Launching reference calculation
[13.03|11:24:18] Reference calculation finished!
[13.03|11:24:18] Checking success for step2_attempt2
[13.03|11:24:30] CheckEnergy: Checking energy for MDStep2500, n_atoms = 46
[13.03|11:24:30] CheckEnergy: normalization coefficient = 46
[13.03|11:24:30] CheckEnergy: Actual Threshold
[13.03|11:24:30] CheckEnergy: dE/46 -0.0038 0.0100 OK!
[13.03|11:24:30] CheckEnergy: ddE/46 -0.0021 0.0020 Not OK! (relative to step2_attempt1_simulation:MDStep2500)
[13.03|11:24:30]
[13.03|11:24:30] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:24:30] CheckForces: ------------
[13.03|11:24:30] CheckForces: Reference job from step2_attempt2_reference_calc1
[13.03|11:24:30] CheckForces: Prediction job from final frame (MDStep2500) of step2_attempt2_simulation
[13.03|11:24:30] CheckForces: ------------
[13.03|11:24:30] CheckForces: Histogram of forces
[13.03|11:24:30] CheckForces: eV/Ang Ref Pred
[13.03|11:24:30] CheckForces: -3 0 0
[13.03|11:24:30] CheckForces: -2 3 3
[13.03|11:24:30] CheckForces: -1 65 64
[13.03|11:24:30] CheckForces: 0 68 70
[13.03|11:24:30] CheckForces: 1 2 1
[13.03|11:24:30] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:24:30] CheckForces: Force components with an error exceeding the threshold:
[13.03|11:24:30] CheckForces: Ref Pred Delta Threshold
[13.03|11:24:30] CheckForces: -0.59 0.05 0.64 0.33
[13.03|11:24:30] CheckForces: -0.70 -0.29 0.41 0.34
[13.03|11:24:30] CheckForces: -0.65 -0.30 0.35 0.33
[13.03|11:24:30] CheckForces: 0.55 0.92 0.37 0.33
[13.03|11:24:30] CheckForces: 0.40 0.03 0.38 0.32
[13.03|11:24:30] CheckForces: Maximum deviation: 0.639 eV/angstrom
[13.03|11:24:30] CheckForces: Actual Threshold
[13.03|11:24:30] CheckForces: # > thr. 5 0 Not OK!
[13.03|11:24:30] CheckForces: MAE 0.085 0.30 OK!
[13.03|11:24:30] CheckForces: R^2 0.897 0.80 OK!
[13.03|11:24:30] CheckForces: --------------------
[13.03|11:24:30]
[13.03|11:24:31] Adding results from step2_attempt2_reference_calc1 to training set
[13.03|11:24:31] Current # training set entries: 48
[13.03|11:24:31] Current # validation set entries: 21
[13.03|11:24:31] Storing data in step2_attempt2_reference_data
[13.03|11:24:31] Deleting step2_attempt1_reference_data
[13.03|11:24:31] Deleting step2_attempt2_reference_calc1
[13.03|11:24:31]
[13.03|11:24:31] Current (cumulative) timings:
[13.03|11:24:31] Time (s) Fraction
[13.03|11:24:31] Ref. calcs 51.39 0.017
[13.03|11:24:31] ML training 2556.16 0.855
[13.03|11:24:31] Simulations 382.11 0.128
[13.03|11:24:31]
[13.03|11:24:31]
[13.03|11:24:31]
[13.03|11:24:31] --- Begin summary ---
[13.03|11:24:31] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:24:31] 1 1 FAILED Inaccurate 0.4793
[13.03|11:24:31] 1 2 FAILED Inaccurate 0.4059
[13.03|11:24:31] 1 3 FAILED Inaccurate 0.4848
[13.03|11:24:31] 1 4 FAILED Inaccurate 0.6954
[13.03|11:24:31] 1 5 SUCCESS Accurate 0.2303
[13.03|11:24:31] 2 1 FAILED Inaccurate 0.6081
[13.03|11:24:31] 2 2 FAILED Inaccurate 0.6387
[13.03|11:24:31] --- End summary ---
[13.03|11:24:31]
[13.03|11:24:31] Running more reference calculations....
[13.03|11:24:31] Running reference calculations on frames [60, 93] from step2_attempt2_simulation/ams.rkf
[13.03|11:24:31] Calculating 2 frames in total
[13.03|11:24:31] Running step2_attempt2_reference_calc2
[13.03|11:24:34] Running step2_attempt2_reference_calc3
[13.03|11:24:37] Reference calculations finished!
[13.03|11:24:38] Adding results from step2_attempt2_reference_calc2 to validation set
[13.03|11:24:38] Adding results from step2_attempt2_reference_calc3 to training set
[13.03|11:24:38] Current # training set entries: 49
[13.03|11:24:38] Current # validation set entries: 22
[13.03|11:24:38] Storing data in step2_attempt2_reference_data
[13.03|11:24:38] Deleting step2_attempt2_reference_calc2
[13.03|11:24:38] Deleting step2_attempt2_reference_calc3
[13.03|11:24:38] Launching reparametrization job: step2_attempt2_training
[13.03|11:24:44] JOB m3gnet STARTED
[13.03|11:24:44] Starting m3gnet.prerun()
[13.03|11:24:44] m3gnet.prerun() finished
[13.03|11:24:44] JOB m3gnet RUNNING
[13.03|11:24:44] Executing m3gnet.run
[13.03|11:25:55] training_set Optimizer: 001 Epoch: 0 Loss: 0.001445
[13.03|11:25:55] validation_set Optimizer: 001 Epoch: 0 Loss: 0.029377
[13.03|11:26:19] training_set Optimizer: 001 Epoch: 10 Loss: 0.000089
[13.03|11:26:19] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005602
[13.03|11:26:43] training_set Optimizer: 001 Epoch: 20 Loss: 0.000085
[13.03|11:26:43] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005480
[13.03|11:27:07] training_set Optimizer: 001 Epoch: 30 Loss: 0.000079
[13.03|11:27:07] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005725
[13.03|11:27:31] training_set Optimizer: 001 Epoch: 40 Loss: 0.000077
[13.03|11:27:31] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005386
[13.03|11:27:55] training_set Optimizer: 001 Epoch: 50 Loss: 0.000072
[13.03|11:27:55] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005935
[13.03|11:28:18] training_set Optimizer: 001 Epoch: 60 Loss: 0.000071
[13.03|11:28:18] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006309
[13.03|11:28:42] training_set Optimizer: 001 Epoch: 70 Loss: 0.000084
[13.03|11:28:42] validation_set Optimizer: 001 Epoch: 70 Loss: 0.008546
[13.03|11:29:07] training_set Optimizer: 001 Epoch: 80 Loss: 0.000076
[13.03|11:29:07] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006999
[13.03|11:29:31] training_set Optimizer: 001 Epoch: 90 Loss: 0.000069
[13.03|11:29:31] validation_set Optimizer: 001 Epoch: 90 Loss: 0.008208
[13.03|11:29:55] training_set Optimizer: 001 Epoch: 100 Loss: 0.000071
[13.03|11:29:55] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005763
[13.03|11:30:19] training_set Optimizer: 001 Epoch: 110 Loss: 0.000074
[13.03|11:30:19] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006883
[13.03|11:30:44] training_set Optimizer: 001 Epoch: 120 Loss: 0.000076
[13.03|11:30:44] validation_set Optimizer: 001 Epoch: 120 Loss: 0.006991
[13.03|11:31:08] training_set Optimizer: 001 Epoch: 130 Loss: 0.000067
[13.03|11:31:08] validation_set Optimizer: 001 Epoch: 130 Loss: 0.008687
[13.03|11:31:32] training_set Optimizer: 001 Epoch: 140 Loss: 0.000074
[13.03|11:31:32] validation_set Optimizer: 001 Epoch: 140 Loss: 0.007081
[13.03|11:31:56] training_set Optimizer: 001 Epoch: 150 Loss: 0.000065
[13.03|11:31:56] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005380
[13.03|11:32:19] training_set Optimizer: 001 Epoch: 160 Loss: 0.000074
[13.03|11:32:19] validation_set Optimizer: 001 Epoch: 160 Loss: 0.011895
[13.03|11:32:43] training_set Optimizer: 001 Epoch: 170 Loss: 0.000068
[13.03|11:32:43] validation_set Optimizer: 001 Epoch: 170 Loss: 0.007006
[13.03|11:33:07] training_set Optimizer: 001 Epoch: 180 Loss: 0.000063
[13.03|11:33:07] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005963
[13.03|11:33:30] training_set Optimizer: 001 Epoch: 190 Loss: 0.000082
[13.03|11:33:30] validation_set Optimizer: 001 Epoch: 190 Loss: 0.011909
[13.03|11:33:54] Execution of m3gnet.run finished with returncode 0
[13.03|11:33:54] JOB m3gnet FINISHED
[13.03|11:33:54] Starting m3gnet.postrun()
[13.03|11:33:54] m3gnet.postrun() finished
[13.03|11:33:55] JOB m3gnet SUCCESSFUL
[13.03|11:34:13] Running all jobs through AMS....
[13.03|11:34:14] Storing results/optimization/training_set_results/best
[13.03|11:34:14] Storing results/optimization/validation_set_results/best
[13.03|11:34:14] PLAMS environment cleaned up successfully
[13.03|11:34:14] PLAMS run finished. Goodbye
[13.03|11:34:15] ParAMSResults training_set validation_set
[13.03|11:34:15] energy MAE 0.0379 0.0457 eV
[13.03|11:34:15] forces MAE 0.0178 0.0488 eV/angstrom
[13.03|11:34:15] Newly created parameter file/dir: step2_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|11:34:15] Done!
[13.03|11:34:16] Deleting step2_attempt1_training
[13.03|11:34:16] ##########################
[13.03|11:34:16] ### Step 2 / Attempt 3 ###
[13.03|11:34:16] ##########################
[13.03|11:34:16] MD Steps: 2000 (cumulative: 2500)
[13.03|11:34:16] Current engine settings:
[13.03|11:34:16]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step2_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:34:16] Running step2_attempt3_simulation...
[13.03|11:35:55] Job step2_attempt3_simulation finished
[13.03|11:35:55] Deleting files that are no longer needed...
[13.03|11:35:56] Launching reference calculation
[13.03|11:35:59] Reference calculation finished!
[13.03|11:35:59] Checking success for step2_attempt3
[13.03|11:36:11] CheckEnergy: Checking energy for MDStep2500, n_atoms = 46
[13.03|11:36:11] CheckEnergy: normalization coefficient = 46
[13.03|11:36:11] CheckEnergy: Actual Threshold
[13.03|11:36:11] CheckEnergy: dE/46 0.0012 0.0100 OK!
[13.03|11:36:11] CheckEnergy: ddE/46 0.0015 0.0020 OK! (relative to step2_attempt2_simulation:MDStep2500)
[13.03|11:36:11]
[13.03|11:36:11] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:36:11] CheckForces: ------------
[13.03|11:36:11] CheckForces: Reference job from step2_attempt3_reference_calc1
[13.03|11:36:11] CheckForces: Prediction job from final frame (MDStep2500) of step2_attempt3_simulation
[13.03|11:36:11] CheckForces: ------------
[13.03|11:36:11] CheckForces: Histogram of forces
[13.03|11:36:11] CheckForces: eV/Ang Ref Pred
[13.03|11:36:11] CheckForces: -2 0 0
[13.03|11:36:11] CheckForces: -1 67 69
[13.03|11:36:11] CheckForces: 0 71 69
[13.03|11:36:11] CheckForces: 1 0 0
[13.03|11:36:11] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:36:11] CheckForces: Force components with an error exceeding the threshold:
[13.03|11:36:11] CheckForces: Ref Pred Delta Threshold
[13.03|11:36:11] CheckForces: 0.42 0.08 0.34 0.32
[13.03|11:36:11] CheckForces: 0.54 0.20 0.34 0.33
[13.03|11:36:11] CheckForces: -0.23 0.26 0.50 0.31
[13.03|11:36:11] CheckForces: Maximum deviation: 0.497 eV/angstrom
[13.03|11:36:11] CheckForces: Actual Threshold
[13.03|11:36:11] CheckForces: # > thr. 3 0 Not OK!
[13.03|11:36:11] CheckForces: MAE 0.062 0.30 OK!
[13.03|11:36:11] CheckForces: R^2 0.866 0.80 OK!
[13.03|11:36:11] CheckForces: --------------------
[13.03|11:36:11]
[13.03|11:36:11] Adding results from step2_attempt3_reference_calc1 to training set
[13.03|11:36:11] Current # training set entries: 50
[13.03|11:36:11] Current # validation set entries: 22
[13.03|11:36:11] Storing data in step2_attempt3_reference_data
[13.03|11:36:12] Deleting step2_attempt2_reference_data
[13.03|11:36:12] Deleting step2_attempt3_reference_calc1
[13.03|11:36:12]
[13.03|11:36:12] Current (cumulative) timings:
[13.03|11:36:12] Time (s) Fraction
[13.03|11:36:12] Ref. calcs 60.57 0.016
[13.03|11:36:12] ML training 3133.36 0.852
[13.03|11:36:12] Simulations 481.66 0.131
[13.03|11:36:12]
[13.03|11:36:12]
[13.03|11:36:12]
[13.03|11:36:12] --- Begin summary ---
[13.03|11:36:12] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:36:12] 1 1 FAILED Inaccurate 0.4793
[13.03|11:36:12] 1 2 FAILED Inaccurate 0.4059
[13.03|11:36:12] 1 3 FAILED Inaccurate 0.4848
[13.03|11:36:12] 1 4 FAILED Inaccurate 0.6954
[13.03|11:36:12] 1 5 SUCCESS Accurate 0.2303
[13.03|11:36:12] 2 1 FAILED Inaccurate 0.6081
[13.03|11:36:12] 2 2 FAILED Inaccurate 0.6387
[13.03|11:36:12] 2 3 FAILED Inaccurate 0.4972
[13.03|11:36:12] --- End summary ---
[13.03|11:36:12]
[13.03|11:36:12] Running more reference calculations....
[13.03|11:36:12] Running reference calculations on frames [60, 93] from step2_attempt3_simulation/ams.rkf
[13.03|11:36:12] Calculating 2 frames in total
[13.03|11:36:12] Running step2_attempt3_reference_calc2
[13.03|11:36:15] Running step2_attempt3_reference_calc3
[13.03|11:36:18] Reference calculations finished!
[13.03|11:36:19] Adding results from step2_attempt3_reference_calc2 to validation set
[13.03|11:36:19] Adding results from step2_attempt3_reference_calc3 to training set
[13.03|11:36:19] Current # training set entries: 51
[13.03|11:36:19] Current # validation set entries: 23
[13.03|11:36:19] Storing data in step2_attempt3_reference_data
[13.03|11:36:19] Deleting step2_attempt3_reference_calc2
[13.03|11:36:19] Deleting step2_attempt3_reference_calc3
[13.03|11:36:19] Launching reparametrization job: step2_attempt3_training
[13.03|11:36:25] JOB m3gnet STARTED
[13.03|11:36:25] Starting m3gnet.prerun()
[13.03|11:36:25] m3gnet.prerun() finished
[13.03|11:36:25] JOB m3gnet RUNNING
[13.03|11:36:25] Executing m3gnet.run
[13.03|11:37:35] training_set Optimizer: 001 Epoch: 0 Loss: 0.001470
[13.03|11:37:35] validation_set Optimizer: 001 Epoch: 0 Loss: 0.035555
[13.03|11:38:01] training_set Optimizer: 001 Epoch: 10 Loss: 0.000072
[13.03|11:38:01] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005566
[13.03|11:38:27] training_set Optimizer: 001 Epoch: 20 Loss: 0.000068
[13.03|11:38:27] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006156
[13.03|11:38:52] training_set Optimizer: 001 Epoch: 30 Loss: 0.000070
[13.03|11:38:52] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005724
[13.03|11:39:17] training_set Optimizer: 001 Epoch: 40 Loss: 0.000062
[13.03|11:39:17] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006248
[13.03|11:39:42] training_set Optimizer: 001 Epoch: 50 Loss: 0.000061
[13.03|11:39:42] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005738
[13.03|11:40:09] training_set Optimizer: 001 Epoch: 60 Loss: 0.000063
[13.03|11:40:09] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005973
[13.03|11:40:34] training_set Optimizer: 001 Epoch: 70 Loss: 0.000059
[13.03|11:40:34] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006253
[13.03|11:41:00] training_set Optimizer: 001 Epoch: 80 Loss: 0.000066
[13.03|11:41:00] validation_set Optimizer: 001 Epoch: 80 Loss: 0.006023
[13.03|11:41:25] training_set Optimizer: 001 Epoch: 90 Loss: 0.000067
[13.03|11:41:25] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006046
[13.03|11:41:51] training_set Optimizer: 001 Epoch: 100 Loss: 0.000061
[13.03|11:41:51] validation_set Optimizer: 001 Epoch: 100 Loss: 0.007432
[13.03|11:42:16] training_set Optimizer: 001 Epoch: 110 Loss: 0.000079
[13.03|11:42:16] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005911
[13.03|11:42:41] training_set Optimizer: 001 Epoch: 120 Loss: 0.000061
[13.03|11:42:41] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005955
[13.03|11:43:07] training_set Optimizer: 001 Epoch: 130 Loss: 0.000063
[13.03|11:43:07] validation_set Optimizer: 001 Epoch: 130 Loss: 0.010854
[13.03|11:43:32] training_set Optimizer: 001 Epoch: 140 Loss: 0.000057
[13.03|11:43:32] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005967
[13.03|11:43:58] training_set Optimizer: 001 Epoch: 150 Loss: 0.000069
[13.03|11:43:58] validation_set Optimizer: 001 Epoch: 150 Loss: 0.007302
[13.03|11:44:24] training_set Optimizer: 001 Epoch: 160 Loss: 0.000059
[13.03|11:44:24] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005792
[13.03|11:44:49] training_set Optimizer: 001 Epoch: 170 Loss: 0.000064
[13.03|11:44:49] validation_set Optimizer: 001 Epoch: 170 Loss: 0.005791
[13.03|11:45:14] training_set Optimizer: 001 Epoch: 180 Loss: 0.000056
[13.03|11:45:14] validation_set Optimizer: 001 Epoch: 180 Loss: 0.007107
[13.03|11:45:40] training_set Optimizer: 001 Epoch: 190 Loss: 0.000063
[13.03|11:45:40] validation_set Optimizer: 001 Epoch: 190 Loss: 0.010186
[13.03|11:46:04] Execution of m3gnet.run finished with returncode 0
[13.03|11:46:05] JOB m3gnet FINISHED
[13.03|11:46:05] Starting m3gnet.postrun()
[13.03|11:46:05] m3gnet.postrun() finished
[13.03|11:46:05] JOB m3gnet SUCCESSFUL
[13.03|11:46:24] Running all jobs through AMS....
[13.03|11:46:25] Storing results/optimization/training_set_results/best
[13.03|11:46:25] Storing results/optimization/validation_set_results/best
[13.03|11:46:25] PLAMS environment cleaned up successfully
[13.03|11:46:25] PLAMS run finished. Goodbye
[13.03|11:46:26] ParAMSResults training_set validation_set
[13.03|11:46:26] energy MAE 0.0208 0.0612 eV
[13.03|11:46:26] forces MAE 0.0250 0.0538 eV/angstrom
[13.03|11:46:27] Newly created parameter file/dir: step2_attempt3_training/results/optimization/m3gnet/m3gnet
[13.03|11:46:27] Done!
[13.03|11:46:27] Deleting step2_attempt2_training
[13.03|11:46:27] ##########################
[13.03|11:46:27] ### Step 2 / Attempt 4 ###
[13.03|11:46:27] ##########################
[13.03|11:46:27] MD Steps: 2000 (cumulative: 2500)
[13.03|11:46:27] Current engine settings:
[13.03|11:46:27]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step2_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:46:27] Running step2_attempt4_simulation...
[13.03|11:48:08] Job step2_attempt4_simulation finished
[13.03|11:48:08] Deleting files that are no longer needed...
[13.03|11:48:08] Launching reference calculation
[13.03|11:48:11] Reference calculation finished!
[13.03|11:48:11] Checking success for step2_attempt4
[13.03|11:48:24] CheckEnergy: Checking energy for MDStep2500, n_atoms = 46
[13.03|11:48:24] CheckEnergy: normalization coefficient = 46
[13.03|11:48:24] CheckEnergy: Actual Threshold
[13.03|11:48:24] CheckEnergy: dE/46 -0.0004 0.0100 OK!
[13.03|11:48:24] CheckEnergy: ddE/46 0.0003 0.0020 OK! (relative to step2_attempt3_simulation:MDStep2500)
[13.03|11:48:24]
[13.03|11:48:24] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|11:48:24] CheckForces: ------------
[13.03|11:48:24] CheckForces: Reference job from step2_attempt4_reference_calc1
[13.03|11:48:24] CheckForces: Prediction job from final frame (MDStep2500) of step2_attempt4_simulation
[13.03|11:48:24] CheckForces: ------------
[13.03|11:48:24] CheckForces: Histogram of forces
[13.03|11:48:24] CheckForces: eV/Ang Ref Pred
[13.03|11:48:24] CheckForces: -2 1 1
[13.03|11:48:24] CheckForces: -1 63 68
[13.03|11:48:24] CheckForces: 0 73 69
[13.03|11:48:24] CheckForces: 1 1 0
[13.03|11:48:24] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|11:48:24] CheckForces: Force components with an error exceeding the threshold:
[13.03|11:48:24] CheckForces: Ref Pred Delta Threshold
[13.03|11:48:24] CheckForces: 0.05 -0.36 0.41 0.30
[13.03|11:48:24] CheckForces: -0.63 -0.24 0.39 0.33
[13.03|11:48:24] CheckForces: 0.01 -0.30 0.31 0.30
[13.03|11:48:24] CheckForces: Maximum deviation: 0.412 eV/angstrom
[13.03|11:48:24] CheckForces: Actual Threshold
[13.03|11:48:24] CheckForces: # > thr. 3 0 Not OK!
[13.03|11:48:24] CheckForces: MAE 0.076 0.30 OK!
[13.03|11:48:24] CheckForces: R^2 0.843 0.80 OK!
[13.03|11:48:24] CheckForces: --------------------
[13.03|11:48:24]
[13.03|11:48:24] Adding results from step2_attempt4_reference_calc1 to training set
[13.03|11:48:24] Current # training set entries: 52
[13.03|11:48:24] Current # validation set entries: 23
[13.03|11:48:24] Storing data in step2_attempt4_reference_data
[13.03|11:48:25] Deleting step2_attempt3_reference_data
[13.03|11:48:25] Deleting step2_attempt4_reference_calc1
[13.03|11:48:25]
[13.03|11:48:25] Current (cumulative) timings:
[13.03|11:48:25] Time (s) Fraction
[13.03|11:48:25] Ref. calcs 69.74 0.016
[13.03|11:48:25] ML training 3740.75 0.851
[13.03|11:48:25] Simulations 582.79 0.133
[13.03|11:48:25]
[13.03|11:48:25]
[13.03|11:48:25]
[13.03|11:48:25] --- Begin summary ---
[13.03|11:48:25] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|11:48:25] 1 1 FAILED Inaccurate 0.4793
[13.03|11:48:25] 1 2 FAILED Inaccurate 0.4059
[13.03|11:48:25] 1 3 FAILED Inaccurate 0.4848
[13.03|11:48:25] 1 4 FAILED Inaccurate 0.6954
[13.03|11:48:25] 1 5 SUCCESS Accurate 0.2303
[13.03|11:48:25] 2 1 FAILED Inaccurate 0.6081
[13.03|11:48:25] 2 2 FAILED Inaccurate 0.6387
[13.03|11:48:25] 2 3 FAILED Inaccurate 0.4972
[13.03|11:48:25] 2 4 FAILED Inaccurate 0.4115
[13.03|11:48:25] --- End summary ---
[13.03|11:48:25]
[13.03|11:48:25] Running more reference calculations....
[13.03|11:48:25] Running reference calculations on frames [60, 93] from step2_attempt4_simulation/ams.rkf
[13.03|11:48:25] Calculating 2 frames in total
[13.03|11:48:25] Running step2_attempt4_reference_calc2
[13.03|11:48:28] Running step2_attempt4_reference_calc3
[13.03|11:48:31] Reference calculations finished!
[13.03|11:48:31] Adding results from step2_attempt4_reference_calc2 to validation set
[13.03|11:48:32] Adding results from step2_attempt4_reference_calc3 to training set
[13.03|11:48:32] Current # training set entries: 53
[13.03|11:48:32] Current # validation set entries: 24
[13.03|11:48:32] Storing data in step2_attempt4_reference_data
[13.03|11:48:32] Deleting step2_attempt4_reference_calc2
[13.03|11:48:32] Deleting step2_attempt4_reference_calc3
[13.03|11:48:32] Launching reparametrization job: step2_attempt4_training
[13.03|11:48:38] JOB m3gnet STARTED
[13.03|11:48:38] Starting m3gnet.prerun()
[13.03|11:48:38] m3gnet.prerun() finished
[13.03|11:48:38] JOB m3gnet RUNNING
[13.03|11:48:38] Executing m3gnet.run
[13.03|11:49:49] training_set Optimizer: 001 Epoch: 0 Loss: 0.001217
[13.03|11:49:49] validation_set Optimizer: 001 Epoch: 0 Loss: 0.015274
[13.03|11:50:15] training_set Optimizer: 001 Epoch: 10 Loss: 0.000069
[13.03|11:50:15] validation_set Optimizer: 001 Epoch: 10 Loss: 0.007221
[13.03|11:50:41] training_set Optimizer: 001 Epoch: 20 Loss: 0.000067
[13.03|11:50:41] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006648
[13.03|11:51:07] training_set Optimizer: 001 Epoch: 30 Loss: 0.000064
[13.03|11:51:07] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005750
[13.03|11:51:33] training_set Optimizer: 001 Epoch: 40 Loss: 0.000069
[13.03|11:51:33] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005581
[13.03|11:52:00] training_set Optimizer: 001 Epoch: 50 Loss: 0.000063
[13.03|11:52:00] validation_set Optimizer: 001 Epoch: 50 Loss: 0.007404
[13.03|11:52:26] training_set Optimizer: 001 Epoch: 60 Loss: 0.000071
[13.03|11:52:26] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007096
[13.03|11:52:52] training_set Optimizer: 001 Epoch: 70 Loss: 0.000061
[13.03|11:52:52] validation_set Optimizer: 001 Epoch: 70 Loss: 0.008880
[13.03|11:53:19] training_set Optimizer: 001 Epoch: 80 Loss: 0.000065
[13.03|11:53:19] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005786
[13.03|11:53:45] training_set Optimizer: 001 Epoch: 90 Loss: 0.000087
[13.03|11:53:45] validation_set Optimizer: 001 Epoch: 90 Loss: 0.011641
[13.03|11:54:11] training_set Optimizer: 001 Epoch: 100 Loss: 0.000055
[13.03|11:54:11] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005763
[13.03|11:54:37] training_set Optimizer: 001 Epoch: 110 Loss: 0.000059
[13.03|11:54:37] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005909
[13.03|11:55:03] training_set Optimizer: 001 Epoch: 120 Loss: 0.000077
[13.03|11:55:03] validation_set Optimizer: 001 Epoch: 120 Loss: 0.013412
[13.03|11:55:29] training_set Optimizer: 001 Epoch: 130 Loss: 0.000056
[13.03|11:55:29] validation_set Optimizer: 001 Epoch: 130 Loss: 0.006084
[13.03|11:55:56] training_set Optimizer: 001 Epoch: 140 Loss: 0.000055
[13.03|11:55:56] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005907
[13.03|11:56:21] training_set Optimizer: 001 Epoch: 150 Loss: 0.000079
[13.03|11:56:21] validation_set Optimizer: 001 Epoch: 150 Loss: 0.007728
[13.03|11:56:48] training_set Optimizer: 001 Epoch: 160 Loss: 0.000052
[13.03|11:56:48] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006141
[13.03|11:57:14] training_set Optimizer: 001 Epoch: 170 Loss: 0.000066
[13.03|11:57:14] validation_set Optimizer: 001 Epoch: 170 Loss: 0.009011
[13.03|11:57:40] training_set Optimizer: 001 Epoch: 180 Loss: 0.000062
[13.03|11:57:40] validation_set Optimizer: 001 Epoch: 180 Loss: 0.006031
[13.03|11:58:06] training_set Optimizer: 001 Epoch: 190 Loss: 0.000055
[13.03|11:58:06] validation_set Optimizer: 001 Epoch: 190 Loss: 0.006335
[13.03|11:58:32] Execution of m3gnet.run finished with returncode 0
[13.03|11:58:32] JOB m3gnet FINISHED
[13.03|11:58:32] Starting m3gnet.postrun()
[13.03|11:58:32] m3gnet.postrun() finished
[13.03|11:58:33] JOB m3gnet SUCCESSFUL
[13.03|11:58:53] Running all jobs through AMS....
[13.03|11:58:54] Storing results/optimization/training_set_results/best
[13.03|11:58:54] Storing results/optimization/validation_set_results/best
[13.03|11:58:54] PLAMS environment cleaned up successfully
[13.03|11:58:54] PLAMS run finished. Goodbye
[13.03|11:58:55] ParAMSResults training_set validation_set
[13.03|11:58:55] energy MAE 0.0923 0.1344 eV
[13.03|11:58:55] forces MAE 0.0176 0.0506 eV/angstrom
[13.03|11:58:55] Newly created parameter file/dir: step2_attempt4_training/results/optimization/m3gnet/m3gnet
[13.03|11:58:55] Done!
[13.03|11:58:55] Deleting step2_attempt3_training
[13.03|11:58:55] ##########################
[13.03|11:58:55] ### Step 2 / Attempt 5 ###
[13.03|11:58:55] ##########################
[13.03|11:58:55] MD Steps: 2000 (cumulative: 2500)
[13.03|11:58:55] Current engine settings:
[13.03|11:58:55]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step2_attempt4_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|11:58:55] Running step2_attempt5_simulation...
[13.03|12:00:34] Job step2_attempt5_simulation finished
[13.03|12:00:34] Deleting files that are no longer needed...
[13.03|12:00:35] Launching reference calculation
[13.03|12:00:38] Reference calculation finished!
[13.03|12:00:38] Checking success for step2_attempt5
[13.03|12:00:50] CheckEnergy: Checking energy for MDStep2500, n_atoms = 46
[13.03|12:00:50] CheckEnergy: normalization coefficient = 46
[13.03|12:00:50] CheckEnergy: Actual Threshold
[13.03|12:00:50] CheckEnergy: dE/46 -0.0012 0.0100 OK!
[13.03|12:00:50] CheckEnergy: ddE/46 0.0006 0.0020 OK! (relative to step2_attempt4_simulation:MDStep2500)
[13.03|12:00:50]
[13.03|12:00:50] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:00:50] CheckForces: ------------
[13.03|12:00:50] CheckForces: Reference job from step2_attempt5_reference_calc1
[13.03|12:00:50] CheckForces: Prediction job from final frame (MDStep2500) of step2_attempt5_simulation
[13.03|12:00:50] CheckForces: ------------
[13.03|12:00:50] CheckForces: Histogram of forces
[13.03|12:00:50] CheckForces: eV/Ang Ref Pred
[13.03|12:00:50] CheckForces: -2 0 0
[13.03|12:00:50] CheckForces: -1 72 61
[13.03|12:00:50] CheckForces: 0 65 76
[13.03|12:00:50] CheckForces: 1 1 1
[13.03|12:00:50] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:00:50] CheckForces: All force components are within the acceptable error!
[13.03|12:00:50] CheckForces: Maximum deviation: 0.207 eV/angstrom
[13.03|12:00:50] CheckForces: Actual Threshold
[13.03|12:00:50] CheckForces: # > thr. 0 0 OK!
[13.03|12:00:50] CheckForces: MAE 0.047 0.30 OK!
[13.03|12:00:50] CheckForces: R^2 0.962 0.80 OK!
[13.03|12:00:50] CheckForces: --------------------
[13.03|12:00:50]
[13.03|12:00:50] Adding results from step2_attempt5_reference_calc1 to training set
[13.03|12:00:50] Current # training set entries: 54
[13.03|12:00:50] Current # validation set entries: 24
[13.03|12:00:50] Storing data in step2_attempt5_reference_data
[13.03|12:00:51] Deleting step2_attempt4_reference_data
[13.03|12:00:51] Deleting step2_attempt5_reference_calc1
[13.03|12:00:51]
[13.03|12:00:51] Current (cumulative) timings:
[13.03|12:00:51] Time (s) Fraction
[13.03|12:00:51] Ref. calcs 78.79 0.015
[13.03|12:00:51] ML training 4364.15 0.852
[13.03|12:00:51] Simulations 681.78 0.133
[13.03|12:00:51]
[13.03|12:00:51]
[13.03|12:00:51] Step 2 finished successfully!
[13.03|12:00:51]
[13.03|12:00:51] --- Begin summary ---
[13.03|12:00:51] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:00:51] 1 1 FAILED Inaccurate 0.4793
[13.03|12:00:51] 1 2 FAILED Inaccurate 0.4059
[13.03|12:00:51] 1 3 FAILED Inaccurate 0.4848
[13.03|12:00:51] 1 4 FAILED Inaccurate 0.6954
[13.03|12:00:51] 1 5 SUCCESS Accurate 0.2303
[13.03|12:00:51] 2 1 FAILED Inaccurate 0.6081
[13.03|12:00:51] 2 2 FAILED Inaccurate 0.6387
[13.03|12:00:51] 2 3 FAILED Inaccurate 0.4972
[13.03|12:00:51] 2 4 FAILED Inaccurate 0.4115
[13.03|12:00:51] 2 5 SUCCESS Accurate 0.2071
[13.03|12:00:51] --- End summary ---
[13.03|12:00:51]
[13.03|12:00:51] ##########################
[13.03|12:00:51] ### Step 3 / Attempt 1 ###
[13.03|12:00:51] ##########################
[13.03|12:00:51] MD Steps: 2000 (cumulative: 4500)
[13.03|12:00:51] Current engine settings:
[13.03|12:00:51]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step2_attempt4_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:00:51] Running step3_attempt1_simulation...
[13.03|12:02:26] Job step3_attempt1_simulation finished
[13.03|12:02:26] Deleting files that are no longer needed...
[13.03|12:02:26] Deleting step1_attempt5_simulation
[13.03|12:02:26] Deleting step2_attempt1_simulation
[13.03|12:02:26] Deleting step2_attempt2_simulation
[13.03|12:02:26] Deleting step2_attempt3_simulation
[13.03|12:02:26] Deleting step2_attempt4_simulation
[13.03|12:02:27] Launching reference calculation
[13.03|12:02:30] Reference calculation finished!
[13.03|12:02:30] Checking success for step3_attempt1
[13.03|12:02:42] CheckEnergy: Checking energy for MDStep4500, n_atoms = 46
[13.03|12:02:42] CheckEnergy: normalization coefficient = 46
[13.03|12:02:42] CheckEnergy: Actual Threshold
[13.03|12:02:42] CheckEnergy: dE/46 -0.0038 0.0100 OK!
[13.03|12:02:42] CheckEnergy: ddE/46 -0.0026 0.0020 Not OK! (relative to step2_attempt5_simulation:MDStep2500)
[13.03|12:02:42]
[13.03|12:02:42] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:02:42] CheckForces: ------------
[13.03|12:02:42] CheckForces: Reference job from step3_attempt1_reference_calc1
[13.03|12:02:42] CheckForces: Prediction job from final frame (MDStep4500) of step3_attempt1_simulation
[13.03|12:02:42] CheckForces: ------------
[13.03|12:02:42] CheckForces: Histogram of forces
[13.03|12:02:42] CheckForces: eV/Ang Ref Pred
[13.03|12:02:42] CheckForces: -2 0 0
[13.03|12:02:42] CheckForces: -1 69 69
[13.03|12:02:42] CheckForces: 0 68 69
[13.03|12:02:42] CheckForces: 1 1 0
[13.03|12:02:42] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:02:42] CheckForces: Force components with an error exceeding the threshold:
[13.03|12:02:42] CheckForces: Ref Pred Delta Threshold
[13.03|12:02:42] CheckForces: -0.12 -0.49 0.37 0.31
[13.03|12:02:42] CheckForces: 1.00 0.34 0.66 0.35
[13.03|12:02:42] CheckForces: -0.99 -0.52 0.47 0.35
[13.03|12:02:42] CheckForces: -0.49 -0.10 0.39 0.32
[13.03|12:02:42] CheckForces: 0.04 -0.28 0.32 0.30
[13.03|12:02:42] CheckForces: Maximum deviation: 0.659 eV/angstrom
[13.03|12:02:42] CheckForces: Actual Threshold
[13.03|12:02:42] CheckForces: # > thr. 5 0 Not OK!
[13.03|12:02:42] CheckForces: MAE 0.072 0.30 OK!
[13.03|12:02:42] CheckForces: R^2 0.856 0.80 OK!
[13.03|12:02:42] CheckForces: --------------------
[13.03|12:02:42]
[13.03|12:02:42] Adding results from step3_attempt1_reference_calc1 to training set
[13.03|12:02:42] Current # training set entries: 55
[13.03|12:02:42] Current # validation set entries: 24
[13.03|12:02:42] Storing data in step3_attempt1_reference_data
[13.03|12:02:43] Deleting step2_attempt5_reference_data
[13.03|12:02:43] Deleting step3_attempt1_reference_calc1
[13.03|12:02:43]
[13.03|12:02:43] Current (cumulative) timings:
[13.03|12:02:43] Time (s) Fraction
[13.03|12:02:43] Ref. calcs 81.73 0.016
[13.03|12:02:43] ML training 4364.15 0.836
[13.03|12:02:43] Simulations 776.76 0.149
[13.03|12:02:43]
[13.03|12:02:43]
[13.03|12:02:43]
[13.03|12:02:43] --- Begin summary ---
[13.03|12:02:43] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:02:43] 1 1 FAILED Inaccurate 0.4793
[13.03|12:02:43] 1 2 FAILED Inaccurate 0.4059
[13.03|12:02:43] 1 3 FAILED Inaccurate 0.4848
[13.03|12:02:43] 1 4 FAILED Inaccurate 0.6954
[13.03|12:02:43] 1 5 SUCCESS Accurate 0.2303
[13.03|12:02:43] 2 1 FAILED Inaccurate 0.6081
[13.03|12:02:43] 2 2 FAILED Inaccurate 0.6387
[13.03|12:02:43] 2 3 FAILED Inaccurate 0.4972
[13.03|12:02:43] 2 4 FAILED Inaccurate 0.4115
[13.03|12:02:43] 2 5 SUCCESS Accurate 0.2071
[13.03|12:02:43] 3 1 FAILED Inaccurate 0.6595
[13.03|12:02:43] --- End summary ---
[13.03|12:02:43]
[13.03|12:02:43] Running more reference calculations....
[13.03|12:02:43] Running reference calculations on frames [160, 193] from step3_attempt1_simulation/ams.rkf
[13.03|12:02:43] Calculating 2 frames in total
[13.03|12:02:43] Running step3_attempt1_reference_calc2
[13.03|12:02:46] Running step3_attempt1_reference_calc3
[13.03|12:02:49] Reference calculations finished!
[13.03|12:02:50] Adding results from step3_attempt1_reference_calc2 to validation set
[13.03|12:02:50] Adding results from step3_attempt1_reference_calc3 to training set
[13.03|12:02:50] Current # training set entries: 56
[13.03|12:02:50] Current # validation set entries: 25
[13.03|12:02:50] Storing data in step3_attempt1_reference_data
[13.03|12:02:50] Deleting step3_attempt1_reference_calc2
[13.03|12:02:50] Deleting step3_attempt1_reference_calc3
[13.03|12:02:50] Launching reparametrization job: step3_attempt1_training
[13.03|12:02:56] JOB m3gnet STARTED
[13.03|12:02:56] Starting m3gnet.prerun()
[13.03|12:02:56] m3gnet.prerun() finished
[13.03|12:02:56] JOB m3gnet RUNNING
[13.03|12:02:56] Executing m3gnet.run
[13.03|12:04:06] training_set Optimizer: 001 Epoch: 0 Loss: 0.002158
[13.03|12:04:06] validation_set Optimizer: 001 Epoch: 0 Loss: 0.016850
[13.03|12:04:33] training_set Optimizer: 001 Epoch: 10 Loss: 0.000086
[13.03|12:04:33] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006704
[13.03|12:05:01] training_set Optimizer: 001 Epoch: 20 Loss: 0.000062
[13.03|12:05:01] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006265
[13.03|12:05:29] training_set Optimizer: 001 Epoch: 30 Loss: 0.000058
[13.03|12:05:29] validation_set Optimizer: 001 Epoch: 30 Loss: 0.006225
[13.03|12:05:57] training_set Optimizer: 001 Epoch: 40 Loss: 0.000057
[13.03|12:05:57] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005607
[13.03|12:06:24] training_set Optimizer: 001 Epoch: 50 Loss: 0.000062
[13.03|12:06:24] validation_set Optimizer: 001 Epoch: 50 Loss: 0.006352
[13.03|12:06:52] training_set Optimizer: 001 Epoch: 60 Loss: 0.000054
[13.03|12:06:52] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007168
[13.03|12:07:20] training_set Optimizer: 001 Epoch: 70 Loss: 0.000060
[13.03|12:07:20] validation_set Optimizer: 001 Epoch: 70 Loss: 0.009132
[13.03|12:07:47] training_set Optimizer: 001 Epoch: 80 Loss: 0.000061
[13.03|12:07:47] validation_set Optimizer: 001 Epoch: 80 Loss: 0.007990
[13.03|12:08:15] training_set Optimizer: 001 Epoch: 90 Loss: 0.000059
[13.03|12:08:15] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005480
[13.03|12:08:42] training_set Optimizer: 001 Epoch: 100 Loss: 0.000060
[13.03|12:08:42] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005820
[13.03|12:09:10] training_set Optimizer: 001 Epoch: 110 Loss: 0.000055
[13.03|12:09:10] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006694
[13.03|12:09:38] training_set Optimizer: 001 Epoch: 120 Loss: 0.000058
[13.03|12:09:38] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005714
[13.03|12:10:06] training_set Optimizer: 001 Epoch: 130 Loss: 0.000058
[13.03|12:10:06] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005704
[13.03|12:10:33] training_set Optimizer: 001 Epoch: 140 Loss: 0.000051
[13.03|12:10:33] validation_set Optimizer: 001 Epoch: 140 Loss: 0.007155
[13.03|12:11:02] training_set Optimizer: 001 Epoch: 150 Loss: 0.000063
[13.03|12:11:02] validation_set Optimizer: 001 Epoch: 150 Loss: 0.008382
[13.03|12:11:29] training_set Optimizer: 001 Epoch: 160 Loss: 0.000060
[13.03|12:11:29] validation_set Optimizer: 001 Epoch: 160 Loss: 0.007330
[13.03|12:11:57] training_set Optimizer: 001 Epoch: 170 Loss: 0.000060
[13.03|12:11:57] validation_set Optimizer: 001 Epoch: 170 Loss: 0.007315
[13.03|12:12:25] training_set Optimizer: 001 Epoch: 180 Loss: 0.000048
[13.03|12:12:25] validation_set Optimizer: 001 Epoch: 180 Loss: 0.008844
[13.03|12:12:53] training_set Optimizer: 001 Epoch: 190 Loss: 0.000063
[13.03|12:12:53] validation_set Optimizer: 001 Epoch: 190 Loss: 0.005901
[13.03|12:13:20] Execution of m3gnet.run finished with returncode 0
[13.03|12:13:20] JOB m3gnet FINISHED
[13.03|12:13:20] Starting m3gnet.postrun()
[13.03|12:13:20] m3gnet.postrun() finished
[13.03|12:13:21] JOB m3gnet SUCCESSFUL
[13.03|12:13:40] Running all jobs through AMS....
[13.03|12:13:41] Storing results/optimization/training_set_results/best
[13.03|12:13:41] Storing results/optimization/validation_set_results/best
[13.03|12:13:41] PLAMS environment cleaned up successfully
[13.03|12:13:41] PLAMS run finished. Goodbye
[13.03|12:13:42] ParAMSResults training_set validation_set
[13.03|12:13:42] energy MAE 0.0853 0.1274 eV
[13.03|12:13:42] forces MAE 0.0168 0.0494 eV/angstrom
[13.03|12:13:42] Newly created parameter file/dir: step3_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|12:13:42] Done!
[13.03|12:13:42] Deleting step2_attempt4_training
[13.03|12:13:42] ##########################
[13.03|12:13:42] ### Step 3 / Attempt 2 ###
[13.03|12:13:42] ##########################
[13.03|12:13:42] MD Steps: 2000 (cumulative: 4500)
[13.03|12:13:42] Current engine settings:
[13.03|12:13:42]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step3_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:13:42] Running step3_attempt2_simulation...
[13.03|12:15:17] Job step3_attempt2_simulation finished
[13.03|12:15:17] Deleting files that are no longer needed...
[13.03|12:15:18] Launching reference calculation
[13.03|12:15:21] Reference calculation finished!
[13.03|12:15:21] Checking success for step3_attempt2
[13.03|12:15:33] CheckEnergy: Checking energy for MDStep4500, n_atoms = 46
[13.03|12:15:33] CheckEnergy: normalization coefficient = 46
[13.03|12:15:33] CheckEnergy: Actual Threshold
[13.03|12:15:33] CheckEnergy: dE/46 -0.0009 0.0100 OK!
[13.03|12:15:33] CheckEnergy: ddE/46 0.0011 0.0020 OK! (relative to step3_attempt1_simulation:MDStep4500)
[13.03|12:15:33]
[13.03|12:15:33] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:15:33] CheckForces: ------------
[13.03|12:15:33] CheckForces: Reference job from step3_attempt2_reference_calc1
[13.03|12:15:33] CheckForces: Prediction job from final frame (MDStep4500) of step3_attempt2_simulation
[13.03|12:15:33] CheckForces: ------------
[13.03|12:15:33] CheckForces: Histogram of forces
[13.03|12:15:33] CheckForces: eV/Ang Ref Pred
[13.03|12:15:33] CheckForces: -2 1 1
[13.03|12:15:33] CheckForces: -1 73 72
[13.03|12:15:33] CheckForces: 0 63 64
[13.03|12:15:33] CheckForces: 1 1 1
[13.03|12:15:33] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:15:33] CheckForces: All force components are within the acceptable error!
[13.03|12:15:33] CheckForces: Maximum deviation: 0.299 eV/angstrom
[13.03|12:15:33] CheckForces: Actual Threshold
[13.03|12:15:33] CheckForces: # > thr. 0 0 OK!
[13.03|12:15:33] CheckForces: MAE 0.040 0.30 OK!
[13.03|12:15:33] CheckForces: R^2 0.941 0.80 OK!
[13.03|12:15:33] CheckForces: --------------------
[13.03|12:15:33]
[13.03|12:15:34] Adding results from step3_attempt2_reference_calc1 to validation set
[13.03|12:15:34] Current # training set entries: 56
[13.03|12:15:34] Current # validation set entries: 26
[13.03|12:15:34] Storing data in step3_attempt2_reference_data
[13.03|12:15:34] Deleting step3_attempt1_reference_data
[13.03|12:15:34] Deleting step3_attempt2_reference_calc1
[13.03|12:15:34]
[13.03|12:15:34] Current (cumulative) timings:
[13.03|12:15:34] Time (s) Fraction
[13.03|12:15:34] Ref. calcs 90.81 0.015
[13.03|12:15:34] ML training 5016.40 0.839
[13.03|12:15:34] Simulations 871.74 0.146
[13.03|12:15:34]
[13.03|12:15:34]
[13.03|12:15:35] Step 3 finished successfully!
[13.03|12:15:35]
[13.03|12:15:35] --- Begin summary ---
[13.03|12:15:35] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:15:35] 1 1 FAILED Inaccurate 0.4793
[13.03|12:15:35] 1 2 FAILED Inaccurate 0.4059
[13.03|12:15:35] 1 3 FAILED Inaccurate 0.4848
[13.03|12:15:35] 1 4 FAILED Inaccurate 0.6954
[13.03|12:15:35] 1 5 SUCCESS Accurate 0.2303
[13.03|12:15:35] 2 1 FAILED Inaccurate 0.6081
[13.03|12:15:35] 2 2 FAILED Inaccurate 0.6387
[13.03|12:15:35] 2 3 FAILED Inaccurate 0.4972
[13.03|12:15:35] 2 4 FAILED Inaccurate 0.4115
[13.03|12:15:35] 2 5 SUCCESS Accurate 0.2071
[13.03|12:15:35] 3 1 FAILED Inaccurate 0.6595
[13.03|12:15:35] 3 2 SUCCESS Accurate 0.2988
[13.03|12:15:35] --- End summary ---
[13.03|12:15:35]
[13.03|12:15:35] ##########################
[13.03|12:15:35] ### Step 4 / Attempt 1 ###
[13.03|12:15:35] ##########################
[13.03|12:15:35] MD Steps: 2000 (cumulative: 6500)
[13.03|12:15:35] Current engine settings:
[13.03|12:15:35]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step3_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:15:35] Running step4_attempt1_simulation...
[13.03|12:17:10] Job step4_attempt1_simulation finished
[13.03|12:17:10] Deleting files that are no longer needed...
[13.03|12:17:10] Deleting step2_attempt5_simulation
[13.03|12:17:10] Deleting step3_attempt1_simulation
[13.03|12:17:11] Launching reference calculation
[13.03|12:17:14] Reference calculation finished!
[13.03|12:17:14] Checking success for step4_attempt1
[13.03|12:17:25] CheckEnergy: Checking energy for MDStep6500, n_atoms = 46
[13.03|12:17:25] CheckEnergy: normalization coefficient = 46
[13.03|12:17:25] CheckEnergy: Actual Threshold
[13.03|12:17:25] CheckEnergy: dE/46 -0.0030 0.0100 OK!
[13.03|12:17:25] CheckEnergy: ddE/46 -0.0020 0.0020 Not OK! (relative to step3_attempt2_simulation:MDStep4500)
[13.03|12:17:25]
[13.03|12:17:25] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:17:25] CheckForces: ------------
[13.03|12:17:25] CheckForces: Reference job from step4_attempt1_reference_calc1
[13.03|12:17:25] CheckForces: Prediction job from final frame (MDStep6500) of step4_attempt1_simulation
[13.03|12:17:25] CheckForces: ------------
[13.03|12:17:25] CheckForces: Histogram of forces
[13.03|12:17:25] CheckForces: eV/Ang Ref Pred
[13.03|12:17:25] CheckForces: -2 0 0
[13.03|12:17:25] CheckForces: -1 70 72
[13.03|12:17:25] CheckForces: 0 68 66
[13.03|12:17:25] CheckForces: 1 0 0
[13.03|12:17:25] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:17:25] CheckForces: Force components with an error exceeding the threshold:
[13.03|12:17:25] CheckForces: Ref Pred Delta Threshold
[13.03|12:17:25] CheckForces: -0.28 -0.75 0.47 0.31
[13.03|12:17:25] CheckForces: -0.26 0.12 0.38 0.31
[13.03|12:17:25] CheckForces: Maximum deviation: 0.469 eV/angstrom
[13.03|12:17:25] CheckForces: Actual Threshold
[13.03|12:17:25] CheckForces: # > thr. 2 0 Not OK!
[13.03|12:17:25] CheckForces: MAE 0.058 0.30 OK!
[13.03|12:17:25] CheckForces: R^2 0.825 0.80 OK!
[13.03|12:17:25] CheckForces: --------------------
[13.03|12:17:25]
[13.03|12:17:25] Adding results from step4_attempt1_reference_calc1 to training set
[13.03|12:17:25] Current # training set entries: 57
[13.03|12:17:25] Current # validation set entries: 26
[13.03|12:17:25] Storing data in step4_attempt1_reference_data
[13.03|12:17:26] Deleting step3_attempt2_reference_data
[13.03|12:17:26] Deleting step4_attempt1_reference_calc1
[13.03|12:17:26]
[13.03|12:17:26] Current (cumulative) timings:
[13.03|12:17:26] Time (s) Fraction
[13.03|12:17:26] Ref. calcs 93.73 0.015
[13.03|12:17:26] ML training 5016.40 0.825
[13.03|12:17:26] Simulations 967.44 0.159
[13.03|12:17:26]
[13.03|12:17:26]
[13.03|12:17:26]
[13.03|12:17:26] --- Begin summary ---
[13.03|12:17:26] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:17:26] 1 1 FAILED Inaccurate 0.4793
[13.03|12:17:26] 1 2 FAILED Inaccurate 0.4059
[13.03|12:17:26] 1 3 FAILED Inaccurate 0.4848
[13.03|12:17:26] 1 4 FAILED Inaccurate 0.6954
[13.03|12:17:26] 1 5 SUCCESS Accurate 0.2303
[13.03|12:17:26] 2 1 FAILED Inaccurate 0.6081
[13.03|12:17:26] 2 2 FAILED Inaccurate 0.6387
[13.03|12:17:26] 2 3 FAILED Inaccurate 0.4972
[13.03|12:17:26] 2 4 FAILED Inaccurate 0.4115
[13.03|12:17:26] 2 5 SUCCESS Accurate 0.2071
[13.03|12:17:26] 3 1 FAILED Inaccurate 0.6595
[13.03|12:17:26] 3 2 SUCCESS Accurate 0.2988
[13.03|12:17:26] 4 1 FAILED Inaccurate 0.4693
[13.03|12:17:26] --- End summary ---
[13.03|12:17:26]
[13.03|12:17:26] Running more reference calculations....
[13.03|12:17:26] Running reference calculations on frames [260, 293] from step4_attempt1_simulation/ams.rkf
[13.03|12:17:26] Calculating 2 frames in total
[13.03|12:17:26] Running step4_attempt1_reference_calc2
[13.03|12:17:29] Running step4_attempt1_reference_calc3
[13.03|12:17:32] Reference calculations finished!
[13.03|12:17:33] Adding results from step4_attempt1_reference_calc2 to validation set
[13.03|12:17:33] Adding results from step4_attempt1_reference_calc3 to training set
[13.03|12:17:33] Current # training set entries: 58
[13.03|12:17:33] Current # validation set entries: 27
[13.03|12:17:33] Storing data in step4_attempt1_reference_data
[13.03|12:17:33] Deleting step4_attempt1_reference_calc2
[13.03|12:17:33] Deleting step4_attempt1_reference_calc3
[13.03|12:17:33] Launching reparametrization job: step4_attempt1_training
[13.03|12:17:39] JOB m3gnet STARTED
[13.03|12:17:39] Starting m3gnet.prerun()
[13.03|12:17:39] m3gnet.prerun() finished
[13.03|12:17:39] JOB m3gnet RUNNING
[13.03|12:17:39] Executing m3gnet.run
[13.03|12:18:49] training_set Optimizer: 001 Epoch: 0 Loss: 0.001849
[13.03|12:18:49] validation_set Optimizer: 001 Epoch: 0 Loss: 0.020577
[13.03|12:19:18] training_set Optimizer: 001 Epoch: 10 Loss: 0.000057
[13.03|12:19:18] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005314
[13.03|12:19:47] training_set Optimizer: 001 Epoch: 20 Loss: 0.000052
[13.03|12:19:47] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005396
[13.03|12:20:15] training_set Optimizer: 001 Epoch: 30 Loss: 0.000051
[13.03|12:20:15] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005385
[13.03|12:20:44] training_set Optimizer: 001 Epoch: 40 Loss: 0.000051
[13.03|12:20:44] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005958
[13.03|12:21:13] training_set Optimizer: 001 Epoch: 50 Loss: 0.000052
[13.03|12:21:13] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005630
[13.03|12:21:42] training_set Optimizer: 001 Epoch: 60 Loss: 0.000054
[13.03|12:21:42] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005948
[13.03|12:22:11] training_set Optimizer: 001 Epoch: 70 Loss: 0.000049
[13.03|12:22:11] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005465
[13.03|12:22:40] training_set Optimizer: 001 Epoch: 80 Loss: 0.000047
[13.03|12:22:40] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005966
[13.03|12:23:08] training_set Optimizer: 001 Epoch: 90 Loss: 0.000052
[13.03|12:23:08] validation_set Optimizer: 001 Epoch: 90 Loss: 0.006403
[13.03|12:23:37] training_set Optimizer: 001 Epoch: 100 Loss: 0.000054
[13.03|12:23:37] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005351
[13.03|12:24:05] training_set Optimizer: 001 Epoch: 110 Loss: 0.000052
[13.03|12:24:05] validation_set Optimizer: 001 Epoch: 110 Loss: 0.007148
[13.03|12:24:34] training_set Optimizer: 001 Epoch: 120 Loss: 0.000049
[13.03|12:24:34] validation_set Optimizer: 001 Epoch: 120 Loss: 0.006739
[13.03|12:25:03] training_set Optimizer: 001 Epoch: 130 Loss: 0.000057
[13.03|12:25:03] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005777
[13.03|12:25:32] training_set Optimizer: 001 Epoch: 140 Loss: 0.000046
[13.03|12:25:32] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005515
[13.03|12:26:01] training_set Optimizer: 001 Epoch: 150 Loss: 0.000051
[13.03|12:26:01] validation_set Optimizer: 001 Epoch: 150 Loss: 0.006542
[13.03|12:26:30] training_set Optimizer: 001 Epoch: 160 Loss: 0.000045
[13.03|12:26:30] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006558
[13.03|12:26:59] training_set Optimizer: 001 Epoch: 170 Loss: 0.000048
[13.03|12:26:59] validation_set Optimizer: 001 Epoch: 170 Loss: 0.005352
[13.03|12:27:28] training_set Optimizer: 001 Epoch: 180 Loss: 0.000049
[13.03|12:27:28] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005430
[13.03|12:27:56] training_set Optimizer: 001 Epoch: 190 Loss: 0.000047
[13.03|12:27:56] validation_set Optimizer: 001 Epoch: 190 Loss: 0.008168
[13.03|12:28:26] Execution of m3gnet.run finished with returncode 0
[13.03|12:28:26] JOB m3gnet FINISHED
[13.03|12:28:26] Starting m3gnet.postrun()
[13.03|12:28:26] m3gnet.postrun() finished
[13.03|12:28:27] JOB m3gnet SUCCESSFUL
[13.03|12:28:46] Running all jobs through AMS....
[13.03|12:28:47] Storing results/optimization/training_set_results/best
[13.03|12:28:47] Storing results/optimization/validation_set_results/best
[13.03|12:28:47] PLAMS environment cleaned up successfully
[13.03|12:28:47] PLAMS run finished. Goodbye
[13.03|12:28:49] ParAMSResults training_set validation_set
[13.03|12:28:49] energy MAE 0.0411 0.0835 eV
[13.03|12:28:49] forces MAE 0.0164 0.0487 eV/angstrom
[13.03|12:28:49] Newly created parameter file/dir: step4_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|12:28:49] Done!
[13.03|12:28:49] Deleting step3_attempt1_training
[13.03|12:28:49] ##########################
[13.03|12:28:49] ### Step 4 / Attempt 2 ###
[13.03|12:28:49] ##########################
[13.03|12:28:49] MD Steps: 2000 (cumulative: 6500)
[13.03|12:28:49] Current engine settings:
[13.03|12:28:49]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:28:49] Running step4_attempt2_simulation...
[13.03|12:30:25] Job step4_attempt2_simulation finished
[13.03|12:30:25] Deleting files that are no longer needed...
[13.03|12:30:26] Launching reference calculation
[13.03|12:30:29] Reference calculation finished!
[13.03|12:30:29] Checking success for step4_attempt2
[13.03|12:30:40] CheckEnergy: Checking energy for MDStep6500, n_atoms = 46
[13.03|12:30:40] CheckEnergy: normalization coefficient = 46
[13.03|12:30:40] CheckEnergy: Actual Threshold
[13.03|12:30:40] CheckEnergy: dE/46 -0.0027 0.0100 OK!
[13.03|12:30:40] CheckEnergy: ddE/46 -0.0007 0.0020 OK! (relative to step4_attempt1_simulation:MDStep6500)
[13.03|12:30:40]
[13.03|12:30:40] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:30:40] CheckForces: ------------
[13.03|12:30:40] CheckForces: Reference job from step4_attempt2_reference_calc1
[13.03|12:30:40] CheckForces: Prediction job from final frame (MDStep6500) of step4_attempt2_simulation
[13.03|12:30:40] CheckForces: ------------
[13.03|12:30:40] CheckForces: Histogram of forces
[13.03|12:30:40] CheckForces: eV/Ang Ref Pred
[13.03|12:30:40] CheckForces: -2 0 0
[13.03|12:30:40] CheckForces: -1 67 64
[13.03|12:30:40] CheckForces: 0 71 74
[13.03|12:30:40] CheckForces: 1 0 0
[13.03|12:30:40] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:30:40] CheckForces: Force components with an error exceeding the threshold:
[13.03|12:30:40] CheckForces: Ref Pred Delta Threshold
[13.03|12:30:40] CheckForces: -0.02 -0.40 0.39 0.30
[13.03|12:30:40] CheckForces: -0.28 0.06 0.33 0.31
[13.03|12:30:40] CheckForces: Maximum deviation: 0.385 eV/angstrom
[13.03|12:30:40] CheckForces: Actual Threshold
[13.03|12:30:40] CheckForces: # > thr. 2 0 Not OK!
[13.03|12:30:40] CheckForces: MAE 0.052 0.30 OK!
[13.03|12:30:40] CheckForces: R^2 0.863 0.80 OK!
[13.03|12:30:40] CheckForces: --------------------
[13.03|12:30:40]
[13.03|12:30:41] Adding results from step4_attempt2_reference_calc1 to training set
[13.03|12:30:41] Current # training set entries: 59
[13.03|12:30:41] Current # validation set entries: 27
[13.03|12:30:41] Storing data in step4_attempt2_reference_data
[13.03|12:30:41] Deleting step4_attempt1_reference_data
[13.03|12:30:41] Deleting step4_attempt2_reference_calc1
[13.03|12:30:41]
[13.03|12:30:41] Current (cumulative) timings:
[13.03|12:30:41] Time (s) Fraction
[13.03|12:30:41] Ref. calcs 103.06 0.015
[13.03|12:30:41] ML training 5691.79 0.830
[13.03|12:30:41] Simulations 1064.00 0.155
[13.03|12:30:41]
[13.03|12:30:41]
[13.03|12:30:41]
[13.03|12:30:41] --- Begin summary ---
[13.03|12:30:41] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:30:41] 1 1 FAILED Inaccurate 0.4793
[13.03|12:30:41] 1 2 FAILED Inaccurate 0.4059
[13.03|12:30:41] 1 3 FAILED Inaccurate 0.4848
[13.03|12:30:41] 1 4 FAILED Inaccurate 0.6954
[13.03|12:30:41] 1 5 SUCCESS Accurate 0.2303
[13.03|12:30:41] 2 1 FAILED Inaccurate 0.6081
[13.03|12:30:41] 2 2 FAILED Inaccurate 0.6387
[13.03|12:30:41] 2 3 FAILED Inaccurate 0.4972
[13.03|12:30:41] 2 4 FAILED Inaccurate 0.4115
[13.03|12:30:41] 2 5 SUCCESS Accurate 0.2071
[13.03|12:30:41] 3 1 FAILED Inaccurate 0.6595
[13.03|12:30:41] 3 2 SUCCESS Accurate 0.2988
[13.03|12:30:41] 4 1 FAILED Inaccurate 0.4693
[13.03|12:30:41] 4 2 FAILED Inaccurate 0.3853
[13.03|12:30:41] --- End summary ---
[13.03|12:30:41]
[13.03|12:30:41] Running more reference calculations....
[13.03|12:30:42] Running reference calculations on frames [260, 293] from step4_attempt2_simulation/ams.rkf
[13.03|12:30:42] Calculating 2 frames in total
[13.03|12:30:42] Running step4_attempt2_reference_calc2
[13.03|12:30:45] Running step4_attempt2_reference_calc3
[13.03|12:30:48] Reference calculations finished!
[13.03|12:30:48] Adding results from step4_attempt2_reference_calc2 to validation set
[13.03|12:30:48] Adding results from step4_attempt2_reference_calc3 to training set
[13.03|12:30:48] Current # training set entries: 60
[13.03|12:30:48] Current # validation set entries: 28
[13.03|12:30:48] Storing data in step4_attempt2_reference_data
[13.03|12:30:49] Deleting step4_attempt2_reference_calc2
[13.03|12:30:49] Deleting step4_attempt2_reference_calc3
[13.03|12:30:49] Launching reparametrization job: step4_attempt2_training
[13.03|12:30:54] JOB m3gnet STARTED
[13.03|12:30:54] Starting m3gnet.prerun()
[13.03|12:30:54] m3gnet.prerun() finished
[13.03|12:30:54] JOB m3gnet RUNNING
[13.03|12:30:54] Executing m3gnet.run
[13.03|12:31:46] training_set Optimizer: 001 Epoch: 0 Loss: 0.001254
[13.03|12:31:46] validation_set Optimizer: 001 Epoch: 0 Loss: 0.023779
[13.03|12:32:15] training_set Optimizer: 001 Epoch: 10 Loss: 0.000049
[13.03|12:32:15] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005230
[13.03|12:32:44] training_set Optimizer: 001 Epoch: 20 Loss: 0.000046
[13.03|12:32:44] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005483
[13.03|12:33:13] training_set Optimizer: 001 Epoch: 30 Loss: 0.000046
[13.03|12:33:13] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005236
[13.03|12:33:42] training_set Optimizer: 001 Epoch: 40 Loss: 0.000046
[13.03|12:33:42] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005648
[13.03|12:34:11] training_set Optimizer: 001 Epoch: 50 Loss: 0.000049
[13.03|12:34:11] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005498
[13.03|12:34:40] training_set Optimizer: 001 Epoch: 60 Loss: 0.000046
[13.03|12:34:40] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006258
[13.03|12:35:09] training_set Optimizer: 001 Epoch: 70 Loss: 0.000046
[13.03|12:35:09] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005777
[13.03|12:35:38] training_set Optimizer: 001 Epoch: 80 Loss: 0.000047
[13.03|12:35:38] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005622
[13.03|12:36:07] training_set Optimizer: 001 Epoch: 90 Loss: 0.000048
[13.03|12:36:07] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005594
[13.03|12:36:36] training_set Optimizer: 001 Epoch: 100 Loss: 0.000060
[13.03|12:36:36] validation_set Optimizer: 001 Epoch: 100 Loss: 0.010379
[13.03|12:37:05] training_set Optimizer: 001 Epoch: 110 Loss: 0.000047
[13.03|12:37:05] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006525
[13.03|12:37:34] training_set Optimizer: 001 Epoch: 120 Loss: 0.000044
[13.03|12:37:34] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005376
[13.03|12:38:03] training_set Optimizer: 001 Epoch: 130 Loss: 0.000059
[13.03|12:38:03] validation_set Optimizer: 001 Epoch: 130 Loss: 0.007216
[13.03|12:38:32] training_set Optimizer: 001 Epoch: 140 Loss: 0.000049
[13.03|12:38:32] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006930
[13.03|12:39:01] training_set Optimizer: 001 Epoch: 150 Loss: 0.000050
[13.03|12:39:01] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005578
[13.03|12:39:30] training_set Optimizer: 001 Epoch: 160 Loss: 0.000049
[13.03|12:39:30] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005478
[13.03|12:39:59] training_set Optimizer: 001 Epoch: 170 Loss: 0.000047
[13.03|12:39:59] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006074
[13.03|12:40:28] training_set Optimizer: 001 Epoch: 180 Loss: 0.000049
[13.03|12:40:28] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005392
[13.03|12:40:57] training_set Optimizer: 001 Epoch: 190 Loss: 0.000041
[13.03|12:40:57] validation_set Optimizer: 001 Epoch: 190 Loss: 0.009123
[13.03|12:41:25] Execution of m3gnet.run finished with returncode 0
[13.03|12:41:25] JOB m3gnet FINISHED
[13.03|12:41:25] Starting m3gnet.postrun()
[13.03|12:41:25] m3gnet.postrun() finished
[13.03|12:41:26] JOB m3gnet SUCCESSFUL
[13.03|12:41:45] Running all jobs through AMS....
[13.03|12:41:46] Storing results/optimization/training_set_results/best
[13.03|12:41:46] Storing results/optimization/validation_set_results/best
[13.03|12:41:46] PLAMS environment cleaned up successfully
[13.03|12:41:46] PLAMS run finished. Goodbye
[13.03|12:41:48] ParAMSResults training_set validation_set
[13.03|12:41:48] energy MAE 0.1240 0.1604 eV
[13.03|12:41:48] forces MAE 0.0151 0.0480 eV/angstrom
[13.03|12:41:48] Newly created parameter file/dir: step4_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|12:41:48] Done!
[13.03|12:41:48] Deleting step4_attempt1_training
[13.03|12:41:48] ##########################
[13.03|12:41:48] ### Step 4 / Attempt 3 ###
[13.03|12:41:48] ##########################
[13.03|12:41:48] MD Steps: 2000 (cumulative: 6500)
[13.03|12:41:48] Current engine settings:
[13.03|12:41:48]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:41:48] Running step4_attempt3_simulation...
[13.03|12:43:27] Job step4_attempt3_simulation finished
[13.03|12:43:27] Deleting files that are no longer needed...
[13.03|12:43:27] Launching reference calculation
[13.03|12:43:30] Reference calculation finished!
[13.03|12:43:30] Checking success for step4_attempt3
[13.03|12:43:42] CheckEnergy: Checking energy for MDStep6500, n_atoms = 46
[13.03|12:43:42] CheckEnergy: normalization coefficient = 46
[13.03|12:43:42] CheckEnergy: Actual Threshold
[13.03|12:43:42] CheckEnergy: dE/46 -0.0090 0.0100 OK!
[13.03|12:43:42] CheckEnergy: ddE/46 -0.0061 0.0020 Not OK! (relative to step4_attempt2_simulation:MDStep6500)
[13.03|12:43:42]
[13.03|12:43:42] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:43:42] CheckForces: ------------
[13.03|12:43:42] CheckForces: Reference job from step4_attempt3_reference_calc1
[13.03|12:43:42] CheckForces: Prediction job from final frame (MDStep6500) of step4_attempt3_simulation
[13.03|12:43:42] CheckForces: ------------
[13.03|12:43:42] CheckForces: Histogram of forces
[13.03|12:43:42] CheckForces: eV/Ang Ref Pred
[13.03|12:43:42] CheckForces: -2 0 0
[13.03|12:43:42] CheckForces: -1 73 70
[13.03|12:43:42] CheckForces: 0 63 66
[13.03|12:43:42] CheckForces: 1 1 2
[13.03|12:43:42] CheckForces: 2 1 0
[13.03|12:43:42] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:43:42] CheckForces: Force components with an error exceeding the threshold:
[13.03|12:43:42] CheckForces: Ref Pred Delta Threshold
[13.03|12:43:42] CheckForces: 1.04 0.39 0.65 0.36
[13.03|12:43:42] CheckForces: 2.03 1.12 0.91 0.44
[13.03|12:43:42] CheckForces: -0.57 -0.24 0.33 0.33
[13.03|12:43:42] CheckForces: -0.16 0.22 0.39 0.31
[13.03|12:43:42] CheckForces: Maximum deviation: 0.912 eV/angstrom
[13.03|12:43:42] CheckForces: Actual Threshold
[13.03|12:43:42] CheckForces: # > thr. 4 0 Not OK!
[13.03|12:43:42] CheckForces: MAE 0.076 0.30 OK!
[13.03|12:43:42] CheckForces: R^2 0.855 0.80 OK!
[13.03|12:43:42] CheckForces: --------------------
[13.03|12:43:42]
[13.03|12:43:43] Adding results from step4_attempt3_reference_calc1 to training set
[13.03|12:43:43] Current # training set entries: 61
[13.03|12:43:43] Current # validation set entries: 28
[13.03|12:43:43] Storing data in step4_attempt3_reference_data
[13.03|12:43:43] Deleting step4_attempt2_reference_data
[13.03|12:43:43] Deleting step4_attempt3_reference_calc1
[13.03|12:43:43]
[13.03|12:43:43] Current (cumulative) timings:
[13.03|12:43:43] Time (s) Fraction
[13.03|12:43:43] Ref. calcs 112.20 0.015
[13.03|12:43:43] ML training 6350.91 0.833
[13.03|12:43:43] Simulations 1162.91 0.152
[13.03|12:43:43]
[13.03|12:43:43]
[13.03|12:43:44]
[13.03|12:43:44] --- Begin summary ---
[13.03|12:43:44] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:43:44] 1 1 FAILED Inaccurate 0.4793
[13.03|12:43:44] 1 2 FAILED Inaccurate 0.4059
[13.03|12:43:44] 1 3 FAILED Inaccurate 0.4848
[13.03|12:43:44] 1 4 FAILED Inaccurate 0.6954
[13.03|12:43:44] 1 5 SUCCESS Accurate 0.2303
[13.03|12:43:44] 2 1 FAILED Inaccurate 0.6081
[13.03|12:43:44] 2 2 FAILED Inaccurate 0.6387
[13.03|12:43:44] 2 3 FAILED Inaccurate 0.4972
[13.03|12:43:44] 2 4 FAILED Inaccurate 0.4115
[13.03|12:43:44] 2 5 SUCCESS Accurate 0.2071
[13.03|12:43:44] 3 1 FAILED Inaccurate 0.6595
[13.03|12:43:44] 3 2 SUCCESS Accurate 0.2988
[13.03|12:43:44] 4 1 FAILED Inaccurate 0.4693
[13.03|12:43:44] 4 2 FAILED Inaccurate 0.3853
[13.03|12:43:44] 4 3 FAILED Inaccurate 0.9123
[13.03|12:43:44] --- End summary ---
[13.03|12:43:44]
[13.03|12:43:44] Running more reference calculations....
[13.03|12:43:44] Running reference calculations on frames [260, 293] from step4_attempt3_simulation/ams.rkf
[13.03|12:43:44] Calculating 2 frames in total
[13.03|12:43:44] Running step4_attempt3_reference_calc2
[13.03|12:43:47] Running step4_attempt3_reference_calc3
[13.03|12:43:50] Reference calculations finished!
[13.03|12:43:50] Adding results from step4_attempt3_reference_calc2 to validation set
[13.03|12:43:50] Adding results from step4_attempt3_reference_calc3 to training set
[13.03|12:43:50] Current # training set entries: 62
[13.03|12:43:50] Current # validation set entries: 29
[13.03|12:43:50] Storing data in step4_attempt3_reference_data
[13.03|12:43:51] Deleting step4_attempt3_reference_calc2
[13.03|12:43:51] Deleting step4_attempt3_reference_calc3
[13.03|12:43:51] Launching reparametrization job: step4_attempt3_training
[13.03|12:43:57] JOB m3gnet STARTED
[13.03|12:43:57] Starting m3gnet.prerun()
[13.03|12:43:57] m3gnet.prerun() finished
[13.03|12:43:57] JOB m3gnet RUNNING
[13.03|12:43:57] Executing m3gnet.run
[13.03|12:45:06] training_set Optimizer: 001 Epoch: 0 Loss: 0.001752
[13.03|12:45:06] validation_set Optimizer: 001 Epoch: 0 Loss: 0.028965
[13.03|12:45:37] training_set Optimizer: 001 Epoch: 10 Loss: 0.000062
[13.03|12:45:37] validation_set Optimizer: 001 Epoch: 10 Loss: 0.006170
[13.03|12:46:09] training_set Optimizer: 001 Epoch: 20 Loss: 0.000049
[13.03|12:46:09] validation_set Optimizer: 001 Epoch: 20 Loss: 0.006187
[13.03|12:46:39] training_set Optimizer: 001 Epoch: 30 Loss: 0.000046
[13.03|12:46:39] validation_set Optimizer: 001 Epoch: 30 Loss: 0.007380
[13.03|12:47:10] training_set Optimizer: 001 Epoch: 40 Loss: 0.000046
[13.03|12:47:10] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005356
[13.03|12:47:41] training_set Optimizer: 001 Epoch: 50 Loss: 0.000044
[13.03|12:47:41] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005352
[13.03|12:48:13] training_set Optimizer: 001 Epoch: 60 Loss: 0.000045
[13.03|12:48:13] validation_set Optimizer: 001 Epoch: 60 Loss: 0.007584
[13.03|12:48:43] training_set Optimizer: 001 Epoch: 70 Loss: 0.000044
[13.03|12:48:43] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005838
[13.03|12:49:14] training_set Optimizer: 001 Epoch: 80 Loss: 0.000046
[13.03|12:49:14] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005287
[13.03|12:49:45] training_set Optimizer: 001 Epoch: 90 Loss: 0.000045
[13.03|12:49:45] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005466
[13.03|12:50:16] training_set Optimizer: 001 Epoch: 100 Loss: 0.000045
[13.03|12:50:16] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005811
[13.03|12:50:47] training_set Optimizer: 001 Epoch: 110 Loss: 0.000050
[13.03|12:50:47] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005336
[13.03|12:51:18] training_set Optimizer: 001 Epoch: 120 Loss: 0.000050
[13.03|12:51:18] validation_set Optimizer: 001 Epoch: 120 Loss: 0.008629
[13.03|12:51:49] training_set Optimizer: 001 Epoch: 130 Loss: 0.000051
[13.03|12:51:49] validation_set Optimizer: 001 Epoch: 130 Loss: 0.006561
[13.03|12:52:20] training_set Optimizer: 001 Epoch: 140 Loss: 0.000054
[13.03|12:52:20] validation_set Optimizer: 001 Epoch: 140 Loss: 0.008225
[13.03|12:52:50] training_set Optimizer: 001 Epoch: 150 Loss: 0.000043
[13.03|12:52:50] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005473
[13.03|12:53:20] training_set Optimizer: 001 Epoch: 160 Loss: 0.000049
[13.03|12:53:20] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005419
[13.03|12:53:51] training_set Optimizer: 001 Epoch: 170 Loss: 0.000044
[13.03|12:53:51] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006144
[13.03|12:54:22] training_set Optimizer: 001 Epoch: 180 Loss: 0.000054
[13.03|12:54:22] validation_set Optimizer: 001 Epoch: 180 Loss: 0.007529
[13.03|12:54:53] training_set Optimizer: 001 Epoch: 190 Loss: 0.000040
[13.03|12:54:53] validation_set Optimizer: 001 Epoch: 190 Loss: 0.005503
[13.03|12:55:23] Execution of m3gnet.run finished with returncode 0
[13.03|12:55:23] JOB m3gnet FINISHED
[13.03|12:55:23] Starting m3gnet.postrun()
[13.03|12:55:23] m3gnet.postrun() finished
[13.03|12:55:24] JOB m3gnet SUCCESSFUL
[13.03|12:55:44] Running all jobs through AMS....
[13.03|12:55:45] Storing results/optimization/training_set_results/best
[13.03|12:55:45] Storing results/optimization/validation_set_results/best
[13.03|12:55:45] PLAMS environment cleaned up successfully
[13.03|12:55:45] PLAMS run finished. Goodbye
[13.03|12:55:46] ParAMSResults training_set validation_set
[13.03|12:55:46] energy MAE 0.1469 0.1118 eV
[13.03|12:55:46] forces MAE 0.0155 0.0478 eV/angstrom
[13.03|12:55:46] Newly created parameter file/dir: step4_attempt3_training/results/optimization/m3gnet/m3gnet
[13.03|12:55:46] Done!
[13.03|12:55:46] Deleting step4_attempt2_training
[13.03|12:55:46] ##########################
[13.03|12:55:46] ### Step 4 / Attempt 4 ###
[13.03|12:55:46] ##########################
[13.03|12:55:46] MD Steps: 2000 (cumulative: 6500)
[13.03|12:55:46] Current engine settings:
[13.03|12:55:46]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:55:46] Running step4_attempt4_simulation...
[13.03|12:57:24] Job step4_attempt4_simulation finished
[13.03|12:57:24] Deleting files that are no longer needed...
[13.03|12:57:25] Launching reference calculation
[13.03|12:57:28] Reference calculation finished!
[13.03|12:57:28] Checking success for step4_attempt4
[13.03|12:57:40] CheckEnergy: Checking energy for MDStep6500, n_atoms = 46
[13.03|12:57:40] CheckEnergy: normalization coefficient = 46
[13.03|12:57:40] CheckEnergy: Actual Threshold
[13.03|12:57:40] CheckEnergy: dE/46 0.0028 0.0100 OK!
[13.03|12:57:40] CheckEnergy: ddE/46 -0.0004 0.0020 OK! (relative to step4_attempt3_simulation:MDStep6500)
[13.03|12:57:40]
[13.03|12:57:40] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:57:40] CheckForces: ------------
[13.03|12:57:40] CheckForces: Reference job from step4_attempt4_reference_calc1
[13.03|12:57:40] CheckForces: Prediction job from final frame (MDStep6500) of step4_attempt4_simulation
[13.03|12:57:40] CheckForces: ------------
[13.03|12:57:40] CheckForces: Histogram of forces
[13.03|12:57:40] CheckForces: eV/Ang Ref Pred
[13.03|12:57:40] CheckForces: -2 1 0
[13.03|12:57:40] CheckForces: -1 63 70
[13.03|12:57:40] CheckForces: 0 73 66
[13.03|12:57:40] CheckForces: 1 1 2
[13.03|12:57:40] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:57:40] CheckForces: All force components are within the acceptable error!
[13.03|12:57:40] CheckForces: Maximum deviation: 0.206 eV/angstrom
[13.03|12:57:40] CheckForces: Actual Threshold
[13.03|12:57:40] CheckForces: # > thr. 0 0 OK!
[13.03|12:57:40] CheckForces: MAE 0.044 0.30 OK!
[13.03|12:57:40] CheckForces: R^2 0.935 0.80 OK!
[13.03|12:57:40] CheckForces: --------------------
[13.03|12:57:40]
[13.03|12:57:41] Adding results from step4_attempt4_reference_calc1 to training set
[13.03|12:57:41] Current # training set entries: 63
[13.03|12:57:41] Current # validation set entries: 29
[13.03|12:57:41] Storing data in step4_attempt4_reference_data
[13.03|12:57:41] Deleting step4_attempt3_reference_data
[13.03|12:57:41] Deleting step4_attempt4_reference_calc1
[13.03|12:57:41]
[13.03|12:57:41] Current (cumulative) timings:
[13.03|12:57:41] Time (s) Fraction
[13.03|12:57:41] Ref. calcs 121.31 0.014
[13.03|12:57:41] ML training 7066.45 0.836
[13.03|12:57:41] Simulations 1260.58 0.149
[13.03|12:57:41]
[13.03|12:57:41]
[13.03|12:57:41] Step 4 finished successfully!
[13.03|12:57:41]
[13.03|12:57:41] --- Begin summary ---
[13.03|12:57:41] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:57:41] 1 1 FAILED Inaccurate 0.4793
[13.03|12:57:41] 1 2 FAILED Inaccurate 0.4059
[13.03|12:57:41] 1 3 FAILED Inaccurate 0.4848
[13.03|12:57:41] 1 4 FAILED Inaccurate 0.6954
[13.03|12:57:41] 1 5 SUCCESS Accurate 0.2303
[13.03|12:57:41] 2 1 FAILED Inaccurate 0.6081
[13.03|12:57:41] 2 2 FAILED Inaccurate 0.6387
[13.03|12:57:41] 2 3 FAILED Inaccurate 0.4972
[13.03|12:57:41] 2 4 FAILED Inaccurate 0.4115
[13.03|12:57:41] 2 5 SUCCESS Accurate 0.2071
[13.03|12:57:41] 3 1 FAILED Inaccurate 0.6595
[13.03|12:57:41] 3 2 SUCCESS Accurate 0.2988
[13.03|12:57:41] 4 1 FAILED Inaccurate 0.4693
[13.03|12:57:41] 4 2 FAILED Inaccurate 0.3853
[13.03|12:57:41] 4 3 FAILED Inaccurate 0.9123
[13.03|12:57:41] 4 4 SUCCESS Accurate 0.2058
[13.03|12:57:41] --- End summary ---
[13.03|12:57:41]
[13.03|12:57:41] ##########################
[13.03|12:57:41] ### Step 5 / Attempt 1 ###
[13.03|12:57:41] ##########################
[13.03|12:57:41] MD Steps: 2000 (cumulative: 8500)
[13.03|12:57:41] Current engine settings:
[13.03|12:57:41]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:57:41] Running step5_attempt1_simulation...
[13.03|12:59:17] Job step5_attempt1_simulation finished
[13.03|12:59:17] Deleting files that are no longer needed...
[13.03|12:59:17] Deleting step3_attempt2_simulation
[13.03|12:59:17] Deleting step4_attempt1_simulation
[13.03|12:59:17] Deleting step4_attempt2_simulation
[13.03|12:59:17] Deleting step4_attempt3_simulation
[13.03|12:59:18] Launching reference calculation
[13.03|12:59:21] Reference calculation finished!
[13.03|12:59:21] Checking success for step5_attempt1
[13.03|12:59:34] CheckEnergy: Checking energy for MDStep8500, n_atoms = 46
[13.03|12:59:34] CheckEnergy: normalization coefficient = 46
[13.03|12:59:34] CheckEnergy: Actual Threshold
[13.03|12:59:34] CheckEnergy: dE/46 0.0021 0.0100 OK!
[13.03|12:59:34] CheckEnergy: ddE/46 -0.0007 0.0020 OK! (relative to step4_attempt4_simulation:MDStep6500)
[13.03|12:59:34]
[13.03|12:59:34] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|12:59:34] CheckForces: ------------
[13.03|12:59:34] CheckForces: Reference job from step5_attempt1_reference_calc1
[13.03|12:59:34] CheckForces: Prediction job from final frame (MDStep8500) of step5_attempt1_simulation
[13.03|12:59:34] CheckForces: ------------
[13.03|12:59:34] CheckForces: Histogram of forces
[13.03|12:59:34] CheckForces: eV/Ang Ref Pred
[13.03|12:59:34] CheckForces: -2 0 0
[13.03|12:59:34] CheckForces: -1 74 70
[13.03|12:59:34] CheckForces: 0 64 68
[13.03|12:59:34] CheckForces: 1 0 0
[13.03|12:59:34] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|12:59:34] CheckForces: All force components are within the acceptable error!
[13.03|12:59:34] CheckForces: Maximum deviation: 0.279 eV/angstrom
[13.03|12:59:34] CheckForces: Actual Threshold
[13.03|12:59:34] CheckForces: # > thr. 0 0 OK!
[13.03|12:59:34] CheckForces: MAE 0.047 0.30 OK!
[13.03|12:59:34] CheckForces: R^2 0.897 0.80 OK!
[13.03|12:59:34] CheckForces: --------------------
[13.03|12:59:34]
[13.03|12:59:34] Adding results from step5_attempt1_reference_calc1 to validation set
[13.03|12:59:34] Current # training set entries: 63
[13.03|12:59:34] Current # validation set entries: 30
[13.03|12:59:34] Storing data in step5_attempt1_reference_data
[13.03|12:59:34] Deleting step4_attempt4_reference_data
[13.03|12:59:34] Deleting step5_attempt1_reference_calc1
[13.03|12:59:34]
[13.03|12:59:34] Current (cumulative) timings:
[13.03|12:59:34] Time (s) Fraction
[13.03|12:59:34] Ref. calcs 124.36 0.015
[13.03|12:59:34] ML training 7066.45 0.827
[13.03|12:59:34] Simulations 1356.52 0.159
[13.03|12:59:34]
[13.03|12:59:34]
[13.03|12:59:35] Step 5 finished successfully!
[13.03|12:59:35]
[13.03|12:59:35] --- Begin summary ---
[13.03|12:59:35] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|12:59:35] 1 1 FAILED Inaccurate 0.4793
[13.03|12:59:35] 1 2 FAILED Inaccurate 0.4059
[13.03|12:59:35] 1 3 FAILED Inaccurate 0.4848
[13.03|12:59:35] 1 4 FAILED Inaccurate 0.6954
[13.03|12:59:35] 1 5 SUCCESS Accurate 0.2303
[13.03|12:59:35] 2 1 FAILED Inaccurate 0.6081
[13.03|12:59:35] 2 2 FAILED Inaccurate 0.6387
[13.03|12:59:35] 2 3 FAILED Inaccurate 0.4972
[13.03|12:59:35] 2 4 FAILED Inaccurate 0.4115
[13.03|12:59:35] 2 5 SUCCESS Accurate 0.2071
[13.03|12:59:35] 3 1 FAILED Inaccurate 0.6595
[13.03|12:59:35] 3 2 SUCCESS Accurate 0.2988
[13.03|12:59:35] 4 1 FAILED Inaccurate 0.4693
[13.03|12:59:35] 4 2 FAILED Inaccurate 0.3853
[13.03|12:59:35] 4 3 FAILED Inaccurate 0.9123
[13.03|12:59:35] 4 4 SUCCESS Accurate 0.2058
[13.03|12:59:35] 5 1 SUCCESS Accurate 0.2789
[13.03|12:59:35] --- End summary ---
[13.03|12:59:35]
[13.03|12:59:35] ##########################
[13.03|12:59:35] ### Step 6 / Attempt 1 ###
[13.03|12:59:35] ##########################
[13.03|12:59:35] MD Steps: 2000 (cumulative: 10500)
[13.03|12:59:35] Current engine settings:
[13.03|12:59:35]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|12:59:35] Running step6_attempt1_simulation...
[13.03|13:01:16] Job step6_attempt1_simulation finished
[13.03|13:01:16] Deleting files that are no longer needed...
[13.03|13:01:16] Deleting step4_attempt4_simulation
[13.03|13:01:17] Launching reference calculation
[13.03|13:01:19] Reference calculation finished!
[13.03|13:01:19] Checking success for step6_attempt1
[13.03|13:01:32] CheckEnergy: Checking energy for MDStep10500, n_atoms = 46
[13.03|13:01:32] CheckEnergy: normalization coefficient = 46
[13.03|13:01:32] CheckEnergy: Actual Threshold
[13.03|13:01:32] CheckEnergy: dE/46 0.0030 0.0100 OK!
[13.03|13:01:32] CheckEnergy: ddE/46 0.0009 0.0020 OK! (relative to step5_attempt1_simulation:MDStep8500)
[13.03|13:01:32]
[13.03|13:01:32] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:01:32] CheckForces: ------------
[13.03|13:01:32] CheckForces: Reference job from step6_attempt1_reference_calc1
[13.03|13:01:32] CheckForces: Prediction job from final frame (MDStep10500) of step6_attempt1_simulation
[13.03|13:01:32] CheckForces: ------------
[13.03|13:01:32] CheckForces: Histogram of forces
[13.03|13:01:32] CheckForces: eV/Ang Ref Pred
[13.03|13:01:32] CheckForces: -2 0 1
[13.03|13:01:32] CheckForces: -1 67 62
[13.03|13:01:32] CheckForces: 0 71 75
[13.03|13:01:32] CheckForces: 1 0 0
[13.03|13:01:32] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:01:32] CheckForces: All force components are within the acceptable error!
[13.03|13:01:32] CheckForces: Maximum deviation: 0.258 eV/angstrom
[13.03|13:01:32] CheckForces: Actual Threshold
[13.03|13:01:32] CheckForces: # > thr. 0 0 OK!
[13.03|13:01:32] CheckForces: MAE 0.035 0.30 OK!
[13.03|13:01:32] CheckForces: R^2 0.953 0.80 OK!
[13.03|13:01:32] CheckForces: --------------------
[13.03|13:01:32]
[13.03|13:01:33] Adding results from step6_attempt1_reference_calc1 to training set
[13.03|13:01:33] Current # training set entries: 64
[13.03|13:01:33] Current # validation set entries: 30
[13.03|13:01:33] Storing data in step6_attempt1_reference_data
[13.03|13:01:33] Deleting step5_attempt1_reference_data
[13.03|13:01:33] Deleting step6_attempt1_reference_calc1
[13.03|13:01:33]
[13.03|13:01:33] Current (cumulative) timings:
[13.03|13:01:33] Time (s) Fraction
[13.03|13:01:33] Ref. calcs 127.26 0.015
[13.03|13:01:33] ML training 7066.45 0.817
[13.03|13:01:33] Simulations 1457.41 0.168
[13.03|13:01:33]
[13.03|13:01:33]
[13.03|13:01:33] Step 6 finished successfully!
[13.03|13:01:33]
[13.03|13:01:33] --- Begin summary ---
[13.03|13:01:33] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:01:33] 1 1 FAILED Inaccurate 0.4793
[13.03|13:01:33] 1 2 FAILED Inaccurate 0.4059
[13.03|13:01:33] 1 3 FAILED Inaccurate 0.4848
[13.03|13:01:33] 1 4 FAILED Inaccurate 0.6954
[13.03|13:01:33] 1 5 SUCCESS Accurate 0.2303
[13.03|13:01:33] 2 1 FAILED Inaccurate 0.6081
[13.03|13:01:33] 2 2 FAILED Inaccurate 0.6387
[13.03|13:01:33] 2 3 FAILED Inaccurate 0.4972
[13.03|13:01:33] 2 4 FAILED Inaccurate 0.4115
[13.03|13:01:33] 2 5 SUCCESS Accurate 0.2071
[13.03|13:01:33] 3 1 FAILED Inaccurate 0.6595
[13.03|13:01:33] 3 2 SUCCESS Accurate 0.2988
[13.03|13:01:33] 4 1 FAILED Inaccurate 0.4693
[13.03|13:01:33] 4 2 FAILED Inaccurate 0.3853
[13.03|13:01:33] 4 3 FAILED Inaccurate 0.9123
[13.03|13:01:33] 4 4 SUCCESS Accurate 0.2058
[13.03|13:01:33] 5 1 SUCCESS Accurate 0.2789
[13.03|13:01:33] 6 1 SUCCESS Accurate 0.2585
[13.03|13:01:33] --- End summary ---
[13.03|13:01:33]
[13.03|13:01:33] ##########################
[13.03|13:01:33] ### Step 7 / Attempt 1 ###
[13.03|13:01:33] ##########################
[13.03|13:01:33] MD Steps: 2000 (cumulative: 12500)
[13.03|13:01:33] Current engine settings:
[13.03|13:01:33]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step4_attempt3_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:01:33] Running step7_attempt1_simulation...
[13.03|13:03:13] Job step7_attempt1_simulation finished
[13.03|13:03:13] Deleting files that are no longer needed...
[13.03|13:03:13] Deleting step5_attempt1_simulation
[13.03|13:03:14] Launching reference calculation
[13.03|13:03:17] Reference calculation finished!
[13.03|13:03:17] Checking success for step7_attempt1
[13.03|13:03:28] CheckEnergy: Checking energy for MDStep12500, n_atoms = 46
[13.03|13:03:28] CheckEnergy: normalization coefficient = 46
[13.03|13:03:28] CheckEnergy: Actual Threshold
[13.03|13:03:28] CheckEnergy: dE/46 0.0023 0.0100 OK!
[13.03|13:03:28] CheckEnergy: ddE/46 -0.0007 0.0020 OK! (relative to step6_attempt1_simulation:MDStep10500)
[13.03|13:03:28]
[13.03|13:03:28] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:03:28] CheckForces: ------------
[13.03|13:03:28] CheckForces: Reference job from step7_attempt1_reference_calc1
[13.03|13:03:28] CheckForces: Prediction job from final frame (MDStep12500) of step7_attempt1_simulation
[13.03|13:03:28] CheckForces: ------------
[13.03|13:03:28] CheckForces: Histogram of forces
[13.03|13:03:28] CheckForces: eV/Ang Ref Pred
[13.03|13:03:28] CheckForces: -2 0 0
[13.03|13:03:28] CheckForces: -1 70 67
[13.03|13:03:28] CheckForces: 0 68 71
[13.03|13:03:28] CheckForces: 1 0 0
[13.03|13:03:28] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:03:28] CheckForces: Force components with an error exceeding the threshold:
[13.03|13:03:28] CheckForces: Ref Pred Delta Threshold
[13.03|13:03:28] CheckForces: -0.77 -0.39 0.38 0.34
[13.03|13:03:28] CheckForces: 0.79 0.28 0.50 0.34
[13.03|13:03:28] CheckForces: Maximum deviation: 0.503 eV/angstrom
[13.03|13:03:28] CheckForces: Actual Threshold
[13.03|13:03:28] CheckForces: # > thr. 2 0 Not OK!
[13.03|13:03:28] CheckForces: MAE 0.061 0.30 OK!
[13.03|13:03:28] CheckForces: R^2 0.876 0.80 OK!
[13.03|13:03:28] CheckForces: --------------------
[13.03|13:03:28]
[13.03|13:03:29] Adding results from step7_attempt1_reference_calc1 to training set
[13.03|13:03:29] Current # training set entries: 65
[13.03|13:03:29] Current # validation set entries: 30
[13.03|13:03:29] Storing data in step7_attempt1_reference_data
[13.03|13:03:29] Deleting step6_attempt1_reference_data
[13.03|13:03:29] Deleting step7_attempt1_reference_calc1
[13.03|13:03:29]
[13.03|13:03:29] Current (cumulative) timings:
[13.03|13:03:29] Time (s) Fraction
[13.03|13:03:29] Ref. calcs 130.21 0.015
[13.03|13:03:29] ML training 7066.45 0.807
[13.03|13:03:29] Simulations 1556.84 0.178
[13.03|13:03:29]
[13.03|13:03:29]
[13.03|13:03:29]
[13.03|13:03:29] --- Begin summary ---
[13.03|13:03:29] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:03:29] 1 1 FAILED Inaccurate 0.4793
[13.03|13:03:29] 1 2 FAILED Inaccurate 0.4059
[13.03|13:03:29] 1 3 FAILED Inaccurate 0.4848
[13.03|13:03:29] 1 4 FAILED Inaccurate 0.6954
[13.03|13:03:29] 1 5 SUCCESS Accurate 0.2303
[13.03|13:03:29] 2 1 FAILED Inaccurate 0.6081
[13.03|13:03:29] 2 2 FAILED Inaccurate 0.6387
[13.03|13:03:29] 2 3 FAILED Inaccurate 0.4972
[13.03|13:03:29] 2 4 FAILED Inaccurate 0.4115
[13.03|13:03:29] 2 5 SUCCESS Accurate 0.2071
[13.03|13:03:29] 3 1 FAILED Inaccurate 0.6595
[13.03|13:03:29] 3 2 SUCCESS Accurate 0.2988
[13.03|13:03:29] 4 1 FAILED Inaccurate 0.4693
[13.03|13:03:29] 4 2 FAILED Inaccurate 0.3853
[13.03|13:03:29] 4 3 FAILED Inaccurate 0.9123
[13.03|13:03:29] 4 4 SUCCESS Accurate 0.2058
[13.03|13:03:29] 5 1 SUCCESS Accurate 0.2789
[13.03|13:03:29] 6 1 SUCCESS Accurate 0.2585
[13.03|13:03:29] 7 1 FAILED Inaccurate 0.5031
[13.03|13:03:29] --- End summary ---
[13.03|13:03:29]
[13.03|13:03:29] Running more reference calculations....
[13.03|13:03:30] Running reference calculations on frames [560, 593] from step7_attempt1_simulation/ams.rkf
[13.03|13:03:30] Calculating 2 frames in total
[13.03|13:03:30] Running step7_attempt1_reference_calc2
[13.03|13:03:33] Running step7_attempt1_reference_calc3
[13.03|13:03:36] Reference calculations finished!
[13.03|13:03:36] Adding results from step7_attempt1_reference_calc2 to validation set
[13.03|13:03:36] Adding results from step7_attempt1_reference_calc3 to training set
[13.03|13:03:36] Current # training set entries: 66
[13.03|13:03:36] Current # validation set entries: 31
[13.03|13:03:36] Storing data in step7_attempt1_reference_data
[13.03|13:03:37] Deleting step7_attempt1_reference_calc2
[13.03|13:03:37] Deleting step7_attempt1_reference_calc3
[13.03|13:03:37] Launching reparametrization job: step7_attempt1_training
[13.03|13:03:43] JOB m3gnet STARTED
[13.03|13:03:43] Starting m3gnet.prerun()
[13.03|13:03:43] m3gnet.prerun() finished
[13.03|13:03:43] JOB m3gnet RUNNING
[13.03|13:03:43] Executing m3gnet.run
[13.03|13:04:54] training_set Optimizer: 001 Epoch: 0 Loss: 0.001232
[13.03|13:04:54] validation_set Optimizer: 001 Epoch: 0 Loss: 0.018746
[13.03|13:05:27] training_set Optimizer: 001 Epoch: 10 Loss: 0.000052
[13.03|13:05:27] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005861
[13.03|13:06:00] training_set Optimizer: 001 Epoch: 20 Loss: 0.000047
[13.03|13:06:00] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005835
[13.03|13:06:34] training_set Optimizer: 001 Epoch: 30 Loss: 0.000051
[13.03|13:06:34] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005603
[13.03|13:07:07] training_set Optimizer: 001 Epoch: 40 Loss: 0.000062
[13.03|13:07:07] validation_set Optimizer: 001 Epoch: 40 Loss: 0.008666
[13.03|13:07:40] training_set Optimizer: 001 Epoch: 50 Loss: 0.000046
[13.03|13:07:40] validation_set Optimizer: 001 Epoch: 50 Loss: 0.006204
[13.03|13:08:14] training_set Optimizer: 001 Epoch: 60 Loss: 0.000051
[13.03|13:08:14] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005965
[13.03|13:08:47] training_set Optimizer: 001 Epoch: 70 Loss: 0.000054
[13.03|13:08:47] validation_set Optimizer: 001 Epoch: 70 Loss: 0.006588
[13.03|13:09:20] training_set Optimizer: 001 Epoch: 80 Loss: 0.000046
[13.03|13:09:20] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005563
[13.03|13:09:54] training_set Optimizer: 001 Epoch: 90 Loss: 0.000051
[13.03|13:09:54] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005928
[13.03|13:10:28] training_set Optimizer: 001 Epoch: 100 Loss: 0.000049
[13.03|13:10:28] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005214
[13.03|13:11:03] training_set Optimizer: 001 Epoch: 110 Loss: 0.000050
[13.03|13:11:03] validation_set Optimizer: 001 Epoch: 110 Loss: 0.005563
[13.03|13:11:36] training_set Optimizer: 001 Epoch: 120 Loss: 0.000056
[13.03|13:11:36] validation_set Optimizer: 001 Epoch: 120 Loss: 0.012101
[13.03|13:12:09] training_set Optimizer: 001 Epoch: 130 Loss: 0.000057
[13.03|13:12:09] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005823
[13.03|13:12:43] training_set Optimizer: 001 Epoch: 140 Loss: 0.000043
[13.03|13:12:43] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006174
[13.03|13:13:16] training_set Optimizer: 001 Epoch: 150 Loss: 0.000058
[13.03|13:13:16] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005449
[13.03|13:13:49] training_set Optimizer: 001 Epoch: 160 Loss: 0.000046
[13.03|13:13:49] validation_set Optimizer: 001 Epoch: 160 Loss: 0.005839
[13.03|13:14:22] training_set Optimizer: 001 Epoch: 170 Loss: 0.000058
[13.03|13:14:22] validation_set Optimizer: 001 Epoch: 170 Loss: 0.008020
[13.03|13:14:55] training_set Optimizer: 001 Epoch: 180 Loss: 0.000043
[13.03|13:14:55] validation_set Optimizer: 001 Epoch: 180 Loss: 0.007656
[13.03|13:15:29] training_set Optimizer: 001 Epoch: 190 Loss: 0.000051
[13.03|13:15:29] validation_set Optimizer: 001 Epoch: 190 Loss: 0.009837
[13.03|13:16:01] Execution of m3gnet.run finished with returncode 0
[13.03|13:16:01] JOB m3gnet FINISHED
[13.03|13:16:01] Starting m3gnet.postrun()
[13.03|13:16:01] m3gnet.postrun() finished
[13.03|13:16:02] JOB m3gnet SUCCESSFUL
[13.03|13:16:23] Running all jobs through AMS....
[13.03|13:16:23] Storing results/optimization/training_set_results/best
[13.03|13:16:23] Storing results/optimization/validation_set_results/best
[13.03|13:16:23] PLAMS environment cleaned up successfully
[13.03|13:16:23] PLAMS run finished. Goodbye
[13.03|13:16:25] ParAMSResults training_set validation_set
[13.03|13:16:25] energy MAE 0.3587 0.3264 eV
[13.03|13:16:25] forces MAE 0.0174 0.0465 eV/angstrom
[13.03|13:16:25] Newly created parameter file/dir: step7_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|13:16:25] Done!
[13.03|13:16:25] Deleting step4_attempt3_training
[13.03|13:16:25] ##########################
[13.03|13:16:25] ### Step 7 / Attempt 2 ###
[13.03|13:16:25] ##########################
[13.03|13:16:25] MD Steps: 2000 (cumulative: 12500)
[13.03|13:16:25] Current engine settings:
[13.03|13:16:25]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:16:25] Running step7_attempt2_simulation...
[13.03|13:18:07] Job step7_attempt2_simulation finished
[13.03|13:18:07] Deleting files that are no longer needed...
[13.03|13:18:08] Launching reference calculation
[13.03|13:18:11] Reference calculation finished!
[13.03|13:18:11] Checking success for step7_attempt2
[13.03|13:18:23] CheckEnergy: Checking energy for MDStep12500, n_atoms = 46
[13.03|13:18:23] CheckEnergy: normalization coefficient = 46
[13.03|13:18:23] CheckEnergy: Actual Threshold
[13.03|13:18:23] CheckEnergy: dE/46 0.0069 0.0100 OK!
[13.03|13:18:23] CheckEnergy: ddE/46 -0.0007 0.0020 OK! (relative to step7_attempt1_simulation:MDStep12500)
[13.03|13:18:23]
[13.03|13:18:23] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:18:23] CheckForces: ------------
[13.03|13:18:23] CheckForces: Reference job from step7_attempt2_reference_calc1
[13.03|13:18:23] CheckForces: Prediction job from final frame (MDStep12500) of step7_attempt2_simulation
[13.03|13:18:23] CheckForces: ------------
[13.03|13:18:23] CheckForces: Histogram of forces
[13.03|13:18:23] CheckForces: eV/Ang Ref Pred
[13.03|13:18:23] CheckForces: -2 1 2
[13.03|13:18:23] CheckForces: -1 64 62
[13.03|13:18:23] CheckForces: 0 72 74
[13.03|13:18:23] CheckForces: 1 1 0
[13.03|13:18:23] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:18:23] CheckForces: Force components with an error exceeding the threshold:
[13.03|13:18:23] CheckForces: Ref Pred Delta Threshold
[13.03|13:18:23] CheckForces: 0.20 -0.12 0.32 0.31
[13.03|13:18:23] CheckForces: 0.48 0.05 0.43 0.32
[13.03|13:18:23] CheckForces: -0.13 0.30 0.43 0.31
[13.03|13:18:23] CheckForces: Maximum deviation: 0.432 eV/angstrom
[13.03|13:18:23] CheckForces: Actual Threshold
[13.03|13:18:23] CheckForces: # > thr. 3 0 Not OK!
[13.03|13:18:23] CheckForces: MAE 0.056 0.30 OK!
[13.03|13:18:23] CheckForces: R^2 0.895 0.80 OK!
[13.03|13:18:23] CheckForces: --------------------
[13.03|13:18:23]
[13.03|13:18:23] Adding results from step7_attempt2_reference_calc1 to training set
[13.03|13:18:23] Current # training set entries: 67
[13.03|13:18:23] Current # validation set entries: 31
[13.03|13:18:23] Storing data in step7_attempt2_reference_data
[13.03|13:18:24] Deleting step7_attempt1_reference_data
[13.03|13:18:24] Deleting step7_attempt2_reference_calc1
[13.03|13:18:24]
[13.03|13:18:24] Current (cumulative) timings:
[13.03|13:18:24] Time (s) Fraction
[13.03|13:18:24] Ref. calcs 139.55 0.014
[13.03|13:18:24] ML training 7834.51 0.813
[13.03|13:18:24] Simulations 1658.60 0.172
[13.03|13:18:24]
[13.03|13:18:24]
[13.03|13:18:24]
[13.03|13:18:24] --- Begin summary ---
[13.03|13:18:24] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:18:24] 1 1 FAILED Inaccurate 0.4793
[13.03|13:18:24] 1 2 FAILED Inaccurate 0.4059
[13.03|13:18:24] 1 3 FAILED Inaccurate 0.4848
[13.03|13:18:24] 1 4 FAILED Inaccurate 0.6954
[13.03|13:18:24] 1 5 SUCCESS Accurate 0.2303
[13.03|13:18:24] 2 1 FAILED Inaccurate 0.6081
[13.03|13:18:24] 2 2 FAILED Inaccurate 0.6387
[13.03|13:18:24] 2 3 FAILED Inaccurate 0.4972
[13.03|13:18:24] 2 4 FAILED Inaccurate 0.4115
[13.03|13:18:24] 2 5 SUCCESS Accurate 0.2071
[13.03|13:18:24] 3 1 FAILED Inaccurate 0.6595
[13.03|13:18:24] 3 2 SUCCESS Accurate 0.2988
[13.03|13:18:24] 4 1 FAILED Inaccurate 0.4693
[13.03|13:18:24] 4 2 FAILED Inaccurate 0.3853
[13.03|13:18:24] 4 3 FAILED Inaccurate 0.9123
[13.03|13:18:24] 4 4 SUCCESS Accurate 0.2058
[13.03|13:18:24] 5 1 SUCCESS Accurate 0.2789
[13.03|13:18:24] 6 1 SUCCESS Accurate 0.2585
[13.03|13:18:24] 7 1 FAILED Inaccurate 0.5031
[13.03|13:18:24] 7 2 FAILED Inaccurate 0.4318
[13.03|13:18:24] --- End summary ---
[13.03|13:18:24]
[13.03|13:18:24] Running more reference calculations....
[13.03|13:18:24] Running reference calculations on frames [560, 593] from step7_attempt2_simulation/ams.rkf
[13.03|13:18:24] Calculating 2 frames in total
[13.03|13:18:24] Running step7_attempt2_reference_calc2
[13.03|13:18:27] Running step7_attempt2_reference_calc3
[13.03|13:18:30] Reference calculations finished!
[13.03|13:18:31] Adding results from step7_attempt2_reference_calc2 to validation set
[13.03|13:18:31] Adding results from step7_attempt2_reference_calc3 to training set
[13.03|13:18:31] Current # training set entries: 68
[13.03|13:18:31] Current # validation set entries: 32
[13.03|13:18:31] Storing data in step7_attempt2_reference_data
[13.03|13:18:32] Deleting step7_attempt2_reference_calc2
[13.03|13:18:32] Deleting step7_attempt2_reference_calc3
[13.03|13:18:32] Launching reparametrization job: step7_attempt2_training
[13.03|13:18:38] JOB m3gnet STARTED
[13.03|13:18:38] Starting m3gnet.prerun()
[13.03|13:18:38] m3gnet.prerun() finished
[13.03|13:18:38] JOB m3gnet RUNNING
[13.03|13:18:38] Executing m3gnet.run
[13.03|13:19:49] training_set Optimizer: 001 Epoch: 0 Loss: 0.001396
[13.03|13:19:49] validation_set Optimizer: 001 Epoch: 0 Loss: 0.032004
[13.03|13:20:23] training_set Optimizer: 001 Epoch: 10 Loss: 0.000048
[13.03|13:20:23] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005151
[13.03|13:20:57] training_set Optimizer: 001 Epoch: 20 Loss: 0.000046
[13.03|13:20:57] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005019
[13.03|13:21:30] training_set Optimizer: 001 Epoch: 30 Loss: 0.000044
[13.03|13:21:30] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005146
[13.03|13:22:04] training_set Optimizer: 001 Epoch: 40 Loss: 0.000044
[13.03|13:22:04] validation_set Optimizer: 001 Epoch: 40 Loss: 0.006891
[13.03|13:22:37] training_set Optimizer: 001 Epoch: 50 Loss: 0.000046
[13.03|13:22:37] validation_set Optimizer: 001 Epoch: 50 Loss: 0.005523
[13.03|13:23:10] training_set Optimizer: 001 Epoch: 60 Loss: 0.000044
[13.03|13:23:10] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005352
[13.03|13:23:44] training_set Optimizer: 001 Epoch: 70 Loss: 0.000046
[13.03|13:23:44] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005516
[13.03|13:24:18] training_set Optimizer: 001 Epoch: 80 Loss: 0.000041
[13.03|13:24:18] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005112
[13.03|13:24:51] training_set Optimizer: 001 Epoch: 90 Loss: 0.000042
[13.03|13:24:51] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005366
[13.03|13:25:25] training_set Optimizer: 001 Epoch: 100 Loss: 0.000048
[13.03|13:25:25] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005331
[13.03|13:25:59] training_set Optimizer: 001 Epoch: 110 Loss: 0.000047
[13.03|13:25:59] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006142
[13.03|13:26:32] training_set Optimizer: 001 Epoch: 120 Loss: 0.000053
[13.03|13:26:32] validation_set Optimizer: 001 Epoch: 120 Loss: 0.010516
[13.03|13:27:06] training_set Optimizer: 001 Epoch: 130 Loss: 0.000046
[13.03|13:27:06] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005148
[13.03|13:27:39] training_set Optimizer: 001 Epoch: 140 Loss: 0.000043
[13.03|13:27:39] validation_set Optimizer: 001 Epoch: 140 Loss: 0.008393
[13.03|13:28:13] training_set Optimizer: 001 Epoch: 150 Loss: 0.000041
[13.03|13:28:13] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005641
[13.03|13:28:47] training_set Optimizer: 001 Epoch: 160 Loss: 0.000042
[13.03|13:28:47] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006208
[13.03|13:29:21] training_set Optimizer: 001 Epoch: 170 Loss: 0.000043
[13.03|13:29:21] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006693
[13.03|13:29:54] training_set Optimizer: 001 Epoch: 180 Loss: 0.000050
[13.03|13:29:54] validation_set Optimizer: 001 Epoch: 180 Loss: 0.005174
[13.03|13:30:27] training_set Optimizer: 001 Epoch: 190 Loss: 0.000040
[13.03|13:30:27] validation_set Optimizer: 001 Epoch: 190 Loss: 0.007556
[13.03|13:31:00] Execution of m3gnet.run finished with returncode 0
[13.03|13:31:01] JOB m3gnet FINISHED
[13.03|13:31:01] Starting m3gnet.postrun()
[13.03|13:31:01] m3gnet.postrun() finished
[13.03|13:31:01] JOB m3gnet SUCCESSFUL
[13.03|13:31:22] Running all jobs through AMS....
[13.03|13:31:23] Storing results/optimization/training_set_results/best
[13.03|13:31:23] Storing results/optimization/validation_set_results/best
[13.03|13:31:23] PLAMS environment cleaned up successfully
[13.03|13:31:23] PLAMS run finished. Goodbye
[13.03|13:31:25] ParAMSResults training_set validation_set
[13.03|13:31:25] energy MAE 0.1874 0.1549 eV
[13.03|13:31:25] forces MAE 0.0149 0.0462 eV/angstrom
[13.03|13:31:25] Newly created parameter file/dir: step7_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|13:31:25] Done!
[13.03|13:31:25] Deleting step7_attempt1_training
[13.03|13:31:25] ##########################
[13.03|13:31:25] ### Step 7 / Attempt 3 ###
[13.03|13:31:25] ##########################
[13.03|13:31:25] MD Steps: 2000 (cumulative: 12500)
[13.03|13:31:25] Current engine settings:
[13.03|13:31:25]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:31:25] Running step7_attempt3_simulation...
[13.03|13:33:05] Job step7_attempt3_simulation finished
[13.03|13:33:05] Deleting files that are no longer needed...
[13.03|13:33:06] Launching reference calculation
[13.03|13:33:09] Reference calculation finished!
[13.03|13:33:09] Checking success for step7_attempt3
[13.03|13:33:21] CheckEnergy: Checking energy for MDStep12500, n_atoms = 46
[13.03|13:33:21] CheckEnergy: normalization coefficient = 46
[13.03|13:33:21] CheckEnergy: Actual Threshold
[13.03|13:33:21] CheckEnergy: dE/46 0.0028 0.0100 OK!
[13.03|13:33:21] CheckEnergy: ddE/46 -0.0007 0.0020 OK! (relative to step7_attempt2_simulation:MDStep12500)
[13.03|13:33:21]
[13.03|13:33:21] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:33:21] CheckForces: ------------
[13.03|13:33:21] CheckForces: Reference job from step7_attempt3_reference_calc1
[13.03|13:33:21] CheckForces: Prediction job from final frame (MDStep12500) of step7_attempt3_simulation
[13.03|13:33:21] CheckForces: ------------
[13.03|13:33:21] CheckForces: Histogram of forces
[13.03|13:33:21] CheckForces: eV/Ang Ref Pred
[13.03|13:33:21] CheckForces: -2 1 0
[13.03|13:33:21] CheckForces: -1 65 65
[13.03|13:33:21] CheckForces: 0 71 72
[13.03|13:33:21] CheckForces: 1 1 1
[13.03|13:33:21] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:33:21] CheckForces: All force components are within the acceptable error!
[13.03|13:33:21] CheckForces: Maximum deviation: 0.180 eV/angstrom
[13.03|13:33:21] CheckForces: Actual Threshold
[13.03|13:33:21] CheckForces: # > thr. 0 0 OK!
[13.03|13:33:21] CheckForces: MAE 0.044 0.30 OK!
[13.03|13:33:21] CheckForces: R^2 0.956 0.80 OK!
[13.03|13:33:21] CheckForces: --------------------
[13.03|13:33:21]
[13.03|13:33:22] Adding results from step7_attempt3_reference_calc1 to training set
[13.03|13:33:22] Current # training set entries: 69
[13.03|13:33:22] Current # validation set entries: 32
[13.03|13:33:22] Storing data in step7_attempt3_reference_data
[13.03|13:33:22] Deleting step7_attempt2_reference_data
[13.03|13:33:22] Deleting step7_attempt3_reference_calc1
[13.03|13:33:22]
[13.03|13:33:22] Current (cumulative) timings:
[13.03|13:33:22] Time (s) Fraction
[13.03|13:33:22] Ref. calcs 148.90 0.014
[13.03|13:33:22] ML training 8607.62 0.819
[13.03|13:33:22] Simulations 1758.64 0.167
[13.03|13:33:22]
[13.03|13:33:22]
[13.03|13:33:23] Step 7 finished successfully!
[13.03|13:33:23]
[13.03|13:33:23] --- Begin summary ---
[13.03|13:33:23] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:33:23] 1 1 FAILED Inaccurate 0.4793
[13.03|13:33:23] 1 2 FAILED Inaccurate 0.4059
[13.03|13:33:23] 1 3 FAILED Inaccurate 0.4848
[13.03|13:33:23] 1 4 FAILED Inaccurate 0.6954
[13.03|13:33:23] 1 5 SUCCESS Accurate 0.2303
[13.03|13:33:23] 2 1 FAILED Inaccurate 0.6081
[13.03|13:33:23] 2 2 FAILED Inaccurate 0.6387
[13.03|13:33:23] 2 3 FAILED Inaccurate 0.4972
[13.03|13:33:23] 2 4 FAILED Inaccurate 0.4115
[13.03|13:33:23] 2 5 SUCCESS Accurate 0.2071
[13.03|13:33:23] 3 1 FAILED Inaccurate 0.6595
[13.03|13:33:23] 3 2 SUCCESS Accurate 0.2988
[13.03|13:33:23] 4 1 FAILED Inaccurate 0.4693
[13.03|13:33:23] 4 2 FAILED Inaccurate 0.3853
[13.03|13:33:23] 4 3 FAILED Inaccurate 0.9123
[13.03|13:33:23] 4 4 SUCCESS Accurate 0.2058
[13.03|13:33:23] 5 1 SUCCESS Accurate 0.2789
[13.03|13:33:23] 6 1 SUCCESS Accurate 0.2585
[13.03|13:33:23] 7 1 FAILED Inaccurate 0.5031
[13.03|13:33:23] 7 2 FAILED Inaccurate 0.4318
[13.03|13:33:23] 7 3 SUCCESS Accurate 0.1800
[13.03|13:33:23] --- End summary ---
[13.03|13:33:23]
[13.03|13:33:23] ##########################
[13.03|13:33:23] ### Step 8 / Attempt 1 ###
[13.03|13:33:23] ##########################
[13.03|13:33:23] MD Steps: 2000 (cumulative: 14500)
[13.03|13:33:23] Current engine settings:
[13.03|13:33:23]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:33:23] Running step8_attempt1_simulation...
[13.03|13:35:01] Job step8_attempt1_simulation finished
[13.03|13:35:01] Deleting files that are no longer needed...
[13.03|13:35:01] Deleting step6_attempt1_simulation
[13.03|13:35:01] Deleting step7_attempt1_simulation
[13.03|13:35:01] Deleting step7_attempt2_simulation
[13.03|13:35:02] Launching reference calculation
[13.03|13:35:05] Reference calculation finished!
[13.03|13:35:05] Checking success for step8_attempt1
[13.03|13:35:17] CheckEnergy: Checking energy for MDStep14500, n_atoms = 46
[13.03|13:35:17] CheckEnergy: normalization coefficient = 46
[13.03|13:35:17] CheckEnergy: Actual Threshold
[13.03|13:35:17] CheckEnergy: dE/46 0.0046 0.0100 OK!
[13.03|13:35:17] CheckEnergy: ddE/46 0.0018 0.0020 OK! (relative to step7_attempt3_simulation:MDStep12500)
[13.03|13:35:17]
[13.03|13:35:18] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:35:18] CheckForces: ------------
[13.03|13:35:18] CheckForces: Reference job from step8_attempt1_reference_calc1
[13.03|13:35:18] CheckForces: Prediction job from final frame (MDStep14500) of step8_attempt1_simulation
[13.03|13:35:18] CheckForces: ------------
[13.03|13:35:18] CheckForces: Histogram of forces
[13.03|13:35:18] CheckForces: eV/Ang Ref Pred
[13.03|13:35:18] CheckForces: -2 0 0
[13.03|13:35:18] CheckForces: -1 67 69
[13.03|13:35:18] CheckForces: 0 70 68
[13.03|13:35:18] CheckForces: 1 1 1
[13.03|13:35:18] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:35:18] CheckForces: All force components are within the acceptable error!
[13.03|13:35:18] CheckForces: Maximum deviation: 0.315 eV/angstrom
[13.03|13:35:18] CheckForces: Actual Threshold
[13.03|13:35:18] CheckForces: # > thr. 0 0 OK!
[13.03|13:35:18] CheckForces: MAE 0.051 0.30 OK!
[13.03|13:35:18] CheckForces: R^2 0.918 0.80 OK!
[13.03|13:35:18] CheckForces: --------------------
[13.03|13:35:18]
[13.03|13:35:18] Adding results from step8_attempt1_reference_calc1 to training set
[13.03|13:35:18] Current # training set entries: 70
[13.03|13:35:18] Current # validation set entries: 32
[13.03|13:35:18] Storing data in step8_attempt1_reference_data
[13.03|13:35:18] Deleting step7_attempt3_reference_data
[13.03|13:35:18] Deleting step8_attempt1_reference_calc1
[13.03|13:35:18]
[13.03|13:35:18] Current (cumulative) timings:
[13.03|13:35:18] Time (s) Fraction
[13.03|13:35:18] Ref. calcs 151.91 0.014
[13.03|13:35:18] ML training 8607.62 0.811
[13.03|13:35:18] Simulations 1856.94 0.175
[13.03|13:35:18]
[13.03|13:35:18]
[13.03|13:35:19] Step 8 finished successfully!
[13.03|13:35:19]
[13.03|13:35:19] --- Begin summary ---
[13.03|13:35:19] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:35:19] 1 1 FAILED Inaccurate 0.4793
[13.03|13:35:19] 1 2 FAILED Inaccurate 0.4059
[13.03|13:35:19] 1 3 FAILED Inaccurate 0.4848
[13.03|13:35:19] 1 4 FAILED Inaccurate 0.6954
[13.03|13:35:19] 1 5 SUCCESS Accurate 0.2303
[13.03|13:35:19] 2 1 FAILED Inaccurate 0.6081
[13.03|13:35:19] 2 2 FAILED Inaccurate 0.6387
[13.03|13:35:19] 2 3 FAILED Inaccurate 0.4972
[13.03|13:35:19] 2 4 FAILED Inaccurate 0.4115
[13.03|13:35:19] 2 5 SUCCESS Accurate 0.2071
[13.03|13:35:19] 3 1 FAILED Inaccurate 0.6595
[13.03|13:35:19] 3 2 SUCCESS Accurate 0.2988
[13.03|13:35:19] 4 1 FAILED Inaccurate 0.4693
[13.03|13:35:19] 4 2 FAILED Inaccurate 0.3853
[13.03|13:35:19] 4 3 FAILED Inaccurate 0.9123
[13.03|13:35:19] 4 4 SUCCESS Accurate 0.2058
[13.03|13:35:19] 5 1 SUCCESS Accurate 0.2789
[13.03|13:35:19] 6 1 SUCCESS Accurate 0.2585
[13.03|13:35:19] 7 1 FAILED Inaccurate 0.5031
[13.03|13:35:19] 7 2 FAILED Inaccurate 0.4318
[13.03|13:35:19] 7 3 SUCCESS Accurate 0.1800
[13.03|13:35:19] 8 1 SUCCESS Accurate 0.3154
[13.03|13:35:19] --- End summary ---
[13.03|13:35:19]
[13.03|13:35:19] ##########################
[13.03|13:35:19] ### Step 9 / Attempt 1 ###
[13.03|13:35:19] ##########################
[13.03|13:35:19] MD Steps: 2000 (cumulative: 16500)
[13.03|13:35:19] Current engine settings:
[13.03|13:35:19]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:35:19] Running step9_attempt1_simulation...
[13.03|13:36:58] Job step9_attempt1_simulation finished
[13.03|13:36:58] Deleting files that are no longer needed...
[13.03|13:36:58] Deleting step7_attempt3_simulation
[13.03|13:36:59] Launching reference calculation
[13.03|13:37:01] Reference calculation finished!
[13.03|13:37:01] Checking success for step9_attempt1
[13.03|13:37:13] CheckEnergy: Checking energy for MDStep16500, n_atoms = 46
[13.03|13:37:13] CheckEnergy: normalization coefficient = 46
[13.03|13:37:13] CheckEnergy: Actual Threshold
[13.03|13:37:13] CheckEnergy: dE/46 0.0037 0.0100 OK!
[13.03|13:37:13] CheckEnergy: ddE/46 -0.0009 0.0020 OK! (relative to step8_attempt1_simulation:MDStep14500)
[13.03|13:37:13]
[13.03|13:37:13] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:37:13] CheckForces: ------------
[13.03|13:37:13] CheckForces: Reference job from step9_attempt1_reference_calc1
[13.03|13:37:13] CheckForces: Prediction job from final frame (MDStep16500) of step9_attempt1_simulation
[13.03|13:37:13] CheckForces: ------------
[13.03|13:37:13] CheckForces: Histogram of forces
[13.03|13:37:13] CheckForces: eV/Ang Ref Pred
[13.03|13:37:13] CheckForces: -2 0 0
[13.03|13:37:13] CheckForces: -1 74 72
[13.03|13:37:13] CheckForces: 0 62 64
[13.03|13:37:13] CheckForces: 1 2 2
[13.03|13:37:13] CheckForces: 2 0 0
[13.03|13:37:13] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:37:13] CheckForces: All force components are within the acceptable error!
[13.03|13:37:13] CheckForces: Maximum deviation: 0.223 eV/angstrom
[13.03|13:37:13] CheckForces: Actual Threshold
[13.03|13:37:13] CheckForces: # > thr. 0 0 OK!
[13.03|13:37:13] CheckForces: MAE 0.036 0.30 OK!
[13.03|13:37:13] CheckForces: R^2 0.966 0.80 OK!
[13.03|13:37:13] CheckForces: --------------------
[13.03|13:37:13]
[13.03|13:37:13] Adding results from step9_attempt1_reference_calc1 to validation set
[13.03|13:37:13] Current # training set entries: 70
[13.03|13:37:13] Current # validation set entries: 33
[13.03|13:37:13] Storing data in step9_attempt1_reference_data
[13.03|13:37:13] Deleting step8_attempt1_reference_data
[13.03|13:37:13] Deleting step9_attempt1_reference_calc1
[13.03|13:37:13]
[13.03|13:37:13] Current (cumulative) timings:
[13.03|13:37:13] Time (s) Fraction
[13.03|13:37:14] Ref. calcs 154.90 0.014
[13.03|13:37:14] ML training 8607.62 0.803
[13.03|13:37:14] Simulations 1955.68 0.182
[13.03|13:37:14]
[13.03|13:37:14]
[13.03|13:37:14] Step 9 finished successfully!
[13.03|13:37:14]
[13.03|13:37:14] --- Begin summary ---
[13.03|13:37:14] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:37:14] 1 1 FAILED Inaccurate 0.4793
[13.03|13:37:14] 1 2 FAILED Inaccurate 0.4059
[13.03|13:37:14] 1 3 FAILED Inaccurate 0.4848
[13.03|13:37:14] 1 4 FAILED Inaccurate 0.6954
[13.03|13:37:14] 1 5 SUCCESS Accurate 0.2303
[13.03|13:37:14] 2 1 FAILED Inaccurate 0.6081
[13.03|13:37:14] 2 2 FAILED Inaccurate 0.6387
[13.03|13:37:14] 2 3 FAILED Inaccurate 0.4972
[13.03|13:37:14] 2 4 FAILED Inaccurate 0.4115
[13.03|13:37:14] 2 5 SUCCESS Accurate 0.2071
[13.03|13:37:14] 3 1 FAILED Inaccurate 0.6595
[13.03|13:37:14] 3 2 SUCCESS Accurate 0.2988
[13.03|13:37:14] 4 1 FAILED Inaccurate 0.4693
[13.03|13:37:14] 4 2 FAILED Inaccurate 0.3853
[13.03|13:37:14] 4 3 FAILED Inaccurate 0.9123
[13.03|13:37:14] 4 4 SUCCESS Accurate 0.2058
[13.03|13:37:14] 5 1 SUCCESS Accurate 0.2789
[13.03|13:37:14] 6 1 SUCCESS Accurate 0.2585
[13.03|13:37:14] 7 1 FAILED Inaccurate 0.5031
[13.03|13:37:14] 7 2 FAILED Inaccurate 0.4318
[13.03|13:37:14] 7 3 SUCCESS Accurate 0.1800
[13.03|13:37:14] 8 1 SUCCESS Accurate 0.3154
[13.03|13:37:14] 9 1 SUCCESS Accurate 0.2225
[13.03|13:37:14] --- End summary ---
[13.03|13:37:14]
[13.03|13:37:14] ###########################
[13.03|13:37:14] ### Step 10 / Attempt 1 ###
[13.03|13:37:14] ###########################
[13.03|13:37:14] MD Steps: 2000 (cumulative: 18500)
[13.03|13:37:14] Current engine settings:
[13.03|13:37:14]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:37:14] Running step10_attempt1_simulation...
[13.03|13:38:50] Job step10_attempt1_simulation finished
[13.03|13:38:50] Deleting files that are no longer needed...
[13.03|13:38:50] Deleting step8_attempt1_simulation
[13.03|13:38:51] Launching reference calculation
[13.03|13:38:54] Reference calculation finished!
[13.03|13:38:54] Checking success for step10_attempt1
[13.03|13:39:06] CheckEnergy: Checking energy for MDStep18500, n_atoms = 46
[13.03|13:39:06] CheckEnergy: normalization coefficient = 46
[13.03|13:39:06] CheckEnergy: Actual Threshold
[13.03|13:39:06] CheckEnergy: dE/46 0.0020 0.0100 OK!
[13.03|13:39:06] CheckEnergy: ddE/46 -0.0017 0.0020 OK! (relative to step9_attempt1_simulation:MDStep16500)
[13.03|13:39:06]
[13.03|13:39:06] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:39:06] CheckForces: ------------
[13.03|13:39:06] CheckForces: Reference job from step10_attempt1_reference_calc1
[13.03|13:39:06] CheckForces: Prediction job from final frame (MDStep18500) of step10_attempt1_simulation
[13.03|13:39:06] CheckForces: ------------
[13.03|13:39:06] CheckForces: Histogram of forces
[13.03|13:39:06] CheckForces: eV/Ang Ref Pred
[13.03|13:39:06] CheckForces: -2 0 0
[13.03|13:39:06] CheckForces: -1 70 68
[13.03|13:39:06] CheckForces: 0 67 69
[13.03|13:39:06] CheckForces: 1 1 1
[13.03|13:39:06] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:39:06] CheckForces: All force components are within the acceptable error!
[13.03|13:39:06] CheckForces: Maximum deviation: 0.308 eV/angstrom
[13.03|13:39:06] CheckForces: Actual Threshold
[13.03|13:39:06] CheckForces: # > thr. 0 0 OK!
[13.03|13:39:06] CheckForces: MAE 0.033 0.30 OK!
[13.03|13:39:06] CheckForces: R^2 0.950 0.80 OK!
[13.03|13:39:06] CheckForces: --------------------
[13.03|13:39:06]
[13.03|13:39:06] Adding results from step10_attempt1_reference_calc1 to validation set
[13.03|13:39:06] Current # training set entries: 70
[13.03|13:39:06] Current # validation set entries: 34
[13.03|13:39:06] Storing data in step10_attempt1_reference_data
[13.03|13:39:07] Deleting step9_attempt1_reference_data
[13.03|13:39:07] Deleting step10_attempt1_reference_calc1
[13.03|13:39:07]
[13.03|13:39:07] Current (cumulative) timings:
[13.03|13:39:07] Time (s) Fraction
[13.03|13:39:07] Ref. calcs 157.86 0.015
[13.03|13:39:07] ML training 8607.62 0.796
[13.03|13:39:07] Simulations 2051.64 0.190
[13.03|13:39:07]
[13.03|13:39:07]
[13.03|13:39:07] Step 10 finished successfully!
[13.03|13:39:07]
[13.03|13:39:07] --- Begin summary ---
[13.03|13:39:07] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:39:07] 1 1 FAILED Inaccurate 0.4793
[13.03|13:39:07] 1 2 FAILED Inaccurate 0.4059
[13.03|13:39:07] 1 3 FAILED Inaccurate 0.4848
[13.03|13:39:07] 1 4 FAILED Inaccurate 0.6954
[13.03|13:39:07] 1 5 SUCCESS Accurate 0.2303
[13.03|13:39:07] 2 1 FAILED Inaccurate 0.6081
[13.03|13:39:07] 2 2 FAILED Inaccurate 0.6387
[13.03|13:39:07] 2 3 FAILED Inaccurate 0.4972
[13.03|13:39:07] 2 4 FAILED Inaccurate 0.4115
[13.03|13:39:07] 2 5 SUCCESS Accurate 0.2071
[13.03|13:39:07] 3 1 FAILED Inaccurate 0.6595
[13.03|13:39:07] 3 2 SUCCESS Accurate 0.2988
[13.03|13:39:07] 4 1 FAILED Inaccurate 0.4693
[13.03|13:39:07] 4 2 FAILED Inaccurate 0.3853
[13.03|13:39:07] 4 3 FAILED Inaccurate 0.9123
[13.03|13:39:07] 4 4 SUCCESS Accurate 0.2058
[13.03|13:39:07] 5 1 SUCCESS Accurate 0.2789
[13.03|13:39:07] 6 1 SUCCESS Accurate 0.2585
[13.03|13:39:07] 7 1 FAILED Inaccurate 0.5031
[13.03|13:39:07] 7 2 FAILED Inaccurate 0.4318
[13.03|13:39:07] 7 3 SUCCESS Accurate 0.1800
[13.03|13:39:07] 8 1 SUCCESS Accurate 0.3154
[13.03|13:39:07] 9 1 SUCCESS Accurate 0.2225
[13.03|13:39:07] 10 1 SUCCESS Accurate 0.3078
[13.03|13:39:07] --- End summary ---
[13.03|13:39:07]
[13.03|13:39:07] ###########################
[13.03|13:39:07] ### Step 11 / Attempt 1 ###
[13.03|13:39:07] ###########################
[13.03|13:39:07] MD Steps: 1500 (cumulative: 20000)
[13.03|13:39:07] Current engine settings:
[13.03|13:39:07]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step7_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:39:07] Running step11_attempt1_simulation...
[13.03|13:40:21] Job step11_attempt1_simulation finished
[13.03|13:40:21] Deleting files that are no longer needed...
[13.03|13:40:21] Deleting step9_attempt1_simulation
[13.03|13:40:22] Launching reference calculation
[13.03|13:40:25] Reference calculation finished!
[13.03|13:40:25] Checking success for step11_attempt1
[13.03|13:40:38] CheckEnergy: Checking energy for MDStep20000, n_atoms = 46
[13.03|13:40:38] CheckEnergy: normalization coefficient = 46
[13.03|13:40:38] CheckEnergy: Actual Threshold
[13.03|13:40:38] CheckEnergy: dE/46 -0.0033 0.0100 OK!
[13.03|13:40:38] CheckEnergy: ddE/46 -0.0052 0.0020 Not OK! (relative to step10_attempt1_simulation:MDStep18500)
[13.03|13:40:38]
[13.03|13:40:38] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:40:38] CheckForces: ------------
[13.03|13:40:38] CheckForces: Reference job from step11_attempt1_reference_calc1
[13.03|13:40:38] CheckForces: Prediction job from final frame (MDStep20000) of step11_attempt1_simulation
[13.03|13:40:38] CheckForces: ------------
[13.03|13:40:38] CheckForces: Histogram of forces
[13.03|13:40:38] CheckForces: eV/Ang Ref Pred
[13.03|13:40:38] CheckForces: -2 2 0
[13.03|13:40:38] CheckForces: -1 64 67
[13.03|13:40:38] CheckForces: 0 71 71
[13.03|13:40:38] CheckForces: 1 1 0
[13.03|13:40:38] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:40:38] CheckForces: Force components with an error exceeding the threshold:
[13.03|13:40:38] CheckForces: Ref Pred Delta Threshold
[13.03|13:40:38] CheckForces: 0.06 -0.28 0.34 0.30
[13.03|13:40:38] CheckForces: -0.55 -0.06 0.49 0.33
[13.03|13:40:38] CheckForces: 1.08 0.28 0.79 0.36
[13.03|13:40:38] CheckForces: 0.57 -0.12 0.68 0.33
[13.03|13:40:38] CheckForces: -0.94 0.17 1.11 0.35
[13.03|13:40:38] CheckForces: 0.70 0.16 0.55 0.34
[13.03|13:40:38] CheckForces: ... and 6 more.
[13.03|13:40:38] CheckForces: Maximum deviation: 1.114 eV/angstrom
[13.03|13:40:38] CheckForces: Actual Threshold
[13.03|13:40:38] CheckForces: # > thr. 12 0 Not OK!
[13.03|13:40:38] CheckForces: MAE 0.108 0.30 OK!
[13.03|13:40:38] CheckForces: R^2 0.634 0.80 Not OK!
[13.03|13:40:38] CheckForces: --------------------
[13.03|13:40:38]
[13.03|13:40:39] Adding results from step11_attempt1_reference_calc1 to training set
[13.03|13:40:39] Current # training set entries: 71
[13.03|13:40:39] Current # validation set entries: 34
[13.03|13:40:39] Storing data in step11_attempt1_reference_data
[13.03|13:40:39] Deleting step10_attempt1_reference_data
[13.03|13:40:39] Deleting step11_attempt1_reference_calc1
[13.03|13:40:39]
[13.03|13:40:39] Current (cumulative) timings:
[13.03|13:40:39] Time (s) Fraction
[13.03|13:40:39] Ref. calcs 160.79 0.015
[13.03|13:40:39] ML training 8607.62 0.790
[13.03|13:40:39] Simulations 2125.46 0.195
[13.03|13:40:39]
[13.03|13:40:39]
[13.03|13:40:39]
[13.03|13:40:39] --- Begin summary ---
[13.03|13:40:39] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:40:39] 1 1 FAILED Inaccurate 0.4793
[13.03|13:40:39] 1 2 FAILED Inaccurate 0.4059
[13.03|13:40:39] 1 3 FAILED Inaccurate 0.4848
[13.03|13:40:39] 1 4 FAILED Inaccurate 0.6954
[13.03|13:40:39] 1 5 SUCCESS Accurate 0.2303
[13.03|13:40:39] 2 1 FAILED Inaccurate 0.6081
[13.03|13:40:39] 2 2 FAILED Inaccurate 0.6387
[13.03|13:40:39] 2 3 FAILED Inaccurate 0.4972
[13.03|13:40:39] 2 4 FAILED Inaccurate 0.4115
[13.03|13:40:39] 2 5 SUCCESS Accurate 0.2071
[13.03|13:40:39] 3 1 FAILED Inaccurate 0.6595
[13.03|13:40:39] 3 2 SUCCESS Accurate 0.2988
[13.03|13:40:39] 4 1 FAILED Inaccurate 0.4693
[13.03|13:40:39] 4 2 FAILED Inaccurate 0.3853
[13.03|13:40:39] 4 3 FAILED Inaccurate 0.9123
[13.03|13:40:39] 4 4 SUCCESS Accurate 0.2058
[13.03|13:40:39] 5 1 SUCCESS Accurate 0.2789
[13.03|13:40:39] 6 1 SUCCESS Accurate 0.2585
[13.03|13:40:39] 7 1 FAILED Inaccurate 0.5031
[13.03|13:40:39] 7 2 FAILED Inaccurate 0.4318
[13.03|13:40:39] 7 3 SUCCESS Accurate 0.1800
[13.03|13:40:39] 8 1 SUCCESS Accurate 0.3154
[13.03|13:40:39] 9 1 SUCCESS Accurate 0.2225
[13.03|13:40:39] 10 1 SUCCESS Accurate 0.3078
[13.03|13:40:39] 11 1 FAILED Inaccurate 1.1138
[13.03|13:40:39] --- End summary ---
[13.03|13:40:39]
[13.03|13:40:39] Running more reference calculations....
[13.03|13:40:40] Running reference calculations on frames [951, 976] from step11_attempt1_simulation/ams.rkf
[13.03|13:40:40] Calculating 2 frames in total
[13.03|13:40:40] Running step11_attempt1_reference_calc2
[13.03|13:40:43] Running step11_attempt1_reference_calc3
[13.03|13:40:46] Reference calculations finished!
[13.03|13:40:46] Adding results from step11_attempt1_reference_calc2 to validation set
[13.03|13:40:47] Adding results from step11_attempt1_reference_calc3 to training set
[13.03|13:40:47] Current # training set entries: 72
[13.03|13:40:47] Current # validation set entries: 35
[13.03|13:40:47] Storing data in step11_attempt1_reference_data
[13.03|13:40:47] Deleting step11_attempt1_reference_calc2
[13.03|13:40:47] Deleting step11_attempt1_reference_calc3
[13.03|13:40:47] Launching reparametrization job: step11_attempt1_training
[13.03|13:40:53] JOB m3gnet STARTED
[13.03|13:40:53] Starting m3gnet.prerun()
[13.03|13:40:53] m3gnet.prerun() finished
[13.03|13:40:53] JOB m3gnet RUNNING
[13.03|13:40:53] Executing m3gnet.run
[13.03|13:42:03] training_set Optimizer: 001 Epoch: 0 Loss: 0.001243
[13.03|13:42:03] validation_set Optimizer: 001 Epoch: 0 Loss: 0.013370
[13.03|13:42:39] training_set Optimizer: 001 Epoch: 10 Loss: 0.000055
[13.03|13:42:39] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005005
[13.03|13:43:15] training_set Optimizer: 001 Epoch: 20 Loss: 0.000051
[13.03|13:43:15] validation_set Optimizer: 001 Epoch: 20 Loss: 0.005222
[13.03|13:43:51] training_set Optimizer: 001 Epoch: 30 Loss: 0.000050
[13.03|13:43:51] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005004
[13.03|13:44:26] training_set Optimizer: 001 Epoch: 40 Loss: 0.000051
[13.03|13:44:26] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005645
[13.03|13:45:02] training_set Optimizer: 001 Epoch: 50 Loss: 0.000045
[13.03|13:45:02] validation_set Optimizer: 001 Epoch: 50 Loss: 0.008028
[13.03|13:45:38] training_set Optimizer: 001 Epoch: 60 Loss: 0.000045
[13.03|13:45:38] validation_set Optimizer: 001 Epoch: 60 Loss: 0.006586
[13.03|13:46:14] training_set Optimizer: 001 Epoch: 70 Loss: 0.000051
[13.03|13:46:14] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005215
[13.03|13:46:50] training_set Optimizer: 001 Epoch: 80 Loss: 0.000045
[13.03|13:46:50] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005659
[13.03|13:47:26] training_set Optimizer: 001 Epoch: 90 Loss: 0.000045
[13.03|13:47:26] validation_set Optimizer: 001 Epoch: 90 Loss: 0.005004
[13.03|13:48:02] training_set Optimizer: 001 Epoch: 100 Loss: 0.000045
[13.03|13:48:02] validation_set Optimizer: 001 Epoch: 100 Loss: 0.005229
[13.03|13:48:38] training_set Optimizer: 001 Epoch: 110 Loss: 0.000048
[13.03|13:48:38] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006265
[13.03|13:49:14] training_set Optimizer: 001 Epoch: 120 Loss: 0.000046
[13.03|13:49:14] validation_set Optimizer: 001 Epoch: 120 Loss: 0.006661
[13.03|13:49:50] training_set Optimizer: 001 Epoch: 130 Loss: 0.000056
[13.03|13:49:50] validation_set Optimizer: 001 Epoch: 130 Loss: 0.005890
[13.03|13:50:26] training_set Optimizer: 001 Epoch: 140 Loss: 0.000048
[13.03|13:50:26] validation_set Optimizer: 001 Epoch: 140 Loss: 0.005158
[13.03|13:51:02] training_set Optimizer: 001 Epoch: 150 Loss: 0.000048
[13.03|13:51:02] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005555
[13.03|13:51:38] training_set Optimizer: 001 Epoch: 160 Loss: 0.000047
[13.03|13:51:38] validation_set Optimizer: 001 Epoch: 160 Loss: 0.004919
[13.03|13:52:13] training_set Optimizer: 001 Epoch: 170 Loss: 0.000047
[13.03|13:52:13] validation_set Optimizer: 001 Epoch: 170 Loss: 0.006033
[13.03|13:52:48] training_set Optimizer: 001 Epoch: 180 Loss: 0.000051
[13.03|13:52:48] validation_set Optimizer: 001 Epoch: 180 Loss: 0.008440
[13.03|13:53:25] training_set Optimizer: 001 Epoch: 190 Loss: 0.000042
[13.03|13:53:25] validation_set Optimizer: 001 Epoch: 190 Loss: 0.007576
[13.03|13:53:59] Execution of m3gnet.run finished with returncode 0
[13.03|13:53:59] JOB m3gnet FINISHED
[13.03|13:53:59] Starting m3gnet.postrun()
[13.03|13:53:59] m3gnet.postrun() finished
[13.03|13:54:00] JOB m3gnet SUCCESSFUL
[13.03|13:54:21] Running all jobs through AMS....
[13.03|13:54:21] Storing results/optimization/training_set_results/best
[13.03|13:54:21] Storing results/optimization/validation_set_results/best
[13.03|13:54:21] PLAMS environment cleaned up successfully
[13.03|13:54:21] PLAMS run finished. Goodbye
[13.03|13:54:23] ParAMSResults training_set validation_set
[13.03|13:54:23] energy MAE 0.0555 0.0844 eV
[13.03|13:54:23] forces MAE 0.0156 0.0451 eV/angstrom
[13.03|13:54:23] Newly created parameter file/dir: step11_attempt1_training/results/optimization/m3gnet/m3gnet
[13.03|13:54:23] Done!
[13.03|13:54:23] Deleting step7_attempt2_training
[13.03|13:54:23] ###########################
[13.03|13:54:23] ### Step 11 / Attempt 2 ###
[13.03|13:54:23] ###########################
[13.03|13:54:23] MD Steps: 1500 (cumulative: 20000)
[13.03|13:54:23] Current engine settings:
[13.03|13:54:23]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step11_attempt1_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|13:54:23] Running step11_attempt2_simulation...
[13.03|13:55:37] Job step11_attempt2_simulation finished
[13.03|13:55:37] Deleting files that are no longer needed...
[13.03|13:55:38] Launching reference calculation
[13.03|13:55:41] Reference calculation finished!
[13.03|13:55:41] Checking success for step11_attempt2
[13.03|13:55:54] CheckEnergy: Checking energy for MDStep20000, n_atoms = 46
[13.03|13:55:54] CheckEnergy: normalization coefficient = 46
[13.03|13:55:54] CheckEnergy: Actual Threshold
[13.03|13:55:54] CheckEnergy: dE/46 -0.0039 0.0100 OK!
[13.03|13:55:54] CheckEnergy: ddE/46 -0.0031 0.0020 Not OK! (relative to step11_attempt1_simulation:MDStep20000)
[13.03|13:55:54]
[13.03|13:55:54] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|13:55:54] CheckForces: ------------
[13.03|13:55:54] CheckForces: Reference job from step11_attempt2_reference_calc1
[13.03|13:55:54] CheckForces: Prediction job from final frame (MDStep20000) of step11_attempt2_simulation
[13.03|13:55:54] CheckForces: ------------
[13.03|13:55:54] CheckForces: Histogram of forces
[13.03|13:55:54] CheckForces: eV/Ang Ref Pred
[13.03|13:55:54] CheckForces: -2 1 0
[13.03|13:55:54] CheckForces: -1 65 66
[13.03|13:55:54] CheckForces: 0 72 72
[13.03|13:55:54] CheckForces: 1 0 0
[13.03|13:55:54] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|13:55:54] CheckForces: Force components with an error exceeding the threshold:
[13.03|13:55:54] CheckForces: Ref Pred Delta Threshold
[13.03|13:55:54] CheckForces: -1.02 -0.55 0.46 0.36
[13.03|13:55:54] CheckForces: Maximum deviation: 0.464 eV/angstrom
[13.03|13:55:54] CheckForces: Actual Threshold
[13.03|13:55:54] CheckForces: # > thr. 1 0 Not OK!
[13.03|13:55:54] CheckForces: MAE 0.051 0.30 OK!
[13.03|13:55:54] CheckForces: R^2 0.863 0.80 OK!
[13.03|13:55:54] CheckForces: --------------------
[13.03|13:55:54]
[13.03|13:55:54] Adding results from step11_attempt2_reference_calc1 to training set
[13.03|13:55:54] Current # training set entries: 73
[13.03|13:55:54] Current # validation set entries: 35
[13.03|13:55:54] Storing data in step11_attempt2_reference_data
[13.03|13:55:55] Deleting step11_attempt1_reference_data
[13.03|13:55:55] Deleting step11_attempt2_reference_calc1
[13.03|13:55:55]
[13.03|13:55:55] Current (cumulative) timings:
[13.03|13:55:55] Time (s) Fraction
[13.03|13:55:55] Ref. calcs 170.34 0.014
[13.03|13:55:55] ML training 9423.86 0.799
[13.03|13:55:55] Simulations 2199.21 0.186
[13.03|13:55:55]
[13.03|13:55:55]
[13.03|13:55:55]
[13.03|13:55:55] --- Begin summary ---
[13.03|13:55:55] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|13:55:55] 1 1 FAILED Inaccurate 0.4793
[13.03|13:55:55] 1 2 FAILED Inaccurate 0.4059
[13.03|13:55:55] 1 3 FAILED Inaccurate 0.4848
[13.03|13:55:55] 1 4 FAILED Inaccurate 0.6954
[13.03|13:55:55] 1 5 SUCCESS Accurate 0.2303
[13.03|13:55:55] 2 1 FAILED Inaccurate 0.6081
[13.03|13:55:55] 2 2 FAILED Inaccurate 0.6387
[13.03|13:55:55] 2 3 FAILED Inaccurate 0.4972
[13.03|13:55:55] 2 4 FAILED Inaccurate 0.4115
[13.03|13:55:55] 2 5 SUCCESS Accurate 0.2071
[13.03|13:55:55] 3 1 FAILED Inaccurate 0.6595
[13.03|13:55:55] 3 2 SUCCESS Accurate 0.2988
[13.03|13:55:55] 4 1 FAILED Inaccurate 0.4693
[13.03|13:55:55] 4 2 FAILED Inaccurate 0.3853
[13.03|13:55:55] 4 3 FAILED Inaccurate 0.9123
[13.03|13:55:55] 4 4 SUCCESS Accurate 0.2058
[13.03|13:55:55] 5 1 SUCCESS Accurate 0.2789
[13.03|13:55:55] 6 1 SUCCESS Accurate 0.2585
[13.03|13:55:55] 7 1 FAILED Inaccurate 0.5031
[13.03|13:55:55] 7 2 FAILED Inaccurate 0.4318
[13.03|13:55:55] 7 3 SUCCESS Accurate 0.1800
[13.03|13:55:55] 8 1 SUCCESS Accurate 0.3154
[13.03|13:55:55] 9 1 SUCCESS Accurate 0.2225
[13.03|13:55:55] 10 1 SUCCESS Accurate 0.3078
[13.03|13:55:55] 11 1 FAILED Inaccurate 1.1138
[13.03|13:55:55] 11 2 FAILED Inaccurate 0.4641
[13.03|13:55:55] --- End summary ---
[13.03|13:55:55]
[13.03|13:55:55] Running more reference calculations....
[13.03|13:55:56] Running reference calculations on frames [951, 976] from step11_attempt2_simulation/ams.rkf
[13.03|13:55:56] Calculating 2 frames in total
[13.03|13:55:56] Running step11_attempt2_reference_calc2
[13.03|13:55:59] Running step11_attempt2_reference_calc3
[13.03|13:56:02] Reference calculations finished!
[13.03|13:56:02] Adding results from step11_attempt2_reference_calc2 to validation set
[13.03|13:56:03] Adding results from step11_attempt2_reference_calc3 to training set
[13.03|13:56:03] Current # training set entries: 74
[13.03|13:56:03] Current # validation set entries: 36
[13.03|13:56:03] Storing data in step11_attempt2_reference_data
[13.03|13:56:03] Deleting step11_attempt2_reference_calc2
[13.03|13:56:03] Deleting step11_attempt2_reference_calc3
[13.03|13:56:03] Launching reparametrization job: step11_attempt2_training
[13.03|13:56:09] JOB m3gnet STARTED
[13.03|13:56:09] Starting m3gnet.prerun()
[13.03|13:56:09] m3gnet.prerun() finished
[13.03|13:56:09] JOB m3gnet RUNNING
[13.03|13:56:09] Executing m3gnet.run
[13.03|13:57:21] training_set Optimizer: 001 Epoch: 0 Loss: 0.000951
[13.03|13:57:21] validation_set Optimizer: 001 Epoch: 0 Loss: 0.012707
[13.03|13:57:58] training_set Optimizer: 001 Epoch: 10 Loss: 0.000041
[13.03|13:57:58] validation_set Optimizer: 001 Epoch: 10 Loss: 0.005108
[13.03|13:58:34] training_set Optimizer: 001 Epoch: 20 Loss: 0.000039
[13.03|13:58:34] validation_set Optimizer: 001 Epoch: 20 Loss: 0.004799
[13.03|13:59:11] training_set Optimizer: 001 Epoch: 30 Loss: 0.000038
[13.03|13:59:11] validation_set Optimizer: 001 Epoch: 30 Loss: 0.005083
[13.03|13:59:48] training_set Optimizer: 001 Epoch: 40 Loss: 0.000038
[13.03|13:59:48] validation_set Optimizer: 001 Epoch: 40 Loss: 0.005064
[13.03|14:00:25] training_set Optimizer: 001 Epoch: 50 Loss: 0.000042
[13.03|14:00:25] validation_set Optimizer: 001 Epoch: 50 Loss: 0.004789
[13.03|14:01:02] training_set Optimizer: 001 Epoch: 60 Loss: 0.000047
[13.03|14:01:02] validation_set Optimizer: 001 Epoch: 60 Loss: 0.005904
[13.03|14:01:39] training_set Optimizer: 001 Epoch: 70 Loss: 0.000038
[13.03|14:01:39] validation_set Optimizer: 001 Epoch: 70 Loss: 0.005936
[13.03|14:02:16] training_set Optimizer: 001 Epoch: 80 Loss: 0.000048
[13.03|14:02:16] validation_set Optimizer: 001 Epoch: 80 Loss: 0.005665
[13.03|14:02:52] training_set Optimizer: 001 Epoch: 90 Loss: 0.000038
[13.03|14:02:52] validation_set Optimizer: 001 Epoch: 90 Loss: 0.004752
[13.03|14:03:30] training_set Optimizer: 001 Epoch: 100 Loss: 0.000041
[13.03|14:03:30] validation_set Optimizer: 001 Epoch: 100 Loss: 0.004848
[13.03|14:04:06] training_set Optimizer: 001 Epoch: 110 Loss: 0.000041
[13.03|14:04:06] validation_set Optimizer: 001 Epoch: 110 Loss: 0.006011
[13.03|14:04:43] training_set Optimizer: 001 Epoch: 120 Loss: 0.000048
[13.03|14:04:43] validation_set Optimizer: 001 Epoch: 120 Loss: 0.005184
[13.03|14:05:20] training_set Optimizer: 001 Epoch: 130 Loss: 0.000047
[13.03|14:05:20] validation_set Optimizer: 001 Epoch: 130 Loss: 0.007459
[13.03|14:05:57] training_set Optimizer: 001 Epoch: 140 Loss: 0.000038
[13.03|14:05:57] validation_set Optimizer: 001 Epoch: 140 Loss: 0.006187
[13.03|14:06:34] training_set Optimizer: 001 Epoch: 150 Loss: 0.000042
[13.03|14:06:34] validation_set Optimizer: 001 Epoch: 150 Loss: 0.005099
[13.03|14:07:10] training_set Optimizer: 001 Epoch: 160 Loss: 0.000040
[13.03|14:07:10] validation_set Optimizer: 001 Epoch: 160 Loss: 0.006542
[13.03|14:07:47] training_set Optimizer: 001 Epoch: 170 Loss: 0.000037
[13.03|14:07:47] validation_set Optimizer: 001 Epoch: 170 Loss: 0.005236
[13.03|14:08:24] training_set Optimizer: 001 Epoch: 180 Loss: 0.000057
[13.03|14:08:24] validation_set Optimizer: 001 Epoch: 180 Loss: 0.004990
[13.03|14:09:01] training_set Optimizer: 001 Epoch: 190 Loss: 0.000046
[13.03|14:09:01] validation_set Optimizer: 001 Epoch: 190 Loss: 0.006967
[13.03|14:09:36] Execution of m3gnet.run finished with returncode 0
[13.03|14:09:36] JOB m3gnet FINISHED
[13.03|14:09:36] Starting m3gnet.postrun()
[13.03|14:09:36] m3gnet.postrun() finished
[13.03|14:09:37] JOB m3gnet SUCCESSFUL
[13.03|14:09:59] Running all jobs through AMS....
[13.03|14:10:00] Storing results/optimization/training_set_results/best
[13.03|14:10:00] Storing results/optimization/validation_set_results/best
[13.03|14:10:00] PLAMS environment cleaned up successfully
[13.03|14:10:00] PLAMS run finished. Goodbye
[13.03|14:10:02] ParAMSResults training_set validation_set
[13.03|14:10:02] energy MAE 0.1285 0.1537 eV
[13.03|14:10:02] forces MAE 0.0168 0.0453 eV/angstrom
[13.03|14:10:02] Newly created parameter file/dir: step11_attempt2_training/results/optimization/m3gnet/m3gnet
[13.03|14:10:02] Done!
[13.03|14:10:02] Deleting step11_attempt1_training
[13.03|14:10:02] ###########################
[13.03|14:10:02] ### Step 11 / Attempt 3 ###
[13.03|14:10:02] ###########################
[13.03|14:10:02] MD Steps: 1500 (cumulative: 20000)
[13.03|14:10:02] Current engine settings:
[13.03|14:10:02]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step11_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|14:10:02] Running step11_attempt3_simulation...
[13.03|14:11:16] Job step11_attempt3_simulation finished
[13.03|14:11:16] Deleting files that are no longer needed...
[13.03|14:11:17] Launching reference calculation
[13.03|14:11:20] Reference calculation finished!
[13.03|14:11:20] Checking success for step11_attempt3
[13.03|14:11:31] CheckEnergy: Checking energy for MDStep20000, n_atoms = 46
[13.03|14:11:31] CheckEnergy: normalization coefficient = 46
[13.03|14:11:31] CheckEnergy: Actual Threshold
[13.03|14:11:31] CheckEnergy: dE/46 -0.0027 0.0100 OK!
[13.03|14:11:31] CheckEnergy: ddE/46 0.0018 0.0020 OK! (relative to step11_attempt2_simulation:MDStep20000)
[13.03|14:11:31]
[13.03|14:11:31] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[13.03|14:11:31] CheckForces: ------------
[13.03|14:11:31] CheckForces: Reference job from step11_attempt3_reference_calc1
[13.03|14:11:31] CheckForces: Prediction job from final frame (MDStep20000) of step11_attempt3_simulation
[13.03|14:11:31] CheckForces: ------------
[13.03|14:11:31] CheckForces: Histogram of forces
[13.03|14:11:31] CheckForces: eV/Ang Ref Pred
[13.03|14:11:31] CheckForces: -2 1 0
[13.03|14:11:31] CheckForces: -1 72 75
[13.03|14:11:31] CheckForces: 0 62 62
[13.03|14:11:31] CheckForces: 1 3 1
[13.03|14:11:31] CheckForces: Threshold for 0 force: 0.30 eV/angstrom
[13.03|14:11:31] CheckForces: All force components are within the acceptable error!
[13.03|14:11:31] CheckForces: Maximum deviation: 0.255 eV/angstrom
[13.03|14:11:31] CheckForces: Actual Threshold
[13.03|14:11:31] CheckForces: # > thr. 0 0 OK!
[13.03|14:11:31] CheckForces: MAE 0.043 0.30 OK!
[13.03|14:11:31] CheckForces: R^2 0.950 0.80 OK!
[13.03|14:11:31] CheckForces: --------------------
[13.03|14:11:31]
[13.03|14:11:31] Adding results from step11_attempt3_reference_calc1 to training set
[13.03|14:11:31] Current # training set entries: 75
[13.03|14:11:31] Current # validation set entries: 36
[13.03|14:11:31] Storing data in step11_attempt3_reference_data
[13.03|14:11:32] Deleting step11_attempt2_reference_data
[13.03|14:11:32] Deleting step11_attempt3_reference_calc1
[13.03|14:11:32]
[13.03|14:11:32] Current (cumulative) timings:
[13.03|14:11:32] Time (s) Fraction
[13.03|14:11:32] Ref. calcs 179.97 0.014
[13.03|14:11:32] ML training 10262.47 0.807
[13.03|14:11:32] Simulations 2273.00 0.179
[13.03|14:11:32]
[13.03|14:11:32]
[13.03|14:11:32] Step 11 finished successfully!
[13.03|14:11:32]
[13.03|14:11:32] --- Begin summary ---
[13.03|14:11:32] Step Attempt Status Reason finalframe_forces_max_delta
[13.03|14:11:32] 1 1 FAILED Inaccurate 0.4793
[13.03|14:11:32] 1 2 FAILED Inaccurate 0.4059
[13.03|14:11:32] 1 3 FAILED Inaccurate 0.4848
[13.03|14:11:32] 1 4 FAILED Inaccurate 0.6954
[13.03|14:11:32] 1 5 SUCCESS Accurate 0.2303
[13.03|14:11:32] 2 1 FAILED Inaccurate 0.6081
[13.03|14:11:32] 2 2 FAILED Inaccurate 0.6387
[13.03|14:11:32] 2 3 FAILED Inaccurate 0.4972
[13.03|14:11:32] 2 4 FAILED Inaccurate 0.4115
[13.03|14:11:32] 2 5 SUCCESS Accurate 0.2071
[13.03|14:11:32] 3 1 FAILED Inaccurate 0.6595
[13.03|14:11:32] 3 2 SUCCESS Accurate 0.2988
[13.03|14:11:32] 4 1 FAILED Inaccurate 0.4693
[13.03|14:11:32] 4 2 FAILED Inaccurate 0.3853
[13.03|14:11:32] 4 3 FAILED Inaccurate 0.9123
[13.03|14:11:32] 4 4 SUCCESS Accurate 0.2058
[13.03|14:11:32] 5 1 SUCCESS Accurate 0.2789
[13.03|14:11:32] 6 1 SUCCESS Accurate 0.2585
[13.03|14:11:32] 7 1 FAILED Inaccurate 0.5031
[13.03|14:11:32] 7 2 FAILED Inaccurate 0.4318
[13.03|14:11:32] 7 3 SUCCESS Accurate 0.1800
[13.03|14:11:32] 8 1 SUCCESS Accurate 0.3154
[13.03|14:11:32] 9 1 SUCCESS Accurate 0.2225
[13.03|14:11:32] 10 1 SUCCESS Accurate 0.3078
[13.03|14:11:32] 11 1 FAILED Inaccurate 1.1138
[13.03|14:11:32] 11 2 FAILED Inaccurate 0.4641
[13.03|14:11:32] 11 3 SUCCESS Accurate 0.2546
[13.03|14:11:32] --- End summary ---
[13.03|14:11:32]
[13.03|14:11:32] The engine settings for the final trained ML engine are:
[13.03|14:11:32]
Engine MLPotential
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir/crest_al/step11_attempt2_training/results/optimization/m3gnet/m3gnet
EndEngine
[13.03|14:11:32] Active learning finished!
[13.03|14:11:32] Goodbye!
[13.03|14:11:33] JOB crest_al FINISHED
[13.03|14:11:33] JOB crest_al SUCCESSFUL
# crest_al_job = SimpleActiveLearningJob.load_external("plams_workdir.003/crest_al")
new_retrained_model_settings = (
crest_al_job.results.get_params_job().results.get_production_engine_settings()
)
Generate conformers with the retrained M3GNet model and score with reference method¶
def generate_and_score(
molecule: plams.Molecule,
gen_name: str,
gen_settings: plams.Settings,
score_name: str,
score_settings: plams.Settings,
):
generate_job = ConformersJob(name=gen_name, molecule=molecule)
generate_job.settings.input.ams.Task = "Generate"
generate_job.settings.input.ams.Generator.Method = "RDKit"
generate_job.settings.input.ams.Generator.RDKit.InitialNConformers = 40
generate_job.settings.input += gen_settings.input
generate_job.run()
score_job = ConformersJob(name=score_name)
score_job.settings.input.ams.Task = "Score"
score_job.settings.input.ams.InputConformersSet = generate_job.results.rkfpath()
score_job.settings.input += score_settings.input
score_job.run()
molecules_gen = generate_job.results.get_conformers()
energies_gen = generate_job.results.get_relative_energies(unit="eV")
molecules_score = score_job.results.get_conformers()
energies_score = score_job.results.get_relative_energies(unit="ev")
return generate_job, molecules_gen, energies_gen, score_job, molecules_score, energies_score
(
generate_conformers_m3gnet_retrained_job,
molecules_pred,
energies_pred,
_,
molecules_ref,
energies_ref,
) = generate_and_score(
starting_structure,
gen_name="generate_conformers_m3gnet_retrained",
gen_settings=crest_al_job.results.get_production_engine_settings(),
score_name="score_conformers_ref2",
score_settings=ref_s,
)
[13.03|14:11:33] JOB generate_conformers_m3gnet_retrained STARTED
[13.03|14:11:33] JOB generate_conformers_m3gnet_retrained RUNNING
[13.03|14:13:02] JOB generate_conformers_m3gnet_retrained FINISHED
[13.03|14:13:02] JOB generate_conformers_m3gnet_retrained SUCCESSFUL
[13.03|14:13:02] JOB score_conformers_ref2 STARTED
[13.03|14:13:02] JOB score_conformers_ref2 RUNNING
[13.03|14:13:07] JOB score_conformers_ref2 FINISHED
[13.03|14:13:07] JOB score_conformers_ref2 SUCCESSFUL
print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred);
Ref_i Pred_i RMSD Ref_dE Pred_dE
0 1 0.0 0.00 0.00
1 8 0.0 0.04 0.06
2 4 0.0 0.05 0.02
3 7 0.0 0.05 0.04
4 6 0.0 0.07 0.03
5 9 0.0 0.07 0.06
6 2 0.0 0.07 0.02
7 16 0.0 0.08 0.11
8 0 0.0 0.08 -0.03
9 15 0.0 0.09 0.10
10 5 0.0 0.10 0.03
11 13 0.0 0.11 0.08
12 3 0.0 0.11 0.02
13 11 0.0 0.14 0.07
14 12 0.0 0.15 0.08
15 10 0.0 0.15 0.07
16 17 0.0 0.15 0.11
17 14 0.0 0.16 0.09
18 18 0.0 0.21 0.12
19 19 0.0 0.21 0.16
20 20 0.0 0.21 0.16
21 22 0.0 0.22 0.20
22 27 0.0 0.27 0.25
23 24 0.0 0.28 0.24
24 21 0.0 0.34 0.20
25 23 0.0 0.34 0.24
26 26 0.0 0.34 0.25
27 30 0.0 0.35 0.29
28 25 0.0 0.36 0.24
29 29 0.0 0.37 0.28
30 28 0.0 0.39 0.27
31 32 0.0 0.39 0.31
32 31 0.0 0.39 0.30
33 33 0.0 0.41 0.40
We can see a significant improvement compared to the M3GNet-UP-2022 results! However, the results are not perfect.
Generate conformers with the reference method and score with the retrained M3GNet model¶
As a second test, we can instead generate the conformers with the reference method and score them with the retrained m3gnet model.
This is quite expensive if the reference method is DFT!
if perform_expensive_tests:
generate_ref_job, molecules_ref, energies_ref, _, molecules_pred, energies_pred = (
generate_and_score(
starting_structure,
gen_name="generate_conformers_ref",
gen_settings=ref_s,
score_name="score_conformers_m3gnet_retrained",
score_settings=crest_al_job.results.get_production_engine_settings(),
)
)
print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred)
plot_conformers(generate_ref_job, 3)
[13.03|14:13:09] JOB generate_conformers_ref STARTED
[13.03|14:13:09] JOB generate_conformers_ref RUNNING
[13.03|14:13:54] JOB generate_conformers_ref FINISHED
[13.03|14:13:54] JOB generate_conformers_ref SUCCESSFUL
[13.03|14:13:54] JOB score_conformers_m3gnet_retrained STARTED
[13.03|14:13:54] JOB score_conformers_m3gnet_retrained RUNNING
[13.03|14:14:14] JOB score_conformers_m3gnet_retrained FINISHED
[13.03|14:14:14] JOB score_conformers_m3gnet_retrained SUCCESSFUL
Ref_i Pred_i RMSD Ref_dE Pred_dE
0 1 0.0 0.00 0.00
1 7 0.0 0.03 0.07
2 6 0.0 0.05 0.05
3 0 0.0 0.05 -0.00
4 5 0.0 0.05 0.05
5 3 0.0 0.05 0.02
6 4 0.0 0.09 0.04
7 12 0.0 0.10 0.09
8 15 0.0 0.10 0.11
9 9 0.0 0.11 0.08
10 13 0.0 0.11 0.10
11 11 0.0 0.11 0.09
12 19 0.0 0.11 0.16
13 8 0.0 0.12 0.07
14 2 0.0 0.12 0.01
15 16 0.0 0.14 0.13
16 10 0.0 0.14 0.09
17 14 0.0 0.15 0.11
18 18 0.0 0.15 0.16
19 17 0.0 0.16 0.14
20 20 0.0 0.18 0.17
21 21 0.0 0.20 0.18
22 22 0.0 0.21 0.22
23 32 0.0 0.24 0.32
24 30 0.0 0.24 0.30
25 23 0.0 0.24 0.24
26 25 0.0 0.25 0.27
27 24 0.0 0.28 0.25
28 26 0.0 0.28 0.27
29 28 0.0 0.29 0.28
30 27 0.0 0.29 0.28
31 29 0.0 0.31 0.29
32 31 0.0 0.32 0.30
33 35 0.0 0.32 0.41
34 33 0.0 0.33 0.35
35 34 0.0 0.33 0.35
36 36 0.0 0.42 0.45
Compare the RMSD between the different conformer sets¶
The Conformer “Score” task performs single-point calculations.
If we instead change the task to “Optimize” we can see how similar the reference-method-optimized conformers are to the retrained-M3GNet-optimized conformers by comparing the RMSD.
The below is quite computationally expensive for a DFT reference engine.
if perform_expensive_tests:
opt_conformers_ref_job2 = ConformersJob(name="opt_conformers_ref2")
opt_conformers_ref_job2.settings.input.ams.Task = "Optimize"
opt_conformers_ref_job2.settings.input.ams.InputConformersSet = (
generate_conformers_m3gnet_retrained_job.results.rkfpath()
)
opt_conformers_ref_job2.settings.input.ams.InputMaxEnergy = (
5.0 # only conformers in the lowest 5.0 kcal/mol = 0.2 eV
)
opt_conformers_ref_job2.settings.input += ref_s.input
opt_conformers_ref_job2.run()
molecules_ref = opt_conformers_ref_job2.results.get_conformers()
energies_ref = opt_conformers_ref_job2.results.get_relative_energies(unit="eV")
molecules_pred = generate_conformers_m3gnet_retrained_job.results.get_conformers()
energies_pred = generate_conformers_m3gnet_retrained_job.results.get_relative_energies(
unit="eV"
)
print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred)
[13.03|14:14:16] JOB opt_conformers_ref2 STARTED
[13.03|14:14:16] JOB opt_conformers_ref2 RUNNING
[13.03|14:14:37] JOB opt_conformers_ref2 FINISHED
[13.03|14:14:37] JOB opt_conformers_ref2 SUCCESSFUL
Ref_i Pred_i RMSD Ref_dE Pred_dE
0 1 0.7 0.00 0.00
1 6 rmsd > 0.7 ang.
2 4 0.3 0.04 0.02
3 8 0.1 0.05 0.06
4 7 0.2 0.05 0.04
5 0 0.2 0.07 -0.03
6 9 0.2 0.07 0.06
7 11 rmsd > 0.7 ang.
8 2 0.1 0.07 0.02
9 16 0.2 0.08 0.11
10 15 0.3 0.09 0.10
11 5 0.3 0.09 0.03
12 10 0.3 0.14 0.07
13 12 0.3 0.14 0.08
14 17 0.2 0.15 0.11
15 18 rmsd > 0.7 ang.
16 20 0.2 0.19 0.16
In the above table a few entries have “rmsd > 0.7 ang.”. This means that the reference geometry optimization causes the structure to change significantly compared to the retrained-m3gnet-optimized geometry.
In such cases it is not so meaningful to compare the relative energies between the reference and prediction, so those points are excluded from the table and from the plot.
Custom active learning loop: add newly generated conformers to training set¶
The Simple Active Learning module in AMS only works for MD simulations, so it cannot automatically add optimized conformers to the training or validation sets.
However, you can do it yourself!
The Conformers “Score” function does not store or calculate the forces. So let’s set up an AMS “Replay” job to recalculate the energies and forces to add to the training set:
replay_s = plams.Settings()
replay_s.input.ams.Task = "Replay"
replay_s.input.ams.Replay.File = generate_conformers_m3gnet_retrained_job.results.rkfpath()
replay_s.input.ams.Properties.Gradients = "Yes"
replay_s += ref_s
replay_job = plams.AMSJob(settings=replay_s, name="replay_new_conformers")
replay_job.run();
[13.03|14:14:37] JOB replay_new_conformers STARTED
[13.03|14:14:37] JOB replay_new_conformers RUNNING
[13.03|14:14:50] JOB replay_new_conformers FINISHED
[13.03|14:14:50] JOB replay_new_conformers SUCCESSFUL
Now import the data into a results importer:
ri = params.ResultsImporter.from_yaml(crest_al_job.results.get_reference_data_directory())
ri.add_trajectory_singlepoints(replay_job, properties=["energy", "forces"])
yaml_dir = "data_with_conformer_singlepoints_yaml"
ri.store(yaml_dir, backup=False)
['data_with_conformer_singlepoints_yaml/job_collection.yaml',
'data_with_conformer_singlepoints_yaml/results_importer_settings.yaml',
'data_with_conformer_singlepoints_yaml/training_set.yaml',
'data_with_conformer_singlepoints_yaml/validation_set.yaml']
Then launch ParAMS:
params_job = params.ParAMSJob.from_yaml(yaml_dir, name="params_with_conformer_singlepoints")
params_job.settings.input += ml_s.input.ams
params_job.settings.input.MachineLearning.LoadModel = (
crest_al_job.results.get_params_results_directory()
)
params_job.settings.input.Task = "MachineLearning"
params_job.settings.input.MachineLearning.LossCoeffs.Energy = 50
params_job.settings.input.MachineLearning.Target.Forces.Enabled = "No"
params_job.settings.input.MachineLearning.MaxEpochs = 100
params_job.run();
[13.03|14:14:52] JOB params_with_conformer_singlepoints STARTED
[13.03|14:14:52] JOB params_with_conformer_singlepoints RUNNING
[13.03|14:24:54] JOB params_with_conformer_singlepoints FINISHED
[13.03|14:24:54] JOB params_with_conformer_singlepoints SUCCESSFUL
If the job failed print the error message:
if not params_job.check():
print(params_job.get_errormsg())
Generate conformers with the new model and score with the reference method:
_, molecules_pred, energies_pred, _, molecules_ref, energies_ref = generate_and_score(
starting_structure,
gen_name="generate_conformers_m3gnet_retrained_again",
gen_settings=params_job.results.get_production_engine_settings(),
score_name="score_conformers_ref2",
score_settings=ref_s,
)
[13.03|14:24:54] JOB generate_conformers_m3gnet_retrained_again STARTED
[13.03|14:24:54] JOB generate_conformers_m3gnet_retrained_again RUNNING
[13.03|14:26:25] JOB generate_conformers_m3gnet_retrained_again FINISHED
[13.03|14:26:25] JOB generate_conformers_m3gnet_retrained_again SUCCESSFUL
[13.03|14:26:25] JOB score_conformers_ref2 STARTED
[13.03|14:26:25] Renaming job score_conformers_ref2 to score_conformers_ref2.002
[13.03|14:26:25] JOB score_conformers_ref2.002 RUNNING
[13.03|14:26:31] JOB score_conformers_ref2.002 FINISHED
[13.03|14:26:31] JOB score_conformers_ref2.002 SUCCESSFUL
And print/plot the results:
print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred);
Ref_i Pred_i RMSD Ref_dE Pred_dE
0 3 0.0 0.00 0.00
1 1 0.0 0.00 -0.05
2 4 0.0 0.00 0.01
3 6 0.0 0.04 0.03
4 5 0.0 0.05 0.02
5 7 0.0 0.06 0.03
6 9 0.0 0.06 0.06
7 2 0.0 0.08 -0.04
8 8 0.0 0.08 0.06
9 11 0.0 0.10 0.07
10 0 0.0 0.11 -0.05
11 12 0.0 0.11 0.08
12 10 0.0 0.14 0.07
13 13 0.0 0.15 0.09
14 17 0.0 0.16 0.16
15 15 0.0 0.16 0.12
16 14 0.0 0.17 0.12
17 18 0.0 0.19 0.17
18 16 0.0 0.20 0.16
19 20 0.0 0.21 0.21
20 22 0.0 0.21 0.22
21 25 0.0 0.23 0.26
22 19 0.0 0.24 0.21
23 21 0.0 0.24 0.22
24 23 0.0 0.25 0.24
25 24 0.0 0.27 0.25
26 26 0.0 0.29 0.27
27 28 0.0 0.30 0.31
28 27 0.0 0.32 0.30
29 29 0.0 0.39 0.34
30 30 0.0 0.40 0.41
31 31 0.0 0.51 0.46
Here we see even better agreement than before.
Conclusion: By manually adding retrained-ml-optimized conformers to the training set, you can improve the conformer prediction even more. This means to do your own “active learning” outside of the Simple Active Learning module in AMS.
Complete Python code¶
#!/usr/bin/env amspython
# coding: utf-8
# When running active learning it's usually a good idea to start off "simple" and make the system/structures gradually more complicated.
#
# For getting a model which predicts conformers accurately, we may take the following approach:
#
# * first train a potential at slightly above room temperature with NVT MD
#
# * continue training a second potential using CREST metadynamics
#
# * generate conformers with the previous model and also train to those
#
# ## Initial imports
import scm.plams as plams
import scm.params as params
from scm.simple_active_learning import SimpleActiveLearningJob
import matplotlib.pyplot as plt
from scm.external_engines.core import interface_is_installed
import os
import numpy as np
from typing import List
from scm.conformers import ConformersJob
from scm.conformers.plams.plot import plot_conformers
assert interface_is_installed("m3gnet"), "You must install the m3gnet backend before following this tutorial!"
plams.init()
# ## Create the initial structure
molecule = plams.from_smiles("OC(CC1c2ccccc2Sc2ccccc21)CN1CCCC1", forcefield="uff")
molecule.delete_all_bonds()
for at in molecule:
at.properties = plams.Settings()
plams.plot_molecule(molecule)
starting_structure = molecule
# ## Reference engine settings
fast_ref_s = plams.Settings()
fast_ref_s.input.DFTB.Model = "GFN1-xTB"
slow_ref_s = plams.Settings()
slow_ref_s.input.ADF.Basis.Type = "TZP"
slow_ref_s.input.ADF.Basis.Core = "None"
slow_ref_s.input.ADF.XC.Hybrid = "B3LYP"
slow_ref_s.input.ADF.XC.DISPERSION = "GRIMME3 BJDAMP"
# Change to slow_ref_s to train to B3LYP-D3(BJ) instead:
ref_s = fast_ref_s.copy()
# ref_s = slow_ref_s.copy()
perform_expensive_tests = ref_s == fast_ref_s
# ## Problem statement: Generate a few conformers with M3GNet-UP-2022 and Score with reference method
m3gnet_up_s = plams.Settings()
m3gnet_up_s.input.MLPotential.Model = "M3GNet-UP-2022"
generate_conformers_m3gnet_up_job = ConformersJob(name="generate_conformers_m3gnet_up", molecule=starting_structure)
generate_conformers_m3gnet_up_job.settings.input.ams.Task = "Generate"
generate_conformers_m3gnet_up_job.settings.input.ams.Generator.Method = "RDKit"
generate_conformers_m3gnet_up_job.settings.input.ams.Generator.RDKit.InitialNConformers = 40
# generate_conformers_m3gnet_up_job.settings += crest_al_job.results.get_production_engine_settings()
generate_conformers_m3gnet_up_job.settings += m3gnet_up_s
# generate_conformers_m3gnet_up_job.settings.runscript.nproc = 1 # run in serial, useful if you run out of memory when running M3GNet on the GPU
generate_conformers_m3gnet_up_job.run()
# Show the 4 most stable conformers with M3GNet-UP-2022:
plot_conformers(generate_conformers_m3gnet_up_job, 4, unit="eV")
# Score the generated conformers with the reference method:
score_conformers_ref_job = ConformersJob(name="score_conformers_ref")
score_conformers_ref_job.settings.input.ams.Task = "Score"
score_conformers_ref_job.settings.input.ams.InputConformersSet = generate_conformers_m3gnet_up_job.results.rkfpath()
score_conformers_ref_job.settings.input += ref_s.input
score_conformers_ref_job.run()
plot_conformers(score_conformers_ref_job, 4, unit="eV")
# Here we can see that the ordering of conformers at the reference level is completely different compared to M3GNet-UP-2022.
#
# The Score job reorders the conformers. To compare the energies more precisely, we can use the minimum pairwise RMSD (which should be 0) to identify how the order of conformers change:
def get_pairwise_rmsds(molecules_ref: List[plams.Molecule], molecules_pred: List[plams.Molecule]) -> np.ndarray:
"""Returns a len(molecules_ref)*len(molecules_pred) numpy array with pairwise rmsds (in angstrom) between the structures"""
rmsds = np.zeros((len(molecules_ref), len(molecules_pred)))
for i, ref_mol in enumerate(molecules_ref):
for j, pred_mol in enumerate(molecules_pred):
rmsds[i, j] = plams.Molecule.rmsd(ref_mol, pred_mol)
return rmsds
def get_reordering(rmsds: np.ndarray) -> np.ndarray:
"""
rmsds: numpy array with shape len(molecules_ref)*len(molecules_pred)
Returns a len(molecules_ref) integer numpy array.
The first element is the index (0-based) in molecules_pred corresponding to the first reference molecule, etc.
"""
rmsds = get_pairwise_rmsds(molecules_ref, molecules_pred)
reordering = np.argmin(rmsds, axis=1)
return reordering
def print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred, ax=None):
"""This functions prints the reordering table including relative energies. It also plots the predicted relative energies versus the reference relative energies"""
print(f"Ref_i Pred_i RMSD Ref_dE Pred_dE")
x, y = [], []
rmsds = get_pairwise_rmsds(molecules_ref, molecules_pred)
reordering = get_reordering(rmsds)
rmsd_threshold = 0.7 # angstrom, for printing/plotting points
for ref_i, pred_i in enumerate(reordering):
rmsd = rmsds[ref_i, pred_i]
ref_relative_e = energies_ref[ref_i]
pred_relative_e = energies_pred[pred_i] - energies_pred[reordering[0]]
if rmsd <= rmsd_threshold:
print(f"{ref_i:6d} {pred_i:6d} {rmsd:4.1f} {ref_relative_e:7.2f} {pred_relative_e:7.2f}")
x.append(ref_relative_e)
y.append(pred_relative_e)
else:
print(f"{ref_i:6d} {pred_i:6d} rmsd > {rmsd_threshold:.1f} ang.")
if ax is None:
_, ax = plt.subplots()
m, M = np.min([np.min(x), np.min(y)]), np.max([np.max(x), np.max(y)])
ax.plot(x, y, ".")
ax.plot([m, M], [m, M], "-")
ax.set_xlabel("ref. deltaE (eV)")
ax.set_ylabel("pred. deltaE (eV)")
return ax
# In the below table the energies are given with respect to the structure that had the lowest reference energy. This is thus different compared to the first image above, where the predicted (m3gnet-up-2022) relative energies were given to predicted lowest energy conformer.
molecules_ref = score_conformers_ref_job.results.get_conformers()
energies_ref = score_conformers_ref_job.results.get_relative_energies(unit="eV")
molecules_pred = generate_conformers_m3gnet_up_job.results.get_conformers()
energies_pred = generate_conformers_m3gnet_up_job.results.get_relative_energies(unit="ev")
ax = print_reordering_table(molecules_ref, molecules_pred, energies_ref, energies_pred)
ax.set_title("M3GNet-UP-2022 relative to reference method")
# Here we see that there is no correlation between the relative stabilities calculated with M3GNet-UP-2022 and the reference method.
#
# Example: The most stable conformer with the reference method (Ref_i=0) corresponds to the tenth most stable conformer with M3GNet-UP-2022 (Pred_i=10)
#
# Example 2: The second most stable conformer with the reference method (Ref_i=1) should be 0.08 eV *less* stable than the ref_i=0 conformer, but with M3GNet-UP-2022 method it is actually 0.01 eV *more* stable!
# ## Optimize a few conformers for initial reference data
#
# Conformers are local minima on the potential energy surface. The Active Learning MD will sample off-equilibrium structures. So let's make sure that there are at least some local minima in the training set by optimizing some of the generated conformers.
#
# Here, we loop over the conformers and run GeometryOptimization jobs. This produces output files in the normal AMS format that can easily be imported into ParAMS
max_N = min(6, len(molecules_pred)) # at most 6 optimizations
opt_s = plams.Settings()
opt_s.input.ams.Task = "GeometryOptimization"
opt_s.input.ams.GeometryOptimization.Convergence.Quality = "Basic"
opt_jobs = [
plams.AMSJob(settings=opt_s + ref_s, name=f"opt_{i}", molecule=mol) for i, mol in enumerate(molecules_pred[:max_N])
]
for opt_job in opt_jobs:
opt_job.run()
# Now import the data into a ParAMS results importer and store in the directory ``my_initial_reference_data``:
yaml_dir = os.path.abspath("my_initial_reference_data")
ri = params.ResultsImporter(settings={"units": {"energy": "eV", "forces": "eV/angstrom"}})
for opt_job in opt_jobs:
ri.add_singlejob(opt_job, properties=["energy", "forces"], task="SinglePoint")
ri.store(yaml_dir, backup=False)
# ## Simple Active Learning setup
#
# ### Molecular dynamics settings (temperature ramp)
nvt_md_s = plams.AMSNVTJob(
nsteps=20000,
timestep=0.5,
temperature=(270, 350, 350),
tau=100,
thermostat="Berendsen",
).settings
# ### ParAMS machine learning settings
ml_s = plams.Settings()
ml_s.input.ams.MachineLearning.Backend = "M3GNet"
ml_s.input.ams.MachineLearning.CommitteeSize = 1
ml_s.input.ams.MachineLearning.M3GNet.Model = "UniversalPotential"
ml_s.input.ams.MachineLearning.MaxEpochs = 200
# ### Active learning settings
#
# Conformer search of a single molecule is quite simple, so we can expect the ML method to perform quite well. We therefore decrease the success criteria thresholds a bit compared to the default values, to ensure that we get accurate results.
#
# Since we will immediately continue with another active learning loop, we disable the "RerunSimulation" as we are not interested in the MD simulation per se.
al_s = plams.Settings()
al_s.input.ams.ActiveLearning.Steps.Type = "Geometric"
al_s.input.ams.ActiveLearning.Steps.Geometric.Start = 10
al_s.input.ams.ActiveLearning.Steps.Geometric.NumSteps = 8
al_s.input.ams.ActiveLearning.InitialReferenceData.Load.Directory = yaml_dir
al_s.input.ams.ActiveLearning.InitialReferenceData.Generate.M3GNetShortMD.Enabled = "Yes"
al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Relative = 0.002
al_s.input.ams.ActiveLearning.SuccessCriteria.Forces.MaxDeviationForZeroForce = 0.30
al_s.input.ams.ActiveLearning.AtEnd.RerunSimulation = "No"
# ### Initial structure
#
# Let's start with the lowest-energy conformer according to the reference method:
starting_structure_al = molecules_ref[0]
# The bonds are not used, so delete them to make the input file less confusing:
starting_structure_al.delete_all_bonds()
# ### Complete job
settings = ref_s + nvt_md_s + ml_s + al_s
job = SimpleActiveLearningJob(settings=settings, molecule=starting_structure_al, name="sal")
print(job.get_input())
# ### Run the simple active learning job
job.run(watch=True)
# ### Final structure from MD simulation
#
new_structure = job.results.get_main_molecule()
plams.plot_molecule(new_structure, rotation="-5x,5y,0z")
# ## Second active learning job: CREST metadynamics
#
# Here we set Steps.Type = "Linear" to run reference calculations every 2000 MD steps.
nsteps = 20000
crest_md_s = plams.Settings()
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.Height = 0.138
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.NGaussiansMax = 50
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.NSteps = 200
crest_md_s.input.ams.MolecularDynamics.CRESTMTD.Width = 0.62
crest_complete_md_s = plams.AMSMDJob(
molecule=new_structure,
nsteps=nsteps,
settings=crest_md_s,
tau=10, # small time constant
thermostat="NHC",
temperature=300,
timestep=0.5,
samplingfreq=20,
).settings
# job = SimpleActiveLearningJob.load_external(plams.config.default_jobmanager.workdir + "/sal.002")
crest_ml_s = ml_s.copy()
crest_ml_s.input.ams.MachineLearning.LoadModel = job.results.get_params_results_directory()
crest_ml_s.input.ams.MachineLearning.Target.Forces.MAE = 0.04
crest_ml_s.input.ams.MachineLearning.MaxEpochs = 200
crest_al_s = plams.Settings()
crest_al_s.input.ams.ActiveLearning.Steps.Type = "Linear"
crest_al_s.input.ams.ActiveLearning.Steps.Linear.Start = 500
crest_al_s.input.ams.ActiveLearning.Steps.Linear.StepSize = 2000
crest_al_s.input.ams.ActiveLearning.InitialReferenceData.Load.FromPreviousModel = "Yes"
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Relative = 0.002
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Total = 0.010
# because we do not set Normalization, the above Energy criteria are energies per atom
# crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Energy.Normalization =
crest_al_s.input.ams.ActiveLearning.SuccessCriteria.Forces.MaxDeviationForZeroForce = 0.30
crest_al_s.input.ams.ActiveLearning.AtEnd.RerunSimulation = "No"
crest_al_s.input.ams.ActiveLearning.MaxReferenceCalculationsPerAttempt = 3
crest_al_job = SimpleActiveLearningJob(
name="crest_al",
settings=ref_s + crest_complete_md_s + crest_ml_s + crest_al_s,
molecule=new_structure,
)
crest_al_job.run(watch=True)
# crest_al_job = SimpleActiveLearningJob.load_external("plams_workdir.003/crest_al")
new_retrained_model_settings = crest_al_job.results.get_params_job().results.get_production_engine_settings()
# ## Generate conformers with the retrained M3GNet model and score with reference method
def generate_and_score(
molecule: plams.Molecule,
gen_name: str,
gen_settings: plams.Settings,
score_name: str,
score_settings: plams.Settings,
):
generate_job = ConformersJob(name=gen_name, molecule=molecule)
generate_job.settings.input.ams.Task = "Generate"
generate_job.settings.input.ams.Generator.Method = "RDKit"
generate_job.settings.inp