6.6. Task: MachineLearning¶
Set Task MachineLearning to fit machine learning potentials (ML
potentials). In ParAMS, all supported types of ML potentials can be trained as committee
models that provide an estimate of the uncertainty of predicted energies and
forces during production simulations.
Training ML Potentials through ParAMS requires
the job collection, and
training and validation sets
You can construct these using the results importers, just as for ReaxFF and DFTB parametrization.
Note
Unlike ReaxFF and DFTB parametrization, no Parameter Interface is needed. This is because ML potentials usually contain many thousands of parameters. It is typically not useful to manually control the values and ranges for all of those parameters.
You also need to specify which Backend to use, for example MatGL.
6.6.1. Requirements for job collection and data sets¶
The machine learning potentials are trained in a quite different way from how ParAMS trains ReaxFF and DFTB.
6.6.1.1. Only singlepoint calculations in the job collection¶
For ML potentials, only singlepoint calculations may enter the job collection. The original reference job can still be of any type (geometry optimization, PES Scan, …).
Example: if you import a DFT-calculated bond scan (PES Scan), you must import it using the “Add PESScan Singlepoints” option, not “add singlejob with Task=’PESScan’”.
Any jobs in the job collection with Task different from “SinglePoint” will be ignored.
6.6.1.2. Only single extractors in the training and validation sets¶
Similarly, for the training and validation sets, the expressions can only contain one extractor acting on a single job. This means that you cannot train reaction energies. Instead, you can (and should) train the total energy. As a result it is extra important that all reference data was calculated using a single level of theory.
When training forces, you must extract all force components from the job. However, depending on the backend, you may be able to set the force weights.
For task MachineLearning, only a small set of extractors (that act on singlepoint jobs) are supported:
energyforces
Examples:
Expression |
Task Optimization |
Task MachineLearning |
|---|---|---|
|
OK |
OK |
|
OK |
OK |
|
OK |
Not OK |
|
OK |
Not OK |
|
OK |
Not OK |
Expressions that do not follow the above requirements will be ignored during the ML training, but they will still be stored on disk. This means that if you after training your ML potential switch to the ParAMS SinglePoint Task, you can use any expressions and job tasks to test/validate/benchmark your trained potential.
6.6.1.3. The engine settings must be the same for all jobs¶
When you train for example DFTB, you can have different engine settings for different jobs. For example, you might want the k-space sampling to be different depending on the system.
However, when training machine learning potentials, you cannot set any job-dependent (structure-dependent) engine settings. Every job (structure) will use the same settings.
6.6.2. Machine Learning Input Structure¶
The input for the ParAMS Task MachineLearning is structured as follows:
MachineLearning has multiple backends that can be selected through the Backend key.
Each backend has a corresponding block (the same name as the value of the Backend key) with settings specific to that backend.
Additionally, there are several shared keywords, such as MaxEpochs that modify the behavior of the backends in the same way.
Each backend might support multiple models and has a corresponding block (the same name as the value of the Model key) with settings specific to that model.
For example the number of layers or how to initialize the parameters.
Some models consist of only a single key rather than a block. For example when a backend supports loading some file that contains model settings and parameters.
Any number of settings may exist at the top level of a backend block that are appropriate for all models.
The MachineLearning%LoadModel loads a previously fitted model from a ParAMS
results directory. The ParAMS results directory must contain the two
subdirectories optimization and settings_and_initial_data. Enabling
MachineLearning%LoadModel enforces the same Backend and
CommitteeSize as in the previous job and will ignore the model keys.
Instead it reads them from the previous ParAMS calculation. Any settings in
the model blocks are ignored. If any settings in the backend blocks are
incompatible with the loaded model then ParAMS will crash or behave unexpectedly.
The exact same backend and model settings are used for every committee member no matter the CommitteeSize, although the models can still be different due to stochastic effects (e.g. random parameters or a stochastic optimization algorithm).
When using LoadModel the committee from the previous calculation is used.
Set RunAMSAtEnd to run the job collection with the newly trained model once training is completed.
This will provide additional output such as scatter plots of prediction against reference values.
Tip
Learn using the ParAMS input for Task MachineLearning from the tutorials.
MachineLearning- Type:
Block
- Description:
Options for Task MachineLearning.
Backend- Type:
Multiple Choice
- Default value:
M3GNet
- Options:
[MatGL, M3GNet, MACE, NequIP]
- Description:
The backend to use. You must separately install the backend before running a training job.
MaxEpochs- Type:
Integer
- Default value:
1000
- Description:
Maximum number of epochs during the training.
EarlyStopping- Type:
Block
- Description:
Stop training when the validation loss has not improved sufficiently. Supported by the MatGL and M3GNet backends.
Enabled- Type:
Bool
- Default value:
Yes
- Description:
Whether to stop training after the validation loss stops improving.
MinDelta- Type:
Float
- Default value:
1e-05
- Description:
Minimum absolute decrease in validation loss that counts as an improvement. Must not be negative.
Patience- Type:
Integer
- Default value:
100
- Description:
Number of consecutive validation checks without a sufficient improvement before stopping. Must be greater than zero.
LossCoeffs- Type:
Block
- Description:
Modify the coefficients for the machine learning loss function. For backends that support weights, this is on top of the supplied dataset weights and sigmas.
AverageForcePerAtom- Type:
Bool
- Default value:
No
- Description:
For each force data entry, divide the loss contribution by the number of concomitant atoms. This is the same as the behavior for ParAMS Optimization, but it is turned off by default in Task MachineLearning. For machine learning, setting this to ‘No’ can be better since larger molecules will contribute more to the loss. For backends that support weights, this is on top of the supplied dataset weights and sigmas.
Energy- Type:
Float
- Default value:
10.0
- GUI name:
Energy coefficient
- Description:
Coefficient for the contribution of loss due to the energy. For backends that support weights, this is on top of the supplied dataset weights and sigmas.
Forces- Type:
Float
- Default value:
1.0
- GUI name:
Forces coefficient
- Description:
Coefficient for the contribution of loss due to the forces. For backends that support weights, this is on top of the supplied dataset weights and sigmas.
Target- Type:
Block
- Description:
Target values for stopping training. If both the training and validation metrics are smaller than the specified values, the training will stop early. Supported by the MatGL and M3GNet backends.
Forces- Type:
Block
- Description:
Forces (as reported by the backend)
Enabled- Type:
Bool
- Default value:
Yes
- Description:
Whether to use target values for forces.
MAE- Type:
Float
- Default value:
0.05
- Unit:
eV/angstrom
- Description:
MAE for forces (as reported by the backend).
LoadModel- Type:
String
- Description:
Load a previously fitted model from a ParAMS results directory. A ParAMS results directory should contain two subdirectories
optimizationandsettings_and_initial_data. The loaded model defines the model source and architecture; fitting controls such as the learning rate and trainable layers still apply when supported by the backend.
CommitteeSize- Type:
Integer
- Default value:
1
- Description:
The number of independently trained ML potentials.
RunAMSAtEnd- Type:
Bool
- Default value:
Yes
- GUI name:
Run AMS at end
- Description:
Whether to run the (committee) ML potential through AMS at the end. This will create the energy/forces scatter plots for the final trained model.
Early stopping is enabled by default for MatGL and M3GNet. It stops training
when the validation loss has not decreased by at least
EarlyStopping%MinDelta for EarlyStopping%Patience consecutive
validation checks. Both supported backends
validate once per epoch, so a validation check currently corresponds to one
epoch. The best validation-loss model is deployed rather than the model from
the final epoch. Early-stopping patience starts over when training a model
loaded from an earlier ParAMS results directory.
For MACE, ParAMS passes EarlyStopping%Patience to MACE’s native
--patience option. However, early stopping does not work with the AMS-bundled MACE version.
6.6.3. Backends: MatGL, M3GNet, MACE, …¶
6.6.3.1. Installation¶
The ML backends are not included by default with AMS or ParAMS, as they can be quite large. Before you can train an ML potential, you need to install the corresponding backend either through the AMS package manager or manually.
Tip
Before training a custom model with ParAMS, we recommend that you first test the ML backend in a production (for example, molecular dynamics or geometry optimization) simulation with some already created parameters. For example, follow the M3GNet GUI tutorial to make sure that the M3GNet backend has been installed correctly.
6.6.3.2. MatGL¶
The MatGL backend trains QET and TensorNet models. You can train a
custom model from scratch, fine-tune one of the four packaged PBE or r2SCAN
2025 models, or load a serialized MatGL Potential directory.
Set MatGL%Model directly to the packaged model name, for example
QET-PBE-2025.
MachineLearning- Type:
Block
- Description:
Options for Task MachineLearning.
MatGL- Type:
Block
- Description:
Options for fitting MatGL QET and TensorNet potentials.
Custom- Type:
Block
- Description:
Create a model from scratch with MatGL 4.0.2 constructor defaults.
Architecture- Type:
Multiple Choice
- Default value:
QET
- Options:
[QET, TensorNet]
- Description:
MatGL architecture to train.
M3GNet- Type:
Block
- Description:
M3GNet constructor settings.
Activation- Type:
Multiple Choice
- Default value:
swish
- Options:
[swish, tanh, sigmoid, softplus2, softexp]
- Description:
Activation function.
Cutoff- Type:
Float
- Default value:
5.0
- Unit:
angstrom
- Description:
Pair interaction cutoff.
DimEdgeEmbedding- Type:
Integer
- Default value:
64
- Description:
Edge embedding dimension.
DimNodeEmbedding- Type:
Integer
- Default value:
64
- Description:
Node embedding dimension.
MaxL- Type:
Integer
- Default value:
3
- Description:
Maximum angular basis order.
MaxN- Type:
Integer
- Default value:
3
- Description:
Maximum radial basis index.
NumBlocks- Type:
Integer
- Default value:
3
- Description:
Number of graph convolution blocks.
NumNeurons- Type:
Integer
- Default value:
64
- Description:
Number of neurons in each update layer.
ThreebodyCutoff- Type:
Float
- Default value:
4.0
- Unit:
angstrom
- Description:
Three-body interaction cutoff.
UsePhi- Type:
Bool
- Default value:
No
- Description:
Include the azimuthal angle in the three-body basis.
UseSmooth- Type:
Bool
- Default value:
No
- Description:
Use the smooth spherical Bessel basis.
QET- Type:
Block
- Description:
QET constructor settings.
Activation- Type:
Multiple Choice
- Default value:
swish
- Options:
[swish, tanh, sigmoid, softplus2, softexp]
- Description:
Activation function.
Cutoff- Type:
Float
- Default value:
5.0
- Unit:
angstrom
- Description:
Pair interaction cutoff.
EnvironmentDependentHardness- Type:
Bool
- Default value:
No
- Description:
Predict QET hardness from the local environment.
EquivarianceGroup- Type:
Multiple Choice
- Default value:
O(3)
- Options:
[O(3), SO(3)]
- Description:
Equivariance and invariance group.
MaxL- Type:
Integer
- Default value:
3
- Description:
Maximum angular basis order.
MaxN- Type:
Integer
- Default value:
3
- Description:
Maximum radial basis index.
NumBlocks- Type:
Integer
- Default value:
2
- Description:
Number of interaction blocks.
NumNeurons- Type:
Integer
- Default value:
64
- Description:
Hidden embedding size.
NumRBF- Type:
Integer
- Default value:
32
- Description:
Number of radial basis functions.
RBFType- Type:
Multiple Choice
- Default value:
Gaussian
- Options:
[Gaussian, SphericalBessel]
- Description:
Radial basis type.
TrainSigma- Type:
Bool
- Default value:
No
- Description:
Train QET Gaussian charge widths.
UseSmooth- Type:
Bool
- Default value:
No
- Description:
Use a smooth spherical Bessel basis.
Width- Type:
Float
- Default value:
0.5
- Description:
Gaussian radial basis width.
TensorNet- Type:
Block
- GUI name:
TensorNet
- Description:
TensorNet constructor settings.
Activation- Type:
Multiple Choice
- Default value:
swish
- Options:
[swish, tanh, sigmoid, softplus2, softexp]
- Description:
Activation function.
Cutoff- Type:
Float
- Default value:
5.0
- Unit:
angstrom
- Description:
Pair interaction cutoff.
EquivarianceGroup- Type:
Multiple Choice
- Default value:
O(3)
- Options:
[O(3), SO(3)]
- Description:
Equivariance and invariance group.
MaxL- Type:
Integer
- Default value:
3
- Description:
Maximum angular basis order.
MaxN- Type:
Integer
- Default value:
3
- Description:
Maximum radial basis index.
NumBlocks- Type:
Integer
- Default value:
2
- Description:
Number of interaction blocks.
NumNeurons- Type:
Integer
- Default value:
64
- Description:
Hidden embedding size.
NumRBF- Type:
Integer
- Default value:
32
- Description:
Number of radial basis functions.
RBFType- Type:
Multiple Choice
- Default value:
Gaussian
- Options:
[Gaussian, SphericalBessel]
- Description:
Radial basis type.
UseSmooth- Type:
Bool
- Default value:
No
- Description:
Use a smooth spherical Bessel basis.
Width- Type:
Float
- Default value:
0.5
- Description:
Gaussian radial basis width.
LearningRate- Type:
Float
- Default value:
0.001
- Description:
Initial learning rate for MatGL weight optimization. The LearningRateSchedule block controls how this rate changes during training.
LearningRateSchedule- Type:
Block
- Description:
Control how the MatGL learning rate changes during training and after a full-checkpoint restart.
FinalFactor- Type:
Float
- Default value:
0.01
- Description:
Final cosine learning rate as a fraction of LearningRate. Must be between 0 and 1. Example: if LearningRate is 0.01 and FinalFactor is 0.1, then the learning rate will decrease from 0.01 to 0.001 during the training.
Restart- Type:
Multiple Choice
- Default value:
Reset
- Options:
[Reset, Continue]
- Description:
For a full-checkpoint restart, Continue starts a new schedule from the checkpoint learning rate without increasing it. Reset starts from LearningRate. In both cases, MaxEpochs is the length of the new schedule. This option has no effect for a weights-only load.
Type- Type:
Multiple Choice
- Default value:
Cosine
- Options:
[Cosine, Constant]
- Description:
Cosine decreases the learning rate over MaxEpochs. Constant keeps it fixed.
Model- Type:
Multiple Choice
- Default value:
QET-PBE-2025
- Options:
[QET-PBE-2025, QET-r2SCAN-2025, TensorNet-PBE-M-2025, TensorNet-r2SCAN-M-2025, Custom, ModelDir]
- Description:
Select a packaged foundation model to fine-tune, create a custom architecture, or load a MatGL potential directory.
ModelDir- Type:
String
- Description:
Path to a MatGL potential directory containing model.json, model.pt, and state.pt.
TrainChargeHeads- Type:
Bool
- Default value:
No
- GUI name:
Train QET charge heads
- Description:
Whether QET electronegativity and hardness heads are optimized. The default, No, retains those heads from the loaded model while training the layers selected by TrainableLayers. Set this to Yes for a custom QET model trained from scratch so that its randomly initialized charge-head weights are trained. This option only affects QET models.
TrainableLayers- Type:
Multiple Choice
- Default value:
All
- Options:
[All, ReadoutOnly, LastBlockAndReadout]
- GUI name:
Trainable layers
- Description:
Select which neural-network parameters to optimize. All is the recommended default, trains the complete model, and should normally be used for a custom model trained from scratch. ReadoutOnly trains only the final energy-related output heads. LastBlockAndReadout additionally trains the final interaction block and its output projection. The analytically fitted elemental energy references are updated independently of this setting. This setting is reapplied when loading or restarting a model.
Choose a model based on the intended application:
QET-PBE-2025andQET-r2SCAN-2025include charge equilibration. ParAMS fits their energies and forces but does not require explicit atomic-charge labels.TensorNet-PBE-M-2025andTensorNet-r2SCAN-M-2025use the equivariant TensorNet architecture.
All MatGL models use MatGL’s universal element ordering. ParAMS rejects unsupported elements before training. A later restart can therefore add a supported element without resizing the embedding.
6.6.3.2.1. Learning-rate schedule¶
By default, MatGL uses cosine annealing. The first epoch uses
MatGL%LearningRate. The learning rate then decreases over
MachineLearning%MaxEpochs toward
LearningRate * LearningRateSchedule%FinalFactor. The default final factor
is 0.01. Set LearningRateSchedule%Type to Constant to disable this
decay.
A full-checkpoint restart starts a new schedule for the additional
MaxEpochs. By default, LearningRateSchedule%Restart is Reset and
the new phase starts from LearningRate. This is suitable for active
learning and other workflows that add training data before continuing. Set
Restart to Continue when extending training on an unchanged data set.
In that mode, the new phase starts from the checkpoint learning rate and never
increases it. The final rate is the smaller of the checkpoint rate and
LearningRate * FinalFactor.
6.6.3.2.2. Choosing which layers to train¶
MatGL%TrainableLayers controls how much of the neural network ParAMS
updates. To understand the choices, it is useful to divide a MatGL model into
two conceptual parts:
The representation and interaction layers convert elements, distances, angles, and the local atomic environments into learned atomic features. Successive interaction blocks refine these features by exchanging information between neighboring atoms.
The readout layers convert the final atomic features into energy-related quantities. The total energy is assembled from these outputs.
ReadoutOnlyFreezes all representation and interaction layers. Only the final energy-related output heads are optimized. The model therefore keeps its original description of atomic environments and only relearns how to map that description to energies.
This is the least flexible option, but it can be useful for a small data set that is similar to the model’s original training domain. It also reduces the number of optimized parameters. Although the representation is frozen, predicted forces can still change: forces are energy derivatives, and changing the readout changes the energy surface. The wall-clock speedup can be modest because force training must still differentiate through the frozen layers with respect to atomic coordinates.
LastBlockAndReadoutOptimizes the readout and the final interaction block. Earlier blocks remain frozen. The final block can adapt the learned atomic features to the new data before the readout converts them into energies.
This option is more flexible than
ReadoutOnlybut changes fewer parameters thanAll. It is a reasonable intermediate choice when the new structures differ somewhat from the foundation-model data, but the available training set is not large enough to justify updating the whole model.AllOptimizes the complete neural network. This is the default and generally the safest starting point for fine-tuning. Use
Allwhen training a custom model from scratch: freezing layers in a new model would preserve their random initial weights.
The exact groups selected by the two restricted presets are:
Architecture |
|
Additional groups in |
|---|---|---|
QET |
|
|
TensorNet |
|
|
6.6.3.2.3. Freezing the QET charge heads¶
By default, MatGL%TrainChargeHeads is No. This keeps the parameters of
QET’s electronegativity (chi_readout) and hardness
(hardness_readout) heads fixed at their loaded values. This setting is
applied after TrainableLayers, so it works with All, ReadoutOnly,
and LastBlockAndReadout. The option only affects QET models.
With All or LastBlockAndReadout, trainable upstream layers can still
change the features supplied to a frozen electronegativity head, and hence its
predictions. For a custom QET model trained from scratch, set
TrainChargeHeads Yes so that its randomly initialized charge-head weights
are trained.
Freezing these heads can be useful when fine-tuning only against energies and forces: QET charges are latent in such a fit, so an energy/force loss does not directly require the fitted charges to remain close to those of the foundation model. The option does not add charge labels or otherwise constrain the charge equilibration solution.
For example:
MachineLearning
Backend MatGL
MatGL
Model QET-PBE-2025
TrainableLayers LastBlockAndReadout
TrainChargeHeads No
End
End
For QET, the charge-equilibration quantities are part of the energy model.
Elemental energy references. MatGL writes the predicted total energy as
where \(n_Z\) is the number of atoms of element \(Z\) and \(\varepsilon_Z\) is its elemental reference energy. The reference energy depends only on the element, not on its atomic environment. It therefore provides a composition-dependent energy baseline, while the neural network learns bonding and environment-dependent energy differences.
ParAMS determines the elemental references by a linear least-squares fit to the training-set energies only. The validation set is not included in this fit. The fitted references remain fixed during gradient training:
For a model trained from scratch, ParAMS initially fits absolute elemental references to the training energies.
For a foundation model,
ModelDirmodel, orLoadModelcontinuation, ParAMS calculates the energy residuals of the loaded model and fits an elemental correction to those residuals. It adds this correction to the references already stored in the model.After training, ParAMS loads the best checkpoint and performs one more residual fit. The deployed model therefore includes a final correction for any approximately constant or composition-dependent energy offset left after neural-network training.
Elemental references do not contribute directly to forces because they are
constant when the atomic positions change without changing the composition.
They can correct a large energy offset without disturbing an already accurate
force field. They are not neural-network parameters and are updated
independently of TrainableLayers, including with ReadoutOnly.
The fit can determine separate elemental references only if the training set contains enough independent compositions. For example, a data set containing only one fixed stoichiometry determines the reference contribution for that stoichiometry, but not a unique reference for every constituent element. ParAMS uses a stable minimum-norm solution when the composition matrix is rank-deficient.
The implementation processes the training set in batches and accumulates only a small matrix whose dimensions are set by the number of supported elements. Its memory use therefore does not grow with the number of structures. The work scales linearly with the training-set size; fitting a residual correction for a loaded model requires one energy-inference pass over the training set.
The selected preset is also reapplied after loading a serialized model or a training checkpoint. For example:
MachineLearning
Backend MatGL
MatGL
Model QET-PBE-2025
TrainableLayers LastBlockAndReadout
End
End
6.6.3.2.4. Artifacts and production use¶
For a single model, the job directory is
results/optimization/matgl. It contains:
matgl/: the best model serialized as a standard MatGLPotential;best.ckpt: the checkpoint with the lowest validation total loss;last.ckpt: the complete final Lightning state used for continuation;metrics.csvandtraining_summary.json: stable epoch metrics and the last and best lifetime epoch numbers.
ParAMS automatically returns production settings with MLPotential Backend
MatGL and ParameterDir pointing to the serialized matgl/ directory.
Only energy and force labels are fitted. Stress and explicit QET charge labels
are not used.
6.6.3.2.5. Restarts¶
Set MachineLearning%LoadModel to a previous ParAMS MatGL results directory
to continue training. If last.ckpt is present, ParAMS restores model
weights, optimizer and scheduler state, the completed epoch, and random-number
generator state. It then starts the learning-rate phase selected by
LearningRateSchedule%Restart. MaxEpochs is the number of additional
epochs and also the length of this new learning-rate phase.
ParAMS copies the previous loss and statistics files and appends new rows with
monotonically increasing lifetime epoch numbers.
If the complete checkpoint is unavailable, ParAMS falls back to weights-only
fine-tuning of the exported best model. Packaged foundation models and
ModelDir inputs also start as weights-only fine-tuning because they do not
contain optimizer state.
6.6.3.3. M3GNet¶
The M3GNet backend trains and loads M3GNet-UP-2022.
MachineLearning- Type:
Block
- Description:
Options for Task MachineLearning.
M3GNet- Type:
Block
- Description:
Options for M3GNet fitting.
Custom- Type:
Block
- Description:
Specify a custom M3GNet model.
Cutoff- Type:
Float
- Default value:
5.0
- Unit:
angstrom
- Description:
Cutoff radius of the graph
MaxL- Type:
Integer
- Default value:
3
- Description:
Include spherical components up to order MaxL. Higher gives a better angular resolution, but increases computational cost substantially.
MaxN- Type:
Integer
- Default value:
3
- Description:
Include radial components up to the MaxN’th root of the spherical Bessel function. Higher gives a better radial resolution, but increases computational cost substantially.
NumBlocks- Type:
Integer
- Default value:
3
- GUI name:
Number of convolution blocks:
- Description:
Number of convolution blocks.
NumNeurons- Type:
Integer
- Default value:
64
- GUI name:
Number of neurons per layer
- Description:
Number of neurons in each layer.
ThreebodyCutoff- Type:
Float
- Default value:
4.0
- Unit:
angstrom
- Description:
Cutoff radius of the three-body interaction.
LearningRate- Type:
Float
- Default value:
0.001
- Description:
Learning rate for the M3GNet weight optimization.
Model- Type:
Multiple Choice
- Default value:
UniversalPotential
- Options:
[UniversalPotential, Custom, ModelDir]
- Description:
How to specify the model for the M3GNet backend. Either a Custom model can be made from scratch or an existing model directory can be loaded to obtain the model settings.
ModelDir- Type:
String
- Description:
Path to the directory defining the model. This folder should contain the files: ‘checkpoint’, ‘m3gnet.data-00000-of-00001’, ‘ m3gnet.index’ and ‘m3gnet.json’
UniversalPotential- Type:
Block
- Description:
Settings for (transfer) learning with the M3GNet Universal Potential.
Featurizer- Type:
Bool
- Default value:
No
- GUI name:
Train featurizer
- Description:
Train the Featurizer layer of the M3GNet universal potential.
Final- Type:
Bool
- Default value:
Yes
- GUI name:
Train final layer
- Description:
Train the Final layer of the M3GNet universal potential.
GraphLayer1- Type:
Bool
- Default value:
No
- GUI name:
Train layer 1 - graph
- Description:
Train the first Graph layer of the M3GNet universal potential.
GraphLayer2- Type:
Bool
- Default value:
No
- GUI name:
Train layer 2 - graph
- Description:
Train the second Graph layer of the M3GNet universal potential.
GraphLayer3- Type:
Bool
- Default value:
Yes
- GUI name:
Train layer 3 - graph
- Description:
Train the third Graph layer of the M3GNet universal potential.
ThreeDInteractions1- Type:
Bool
- Default value:
No
- GUI name:
Train layer 1 - 3D interactions
- Description:
Train the first ThreeDInteractions (three-body terms) layer of the M3GNet universal potential.
ThreeDInteractions2- Type:
Bool
- Default value:
No
- GUI name:
Train layer 2 - 3D interactions
- Description:
Train the second ThreeDInteractions (three-body terms) layer of the M3GNet universal potential.
ThreeDInteractions3- Type:
Bool
- Default value:
Yes
- GUI name:
Train layer 3 - 3D interactions
- Description:
Train the third ThreeDInteractions (three-body terms) layer of the M3GNet universal potential.
M3GNet produces the parameter directory <calculation name>.results/optimization/m3gnet/results/model which contains the parameterized model and can be used with the MLPotential engine. Set Backend M3GNet and ParameterDir to the path of the deployed model.
The M3GNet universal potential has the following architecture/structure:
Layer (type) |
Param # |
|---|---|
radius_cutoff_graph_converter (RadiusCutoffGraphConverter) |
0 (unused) |
graph_featurizer (GraphFeaturizer) |
6080 |
graph_update_func (GraphUpdateFunc) |
192 |
spherical_bessel_with_harmonics (SphericalBesselWithHarmonics) |
0 |
three_d_interaction (ThreeDInteraction) |
1737 |
three_d_interaction_1 (ThreeDInteraction) |
1737 |
three_d_interaction_2 (ThreeDInteraction) |
1737 |
graph_network_layer (GraphNetworkLayer) |
66432 |
graph_network_layer_1 (GraphNetworkLayer) |
66432 |
graph_network_layer_2 (GraphNetworkLayer) |
66432 |
pipe_24 (Pipe) |
16770 |
atom_ref_2 (AtomRef) |
0 |
Total params: 227,549
6.6.3.4. MACE¶
The options for MACE are:
MachineLearning- Type:
Block
- Description:
Options for Task MachineLearning.
MACE- Type:
Block
- Description:
Options for MACE fitting.
Custom- Type:
Block
- Description:
Specify a custom MACE model.
DataType- Type:
Multiple Choice
- Default value:
float64
- Options:
[float32, float64]
- Description:
Using
float32is faster but less accurate, and generally recommended for MD. Conversely usingfloat64is slower but more accurate, and recommended for geometry optimization.
LMax- Type:
Integer
- Default value:
1
- GUI name:
Max spherical harmonic order
- Description:
Maximum spherical harmonic order of the messages in the message passing step; 0 is invariant, 1 and 2 are higher order (equivariant). Defaults to
1. A higher value increases accuracy, but increases model size and computational cost.
NumChannels- Type:
Integer
- Default value:
128
- GUI name:
Number of channels
- Description:
Number of channels per angular momentum order. Defaults to
128. Set to64for a smaller model or256for a larger model. A higher value increases accuracy, but increases model size and computational cost.
RMax- Type:
Float
- Default value:
5.0
- Unit:
angstrom
- GUI name:
Distance cutoff
- Description:
Distance cutoff for interactions.
Foundation- Type:
Block
- Description:
Settings for (transfer) learning with the MACE foundation model.
Type- Type:
Multiple Choice
- Default value:
MACE-MPA-0
- Options:
[MACE-MPA-0, MACE-MP-0-Large, MACE-MP-0-Medium, MACE-MP-0-Small]
- Description:
Which MACE foundation model to use.
LearningRate- Type:
Float
- Default value:
0.01
- Description:
Learning rate for the MACE weight optimization
LoRA- Type:
Block
- Description:
Settings for LoRA (Low-Rank Adaptation) fine-tuning. LoRA freezes the base model weights and trains only small low-rank adapters, reducing overfitting and catastrophic forgetting when fine-tuning a foundation model on a small dataset. Only applied when fine-tuning a foundation/restart model.
Alpha- Type:
Float
- Default value:
1.0
- GUI name:
LoRA alpha
- Description:
Scaling factor for the LoRA update. The effective scaling applied to the low-rank path is Alpha / Rank.
Enabled- Type:
Bool
- Default value:
Yes
- GUI name:
LoRA enabled
- Description:
Whether to enable LoRA fine-tuning, defaults to
True.
Rank- Type:
Integer
- Default value:
4
- GUI name:
LoRA rank
- Description:
Rank of the LoRA matrices. Higher rank increases capacity and the number of trainable parameters.
LossCoeffs- Type:
Block
- Description:
Modify the coefficients for the machine learning loss function, by applying scale factors to the MachineLearning%LossCoeffs values.
EnergyScaleFactor- Type:
Float
- Default value:
0.1
- Description:
Scale factor to apply to the energy loss coefficient, i.e. EnergyScaleFactor * MachineLearning%LossCoeffs%Energy.
ForcesScaleFactor- Type:
Float
- Default value:
100.0
- Description:
Scale factor to apply to the forces loss coefficient i.e. ForcesScaleFactor * MachineLearning%LossCoeffs%Forces.
Model- Type:
Multiple Choice
- Default value:
Foundation
- Options:
[Foundation, Custom, ModelFile]
- Description:
How to specify the model for the MACE backend. A foundation model can be used, a custom model can be made from scratch or an existing model file can be loaded to obtain the model settings.
ModelFile- Type:
String
- Description:
Path to the .model file defining the model.
StageTwo- Type:
Block
- Description:
Settings for stage two of training.
Enabled- Type:
Bool
- Default value:
Yes
- GUI name:
Stage two enabled
- Description:
Whether to enable stage two training, defaults to
True.
LearningRate- Type:
Float
- Default value:
0.001
- GUI name:
Stage two learning rate
- Description:
Learning rate for the MACE weight optimization for stage two training
LossCoeffs- Type:
Block
- Description:
Modify the coefficients for the machine learning loss function for stage two training, by applying scale factors to the MachineLearning%LossCoeffs values.
EnergyScaleFactor- Type:
Float
- Default value:
100.0
- GUI name:
Stage two energy scale factor
- Description:
Scale factor to apply to the energy loss coefficient for stage two training, i.e. EnergyScaleFactor * MachineLearning%LossCoeffs%Energy.
ForcesScaleFactor- Type:
Float
- Default value:
100.0
- GUI name:
Stage two forces scale factor
- Description:
Scale factor to apply to the forces loss coefficient for stage two training i.e. ForcesScaleFactor * MachineLearning%LossCoeffs%Forces.
Start- Type:
Float
- Default value:
0.8
- GUI name:
Stage two start
- Description:
When to start stage two training, as a proportion of the MachineLearning%MaxEpochs
6.6.3.5. NequIP¶
Important
Training NequIP potentials with ParAMS is not a fully supported feature. To use NequIP with AMS, or to train NequIP with ParAMS, you need to manually install it into the AMS Python environment.
SCM does not provide any packages for NequIP and cannot provide support for the installation. But we have compiled some helpful tips in the Engine ASE documentation that may help you with the installation.
The options for NequIP are:
MachineLearning- Type:
Block
- Description:
Options for Task MachineLearning.
NequIP- Type:
Block
- Description:
Options for NequIP fitting.
Custom- Type:
Block
- Description:
Specify a custom NequIP model.
LMax- Type:
Integer
- Default value:
1
- Description:
Maximum L value. 1 is probably high enough.
MetricsKey- Type:
Multiple Choice
- Default value:
validation_loss
- Options:
[training_loss, validation_loss]
- Description:
Which metric to use to generate the ‘best’ model.
NumLayers- Type:
Integer
- Default value:
4
- Description:
Number of interaction layers in the NequIP neural network.
RMax- Type:
Float
- Default value:
3.5
- Unit:
angstrom
- GUI name:
Distance cutoff
- Description:
Distance cutoff for interactions.
LearningRate- Type:
Float
- Default value:
0.005
- Description:
Learning rate for the NequIP weight optimization
Model- Type:
Multiple Choice
- Default value:
Custom
- Options:
[Custom, ModelFile]
- Description:
How to specify the model for the NequIP backend. Either a Custom model can be made from scratch or an existing ‘model.pth’ file can be loaded to obtain the model settings.
ModelFile- Type:
String
- Description:
Path to the model.pth file defining the model.
UseRescalingFromLoadedModel- Type:
Bool
- Default value:
Yes
- Description:
When loading a model with LoadModel or NequiP%ModelFile do not recalculate the dataset rescaling but use the value from the loaded model.
NequIP produces the file <calculation name>.results/optimization/nequip/results/model.pth which contains the deployed model and can be used with the MLPotential engine. Set Backend NequIP and ParameterFile to the path of the deployed model.
6.6.4. ML Parallelization¶
Parallelization options can be set with ParallelLevels.
Note that Task MachineLearning does not perform AMS jobs during optimization, so the parallelization options are different.
Select the maximum number of parallel committee members with CommitteeMembers or set it to zero to run all committee members in parallel (up to the maximum number of cores or the NSCM environment variable). Select the number of cores each committee is allowed to use with Cores or set it to zero (default) to evenly distribute the available cores over the committee members running in parallel.
Some backends may spawn additional threads for database management, but they should not be using substantial CPU time. GPU offloading is supported through TensorFlow or PyTorch depending on the backend. Currently there are no settings available in ParAMS for GPU offloading; the backends use GPU resources according to their documentation.
ParallelLevels- Type:
Block
- GUI name:
Parallelization distribution:
- Description:
Distribution of threads/processes between the parallelization levels.
CommitteeMembers- Type:
Integer
- Default value:
1
- GUI name:
Number of parallel committee members
- Description:
Maximum number of committee member optimizations to run in parallel. If set to zero will take the minimum of MachineLearning%CommitteeSize and the number of available cores (NSCM)
Cores- Type:
Integer
- Default value:
0
- GUI name:
Processes (per Job)
- Description:
Number of cores to use per committee member optimization. By default (0) the available cores (NSCM) divided equally among committee members. When using GPU offloading, consider setting this to 1.