Appendix D: Advanced package manager usage
******************************************


Overview
========

AMSpackages helps you maintain optional components of the Amsterdam
Modeling Suite. Features installed through AMSpackages integrate with
both the GUI and command line programs, and can be managed through
either interface. All package installs require a valid license.

Note:

  AMSpackages behavior changed significantly in AMS2026. This page
  describes the package manager for AMS2026 and later. If you are
  using AMS2025 or an older release, see the documentation for
  previous AMS versions.

New in version AMS2026.101:

* The default behavior of AMSpackages now installs packages for all
  users.

* The default location for shared packages is now inside "AMSHOME" on
  all platforms.

* A "--single-user"/"-u" flag was added to fall back to the old per-
  user behavior.


Managing packages from the command line
=======================================

On the command line, users can use ""${AMSBIN}/amspackages"" to access
the package manager interface. There are several subcommands that can
be used to access several features such as installing, updating, or
removing packages. Below the most commonly used options are explained.

To see a full list of options you can use ""${AMSBIN}/amspackages"
--help". Any of the subcommands may also have specific options, which
can be seen when using "--help" behind the subcommand. For instance,
for ""${AMSBIN}/amspackages" install --help".


Helpful commands and global options
===================================

Besides the main install, update, and remove operations, the following
commands are often useful when working with AMSpackages from the
command line:

* ""${AMSBIN}/amspackages" list" shows available and installed
  packages

* ""${AMSBIN}/amspackages" check lfdft" checks whether a package is
  installed

* ""${AMSBIN}/amspackages" loc lfdft" prints the install location of a
  package

* ""${AMSBIN}/amspackages" clean" removes cached downloads and
  leftovers from aborted installations

* ""${AMSBIN}/amspackages" debug" prints debugging information in
  markdown format

* ""${AMSBIN}/amspackages" env" shows the environment variables
  exported by AMSpackages

Useful global options include:

* "-v" or "--verbose" for more detailed output. You can provide this
  option multiple times.

* "--repository" to use a specific online or local repository
  definition file

* "--timeout" to change the network timeout for downloads

* "--no-cache" to skip cached repository data

* "--no-shared" to ignore packages installed in the shared location

For example:

   "${AMSBIN}/amspackages" -v list --format wide
   "${AMSBIN}/amspackages" --repository /path/to/AMS2026.1.yml list
   "${AMSBIN}/amspackages" --no-cache check lfdft


Finding available packages
==========================

New in version AMS2024.101.

* The "display" command was renamed to "list".

* "yaml" was added as an available format.

The "list" command will show available packages.

If you wish to refer to a package, you can use the "ID" to install,
remove it, or check if it is installed. By default, list provides a
long list of packages, but a wide view is also available.

For example:

   > "${AMSBIN}/amspackages" list
   AMSpackages Overview
   Date: Tue May 11 12:07:01 2021
   Repository: https://downloads.scm.com/Downloads/packages/
   Package directory: /home/user/.scm/packages/AMS2021.1.packages
   Name                                                [id]|     Instl. Version     |   Disk usage   |     Avail. Version     | Licensed? | Req. licenses
   ========================================================|========================|================|========================|===========|================
   ADFCRS-2018 Database                            [adfcrs]|                        |                |     2018: build 1      |    YES    | CRS 2021.1
   LFDFT atomic database                            [lfdft]|      1.0: build 0      |    1.42 GiB    |      1.0: build 0      |    YES    | ADF 2021.1
   All ML Potential backends                 [mlpotentials]|                        |                |     1.0.0: build 0     |    YES    | MLPOT 2021.1
   PiNN ML backend                                   [pinn]|                        |                |     0.3.1: build 0     |    YES    | MLPOT 2021.1
   Quantum ESPRESSO                                    [qe]|                        |                |      6.3: build 1      |    YES    |
   SchNetPack ML backend                       [schnetpack]|                        |                |     0.3.1: build 0     |    YES    | MLPOT 2021.1
   sGDML ML backend                                 [sgdml]|                        |                |     0.4.4: build 0     |    YES    | MLPOT 2021.1
   TorchANI ML backend                           [torchani]|                        |                |     2.2.0: build 0     |    YES    | MLPOT 2021.1

Alternatively, machine parsable output can be obtained using "--
format=yaml".

The "list" command also supports "--format=long" and "--format=yaml".
The wide format is convenient for terminal use, while yaml is useful
in scripts and automated checks.


Available AMSpackages
^^^^^^^^^^^^^^^^^^^^^

+-----------------------------------+-----------------------------------+-----------------------------------+
| Package name                      | Package ID                        | Available version                 |
|===================================|===================================|===================================|
| ADFCRS-2018 Database              | adfcrs                            | 2018: build 5                     |
+-----------------------------------+-----------------------------------+-----------------------------------+
| AIMNet2 Environment (CPU Only)    | aimnet2-cpu                       | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| AIMNet2 Environment (CUDA 12.8)   | aimnet2-cu128                     | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| FAIRChem Environment (CPU Only)   | fairchem-cpu                      | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| FAIRChem Environment (CUDA 12.8)  | fairchem-cu128                    | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| Infretis Environment              | infretis                          | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| LFDFT atomic database             | lfdft                             | 1.0: build 0                      |
+-----------------------------------+-----------------------------------+-----------------------------------+
| M3GNet Environment                | m3gnet                            | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| MACE Environment (CPU Only)       | mace-cpu                          | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| MACE Environment (CUDA 12.8)      | mace-cu128                        | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| Subgraph Sigma Profile Estimation | molsg_sg1db                       | 1.0: build 3                      |
| (SG1) Database                    |                                   |                                   |
+-----------------------------------+-----------------------------------+-----------------------------------+
| NequIP Environment (CPU Only)     | nequip-cpu                        | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| NequIP Environment (CUDA 12.8)    | nequip-cu128                      | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| OLED material database            | oledmatdb                         | 2023.1: build 2                   |
+-----------------------------------+-----------------------------------+-----------------------------------+
| Quantum ESPRESSO (AMSPIPE)        | qe                                | 7.1: build 206                    |
+-----------------------------------+-----------------------------------+-----------------------------------+
| TorchANI Environment (CPU Only)   | torchani-cpu                      | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| TorchANI Environment (CUDA 12.8)  | torchani-cu128                    | 1.0.0: build 1207                 |
+-----------------------------------+-----------------------------------+-----------------------------------+
| Zacros-post                       | zacros_post                       | 1.02: build 2                     |
+-----------------------------------+-----------------------------------+-----------------------------------+


Installing a package
====================

To install a package, use the "install" command and specify the ID of
the package.

   "${AMSBIN}/amspackages" install lfdft

Multiple packages can be specified on the same line for installation.
If you wish to install all available packages, *all* can be used as a
shortcut.

   "${AMSBIN}/amspackages" install all

By default, AMSpackages installs packages in shared/admin mode inside
"AMSHOME". If "AMSHOME" is not writable, it automatically falls back
to single-user mode. To force single-user installation, use "-u" or "
--single-user":

   "${AMSBIN}/amspackages" --single-user install lfdft

Some packages such as the Machine Learning potentials are installed
into your amspython environment. Other packages are installed in the
shared package location inside "AMSHOME" by default.

In single-user mode, the default package locations are:

* for Linux users: "$HOME/.scm/packages"

* for MacOS users: "$HOME/Library/Application Support/SCM/packages"

* for Windows users: "%LOCALAPPDATA%/SCM/packages"


Installing a package from a script
==================================

If you want to install a package non-interactively, for instance
inside of a batch script, use the "--yes" option after the install
command. This will answer yes to any interactive prompts.

   "${AMSBIN}/amspackages" install --yes lfdft

As an alternative you can set an environment variable. Add "export
SCM_AMSPKGS_NONINTERACTIVE=True" to your script to answer yes to all
prompts.

Other useful install options are:

* "--force-reinstall" to reinstall an already installed package

* "--downgrade" to allow installation of an older available version

* "--force" to install despite user modifications in the target
  package directory


Check if a package is installed
===============================

If you want to make sure that a package is installed, you can use the
"check" command.

   "${AMSBIN}/amspackages" check lfdft

The exit status will be 0 if a package is installed, and non-zero
otherwise. For detailed information, you can pass the verbose ("-v")
flag. This will tell you what version is installed.

   > amspackages -v check lfdft
   04-20 17:26:28 lfdft is installed: Ligand Field DFT v[1.0] - build:0 [974661607e9f46c78b6d875e12edfb7a]

Additional check options are available for stricter verification:

* "--modifications" fails if the package was changed externally

* "--licenses" also validates license availability and version
  compatibility

* "--pip" checks for missing Python packages using pip

For example:

   "${AMSBIN}/amspackages" check --licenses --modifications lfdft


Removing a package
==================

Removing a package is done by using the "remove" command. This command
can take a list of packages as arguments. For example, to remove the
Ligand Field DFT package use:

   "${AMSBIN}/amspackages" remove lfdft

While we advise against it, there may be times when a user has
accidentally or intentionally made changes or saved new files inside
the directories of one of the packages. When trying to remove such a
package, to prevent data loss you will be warned that a package is
modified. In order to remove packages despite modifications, use the "
--force" flag.

   "${AMSBIN}/amspackages" remove --force lfdft


Updating a package
==================

At times a newer version or build of a package may be available. To
install an update, use the "update" command.

   "${AMSBIN}/amspackages" update lfdft

You can also run "update" without package arguments to update all
installed packages.

Note:

  The update mechanism usually removes user modifications. Therefore
  just like the remove command, you will have to specify the "--force"
  flag to update such packages.

If you install a new version of AMS, reinstall any optional packages
in that new installation instead of copying them from an older AMS
version.

Other useful update options are:

* "--fix" or "--fix-unsupported" to replace unsupported package
  versions, for example after manual pip changes

* "--downgrade" to allow updating to an older available version when
  needed

* "--yes" for non-interactive updates


Locating an installed package
=============================

If you need the exact installation path of a package, use the "loc"
command.

   "${AMSBIN}/amspackages" loc lfdft

This is useful for troubleshooting, verifying where a package ended
up, or locating package files manually.


Cleaning cached files
=====================

AMSpackages can keep downloaded archives, uv cache data, and partially
installed package directories. To reclaim disk space or clean up after
interrupted operations, use:

   "${AMSBIN}/amspackages" clean

You can keep selected caches by using:

* "--no-downloads" to keep downloaded package files

* "--no-uv-cache" to keep the uv cache

* "--no-installs" to avoid cleaning package folders left by aborted
  installs


Debugging and environment inspection
====================================

For support and troubleshooting, the following commands are useful:

* ""${AMSBIN}/amspackages" debug" prints a markdown report with
  debugging information

* ""${AMSBIN}/amspackages" debug --no-log" omits the appended log file

* ""${AMSBIN}/amspackages" env" shows the environment variables
  exported in the pkgrc file

These commands are especially useful when preparing information for
SCM support or when diagnosing configuration issues in shared
installations.


Using a local mirror with AMSpackages
=====================================

If you need to create or update the local mirror itself, see Appendix
E: Mirroring the SCM package repository. This section describes only
how to point AMSpackages to a mirror that already exists.

Recommended:

Place your downloaded copy of the AMS repository in AMSHOME. If the
root file is located at: "${AMSHOME}/repository/AMS2026.yml",
amspackages will automatically recognize and use it. If you need to
use an an alternative location, on the command line you can point the
package manager to the downloaded repository as follows

   amspackages --repository "/full/path/to/Downloads/packages/AMS2026.1.yml" list

Or, you can set the "SCM_AMSPKGS_REPO" environment variable in your
shell, or in the configuration file.

This will allow other programs that use the package manager to use the
same repository for checking and installing packages.

In your shell (e.g. in *.bashrc*):

   export SCM_AMSPKGS_REPO="/full/path/to/Downloads/packages/AMS2026.1.yml"

In the configuration file:

   SCM_AMSPKGS_REPO: /full/path/to/Downloads/packages/AMS2026.1.yml


Instructions for administrators
===============================

It is typical for several users to access and use the same
installation of AMS. On shared systems, such as a compute cluster, it
is often desirable to share installed optional components. This
prevents redundant installations of the same feature. On shared
installations, the default behavior is already to install shared
packages inside "AMSHOME" when that location is writable. Users who
cannot write to "AMSHOME" can use single-user mode instead. If you
want to force shared/admin mode explicitly, use the "--admin" flag.
For instance, to install a shared package explicitly, use

   "${AMSBIN}/amspackages" --admin install lfdft

Instead of in the user directory, packages will be installed in the
shared installation. Users need read-only access to the shared
installation directory.

AMSpackages supports environment variables that can override default
options. The shared package location can be overridden using
"SCM_AMSPKGS_SHAREDDIR". The single-user package location can be
overridden using "SCM_AMSPKGS_USERDIR". You should add any shared-
location override to the persistent configurations, so users can find
the installation path that you used for installing.


Installing python packages for users
====================================

Warning:

  SCM does not provide technical support for packages from pypi.org or
  other sources. Do not install python packages from untrusted
  developers.

Python packages may be pre-installed into amspython by admins. The
simplest method is to create the folder "python" in AMSHOME, the root
of the AMS installation.

After that, you must open a new shell before installing packages.

   "${AMSBIN}/amspython" -m pip install lxml # installs the lxml package from pypi.org

When using the paths above, the venv containing all python packages
will be located in

   $AMSHOME/python

Alternatively, one can export "SCM_PYTHONDIR" to a directory of their
choosing. Be sure to also export this variable at run time / for all
users or the installed packages may not be detected.


Persistent configurations
=========================


Configuration file
^^^^^^^^^^^^^^^^^^

YAML syntax

The configuration file uses YAML syntax. Environment variables can be
specified using the variable name, followed by ":", a space, and then
the value.

The "amspackages" command has a lot of different options that users or
admins may need to change to suit their particular system. The full
list of configuration options can always be seen when running
"${AMSBIN}/amspackages --help".

The usage text contains the name of the environment variables that
AMSpackages looks for between "[]" brackets. A user can export these
in their shell environment, or an administrator can add them to the
configuration file to make them persistent throughout the usage of the
package manager. These options will also affect the operation of the
package manager gui. The configuration file will make settings
persistent for all users of the installation of AMS. The file is
located inside AMSBIN, under

   "${AMSBIN}/amspackageslib/config.yml"

You can open this file in a plain text editor to make changes. Inside
you can find several variables that are set by default. Please take
care not to accidentally remove the pre-existing variables, as they
are necessary for the package manager to function. You can add your
own variables to the file, and edit variables such as the repository,
for instance to use a local copy.

Some example variables you may wish to set

   SCM_AMSPKGS_REPO: /downloads/SCM/packages/AMS2025.1.yml # A local copy of the package repository
   SCM_AMSPKGS_SHAREDDIR: /shared/SCM/packages/ # Admin provided packages will be installed here
   SCM_AMSPKGS_USERDIR: ~/.scm/packages # User packages will be installed here.

Note that using variables (for example "$HOME") inside the names of
paths is not supported, but you can use a "~" as a shortcut for home
in path names. If you require a different configuration per user, it
is recommended to set an environment variable inside the user
environment instead of using the configuration file.

Note:

  Variables not prefixed with "SCM_AMSPKGS", for example
  **SCM_PYTHONDIR** can not be set in the config.yml file. They should
  always be set and exported in the shell environment or they might
  not take effect.


Environment variables
=====================

AMSpackages can be configured through the following environment
variables.

**SCM_AMSPKGS_CONFIG**: If set, variables will be read from this file
instead of the default configuration file.

**SCM_AMSPKGS_REPO**: The location of the repository root definition
file. Can be either a URL or a path to a local yaml file.

**SCM_AMSPKGS_USERDIR**: The location for single-user package
directories.

**SCM_AMSPKGS_SHAREDDIR**: The location for shared/admin package
directories.

**SCM_AMSPKGS_VERBOSITY**: Increase output verbosity (integer value
between 0-4).

**SCM_AMSPKGS_SSL**: Set this variable to "False" to disable SSL, or
point it to a path containing certificates.

**SCM_AMSPKGS_NONINTERACTIVE**: Set this variable to "True" to run
without user input (e.g. yes to all prompts).

**SCM_AMSPKGS_ADMIN**: Set this variable to "True" to force
shared/admin mode.

**SCM_AMSPKGS_NOSHARED**: Set this variable to "True" to ignore any
packages installed in the shared installation..

**SCM_AMSPKGS_NOCACHE**: Set this variable to "True" to ignore cached
repository information.

**SCM_AMSPKGS_PIPSTRICT**: Set this variable to "True" to always
strictly match exact package versions for python packages.

**SCM_AMSPKGS_LIST_FORMAT**: Set to "wide", "long", or "yaml" for
default display mode in the "amspackages list" output.
