Appendix E: Mirroring the SCM package repository

Overview

A local package mirror is a copy of the SCM package repository that you keep on your own machine, shared filesystem, or internal network. This is useful when target systems cannot access the internet directly, when compute nodes are behind a firewall, or when you want to prepare one repository copy and reuse it across multiple AMS installations.

This appendix describes how to create a local copy of the SCM package repository. For information on telling AMSpackages to use that mirror, see Appendix D: Advanced package manager usage.

Warning

Never install packages from an untrusted source. Only use a mirror copied directly from SCM.

Using amsrepomirror

The preferred way to prepare and update a local SCM package mirror is with amsrepomirror. This tool is distributed separately from AMS and is intended to simplify repository mirroring compared with the manual methods below.

Download amsrepomirror

Download the version for your operating system:

Requirements

amsrepomirror requires:

  • network access to downloads.scm.com;

  • a valid SCM username and password; and

  • enough free disk space for the selected packages.

Downloaded files are placed in a directory named repository below the target directory you select.

Create a mirror

The main window guides you through four steps.

Step 1: Enter your credentials

Enter your SCM username and password. Both fields must contain text before the next step becomes available.

If no AMS versions are shown, check the startup warning. The application may have been unable to reach the listings server because of a network or firewall problem. Correct the problem and restart amsrepomirror.

Step 2: Select files and an AMS version

Use the checkboxes under Include files for: to control which platform-specific files are downloaded:

  • Windows, Linux, and macOS are included by default.

  • COSMO-RS is excluded by default because it requires an additional license.

Select the required AMS version from Download list:, then click Load. A successful load displays the green Download list loaded. message.

If the server rejects the credentials, check the username and password. Other load failures may indicate a temporary network problem.

Step 3: Select a target directory

Enter a directory or click Browse…. amsrepomirror creates the following directory below it:

<selected directory>/repository/

Step 4: Start the download

Click Begin Download. While the mirror is being downloaded:

  • the progress bar and counter show overall completion;

  • worker lines show active files and transfer speeds;

  • Begin Download is disabled; and

  • Cancel Download is enabled.

When every file is complete, Completed! and a Quit button appear below the progress bar. Click Quit to close amsrepomirror and its open subwindows.

Download settings

Open Edit → Settings… before starting a download.

  • Parallel downloads controls how many files are downloaded at once. The range is 1 to 5 and the default is 5.

  • Timeout (s) is the maximum time a file may make no byte progress. The default is 300 seconds.

  • Max retries is the number of retries after the first failed attempt. The default is 1.

The timeout does not limit the total duration of a large file transfer. A transfer may continue for longer than the timeout while it is still receiving data.

Click Save to use the edited settings. Click Cancel to restore the last saved values. Settings apply only to the current application session.

Cancel and resume

Clicking Cancel Download stops active downloads. Partial active files are deleted, successfully completed files remain on disk, and pending files remain in the queue.

Click Begin Download again to resume. amsrepomirror also checks existing file sizes and skips local files whose size already matches the server.

If a download fails after all retries, the batch stops and displays the error. Correct the problem and click Begin Download to retry the remaining queue.

Troubleshooting

No download lists

Check network access, proxy or firewall rules, and access to downloads.scm.com. Restart the application after connectivity is restored.

401 Unauthorized

The listing server rejected the credentials. Re-enter the SCM username and password, then try loading the list again.

A download stops

Read the warning shown in the main window. Check the available disk space and network connectivity. You can increase the timeout and retry values through Edit → Settings…. Click Begin Download to resume the pending queue.

COSMO-RS files are missing

COSMO-RS is excluded by default. Start a new application session, select COSMO-RS (requires additional license), and load the list again. Confirm that the required license is available before downloading these packages.

For questions about licenses, contact license@scm.com.

Manual mirroring methods

If you are not using amsrepomirror, the repository can still be mirrored manually. You can find a listing of the repository contents on our website.

Use the listing that matches your version of AMS, or you may have unexpected issues with installed packages. You will need to download these files and preserve their directory structure.

MacOS and Linux

Method 1: wget

Note

On MacOS, you will need to install wget first. You can use brew, or macports to install it.

On Unix systems, wget can be used to achieve this from the command line. To download the repository for AMS2025.1 you would use the following from the command line:

wget --user scmuser --password scmpasswd -xi https://downloads.scm.com/Downloads/packages/listings/AMS2026.1.txt

This will create the downloads.scm.com folder in your current working directory, containing just the path to the repository.

Method 2: python

You can download the following python script to download a copy of the repository. If you have a local installation of AMS, then you can run it with amspython. The script can resume after interruption, and update files that have changed in size.

It uses the following command line options

usage: download_repository.py [-h] [--strict-authentication] [--no-ssl] [--local-platform-only] [--delete-unlisted] LISTING_URL DOWNLOAD_FOLDER USER PASSWORD

positional arguments:
  LISTING_URL           Url pointing to the file listings for the repository.
  DOWNLOAD_FOLDER       A local directory for storing your download.
  USER                  Username for downloading from the website.
  PASSWORD              Password for downloading from the website.

optional arguments:
  -h, --help            show this help message and exit
  --strict-authentication
                        If supplied this script will exit on 401 errors. By default, this script will skip files on 401 errors.
  --no-ssl              Don't use SSL verification.
  --local-platform-only
                        Skip downloading files for platforms other than the local one (e.g. skip MacOS & Linux packages on Windows).
  --delete-unlisted     Delete files that are not present in the listing from the destination (e.g. clean up old versions of packages).

All positional arguments are REQUIRED.

For example, you can run it as

amspython download_repository.py https://downloads.scm.com/Downloads/packages/listings/AMS2026.1.txt ~/Downloads/my/repo scmuser scmpassword

If you want to use your own version of python, you need to install tqdm, and requests. Note that the script requires python 3.6 or greater.

python -m pip install tqdm requests
python download_repository.py --help

Windows

For Windows users, we provide this powershell script.

Download this script, along with the listing file. You can right-click the script file in the file browser and select Run with Powershell to start it.

It will open a blue window displaying the progress, as well as some interactive prompts that will require you to provide the listing file, the location to store the download, and your SCM user account and password.

Warning

Don’t close the blue powershell window while the program is running.

The script will automatically start downloading the files to the folder of your choosing. It may take a while to download all the files. When it is done, the blue window will display a message saying that it is safe to close.

Note

Your computer may not allow you to run powershell scripts by default. If you receive a warning about not being allowed to run scripts, first open a open a powershell window using win + R on your keyboard, and typing powershell. In the powershell window that opens type powershell -ep Bypass C:\Path\To\Script. Fill in the correct path to the script you downloaded.

Below, you can see a demonstration of how you can download and use the local copy on Windows.

You will need an internet connection to see the video.

Using the mirror with AMSpackages

After creating a mirror, point AMSpackages to it as described in Appendix D: Advanced package manager usage. In many common cases, AMSpackages will also detect a local mirror automatically when a folder named repository is placed inside AMSHOME, and it contains the matching AMS2026.1.yml file.