Improving ADF performance on supercomputers

In some cases you may find that ADF is wasting time waiting for I/O. A large difference between Elapsed time and CPU time indicates that this is happening. The reason is typically that:

ADF is sharing an I/O system with another application that uses I/O heavily
the I/O bandwidth of the system is insufficient.

In such a case, if your machine has enough memory, you can make sure that ADF keeps its files (partially) in memory. In this way, I/O and the corresponding delays are kept to a minimum.

Problem

You notice for your ADF job that there is a gap between the Elapsed time and CPU time. These are printed by ADF near the end of your output file. This delay may well be (partially) due to the fact that the program is waiting for I/O.

Solution

Increase the value for the SCM_IOBUFFERSIZE environment variable. I/O can be avoided completely in this manner if you have enough memory available. If your system has 1 GB of memory per CPU, 128 or 256 are reasonable values for SCM_IOBUFFERSIZE. These numbers refer to the number of MBytes per CPU reserved for I/O buffering.
Lengthy E-mail from SCM on this topic

Search: