KF files
********

KF is the main format for storing binary data used in all Amsterdam
Modeling Suite programs. All "TAPEXX" and ".rkf" files are KF files.
PLAMS offers a dictionary-like interface to the KF format which allows
for reading, writing, modifying, and creating KF files efficiently.

class KFFile(path, autosave=True)

   A class for reading and writing binary files in KF format.

   This class acts as a wrapper around "KFReader" collecting all the
   data written by user in some "temporary zone" and using Fortran
   binaries "udmpkf" and "cpkf" to write this data to the physical
   file when needed.

   The constructor argument *path* should be a string with a path to
   an existing KF file or a new KF file that you wish to create. If a
   path to existing file is passed, new "KFReader" instance is created
   allowing to read all the data from this file.

   When "write()" method is used, the new data is not immediately
   written to a disk. Instead of that, it is temporarily stored in
   "tmpdata" dictionary. When method "save()" is invoked, contents of
   that dictionary are written to a physical file and "tmpdata" is
   emptied.

   Other methods like "read()" or "delete_section()" are aware of
   "tmpdata" and work flawlessly, regardless if "save()" was called or
   not.

   By default, "save()" is automatically invoked after each "write()",
   so physical file on a disk is always "actual". This behavior can be
   adjusted with *autosave* constructor parameter. Having autosave
   enabled is usually a good idea, however, if you need to write a lot
   of small pieces of data to your file, the overhead of calling
   "udmpkf" and "cpkf" after *every* "write()" can lead to significant
   delays. In such a case it is advised to disable autosave and call
   "save()" manually, when needed.

   Dictionary-like bracket notation can be used as a shortcut to read
   and write variables:

      mykf = KFFile('someexistingkffile.kf')
      #all three below are equivalent
      x = mykf['General%termination status']
      x = mykf[('General','termination status')]
      x = mykf.read('General','termination status')

      #all three below are equivalent
      mykf['Geometry%xyz'] = somevariable
      mykf[('Geometry','xyz')] = somevariable
      mykf.write('Geometry','xyz', somevariable)

   Parameters:
      * **path** (*str*) --

      * **autosave** (*bool*) --

   __init__(path, autosave=True)

      Parameters:
         * **path** (*str*) --

         * **autosave** (*bool*) --

   read(section: str, variable: str, return_as_list: Literal[1]) -> str | int | float | bool
   read(section: str, variable: str, return_as_list: bool = False) -> int | float | bool | str | Sequence[int | float | bool]

      Extract and return data for a *variable* located in a *section*.

      By default, for single-value numerical or boolean variables
      returned value is a single number or bool. For longer variables
      this method returns a list of values. For string variables a
      single string is returned. This behavior can be changed by
      setting *return_as_list* parameter to "True". In that case the
      returned value is always a list of numbers (possibly of length
      1) or a single string.

   read_int(section, variable)

      Obtain an integer for a *variable* located in *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a single integer

      Returns:
         The integer value of *variable* located in *section*

      Return type:
         int

   read_ints(section, variable)

      Obtain a list of integers for a *variable* located in *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a list of integers

      Returns:
         The list of integers of *variable* located in *section*

      Return type:
         *List*[int]

   read_real(section, variable)

      Obtain a real/float for a *variable* located in *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a single real/float

      Returns:
         The real/float value of *variable* located in *section*

      Return type:
         float

   read_reals(section, variable)

      Obtain a list of reals/floats for a *variable* located in
      *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a list of reals/floats

      Returns:
         The list of reals/floats of *variable* located in *section*

      Return type:
         *List*[float]

   read_logical(section, variable)

      Obtain a logical/bool for a *variable* located in *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a single logical/bool

      Returns:
         The logical/bool value of *variable* located in *section*

      Return type:
         bool

   read_logicals(section, variable)

      Obtain a list of logicals/bools for a *variable* located in
      *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a list of logicals/bools

      Returns:
         The list of logicals/bools of *variable* located in *section*

      Return type:
         *List*[bool]

   read_string(section, variable)

      Obtain a string for a *variable* located in *section*.

      Parameters:
         * **section** (*str*) -- Section where the variable is
           located

         * **variable** (*str*) -- Variable to obtain

      Raises:
         **KFTypedReadError** -- Error is raised if the *variable* in
         *section* does not contain a single string

      Returns:
         The string value of *variable* located in *section*

      Return type:
         str

   write(section, variable, value, value_type=None)

      Write a *variable* with a *value* in a *section* . If such a
      variable already exists in this section, the old value is
      overwritten.

      Parameters:
         * **section** (*str*) --

         * **variable** (*str*) --

         * **value** (*int** | **float** | **bool** | **str** |
           **Sequence**[**int** | **float** | **bool** | **str**]*) --

         * **value_type** (*int** | **float** | **bool** | **str** |
           **Sequence**[**int** | **float** | **bool** | **str**] **|
           **None*) --

      Return type:
         None

   save()

      Save all changes stored in "tmpdata" to physical file on a disk.

      Return type:
         None

   delete_section(section)

      Delete the entire *section* from this KF file.

      Parameters:
         **section** (*str*) --

      Return type:
         None

   sections()

      Return a list with all section names, ordered alphabetically.

      Return type:
         *List*[str]

   read_section(section)

      Return a dictionary with all variables from a given *section*.

      Note:

        Some sections can contain very large amount of data. Turning
        them into dictionaries can cause memory shortage or
        performance issues. Use this method carefully.

      Parameters:
         **section** (*str*) --

      Return type:
         *Dict*[str, int | float | bool | str | *Sequence*[int | float
         | bool]]

   keys()

      Returns all sections in the current instance

      Return type:
         *Set*[str]

   get_skeleton()

      Return a dictionary reflecting the structure of this KF file.

      Each key in that dictionary corresponds to a section name of the
      KF file with the value being a set of variable names.

      Return type:
         *Dict*[str, *Set*[str]]

   _variable_type_equals(section, variable, variable_type)

      Check the type of a variable.

      Parameters:
         * **section** (*str*) --

         * **variable** (*str*) --

         * **variable_type** (*int*) --

      Return type:
         bool

   __getitem__(name)

      Allow to use "x = mykf['section%variable']" or "x =
      mykf[('section','variable')]" instead of "x = kf.read('section',
      'variable')".

      Parameters:
         **name** (*str** | **Tuple**[**str**, **str**]*) --

      Return type:
         int | float | bool | str | *Sequence*[int | float | bool]

   __setitem__(name, value)

      Allow to use "mykf['section%variable'] = value" or
      "mykf[('section','variable')] = value" instead of
      "kf.write('section', 'variable', value)".

      Parameters:
         * **name** (*str** | **Tuple**[**str**, **str**]*) --

         * **value** (*int** | **float** | **bool** | **str** |
           **Sequence**[**int** | **float** | **bool** | **str**]*) --

      Return type:
         None

   __iter__()

      Iteration yields pairs of section name and variable name.

      Return type:
         *Iterator*[*Tuple*[str, str]]

   __contains__(arg)

      Implements Python "in" operator for KFFiles. *arg* can be a
      single string with a section name or a pair of strings (section,
      variable).

      Parameters:
         **arg** (*str** | **Tuple**[**str**, **str**]*) --

      Return type:
         bool

   static _split(name)

      Ensure that a key used in bracket notation is of the form
      "'section%variable'" or "('section','variable')". If so, return
      a tuple "('section','variable')".

      Parameters:
         **name** (*str** | **Tuple**[**str**, **str**]*) --

      Return type:
         *Tuple*[str, str]

   static _str(val)

      Return a string representation of *val* in the form that can be
      understood by "udmpkf".

      Parameters:
         **val** (*Any*) --

      Return type:
         str

class KFReader(path, blocksize=4096, autodetect=True)

   A class for efficient Python-native reader of binary files in KF
   format.

   This class offers read-only access to any fragment of data from a
   KF file. Unlike other Python KF readers, this one does not use the
   Fortran binary "dmpkf" to process KF files, but instead reads and
   interprets raw binary data straight from the file, on Python level.
   That approach results in significant speedup (by a factor of few
   hundreds for large files extracted variable by variable).

   The constructor argument *path* should be a string with a path
   (relative or absolute) to an existing KF file.

   *blocksize* indicates the length of basic KF file block. So far,
   all KF files produced by any of Amsterdam Modeling Suite programs
   have the same block size of 4096 bytes. Unless you're doing
   something *very* special, you should not touch this value.

   Organization of data inside KF file can depend on a machine on
   which this file was produced. Two parameters can vary: the length
   of integer (32 or 64 bit) and endian (little or big). These
   parameters have to be determined before any reading can take place,
   otherwise the results will have no sense. If the constructor
   argument *autodetect* is "True", the constructor attempts to
   automatically detect the format of a given KF file, allowing to
   read files created on a machine with different endian or integer
   length. This automatic detection is enabled by default and it is
   advised to leave it that way. If you wish to disable it, you should
   set "endian" and "word" attributes manually before reading anything
   (see the code for details).

   Note:

     This class consists of quite technical, low level code. If you
     don't need to modify or extend "KFReader", you can safely ignore
     all private methods, all you need is "read()" and occasionally
     "__iter__()"

   Parameters:
      * **path** (*str*) --

      * **blocksize** (*int*) --

      * **autodetect** (*bool*) --

   __init__(path, blocksize=4096, autodetect=True)

      Parameters:
         * **path** (*str*) --

         * **blocksize** (*int*) --

         * **autodetect** (*bool*) --

   read(section, variable)

      Extract and return data for a *variable* located in a *section*.

      For single-value numerical or boolean variables returned value
      is a single number or bool. For longer variables this method
      returns a list of values. For string variables a single string
      is returned.

      Parameters:
         * **section** (*str*) --

         * **variable** (*str*) --

      Return type:
         int | float | bool | str | *Sequence*[int | float | bool]

   variable_type(section, variable)

      Return the integer code of the variable's type (int:1, float:2,
      string:3, bool:4)

      Parameters:
         * **section** (*str*) --

         * **variable** (*str*) --

      Return type:
         int

   __iter__()

      Iteration yields pairs of section name and variable name.

      Return type:
         *Iterator*[*Tuple*[str, str]]

   _autodetect()

      Try to automatically detect the format (int size and endian) of
      this KF file.

      Return type:
         None

   _read_block(f, pos)

      Read a single block of binary data from posistion *pos* in file
      *f*.

      Parameters:
         * **f** (*BinaryIO*) --

         * **pos** (*int*) --

      Return type:
         bytes

   _parse(block, format)

      Translate a *block* of binary data into list of values in
      specified *format*.

      *format* should be a list of pairs *(a,t)* where *t* is one of
      the following characters: "'s'" for string (bytes), "'i'" for
      32-bit integer, "'q'" for 64-bit integer and *a* is the number
      of occurrences (or length of a string).

      For example, if *format* is equal to
      "[(32,'s'),(4,'i'),(2,'d'),(2,'i')]", the contents of *block*
      are divided into 72 bytes (32*1 + 4*4 + 2*8 + 2*4 = 72) chunks
      (possibly droping the last one, if it's shorter than 72 bytes).
      Then each chunk is translated to a 9-tuple of bytes, 4 ints, 2
      floats and 2 ints. List of such tuples is the returned value.

      Parameters:
         * **block** (*bytes*) --

         * **format** (*Iterable**[**Tuple**[**int**, **str**]**]*) --

      Return type:
         *List*[*Tuple*[*Any*, ...]]

   _get_data(datablock, vtype)

      Extract all data of a given type from a single data block.
      Returned value is a list of values (int, float, or bool) or a
      single "bytes" object.

      Parameters:
         * **datablock** (*bytes*) --

         * **vtype** (*int*) --

      Return type:
         bytes | *List*[int | float | bool]

   _create_index()

      Find and parse relevant index blocks of KFFile to extract the
      information about location of all sections and variables.

      Two dictionaries are populated during this process. "_data"
      contains, for each section, a list of triples describing how
      logical blocks of data are mapped into physical ones. For
      example, "_data['General'] = [(3,6,12), (9,40,45)]" means that
      logical blocks 3-8 of section "General" are located in physical
      blocks 6-11 and logical blocks 9-13 in physical blocks 40-44.
      This list is always sorted via first tuple elements allowing
      efficient access to arbitrary logical block of each section.

      The second dictionary, "_sections", is used to locate each
      variable within its section. For each section, it contains
      another dictionary of each variable of this section. So
      "_section[sec][var]" contains all information needed to extract
      variable "var" from section "sec". This is a 4-tuple containing
      the following information: variable type, logic block in which
      the variable first occurs, position within this block where its
      data start and the length of the variable. Combining this
      information with mapping stored in "_data" allows to extract
      each single variable.

      Return type:
         None

   static _datablocks(lst, n=1)

      Transform a tuple of lists "([x1,x2,...],
      [(a1,b1),(a2,b2),...])" into an iterator over
      "range(a1,b1)+range(a2,b2)+..." Iteration starts from nth
      element of this list.

      Parameters:
         * **lst** (*Tuple**[**List**[**int**]**,
           **List**[**Tuple**[**int**, **int**]**]**]*) --

         * **n** (*int*) --

      Return type:
         *Iterator*[int]

class KFHistory(kf, section)

   A class for reading "History" sections of files in the KF format.

   This class acts as a wrapper around "KFReader" enabling convenient
   iteration over entries (frames) of History sections.

   The constructor argument *kf* should be a "KFReader" instance
   attached to an existing KF file. The *section* argument then holds
   a name of the desired History-like section, such as "History" or
   "MDHistory".

   The "read_all()" method can be used to easily read all values of a
   particular history item into a single numpy array.

   To iterate over the frames in a history section, use "iter()" or
   "iter_optional()". The former raises an exception if the selected
   variable is not present in the history, while the latter returns a
   given default value instead.

   Usage:

      kf = KFReader('somefile.rkf')
      mdhistory = KFHistory(kf, 'MDHistory')

      for T, p in mdhistory.iter('Temperature'), mdhistory.iter_optional('Pressure', 0):
          print(T, p)

   Parameters:
      * **kf** (*KFReader*) --

      * **section** (*str*) --

   __init__(kf, section)

      Parameters:
         * **kf** (*KFReader*) --

         * **section** (*str*) --

   read_all(name)

      Return a numpy array containing the values of history item
      *name* from all frames.

      Parameters:
         **name** (*str*) --

      Return type:
         *ndarray*

   iter(name)

      Iterate over the values of history item *name*.

      Parameters:
         **name** (*str*) --

      Return type:
         *Iterator*[int | float | bool | str | *Sequence*[int | float
         | bool]]

   iter_optional(name, default=None)

      Iterate over the values of history item *name*, returning
      *default* if the item is not present.

      Parameters:
         * **name** (*str*) --

         * **default** (*int** | **float** | **bool** | **str** |
           **Sequence**[**int** | **float** | **bool**] **| **None*)
           --

      Return type:
         *Iterator*[int | float | bool | str | *Sequence*[int | float
         | bool] | None]
