Job Analysis
************

Note:

  The "JobAnalysis" class is available in AMS2025+

The "JobAnalysis" class is a tool that aims to simplify the process of
analyzing the status, inputs and outputs of multiple jobs. It helps to
create analysis tables, with jobs as rows and "analysis fields" as
columns. An analysis field is simply a definition for how to extract a
value from each job. These tables can then be easily visualized in a
Jupyter notebook or script, or exported for use with other analysis
packages like pandas.

For a worked example demonstrating the capabilities and uses of the
"JobAnalysis" class, see the Job analysis example.


Adding Jobs
===========

Jobs can be added to a "JobAnalysis" in two ways, either passed
directly on initialization or using "add_job()", or alternatively
loaded from paths using "load_job()".

When loading jobs from a path, an attempt is first made to load from a
".dill" file. If this fails, the tool will attempt to use a series of
loaders to create the job. By default this will be "load_external()",
but a custom set of loaders can be provided for other job types.

For example, the following snippet will add two jobs to the
"JobAnalysis" and then load a further job using the
"ParAMSJob.load_external" method:

   ja = (JobAnalysis(jobs=[job1, job2])
         .load_job("path/to/job3", loaders=[ParAMSJob.load_external]))

This also illustrates one of the design features of "JobAnalysis",
each operation returns an updated copy of the "JobAnalysis", allowing
the use of fluent syntax to chain methods.


Adding Analysis Fields
======================

Analysis fields can be added to a "JobAnalysis" in a number of ways.
Firstly, there are a small number of predefined "standard" analysis
fields which are common across jobs e.g. "Name", "Path" etc. These can
be added to the analysis via the methods, such as
"add_standard_field()" / "add_standard_fields()". To add these
standard fields, the key(s) of the relevant fields must be supplied. A
full list of these is available in the tooltip.

Secondly, custom analysis fields can be added using the methods
"add_field()" or "set_field()". When doing this, a unique identifier
for the field (the key) must be provided, along with a function
defining how to extract a value for the field from a job. In addition,
optional arguments can be provided to set the displayed name for the
field, and its formatting when returned in a table.

Finally, additional methods are present to facilitate adding fields
for values from the job "settings". These are "add_settings_field()"
and "add_settings_input_fields()". The former is used to create an
analysis field given a specific nested settings key, the latter
automatically creates fields for all settings keys under the
"settings.input", which includes AMS input settings. The key for these
settings fields will be the concatenated settings key in Pascal case
e.g. "("input", "ams", "task")" will have field key "InputAmsTask".

For example, the following snippet will add the "Formula" field to the
"JobAnalysis", followed by a custom field displaying the number of
atoms, and finally the settings input fields:

   ja = (ja
    .add_standard_field("Formula")
    .add_field("NAtoms", lambda j: len(j.molecule))
    .add_settings_input_fields())


Modifying Analysis
==================

"JobAnalysis" supports some modification and data manipulation
methods. These allow jobs and results to be interrogated and more
easily visualized.

Fields can be filtered using "filter_fields()". This accepts a
predicate as an argument, which accepts all the values for a field.
Fields are only retained for which the predicate evaluates to "True".
There are some pre-configured predicates, such as
"remove_empty_fields()" and "remove_uniform_fields()". These can be
useful for removing noise from the analysis table, for fields which
only have empty/the same values.

Jobs can also be filtered using "filter_jobs()". This accepts a
predicate as an argument, which accepts all the field keys and values
for a job. Again, jobs are only retained for which the predicate
evaluates to "True". This can be useful for removing jobs which are
not of interest, for example those with a given status.

For example, the following snippet will retain only jobs which did not
succeed:

   ja = (ja
    .filter_field(lambda vals: all([v is not None for v in vals]))
    .filter_jobs(lambda data: not data["OK"]))

Fields can also be sorted, renamed and formatted to aid with
presentation. These changes will only be reflected in the generated
tables, not when calling "get_analysis()".

For example:

   ja = (ja
    .rename_field("InputAmsTask", "Task")
    .format_field("Energy", ".4f")
    .sort_fields(["Name", "Formula", "Energy"])
    .sort_jobs(field_keys=["Energy"]))


Extracting Analysis
===================

There are various ways to extract data from "JobAnalysis". The
simplest way to get a visual representation is to call "get_table()"
(or "display_table()" if running in a notebook). This generates a
table in either "markdown", "html" or "rst" formats.

Alternatively, data can be retrieved in code by calling
"get_analysis()". This gets a pure Python dictionary of the data,
where the analysis field keys are the dictionary keys. The dictionary
values are lists of values, with the elements as the field value for
each job.

For more complex or involved analysis, the best approach is to export
the data to a pandas data frame. Pandas is a fast and powerful data
analysis tool, which can perform complex manipulations. It can be
installed via "amspackages".

As a final option, data can also be saved to a CSV file using
"to_csv_file()".


API
===

class JobAnalysis(paths=None, jobs=None, loaders=None, standard_fields=('Path', 'Name', 'OK', 'Check', 'ErrorMsg'), await_results=True)

   Analysis tool for Jobs, which generates tables of data consisting
   of fields and their respective value for each job.

   The jobs and fields which are included in the analysis are
   customizable, to allow for flexible comparison.

   Parameters:
      * **paths** (*Sequence**[**str** | **PathLike**] **| **None*) --

      * **jobs** (*Sequence**[**Job**] **| **None*) --

      * **loaders** (*Sequence**[**Callable**[**[**str**]**,
        **Job**]**] **| **None*) --

      * **standard_fields**
        (*Sequence**[**JobAnalysis.StandardField**] **| **None*) --

      * **await_results** (*bool*) --

   __init__(paths=None, jobs=None, loaders=None, standard_fields=('Path', 'Name', 'OK', 'Check', 'ErrorMsg'), await_results=True)

      Initialize new instance of "JobAnalysis" with a set of jobs.

         >>> ja = JobAnalysis(jobs=[job1, job2], standard_fields=["Name", "OK"])
         >>> ja

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Parameters:
         * **paths** (*Sequence**[**str** | **PathLike**] **| **None*)
           -- one or more paths to folders from which to load jobs to
           add to the analysis

         * **jobs** (*Sequence**[**Job**] **| **None*) -- one or more
           jobs to add to the analysis

         * **loaders** (*Sequence**[**Callable**[**[**str**]**,
           **Job**]**] **| **None*) -- custom loading functions to
           generate jobs from a job folder

         * **standard_fields** (*Sequence**[**Literal**[**'Path'**,
           **'Name'**, **'OK'**, **'Check'**, **'ErrorMsg'**,
           **'ParentPath'**, **'ParentName'**, **'Formula'**,
           **'Smiles'**, **'GyrationRadius'**, **'CPUTime'**,
           **'SysTime'**, **'ElapsedTime'**]**] **| **None*) -- keys
           of standard fields to include in analysis, defaults to
           "("Path", "Name", "OK", "Check", "ErrorMsg")"

         * **await_results** (*bool*) -- whether to wait for the
           results of any passed jobs to finish, defaults to "True"

   copy()

      Produce a copy of this analysis with the same jobs and fields.

         >>> ja.copy()

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Returns:
         copy of the analysis

      Return type:
         *JobAnalysis*

   property jobs: Dict[str, Job]

      Jobs currently included in analysis.

         >>> ja.jobs

         {
             '/path/job1': <scm.plams.interfaces.adfsuite.ams.AMSJob object at 0x1085a13d0>,
             '/path/job2': <scm.plams.interfaces.adfsuite.ams.AMSJob object at 0x15e389970>
         }

      Returns:
         Dictionary of the job path and the "Job"

   property field_keys: List[str]

      Keys of current fields, as they appear in the analysis.

         >>> ja.field_keys

         ['Name', 'OK']

      Returns:
         list of field keys

   get_analysis()

      Gets analysis data. This is effectively a table in the form of a
      dictionary, where the keys are the field keys and the values are
      a list of data for each job.

         >>> ja.field_keys

         {
             'Name': ['job1', 'job2'],
             'OK': [True, True]
         }

      Returns:
         analysis data as a dictionary of field keys/lists of job
         values

      Return type:
         *Dict*[str, *List*]

   _expand_analysis(analysis)

      Expand analysis fields, converting individual job rows into
      multiple rows

      Parameters:
         **analysis** (*Dict**[**str**, **Any**]*) -- analysis data as
         a dictionary of field keys/lists of job values

      Returns:
         analysis data as a dictionary of field keys/lists of
         (multiple) job values

      Return type:
         *Dict*[str, *Any*]

   to_dataframe()

      Converts analysis data to a dataframe. The column names are the
      field keys and the column values are the values for each job.
      This method requires the pandas package.

         >>> print(ja.to_dataframe())

            Name    OK
         0  job1  True
         1  job2  True

      Returns:
         analysis data as a dataframe

      Return type:
         DataFrame

   to_table(max_col_width=-1, max_rows=30, fmt='markdown')

      Converts analysis data to a pretty-printed table.

         >>> print(ja.to_table())

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Parameters:
         * **max_col_width** (*int*) -- can be integer positive value
           or -1, defaults to -1 (no maximum width)

         * **max_rows** (*int*) -- can be integer positive value or
           -1, defaults to 30

         * **fmt** (*Literal**[**'markdown'**, **'html'**,
           **'rst'**]*) -- format of the table, either markdown
           (default), html or rst

      Returns:
         string representation of the table

      Return type:
         str

   display_table(max_col_width=-1, max_rows=30, fmt='markdown')

      Converts analysis data to a pretty-printed table which is then
      displayed using IPython.

         >>> ja.display_table()

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Parameters:
         * **max_col_width** (*int*) -- can be integer positive value
           or -1, defaults to -1 (no maximum width)

         * **max_rows** (*int*) -- can be integer positive value or
           -1, defaults to 30

         * **fmt** (*Literal**[**'markdown'**, **'html'**,
           **'rst'**]*) -- format of the table, either markdown
           (default), html or rst

      Return type:
         None

   to_csv_file(path)

      Write the analysis to a csv file with the specified path.

         >>> ja.to_csv_file("./a.csv")
         >>> with open("./a.csv") as csv:
         >>>     print(csv.read())

         Name,OK
         job1,True
         job2,True

      Parameters:
         **path** (*str** | **PathLike*) -- path to save the csv file

      Return type:
         None

   get_timeline(max_intervals=5, fmt='markdown')

      Get depiction of timeline of jobs as they were run. Each job is
      represented as a horizontal bar of symbols, where each symbol
      indicates a different job status.

      These are as follows:

      * "created": "."

      * "started": "-"

      * "registered": "+"

      * "running": "="

      * "finished": "*"

      * "crashed": "x"

      * "failed": "X"

      * "successful": ">"

      * "copied": "#"

      * "preview": "~"

      * "deleted": "!"

      e.g.

         >>> print(ja.get_timeline())

         | JobName    | ↓2025-02-03 15:16:52 | ↓2025-02-03 15:17:10 | ↓2025-02-03 15:17:28 | ↓2025-02-03 15:17:46 | ↓2025-02-03 15:18:03 | WaitDuration | RunDuration | TotalDuration |
         |------------|----------------------|----------------------|----------------------|----------------------|----------------------|--------------|-------------|---------------|
         | generate   | ==================== | ==================== | ==================== | ==========>          |                      | 0s           | 1m2s        | 1m2s          |
         | reoptimize |                      |                      |                      |           ====>      |                      | 0s           | 3s          | 3s            |
         | score      |                      |                      |                      |                ===>  |                      | 0s           | 2s          | 2s            |
         | filter     |                      |                      |                      |                   =* | >                    | 0s           | 1s          | 1s            |

      If multiple status changes occur within the same resolution
      period, the latest will be displayed.

      Parameters:
         * **max_intervals** (*int*) -- maximum number of datetime
           intervals to display i.e. the width and resolution of the
           timeline

         * **fmt** (*Literal**[**'markdown'**, **'html'**,
           **'rst'**]*) -- format of the table, either markdown
           (default) or html

      Returns:
         string representation of timeline as a markdown (default),
         html or rst table

      Return type:
         str

   display_timeline(max_intervals=5, fmt='markdown')

      Get depiction of timeline of jobs as they were run and display
      using IPython. Each job is represented as a horizontal bar of
      symbols, where each symbol indicates a different job status.

      These are as follows:

      * "created": "."

      * "started": "-"

      * "registered": "+"

      * "running": "="

      * "finished": "*"

      * "crashed": "x"

      * "failed": "X"

      * "successful": ">"

      * "copied": "#"

      * "preview": "~"

      * "deleted": "!"

      e.g.

         >>> ja.display_timeline()

         | JobName    | ↓2025-02-03 15:16:52 | ↓2025-02-03 15:17:10 | ↓2025-02-03 15:17:28 | ↓2025-02-03 15:17:46 | ↓2025-02-03 15:18:03 | WaitDuration | RunDuration | TotalDuration |
         |------------|----------------------|----------------------|----------------------|----------------------|----------------------|--------------|-------------|---------------|
         | generate   | ==================== | ==================== | ==================== | ==========>          |                      | 0s           | 1m2s        | 1m2s          |
         | reoptimize |                      |                      |                      |           ====>      |                      | 0s           | 3s          | 3s            |
         | score      |                      |                      |                      |                ===>  |                      | 0s           | 2s          | 2s            |
         | filter     |                      |                      |                      |                   =* | >                    | 0s           | 1s          | 1s            |

      If multiple status changes occur within the same resolution
      period, the latest will be displayed. :param max_intervals:
      maximum number of datetime intervals to display i.e. the width
      and resolution of the timeline :param fmt: format of the table,
      either markdown (default), html or rst

      Parameters:
         * **max_intervals** (*int*) --

         * **fmt** (*Literal**[**'markdown'**, **'html'**,
           **'rst'**]*) --

      Return type:
         None

   add_job(job)

      Add a job to the analysis. This adds a row to the analysis data.

         >>> ja.add_job(job3)

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |
         | job_3 | True |

      Parameters:
         **job** (*Job*) -- "Job" to add to the analysis

      Returns:
         copy of "JobAnalysis" with the additional job

      Return type:
         *JobAnalysis*

   remove_job(job)

      Remove a job from the analysis. This removes a row from the
      analysis data.

         >>> ja.remove_job(job2)

         | Name  | OK   |
         |-------|------|
         | job_1 | True |

      Parameters:
         **job** (*str** | **PathLike** | **Job*) -- "Job" or path to
         a job to remove from the analysis

      Returns:
         copy of "JobAnalysis" with the job removed

      Return type:
         *JobAnalysis*

   load_job(path, loaders=None)

      Add job to the analysis by loading from a given path to the job
      folder. If no dill file is present in that location, or the dill
      unpickling fails, the loaders will be used to load the given job
      from the folder.

         >>> ja.load_job("path/job3")

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |
         | job_3 | True |

      Parameters:
         * **path** (*str** | **PathLike*) -- path to folder from
           which to load the job

         * **loaders** (*Sequence**[**Callable**[**[**str**]**,
           **Job**]**] **| **None*) -- functions to try and load jobs,
           defaults to "load_external()" followed by "load_external()"

      Returns:
         copy of "JobAnalysis" with the additional loaded job

      Return type:
         *JobAnalysis*

   filter_jobs(predicate)

      Retain jobs from the analysis where the given predicate for
      field values evaluates to "True". In other words, this removes
      rows(s) from the analysis data where the filter function
      evaluates to "False" given a dictionary of the row data.

         >>> ja

         | Name  | OK    |
         |-------|-------|
         | job_1 | True  |
         | job_2 | True  |
         | job_3 | False |

         >>> ja.filter_jobs(lambda data: not data["OK"])

         | Name  | OK    |
         |-------|-------|
         | job_3 | False |

      Parameters:
         **predicate** (*Callable**[**[**Dict**[**str**,
         **Any**]**]**, **bool**]*) -- filter function which takes a
         dictionary of field keys and their values and evaluates to
         "True"/"False"

      Returns:
         copy of "JobAnalysis" with the job filter applied

      Return type:
         *JobAnalysis*

   sort_jobs(field_keys=None, sort_key=None, reverse=False)

      Sort jobs according to a single or multiple fields. This is the
      order the rows will appear in the analysis data.

      Either one of "field_keys" or "key" must be provided. If
      "field_keys" is provided, the values from these field(s) will be
      used to sort, in the order they are specified. If "sort_key" is
      provided, the sorting function will be applied to all fields.

         >>> ja.sort_jobs(field_keys=["Name"], reverse=True)

         | Name  | OK    |
         |-------|-------|
         | job_2 | True  |
         | job_1 | True  |

      Parameters:
         * **field_keys** (*Sequence**[**str**] **| **None*) -- field
           keys to sort by,

         * **sort_key** (*Callable**[**[**Dict**[**str**,
           **Any**]**]**, **Any**] **| **None*) -- sorting function
           which takes a dictionary of field keys and their values

         * **reverse** (*bool*) -- reverse sort order, defaults to
           "False"

      Returns:
         copy of "JobAnalysis" with the job sorted

      Return type:
         *JobAnalysis*

   add_field(key, value_extractor, display_name=None, fmt=None, expansion_depth=0)

      Add a new field to the analysis. This adds a column to the
      analysis data.

         >>> ja.add_field("N", lambda j: len(j.molecule), display_name="Num Atoms")

         | Name  | OK    | Num Atoms |
         |-------|-------|-----------|
         | job_1 | True  | 4         |
         | job_2 | True  | 6         |

      Parameters:
         * **key** (*str*) -- unique identifier for the field

         * **value_extractor** (*Callable**[**[**Job**]**, **Any**]*)
           -- callable to extract the value for the field from a job

         * **display_name** (*str** | **None*) -- name which will
           appear for the field when displayed in table

         * **fmt** (*str** | **None*) -- string format for how field
           values are displayed in table

         * **expansion_depth** (*int*) -- whether to expand field of
           multiple values into multiple rows, and recursively to what
           depth

      Returns:
         copy of "JobAnalysis" with the additional field

      Return type:
         *JobAnalysis*

   set_field(key, value_extractor, display_name=None, fmt=None, expansion_depth=0)

      Set a field in the analysis. This adds or modifies a column in
      the analysis data.

         >>> ja.set_field("N", lambda j: len(j.molecule), display_name="Num Atoms")

         | Name  | OK    | Num Atoms |
         |-------|-------|-----------|
         | job_1 | True  | 4         |
         | job_2 | True  | 6         |

      Parameters:
         * **key** (*str*) -- unique identifier for the field

         * **value_extractor** (*Callable**[**[**Job**]**, **Any**]*)
           -- callable to extract the value for the field from a job

         * **display_name** (*str** | **None*) -- name which will
           appear for the field when displayed in table

         * **fmt** (*str** | **None*) -- string format for how field
           values are displayed in table

         * **expansion_depth** (*int*) -- whether to expand field of
           multiple values into multiple rows, and recursively to what
           depth

      Returns:
         copy of "JobAnalysis" with the column set

      Return type:
         *JobAnalysis*

   format_field(key, fmt=None)

      Apply a string formatting to a given field. This will apply when
      "to_table" is called.

         >>> ja.format_field("N", "03.0f")

         | Name  | OK    | Num Atoms |
         |-------|-------|-----------|
         | job_1 | True  | 004       |
         | job_2 | True  | 006       |

      Parameters:
         * **key** (*str*) -- unique identifier of the field

         * **fmt** (*str** | **None*) -- string format of the field
           e.g. ".2f"

      Returns:
         copy of "JobAnalysis" with field formatting applied

      Return type:
         *JobAnalysis*

   rename_field(key, display_name)

      Give a display name to a field in the analysis. This is the
      header of the column in the analysis data.

         >>> ja.rename_field("N", "N Atoms")

         | Name  | OK    | N Atoms |
         |-------|-------|---------|
         | job_1 | True  | 004     |
         | job_2 | True  | 006     |

      Parameters:
         * **key** (*str*) -- unique identifier for the field

         * **display_name** (*str*) -- name of the field

      Returns:
         copy of "JobAnalysis" with the renamed field

      Return type:
         *JobAnalysis*

   expand_field(key, depth=1)

      Expand field of multiple values into multiple rows for each job.
      For nested values, the depth can be provided to determine the
      level of recursive expansion.

         >>> (ja
         >>>  .add_field("Step", lambda j: get_steps(j))
         >>>  .add_field("Energy", lambda j: get_energies(j)))

         | Name  | OK    | Step      | Energy             |
         |-------|-------|-----------|--------------------|
         | job_1 | True  | [1, 2, 3] | [42.1, 43.2, 42.5] |
         | job_2 | True  | [1, 2]    | [84.5, 112.2]      |

         >>> (ja
         >>>  .expand_field("Step")
         >>>  .expand_field("Energy"))

         | Name  | OK    | Step | Energy |
         |-------|-------|------|--------|
         | job_1 | True  | 1    | 42.1   |
         | job_1 | True  | 2    | 43.2   |
         | job_1 | True  | 3    | 42.5   |
         | job_2 | True  | 1    | 84.5   |
         | job_2 | True  | 1    | 112.2  |

      Parameters:
         * **key** (*str*) -- unique identifier of field to expand

         * **depth** (*int*) -- depth of recursive expansion, defaults
           to 1

      Returns:
         copy of "JobAnalysis" with the field expanded

      Return type:
         *JobAnalysis*

   collapse_field(key)

      Collapse field of multiple rows into single row of multiple
      values for each job.

         >>> ja

         | Name  | OK    | Step | Energy |
         |-------|-------|------|--------|
         | job_1 | True  | 1    | 42.1   |
         | job_1 | True  | 2    | 43.2   |
         | job_1 | True  | 3    | 42.5   |
         | job_2 | True  | 1    | 84.5   |
         | job_2 | True  | 1    | 112.2  |

         >>> (ja
         >>>  .collapse_field("Step")
         >>>  .collapse_field("Energy"))

         | Name  | OK    | Step      | Energy             |
         |-------|-------|-----------|--------------------|
         | job_1 | True  | [1, 2, 3] | [42.1, 43.2, 42.5] |
         | job_2 | True  | [1, 2]    | [84.5, 112.2]      |

      Parameters:
         **key** (*str*) -- unique identifier of field to collapse

      Returns:
         copy of "JobAnalysis" with the field collapsed applied

      Return type:
         *JobAnalysis*

   reorder_fields(order)

      Reorder fields based upon the given sequence of field keys. This
      is the order the columns will appear in the analysis data.

      Any specified fields will be placed first, with remaining fields
      placed after with their order unchanged.

         >>> ja.reorder_fields(["Name", "Step"])

         | Name  | Step | OK    | Energy |
         |-------|------|-------|--------|
         | job_1 | 1    | True  | 42.1   |
         | job_1 | 2    | True  | 43.2   |
         | job_1 | 3    | True  | 42.5   |
         | job_2 | 1    | True  | 84.5   |
         | job_2 | 1    | True  | 112.2  |

      Parameters:
         **order** (*Sequence**[**str**]*) -- sequence of fields to be
         placed at the start of the field ordering

      Returns:
         copy of "JobAnalysis" with the fields reordered

      Return type:
         *JobAnalysis*

   sort_fields(sort_key, reverse=False)

      Sort fields according to a sort key. This is the order the
      columns will appear in the analysis data.

         >>> ja.sort_fields(lambda k: len(k))

         | OK    | Name  | Step | Energy |
         |-------|-------|------|--------|
         | True  | job_1 | 1    | 42.1   |
         | True  | job_1 | 2    | 43.2   |
         | True  | job_1 | 3    | 42.5   |
         | True  | job_2 | 1    | 84.5   |
         | True  | job_2 | 1    | 112.2  |

      Parameters:
         * **sort_key** (*Callable**[**[**str**]**, **Any**]*) --
           sorting function which accepts the field key

         * **reverse** (*bool*) -- reverse sort order, defaults to
           "False"

      Returns:
         copy of "JobAnalysis" with the fields sorted

      Return type:
         *JobAnalysis*

   remove_field(key)

      Remove a field from the analysis. This removes a column from the
      analysis data.

         >>> ja.remove_field("OK")

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Parameters:
         **key** (*str*) -- unique identifier of the field

      Returns:
         copy of "JobAnalysis" with the field removed

      Return type:
         *JobAnalysis*

   remove_fields(keys)

      Remove multiple fields from the analysis. This removes columns
      from the analysis data.

         >>> ja.remove_fields(["OK", "N"])

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

      Parameters:
         **keys** (*Sequence**[**str**]*) -- unique identifiers of the
         fields

      Returns:
         copy of "JobAnalysis" with the fields removed

      Return type:
         *JobAnalysis*

   filter_fields(predicate)

      Retain fields from the analysis where the given predicate
      evaluates to "True" given the field values. In other words, this
      removes column(s) from the analysis data where the filter
      function evaluates to "False" given all the row values.

         >>> ja

         | OK    | Name  | Step | Energy |
         |-------|-------|------|--------|
         | True  | job_1 | 1    | 42.1   |
         | True  | job_1 | 2    | 43.2   |
         | True  | job_1 | 3    | 42.5   |
         | True  | job_2 | 1    | 84.5   |
         | True  | job_2 | 1    | 112.2  |

         >>> ja.filter_fields(lambda vals: all([not isinstance(v, int) or v > 50 for v in vals]))

         | Name  | Energy |
         |-------|--------|
         | job_1 | 42.1   |
         | job_1 | 43.2   |
         | job_1 | 42.5   |
         | job_2 | 84.5   |
         | job_2 | 112.2  |

      Parameters:
         **predicate** (*Callable**[**[**List**[**Any**]**]**,
         **bool**]*) -- filter function which takes values and
         evaluates to "True"/"False"

      Returns:
         copy of "JobAnalysis" with the field filter applied

      Return type:
         *JobAnalysis*

   remove_empty_fields()

      Remove field(s) from the analysis which have "None" for all
      values. This removes column(s) from the analysis data, where all
      rows have empty values.

         >>> ja.add_standard_field("ParentName")

         | Name  | OK    | ParentName |
         |-------|-------|------------|
         | job_1 | True  | None       |
         | job_2 | True  | None       |

         >>> ja.remove_empty_fields()

         | Name  | OK    |
         |-------|-------|
         | job_1 | True  |
         | job_2 | True  |

      Returns:
         copy of "JobAnalysis" with empty fields removed

      Return type:
         *JobAnalysis*

   remove_uniform_fields(tol=1e-08, ignore_empty=False)

      Remove field(s) from the analysis which evaluate the same for
      all values. This removes column(s) from the analysis data, where
      all rows have the same value.

         >>> ja.add_standard_field("ParentName")

         | Name  | OK    | ParentName |
         |-------|-------|------------|
         | job_1 | True  | None       |
         | job_2 | True  | None       |
         | job_3 | True  | p_job_4    |

         >>> ja.remove_uniform_fields()

         | Name  | ParentName |
         |-------|------------|
         | job_1 | None       |
         | job_2 | None       |
         | job_3 | p_job_4    |

         >>> ja.remove_uniform_fields(ignore_empty=True)

         | Name  |
         |-------|
         | job_1 |
         | job_2 |
         | job_3 |

      Parameters:
         * **tol** (*float*) -- absolute tolerance for numeric value
           comparison, all values must fall within this range

         * **ignore_empty** (*bool*) -- when "True" ignore "None"
           values and empty containers in comparison, defaults to
           "False"

      Returns:
         copy of "JobAnalysis" with uniform fields removed

      Return type:
         *JobAnalysis*

   add_standard_fields(keys)

      Adds multiple standard fields to the analysis.

      These are:

      * "Path": for "Job" attribute "path"

      * "Name": for "Job" attribute "name"

      * "OK": for "Job" method "ok()"

      * "Check": for "Job" method "check()"

      * "ErrorMsg": for "Job" method "get_errormsg()"

      * "ParentPath": for attribute "path" of "Job" attribute "parent"

      * "ParentName": for attribute "name" of "Job" attribute "parent"

      * "Formula": for method "get_formula()" of "Job" attribute
        "molecule"

      * "Smiles": for function "to_smiles()" for "Job" attribute
        "molecule"

      * "GyrationRadius": for function "gyration_radius()" for "Job"
        attribute "molecule"

      * "CPUTime": for method "readrkf()" with "General/CPUTime" for
        "Job" attribute "results"

      * "SysTime": for method "readrkf()" with "General/SysTime" for
        "Job" attribute "results"

      * "ElapsedTime": for method "readrkf()" with
        "General/ElapsedTime" for "Job" attribute "results"

         >>> ja

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

         >>> ja.add_standard_fields(["Path", "Smiles"])

         | Name  | Path        | Smiles |
         |-------|-------------|--------|
         | job_1 | /path/job_1 | N      |
         | job_2 | /path/job_2 | C=C    |

      Parameters:
         **keys** (*Sequence**[**Literal**[**'Path'**, **'Name'**,
         **'OK'**, **'Check'**, **'ErrorMsg'**, **'ParentPath'**,
         **'ParentName'**, **'Formula'**, **'Smiles'**,
         **'GyrationRadius'**, **'CPUTime'**, **'SysTime'**,
         **'ElapsedTime'**]**]*) -- sequence of keys for the analysis
         fields

      Returns:
         copy of "JobAnalysis" with the standard fields added

      Return type:
         *JobAnalysis*

   add_standard_field(key)

      Adds a standard field to the analysis.

      These are:

      * "Path": for "Job" attribute "path"

      * "Name": for "Job" attribute "name"

      * "OK": for "Job" method "ok()"

      * "Check": for "Job" method "check()"

      * "ErrorMsg": for "Job" method "get_errormsg()"

      * "ParentPath": for attribute "path" of "Job" attribute "parent"

      * "ParentName": for attribute "name" of "Job" attribute "parent"

      * "Formula": for method "get_formula()" of "Job" attribute
        "molecule"

      * "Smiles": for function "to_smiles()" for "Job" attribute
        "molecule"

      * "GyrationRadius": for function "gyration_radius()" for "Job"
        attribute "molecule"

      * "CPUTime": for method "readrkf()" with "General/CPUTime" for
        "Job" attribute "results"

      * "SysTime": for method "readrkf()" with "General/SysTime" for
        "Job" attribute "results"

      * "ElapsedTime": for method "readrkf()" with
        "General/ElapsedTime" for "Job" attribute "results"

         >>> ja

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

         >>> ja.add_standard_field("Path")

         | Name  | Path        |
         |-------|-------------|
         | job_1 | /path/job_1 |
         | job_2 | /path/job_2 |

      Parameters:
         **key** (*Literal**[**'Path'**, **'Name'**, **'OK'**,
         **'Check'**, **'ErrorMsg'**, **'ParentPath'**,
         **'ParentName'**, **'Formula'**, **'Smiles'**,
         **'GyrationRadius'**, **'CPUTime'**, **'SysTime'**,
         **'ElapsedTime'**]*) -- key for the analysis field

      Returns:
         copy of "JobAnalysis" with the additional standard field

      Return type:
         *JobAnalysis*

   static get_settings_field_key(key_tuple)

      Gets the key for a "JobAnalysis" settings field corresponding to
      a given nested "Settings" key. This will be a Pascal-case string
      of the settings nested key path e.g.``("input", "ams", "task")``
      will give the key "InputAmsTask".

         >>> JobAnalysis.get_settings_field_key(("input", "ams", "task"))

         InputAmsTask

      Parameters:
         **key_tuple** (*Tuple**[**Hashable**, **...**]*) -- nested
         tuple of keys in the "Settings" object

      Returns:
         corresponding key used for the "JobAnalysis" field

      Return type:
         str

   add_settings_field(key_tuple, display_name=None, fmt=None, expansion_depth=0)

      Add a field for a nested key from the job settings to the
      analysis. The key of the field will be a Pascal-case string of
      the settings nested key path e.g. "("input", "ams", "task")"
      will appear as field "InputAmsTask".

         >>> ja.add_settings_field(("input", "ams", "task"), display_name="Task")

         | Name  | Task        |
         |-------|-------------|
         | job_1 | SinglePoint |
         | job_2 | SinglePoint |

      Parameters:
         * **key_tuple** (*Tuple**[**Hashable**, **...**]*) -- nested
           tuple of keys in the settings object

         * **display_name** (*str** | **None*) -- name which will
           appear for the field when displayed in table

         * **fmt** (*str** | **None*) -- string format for how field
           values are displayed in table

         * **expansion_depth** (*int*) -- whether to expand field of
           multiple values into multiple rows, and recursively to what
           depth

      Returns:
         copy of "JobAnalysis" with the settings fields added

      Return type:
         *JobAnalysis*

   add_settings_fields(predicate=None, flatten_list=True)

      Add a field for all nested keys which satisfy the predicate from
      the job settings to the analysis. The key of the fields will be
      a Pascal-case string of the settings nested key path e.g.
      ("input", "ams", "task") will appear as field "InputAmsTask".

         >>> ja.add_settings_fields(lambda k: len(k) >= 3 and k[2].lower() == "xc")

         | Name  | InputAdfXcDispersion | InputAdfXcGga |
         |-------|----------------------|---------------|
         | job_1 | Grimme3              | PBE           |
         | job_2 | Grimme3              | PBE           |

      Parameters:
         * **predicate** (*Callable**[**[**Tuple**[**Hashable**,
           **...**]**]**, **bool**] **| **None*) -- optional predicate
           which evaluates to "True" or "False" given a nested key, by
           default will be "True" for every key

         * **flatten_list** (*bool*) -- whether to flatten lists in
           settings objects

      Returns:
         copy of "JobAnalysis" with settings fields added

      Return type:
         *JobAnalysis*

   static get_rkf_field_key(section, variable, file='ams')

      Gets the key for a "JobAnalysis" rkf field corresponding to a
      given file, section and varible. This will be a Pascal-case
      string of the file, section and varible e.g. "AmsGeneralTask".

         >>> JobAnalysis.get_rkf_field_key("General", "task")

         AmsGeneralTask

      Parameters:
         * **section** (*str*) -- section of the rkf file to read

         * **variable** (*str*) -- variable in the section of the rkf
           file to read

         * **file** (*str*) -- rkf file to use (without the .rkf
           extension)

      Returns:
         corresponding key used for the "JobAnalysis" field

      Return type:
         str

   add_rkf_field(section, variable, file='ams', display_name=None, fmt=None, expansion_depth=0)

      Add a field for given variable in a section of a rkf file. The
      key of the fields will be a Pascal-case string of the file,
      section and varible e.g. "AmsGeneralTask"

         >>> ja.add_rkf_field("General", "task")

         | Name  | ams:General%task |
         |-------|------------------|
         | job_1 | singlepoint      |
         | job_2 | singlepoint      |

      Parameters:
         * **section** (*str*) -- section of the rkf file to read

         * **variable** (*str*) -- variable in the section of the rkf
           file to read

         * **file** (*str*) -- rkf file to use (without the .rkf
           extension)

         * **display_name** (*str** | **None*) -- name which will
           appear for the field when displayed in table

         * **fmt** (*str** | **None*) -- string format for how field
           values are displayed in table

         * **expansion_depth** (*int*) -- whether to expand field of
           multiple values into multiple rows, and recursively to what
           depth

      Returns:
         copy of "JobAnalysis" with the additional field

      Return type:
         *JobAnalysis*

   add_settings_input_fields(include_system_block=False, flatten_list=True)

      Add a field for each input key in the "settings" object across
      all currently added jobs.

         >>> ja.add_settings_input_fields()

         | Name  | InputAdfBasisType | InputAdfXcDispersion | InputAdfXcGga | InputAmsTask |
         |-------|-------------------|----------------------|---------------|--------------|
         | job_1 | TZP               | Grimme3              | PBE           | SinglePoint  |
         | job_2 | TZP               | Grimme3              | PBE           | SinglePoint  |

      Parameters:
         * **include_system_block** (*bool*) -- whether to include
           keys for the system block, defaults to "False"

         * **flatten_list** (*bool*) -- whether to flatten lists in
           settings objects

      Returns:
         copy of "JobAnalysis" with the settings input fields added

      Return type:
         *JobAnalysis*

   remove_settings_fields()

      Remove all fields which were added as settings fields.

         >>> ja.add_settings_input_fields().remove_settings_fields()

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

      Returns:
         copy of "JobAnalysis" with settings input fields removed

      Return type:
         *JobAnalysis*

   __str__()

      Get string representation of analysis as Markdown table with a
      maximum of 5 rows and column width of 12.

         >>> str(ja)

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Returns:
         markdown table of analysis

      Return type:
         str

   __repr__()

      Get string representation of analysis as Markdown table with a
      maximum of 5 rows and column width of 12.

         >>> ja

         | Name  | OK   |
         |-------|------|
         | job_1 | True |
         | job_2 | True |

      Returns:
         markdown table of analysis

      Return type:
         str

   __getitem__(key)

      Get analysis data for a given field.

         >>> ja["Name"]

         ['job_1', 'job_2']

      Parameters:
         **key** (*str*) -- unique identifier for the field

      Returns:
         list of values for each job

      Return type:
         *List*[*Any*]

   __setitem__(key, value)

      Set analysis for given field.

         >>> ja["N"] = lambda j: len(j.molecule)
         >>> ja["N"]

         [4, 6]

      Parameters:
         * **key** (*str*) -- unique identifier for the field

         * **value** (*Callable**[**[**Job**]**, **Any**]*) --
           callable to extract the value for the field from a job

      Return type:
         None

   __delitem__(key)

      Delete analysis for given field.

         >>> del ja["OK"]
         >>> ja

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

      Parameters:
         **key** (*str*) -- unique identifier for the field

      Return type:
         None

   __getattr__(key)

      Fallback to get analysis for given field when an attribute is
      not present.

         >>> ja.Name

         ['job_1', 'job_2']

      Parameters:
         **key** (*str*) -- unique identifier for the field

      Returns:
         list of values for each job

      Return type:
         *List*[*Any*]

   __setattr__(key, value)

      Fallback to set analysis for given field.

         >>> ja.N = lambda j: len(j.molecule)
         >>> ja.N

         [4, 6]

      Parameters:
         * **key** (*str*) -- unique identifier for the field

         * **value** (*Any*) -- callable to extract the value for the
           field from a job

      Return type:
         None

   __delattr__(key)

      Fallback to set analysis for given field.

         >>> del ja.OK
         >>> ja

         | Name  |
         |-------|
         | job_1 |
         | job_2 |

      Parameters:
         **key** (*str*) -- unique identifier for the field

      Return type:
         None

   __contains__(key)

      Check whether a field is part of the analysis

         >>> "Name" in ja

         True

      Parameters:
         **key** (*str*) -- unique identifier for the field

      Returns:
         boolean flag for presence of field in the analysis

      Return type:
         bool
