Oracle Module¶

Abstract Base Class¶

class orchestrator.oracle.oracle_base.Oracle(**kwargs)[source]¶

Bases: Recorder, ABC

Abstract base class for ground truth calculations

The oracle class manages the construction and parsing of ground truth calculations to provide training or reference data. The input will typically consist of an atomic configuration and calculation parameters, the output will include the energy of the system, forces on each atom, and/or the stress on the cell.

__init__(**kwargs)[source]¶

set variables and initialize the recorder and default workflow

Parameters:: kwargs – arguments for instantiating Oracle. These arguments are defined by the concrete classes, and include items such as the path to an input file template (input_template), potential name (potential), and executable (code_path)

default_wf¶: default workflow to use within the Oracle class

run(path_type, input_args, configs, workflow=None, job_details=None)[source]¶

Main function to call ground truth calculation

The run method includes half of the main functionality of the oracle, taking atomic configurations as input and handling the submission of calculations to obtain the ground truth data. Configs is a dataset of 1 or more structures. run() will create independent jobs for each structure using the supplied workflow, with job_details parameterizing the job submission.

Parameters:

path_type (str) – specifier for the workflow path, to differentiate calculation types
input_args (dict) – input arguments to fill out the input file
configs (list) – list of configurations as ASE atoms to run ground truth calculations for
workflow (Workflow) – the workflow for managing job submission, if none are supplied, will use the default workflow defined in this class
Default: None
job_details (dict) – dict that includes any additional parameters for running the job (passed to submit_job())
Default: None

Returns:

a list of calculation IDs from the workflow.

Return type:

list

save_labeled_configs(paths, storage, dataset_name=None, dataset_handle=None, workflow=None)[source]¶

extract and save computed data to storage

The save_labeled_configs method includes the other half of the main functionality of the Oracle. Once the calculations are complete, the data they generate must be integrated with the strucutral configuration in a consistent framework to be used for training. This is done by parsing and ingesting the configuration and attached data (energies, forces, stresses) into a dataset handled by the Storage module.

Parameters:

paths (list[Union[int, str]]) – calc_ids or explicit paths associated with each config. If calc_ids are supplied, the path is extracted from the JobStatus. Calc IDs are generally prefered as they can also carry metadata with them.
storage (Storage) – specific module that handles the staroge of data.
dataset_name (Optional[str]) – The name of the datset in the Storage where the configurations should be saved. If a dataset with this name already exists, it will be added to. Otherwise a new dataset with this name will be created. If None and dataset_handle is not provided, then the class default (date stamped) is used.
Default: None.
dataset_handle (Optional[str]) – the handle to identify the dataset in Storage. If provided, will add data to this dataset. If None and dataset_name is not provided the class default name (date stamped) is used.
Default: None
workflow (Optional[Workflow]) – the workflow for managing job submission, if none are supplied, will use the default workflow defined in this class. Should be consistent with the workflow supplied for the run calls.
Default: None

Return type:

str

Returns:

dataset handle

data_from_calc_ids(calc_ids=None, workflow=None)[source]¶

Given a list of calc_ids, will iterate over the list and make relevant checks to ensure the returned values have the same inputs.

Parameters:: calc_ids (Optional[list[int]]) – List of calculation ids that are used to parse relevant data.
Return type:: tuple[list[Atoms], dict]
Returns:: List of Atoms object and a nested dictionary containing the input parameters of the code and the universal values.

abstract write_input(run_path, input_args, config)[source]¶

generate an input file for running the ground truth calculation

This method will write the requisite input file in the run_path using the input_args of a given configuration. Specific implementations may leverage additional helper functions to construct the input.

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) – input arguments for the oracle input file, typically controlled using a template
config (Atoms) – the configuration as an Atoms object

Returns:

name of written input file

Return type:

str

abstract get_run_command(**job_details)[source]¶

return the command to run an oracle calculation

this method formats the run command based on the code_path internal variable set at instantiation of the Oracle, while the args dictionary can be used to pass any necessary extra parameters to the specific implementations.

Parameters:: job_details – dictionary for parameters to decorate or enable the run command. Keys are defined in concrete classes.
Default: None
Returns:: implementation dependent
Return type:: implementation dependent

abstract parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

process calculation output to extract data in a consistent format

Parse the output from the Oracle calculation into ASE Atoms objects. The resulting Atoms will include the atomic configurations, cell info, and possibly energies, forces, and stresses. Units are: total system energy in eV, forces on each atom in eV/A, and stress on the system in eV/A^3

Parameters:

run_path (str) – directory where the oracle output file resides.
calc_id (Optional[int]) – Job ID of the calculation to parse.
workflow (Optional[Workflow]) – Workflow object of Orchestrator.

Return type:

Atoms

Returns:

Atoms of the configurations with attached properties and metadata.

static get_calc_id_from_data_source_tag(config)[source]¶

helper method for retrieving the calc_id/pk from configs’ metdata

Parameters:: config (Atoms) – config parsed from an oracle calculation
Returns:: calc_id
Return type:: int

LAMMPS+X¶

class orchestrator.oracle.lammps.LAMMPSOracle(code_path, potential, **kwargs)[source]¶

Bases: Oracle

Class for reating, running, and parsing LAMMPS single point calculations

Resposible for making LAMMPS input file, run the code and extract energy, forces, and stress tensor from the LAMMPS output

__init__(code_path, potential, **kwargs)[source]¶

set variables and initialize the recorder and default workflow

Parameters:

code_path (str) – path of the LAMMPS executable
potential (str) – string of the potential to use

write_input(run_path, input_args, config)[source]¶

Write LAMMPS input file - implemented by subclasses

Write LAMMPS input script to run a single point calculation and output energy, forces and stress for the configuration given in the conf.lmp file. This method writes both the lammps.in and conf.lmp files. Use ASE to write the configuration file.

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) – dictionary of input arguments for the lammps input file template (not currently used, written in function)
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Returns:

name of written input file

Return type:

str

get_run_command(input_file='lammps.in', **unused_job_details)[source]¶

return the command to execute a LAMMPS calculation

this method formats the run command based on the code_path internal variable set at instantiation of the Oracle, which the Workflow will execute in the proper run_path.

Parameters:: input_file (str) – name of the input file that was written by write_input()
Returns:: single line string with code execution statement
Return type:: str

AiiDA¶

VASP¶

class orchestrator.oracle.aiida.vasp.AiidaVaspOracle(code_str=None, workchain=None, settings=None, clean_workdir=True, group=None, **kwargs)[source]¶

Bases: AiidaOracle

Class for creating, running, and parsing VASP calculations

Responsible for making any needed input files, run the code, and extract energy, forces, and stress tensor from output.

__init__(code_str=None, workchain=None, settings=None, clean_workdir=True, group=None, **kwargs)[source]¶

Class for creating, running, and parsing VASP calculations.

Parameters:

code_str (Optional[str]) – Name of the code in the AiiDA database. e.g. vasp_std@server
workchain (Optional[str]) – Name of the workchain in AiiDA for VASP. e.g. vasp.relax
settings (Optional[dict]) – Controls the parsing behavior and other attributes.
clean_workdir (bool) – Will clean the working directory on the remote machine if True.
group (Optional[str]) – Creates a group node in AiiDA to store all of the calculations for easy parsing afterwards based on the string name.

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

Process calculation output to extract data in a consistent format

Parse the output from the Espresso calculation into ASE Atoms objects. The resulting Atoms will include the atomic configurations, cell info, and possibly energies, forces, and stresses. Units are: total system energy in eV, forces on each atom in eV/A, and stress on the system in eV/A^3

Parameters:

run_path (str) – Unique AiiDA identifier to load a node from the database
calc_id (Optional[int]) – Calculation ID returned from an Oracle.
workflow (Optional[Workflow]) – Workflow object from orchestrator that has attached metadata.

Return type:

Atoms

Returns:

ASE Atoms object of the configuration and attached properties as well as a dictionary of metadata that should be stored with the configuration.

default_settings()[source]¶

The settings object controls parsing of the VASP calculation.

Return type:: AttributeDict
Returns:: Attribute dictionary with parser settings.

default_relax_options(overrides=None)[source]¶

To perform structure relaxations with AiiDA for VASP, a relax object is created to control the various options.

Parameters:: overrides (Optional[dict]) – Values that will change the default behavior. This can include changing relax.perform from False to True to perform a geometry optimization.
Return type:: AttributeDict
Returns:: Attribute dictionary with relaxation objects

static translate_universal_parameters(parameters)[source]¶

Orchestrator has predefined universal input values for varying codes to allow some transferability. Each Oracle will need a function to translate those values from the specific code. This function will take the VASP INCAR values and convert them to the universal values to be stored with the dataset on the initial submission.

Parameters:: parameters (dict) – Dictionary containing all the INCAR parameters to run the simulation.
Return type:: dict
Returns:: Dictionary of universal input parameters for database storage.

Quantum Espresso¶

class orchestrator.oracle.aiida.espresso.AiidaEspressoOracle(code_str=None, workchain=None, clean_workdir=True, group=None, **kwargs)[source]¶

Bases: AiidaOracle

Class for submitting quantum espresso calculations through AiiDA.

Resposible for formatting the input for a workchain.

Parameters:: oracle_args – arguments for instantiating a EspressoOracle. This includes any input values needed for the specified workchain.

__init__(code_str=None, workchain=None, clean_workdir=True, group=None, **kwargs)[source]¶

Class for creating, running, and parsing Quantum Espresso calculations.

Parameters:

code_str (Optional[str]) – Name of the code in the AiiDA database. e.g. pw@server
workchain (Optional[str]) – Name of the workchain in AiiDA for Quantum Espresso. e.g. quantumespresso.relax
clean_workdir (bool) – Will clean the working directory on the remote machine if True.
group (Optional[str]) – Creates a group node in AiiDA to store all of the calculations for easy parsing afterwards based on the string name.

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

Process calculation output to extract data in a consistent format.

Parameters:

run_path (str) – Unique AiiDA identifier to load a node from the database
calc_id (Optional[int]) – Calculation ID returned from an Oracle.
workflow (Optional[Workflow]) – Workflow object from orchestrator that has attached metadata.

Return type:

Atoms

Returns:

tuple of Atoms of the configurations and attached properties as well as a dictionary of metadata that should be stored with the configuration.

get_relax_type(parameters)[source]¶

Selects the correct RelaxType based on the input parameters for Quantum Espresso.

Parameters:: parameters (dict) – Input parameters for a Quantum Espresso calculation.
Return type:: RelaxType
Returns:: The specified relaxation type

static translate_universal_parameters(parameters, structure)[source]¶

Orchestrator has predefined universal input values for varying codes to allow some transferability. Each Oracle will need a function to translate those values from the specific code. This function will take the Quantum Espresso input values and convert them to the universal values to be stored with the dataset on the initial submission.

Parameters:

parameters (dict) – Dictionary containing all the pw.in parameters to run the simulation.
structure (Atoms) – ASE Atoms object that will be used to map the magnetic moments.

Return type:

dict

Returns:

Dictionary of universal input parameters for database storage.

Concrete Implementations¶

Espresso¶

class orchestrator.oracle.espresso.EspressoOracle(code_path, input_template, **kwargs)[source]¶

Bases: Oracle

Class for creating, running, and parsing quantum espresso calculations

Resposible for making PWscf input file, run the code and extract energy, forces, and stress tensor from PWscf output

__init__(code_path, input_template, **kwargs)[source]¶

set variables and initialize the recorder and default workflow

Parameters:

code_path (str) – path of the QE executable
input_template (str) – path to a templated input file to use to write inputs

write_atomic_positions(pos, atype)[source]¶

Write atomic coordinates in PWscf input file format

Helper function to write atomic coordinates in the format of quantum espresso DFT calculations. Returns the combined string.

Parameters:

pos (np.ndarray) – atomic positions in cartesian coords in units of angstroms, array will be size [Nx3]
atype (np.ndarray) – atomic symbols cooresponding to each position entry in pos, array of size [N]

Returns:

formatted atomic coordinates for QE input

Return type:

str

write_cell(cell)[source]¶

Write cell lattice in PWscf input file format

Helper function to write unit cell information in the format of quantum espresso DFT calculations. Returns the formatted string

Parameters:: cell (np.ndarray) – calculation unit cell in Angstrom ([3x3] array)
Returns:: formatted cell for QE input
Return type:: str

write_input(run_path, input_args, config)[source]¶

Write PWscf input file

This is the main method that utilizes the helper functions and the input template to write the full QE input file.

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) – dictionary of input arguments for the espresso input file template
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Returns:

name of written input file

Return type:

str

get_run_command(input_file='espresso.in', npools=1, nband=1, nimage=1, **unused_job_details)[source]¶

return the command to run a quantum espresso calculation

this method formats the run command based on the code_path internal variable set at instantiation of the Oracle, which the Workflow will execute in the proper run_path. args include parallelization schemes for espresso, including: ‘nimage’, ‘npool’, and ‘nband’. Each will be set to 1 if not specified. These are generally passed in as a dictionary, which is expanded with the ** operator.

Parameters:

input_file (str) – name of the input file that was written by write_input()
npools (int) – k-point parallelization
Default: 1
nband (int) – band parallelization
Default: 1
nimage (int) – image parallelization
Default: 1

Returns:

single line string with code execution statement

Return type:

str

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

process calculation output to extract data in a consistent format

Parameters:

run_path (str) – directory where the oracle output file resides
calc_id (Optional[int]) – Job ID of the calculation to parse.
workflow (Optional[Workflow]) – Workflow object of Orchestrator.

Return type:

Atoms

Returns:

Atoms of the configuration and attached properties and a dictionary of metadata that should be stored with the configuration.

ry_to_metal_units()[source]¶

Constants to convert Rydberg units to metal units

Espresso specific helper for unit conversions as QE outputs in Ry and bohr instead of eV and angstrom

get_pw_parameters(runpath=None)[source]¶

Read in the input parameters from a pw.x calculation.

Parameters:: runpath (Optional[str]) – directory where the oracle calculation files reside.
Return type:: dict
Returns:: Dict of QE input parameters.

translate_universal_parameters(runpath)[source]¶

Orchestrator has predefined universal input values for varying codes to allow some transferability. Each Oracle will need a function to translate those values from the specific code. This function will take the Quantum Espresso input values and convert them to the universal values to be stored with the dataset on the initial submission.

Parameters:

parameters – Dictionary containing all the pw.in parameters to run the simulation.
runpath (str) – Directory where the oracle calculation files reside.

Return type:

dict

Returns:

Dictionary of universal input parameters for database storage.

VASP¶

class orchestrator.oracle.vasp.VaspOracle(code_path=None, **kwargs)[source]¶

Bases: Oracle

Class for creating, running, and parsing quantum espresso calculations

Resposible for making PWscf input file, run the code and extract energy, forces, and stress tensor from PWscf output

__init__(code_path=None, **kwargs)[source]¶

set variables and initialize the recorder and default workflow

Parameters:: code_path (Union[str, PathLike, None]) – path of the VASP executable

write_input(run_path, input_args, config)[source]¶

Write input files for VASP calculation.

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) –
dictionary of input arguments for the INCAR file. This will contain multiple values with the following format:
```
input_args = {
    incar: {INCAR parameters},
    kpoints: [5, 5, 5],
    kspacing: 0.1,
    pseudo_path: '/path/to/pseudos',
    pseudo_mapping: {
        'Fe': 'Fe_pv'
    }
}
```
pseudo_mapping can be set to use a specific pseudo based on the folder name. Otherwise, the default is to use the basic pseudo. Kpoints or kspacing (no 2pi factor) may be defined. kspacing is used by default
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Return type:

str

Returns:

name of written input file

get_run_command(**kwargs)[source]¶

return the command to run a VASP calculation

this method formats the run command based on the code_path internal variable set at instantiation of the Oracle, which the Workflow will execute in the proper run_path.

Return type:: str
Returns:: single line string with code execution statement

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

process calculation output to extract data in a consistent format

Parse the output from the VASP calculation into ASE Atoms objects. The resulting Atoms will include the atomic configurations, cell info, and possibly energies, forces, and stresses. Units are: total system energy in eV, forces on each atom in eV/A, and stress on the system in eV/A^3

Parameters:

run_path (str) – directory where the oracle output file resides
calc_id (Optional[int]) – Job ID of the calculation to parse.
workflow (Optional[Workflow]) – Workflow object of Orchestrator.

Return type:

Atoms

Returns:

Atoms of the configuration and attached properties and a dictionary of metadata that should be stored with the configuration.

get_parameters(runpath=None)[source]¶

Read in the input parameters from a VASP calculation.

Parameters:: runpath (Optional[str]) – directory where the oracle calculation files reside.
Return type:: dict
Returns:: Dict of VASP input parameters.

static translate_universal_parameters(parameters)[source]¶

Parameters:: parameters (dict) – Dictionary containing all the INCAR parameters to run the simulation.
Return type:: dict
Returns:: Dictionary of universal input parameters for database storage.

KIM¶

class orchestrator.oracle.kim.KIMOracle(potential, **kwargs)[source]¶

Bases: Oracle

Class for creating, running, and parsing ASE calculations with KIM

Resposible for making ASE “input file”, running the calculator, and extracting energy and forces from the output. Calculating the stress tensor is not currently supported.

__init__(potential, **kwargs)[source]¶

set variables and initialize the recorder and default workflow

Parameters:: potential (str) – string of the KIM potential installed in the KIM API

write_input(run_path, input_args, config)[source]¶

Make ASE calculator (Atoms)

When using ASE+KIM potential, calculations are not done externally, but instead directly in the python environment from the Atoms data structure. Thus no input file is actually written, but instead the internal atoms attribute is set.

Parameters:

run_path (str) – directory path where the file is written, not used
input_args (dict) – dictionary of input arguments for Atoms (not currently used, constructed internally)
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Returns:

name of written input file

Return type:

str

get_run_command(run_path, **unused_job_details)[source]¶

assign the KIM calculator to the Atoms attribute

this method bypasses the actual submission of a job to any workflow, instead using the internal python environment/KIM to calculate ground truth values. Calculation outputs are saved in the run_path specified in the args dict.

Parameters:: run_path (str) – output location of the calculation
Returns:: None
Return type:: None

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

process calculation output to extract data in the format for Storage

Parse the output from the KIM calculation into ASE Atoms objects. The resulting Atoms will include the atomic configurations, cell info, and possibly energies, forces, and stresses. Units are: total system energy in eV, forces on each atom in eV/A, and stress on the system in eV/A^3

Parameters:

run_path (str) – directory where the oracle output file resides
calc_id (Optional[int]) – Calculation ID returned from an Oracle.
workflow (Optional[Workflow]) – Workflow object from orchestrator that has attached metadata.

Return type:

Atoms

Returns:

Atoms of the configuration and attached properties and a dictionary of metadata that should be stored with the configuration.

LAMMPS+KIM¶

class orchestrator.oracle.lammps.LAMMPSKIMOracle(code_path, potential, **kwargs)[source]¶

Bases: LAMMPSOracle

Class for reating, running, and parsing LAMMPS+KIM calculations

Resposible for making LAMMPS input file, run the code and extract energy, forces, and stress tensor from the LAMMPS output

write_input(run_path, input_args, config)[source]¶

Write LAMMPS input file using a KIM potential

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) – dictionary of input arguments for the lammps input file template (not currently used, written in function)
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Returns:

name of written input file

Return type:

str

parse_for_storage(run_path='', calc_id=None, workflow=None)[source]¶

process calculation output to extract data in a consistent format

Parse the output from the LAMMPS calculation into ASE Atoms objects. The resulting Atoms will include the atomic configurations, energies, forces, and stresses. Units are: total system energy in eV, forces on each atom in eV/A, and stress on the system in eV/A^3

Parameters:

run_path (str) – directory where the oracle output file resides
calc_id (Optional[int]) – Calculation ID returned from an Oracle.
workflow (Optional[Workflow]) – Workflow object from orchestrator that has attached metadata.

Return type:

Atoms

Returns:

Atoms of the configuration and attached properties and a dictionary of metadata that should be stored with the configuration.

LAMMPS+SNAP¶

class orchestrator.oracle.lammps.LAMMPSSnapOracle(code_path, potential, **kwargs)[source]¶

Bases: LAMMPSOracle

Class for creating, running, and parsing LAMMPS+SNAP calculations

Resposible for making LAMMPS input file, run the code and extract energy, forces, and stress tensor from the LAMMPS output

write_input(run_path, input_args, config)[source]¶

Write LAMMPS input file using a SNAP potential

Parameters:

run_path (str) – directory path where the file is written
input_args (dict) – dictionary of input arguments for the lammps input file template which can contain the model_path key
config (Atoms) – ASE Atoms object of the configuration, containing position, cell, and atom type information at minimum

Returns:

name of written input file

Return type:

str

parse_for_storage(run_path)[source]¶

process calculation output to extract data in a consistent format

Parameters:: run_path (str) – directory where the oracle output file resides
Returns:: Atoms of the configuration and attached properties and a dictionary of metadata that should be stored with the configuration.
Return type:: Atoms

Oracle Builder¶

orchestrator.oracle.factory.oracle_factory = <orchestrator.utils.module_factory.ModuleFactory object>¶: default factory for oracles, includes QE, LAMMPS, and KIM

class orchestrator.oracle.factory.AiidaOracle[source]¶: Bases: object

class orchestrator.oracle.factory.OracleBuilder(factory=<orchestrator.utils.module_factory.ModuleFactory object>)[source]¶

Bases: ModuleBuilder

Constructor for oracles added in the factory

Set the factory to be used for the builder. The default is to use the oracle_factory generated at the end of this module. A user defined ModuleFactory can optionally be supplied instead.

Parameters:: factory (ModuleFactory) – a oracle factory
Default: oracle_factory

__init__(factory=<orchestrator.utils.module_factory.ModuleFactory object>)[source]¶

Constructor for the OracleBuilder, sets the factory to build from

Parameters:: factory (Optional[ModuleFactory]) – An oracle factory
Default: oracle_factory

build(oracle_type, oracle_args=None)[source]¶

Return an instance of the specified oracle.

The build method takes the specifier and input arguments to construct a concrete oracle instance.

Parameters:

oracle_type (str) – Token of an oracle which has been added to the factory.
oracle_args (Optional[dict]) – Dictionary of arguments needed to instantiate a given oracle class.

Return type:

Oracle

Returns:

Instantiated concrete Oracle.

orchestrator.oracle.factory.oracle_builder = <orchestrator.oracle.factory.OracleBuilder object>¶: oracle builder object which can be imported for use in other modules

orchestrator.oracle.factory.aiida_oracle_builder = <orchestrator.oracle.factory.OracleBuilder object>¶: oracle builder object which can be used to build aiida oracles