disc_from_exe module¶

Make a discipline from an executable.

class gemseo.wrappers.disc_from_exe.DiscFromExe(input_template, output_template, output_folder_basepath, executable_command, input_filename, output_filename, folders_iter=FoldersIter.NUMBERED, name=None, parse_outfile_method=Parser.TEMPLATE, write_input_file_method=None, parse_out_separator='=', use_shell=True)[source]¶

Bases: MDODiscipline

Generic wrapper for executables.

This MDODiscipline uses template files describing the input and output variables.

The templates can be generated with a graphical user interface (GUI). by executing the module template_grammar_editor.

An input template file is a JSON file formatted as

{
   "a": GEMSEO_INPUT{a::1.0},
   "b": GEMSEO_INPUT{b::2.0},
   "c": GEMSEO_INPUT{c::3.0}
}

where "a" is the name of an input, and 1.0 is its default value. Similarly, an output template file is a JSON file formatted as

{
   "a": GEMSEO_OUTPUT{a::1.0},
   "b": GEMSEO_OUTPUT{b::2.0},
   "c": GEMSEO_OUTPUT{c::3.0}
}

where "a" is the name of an output, and 1.0 is its default value.

The current limitations are

Only one input and one output template file, otherwise, inherit from this class and modify the parsers. Only limited input writing and output parser strategies are implemented. To change that, you can pass custom parsing and writing methods to the constructor.
The only limitation in the current file format is that it must be a plain text file and not a binary file. In this case, the way of interfacing it is to provide a specific parser to the DiscFromExe, with the write_input_file_method() and parse_outfile_method() arguments of the constructor.
For security reasons, the executable is executed via the Python subprocess library with no shell. If a shell is needed, you may override this in a derived class.

Initialize self. See help(type(self)) for accurate signature.

Parameters:

input_template (str | Path) – The path to the input template file. The input locations in the file are marked by GEMSEO_INPUT{input_name::1.0}, where input_name is the name of the input variable, and 1.0 is its default value.
output_template (str | Path) – The path to the output template file. The output locations in the file are marked by GEMSEO_OUTPUT{output_name::1.0}, where output_name is the name of the output variable, and 1.0 is its default value.
output_folder_basepath (str | Path) – The base path of the execution directories.
executable_command (str) – The command to run the executable. E.g. python my_script.py -i input.txt -o output.txt
input_filename (str | Path) – The name of the input file to be generated in the output folder. E.g. "input.txt".
output_filename (str | Path) – The name of the output file to be generated in the output folder. E.g. "output.txt".
folders_iter (FoldersIter) –
The type of unique identifiers for the output folders. If NUMBERED, the generated output folders will be f"output_folder_basepath{i+1}", where i is the maximum value of the already existing f"output_folder_basepath{i}" folders. Otherwise, a unique number based on the UUID function is generated. This last option shall be used if multiple MDO processes are run in the same work folder.

By default it is set to “NUMBERED”.
name (str | None) – The name of the discipline. If None, use the class name.
parse_outfile_method (Parser | OutputParser) –
The optional method that can be provided by the user to parse the output template file. If the KEY_VALUE is used as output parser, the user may specify the separator key.

By default it is set to “TEMPLATE”.
write_input_file_method (InputWriter | None) – The method to write the input data file. If None, use write_input_file().
parse_out_separator (str) –
The separator used for the KEY_VALUE output parser.

By default it is set to “=”.
use_shell (bool) –
This argument is ignored and will be removed, the shell is not used.

By default it is set to True.

Raises:

TypeError – If the provided parse_outfile_method is not callable. If the provided write_input_file_method is not callable.

class ApproximationMode(value)¶

Bases: StrEnum

The approximation derivation modes.

COMPLEX_STEP = 'complex_step'¶: The complex step method used to approximate the Jacobians by perturbing each variable with a small complex number.

FINITE_DIFFERENCES = 'finite_differences'¶: The finite differences method used to approximate the Jacobians by perturbing each variable with a small real number.

class CacheType(value)¶

Bases: StrEnum

The name of the cache class.

HDF5 = 'HDF5Cache'¶

MEMORY_FULL = 'MemoryFullCache'¶

NONE = ''¶: No cache is used.

SIMPLE = 'SimpleCache'¶

class ExecutionStatus(value)¶

Bases: StrEnum

The execution statuses of a discipline.

DONE = 'DONE'¶

FAILED = 'FAILED'¶

LINEARIZE = 'LINEARIZE'¶

PENDING = 'PENDING'¶

RUNNING = 'RUNNING'¶

VIRTUAL = 'VIRTUAL'¶

class GrammarType(value)¶

Bases: StrEnum

The name of the grammar class.

JSON = 'JSONGrammar'¶

PYDANTIC = 'PydanticGrammar'¶

SIMPLE = 'SimpleGrammar'¶

class InitJacobianType(value)¶

Bases: StrEnum

The way to initialize Jacobian matrices.

DENSE = 'dense'¶: The Jacobian is initialized as a NumPy ndarray filled in with zeros.

EMPTY = 'empty'¶: The Jacobian is initialized as an empty NumPy ndarray.

SPARSE = 'sparse'¶: The Jacobian is initialized as a SciPy CSR array with zero elements.

class LinearizationMode(value)¶

Bases: StrEnum

An enumeration.

ADJOINT = 'adjoint'¶

AUTO = 'auto'¶

COMPLEX_STEP = 'complex_step'¶

DIRECT = 'direct'¶

FINITE_DIFFERENCES = 'finite_differences'¶

REVERSE = 'reverse'¶

class ReExecutionPolicy(value)¶

Bases: StrEnum

The re-execution policy of a discipline.

DONE = 'RE_EXEC_DONE'¶

NEVER = 'RE_EXEC_NEVER'¶

classmethod activate_time_stamps()¶

Activate the time stamps.

For storing start and end times of execution and linearizations.

Return type:: None

add_differentiated_inputs(inputs=None)¶

Add the inputs against which to differentiate the outputs.

If the discipline grammar type is MDODiscipline.GrammarType.JSON and an input is either a non-numeric array or not an array, it will be ignored. If an input is declared as an array but the type of its items is not defined, it is assumed as a numeric array.

If the discipline grammar type is MDODiscipline.GrammarType.SIMPLE and an input is not an array, it will be ignored. Keep in mind that in this case the array subtype is not checked.

Parameters:: inputs (Iterable[str] | None) – The input variables against which to differentiate the outputs. If None, all the inputs of the discipline are used.
Raises:: ValueError – When the inputs wrt which differentiate the discipline are not inputs of the latter.
Return type:: None

add_differentiated_outputs(outputs=None)¶

Add the outputs to be differentiated.

If the discipline grammar type is MDODiscipline.GrammarType.JSON and an output is either a non-numeric array or not an array, it will be ignored. If an output is declared as an array but the type of its items is not defined, it is assumed as a numeric array.

If the discipline grammar type is MDODiscipline.GrammarType.SIMPLE and an output is not an array, it will be ignored. Keep in mind that in this case the array subtype is not checked.

Parameters:: outputs (Iterable[str] | None) – The output variables to be differentiated. If None, all the outputs of the discipline are used.
Raises:: ValueError – When the outputs to differentiate are not discipline outputs.
Return type:: None

add_namespace_to_input(name, namespace)¶

Add a namespace prefix to an existing input grammar element.

The updated input grammar element name will be namespace + namespaces_separator + name.

Parameters:

name (str) – The element name to rename.
namespace (str) – The name of the namespace.

Return type:

None

add_namespace_to_output(name, namespace)¶

Add a namespace prefix to an existing output grammar element.

The updated output grammar element name will be namespace + namespaces_separator + name.

Parameters:

name (str) – The element name to rename.
namespace (str) – The name of the namespace.

Return type:

None

add_status_observer(obs)¶

Add an observer for the status.

Add an observer for the status to be notified when self changes of status.

Parameters:: obs (Any) – The observer to add.
Return type:: None

auto_get_grammar_file(is_input=True, name=None, comp_dir=None)¶

Use a naming convention to associate a grammar file to the discipline.

Search in the directory comp_dir for either an input grammar file named name + "_input.json" or an output grammar file named name + "_output.json".

Parameters:

is_input (bool) –
Whether to search for an input or output grammar file.

By default it is set to True.
name (str | None) – The name to be searched in the file names. If None, use the name of the discipline class.
comp_dir (str | Path | None) – The directory in which to search the grammar file. If None, use the GRAMMAR_DIRECTORY if any, or the directory of the discipline class module.

Returns:

The grammar file path.

Return type:

Path

check_input_data(input_data, raise_exception=True)¶

Check the input data validity.

Parameters:

input_data (Mapping[str, Any]) – The input data needed to execute the discipline according to the discipline input grammar.
raise_exception (bool) –
Whether to raise on error.

By default it is set to True.

Return type:

None

check_jacobian(input_data=None, derr_approx=ApproximationMode.FINITE_DIFFERENCES, step=1e-07, threshold=1e-08, linearization_mode='auto', inputs=None, outputs=None, parallel=False, n_processes=2, use_threading=False, wait_time_between_fork=0, auto_set_step=False, plot_result=False, file_path='jacobian_errors.pdf', show=False, fig_size_x=10, fig_size_y=10, reference_jacobian_path=None, save_reference_jacobian=False, indices=None)¶

Check if the analytical Jacobian is correct with respect to a reference one.

If reference_jacobian_path is not None and save_reference_jacobian is True, compute the reference Jacobian with the approximation method and save it in reference_jacobian_path.

If reference_jacobian_path is not None and save_reference_jacobian is False, do not compute the reference Jacobian but read it from reference_jacobian_path.

If reference_jacobian_path is None, compute the reference Jacobian without saving it.

Parameters:

input_data (Mapping[str, ndarray] | None) – The input data needed to execute the discipline according to the discipline input grammar. If None, use the MDODiscipline.default_inputs.
derr_approx (ApproximationMode) –
The approximation method, either “complex_step” or “finite_differences”.

By default it is set to “finite_differences”.
threshold (float) –
The acceptance threshold for the Jacobian error.

By default it is set to 1e-08.
linearization_mode (str) –
the mode of linearization: direct, adjoint or automated switch depending on dimensions of inputs and outputs (Default value = ‘auto’)

By default it is set to “auto”.
inputs (Iterable[str] | None) – The names of the inputs wrt which to differentiate the outputs.
outputs (Iterable[str] | None) – The names of the outputs to be differentiated.
step (float) –
The differentiation step.

By default it is set to 1e-07.
parallel (bool) –
Whether to differentiate the discipline in parallel.

By default it is set to False.
n_processes (int) –
The maximum simultaneous number of threads, if use_threading is True, or processes otherwise, used to parallelize the execution.

By default it is set to 2.
use_threading (bool) –
Whether to use threads instead of processes to parallelize the execution; multiprocessing will copy (serialize) all the disciplines, while threading will share all the memory This is important to note if you want to execute the same discipline multiple times, you shall use multiprocessing.

By default it is set to False.
wait_time_between_fork (float) –
The time waited between two forks of the process / thread.

By default it is set to 0.
auto_set_step (bool) –
Whether to compute the optimal step for a forward first order finite differences gradient approximation.

By default it is set to False.
plot_result (bool) –
Whether to plot the result of the validation (computed vs approximated Jacobians).

By default it is set to False.
file_path (str | Path) –
The path to the output file if plot_result is True.

By default it is set to “jacobian_errors.pdf”.
show (bool) –
Whether to open the figure.

By default it is set to False.
fig_size_x (float) –
The x-size of the figure in inches.

By default it is set to 10.
fig_size_y (float) –
The y-size of the figure in inches.

By default it is set to 10.
reference_jacobian_path (str | Path | None) – The path of the reference Jacobian file.
save_reference_jacobian (bool) –
Whether to save the reference Jacobian.

By default it is set to False.
indices (Iterable[int] | None) – The indices of the inputs and outputs for the different sub-Jacobian matrices, formatted as {variable_name: variable_components} where variable_components can be either an integer, e.g. 2 a sequence of integers, e.g. [0, 3], a slice, e.g. slice(0,3), the ellipsis symbol (…) or None, which is the same as ellipsis. If a variable name is missing, consider all its components. If None, consider all the components of all the inputs and outputs.

Returns:

Whether the analytical Jacobian is correct with respect to the reference one.

Return type:

bool

check_output_data(raise_exception=True)¶

Check the output data validity.

Parameters:

raise_exception (bool) –

Whether to raise an exception when the data is invalid.

By default it is set to True.

Return type:

None

classmethod deactivate_time_stamps()¶

Deactivate the time stamps.

For storing start and end times of execution and linearizations.

Return type:: None

execute(input_data=None)¶

Execute the discipline.

This method executes the discipline:

Adds the default inputs to the input_data if some inputs are not defined in input_data but exist in MDODiscipline.default_inputs.
Checks whether the last execution of the discipline was called with identical inputs, i.e. cached in MDODiscipline.cache; if so, directly returns self.cache.get_output_cache(inputs).
Caches the inputs.
Checks the input data against MDODiscipline.input_grammar.
If MDODiscipline.data_processor is not None, runs the preprocessor.
Updates the status to MDODiscipline.ExecutionStatus.RUNNING.
Calls the MDODiscipline._run() method, that shall be defined.
If MDODiscipline.data_processor is not None, runs the postprocessor.
Checks the output data.
Caches the outputs.
Updates the status to MDODiscipline.ExecutionStatus.DONE or MDODiscipline.ExecutionStatus.FAILED.
Updates summed execution time.

Parameters:: input_data (Mapping[str, Any] | None) – The input data needed to execute the discipline according to the discipline input grammar. If None, use the MDODiscipline.default_inputs.
Returns:: The discipline local data after execution.
Raises:: RuntimeError – When residual_variables are declared but self.run_solves_residuals is False. This is not supported yet.
Return type:: dict[str, Any]

static from_pickle(file_path)¶

Deserialize a discipline from a file.

Parameters:: file_path (str | Path) – The path to the file containing the discipline.
Returns:: The discipline instance.
Return type:: MDODiscipline

get_all_inputs()¶

Return the local input data as a list.

The order is given by MDODiscipline.get_input_data_names().

Returns:: The local input data.
Return type:: list[Any]

get_all_outputs()¶

Return the local output data as a list.

The order is given by MDODiscipline.get_output_data_names().

Returns:: The local output data.
Return type:: list[Any]

static get_data_list_from_dict(keys, data_dict)¶

Filter the dict from a list of keys or a single key.

If keys is a string, then the method return the value associated to the key. If keys is a list of strings, then the method returns a generator of value corresponding to the keys which can be iterated.

Parameters:

keys (str | Iterable) – One or several names.
data_dict (dict[str, Any]) – The mapping from which to get the data.

Returns:

Either a data or a generator of data.

Return type:

Any | Iterator[Any]

get_disciplines_in_dataflow_chain()¶

Return the disciplines that must be shown as blocks in the XDSM.

By default, only the discipline itself is shown. This function can be differently implemented for any type of inherited discipline.

Returns:: The disciplines shown in the XDSM chain.
Return type:: list[gemseo.core.discipline.MDODiscipline]

get_expected_dataflow()¶

Return the expected data exchange sequence.

This method is used for the XDSM representation.

The default expected data exchange sequence is an empty list.