gemseo / algos

database module

A database of function calls and design variables.

class gemseo.algos.database.Database(input_hdf_file=None, name=None)[source]

Bases: object

Storage of MDOFunction evaluations.

A Database is typically attached to an OptimizationProblem to store the evaluations of its objective, constraints and observables.

Then, a Database can be an optimization history or a collection of samples in the case of a DOE.

It is useful when simulations are costly because it avoids re-evaluating functions at points where they have already been evaluated

See also

NormDBFunction

It can also be post-processed by an OptPostProcessor to visualize its content, e.g. OptHistoryView generating a series of graphs to visualize the histories of the objective, constraints and design variables.

A Database can be serialized to HDF5 for portability and cold post-processing.

Note

Serializing an OptimizationProblem using its method export_hdf also serializes its Database.

The database is based on a two-levels dictionary-like mapping such as {key_level_1: {key_level_2: value_level_2}} with:

  • key_level_1: the values of the input design variables that have been used during the evaluations, if the types of the design variables are different, then they are promoted to the unique type that can represent all them, for instance integer would be promoted to float;

  • key_level_2: the name of the output functions that have been returned, the name of the gradient (the gradient of a function called func is typically denoted as @func), the iteration numbers which correspond to the evaluation (this key is typically denoted Iter) and any additional information related to the methods which use the database (penalization parameter, trust region radii…);

  • value_level_2: depending on the key_level_2, but is typically a float for an output function, an array for a gradient or a list for the iterations (when several iterations are stored for a same point, this means that the point has been replicated during the process at these iterations).

Parameters:
  • input_hdf_file (str | Path | None) – The path of an HDF file containing an initial database if any. It should have been generated with Database.export_hdf().

  • name (str) – The name to be given to the database. If None, use the class name.

add_new_iter_listener(listener_func)[source]

Add a listener to be called when a new iteration is stored to the database.

Parameters:

listener_func (Callable) – The function to be called, it must have one argument that is the current x_vector.

Raises:

TypeError – If the argument is not a callable.

Return type:

None

add_store_listener(listener_func)[source]

Add a listener to be called when an item is stored to the database.

Parameters:

listener_func (Callable) – The function to be called.

Raises:

TypeError – If the argument is not a callable

Return type:

None

clean_from_iterate(iterate)[source]

Delete the iterates after a given iteration number.

Parameters:

iterate (int) – The iteration number.

Return type:

None

clear(reset_iteration_counter=False)[source]

Clear the database.

Parameters:

reset_iteration_counter

Whether to reset the iteration counter.

By default it is set to False.

Return type:

None

clear_listeners()[source]

Clear all the listeners.

Return type:

None

contains_dataname(data_name, skip_grad=False)[source]

Check if the database has an output function with the required name.

Parameters:
  • data_name (str) – The required name of the output function.

  • skip_grad (bool) –

    True if the name of the gradients are skipped during the search, False otherwise.

    By default it is set to False.

Returns:

Whether the required output function name is in the database.

Return type:

bool

contains_x(x_vect)[source]

Check if the history contains a specific value of input design variables.

Parameters:

x_vect (ndarray) – The input values that is checked.

Returns:

True is the input values are in the history, False otherwise.

Return type:

bool

export_hdf(file_path='optimization_history.h5', append=False)[source]

Export the optimization database to an HDF file.

Parameters:
  • file_path (str | Path) –

    The path of the HDF file.

    By default it is set to “optimization_history.h5”.

  • append (bool) –

    Whether to append the data to the file.

    By default it is set to False.

Return type:

None

export_to_ggobi(functions=None, file_path='opt_hist.xml', design_variables_names=None)[source]

Export the database to a xml file for ggobi tool.

Parameters:
  • functions (Iterable[str] | None) – The names of output functions.

  • file_path (str | Path) –

    The path to the xml file.

    By default it is set to “opt_hist.xml”.

  • design_variables_names (str | Iterable[str] | None) – The names of the input design variables.

Return type:

None

filter(names)[source]

Filter the database so that only the required output functions are kept.

Parameters:

names (Iterable[str]) – The names of output functions that must be kept.

Return type:

None

get(x_vect, default=None)[source]

Return the value of the required key if the key is in the dictionary.

Parameters:
  • x_vect (ndarray) – The required key.

  • default (Optional[Any]) – The values that is returned if the key is not found.

Returns:

The value corresponding to the key or the prescribed default value if the key is not found.

Raises:

TypeError – If the type of the required key is neither an array nor a hashable array.

Return type:

Any

get_all_data_names(skip_grad=True, skip_iter=False)[source]

Return all the names of the output functions contained in the database.

Parameters:
  • skip_grad (bool) –

    True if the names of the keys corresponding to the gradients are not returned, False otherwise.

    By default it is set to True.

  • skip_iter (bool) –

    True if the names of the keys corresponding to the iteration numbers are not returned, False otherwise.

    By default it is set to False.

Returns:

The names of the output functions.

Return type:

set[str]

get_complete_history(functions=None, add_missing_tag=False, missing_tag='NA', all_iterations=False, stacked_data=None)[source]

Return the complete history of the optimization.

This includes the design variables, functions and gradients.

Parameters:
  • functions (Iterable[str] | None) – The names of output functions.

  • add_missing_tag (bool) –

    If True, the tag specified in missing_tag is added to iterations where data are not available.

    By default it is set to False.

  • missing_tag (str | float) –

    The tag that is written when data are missing.

    By default it is set to “NA”.

  • all_iterations (bool) –

    If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index)

    By default it is set to False.

  • stacked_data (Iterable[str] | None) – The names of outputs corresponding to data stored as iterable (e.g. iterations, penalization parameters or trust region radii).

Returns:

The history of the output values. The history of the input values.

Return type:

tuple[list[list[float | ndarray]], list[ndarray]]

get_f_of_x(fname, x_vect, dist_tol=0.0)[source]

Return the output function values of any input values in the database.

Parameters:
  • fname (str) – The name of the required output function.

  • x_vect (ndarray) – The required input values.

  • dist_tol (float) –

    In the N-dimensional space of input design variables, this value is the threshold of the normalized distance with respect to the required point (prescribed input values) below which any point in the database can be selected.

    By default it is set to 0.0.

Returns:

The values of the output function corresponding to the required input values. None if no point matches.

Return type:

None | float | ndarray | list[int]

get_func_grad_history(funcname, x_hist=False)[source]

Return the history of the gradient values of any function.

The function can also return the history of the input values.

Parameters:
  • funcname (str) – The name of the function.

  • x_hist (bool) –

    Whether the input values history is also returned.

    By default it is set to False.

Returns:

The gradient values history of the function. The input values history if required.

Return type:

ndarray | tuple[ndarray, list[ndarray]]

get_func_history(funcname, x_hist=False)[source]

Return the history of the output function values.

This function can also return the history of the input values.

Parameters:
  • funcname (str) – The name of the function.

  • x_hist (bool) –

    Whether the input values history is also returned.

    By default it is set to False.

Returns:

The function values history. The input values history if required.

Return type:

ndarray | tuple[ndarray, list[ndarray]]

classmethod get_gradient_name(name)[source]

Return the name of the gradient related to a function.

This name is the concatenation of a GRAD_TAG, e.g. ‘@’, and the name of the function, e.g. ‘f’. With this example, the name of the gradient is ‘@f’.

Parameters:

name (str) – The name of a function.

Returns:

The name of the gradient based on the name of the function.

Return type:

str

static get_hashed_key(x_vect, copy=False)[source]

Convert an array to a hashable array.

This array basically represent a key of the first level of the database.

Parameters:
  • x_vect (ndarray | HashableNdarray) – An array.

  • copy (bool) –

    Whether to copy the original array.

    By default it is set to False.

Returns:

The input array converted to a hashable array.

Raises:

TypeError – If the input is not an array or HashableNdarray.

Return type:

HashableNdarray

get_history_array(functions=None, design_variables_names=None, add_missing_tag=False, missing_tag='NA', add_dv=True, all_iterations=False, stacked_data=None)[source]

Return the history of the optimization process.

Parameters:
  • functions (Iterable[str] | None) – The names of output functions that must be returned.

  • design_variables_names (str | Iterable[str] | None) – The names of the input design variables.

  • add_missing_tag (bool) –

    If True, add the tag specified in missing_tag for data that are not available.

    By default it is set to False.

  • missing_tag (str | float) –

    The tag that is added for data that are not available.

    By default it is set to “NA”.

  • add_dv (bool) –

    If True, the input design variables are returned in the history.

    By default it is set to True.

  • all_iterations (bool) –

    If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index).

    By default it is set to False.

  • stacked_data (Iterable[str] | None) – The names of outputs corresponding to data stored as iterable (e.g. iterations, penalization parameters or trust region radii).

Returns:

The values of the history. The names of the columns corresponding to the values of the history. The names of the output functions.

Return type:

tuple[ndarray, list[str], Iterable[str]]

get_index_of(x_vect)[source]

Return the index of an input values in the database.

Parameters:

x_vect (ndarray) – The input values.

Returns:

The index of the input values in the database.

Raises:

KeyError – If the required key is not found.

Return type:

int

get_last_n_x(n)[source]

Return the last n ordered calls of the input design variables.

Parameters:

n (int) – The number of last returned calls.

Returns:

The values of the input design variables for the last n calls.

Raises:

ValueError – If the number n is higher than the size of the database.

Return type:

list[numpy.ndarray]

get_max_iteration()[source]

Return the maximum number of iterations.

Returns:

The maximum number of iterations.

Return type:

int

get_value(x_vect)[source]

Return a value in the database.

Parameters:

x_vect (ndarray) – The key associated to the value.

Returns:

The values that correspond to the required key.

Raises:

KeyError – If the key does not exist.

Return type:

Mapping[str, Union[float, ndarray, List[int]]]

get_x_by_iter(iteration)[source]

Return the values of the input design variables at a specified iteration.

Parameters:

iteration (int) – The required iteration.

Returns:

The values of the input design variables.

Raises:

ValueError

  • If the database is empty. * If the required iteration is higher than the maximum number of iterations in the database.

Return type:

ndarray

get_x_history()[source]

Return the history of the input design variables ordered by calls.

Returns:

This values of input design variables.

Return type:

list[numpy.ndarray]

import_from_opendace(database_file)[source]

Load the current database from an opendace xml database.

Parameters:

database_file (str | Path) – The path to an opendace database.

Return type:

None

import_hdf(filename='optimization_history.h5')[source]

Import a database from an HDF file.

Parameters:

filename (str | Path) –

The path of the HDF file.

By default it is set to “optimization_history.h5”.

Return type:

None

is_func_grad_history_empty(funcname)[source]

Check if the history of the gradient of any function is empty.

Parameters:

funcname (str) – The name of the function.

Returns:

True if the history of the gradient is empty, False otherwise.

Return type:

bool

is_new_eval(x_vect)[source]

Whether storing the given values would generate a new iteration.

Parameters:

x_vect (ndarray) – The design variables values.

Returns:

Whether a new iteration would occur after storing the values.

Return type:

bool

items()[source]

Database items generator.

Yields:

The next key and value in the database.

Return type:

ItemsView[ndarray, Mapping[str, Union[float, ndarray, List[int]]]]

keys()[source]

Database keys generator.

Yields:

The next key in the database.

Return type:

KeysView[ndarray]

notify_newiter_listeners(x_vect=None)[source]

Notify the listeners that a new iteration is ongoing.

Parameters:

x_vect (ndarray | None) – The values of the design variables. If None, use the values of the last iteration.

Return type:

None

notify_store_listeners(x_vect=None)[source]

Notify the listeners that a new entry was stored in the database.

Parameters:

x_vect (ndarray | None) – The values of the design variables. If None, use the values of the last iteration.

Return type:

None

pop(key)[source]

Remove the required key from the database and return the corresponding value.

Parameters:

key (ndarray) – The required key.

Returns:

The values corresponding to the key.

Raises:

KeyError – If the key is not found.

Return type:

Mapping[str, Union[float, ndarray, List[int]]]

remove_empty_entries()[source]

Remove items when the key is associated to an empty value.

Return type:

None

static set_dv_names(n_dv)[source]

Return the default input variables names.

Parameters:

n_dv (int) – The number of variables.

Returns:

The names of the variables.

Return type:

list[str]

setdefault(key, default)[source]

Set a default database entry.

Parameters:
Returns:

The value which is set.

Raises:

TypeError

  • If the key is not ndarray type. * If the value is not dictionary type.

Return type:

Mapping[str, Union[float, ndarray, List[int]]]

store(x_vect, values_dict, add_iter=True)[source]

Store the output values associated to the input values.

Parameters:
  • x_vect (ndarray) – The input values.

  • values_dict (Mapping[str, Union[float, ndarray, List[int]]]) – The output values corresponding to the input values.

  • add_iter (bool) –

    True if iterations are added to the output values, False otherwise.

    By default it is set to True.

Return type:

None

values()[source]

Database values generator.

Yields:

The next value in the database.

Return type:

ValuesView[Mapping[str, Union[float, ndarray, List[int]]]]

GRAD_TAG = '@'
ITER_TAG = 'Iter'
KEYSSEPARATOR = '__KEYSSEPARATOR__'
property last_item: Mapping[str, Union[float, ndarray, List[int]]]

The last item stored in the database.

missing_value_tag = 'NA'
name: str

The name of the database.

class gemseo.algos.database.HashableNdarray(wrapped, tight=False)[source]

Bases: object

HashableNdarray wrapper for ndarray objects.

Instances of ndarray are not HashableNdarray, meaning they cannot be added to sets, nor used as keys in dictionaries. This is by design, ndarray objects are mutable, and therefore cannot reliably implement the __hash__() method.

The HashableNdarray class allows a way around this limitation. It implements the required methods for HashableNdarray objects in terms of an encapsulated ndarray object. This can be either a copied instance (which is safer) or the original object (which requires the user to be careful enough not to modify it).

Parameters:
  • wrapped (ndarray) – The array that must be wrapped.

  • tight (bool) –

    If True, the wrapped array is copied.

    By default it is set to False.

unwrap()[source]

Return the encapsulated ndarray.

Returns:

The encapsulated ndarray, or a copy if the wrapper is tight.

Return type:

ndarray

property tight: bool

Whether the wrapped array is a copy of the original one.copied.

Examples using Database

BiLevel-based MDO on the Sobieski SSBJ test case

BiLevel-based MDO on the Sobieski SSBJ test case

BiLevel-based MDO on the Sobieski SSBJ test case