gemseo / algos

database module

A database of function calls and design variables

Classes:

Database([input_hdf_file, name])	Class to store evaluations of functions, such as DOE or optimization histories.
HashableNdarray(wrapped[, tight])	HashableNdarray wrapper for ndarray objects.

class gemseo.algos.database.Database(input_hdf_file=None, name=None)

Bases: object

Class to store evaluations of functions, such as DOE or optimization histories.

Avoid multiple calls of the same functions, useful when simulations are costly.

It is also used to store inputs and retrieve them for optimization graphical post processing and plots generation.

Can be serialized to HDF5 for portability and cold post processing.

The database is based on a two-levels dictionary-like mapping such as {key_level_1: {key_level_2: value_level_2} } with:

• key_level_1: the values of the input design variables that have been used during the evaluations;

• key_level_2: the name of the output functions that have been returned, the name of the gradient (the gradient of a function called func is typically denoted as @func), the iteration numbers which correspond to the evaluation (this key is typically denoted Iter) and any additional information related to the methods which use the database (penalization parameter, trust region radii…);

• value_level_2: depending on the key_level_2, but is typically a float for an output function, an array for a gradient or a list for the iterations (when several iterations are stored for a same point, this means that the point has been replicated during the process at these iterations).

name

The name of the database.

Type

str

Parameters

• input_hdf_file (Optional[str]) –

  The path to a HDF5 file from which the database is created. If None, do not import a database.

  By default it is set to None.

• name (Optional[str]) –

  The name to be given to the database. If None, use the class name.

  By default it is set to None.

Return type

None

Attributes:

GRAD_TAG
ITER_TAG
KEYSSEPARATOR
missing_value_tag

Methods:

add_new_iter_listener(listener_func)	Add a listener to be called when a new iteration is stored to the database.
add_store_listener(listener_func)	Add a listener to be called when an item is stored to the database.
clean_from_iterate(iterate)	Delete the iterates after a given iteration number.
clear()	Clear the database.
clear_listeners()	Clear all the listeners.
contains_dataname(data_name[, skip_grad])	Check if the database has an output function with the required name.
contains_x(x_vect)	Check if the history contains a specific value of input design variables.
export_hdf([file_path, append])	Export the optimization database to a hdf file.
export_to_ggobi([functions, file_path, ...])	Export the database to a xml file for ggobi tool.
filter(data_list_to_keep)	Filter the database so that only the required output functions are kept.
get(x_vect[, default])	Return the value of the required key if the key is in the dictionary.
get_all_data_names([skip_grad, skip_iter])	Return all the names of the output functions contained in the database.
get_complete_history([functions, ...])	Return the complete history of the optimization: design variables, functions and gradients.
get_f_of_x(fname, x_vect[, dist_tol])	Return the output function values of any input values in the database.
get_func_grad_history(funcname[, x_hist])	Return the history of the gradient values of any function.
get_func_history(funcname[, x_hist])	Return the history of the output function values.
get_gradient_name(name)	Return the name of the gradient related to a function.
get_history_array([functions, ...])	Return the history of the optimization process.
get_index_of(x_vect)	Return the index of an input values in the database.
get_last_n_x(n)	Return the last n ordered calls of the input design variables.
get_max_iteration()	Return the maximum number of iterations.
get_value(x_vect)	Return a value in the database.
get_x_by_iter(iteration)	Return the values of the input design variables at a specified iteration.
get_x_history()	Return the history of the input design variables ordered by calls.
import_from_opendace(database_file)	Load the current database from an opendace xml database.
import_hdf([filename])	Import a database from a hdf file.
is_func_grad_history_empty(funcname)	Check if the history of the gradient of any function is empty.
is_new_eval(x_vect)	Whether storing the given values would generate a new iteration.
items()	Database items generator.
keys()	Database keys generator.
notify_newiter_listeners([x_vect])	Notify the listeners that a new iteration is ongoing.
notify_store_listeners()	Notify the listeners that a new entry was stored in the database.
pop(key)	Remove the required key from the database and return the corresponding value.
remove_empty_entries()	Remove items when the key is associated to an empty value.
set_dv_names(n_dv)	Return the default input variables names.
setdefault(key, default)	Set a default database entry.
store(x_vect, values_dict[, add_iter])	Store the output values associated to the input values.
values()	Database values generator.

GRAD_TAG = '@'

ITER_TAG = 'Iter'

KEYSSEPARATOR = '__KEYSSEPARATOR__'

add_new_iter_listener(listener_func)

Add a listener to be called when a new iteration is stored to the database.

Parameters

listener_func (Callable) – The function to be called.

Raises

TypeError – If the argument is not a callable.

Return type

None

add_store_listener(listener_func)

Add a listener to be called when an item is stored to the database.

Parameters

listener_func (Callable) – The function to be called.

Raises

TypeError – If the argument is not a callable

Return type

None

clean_from_iterate(iterate)

Delete the iterates after a given iteration number.

Parameters

iterate (int) – The iteration number.

Return type

None

clear()

Clear the database.

Return type

None

clear_listeners()

Clear all the listeners.

Return type

None

contains_dataname(data_name, skip_grad=False)

Check if the database has an output function with the required name.

Parameters

• data_name (str) – The required name of the output function.

• skip_grad (bool) –

  True if the name of the gradients are skipped during the search, False otherwise.

  By default it is set to False.

Returns

Whether the required output function name is in the database.

Return type

bool

contains_x(x_vect)

Check if the history contains a specific value of input design variables.

Parameters

x_vect (numpy.ndarray) – The input values that is checked.

Returns

True is the input values are in the history, False otherwise.

Return type

bool

export_hdf(file_path='optimization_history.h5', append=False)

Export the optimization database to a hdf file.

Parameters

• file_path (str) –

  The name of the hdf file.

  By default it is set to optimization_history.h5.

• append (bool) –

  If True, append the data to the file; False otherwise.

  By default it is set to False.

Return type

None

export_to_ggobi(functions=None, file_path='opt_hist.xml', design_variables_names=None)

Export the database to a xml file for ggobi tool.

Parameters

• functions (Optional[Iterable[str]]) –

  The names of output functions.

  By default it is set to None.

• file_path (str) –

  The path to the xml file.

  By default it is set to opt_hist.xml.

• design_variables_names (Optional[Union[str, Iterable[str]]]) –

  The names of the input design variables.

  By default it is set to None.

Return type

None

filter(data_list_to_keep)

Filter the database so that only the required output functions are kept.

Parameters

data_list_to_keep (Iterable[str]) – The name of output functions that must be kept.

Return type

None

get(x_vect, default=None)

Return the value of the required key if the key is in the dictionary.

Parameters

• x_vect (numpy.ndarray) – The required key.

• default (Optional[Any]) –

  The values that is returned if the key is not found.

  By default it is set to None.

Returns

The value corresponding to the key or the prescribed default value if the key is not found.

Raises

TypeError – If the type of the required key is neither an array nor a hashable array.

Return type

Any

get_all_data_names(skip_grad=True, skip_iter=False)

Return all the names of the output functions contained in the database.

Parameters

• skip_grad (bool) –

  True if the names of the keys corresponding to the gradients are not returned, False otherwise.

  By default it is set to True.

• skip_iter (bool) –

  True if the names of the keys corresponding to the iteration numbers are not returned, False otherwise.

  By default it is set to False.

Returns

The names of the output functions.

Return type

Set[str]

get_complete_history(functions=None, add_missing_tag=False, missing_tag='NA', all_iterations=False, stacked_data=None)

Return the complete history of the optimization: design variables, functions and gradients.

Parameters

• functions (Optional[Iterable[str]]) –

  The names of output functions.

  By default it is set to None.

• add_missing_tag (bool) –

  If True, the tag specified in missing_tag is added to iterations where data are not available.

  By default it is set to False.

• missing_tag (str) –

  The tag that is written when data are missing.

  By default it is set to NA.

• all_iterations (bool) –

  If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index)

  By default it is set to False.

• stacked_data (Optional[Iterable[str]]) –

  The names of outputs corresponding to data stored as iterable (e.g. iterations, penalization parameters or trust region radii).

  By default it is set to None.

Returns

The history of the output values. The history of the input values.

Return type

Tuple[List[List[Union[float, numpy.ndarray]]], List[numpy.ndarray]]

get_f_of_x(fname, x_vect, dist_tol=0.0)

Return the output function values of any input values in the database.

Parameters

• fname (str) – The name of the required output function.

• x_vect (ndarray) – The required input values.

• dist_tol (float) –

  In the N-dimensional space of input design variables, this value is the threshold of the normalized distance with respect to the required point (prescribed input values) below which any point in the database can be selected.

  By default it is set to 0.0.

Returns

The values of the output function corresponding to the required input values. None if no point matches.

Return type

Optional[float, ndarray, List[int]]

get_func_grad_history(funcname, x_hist=False)

Return the history of the gradient values of any function.

The function can also return the history of the input values.

Parameters

• funcname (str) – The name of the function.

• x_hist (bool) –

  Whether the input values history is also returned.

  By default it is set to False.

Returns

The gradient values history of the function. The input values history if required.

Return type

Union[numpy.ndarray, Tuple[numpy.ndarray, List[numpy.ndarray]]]

get_func_history(funcname, x_hist=False)

Return the history of the output function values.

This function can also return the history of the input values.

Parameters

• funcname (str) – The name of the function.

• x_hist (bool) –

  Whether the input values history is also returned.

  By default it is set to False.

Returns

The function values history. The input values history if required.

Return type

Union[numpy.ndarray, Tuple[numpy.ndarray, List[numpy.ndarray]]]

classmethod get_gradient_name(name)

Return the name of the gradient related to a function.

This name is the concatenation of a GRAD_TAG, e.g. ‘@’, and the name of the function, e.g. ‘f’. With this example, the name of the gradient is ‘@f’.

Parameters

name (str) – The name of a function.

Returns

The name of the gradient based on the name of the function.

Return type

str

get_history_array(functions=None, design_variables_names=None, add_missing_tag=False, missing_tag='NA', add_dv=True, all_iterations=False, stacked_data=None)

Return the history of the optimization process.

Parameters

• functions (Optional[Iterable[str]]) –

  The names of output functions that must be returned.

  By default it is set to None.

• design_variables_names (Optional[Union[str, Iterable[str]]]) –

  The names of the input design variables.

  By default it is set to None.

• add_missing_tag (bool) –

  If True, add the tag specified in missing_tag for data that are not available.

  By default it is set to False.

• missing_tag (str) –

  The tag that is added for data that are not available.

  By default it is set to NA.

• add_dv (bool) –

  If True, the input design variables are returned in the history.

  By default it is set to True.

• all_iterations (bool) –

  If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index).

  By default it is set to False.

• stacked_data (Optional[Iterable[str]]) –

  The names of outputs corresponding to data stored as iterable (e.g. iterations, penalization parameters or trust region radii).

  By default it is set to None.

Returns

The values of the history. The names of the columns corresponding to the values of the history. The names of the output functions.

Return type

Tuple[numpy.ndarray, List[str], Iterable[str]]

get_index_of(x_vect)

Return the index of an input values in the database.

Parameters

x_vect (numpy.ndarray) – The input values.

Returns

The index of the input values in the database.

Raises

KeyError – If the required key is not found.

Return type

int

get_last_n_x(n)

Return the last n ordered calls of the input design variables.

Parameters

n (int) – The number of last returned calls.

Returns

The values of the input design variables for the last n calls.

Raises

ValueError – If the number n is higher than the size of the database.

Return type

List[numpy.ndarray]

get_max_iteration()

Return the maximum number of iterations.

Returns

The max number of iterations.

Return type

int

get_value(x_vect)

Return a value in the database.

Parameters

x_vect (numpy.ndarray) – The key associated to the value.

Returns

The values that correspond to the required key.

Raises

KeyError – If the key does not exist.

Return type

Mapping[str, Union[float, numpy.ndarray, List[int]]]

get_x_by_iter(iteration)

Return the values of the input design variables at a specified iteration.

Parameters

iteration (int) – The required iteration.

Returns

The values of the input design variables.

Raises

ValueError –

• If the database is empty. * If the required iteration is higher than the maximum number of iterations in the database.

Return type

numpy.ndarray

get_x_history()

Return the history of the input design variables ordered by calls.

Returns

This values of input design variables.

Return type

List[numpy.ndarray]

import_from_opendace(database_file)

Load the current database from an opendace xml database.

Parameters

database_file (str) – The path to an opendace database.

Return type

None

import_hdf(filename='optimization_history.h5')

Import a database from a hdf file.

Parameters

filename (str) –

The path to the HDF5 file.

By default it is set to optimization_history.h5.

Return type

None

is_func_grad_history_empty(funcname)

Check if the history of the gradient of any function is empty.

Parameters

funcname (str) – The name of the function.

Returns

True if the history of the gradient is empty, False otherwise.

Return type

bool

is_new_eval(x_vect)

Whether storing the given values would generate a new iteration.

Parameters

x_vect (numpy.ndarray) – The design variables values.

Returns

Whether a new iteration would occur after storing the values.

Return type

bool

items()

Database items generator.

Yields

The next key and value in the database.

Return type

ItemsView[numpy.ndarray, Mapping[str, Union[float, numpy.ndarray, List[int]]]]

keys()

Database keys generator.

Yields

The next key in the database.

Return type

KeysView[numpy.ndarray]

missing_value_tag = 'NA'

notify_newiter_listeners(x_vect=None)

Notify the listeners that a new iteration is ongoing.

Parameters

x_vect (Optional[numpy.ndarray]) –

The values of the design variables. If None, use the values of the last iteration.

By default it is set to None.

Return type

None

notify_store_listeners()

Notify the listeners that a new entry was stored in the database.

Return type

None

pop(key)

Remove the required key from the database and return the corresponding value.

Parameters

key (numpy.ndarray) – The required key.

Returns

The values corresponding to the key.

Raises

KeyError – If the key is not found.

Return type

Mapping[str, Union[float, numpy.ndarray, List[int]]]

remove_empty_entries()

Remove items when the key is associated to an empty value.

Return type

None

static set_dv_names(n_dv)

Return the default input variables names.

Parameters

n_dv (int) – The number of variables.

Returns

The names of the variables.

Return type

List[str]

setdefault(key, default)

Set a default database entry.

Parameters

• key (numpy.ndarray) – The key of the default item.

• default (Mapping[str, Union[float, numpy.ndarray, List[int]]]) – The value of the item.

Returns

The value which is set.

Raises

TypeError –

• If the key is not ndarray type. * If the value is not dictionary type.

Return type

Mapping[str, Union[float, numpy.ndarray, List[int]]]

store(x_vect, values_dict, add_iter=True)

Store the output values associated to the input values.

Parameters

• x_vect (numpy.ndarray) – The input values.

• values_dict (Mapping[str, Union[float, numpy.ndarray, List[int]]]) – The output values corresponding to the input values.

• add_iter (bool) –

  True if iterations are added to the output values, False otherwise.

  By default it is set to True.

Return type

None

values()

Database values generator.

Yields

The next value in the database.

Return type

ValuesView[Mapping[str, Union[float, numpy.ndarray, List[int]]]]

class gemseo.algos.database.HashableNdarray(wrapped, tight=False)

Bases: object

HashableNdarray wrapper for ndarray objects.

Instances of ndarray are not HashableNdarray, meaning they cannot be added to sets, nor used as keys in dictionaries. This is by design, ndarray objects are mutable, and therefore cannot reliably implement the __hash__() method.

The HashableNdarray class allows a way around this limitation. It implements the required methods for HashableNdarray objects in terms of an encapsulated ndarray object. This can be either a copied instance (which is safer) or the original object (which requires the user to be careful enough not to modify it).

Parameters

• wrapped (ndarray) – The array that must be wrapped.

• tight (bool) –

  If True, the wrapped array is copied.

  By default it is set to False.

Return type

None

Methods:

unwrap()

Return the encapsulated ndarray.

unwrap()

Return the encapsulated ndarray.

Returns

The encapsulated ndarray, or a copy if the wrapper is tight.

Return type

numpy.ndarray