database module¶
A database of function calls and design variables.
- class gemseo.algos.database.Database(input_hdf_file=None, name=None)[source]¶
Bases:
object
Class to store evaluations of functions, such as DOE or optimization histories.
Avoid multiple calls of the same functions, useful when simulations are costly.
It is also used to store inputs and retrieve them for optimization graphical post-processing and plots generation.
Can be serialized to HDF5 for portability and cold post-processing.
The database is based on a two-levels dictionary-like mapping such as
{key_level_1: {key_level_2: value_level_2}}
with:key_level_1
: the values of the input design variables that have been used during the evaluations;key_level_2
: the name of the output functions that have been returned, the name of the gradient (the gradient of a function calledfunc
is typically denoted as@func
), the iteration numbers which correspond to the evaluation (this key is typically denotedIter
) and any additional information related to the methods which use the database (penalization parameter, trust region radii…);value_level_2
: depending on thekey_level_2
, but is typically a float for an output function, an array for a gradient or a list for the iterations (when several iterations are stored for a same point, this means that the point has been replicated during the process at these iterations).
- Parameters
- Return type
None
- add_new_iter_listener(listener_func)[source]¶
Add a listener to be called when a new iteration is stored to the database.
- Parameters
listener_func (Callable) – The function to be called, it must have one argument that is the current x_vector.
- Raises
TypeError – If the argument is not a callable.
- Return type
None
- add_store_listener(listener_func)[source]¶
Add a listener to be called when an item is stored to the database.
- Parameters
listener_func (Callable) – The function to be called.
- Raises
TypeError – If the argument is not a callable
- Return type
None
- clean_from_iterate(iterate)[source]¶
Delete the iterates after a given iteration number.
- Parameters
iterate (int) – The iteration number.
- Return type
None
- clear(reset_iteration_counter=False)[source]¶
Clear the database.
- Parameters
reset_iteration_counter –
Whether to reset the iteration counter.
By default it is set to False.
- Return type
None
- contains_dataname(data_name, skip_grad=False)[source]¶
Check if the database has an output function with the required name.
- contains_x(x_vect)[source]¶
Check if the history contains a specific value of input design variables.
- Parameters
x_vect (numpy.ndarray) – The input values that is checked.
- Returns
True is the input values are in the history, False otherwise.
- Return type
- export_hdf(file_path='optimization_history.h5', append=False)[source]¶
Export the optimization database to a hdf file.
- export_to_ggobi(functions=None, file_path='opt_hist.xml', design_variables_names=None)[source]¶
Export the database to a xml file for ggobi tool.
- Parameters
functions (Iterable[str] | None) –
The names of output functions.
By default it is set to None.
file_path (str) –
The path to the xml file.
By default it is set to opt_hist.xml.
design_variables_names (str | Iterable[str] | None) –
The names of the input design variables.
By default it is set to None.
- Return type
None
- filter(names)[source]¶
Filter the database so that only the required output functions are kept.
- Parameters
names (Iterable[str]) – The names of output functions that must be kept.
- Return type
None
- get(x_vect, default=None)[source]¶
Return the value of the required key if the key is in the dictionary.
- Parameters
x_vect (numpy.ndarray) – The required key.
default (Optional[Any]) –
The values that is returned if the key is not found.
By default it is set to None.
- Returns
The value corresponding to the key or the prescribed default value if the key is not found.
- Raises
TypeError – If the type of the required key is neither an array nor a hashable array.
- Return type
Any
- get_all_data_names(skip_grad=True, skip_iter=False)[source]¶
Return all the names of the output functions contained in the database.
- Parameters
- Returns
The names of the output functions.
- Return type
- get_complete_history(functions=None, add_missing_tag=False, missing_tag='NA', all_iterations=False, stacked_data=None)[source]¶
Return the complete history of the optimization: design variables, functions and gradients.
- Parameters
functions (Iterable[str] | None) –
The names of output functions.
By default it is set to None.
add_missing_tag (bool) –
If True, the tag specified in
missing_tag
is added to iterations where data are not available.By default it is set to False.
The tag that is written when data are missing.
By default it is set to NA.
all_iterations (bool) –
If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index)
By default it is set to False.
stacked_data (Iterable[str] | None) –
The names of outputs corresponding to data stored as
iterable
(e.g. iterations, penalization parameters or trust region radii).By default it is set to None.
- Returns
The history of the output values. The history of the input values.
- Return type
- get_f_of_x(fname, x_vect, dist_tol=0.0)[source]¶
Return the output function values of any input values in the database.
- Parameters
fname (str) – The name of the required output function.
x_vect (ndarray) – The required input values.
dist_tol (float) –
In the N-dimensional space of input design variables, this value is the threshold of the normalized distance with respect to the required point (prescribed input values) below which any point in the database can be selected.
By default it is set to 0.0.
- Returns
The values of the output function corresponding to the required input values.
None
if no point matches.- Return type
- get_func_grad_history(funcname, x_hist=False)[source]¶
Return the history of the gradient values of any function.
The function can also return the history of the input values.
- get_func_history(funcname, x_hist=False)[source]¶
Return the history of the output function values.
This function can also return the history of the input values.
- classmethod get_gradient_name(name)[source]¶
Return the name of the gradient related to a function.
This name is the concatenation of a GRAD_TAG, e.g. ‘@’, and the name of the function, e.g. ‘f’. With this example, the name of the gradient is ‘@f’.
- get_hashed_key(x_vect, copy=False)[source]¶
Convert an array to a hashable array.
This array basically represent a key of the first level of the database.
- Parameters
x_vect (ndarray | HashableNdarray) – An array.
copy (bool) –
Whether to copy the original array.
By default it is set to False.
- Returns
The input array converted to a hashable array.
- Raises
TypeError – If the input is not an array or HashableNdarray.
- Return type
- get_history_array(functions=None, design_variables_names=None, add_missing_tag=False, missing_tag='NA', add_dv=True, all_iterations=False, stacked_data=None)[source]¶
Return the history of the optimization process.
- Parameters
functions (Iterable[str] | None) –
The names of output functions that must be returned.
By default it is set to None.
design_variables_names (str | Iterable[str] | None) –
The names of the input design variables.
By default it is set to None.
add_missing_tag (bool) –
If True, add the tag specified in
missing_tag
for data that are not available.By default it is set to False.
The tag that is added for data that are not available.
By default it is set to NA.
add_dv (bool) –
If True, the input design variables are returned in the history.
By default it is set to True.
all_iterations (bool) –
If True, the points which are called at several iterations will be duplicated in the history (each duplicate corresponding to a different calling index); otherwise each point will appear only once (with the latest calling index).
By default it is set to False.
stacked_data (Iterable[str] | None) –
The names of outputs corresponding to data stored as
iterable
(e.g. iterations, penalization parameters or trust region radii).By default it is set to None.
- Returns
The values of the history. The names of the columns corresponding to the values of the history. The names of the output functions.
- Return type
- get_index_of(x_vect)[source]¶
Return the index of an input values in the database.
- Parameters
x_vect (numpy.ndarray) – The input values.
- Returns
The index of the input values in the database.
- Raises
KeyError – If the required key is not found.
- Return type
- get_last_n_x(n)[source]¶
Return the last n ordered calls of the input design variables.
- Parameters
n (int) – The number of last returned calls.
- Returns
The values of the input design variables for the last n calls.
- Raises
ValueError – If the number n is higher than the size of the database.
- Return type
- get_max_iteration()[source]¶
Return the maximum number of iterations.
- Returns
The maximum number of iterations.
- Return type
- get_value(x_vect)[source]¶
Return a value in the database.
- Parameters
x_vect (numpy.ndarray) – The key associated to the value.
- Returns
The values that correspond to the required key.
- Raises
KeyError – If the key does not exist.
- Return type
Mapping[str, Union[float, numpy.ndarray, List[int]]]
- get_x_by_iter(iteration)[source]¶
Return the values of the input design variables at a specified iteration.
- Parameters
iteration (int) – The required iteration.
- Returns
The values of the input design variables.
- Raises
If the database is empty. * If the required iteration is higher than the maximum number of iterations in the database.
- Return type
- get_x_history()[source]¶
Return the history of the input design variables ordered by calls.
- Returns
This values of input design variables.
- Return type
- import_from_opendace(database_file)[source]¶
Load the current database from an opendace xml database.
- Parameters
database_file (str) – The path to an opendace database.
- Return type
None
- import_hdf(filename='optimization_history.h5')[source]¶
Import a database from a hdf file.
- Parameters
filename (str) –
The path to the HDF5 file.
By default it is set to optimization_history.h5.
- Return type
None
- is_func_grad_history_empty(funcname)[source]¶
Check if the history of the gradient of any function is empty.
- is_new_eval(x_vect)[source]¶
Whether storing the given values would generate a new iteration.
- Parameters
x_vect (numpy.ndarray) – The design variables values.
- Returns
Whether a new iteration would occur after storing the values.
- Return type
- items()[source]¶
Database items generator.
- Yields
The next key and value in the database.
- Return type
ItemsView[numpy.ndarray, Mapping[str, Union[float, numpy.ndarray, List[int]]]]
- keys()[source]¶
Database keys generator.
- Yields
The next key in the database.
- Return type
KeysView[numpy.ndarray]
- notify_newiter_listeners(x_vect=None)[source]¶
Notify the listeners that a new iteration is ongoing.
- Parameters
x_vect (ndarray | None) –
The values of the design variables. If None, use the values of the last iteration.
By default it is set to None.
- Return type
None
- notify_store_listeners()[source]¶
Notify the listeners that a new entry was stored in the database.
- Return type
None
- pop(key)[source]¶
Remove the required key from the database and return the corresponding value.
- Parameters
key (numpy.ndarray) – The required key.
- Returns
The values corresponding to the key.
- Raises
KeyError – If the key is not found.
- Return type
Mapping[str, Union[float, numpy.ndarray, List[int]]]
- remove_empty_entries()[source]¶
Remove items when the key is associated to an empty value.
- Return type
None
- setdefault(key, default)[source]¶
Set a default database entry.
- Parameters
key (numpy.ndarray) – The key of the default item.
default (Mapping[str, Union[float, numpy.ndarray, List[int]]]) – The value of the item.
- Returns
The value which is set.
- Raises
If the key is not ndarray type. * If the value is not dictionary type.
- Return type
Mapping[str, Union[float, numpy.ndarray, List[int]]]
- store(x_vect, values_dict, add_iter=True)[source]¶
Store the output values associated to the input values.
- Parameters
x_vect (numpy.ndarray) – The input values.
values_dict (Mapping[str, Union[float, numpy.ndarray, List[int]]]) – The output values corresponding to the input values.
add_iter (bool) –
True if iterations are added to the output values, False otherwise.
By default it is set to True.
- Return type
None
- values()[source]¶
Database values generator.
- Yields
The next value in the database.
- Return type
ValuesView[Mapping[str, Union[float, numpy.ndarray, List[int]]]]
- GRAD_TAG = '@'¶
- ITER_TAG = 'Iter'¶
- KEYSSEPARATOR = '__KEYSSEPARATOR__'¶
- property last_item: Mapping[str, Union[float, numpy.ndarray, List[int]]]¶
The last item stored in the database.
- missing_value_tag = 'NA'¶
- class gemseo.algos.database.HashableNdarray(wrapped, tight=False)[source]¶
Bases:
object
HashableNdarray wrapper for ndarray objects.
Instances of ndarray are not HashableNdarray, meaning they cannot be added to sets, nor used as keys in dictionaries. This is by design, ndarray objects are mutable, and therefore cannot reliably implement the __hash__() method.
The HashableNdarray class allows a way around this limitation. It implements the required methods for HashableNdarray objects in terms of an encapsulated ndarray object. This can be either a copied instance (which is safer) or the original object (which requires the user to be careful enough not to modify it).
- Parameters
wrapped (numpy.ndarray) – The array that must be wrapped.
tight (bool) –
If True, the wrapped array is copied.
By default it is set to False.
- Return type
None