gemseo / core

Show inherited members

cache module

Caching module to avoid multiple evaluations of a discipline.

class gemseo.core.cache.AbstractCache(tolerance=0.0, name=None)

Bases: Mapping[DataMapping, CacheEntry]

An abstract base class for caches with a dictionary-like interface.

Caches are mainly used to store the MDODiscipline evaluations.

A cache entry is defined by:

• an input data in the form of a dictionary of objects associated with input names, i.e. {"input_name": object},

• an output data in the form of a dictionary of NumPy arrays associated with output names, i.e. {"output_name": array}.

• an optional Jacobian data, in the form of a nested dictionary of NumPy arrays associating output and input names, i.e. {"output_name": {"input_name": array}}.

Examples

The evaluation of the function \(y=f(x)=(x^2, 2x^3\))` and its derivative at \(x=1\) leads to cache the entry defined by:

• the input data: \(1.\),

• the output data: \((1., 2.)\),

• the Jacobian data: \((2., 6.)^T\).

>>> input_data = {"x": array([1.0])}
>>> output_data = {"y": array([1.0, 2.0])}
>>> jacobian_data = {"y": {"x": array([[2.0], [6.0]])}}

For this input_data, one can cache the output data:

>>> cache.cache_outputs(input_data, output_data)

as well as the Jacobian data:

>>> cache.cache_jacobian(input_data, jacobian_data)

Caches have a abc.Mapping interface making them easy to set (cache[input_data] = (output_data, jacobian_data)), access (cache_entry = cache[input_data]) and update (cache.update(other_cache)).

Notes

cache_entry is a CacheEntry with the ordered fields input, output and jacobian accessible either by index, e.g. input_data = cache_entry[0], or by name, e.g. input_data = cache_entry.inputs.

Notes

If an output name is also an input name, the output name is suffixed with [out].

One can also get the number of cache entries with size = len(cache) and iterate over the cache, e.g. for input_data, output_data, _ in cache for index, (input_data, _, jacobian_data) in enumerate(cache) or [entry.outputs for entry in cache].

See also

SimpleCache to store the last discipline evaluation. MemoryFullCache to store all the discipline evaluations in memory. HDF5Cache to store all the discipline evaluations in a HDF5 file.

Parameters:

• tolerance (float) –

  The tolerance below which two input arrays are considered equal: norm(new_array-cached_array)/(1+norm(cached_array)) <= tolerance. If this is the case for all the input names, then the cached output data shall be returned rather than re-evaluating the discipline. This tolerance could be useful to optimize CPU time. It could be something like 2 * numpy.finfo(float).eps.

  By default it is set to 0.0.

• name (str) – A name for the cache. If None, use the class name.

class Group(value)

Bases: StrEnum

A data group.

INPUTS = 'inputs'

The label for the input variables.

JACOBIAN = 'jacobian'

The label for the Jacobian.

OUTPUTS = 'outputs'

The label for the output variables.

abstract cache_jacobian(input_data, jacobian_data)

Cache the input and Jacobian data.

Parameters:

• input_data (DataMapping) – The data containing the input data to cache.

• jacobian_data (JacobianData) – The Jacobian data to cache.

Return type:

None

abstract cache_outputs(input_data, output_data)

Cache input and output data.

Parameters:

• input_data (DataMapping) – The data containing the input data to cache.

• output_data (DataMapping) – The data containing the output data to cache.

Return type:

None

clear()

Clear the cache.

Return type:

None

abstract get_all_entries()

Return an iterator over all the entries.

The tolerance is ignored.

Yields:

The entries.

Return type:

Iterator[CacheEntry]

to_dataset(name='', categorize=True, input_names=(), output_names=())

Build a Dataset from the cache.

Parameters:

• name (str) –

  A name for the dataset. If empty, use the name of the cache.

  By default it is set to “”.

• categorize (bool) –

  Whether to distinguish between the different groups of variables. Otherwise, group all the variables in Dataset.PARAMETER_GROUP`.

  By default it is set to True.

• input_names (Iterable[str]) –

  The names of the inputs to be exported. If empty, use all the inputs.

  By default it is set to ().

• output_names (Iterable[str]) –

  The names of the outputs to be exported. If empty, use all the outputs. If an output name is also an input name, the output name is suffixed with [out].

  By default it is set to ().

Returns:

A dataset version of the cache.

Return type:

Dataset

property input_names: list[str]

The names of the inputs of the last entry.

abstract property last_entry: CacheEntry

The last cache entry.

name: str

The name of the cache.

property names_to_sizes: dict[str, int]

The sizes of the variables of the last entry.

For a Numpy array, its size is used. For a container, its length is used. Otherwise, a size of 1 is used.

property output_names: list[str]

The names of the outputs of the last entry.

tolerance: float

The tolerance below which two input arrays are considered equal.

class gemseo.core.cache.AbstractFullCache(tolerance=0.0, name=None)

Bases: AbstractCache

Abstract cache to store all the data, either in memory or on the disk.

See also

MemoryFullCache: store all the data in memory. HDF5Cache: store all the data in an HDF5 file.

Parameters:

• tolerance (float) –

  The tolerance below which two input arrays are considered equal: norm(new_array-cached_array)/(1+norm(cached_array)) <= tolerance. If this is the case for all the input names, then the cached output data shall be returned rather than re-evaluating the discipline. This tolerance could be useful to optimize CPU time. It could be something like 2 * numpy.finfo(float).eps.

  By default it is set to 0.0.

• name (str | None) – A name for the cache. If None, use the class name.

cache_jacobian(input_data, jacobian_data)

Cache the input and Jacobian data.

Parameters:

• input_data (DataMapping) – The data containing the input data to cache.

• jacobian_data (JacobianData) – The Jacobian data to cache.

Return type:

None

cache_outputs(input_data, output_data)

Cache input and output data.

Parameters:

• input_data (DataMapping) – The data containing the input data to cache.

• output_data (DataMapping) – The data containing the output data to cache.

Return type:

None

clear()

Clear the cache.

Return type:

None

get_all_entries()

Return an iterator over all the entries.

The tolerance is ignored.

Yields:

The entries.

Return type:

Iterator[CacheEntry]

to_ggobi(file_path, input_names=None, output_names=None)

Export the cache to an XML file for ggobi tool.

Parameters:

• file_path (str) – The path of the file to export the cache.

• input_names (Iterable[str] | None) – The names of the inputs to export. If None, export all of them.

• output_names (Iterable[str] | None) – The names of the outputs to export. If None, export all of them.

Return type:

None

update(other_cache)

Update from another cache.

Parameters:

other_cache (AbstractFullCache) – The cache to update the current one.

Return type:

None

property last_entry: CacheEntry

The last cache entry.

lock: RLockType

The lock used for both multithreading and multiprocessing.

Ensure safe multiprocessing and multithreading concurrent access to the cache.

lock_hashes: RLockType

The lock used for both multithreading and multiprocessing.

Ensure safe multiprocessing and multithreading concurrent access to the cache.

name: str

The name of the cache.

tolerance: float

The tolerance below which two input arrays are considered equal.

class gemseo.core.cache.CacheEntry(inputs, outputs, jacobian)

Bases: NamedTuple

An entry of a cache.

Create new instance of CacheEntry(inputs, outputs, jacobian)

Parameters:

• inputs (DataMapping) –

• outputs (DataMapping) –

• jacobian (JacobianData) –

inputs: DataMapping

The input data.

jacobian: JacobianData

The Jacobian data.

outputs: DataMapping

The output data.

gemseo.core.cache.DATA_COMPARATOR(dict_of_arrays, other_dict_of_arrays, tolerance=0.0)

The comparator of input data structures.

It is used to check whether an input data has been cached in.

Parameters:

• dict_of_arrays (Mapping[str, ndarray | spmatrix | sparray] | Mapping[str, Mapping[str, ndarray | spmatrix | sparray]]) –

• other_dict_of_arrays (Mapping[str, ndarray | spmatrix | sparray] | Mapping[str, Mapping[str, ndarray | spmatrix | sparray]]) –

• tolerance (float) –

  By default it is set to 0.0.

Return type:

bool

gemseo.core.cache.hash_data_dict(data)

Hash data using xxh3_64 from the xxhash library.

Parameters:

data (DataMapping) – The data to hash.

Returns:

The hash value of the data.

Return type:

int

Examples

>>> from gemseo.core.cache import hash_data_dict
>>> from numpy import array
>>> data = {"x": array([1.0, 2.0]), "y": array([3.0])}
>>> hash_data_dict(data)
13252388834746642440
>>> hash_data_dict(data, "x")
4006190450215859422

gemseo.core.cache.to_real(data)

Convert a NumPy array to a float NumPy array.

Parameters:

data (RealOrComplexArray) – The NumPy array to be converted to real.

Returns:

A float NumPy array.

Return type:

RealArray

Examples using AbstractCache

HDF5 cache

HDF5 cache

Memory full cache

Memory full cache

Simple cache

Simple cache

Convert a cache to a dataset

Convert a cache to a dataset

Examples using AbstractFullCache

HDF5 cache

HDF5 cache

Memory full cache

Memory full cache

Convert a cache to a dataset

Convert a cache to a dataset

Examples using CacheEntry

HDF5 cache

HDF5 cache

Memory full cache

Memory full cache

Simple cache

Simple cache