cache module¶
Caching module to avoid multiple evaluations of a discipline.
- class gemseo.core.cache.AbstractCache(tolerance=0.0, name=None)[source]¶
Bases:
Mapping
[DataMapping
,CacheEntry
]An abstract base class for caches with a dictionary-like interface.
Caches are mainly used to store the
MDODiscipline
evaluations.A cache entry is defined by:
an input data in the form of a dictionary of objects associated with input names, i.e.
{"input_name": object}
,an output data in the form of a dictionary of NumPy arrays associated with output names, i.e.
{"output_name": array}
.an optional Jacobian data, in the form of a nested dictionary of NumPy arrays associating output and input names, i.e.
{"output_name": {"input_name": array}}
.
Examples
The evaluation of the function \(y=f(x)=(x^2, 2x^3\))` and its derivative at \(x=1\) leads to cache the entry defined by:
the input data: \(1.\),
the output data: \((1., 2.)\),
the Jacobian data: \((2., 6.)^T\).
>>> input_data = {"x": array([1.0])} >>> output_data = {"y": array([1.0, 2.0])} >>> jacobian_data = {"y": {"x": array([[2.0], [6.0]])}}
For this
input_data
, one can cache the output data:>>> cache.cache_outputs(input_data, output_data)
as well as the Jacobian data:
>>> cache.cache_jacobian(input_data, jacobian_data)
Caches have a
abc.Mapping
interface making them easy to set (cache[input_data] = (output_data, jacobian_data)
), access (cache_entry = cache[input_data]
) and update (cache.update(other_cache)
).Notes
cache_entry
is aCacheEntry
with the ordered fields input, output and jacobian accessible either by index, e.g.input_data = cache_entry[0]
, or by name, e.g.input_data = cache_entry.inputs
.Notes
If an output name is also an input name, the output name is suffixed with
[out]
.One can also get the number of cache entries with
size = len(cache)
and iterate over the cache, e.g.for input_data, output_data, _ in cache
for index, (input_data, _, jacobian_data) in enumerate(cache)
or[entry.outputs for entry in cache]
.See also
SimpleCache
to store the last discipline evaluation.MemoryFullCache
to store all the discipline evaluations in memory.HDF5Cache
to store all the discipline evaluations in a HDF5 file.- Parameters:
tolerance (float) –
The tolerance below which two input arrays are considered equal:
norm(new_array-cached_array)/(1+norm(cached_array)) <= tolerance
. If this is the case for all the input names, then the cached output data shall be returned rather than re-evaluating the discipline. This tolerance could be useful to optimize CPU time. It could be something like2 * numpy.finfo(float).eps
.By default it is set to 0.0.
name (str) – A name for the cache. If
None
, use the class name.
- class Group(value)[source]¶
Bases:
StrEnum
A data group.
- INPUTS = 'inputs'¶
The label for the input variables.
- JACOBIAN = 'jacobian'¶
The label for the Jacobian.
- OUTPUTS = 'outputs'¶
The label for the output variables.
- abstract cache_jacobian(input_data, jacobian_data)[source]¶
Cache the input and Jacobian data.
- Parameters:
input_data (DataMapping) – The data containing the input data to cache.
jacobian_data (JacobianData) – The Jacobian data to cache.
- Return type:
None
- abstract cache_outputs(input_data, output_data)[source]¶
Cache input and output data.
- Parameters:
input_data (DataMapping) – The data containing the input data to cache.
output_data (DataMapping) – The data containing the output data to cache.
- Return type:
None
- get(k[, d]) D[k] if k in D, else d. d defaults to None. ¶
- abstract get_all_entries()[source]¶
Return an iterator over all the entries.
The tolerance is ignored.
- Yields:
The entries.
- Return type:
- items() a set-like object providing a view on D's items ¶
- keys() a set-like object providing a view on D's keys ¶
- to_dataset(name='', categorize=True, input_names=(), output_names=())[source]¶
Build a
Dataset
from the cache.- Parameters:
name (str) –
A name for the dataset. If empty, use the name of the cache.
By default it is set to “”.
categorize (bool) –
Whether to distinguish between the different groups of variables. Otherwise, group all the variables in
Dataset.PARAMETER_GROUP`
.By default it is set to True.
The names of the inputs to be exported. If empty, use all the inputs.
By default it is set to ().
output_names (Iterable[str]) –
The names of the outputs to be exported. If empty, use all the outputs. If an output name is also an input name, the output name is suffixed with
[out]
.By default it is set to ().
- Returns:
A dataset version of the cache.
- Return type:
- values() an object providing a view on D's values ¶
- abstract property last_entry: CacheEntry¶
The last cache entry.
- class gemseo.core.cache.AbstractFullCache(tolerance=0.0, name=None)[source]¶
Bases:
AbstractCache
Abstract cache to store all the data, either in memory or on the disk.
See also
MemoryFullCache
: store all the data in memory.HDF5Cache
: store all the data in an HDF5 file.- Parameters:
tolerance (float) –
The tolerance below which two input arrays are considered equal:
norm(new_array-cached_array)/(1+norm(cached_array)) <= tolerance
. If this is the case for all the input names, then the cached output data shall be returned rather than re-evaluating the discipline. This tolerance could be useful to optimize CPU time. It could be something like2 * numpy.finfo(float).eps
.By default it is set to 0.0.
name (str | None) – A name for the cache. If
None
, use the class name.
- class Group(value)¶
Bases:
StrEnum
A data group.
- INPUTS = 'inputs'¶
The label for the input variables.
- JACOBIAN = 'jacobian'¶
The label for the Jacobian.
- OUTPUTS = 'outputs'¶
The label for the output variables.
- cache_jacobian(input_data, jacobian_data)[source]¶
Cache the input and Jacobian data.
- Parameters:
input_data (DataMapping) – The data containing the input data to cache.
jacobian_data (JacobianData) – The Jacobian data to cache.
- Return type:
None
- cache_outputs(input_data, output_data)[source]¶
Cache input and output data.
- Parameters:
input_data (DataMapping) – The data containing the input data to cache.
output_data (DataMapping) – The data containing the output data to cache.
- Return type:
None
- get(k[, d]) D[k] if k in D, else d. d defaults to None. ¶
- get_all_entries()[source]¶
Return an iterator over all the entries.
The tolerance is ignored.
- Yields:
The entries.
- Return type:
- items() a set-like object providing a view on D's items ¶
- keys() a set-like object providing a view on D's keys ¶
- to_dataset(name='', categorize=True, input_names=(), output_names=())¶
Build a
Dataset
from the cache.- Parameters:
name (str) –
A name for the dataset. If empty, use the name of the cache.
By default it is set to “”.
categorize (bool) –
Whether to distinguish between the different groups of variables. Otherwise, group all the variables in
Dataset.PARAMETER_GROUP`
.By default it is set to True.
The names of the inputs to be exported. If empty, use all the inputs.
By default it is set to ().
output_names (Iterable[str]) –
The names of the outputs to be exported. If empty, use all the outputs. If an output name is also an input name, the output name is suffixed with
[out]
.By default it is set to ().
- Returns:
A dataset version of the cache.
- Return type:
- to_ggobi(file_path, input_names=None, output_names=None)[source]¶
Export the cache to an XML file for ggobi tool.
- update(other_cache)[source]¶
Update from another cache.
- Parameters:
other_cache (AbstractFullCache) – The cache to update the current one.
- Return type:
None
- values() an object providing a view on D's values ¶
- property last_entry: CacheEntry¶
The last cache entry.
- lock: RLockType¶
The lock used for both multithreading and multiprocessing.
Ensure safe multiprocessing and multithreading concurrent access to the cache.
- lock_hashes: RLockType¶
The lock used for both multithreading and multiprocessing.
Ensure safe multiprocessing and multithreading concurrent access to the cache.
- class gemseo.core.cache.CacheEntry(inputs, outputs, jacobian)[source]¶
Bases:
NamedTuple
An entry of a cache.
Create new instance of CacheEntry(inputs, outputs, jacobian)
- Parameters:
inputs (DataMapping) –
outputs (DataMapping) –
jacobian (JacobianData) –
- count(value, /)¶
Return number of occurrences of value.
- index(value, start=0, stop=9223372036854775807, /)¶
Return first index of value.
Raises ValueError if the value is not present.
- inputs: DataMapping¶
The input data.
- jacobian: JacobianData¶
The Jacobian data.
- outputs: DataMapping¶
The output data.
- gemseo.core.cache.DATA_COMPARATOR(dict_of_arrays, other_dict_of_arrays, tolerance=0.0)¶
The comparator of input data structures.
It is used to check whether an input data has been cached in.
- Parameters:
- Return type:
- gemseo.core.cache.hash_data_dict(data)[source]¶
Hash data using xxh3_64 from the xxhash library.
- Parameters:
data (DataMapping) – The data to hash.
- Returns:
The hash value of the data.
- Return type:
Examples
>>> from gemseo.core.cache import hash_data_dict >>> from numpy import array >>> data = {"x": array([1.0, 2.0]), "y": array([3.0])} >>> hash_data_dict(data) 13252388834746642440 >>> hash_data_dict(data, "x") 4006190450215859422
- gemseo.core.cache.to_real(data)[source]¶
Convert a NumPy array to a float NumPy array.
- Parameters:
data (RealOrComplexArray) – The NumPy array to be converted to real.
- Returns:
A float NumPy array.
- Return type:
RealArray