gemseo / core

cache module

Caching module to avoid multiple evaluations of a discipline

Classes:

AbstractCache([tolerance, name])

Abstract class for caches: Defines the common methods for caching inputs, outputs, and jacobians of a MDODiscipline.

AbstractFullCache([tolerance, name])

Abstract cache to store all data, either in memory or on the disk.

Functions:

check_cache_approx(data_dict, cache_dict[, ...])

Checks if the data_dict is approximately equal to the cache dict at self.tolerance (absolute + relative)

check_cache_equal(data_dict, cache_dict)

Check if the data dictionary is equal to the cache data dictionary.

hash_data_dict(data[, names_tokeep])

Hash a data dict using sha1.

to_real(data)

Convert complex to real numpy array.

class gemseo.core.cache.AbstractCache(tolerance=0.0, name=None)[source]

Bases: object

Abstract class for caches: Defines the common methods for caching inputs, outputs, and jacobians of a MDODiscipline.

See also

SimpleCache

store the last evaluation

MemoryFullCache

store all data in memory

HDF5Cache

store all data in an HDF5 file

Initialize cache tolerance. By default, don’t use approximate cache. It is up to the user to choose to optimize CPU time with this or not.

could be something like 2 * finfo(float).eps

Parameters
  • tolerance (float) –

    Tolerance that defines if two input vectors are equal and cached data shall be returned. If 0, no approximation is made. Default: 0.

    By default it is set to 0.0.

  • name (str) –

    Name of the cache.

    By default it is set to None.

Attributes:

INPUTS_GROUP

JACOBIAN_GROUP

OUTPUTS_GROUP

SAMPLE_GROUP

inputs_names

Return the inputs names.

max_length

Get the maximal length of the cache (the maximal number of stored elements).

outputs_names

Return the outputs names.

samples_indices

List of samples indices.

varsizes

Return the variables sizes.

Methods:

cache_jacobian(input_data, input_names, jacobian)

Cache jacobian data to avoid re evaluation.

cache_outputs(input_data, input_names, ...)

Cache data to avoid re evaluation.

clear()

Clear the cache.

get_all_data(**options)

Read all the data in the cache.

get_data(index, **options)

Returns an elementary sample.

get_last_cached_inputs()

Retrieve the last execution inputs.

get_last_cached_outputs()

Retrieve the last execution outputs.

get_length()

Get the length of the cache, ie the number of stored elements.

get_outputs(input_data[, input_names])

Check if the discipline has already been evaluated for the given input data dictionary.

INPUTS_GROUP = 'inputs'
JACOBIAN_GROUP = 'jacobian'
OUTPUTS_GROUP = 'outputs'
SAMPLE_GROUP = 'sample'
cache_jacobian(input_data, input_names, jacobian)[source]

Cache jacobian data to avoid re evaluation.

Parameters
  • input_data (dict) – Input data to cache.

  • input_names (list(str)) – List of input data names.

  • jacobian (dict) – Jacobian to cache.

cache_outputs(input_data, input_names, output_data, output_names=None)[source]

Cache data to avoid re evaluation.

Parameters
  • input_data (dict) – Input data to cache.

  • input_names (list(str)) – List of input data names.

  • output_data (dict) – Output data to cache.

  • output_names (list(str)) –

    List of output data names. If None, use all output names. Default: None.

    By default it is set to None.

clear()[source]

Clear the cache.

get_all_data(**options)[source]

Read all the data in the cache.

Returns

all_data – A dictionary of dictionaries for inputs, outputs and jacobian where keys are data indices.

Return type

dict

get_data(index, **options)[source]

Returns an elementary sample.

Parameters
  • index (int) – sample index.

  • options – getter options

get_last_cached_inputs()[source]

Retrieve the last execution inputs.

Returns

inputs – Last cached inputs.

Return type

dict

get_last_cached_outputs()[source]

Retrieve the last execution outputs.

Returns

outputs – Last cached outputs.

Return type

dict

get_length()[source]

Get the length of the cache, ie the number of stored elements.

Returns

length – Length of the cache.

Return type

int

get_outputs(input_data, input_names=None)[source]

Check if the discipline has already been evaluated for the given input data dictionary. If True, return the associated cache, otherwise return None.

Parameters
  • input_data (dict) – Input data dictionary to test for caching.

  • input_names (list(str)) –

    List of input data names. If None, takes them all

    By default it is set to None.

Returns

  • output_data (dict) – Output data if there is no need to evaluate the discipline. None otherwise.

  • jacobian (dict) – Jacobian if there is no need to evaluate the discipline. None otherwise.

property inputs_names

Return the inputs names.

property max_length

Get the maximal length of the cache (the maximal number of stored elements).

Returns

length – Maximal length of the cache.

Return type

int

property outputs_names

Return the outputs names.

property samples_indices

List of samples indices.

property varsizes

Return the variables sizes.

class gemseo.core.cache.AbstractFullCache(tolerance=0.0, name=None)[source]

Bases: gemseo.core.cache.AbstractCache

Abstract cache to store all data, either in memory or on the disk.

See also

MemoryFullCache

store all data in memory

HDF5Cache

store all data in an HDF5 file

Initialize cache tolerance. By default, don’t use approximate cache. It is up to the user to choose to optimize CPU time with this or not.

could be something like 2 * finfo(float).eps

Parameters
  • tolerance (float) –

    Tolerance that defines if two input vectors are equal and cached data shall be returned. If 0, no approximation is made. Default: 0.

    By default it is set to 0.0.

  • name (str) –

    Name of the cache.

    By default it is set to None.

Attributes:

INPUTS_GROUP

JACOBIAN_GROUP

OUTPUTS_GROUP

SAMPLE_GROUP

inputs_names

Return the inputs names.

max_length

Get the maximal length of the cache (the maximal number of stored elements).

outputs_names

Return the outputs names.

samples_indices

List of samples indices.

varsizes

Return the variables sizes.

Methods:

cache_jacobian(input_data, input_names, jacobian)

Cache jacobian data to avoid re evaluation.

cache_outputs(input_data, input_names, ...)

Cache data to avoid re evaluation.

clear()

Clears the cache.

export_to_dataset([name, by_group, ...])

Set Dataset from a cache.

export_to_ggobi(file_path[, inputs_names, ...])

Export history to xml file format for ggobi tool.

get_all_data([as_iterator])

Return all the data in the cache.

get_data(index, **options)

Gets the data associated to a sample ID.

get_last_cached_inputs()

Retrieve the last execution inputs.

get_last_cached_outputs()

Retrieve the last execution outputs.

get_length()

Get the length of the cache, ie the number of stored elements.

get_outputs(input_data[, input_names])

Check if the discipline has already been evaluated for the given input data dictionary.

merge(other_cache)

Merges an other cache with self.

INPUTS_GROUP = 'inputs'
JACOBIAN_GROUP = 'jacobian'
OUTPUTS_GROUP = 'outputs'
SAMPLE_GROUP = 'sample'
cache_jacobian(input_data, input_names, jacobian)[source]

Cache jacobian data to avoid re evaluation.

Parameters
  • input_data (dict) – Input data to cache.

  • input_names (list(str)) – List of input data names.

  • jacobian (dict) – Jacobian to cache.

cache_outputs(input_data, input_names, output_data, output_names=None)[source]

Cache data to avoid re evaluation.

Parameters
  • input_data (dict) – Input data to cache.

  • input_names (list(str)) – List of input data names.

  • output_data (dict) – Output data to cache.

  • output_names (list(str)) –

    List of output data names. If None, use all output names. Default: None.

    By default it is set to None.

clear()[source]

Clears the cache.

export_to_dataset(name=None, by_group=True, categorize=True, inputs_names=None, outputs_names=None)[source]

Set Dataset from a cache.

Parameters
  • name (str) –

    dataset name.

    By default it is set to None.

  • by_group (bool) –

    if True, store the data by group. Otherwise, store them by variables. Default: True

    By default it is set to True.

  • categorize (bool) –

    distinguish between the different groups of variables. Default: True.

    By default it is set to True.

  • inputs_names (list(str)) –

    list of inputs names. If None, use all inputs. Default: None.

    By default it is set to None.

  • outputs_names (list(str)) –

    list of outputs names. If None, use all outputs. Default: None.

    By default it is set to None.

export_to_ggobi(file_path, inputs_names=None, outputs_names=None)[source]

Export history to xml file format for ggobi tool.

Parameters
  • file_path (str) – Path to export the file.

  • inputs_names (list(str)) –

    List of inputs to include in the export. By default, take all of them.

    By default it is set to None.

  • outputs_names (list(str)) –

    Names of outputs to export. By default, take all of them.

    By default it is set to None.

get_all_data(as_iterator=False)[source]

Return all the data in the cache.

Parameters

as_iterator (bool) –

If True, return an iterator. Otherwise a dictionary. Default: False.

By default it is set to False.

Returns

all_data – A dictionary of dictionaries for inputs, outputs and jacobian where keys are data indices.

Return type

dict

get_data(index, **options)[source]

Gets the data associated to a sample ID.

Parameters
  • index (str) – sample ID.

  • options – options passed to the _read_data() method.

Returns

input data, output data and jacobian.

Return type

dict

get_last_cached_inputs()[source]

Retrieve the last execution inputs.

Returns

inputs – Last cached inputs.

Return type

dict

get_last_cached_outputs()[source]

Retrieve the last execution outputs.

Returns

outputs – Last cached outputs.

Return type

dict

get_length()[source]

Get the length of the cache, ie the number of stored elements.

Returns

length – Length of the cache.

Return type

int

get_outputs(input_data, input_names=None)[source]

Check if the discipline has already been evaluated for the given input data dictionary. If True, return the associated cache, otherwise return None.

Parameters
  • input_data (dict) – Input data dictionary to test for caching.

  • input_names (list(str)) –

    List of input data names.

    By default it is set to None.

Returns

  • output_data (dict) – Output data if there is no need to evaluate the discipline. None otherwise.

  • jacobian (dict) – Jacobian if there is no need to evaluate the discipline. None otherwise.

property inputs_names

Return the inputs names.

property max_length

Get the maximal length of the cache (the maximal number of stored elements).

Returns

length – Maximal length of the cache.

Return type

int

merge(other_cache)[source]

Merges an other cache with self.

Parameters

other_cache (AbstractFullCache) – Cache to merge with the current one.

property outputs_names

Return the outputs names.

property samples_indices

List of samples indices.

property varsizes

Return the variables sizes.

gemseo.core.cache.check_cache_approx(data_dict, cache_dict, cache_tol=0.0)[source]

Checks if the data_dict is approximately equal to the cache dict at self.tolerance (absolute + relative)

Parameters

data_dict – data dict to check

Returns

True if the dict are approximately equal

gemseo.core.cache.check_cache_equal(data_dict, cache_dict)[source]

Check if the data dictionary is equal to the cache data dictionary.

Parameters
  • data_dict (dict) – Data dictionary to check.

  • cache_dict (dict) – Cache data dictionary to check.

Returns

is_equal – True if the dictionaries are equal.

Return type

bool

Examples

>>> from numpy import array
>>> data_1 = {'x': array([1.]), 'y': array([2.])}
>>> data_2 = {'x': array([1.]), 'y': array([3.])}
>>> check_cache_equal(data_1, data_1)
True
>>> check_cache_equal(data_1, data_2)
False
gemseo.core.cache.hash_data_dict(data, names_tokeep=None)[source]
Hash a data dict using sha1.
for group_num, group in node_group.items():

hash_value = int(array(read_hash)[0])

Parameters
  • data (dict) – The data dictionary.

  • names_tokeep (list(str)) –

    Names of the data to keep for hashing. If None, use sorted(data.keys()).

    By default it is set to None.

Returns

hash – Hash value of the data dictionary.

Return type

int

Examples

>>> from gemseo.core.cache import hash_data_dict
>>> from numpy import array
>>> data = {'x':array([1.,2.]),'y':array([3.])}
>>> hash_data_dict(data)
1871784392126344814771968055738742895695521374568L
>>> hash_data_dict(data,'x')
756520441774735697349528776513537427923146459919L
gemseo.core.cache.to_real(data)[source]

Convert complex to real numpy array.