.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/dataset/plot_dataset_from_array.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_dataset_plot_dataset_from_array.py: Dataset from a numpy array ========================== In this example, we will see how to build a :class:`.Dataset` from an numpy array. For that, we need to import this :class:`.Dataset` class: .. GENERATED FROM PYTHON SOURCE LINES 30-43 .. code-block:: default from __future__ import absolute_import, division, print_function, unicode_literals from future import standard_library from numpy import concatenate from numpy.random import rand from gemseo.api import configure_logger from gemseo.core.dataset import Dataset configure_logger() standard_library.install_aliases() .. GENERATED FROM PYTHON SOURCE LINES 44-51 Synthetic data -------------- Let us consider three parameters: - x_1 with dimension 1, - x_2 with dimension 2, - y_1 with dimension 3. .. GENERATED FROM PYTHON SOURCE LINES 51-57 .. code-block:: default dim_x1 = 1 dim_x2 = 2 dim_y1 = 3 sizes = {"x_1": dim_x1, "x_2": dim_x2, "y_1": dim_y1} groups = {"x_1": "inputs", "x_2": "inputs", "y_1": "outputs"} .. GENERATED FROM PYTHON SOURCE LINES 58-64 We generate 5 random samples of the inputs where: - x_1 is stored in the first column, - x_2 is stored in the 2nd and 3rd columns and 5 random samples of the outputs. .. GENERATED FROM PYTHON SOURCE LINES 64-73 .. code-block:: default n_samples = 5 inputs = rand(n_samples, dim_x1 + dim_x2) inputs_names = ["x_1", "x_2"] outputs = rand(n_samples, dim_y1) outputs_names = ["y_1"] data = concatenate((inputs, outputs), 1) data_names = inputs_names + outputs_names .. GENERATED FROM PYTHON SOURCE LINES 74-79 Create a dataset ---------------- using default names ~~~~~~~~~~~~~~~~~~~ We build a :class:`.Dataset` and initialize from the whole data: .. GENERATED FROM PYTHON SOURCE LINES 79-84 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data) print(dataset) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset | Number of samples: 5 | Number of variables: 6 | Variables names and sizes by group: | - parameters: x_0 (1), x_1 (1), x_2 (1), x_3 (1), x_4 (1), x_5 (1) | Number of dimensions (total = 6) by group: | - parameters: 6 .. GENERATED FROM PYTHON SOURCE LINES 85-89 using particular names ~~~~~~~~~~~~~~~~~~~~~~ We can also use the names of the variables, rather than the default ones fixed by the class: .. GENERATED FROM PYTHON SOURCE LINES 89-94 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data, data_names, sizes) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset | Number of samples: 5 | Number of variables: 3 | Variables names and sizes by group: | - parameters: x_1 (1), x_2 (2), y_1 (3) | Number of dimensions (total = 6) by group: | - parameters: 6 {'parameters': array([[0.23124243, 0.07393255, 0.35388856, 0.33223059, 0.84173333, 0.87194202], [0.20172961, 0.52456865, 0.80053636, 0.62157433, 0.34984932, 0.11470781], [0.25730461, 0.81341756, 0.39107675, 0.22729595, 0.00813847, 0.1426607 ], [0.43760277, 0.41877463, 0.58286326, 0.28062242, 0.36643361, 0.43297368], [0.79629287, 0.27208874, 0.26660728, 0.86119936, 0.64135187, 0.19488591]])} .. GENERATED FROM PYTHON SOURCE LINES 95-101 .. warning:: The number of variables names must be equal to the number of columns of the data array. Otherwise, the user has to specify the sizes of the different variables by means of a dictionary and be careful that the total size is equal to this number of columns. .. GENERATED FROM PYTHON SOURCE LINES 103-106 using particular groups ~~~~~~~~~~~~~~~~~~~~~~~ We can also use the notions of groups of variables: .. GENERATED FROM PYTHON SOURCE LINES 106-111 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data, data_names, sizes, groups) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset | Number of samples: 5 | Number of variables: 3 | Variables names and sizes by group: | - inputs: x_1 (1), x_2 (2) | - outputs: y_1 (3) | Number of dimensions (total = 6) by group: | - inputs: 3 | - outputs: 3 {'inputs': array([[0.23124243, 0.07393255, 0.35388856], [0.20172961, 0.52456865, 0.80053636], [0.25730461, 0.81341756, 0.39107675], [0.43760277, 0.41877463, 0.58286326], [0.79629287, 0.27208874, 0.26660728]]), 'outputs': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])} .. GENERATED FROM PYTHON SOURCE LINES 112-117 .. note:: The groups are specified by means of a dictionary where indices are the variables names and values are the groups. If a variable is missing, the default group 'parameters' is considered. .. GENERATED FROM PYTHON SOURCE LINES 119-122 storing by names ~~~~~~~~~~~~~~~~ We can also store the data by variables names rather than by groups. .. GENERATED FROM PYTHON SOURCE LINES 122-127 .. code-block:: default dataset = Dataset(name="random_dataset", by_group=False) dataset.set_from_array(data, data_names, sizes, groups) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset | Number of samples: 5 | Number of variables: 3 | Variables names and sizes by group: | - inputs: x_1 (1), x_2 (2) | - outputs: y_1 (3) | Number of dimensions (total = 6) by group: | - inputs: 3 | - outputs: 3 {'x_1': array([[0.23124243], [0.20172961], [0.25730461], [0.43760277], [0.79629287]]), 'x_2': array([[0.07393255, 0.35388856], [0.52456865, 0.80053636], [0.81341756, 0.39107675], [0.41877463, 0.58286326], [0.27208874, 0.26660728]]), 'y_1': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])} .. GENERATED FROM PYTHON SOURCE LINES 128-139 .. note:: The choice to be made between a storage by group and a storage by variables names aims to limit the number of memory copies of numpy arrays. It mainly depends on how the dataset is used and for what purposes. For example, if we want to build a machine learning algorithm from both input and output data, we only have to access the data by group and in this case, storing the data by group is recommended. Conversely, if we want to use the dataset for post-processing purposes, by accessing the variables of the dataset from their names, the storage by variables names is preferable. .. GENERATED FROM PYTHON SOURCE LINES 141-146 Access properties ----------------- Variables names ~~~~~~~~~~~~~~~ We can access the variables names: .. GENERATED FROM PYTHON SOURCE LINES 146-148 .. code-block:: default print(dataset.variables) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ['x_1', 'x_2', 'y_1'] .. GENERATED FROM PYTHON SOURCE LINES 149-152 Variables sizes ~~~~~~~~~~~~~~~ We can access the variables sizes: .. GENERATED FROM PYTHON SOURCE LINES 152-154 .. code-block:: default print(dataset.sizes) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': 1, 'x_2': 2, 'y_1': 3} .. GENERATED FROM PYTHON SOURCE LINES 155-158 Variables groups ~~~~~~~~~~~~~~~~ We can access the variables groups: .. GENERATED FROM PYTHON SOURCE LINES 158-160 .. code-block:: default print(dataset.groups) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ['inputs', 'outputs'] .. GENERATED FROM PYTHON SOURCE LINES 161-166 Access data ----------- Access by group ~~~~~~~~~~~~~~~ We can get the data by group, either as an array (default option): .. GENERATED FROM PYTHON SOURCE LINES 166-167 .. code-block:: default print(dataset.get_data_by_group("inputs")) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0.23124243 0.07393255 0.35388856] [0.20172961 0.52456865 0.80053636] [0.25730461 0.81341756 0.39107675] [0.43760277 0.41877463 0.58286326] [0.79629287 0.27208874 0.26660728]] .. GENERATED FROM PYTHON SOURCE LINES 168-169 or as a dictionary indexed by the variables names: .. GENERATED FROM PYTHON SOURCE LINES 169-171 .. code-block:: default print(dataset.get_data_by_group("inputs", True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.23124243], [0.20172961], [0.25730461], [0.43760277], [0.79629287]]), 'x_2': array([[0.07393255, 0.35388856], [0.52456865, 0.80053636], [0.81341756, 0.39107675], [0.41877463, 0.58286326], [0.27208874, 0.26660728]])} .. GENERATED FROM PYTHON SOURCE LINES 172-176 Access by variable name ~~~~~~~~~~~~~~~~~~~~~~~ We can get the data by variables names, either as a dictionary indexed by the variables names (default option): .. GENERATED FROM PYTHON SOURCE LINES 176-177 .. code-block:: default print(dataset.get_data_by_names(["x_1", "y_1"])) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.23124243], [0.20172961], [0.25730461], [0.43760277], [0.79629287]]), 'y_1': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])} .. GENERATED FROM PYTHON SOURCE LINES 178-179 or as an array: .. GENERATED FROM PYTHON SOURCE LINES 179-181 .. code-block:: default print(dataset.get_data_by_names(["x_1", "y_1"], False)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0.23124243 0.33223059 0.84173333 0.87194202] [0.20172961 0.62157433 0.34984932 0.11470781] [0.25730461 0.22729595 0.00813847 0.1426607 ] [0.43760277 0.28062242 0.36643361 0.43297368] [0.79629287 0.86119936 0.64135187 0.19488591]] .. GENERATED FROM PYTHON SOURCE LINES 182-185 Access all data ~~~~~~~~~~~~~~~ We can get all the data, either as a large array: .. GENERATED FROM PYTHON SOURCE LINES 185-186 .. code-block:: default print(dataset.get_all_data()) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ({'inputs': array([[0.23124243, 0.07393255, 0.35388856], [0.20172961, 0.52456865, 0.80053636], [0.25730461, 0.81341756, 0.39107675], [0.43760277, 0.41877463, 0.58286326], [0.79629287, 0.27208874, 0.26660728]]), 'outputs': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])}, {'inputs': ['x_1', 'x_2'], 'outputs': ['y_1']}, {'x_1': 1, 'x_2': 2, 'y_1': 3}) .. GENERATED FROM PYTHON SOURCE LINES 187-188 or as a dictionary indexed by variables names: .. GENERATED FROM PYTHON SOURCE LINES 188-189 .. code-block:: default print(dataset.get_all_data(as_dict=True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'inputs': {'x_1': array([[0.23124243], [0.20172961], [0.25730461], [0.43760277], [0.79629287]]), 'x_2': array([[0.07393255, 0.35388856], [0.52456865, 0.80053636], [0.81341756, 0.39107675], [0.41877463, 0.58286326], [0.27208874, 0.26660728]])}, 'outputs': {'y_1': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])}} .. GENERATED FROM PYTHON SOURCE LINES 190-192 We can get these data sorted by category, either with a large array for each category: .. GENERATED FROM PYTHON SOURCE LINES 192-193 .. code-block:: default print(dataset.get_all_data(by_group=False)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none (array([[0.23124243, 0.07393255, 0.35388856, 0.33223059, 0.84173333, 0.87194202], [0.20172961, 0.52456865, 0.80053636, 0.62157433, 0.34984932, 0.11470781], [0.25730461, 0.81341756, 0.39107675, 0.22729595, 0.00813847, 0.1426607 ], [0.43760277, 0.41877463, 0.58286326, 0.28062242, 0.36643361, 0.43297368], [0.79629287, 0.27208874, 0.26660728, 0.86119936, 0.64135187, 0.19488591]]), ['x_1', 'x_2', 'y_1'], {'x_1': 1, 'x_2': 2, 'y_1': 3}) .. GENERATED FROM PYTHON SOURCE LINES 194-195 or with a dictionary of variables names: .. GENERATED FROM PYTHON SOURCE LINES 195-196 .. code-block:: default print(dataset.get_all_data(by_group=False, as_dict=True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.23124243], [0.20172961], [0.25730461], [0.43760277], [0.79629287]]), 'x_2': array([[0.07393255, 0.35388856], [0.52456865, 0.80053636], [0.81341756, 0.39107675], [0.41877463, 0.58286326], [0.27208874, 0.26660728]]), 'y_1': array([[0.33223059, 0.84173333, 0.87194202], [0.62157433, 0.34984932, 0.11470781], [0.22729595, 0.00813847, 0.1426607 ], [0.28062242, 0.36643361, 0.43297368], [0.86119936, 0.64135187, 0.19488591]])} .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.014 seconds) .. _sphx_glr_download_examples_dataset_plot_dataset_from_array.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_dataset_from_array.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_dataset_from_array.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_