.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/dataset/plot_dataset_from_array.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_dataset_plot_dataset_from_array.py: Dataset from a numpy array ========================== In this example, we will see how to build a :class:`.Dataset` from an numpy array. For that, we need to import this :class:`.Dataset` class: .. GENERATED FROM PYTHON SOURCE LINES 30-41 .. code-block:: default from __future__ import division, unicode_literals from numpy import concatenate from numpy.random import rand from gemseo.api import configure_logger from gemseo.core.dataset import Dataset configure_logger() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 42-49 Synthetic data -------------- Let us consider three parameters: - x_1 with dimension 1, - x_2 with dimension 2, - y_1 with dimension 3. .. GENERATED FROM PYTHON SOURCE LINES 49-55 .. code-block:: default dim_x1 = 1 dim_x2 = 2 dim_y1 = 3 sizes = {"x_1": dim_x1, "x_2": dim_x2, "y_1": dim_y1} groups = {"x_1": "inputs", "x_2": "inputs", "y_1": "outputs"} .. GENERATED FROM PYTHON SOURCE LINES 56-62 We generate 5 random samples of the inputs where: - x_1 is stored in the first column, - x_2 is stored in the 2nd and 3rd columns and 5 random samples of the outputs. .. GENERATED FROM PYTHON SOURCE LINES 62-71 .. code-block:: default n_samples = 5 inputs = rand(n_samples, dim_x1 + dim_x2) inputs_names = ["x_1", "x_2"] outputs = rand(n_samples, dim_y1) outputs_names = ["y_1"] data = concatenate((inputs, outputs), 1) data_names = inputs_names + outputs_names .. GENERATED FROM PYTHON SOURCE LINES 72-77 Create a dataset ---------------- using default names ~~~~~~~~~~~~~~~~~~~ We build a :class:`.Dataset` and initialize from the whole data: .. GENERATED FROM PYTHON SOURCE LINES 77-82 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data) print(dataset) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset Number of samples: 5 Number of variables: 6 Variables names and sizes by group: parameters: x_0 (1), x_1 (1), x_2 (1), x_3 (1), x_4 (1), x_5 (1) Number of dimensions (total = 6) by group: parameters: 6 .. GENERATED FROM PYTHON SOURCE LINES 83-87 using particular names ~~~~~~~~~~~~~~~~~~~~~~ We can also use the names of the variables, rather than the default ones fixed by the class: .. GENERATED FROM PYTHON SOURCE LINES 87-92 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data, data_names, sizes) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset Number of samples: 5 Number of variables: 3 Variables names and sizes by group: parameters: x_1 (1), x_2 (2), y_1 (3) Number of dimensions (total = 6) by group: parameters: 6 {'parameters': array([[0.28090745, 0.28645218, 0.35828105, 0.97849129, 0.67800282, 0.03710835], [0.61126982, 0.42011062, 0.71956156, 0.65640073, 0.8621352 , 0.26646334], [0.37495882, 0.6473052 , 0.12554212, 0.94040597, 0.63485572, 0.63484002], [0.3663889 , 0.17240851, 0.02501899, 0.37819503, 0.14520686, 0.81271455], [0.16533173, 0.28728618, 0.7065517 , 0.89159101, 0.78870336, 0.26617046]])} .. GENERATED FROM PYTHON SOURCE LINES 93-99 .. warning:: The number of variables names must be equal to the number of columns of the data array. Otherwise, the user has to specify the sizes of the different variables by means of a dictionary and be careful that the total size is equal to this number of columns. .. GENERATED FROM PYTHON SOURCE LINES 101-104 using particular groups ~~~~~~~~~~~~~~~~~~~~~~~ We can also use the notions of groups of variables: .. GENERATED FROM PYTHON SOURCE LINES 104-109 .. code-block:: default dataset = Dataset(name="random_dataset") dataset.set_from_array(data, data_names, sizes, groups) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset Number of samples: 5 Number of variables: 3 Variables names and sizes by group: inputs: x_1 (1), x_2 (2) outputs: y_1 (3) Number of dimensions (total = 6) by group: inputs: 3 outputs: 3 {'inputs': array([[0.28090745, 0.28645218, 0.35828105], [0.61126982, 0.42011062, 0.71956156], [0.37495882, 0.6473052 , 0.12554212], [0.3663889 , 0.17240851, 0.02501899], [0.16533173, 0.28728618, 0.7065517 ]]), 'outputs': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])} .. GENERATED FROM PYTHON SOURCE LINES 110-115 .. note:: The groups are specified by means of a dictionary where indices are the variables names and values are the groups. If a variable is missing, the default group 'parameters' is considered. .. GENERATED FROM PYTHON SOURCE LINES 117-120 storing by names ~~~~~~~~~~~~~~~~ We can also store the data by variables names rather than by groups. .. GENERATED FROM PYTHON SOURCE LINES 120-125 .. code-block:: default dataset = Dataset(name="random_dataset", by_group=False) dataset.set_from_array(data, data_names, sizes, groups) print(dataset) print(dataset.data) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none random_dataset Number of samples: 5 Number of variables: 3 Variables names and sizes by group: inputs: x_1 (1), x_2 (2) outputs: y_1 (3) Number of dimensions (total = 6) by group: inputs: 3 outputs: 3 {'x_1': array([[0.28090745], [0.61126982], [0.37495882], [0.3663889 ], [0.16533173]]), 'x_2': array([[0.28645218, 0.35828105], [0.42011062, 0.71956156], [0.6473052 , 0.12554212], [0.17240851, 0.02501899], [0.28728618, 0.7065517 ]]), 'y_1': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])} .. GENERATED FROM PYTHON SOURCE LINES 126-137 .. note:: The choice to be made between a storage by group and a storage by variables names aims to limit the number of memory copies of numpy arrays. It mainly depends on how the dataset is used and for what purposes. For example, if we want to build a machine learning algorithm from both input and output data, we only have to access the data by group and in this case, storing the data by group is recommended. Conversely, if we want to use the dataset for post-processing purposes, by accessing the variables of the dataset from their names, the storage by variables names is preferable. .. GENERATED FROM PYTHON SOURCE LINES 139-144 Access properties ----------------- Variables names ~~~~~~~~~~~~~~~ We can access the variables names: .. GENERATED FROM PYTHON SOURCE LINES 144-146 .. code-block:: default print(dataset.variables) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ['x_1', 'x_2', 'y_1'] .. GENERATED FROM PYTHON SOURCE LINES 147-150 Variables sizes ~~~~~~~~~~~~~~~ We can access the variables sizes: .. GENERATED FROM PYTHON SOURCE LINES 150-152 .. code-block:: default print(dataset.sizes) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': 1, 'x_2': 2, 'y_1': 3} .. GENERATED FROM PYTHON SOURCE LINES 153-156 Variables groups ~~~~~~~~~~~~~~~~ We can access the variables groups: .. GENERATED FROM PYTHON SOURCE LINES 156-158 .. code-block:: default print(dataset.groups) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ['inputs', 'outputs'] .. GENERATED FROM PYTHON SOURCE LINES 159-164 Access data ----------- Access by group ~~~~~~~~~~~~~~~ We can get the data by group, either as an array (default option): .. GENERATED FROM PYTHON SOURCE LINES 164-165 .. code-block:: default print(dataset.get_data_by_group("inputs")) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0.28090745 0.28645218 0.35828105] [0.61126982 0.42011062 0.71956156] [0.37495882 0.6473052 0.12554212] [0.3663889 0.17240851 0.02501899] [0.16533173 0.28728618 0.7065517 ]] .. GENERATED FROM PYTHON SOURCE LINES 166-167 or as a dictionary indexed by the variables names: .. GENERATED FROM PYTHON SOURCE LINES 167-169 .. code-block:: default print(dataset.get_data_by_group("inputs", True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.28090745], [0.61126982], [0.37495882], [0.3663889 ], [0.16533173]]), 'x_2': array([[0.28645218, 0.35828105], [0.42011062, 0.71956156], [0.6473052 , 0.12554212], [0.17240851, 0.02501899], [0.28728618, 0.7065517 ]])} .. GENERATED FROM PYTHON SOURCE LINES 170-174 Access by variable name ~~~~~~~~~~~~~~~~~~~~~~~ We can get the data by variables names, either as a dictionary indexed by the variables names (default option): .. GENERATED FROM PYTHON SOURCE LINES 174-175 .. code-block:: default print(dataset.get_data_by_names(["x_1", "y_1"])) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.28090745], [0.61126982], [0.37495882], [0.3663889 ], [0.16533173]]), 'y_1': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])} .. GENERATED FROM PYTHON SOURCE LINES 176-177 or as an array: .. GENERATED FROM PYTHON SOURCE LINES 177-179 .. code-block:: default print(dataset.get_data_by_names(["x_1", "y_1"], False)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0.28090745 0.97849129 0.67800282 0.03710835] [0.61126982 0.65640073 0.8621352 0.26646334] [0.37495882 0.94040597 0.63485572 0.63484002] [0.3663889 0.37819503 0.14520686 0.81271455] [0.16533173 0.89159101 0.78870336 0.26617046]] .. GENERATED FROM PYTHON SOURCE LINES 180-183 Access all data ~~~~~~~~~~~~~~~ We can get all the data, either as a large array: .. GENERATED FROM PYTHON SOURCE LINES 183-184 .. code-block:: default print(dataset.get_all_data()) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none ({'inputs': array([[0.28090745, 0.28645218, 0.35828105], [0.61126982, 0.42011062, 0.71956156], [0.37495882, 0.6473052 , 0.12554212], [0.3663889 , 0.17240851, 0.02501899], [0.16533173, 0.28728618, 0.7065517 ]]), 'outputs': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])}, {'inputs': ['x_1', 'x_2'], 'outputs': ['y_1']}, {'x_1': 1, 'x_2': 2, 'y_1': 3}) .. GENERATED FROM PYTHON SOURCE LINES 185-186 or as a dictionary indexed by variables names: .. GENERATED FROM PYTHON SOURCE LINES 186-187 .. code-block:: default print(dataset.get_all_data(as_dict=True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'inputs': {'x_1': array([[0.28090745], [0.61126982], [0.37495882], [0.3663889 ], [0.16533173]]), 'x_2': array([[0.28645218, 0.35828105], [0.42011062, 0.71956156], [0.6473052 , 0.12554212], [0.17240851, 0.02501899], [0.28728618, 0.7065517 ]])}, 'outputs': {'y_1': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])}} .. GENERATED FROM PYTHON SOURCE LINES 188-190 We can get these data sorted by category, either with a large array for each category: .. GENERATED FROM PYTHON SOURCE LINES 190-191 .. code-block:: default print(dataset.get_all_data(by_group=False)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none (array([[0.28090745, 0.28645218, 0.35828105, 0.97849129, 0.67800282, 0.03710835], [0.61126982, 0.42011062, 0.71956156, 0.65640073, 0.8621352 , 0.26646334], [0.37495882, 0.6473052 , 0.12554212, 0.94040597, 0.63485572, 0.63484002], [0.3663889 , 0.17240851, 0.02501899, 0.37819503, 0.14520686, 0.81271455], [0.16533173, 0.28728618, 0.7065517 , 0.89159101, 0.78870336, 0.26617046]]), ['x_1', 'x_2', 'y_1'], {'x_1': 1, 'x_2': 2, 'y_1': 3}) .. GENERATED FROM PYTHON SOURCE LINES 192-193 or with a dictionary of variables names: .. GENERATED FROM PYTHON SOURCE LINES 193-194 .. code-block:: default print(dataset.get_all_data(by_group=False, as_dict=True)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none {'x_1': array([[0.28090745], [0.61126982], [0.37495882], [0.3663889 ], [0.16533173]]), 'x_2': array([[0.28645218, 0.35828105], [0.42011062, 0.71956156], [0.6473052 , 0.12554212], [0.17240851, 0.02501899], [0.28728618, 0.7065517 ]]), 'y_1': array([[0.97849129, 0.67800282, 0.03710835], [0.65640073, 0.8621352 , 0.26646334], [0.94040597, 0.63485572, 0.63484002], [0.37819503, 0.14520686, 0.81271455], [0.89159101, 0.78870336, 0.26617046]])} .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.021 seconds) .. _sphx_glr_download_examples_dataset_plot_dataset_from_array.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_dataset_from_array.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_dataset_from_array.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_