.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/dataset/creation/plot_dataset.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_dataset_creation_plot_dataset.py: Dataset ======= In this example, we will see how to build and manipulate a :class:`.Dataset`. From a conceptual point of view, a :class:`.Dataset` is a tabular data structure whose rows are the entries, a.k.a. observations or indices, and whose columns are the features, a.k.a. quantities of interest. These features can be grouped by variable identifier which is a tuple ``(group_name, variable_name)`` and has a dimension equal to the number of components of the variable, a.k.a. dimension. A feature is a tuple ``(group_name, variable_name, component)``. From a software point of view, a :class:`.Dataset` is a particular `pandas DataFrame `__. .. GENERATED FROM PYTHON SOURCE LINES 41-49 .. code-block:: Python from __future__ import annotations from numpy import array from pandas import DataFrame from gemseo.datasets.dataset import Dataset .. GENERATED FROM PYTHON SOURCE LINES 50-54 Instantiation ------------- At instantiation, .. GENERATED FROM PYTHON SOURCE LINES 54-56 .. code-block:: Python dataset = Dataset() .. GENERATED FROM PYTHON SOURCE LINES 57-58 a dataset has the same name as its class: .. GENERATED FROM PYTHON SOURCE LINES 58-60 .. code-block:: Python dataset.name .. rst-class:: sphx-glr-script-out .. code-block:: none 'Dataset' .. GENERATED FROM PYTHON SOURCE LINES 61-62 We can use a more appropriate name at instantiation: .. GENERATED FROM PYTHON SOURCE LINES 62-65 .. code-block:: Python dataset_with_custom_name = Dataset(dataset_name="Measurements") dataset_with_custom_name.name .. rst-class:: sphx-glr-script-out .. code-block:: none 'Measurements' .. GENERATED FROM PYTHON SOURCE LINES 66-67 or change it after instantiation: .. GENERATED FROM PYTHON SOURCE LINES 67-70 .. code-block:: Python dataset_with_custom_name.name = "simulations" dataset_with_custom_name.name .. rst-class:: sphx-glr-script-out .. code-block:: none 'simulations' .. GENERATED FROM PYTHON SOURCE LINES 71-72 Let us check that the class :class:`.Dataset` derives from ``pandas.DataFrame``: .. GENERATED FROM PYTHON SOURCE LINES 72-74 .. code-block:: Python isinstance(dataset, DataFrame) .. rst-class:: sphx-glr-script-out .. code-block:: none True .. GENERATED FROM PYTHON SOURCE LINES 75-80 Add a variable -------------- Then, we can add data by variable name: .. GENERATED FROM PYTHON SOURCE LINES 80-83 .. code-block:: Python dataset.add_variable("a", array([[1, 2], [3, 4]])) dataset .. raw:: html
GROUP parameters
VARIABLE a
COMPONENT 0 1
0 1 2
1 3 4


.. GENERATED FROM PYTHON SOURCE LINES 84-89 Note that the columns of the dataset use the multi-level index ``(GROUP, VARIABLE, COMPONENT)``. By default, the variable is placed in the group .. GENERATED FROM PYTHON SOURCE LINES 89-91 .. code-block:: Python dataset.DEFAULT_GROUP .. rst-class:: sphx-glr-script-out .. code-block:: none 'parameters' .. GENERATED FROM PYTHON SOURCE LINES 92-93 The attribute ``group_name`` allows to use another group: .. GENERATED FROM PYTHON SOURCE LINES 93-96 .. code-block:: Python dataset.add_variable("b", array([[-1, -2, -3], [-4, -5, -6]]), "inputs") dataset .. raw:: html
GROUP parameters inputs
VARIABLE a b
COMPONENT 0 1 0 1 2
0 1 2 -1 -2 -3
1 3 4 -4 -5 -6


.. GENERATED FROM PYTHON SOURCE LINES 97-101 In the same way, for a variable of dimension 2, the components are 0 and 1. We can use other values with the attribute ``components``: .. GENERATED FROM PYTHON SOURCE LINES 101-104 .. code-block:: Python dataset.add_variable("c", array([[1.5], [3.5]]), components=[3]) dataset .. raw:: html
GROUP parameters inputs parameters
VARIABLE a b c
COMPONENT 0 1 0 1 2 3
0 1 2 -1 -2 -3 1.5
1 3 4 -4 -5 -6 3.5


.. GENERATED FROM PYTHON SOURCE LINES 105-109 Add a group of variables ------------------------ Note that the data can also be added by group: .. GENERATED FROM PYTHON SOURCE LINES 109-114 .. code-block:: Python dataset.add_group( "G1", array([[-1.1, -2.1, -3.1], [-4.1, -5.1, -6.1]]), ["p", "q"], {"p": 2, "q": 1} ) dataset .. raw:: html
GROUP parameters inputs parameters G1
VARIABLE a b c p q
COMPONENT 0 1 0 1 2 3 0 1 0
0 1 2 -1 -2 -3 1.5 -1.1 -2.1 -3.1
1 3 4 -4 -5 -6 3.5 -4.1 -5.1 -6.1


.. GENERATED FROM PYTHON SOURCE LINES 115-117 The dimensions of the variables ``{"p": 2, "q": 1}`` are not mandatory when the number of variable names is equal to the number of columns of the data array: .. GENERATED FROM PYTHON SOURCE LINES 117-120 .. code-block:: Python dataset.add_group("G2", array([[1.1, 2.1, 3.1], [4.1, 5.1, 6.1]]), ["x", "y", "z"]) dataset .. raw:: html
GROUP parameters inputs parameters G1 G2
VARIABLE a b c p q x y z
COMPONENT 0 1 0 1 2 3 0 1 0 0 0 0
0 1 2 -1 -2 -3 1.5 -1.1 -2.1 -3.1 1.1 2.1 3.1
1 3 4 -4 -5 -6 3.5 -4.1 -5.1 -6.1 4.1 5.1 6.1


.. GENERATED FROM PYTHON SOURCE LINES 121-126 In the same way, the name of the variable is not mandatory; when missing, ``"x"`` will be considered with a dimension equal to the number of columns of the data array: .. GENERATED FROM PYTHON SOURCE LINES 126-129 .. code-block:: Python dataset.add_group("G3", array([[1.2, 2.2], [3.2, 4.2]])) dataset .. raw:: html
GROUP parameters inputs parameters G1 G2 G3
VARIABLE a b c p q x y z x
COMPONENT 0 1 0 1 2 3 0 1 0 0 0 0 0 1
0 1 2 -1 -2 -3 1.5 -1.1 -2.1 -3.1 1.1 2.1 3.1 1.2 2.2
1 3 4 -4 -5 -6 3.5 -4.1 -5.1 -6.1 4.1 5.1 6.1 3.2 4.2


.. GENERATED FROM PYTHON SOURCE LINES 130-135 Convert to a dictionary of arrays --------------------------------- Sometimes, it can be useful to have a dictionary view of the dataset with NumPy arrays as values: .. GENERATED FROM PYTHON SOURCE LINES 135-137 .. code-block:: Python dataset.to_dict_of_arrays() .. rst-class:: sphx-glr-script-out .. code-block:: none {'G1': {'p': array([[-1.1, -2.1], [-4.1, -5.1]]), 'q': array([[-3.1], [-6.1]])}, 'G2': {'x': array([[1.1], [4.1]]), 'y': array([[2.1], [5.1]]), 'z': array([[3.1], [6.1]])}, 'G3': {'x': array([[1.2, 2.2], [3.2, 4.2]])}, 'inputs': {'b': array([[-1, -2, -3], [-4, -5, -6]])}, 'parameters': {'a': array([[1, 2], [3, 4]]), 'c': array([[1.5], [3.5]])}} .. GENERATED FROM PYTHON SOURCE LINES 138-139 We can also flatten this dictionary: .. GENERATED FROM PYTHON SOURCE LINES 139-142 .. code-block:: Python dataset.to_dict_of_arrays(False) .. rst-class:: sphx-glr-script-out .. code-block:: none {'p': array([[-1.1, -2.1], [-4.1, -5.1]]), 'q': array([[-3.1], [-6.1]]), 'G2:x': array([[1.1], [4.1]]), 'y': array([[2.1], [5.1]]), 'z': array([[3.1], [6.1]]), 'G3:x': array([[1.2, 2.2], [3.2, 4.2]]), 'b': array([[-1, -2, -3], [-4, -5, -6]]), 'a': array([[1, 2], [3, 4]]), 'c': array([[1.5], [3.5]])} .. GENERATED FROM PYTHON SOURCE LINES 143-151 Get information --------------- Some properties ~~~~~~~~~~~~~~~ At any time, we can access to the names of the groups of variables: .. GENERATED FROM PYTHON SOURCE LINES 151-153 .. code-block:: Python dataset.group_names .. rst-class:: sphx-glr-script-out .. code-block:: none ['G1', 'G2', 'G3', 'inputs', 'parameters'] .. GENERATED FROM PYTHON SOURCE LINES 154-155 and to the total number of components per group: .. GENERATED FROM PYTHON SOURCE LINES 155-157 .. code-block:: Python dataset.group_names_to_n_components .. rst-class:: sphx-glr-script-out .. code-block:: none {'G1': 3, 'G2': 3, 'G3': 2, 'inputs': 3, 'parameters': 3} .. GENERATED FROM PYTHON SOURCE LINES 158-161 Concerning the variables, note that we can use the same variable name in two different groups. The (unique) variable names can be accessed with .. GENERATED FROM PYTHON SOURCE LINES 161-163 .. code-block:: Python dataset.variable_names .. rst-class:: sphx-glr-script-out .. code-block:: none ['a', 'b', 'c', 'p', 'q', 'x', 'y', 'z'] .. GENERATED FROM PYTHON SOURCE LINES 164-165 while the total number of components per variable name can be accessed with .. GENERATED FROM PYTHON SOURCE LINES 165-167 .. code-block:: Python dataset.variable_names_to_n_components .. rst-class:: sphx-glr-script-out .. code-block:: none {'a': 2, 'b': 3, 'c': 1, 'p': 2, 'q': 1, 'x': 3, 'y': 1, 'z': 1} .. GENERATED FROM PYTHON SOURCE LINES 168-170 Lastly, the variable identifiers ``(group_name, variable_name)`` can be accessed with .. GENERATED FROM PYTHON SOURCE LINES 170-172 .. code-block:: Python dataset.variable_identifiers .. rst-class:: sphx-glr-script-out .. code-block:: none [('G1', 'p'), ('G1', 'q'), ('G2', 'x'), ('G2', 'y'), ('G2', 'z'), ('G3', 'x'), ('inputs', 'b'), ('parameters', 'a'), ('parameters', 'c')] .. GENERATED FROM PYTHON SOURCE LINES 173-177 Some getters ~~~~~~~~~~~~ We can also easily access to the group of a variable: .. GENERATED FROM PYTHON SOURCE LINES 177-179 .. code-block:: Python dataset.get_group_names("x") .. rst-class:: sphx-glr-script-out .. code-block:: none ['G2', 'G3'] .. GENERATED FROM PYTHON SOURCE LINES 180-181 and to the names of the variables included in a group: .. GENERATED FROM PYTHON SOURCE LINES 181-183 .. code-block:: Python dataset.get_variable_names("G1") .. rst-class:: sphx-glr-script-out .. code-block:: none ['p', 'q'] .. GENERATED FROM PYTHON SOURCE LINES 184-185 The components of a variable located in a group can be accessed with .. GENERATED FROM PYTHON SOURCE LINES 185-187 .. code-block:: Python dataset.get_variable_components("G2", "y") .. rst-class:: sphx-glr-script-out .. code-block:: none [0] .. GENERATED FROM PYTHON SOURCE LINES 188-190 Lastly, the columns of the dataset have string representations: .. GENERATED FROM PYTHON SOURCE LINES 190-192 .. code-block:: Python dataset.get_columns() .. rst-class:: sphx-glr-script-out .. code-block:: none ['a[0]', 'a[1]', 'b[0]', 'b[1]', 'b[2]', 'c', 'p[0]', 'p[1]', 'q', 'x', 'y', 'z', 'x[0]', 'x[1]'] .. GENERATED FROM PYTHON SOURCE LINES 193-194 that can be split into tuples: .. GENERATED FROM PYTHON SOURCE LINES 194-196 .. code-block:: Python dataset.get_columns(as_tuple=True) .. rst-class:: sphx-glr-script-out .. code-block:: none [('parameters', 'a', 0), ('parameters', 'a', 1), ('inputs', 'b', 0), ('inputs', 'b', 1), ('inputs', 'b', 2), ('parameters', 'c', 3), ('G1', 'p', 0), ('G1', 'p', 1), ('G1', 'q', 0), ('G2', 'x', 0), ('G2', 'y', 0), ('G2', 'z', 0), ('G3', 'x', 0), ('G3', 'x', 1)] .. GENERATED FROM PYTHON SOURCE LINES 197-198 We can also consider a subset of the columns: .. GENERATED FROM PYTHON SOURCE LINES 198-200 .. code-block:: Python dataset.get_columns(["c", "y"]) .. rst-class:: sphx-glr-script-out .. code-block:: none ['c', 'y'] .. GENERATED FROM PYTHON SOURCE LINES 201-204 Renaming -------- It is quite easy to rename a group: .. GENERATED FROM PYTHON SOURCE LINES 204-207 .. code-block:: Python dataset.rename_group("G1", "foo") dataset.group_names .. rst-class:: sphx-glr-script-out .. code-block:: none ['G2', 'G3', 'foo', 'inputs', 'parameters'] .. GENERATED FROM PYTHON SOURCE LINES 208-209 or a variable: .. GENERATED FROM PYTHON SOURCE LINES 209-213 .. code-block:: Python dataset.rename_variable("x", "bar", "G2") dataset.rename_variable("y", "baz") dataset.variable_names .. rst-class:: sphx-glr-script-out .. code-block:: none ['a', 'b', 'bar', 'baz', 'c', 'p', 'q', 'x', 'z'] .. GENERATED FROM PYTHON SOURCE LINES 214-217 Note that the group name ``"G2"`` allows to rename ``"x"`` only in ``"G2"``; without this information, the method would have renamed ``"x"`` in both ``"G2"`` and ``"G3"``. .. GENERATED FROM PYTHON SOURCE LINES 219-224 Transformation to a variable ---------------------------- One can use a function applying to a NumPy array to transform the data associated with a variable, for instance a twofold increase: .. GENERATED FROM PYTHON SOURCE LINES 224-226 .. code-block:: Python dataset.transform_data(lambda x: 2 * x, variable_names="bar") .. GENERATED FROM PYTHON SOURCE LINES 227-233 Get a view of the dataset ------------------------- The method :meth:`~.Dataset.get_view` returns a view of the dataset by using masks built from variable names, group names, components and row indices. For instance, we can get a view of the variables ``"b"`` and ``"x"``: .. GENERATED FROM PYTHON SOURCE LINES 233-235 .. code-block:: Python dataset.get_view(variable_names=["b", "x"]) .. raw:: html
GROUP inputs G3
VARIABLE b x
COMPONENT 0 1 2 0 1
0 -1 -2 -3 1.2 2.2
1 -4 -5 -6 3.2 4.2


.. GENERATED FROM PYTHON SOURCE LINES 236-237 or a view of the group ``"inputs"``: .. GENERATED FROM PYTHON SOURCE LINES 237-239 .. code-block:: Python dataset.get_view("inputs") .. raw:: html
GROUP inputs
VARIABLE b
COMPONENT 0 1 2
0 -1 -2 -3
1 -4 -5 -6


.. GENERATED FROM PYTHON SOURCE LINES 240-241 We can also combine the keys: .. GENERATED FROM PYTHON SOURCE LINES 241-243 .. code-block:: Python dataset.get_view(variable_names=["b", "x"], components=[0]) .. raw:: html
GROUP inputs G3
VARIABLE b x
COMPONENT 0 0
0 -1 1.2
1 -4 3.2


.. GENERATED FROM PYTHON SOURCE LINES 244-249 Update some data ---------------- To complete this example, we can update the data by using masks built from variable names, group names, components and row indices: .. GENERATED FROM PYTHON SOURCE LINES 249-251 .. code-block:: Python dataset.update_data([[10, 10, 10]], "inputs", indices=[1]) dataset .. raw:: html
GROUP parameters inputs parameters foo G2 G3
VARIABLE a b c p q bar baz z x
COMPONENT 0 1 0 1 2 3 0 1 0 0 0 0 0 1
0 1 2 -1 -2 -3 1.5 -1.1 -2.1 -3.1 2.2 2.1 3.1 1.2 2.2
1 3 4 10 10 10 3.5 -4.1 -5.1 -6.1 8.2 5.1 6.1 3.2 4.2


.. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.099 seconds) .. _sphx_glr_download_examples_dataset_creation_plot_dataset.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_dataset.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_dataset.py ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_