{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Dataset from a numpy array\n\nIn this example, we will see how to build a :class:`.Dataset` from an numpy\narray. For that, we need to import this :class:`.Dataset` class:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from gemseo.api import configure_logger\nfrom gemseo.core.dataset import Dataset\nfrom numpy import concatenate\nfrom numpy.random import rand\n\nconfigure_logger()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Synthetic data\nLet us consider three parameters:\n\n- x_1 with dimension 1,\n- x_2 with dimension 2,\n- y_1 with dimension 3.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dim_x1 = 1\ndim_x2 = 2\ndim_y1 = 3\nsizes = {\"x_1\": dim_x1, \"x_2\": dim_x2, \"y_1\": dim_y1}\ngroups = {\"x_1\": \"inputs\", \"x_2\": \"inputs\", \"y_1\": \"outputs\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We generate 5 random samples of the inputs where:\n\n- x_1 is stored in the first column,\n- x_2 is stored in the 2nd and 3rd columns\n\nand 5 random samples of the outputs.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "n_samples = 5\ninputs = rand(n_samples, dim_x1 + dim_x2)\ninputs_names = [\"x_1\", \"x_2\"]\noutputs = rand(n_samples, dim_y1)\noutputs_names = [\"y_1\"]\ndata = concatenate((inputs, outputs), 1)\ndata_names = inputs_names + outputs_names"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a dataset\nusing default names\n~~~~~~~~~~~~~~~~~~~\nWe build a :class:`.Dataset` and initialize from the whole data:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dataset = Dataset(name=\"random_dataset\")\ndataset.set_from_array(data)\nprint(dataset)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### using particular names\nWe can also use the names of the variables, rather than the default ones\nfixed by the class:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dataset = Dataset(name=\"random_dataset\")\ndataset.set_from_array(data, data_names, sizes)\nprint(dataset)\nprint(dataset.data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-danger\"><h4>Warning</h4><p>The number of variables names must be equal to the number of columns of\n   the data array. Otherwise, the user has to specify the sizes of the\n   different variables by means of a dictionary and be careful that the\n   total size is equal to this number of columns.</p></div>\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### using particular groups\nWe can also use the notions of groups of variables:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dataset = Dataset(name=\"random_dataset\")\ndataset.set_from_array(data, data_names, sizes, groups)\nprint(dataset)\nprint(dataset.data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-info\"><h4>Note</h4><p>The groups are specified by means of a dictionary where indices are the\n   variables names and values are the groups. If a variable is missing,\n   the default group 'parameters' is considered.</p></div>\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### storing by names\nWe can also store the data by variables names rather than by groups.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dataset = Dataset(name=\"random_dataset\", by_group=False)\ndataset.set_from_array(data, data_names, sizes, groups)\nprint(dataset)\nprint(dataset.data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-info\"><h4>Note</h4><p>The choice to be made between a storage by group and a storage by\n   variables names aims to limit the number of memory copies of numpy arrays.\n   It mainly depends on how the dataset is used and for what purposes.\n   For example, if we want to build a machine learning algorithm from both\n   input and output data, we only have to access the data by group and in\n   this case, storing the data by group is recommended. Conversely, if we\n   want to use the dataset for post-processing purposes, by accessing the\n   variables of the dataset from their names, the storage by variables names\n   is preferable.</p></div>\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Access properties\nVariables names\n~~~~~~~~~~~~~~~\nWe can access the variables names:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.variables)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Variables sizes\nWe can access the variables sizes:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.sizes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Variables groups\nWe can access the variables groups:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.groups)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Access data\nAccess by group\n~~~~~~~~~~~~~~~\nWe can get the data by group, either as an array (default option):\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_data_by_group(\"inputs\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "or as a dictionary indexed by the variables names:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_data_by_group(\"inputs\", True))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Access by variable name\nWe can get the data by variables names,\neither as a dictionary indexed by the variables names (default option):\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_data_by_names([\"x_1\", \"y_1\"]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "or as an array:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_data_by_names([\"x_1\", \"y_1\"], False))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Access all data\nWe can get all the data, either as a large array:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_all_data())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "or as a dictionary indexed by variables names:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_all_data(as_dict=True))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can get these data sorted by category, either with a large array for each\ncategory:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_all_data(by_group=False))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "or with a dictionary of variables names:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(dataset.get_all_data(by_group=False, as_dict=True))"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}