..
   Copyright 2021 IRT Saint-Exupéry, https://www.irt-saintexupery.com

   This work is licensed under the Creative Commons Attribution-ShareAlike 4.0
   International License. To view a copy of this license, visit
   http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to Creative
   Commons, PO Box 1866, Mountain View, CA 94042, USA.

.. _gen_regression_algos:

Regression algorithms
=====================

.. warning::
   Some capabilities may require the :ref:`installation <installation>` of |g| with :ref:`all its features <dependencies>`
   and some others may depend on :ref:`plugins <plugins>`.

.. note::
   All the features of the wrapped libraries may not be exposed through |g|.



.. raw:: html

    <style>
    th {
      cursor: pointer;
    }

    th, td {
      text-align: center;
    }

    tr:hover {
        border: 1px solid black;
    }

    tr:nth-child(even) {
      background-color: #f2f2f2
    }
    </style>

    <script>
    function sortTable(n) {
      var table, rows, switching, i, x, y, shouldSwitch, dir, switchcount = 0;
      table = document.getElementById("myTable");
      switching = true;
      dir = "asc";
      while (switching) {
        switching = false;
        rows = table.rows;
        for (i = 1; i < (rows.length - 1); i++) {
          shouldSwitch = false;
          x = rows[i].getElementsByTagName("TD")[n];
          y = rows[i + 1].getElementsByTagName("TD")[n];
          if (dir == "asc") {
            if (x.innerHTML.toLowerCase() > y.innerHTML.toLowerCase()) {
              shouldSwitch= true;
              break;
            }
          } else if (dir == "desc") {
            if (x.innerHTML.toLowerCase() < y.innerHTML.toLowerCase()) {
              shouldSwitch = true;
              break;
            }
          }
        }
        if (shouldSwitch) {
          rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
          switching = true;
          switchcount ++;
        } else {
          if (switchcount == 0 && dir == "asc") {
            dir = "desc";
            switching = true;
          }
        }
      }
    }
    </script>



.. _GaussianProcessRegressor_options:

GaussianProcessRegressor
------------------------



Module: :class:`gemseo.mlearning.regression.gpr`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />








































































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**alpha** : *float | RealArray, optional*

The nugget effect to regularize the model.


By default it is set to 1e-10.


.. raw:: html

    </li>

    <li>

**bounds** : *__Bounds | Mapping[str, __Bounds] | None, optional*

The lower and upper bounds of the parameter length scales
when ``kernel`` is ``None``.
Either a unique lower-upper pair common to all the inputs
or lower-upper pairs for some of them.
When ``bounds`` is ``None`` or when an input has no pair,
the lower bound is 0.01 and the upper bound is 100.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**kernel** : *Kernel | None, optional*

The kernel specifying the covariance model.
If ``None``, use a Matérn(2.5).


By default it is set to None.


.. raw:: html

    </li>

    <li>

**n_restarts_optimizer** : *int, optional*

The number of restarts of the optimizer.


By default it is set to 10.


.. raw:: html

    </li>

    <li>

**optimizer** : *str | Callable, optional*

The optimization algorithm to find the parameter length scales.


By default it is set to fmin_l_bfgs_b.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**random_state** : *int | None, optional*

The random state passed to the random number generator.
Use an integer for reproducible results.


By default it is set to 0.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _LinearRegressor_options:

LinearRegressor
---------------



Module: :class:`gemseo.mlearning.regression.linreg`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />





































































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**fit_intercept** : *bool, optional*

Whether to fit the intercept.


By default it is set to True.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**l2_penalty_ratio** : *float, optional*

The penalty ratio related to the l2 regularization.
If 1, use the Ridge penalty.
If 0, use the Lasso penalty.
Between 0 and 1, use the ElasticNet penalty.


By default it is set to 1.0.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**penalty_level** : *float, optional*

The penalty level greater or equal to 0.
If 0, there is no penalty.


By default it is set to 0.0.


.. raw:: html

    </li>

    <li>

**random_state** : *int | None, optional*

The random state passed to the random number generator
when there is a penalty.
Use an integer for reproducible results.


By default it is set to 0.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    <li>

****parameters** : *float | int | str | bool | None*

The parameters of the machine learning algorithm.



.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _MOERegressor_options:

MOERegressor
------------



Module: :class:`gemseo.mlearning.regression.moe`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />










































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**hard** : *bool, optional*

Whether clustering/classification should be hard or soft.


By default it is set to True.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _PCERegressor_options:

PCERegressor
------------



Module: :class:`gemseo.mlearning.regression.pce`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />


























































































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset | None*

The learning dataset required
in the case of the least-squares regression
or when ``discipline`` is ``None`` in the case of quadrature.

.. raw:: html

    </li>

    <li>

**probability_space** : *ParameterSpace*

The set of random input variables
defined by :class:`.OTDistribution` instances.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**cleaning_options** : *CleaningOptions | None, optional*

The options of the `CleaningStrategy`_.
If ``None``, use :attr:`.DEFAULT_CLEANING_OPTIONS`.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**degree** : *int, optional*

The polynomial degree of the PCE.


By default it is set to 2.


.. raw:: html

    </li>

    <li>

**discipline** : *MDODiscipline | None, optional*

The discipline to be sampled
if ``use_quadrature`` is ``True`` and ``data`` is ``None``.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**hyperbolic_parameter** : *float, optional*

The :math:`q`-quasi norm parameter
of the `hyperbolic and anisotropic enumerate function`_,
defined over the interval :math:`]0,1]`.


By default it is set to 1.0.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**n_quadrature_points** : *int, optional*

The total number of quadrature points
used by the quadrature strategy
to compute the marginal number of points by input dimension
when ``discipline`` is not ``None``.
If ``0``, use :math:`(1+P)^d` points,
where :math:`d` is the dimension of the input space
and :math:`P` is the polynomial degree of the PCE.


By default it is set to 0.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    <li>

**use_cleaning** : *bool, optional*

Whether
to use the `CleaningStrategy`_ algorithm.
Otherwise,
use a fixed truncation strategy (`FixedStrategy`_).


By default it is set to False.


.. raw:: html

    </li>

    <li>

**use_lars** : *bool, optional*

Whether to use the `LARS`_ algorithm
in the case of the least-squares regression.


By default it is set to False.


.. raw:: html

    </li>

    <li>

**use_quadrature** : *bool, optional*

Whether to estimate the coefficients of the PCE
by a quadrature rule;
if so, use the quadrature points stored in ``data``
or sample ``discipline``.
otherwise, estimate the coefficients by least-squares regression.


By default it is set to False.


.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _PolynomialRegressor_options:

PolynomialRegressor
-------------------



Module: :class:`gemseo.mlearning.regression.polyreg`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />





































































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    <li>

**degree** : *int*

The polynomial degree.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**fit_intercept** : *bool, optional*

Whether to fit the intercept.


By default it is set to True.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**l2_penalty_ratio** : *float, optional*

The penalty ratio
related to the l2 regularization.
If 1, the penalty is the Ridge penalty.
If 0, this is the Lasso penalty.
Between 0 and 1, the penalty is the ElasticNet penalty.


By default it is set to 1.0.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**penalty_level** : *float, optional*

The penalty level greater or equal to 0.
If 0, there is no penalty.


By default it is set to 0.0.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    <li>

****parameters** : *float | int | str | bool | None*

The parameters of the machine learning algorithm.



.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _RBFRegressor_options:

RBFRegressor
------------



Module: :class:`gemseo.mlearning.regression.rbf`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />


































































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**der_function** : *Callable[[RealArray], RealArray] | None, optional*

The derivative of the radial basis function,
only to be provided if ``function`` is a callable
and if the use of the model with its derivative is required.
If ``None`` and if ``function`` is a callable,
an error will be raised.
If ``None`` and if ``function`` is a string,
the class will look for its internal implementation
and will raise an error if it is missing.
The ``der_function`` shall take three arguments
(``input_data``, ``norm_input_data``, ``eps``).
For an RBF of the form function(:math:`r`),
der_function(:math:`x`, :math:`|x|`, :math:`\epsilon`) shall
return :math:`\epsilon^{-1} x/|x| f'(|x|/\epsilon)`.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**epsilon** : *float | None, optional*

An adjustable constant for Gaussian or multiquadric functions.
If ``None``, use the average distance between input data.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**function** : *Function | Callable[[float, float], float], optional*

The radial basis function taking a radius :math:`r` as input,
representing a distance between two points.
If it is a string,
then it must be one of the following:

- ``"multiquadric"`` for :math:`\sqrt{(r/\epsilon)^2 + 1}`,
- ``"inverse"`` for :math:`1/\sqrt{(r/\epsilon)^2 + 1}`,
- ``"gaussian"`` for :math:`\exp(-(r/\epsilon)^2)`,
- ``"linear"`` for :math:`r`,
- ``"cubic"`` for :math:`r^3`,
- ``"quintic"`` for :math:`r^5`,
- ``"thin_plate"`` for :math:`r^2\log(r)`.

If it is a callable,
then it must take the two arguments ``self`` and ``r`` as inputs,
e.g. ``lambda self, r: sqrt((r/self.epsilon)**2 + 1)``
for the multiquadric function.
The epsilon parameter will be available as ``self.epsilon``.
Other keyword arguments passed in will be available as well.


By default it is set to multiquadric.


.. raw:: html

    </li>

    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**norm** : *str | Callable[[RealArray, RealArray], float], optional*

The distance metric to be used,
either a distance function name `known by SciPy
<https://docs.scipy.org/doc/scipy/reference/generated/
scipy.spatial.distance.cdist.html>`_
or a function that computes the distance between two points.


By default it is set to euclidean.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**smooth** : *float, optional*

The degree of smoothness,
``0`` involving an interpolation of the learning points.


By default it is set to 0.0.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



.. _RandomForestRegressor_options:

RandomForestRegressor
---------------------



Module: :class:`gemseo.mlearning.regression.random_forest`





.. raw:: html

    <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../_static/css/gemseo.css" type="text/css" />
















































    <dl class="field-list simple">
    <dt class="field-odd">Required parameters</dt>
    <dd class="field-odd"><ul class="simple">

    <li>

**data** : *IODataset*

The learning dataset.

.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>



    <dl class="field-list simple">
    <dt class="field-odd">Optional parameters</dt>
    <dd class="field-odd"><ul class="simple">


    <li>

**input_names** : *Iterable[str] | None, optional*

The names of the input variables.
If ``None``, consider all the input variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**n_estimators** : *int, optional*

The number of trees in the forest.


By default it is set to 100.


.. raw:: html

    </li>

    <li>

**output_names** : *Iterable[str] | None, optional*

The names of the output variables.
If ``None``, consider all the output variables of the learning dataset.


By default it is set to None.


.. raw:: html

    </li>

    <li>

**random_state** : *int | None, optional*

The random state passed to the random number generator.
Use an integer for reproducible results.


By default it is set to 0.


.. raw:: html

    </li>

    <li>

**transformer** : *TransformerType, optional*

The strategies to transform the variables.
The values are instances of :class:`.BaseTransformer`
while the keys are the names of
either the variables
or the groups of variables,
e.g. ``"inputs"`` or ``"outputs"``
in the case of the regression algorithms.
If a group is specified,
the :class:`.BaseTransformer` will be applied
to all the variables of this group.
If :attr:`.IDENTITY`, do not transform the variables.


By default it is set to {}.


.. raw:: html

    </li>

    </ul>
    </dd>
    </dl>