.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/mlearning/regression_model/plot_advanced_moe.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_mlearning_regression_model_plot_advanced_moe.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_mlearning_regression_model_plot_advanced_moe.py:


Advanced mixture of experts
===========================

.. GENERATED FROM PYTHON SOURCE LINES 25-34

.. code-block:: Python


    from __future__ import annotations

    from gemseo import create_benchmark_dataset
    from gemseo.mlearning import create_regression_model
    from gemseo.mlearning.quality_measures.f1_measure import F1Measure
    from gemseo.mlearning.quality_measures.mse_measure import MSEMeasure
    from gemseo.mlearning.quality_measures.silhouette_measure import SilhouetteMeasure








.. GENERATED FROM PYTHON SOURCE LINES 35-37

In this example,
we seek to estimate the Rosenbrock function from the :class:`.RosenbrockDataset`.

.. GENERATED FROM PYTHON SOURCE LINES 37-39

.. code-block:: Python

    dataset = create_benchmark_dataset("RosenbrockDataset", opt_naming=False)








.. GENERATED FROM PYTHON SOURCE LINES 40-56

For that purpose,
we will use an :class:`.MOERegressor` in an advanced way:
we will not set the clustering, classification and regression algorithms
but select them according to their performance
from several candidates that we will provide.
Moreover,
for a given candidate,
we will propose several settings,
compare their performances
and select the best one.

Initialization
--------------
First,
we initialize an :class:`.MOERegressor` with soft classification
by means of the high-level machine learning function :func:`.create_regression_model`.

.. GENERATED FROM PYTHON SOURCE LINES 56-58

.. code-block:: Python

    model = create_regression_model("MOERegressor", dataset, hard=False)








.. GENERATED FROM PYTHON SOURCE LINES 59-70

Clustering
----------
Then,
we add two clustering algorithms
with different numbers of clusters (called *components* for the Gaussian Mixture)
and set the :class:`.SilhouetteMeasure` as clustering measure
to be evaluated from the learning set.
During the learning stage,
the mixture of experts will select the clustering algorithm
and the number of clusters
minimizing this measure.

.. GENERATED FROM PYTHON SOURCE LINES 70-74

.. code-block:: Python

    model.set_clustering_measure(SilhouetteMeasure)
    model.add_clusterer_candidate("KMeans", n_clusters=[2, 3, 4])
    model.add_clusterer_candidate("GaussianMixture", n_components=[3, 4, 5])








.. GENERATED FROM PYTHON SOURCE LINES 75-84

Classification
--------------
We also add classification algorithms
with different settings
and set the :class:`.F1Measure` as classification measure
to be evaluated from the learning set.
During the learning stage,
the mixture of experts will select the classification algorithm and the settings
minimizing this measure.

.. GENERATED FROM PYTHON SOURCE LINES 84-88

.. code-block:: Python

    model.set_classification_measure(F1Measure)
    model.add_classifier_candidate("KNNClassifier", n_neighbors=[3, 4, 5])
    model.add_classifier_candidate("RandomForestClassifier", n_estimators=[100])








.. GENERATED FROM PYTHON SOURCE LINES 89-96

Regression
----------
We also add regression algorithms
and set the :class:`.MSEMeasure` as regression measure
to be evaluated from the learning set.
During the learning stage, for each cluster,
the mixture of experts will select the regression algorithm minimizing this measure.

.. GENERATED FROM PYTHON SOURCE LINES 96-100

.. code-block:: Python

    model.set_regression_measure(MSEMeasure)
    model.add_regressor_candidate("LinearRegressor")
    model.add_regressor_candidate("RBFRegressor")








.. GENERATED FROM PYTHON SOURCE LINES 101-114

.. note::

   We could also add candidates for some learning stages,
   e.g. clustering and regression,
   and set the machine learning algorithms for the remaining ones,
   e.g. classification.

Training
--------
Lastly,
we learn the data
and select the best machine learning algorithm
for both clustering, classification and regression steps.

.. GENERATED FROM PYTHON SOURCE LINES 114-116

.. code-block:: Python

    model.learn()








.. GENERATED FROM PYTHON SOURCE LINES 117-126

Result
------
We can get information on this model,
on the sub-machine learning models selected among the candidates
and on their selected settings.
We can see that
a :class:`.KMeans` with four clusters has been selected for the clustering stage,
as well as a :class:`.RandomForestClassifier` for the classification stage
and a :class:`.RBFRegressor` for each cluster.

.. GENERATED FROM PYTHON SOURCE LINES 126-128

.. code-block:: Python

    model






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div style='background-color: #fafae2; padding: 10px;'>MOERegressor(hard=False)<br/><ul><li>built from 100 learning samples</li><li>Clustering<ul><li>KMeans(n_clusters=4, random_state=0, var_names=None)</li></ul></li><li>Classification<ul><li>RandomForestClassifier(n_estimators=100, random_state=0)</li></ul></li><li>Regression<ul><li>Local model 0<ul><li>RBFRegressor(epsilon=None, function=multiquadric, norm=euclidean, smooth=0.0)</li></ul></li><li>Local model 1<ul><li>RBFRegressor(epsilon=None, function=multiquadric, norm=euclidean, smooth=0.0)</li></ul></li><li>Local model 2<ul><li>RBFRegressor(epsilon=None, function=multiquadric, norm=euclidean, smooth=0.0)</li></ul></li><li>Local model 3<ul><li>RBFRegressor(epsilon=None, function=multiquadric, norm=euclidean, smooth=0.0)</li></ul></li></ul></li></ul></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 129-147

.. note::

   By adding candidates,
   and depending on the complexity of the function to be approximated,
   one could obtain different regression models according to the clusters.
   For example,
   one could use a :class:`.PolynomialRegressor` with order 2
   on a sub-part of the input space
   and a :class:`.GaussianProcessRegressor`
   on another sub-part of the input space.

Once built,
this mixture of experts can be used as any :class:`.BaseMLRegressionAlgo`.

.. seealso::

   :ref:`Another example <sphx_glr_examples_mlearning_regression_model_plot_moe.py>`
   proposes a standard use of :class:`.MOERegressor`.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.422 seconds)


.. _sphx_glr_download_examples_mlearning_regression_model_plot_advanced_moe.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_advanced_moe.ipynb <plot_advanced_moe.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_advanced_moe.py <plot_advanced_moe.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_