.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/mlearning/quality_measure/plot_mse.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_mlearning_quality_measure_plot_mse.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_mlearning_quality_measure_plot_mse.py:


MSE for regression models
=========================

.. GENERATED FROM PYTHON SOURCE LINES 20-32

.. code-block:: Python


    from matplotlib import pyplot as plt
    from numpy import array
    from numpy import linspace
    from numpy import newaxis
    from numpy import sin

    from gemseo.datasets.io_dataset import IODataset
    from gemseo.mlearning.quality_measures.mse_measure import MSEMeasure
    from gemseo.mlearning.regression.polyreg import PolynomialRegressor
    from gemseo.mlearning.regression.rbf import RBFRegressor








.. GENERATED FROM PYTHON SOURCE LINES 33-52

Given a dataset :math:`(x_i,y_i,\hat{y}_i)_{1\leq i \leq N}`
where :math:`x_i` is an input point,
:math:`y_i` is an output observation
and :math:`\hat{y}_i=\hat{f}(x_i)` is an output prediction
computed by a regression model :math:`\hat{f}`,
the mean squared error (MSE) metric is written

.. math::

  \text{MSE} = \frac{1}{N}\sum_{i=1}^N(y_i-\hat{y}_i)^2 \geq 0.

The lower, the better.
From a quantitative point of view,
this depends on the order of magnitude of the outputs.
The square root of this average is often easier to interpret,
as it is expressed in the units of the output (see :class:`.RMSEMeasure`).

To illustrate this quality measure,
let us consider the function :math:`f(x)=(6x-2)^2\sin(12x-4)` :cite:`forrester2008`:

.. GENERATED FROM PYTHON SOURCE LINES 52-58

.. code-block:: Python



    def f(x):
        return (6 * x - 2) ** 2 * sin(12 * x - 4)









.. GENERATED FROM PYTHON SOURCE LINES 59-63

and try to approximate it with a polynomial of order 3.

For this,
we can take these 7 learning input points

.. GENERATED FROM PYTHON SOURCE LINES 63-65

.. code-block:: Python

    x_train = array([0.1, 0.3, 0.5, 0.6, 0.8, 0.9, 0.95])








.. GENERATED FROM PYTHON SOURCE LINES 66-67

and evaluate the model ``f`` over this design of experiments (DOE):

.. GENERATED FROM PYTHON SOURCE LINES 67-69

.. code-block:: Python

    y_train = f(x_train)








.. GENERATED FROM PYTHON SOURCE LINES 70-72

Then,
we create an :class:`.IODataset` from these 7 learning samples:

.. GENERATED FROM PYTHON SOURCE LINES 72-76

.. code-block:: Python

    dataset_train = IODataset()
    dataset_train.add_input_group(x_train[:, newaxis], ["x"])
    dataset_train.add_output_group(y_train[:, newaxis], ["y"])








.. GENERATED FROM PYTHON SOURCE LINES 77-78

and build a :class:`.PolynomialRegressor` with ``degree=3`` from it:

.. GENERATED FROM PYTHON SOURCE LINES 78-81

.. code-block:: Python

    polynomial = PolynomialRegressor(dataset_train, 3)
    polynomial.learn()








.. GENERATED FROM PYTHON SOURCE LINES 82-84

Before using it,
we are going to measure its quality with the MSE metric:

.. GENERATED FROM PYTHON SOURCE LINES 84-88

.. code-block:: Python

    mse = MSEMeasure(polynomial)
    result = mse.compute_learning_measure()
    result, result**0.5 / (y_train.max() - y_train.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([5.6443418]), array([0.137707]))



.. GENERATED FROM PYTHON SOURCE LINES 89-94

This result is medium (14% of the learning output range),
and we can be expected to a poor generalization quality.
As the cost of this academic function is zero,
we can approximate this generalization quality with a large test dataset
whereas the usual test size is about 20% of the training size.

.. GENERATED FROM PYTHON SOURCE LINES 94-102

.. code-block:: Python

    x_test = linspace(0.0, 1.0, 100)
    y_test = f(x_test)
    dataset_test = IODataset()
    dataset_test.add_input_group(x_test[:, newaxis], ["x"])
    dataset_test.add_output_group(y_test[:, newaxis], ["y"])
    result = mse.compute_test_measure(dataset_test)
    result, result**0.5 / (y_test.max() - y_test.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([11.00451361]), array([0.15181886]))



.. GENERATED FROM PYTHON SOURCE LINES 103-106

The quality is higher than 15% of the test output range, which is pretty mediocre.
This can be explained by a broader generalization domain
than that of learning, which highlights the difficulties of extrapolation:

.. GENERATED FROM PYTHON SOURCE LINES 106-114

.. code-block:: Python

    plt.plot(x_test, y_test, "-b", label="Reference")
    plt.plot(x_train, y_train, "ob")
    plt.plot(x_test, polynomial.predict(x_test[:, newaxis]), "-r", label="Prediction")
    plt.plot(x_train, polynomial.predict(x_train[:, newaxis]), "or")
    plt.legend()
    plt.grid()
    plt.show()




.. image-sg:: /examples/mlearning/quality_measure/images/sphx_glr_plot_mse_001.png
   :alt: plot mse
   :srcset: /examples/mlearning/quality_measure/images/sphx_glr_plot_mse_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 115-116

Using the learning domain would slightly improve the quality:

.. GENERATED FROM PYTHON SOURCE LINES 116-125

.. code-block:: Python

    x_test = linspace(x_train.min(), x_train.max(), 100)
    y_test_in_large_domain = y_test
    y_test = f(x_test)
    dataset_test_in_learning_domain = IODataset()
    dataset_test_in_learning_domain.add_input_group(x_test[:, newaxis], ["x"])
    dataset_test_in_learning_domain.add_output_group(y_test[:, newaxis], ["y"])
    mse.compute_test_measure(dataset_test_in_learning_domain)
    result, result**0.5 / (y_test.max() - y_test.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([11.00451361]), array([0.18111514]))



.. GENERATED FROM PYTHON SOURCE LINES 126-129

Lastly,
to get better results without new learning points,
we would have to change the regression model:

.. GENERATED FROM PYTHON SOURCE LINES 129-132

.. code-block:: Python

    rbf = RBFRegressor(dataset_train)
    rbf.learn()








.. GENERATED FROM PYTHON SOURCE LINES 133-135

The quality of this :class:`.RBFRegressor` is quite good,
both on the learning side:

.. GENERATED FROM PYTHON SOURCE LINES 135-139

.. code-block:: Python

    mse_rbf = MSEMeasure(rbf)
    result = mse_rbf.compute_learning_measure()
    result, result**0.5 / (y_train.max() - y_train.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([1.50692212e-28]), array([7.11532547e-16]))



.. GENERATED FROM PYTHON SOURCE LINES 140-141

and on the validation side:

.. GENERATED FROM PYTHON SOURCE LINES 141-144

.. code-block:: Python

    result = mse_rbf.compute_test_measure(dataset_test_in_learning_domain)
    result, result**0.5 / (y_test.max() - y_test.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([0.02227183]), array([0.00814793]))



.. GENERATED FROM PYTHON SOURCE LINES 145-146

including the larger domain:

.. GENERATED FROM PYTHON SOURCE LINES 146-149

.. code-block:: Python

    result = mse_rbf.compute_test_measure(dataset_test)
    result, result**0.5 / (y_test_in_large_domain.max() - y_test_in_large_domain.min())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    (array([0.29357082]), array([0.02479686]))



.. GENERATED FROM PYTHON SOURCE LINES 150-151

A final plot to convince us:

.. GENERATED FROM PYTHON SOURCE LINES 151-158

.. code-block:: Python

    plt.plot(x_test, y_test, "-b", label="Reference")
    plt.plot(x_train, y_train, "ob")
    plt.plot(x_test, rbf.predict(x_test[:, newaxis]), "-r", label="Prediction")
    plt.plot(x_train, rbf.predict(x_train[:, newaxis]), "or")
    plt.legend()
    plt.grid()
    plt.show()



.. image-sg:: /examples/mlearning/quality_measure/images/sphx_glr_plot_mse_002.png
   :alt: plot mse
   :srcset: /examples/mlearning/quality_measure/images/sphx_glr_plot_mse_002.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.315 seconds)


.. _sphx_glr_download_examples_mlearning_quality_measure_plot_mse.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_mse.ipynb <plot_mse.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_mse.py <plot_mse.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_