.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/mlearning/regression_model/plot_moe.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_mlearning_regression_model_plot_moe.py: Mixture of experts ================== In this demo, we load a dataset (the Rosenbrock function in 2D) and apply a mixture of experts regression model to obtain an approximation. .. GENERATED FROM PYTHON SOURCE LINES 30-42 .. code-block:: default from __future__ import division, unicode_literals import matplotlib.pyplot as plt from numpy import array, hstack, linspace, meshgrid, nonzero, sqrt, zeros from gemseo.api import configure_logger, load_dataset from gemseo.mlearning.api import create_regression_model from gemseo.mlearning.transform.scaler.min_max_scaler import MinMaxScaler configure_logger() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 43-47 Dataset (Rosenbrock) -------------------- We here consider the Rosenbrock function with two inputs, on the interval :math:`[-2, 2] \times [-2, 2]`. .. GENERATED FROM PYTHON SOURCE LINES 49-54 Load dataset ~~~~~~~~~~~~ A prebuilt dataset for the Rosenbrock function with two inputs is given as a dataset parametrization, based on a full factorial DOE of the input space with 100 points. .. GENERATED FROM PYTHON SOURCE LINES 54-56 .. code-block:: default dataset = load_dataset("RosenbrockDataset", opt_naming=False) .. GENERATED FROM PYTHON SOURCE LINES 57-61 Print information ~~~~~~~~~~~~~~~~~ Information about the dataset can easily be displayed by printing the dataset directly. .. GENERATED FROM PYTHON SOURCE LINES 61-63 .. code-block:: default print(dataset) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Rosenbrock Number of samples: 100 Number of variables: 2 Variables names and sizes by group: inputs: x (2) outputs: rosen (1) Number of dimensions (total = 3) by group: inputs: 2 outputs: 1 .. GENERATED FROM PYTHON SOURCE LINES 64-67 Show dataset ~~~~~~~~~~~~ The dataset object can present the data in tabular form. .. GENERATED FROM PYTHON SOURCE LINES 67-69 .. code-block:: default print(dataset.export_to_dataframe()) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none inputs outputs x rosen 0 1 0 0 -2.000000 -2.0 3609.000000 1 -1.555556 -2.0 1959.952599 2 -1.111111 -2.0 1050.699741 3 -0.666667 -2.0 600.308642 4 -0.222222 -2.0 421.490779 .. ... ... ... 95 0.222222 2.0 381.095717 96 0.666667 2.0 242.086420 97 1.111111 2.0 58.600975 98 1.555556 2.0 17.927907 99 2.000000 2.0 401.000000 [100 rows x 3 columns] .. GENERATED FROM PYTHON SOURCE LINES 70-74 Mixture of experts (MoE) ------------------------ In this section we load a mixture of experts regression model through the machine learning API, using clustering, classification and regression models. .. GENERATED FROM PYTHON SOURCE LINES 76-80 Mixture of experts model ~~~~~~~~~~~~~~~~~~~~~~~~ We construct the MoE model using the predefined parameters, and fit the model to the dataset through the learn() method. .. GENERATED FROM PYTHON SOURCE LINES 80-89 .. code-block:: default model = create_regression_model( "MixtureOfExperts", dataset, transformer={"outputs": MinMaxScaler()} ) model.set_clusterer("KMeans", n_clusters=3) model.set_classifier("KNNClassifier", n_neighbors=5) model.set_regressor("GaussianProcessRegression") model.learn() .. GENERATED FROM PYTHON SOURCE LINES 90-96 Tests ~~~~~ Here, we test the mixture of experts method applied to two points: (1, 1), the global minimum, where the function is zero, and (-2, -2), an extreme point where the function has a high value (max on the domain). The classes are expected to be different at the two points. .. GENERATED FROM PYTHON SOURCE LINES 96-108 .. code-block:: default input_value = {"x": array([1, 1])} another_input_value = {"x": array([[1, 1], [-2, -2]])} for value in [input_value, another_input_value]: print("Input value:", value) print("Class:", model.predict_class(value)) print("Prediction:", model.predict(value)) print("Local model predictions:") for cls in range(model.n_clusters): print("Local model {}: {}".format(cls, model.predict_local_model(value, cls))) print() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Input value: {'x': array([1, 1])} Class: {'labels': array([0])} Prediction: {'rosen': array([4.17031586])} Local model predictions: Local model 0: {'rosen': array([4.17031586])} Local model 1: {'rosen': array([532.87624246])} Local model 2: {'rosen': array([-46.32257302])} Input value: {'x': array([[ 1, 1], [-2, -2]])} Class: {'labels': array([[0], [2]])} Prediction: {'rosen': array([[ 4.17031586], [3608.99981798]])} Local model predictions: Local model 0: {'rosen': array([[ 4.17031586], [409.41192877]])} Local model 1: {'rosen': array([[ 532.87624246], [3514.58059484]])} Local model 2: {'rosen': array([[ -46.32257302], [3608.99981798]])} .. GENERATED FROM PYTHON SOURCE LINES 109-114 Plot clusters ~~~~~~~~~~~~~ Here, we plot the 10x10 = 100 Rosenbrock function data points, with colors representing the obtained clusters. The Rosenbrock function is represented by a contour plot in the background. .. GENERATED FROM PYTHON SOURCE LINES 114-137 .. code-block:: default n_samples = dataset.n_samples # Dataset is based on a DOE of 100=10^2 fullfact. input_dim = int(sqrt(n_samples)) assert input_dim ** 2 == n_samples # Check that n_samples is a square number colors = ["b", "r", "g", "o", "y"] inputs = dataset.get_data_by_group(dataset.INPUT_GROUP) outputs = dataset.get_data_by_group(dataset.OUTPUT_GROUP) x = inputs[:input_dim, 0] y = inputs[:input_dim, 0] Z = zeros((input_dim, input_dim)) for i in range(input_dim): Z[i, :] = outputs[input_dim * i : input_dim * (i + 1), 0] fig = plt.figure() cnt = plt.contour(x, y, Z, 50) fig.colorbar(cnt) for index in range(model.n_clusters): samples = nonzero(model.labels == index)[0] plt.scatter(inputs[samples, 0], inputs[samples, 1], color=colors[index]) plt.scatter(1, 1, marker="x") plt.show() .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_001.png :alt: plot moe :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 138-141 Plot data and predictions from final model ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We construct a refined input space, and compute the model predictions. .. GENERATED FROM PYTHON SOURCE LINES 141-164 .. code-block:: default refinement = 200 fine_x = linspace(x[0], x[-1], refinement) fine_y = linspace(y[0], y[-1], refinement) fine_x, fine_y = meshgrid(fine_x, fine_y) fine_input = {"x": hstack([fine_x.flatten()[:, None], fine_y.flatten()[:, None]])} fine_z = model.predict(fine_input) # Reshape fine_z = fine_z["rosen"].reshape((refinement, refinement)) plt.figure() plt.imshow(Z) plt.colorbar() plt.title("Original data") plt.show() plt.figure() plt.imshow(fine_z) plt.colorbar() plt.title("Predictions") plt.show() .. rst-class:: sphx-glr-horizontal * .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_002.png :alt: Original data :class: sphx-glr-multi-img * .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_003.png :alt: Predictions :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 165-167 Plot local models ~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 167-177 .. code-block:: default for i in range(model.n_clusters): plt.figure() plt.imshow( model.predict_local_model(fine_input, i)["rosen"].reshape( (refinement, refinement) ) ) plt.colorbar() plt.title("Local model {}".format(i)) plt.show() .. rst-class:: sphx-glr-horizontal * .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_004.png :alt: Local model 0 :class: sphx-glr-multi-img * .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_005.png :alt: Local model 1 :class: sphx-glr-multi-img * .. image:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_006.png :alt: Local model 2 :class: sphx-glr-multi-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 3.140 seconds) .. _sphx_glr_download_examples_mlearning_regression_model_plot_moe.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_moe.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_moe.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_