.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/mlearning/regression_model/plot_moe.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_mlearning_regression_model_plot_moe.py: Mixture of experts ================== In this demo, we load a dataset (the Rosenbrock function in 2D) and apply a mixture of experts regression model to obtain an approximation. .. GENERATED FROM PYTHON SOURCE LINES 28-44 .. code-block:: default import matplotlib.pyplot as plt from gemseo.api import configure_logger from gemseo.api import load_dataset from gemseo.mlearning.api import create_regression_model from gemseo.mlearning.transform.scaler.min_max_scaler import MinMaxScaler from numpy import array from numpy import hstack from numpy import linspace from numpy import meshgrid from numpy import nonzero from numpy import sqrt from numpy import zeros configure_logger() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 45-49 Dataset (Rosenbrock) -------------------- We here consider the Rosenbrock function with two inputs, on the interval :math:`[-2, 2] \times [-2, 2]`. .. GENERATED FROM PYTHON SOURCE LINES 51-56 Load dataset ~~~~~~~~~~~~ A prebuilt dataset for the Rosenbrock function with two inputs is given as a dataset parametrization, based on a full factorial DOE of the input space with 100 points. .. GENERATED FROM PYTHON SOURCE LINES 56-58 .. code-block:: default dataset = load_dataset("RosenbrockDataset", opt_naming=False) .. GENERATED FROM PYTHON SOURCE LINES 59-63 Print information ~~~~~~~~~~~~~~~~~ Information about the dataset can easily be displayed by printing the dataset directly. .. GENERATED FROM PYTHON SOURCE LINES 63-65 .. code-block:: default print(dataset) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Rosenbrock Number of samples: 100 Number of variables: 2 Variables names and sizes by group: inputs: x (2) outputs: rosen (1) Number of dimensions (total = 3) by group: inputs: 2 outputs: 1 .. GENERATED FROM PYTHON SOURCE LINES 66-69 Show dataset ~~~~~~~~~~~~ The dataset object can present the data in tabular form. .. GENERATED FROM PYTHON SOURCE LINES 69-71 .. code-block:: default print(dataset.export_to_dataframe()) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none outputs inputs rosen x 0 0 1 0 3609.000000 -2.000000 -2.0 1 1959.952599 -1.555556 -2.0 2 1050.699741 -1.111111 -2.0 3 600.308642 -0.666667 -2.0 4 421.490779 -0.222222 -2.0 .. ... ... ... 95 381.095717 0.222222 2.0 96 242.086420 0.666667 2.0 97 58.600975 1.111111 2.0 98 17.927907 1.555556 2.0 99 401.000000 2.000000 2.0 [100 rows x 3 columns] .. GENERATED FROM PYTHON SOURCE LINES 72-76 Mixture of experts (MoE) ------------------------ In this section we load a mixture of experts regression model through the machine learning API, using clustering, classification and regression models. .. GENERATED FROM PYTHON SOURCE LINES 78-82 Mixture of experts model ~~~~~~~~~~~~~~~~~~~~~~~~ We construct the MoE model using the predefined parameters, and fit the model to the dataset through the learn() method. .. GENERATED FROM PYTHON SOURCE LINES 82-91 .. code-block:: default model = create_regression_model( "MOERegressor", dataset, transformer={"outputs": MinMaxScaler()} ) model.set_clusterer("KMeans", n_clusters=3) model.set_classifier("KNNClassifier", n_neighbors=5) model.set_regressor("GaussianProcessRegressor") model.learn() .. GENERATED FROM PYTHON SOURCE LINES 92-98 Tests ~~~~~ Here, we test the mixture of experts method applied to two points: (1, 1), the global minimum, where the function is zero, and (-2, -2), an extreme point where the function has a high value (max on the domain). The classes are expected to be different at the two points. .. GENERATED FROM PYTHON SOURCE LINES 98-110 .. code-block:: default input_value = {"x": array([1, 1])} another_input_value = {"x": array([[1, 1], [-2, -2]])} for value in [input_value, another_input_value]: print("Input value:", value) print("Class:", model.predict_class(value)) print("Prediction:", model.predict(value)) print("Local model predictions:") for cls in range(model.n_clusters): print(f"Local model {cls}: {model.predict_local_model(value, cls)}") print() .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Input value: {'x': array([1, 1])} Class: {'labels': array([0])} Prediction: {'rosen': array([4.17031586])} Local model predictions: Local model 0: {'rosen': array([4.17031586])} Local model 1: {'rosen': array([532.88370038])} Local model 2: {'rosen': array([-46.32257305])} Input value: {'x': array([[ 1, 1], [-2, -2]])} Class: {'labels': array([[0], [2]])} Prediction: {'rosen': array([[ 4.17031587], [3608.99981798]])} Local model predictions: Local model 0: {'rosen': array([[ 4.17031587], [409.41192826]])} Local model 1: {'rosen': array([[ 532.88370038], [3514.58098638]])} Local model 2: {'rosen': array([[ -46.32257305], [3608.99981798]])} .. GENERATED FROM PYTHON SOURCE LINES 111-116 Plot clusters ~~~~~~~~~~~~~ Here, we plot the 10x10 = 100 Rosenbrock function data points, with colors representing the obtained clusters. The Rosenbrock function is represented by a contour plot in the background. .. GENERATED FROM PYTHON SOURCE LINES 116-139 .. code-block:: default n_samples = dataset.n_samples # Dataset is based on a DOE of 100=10^2 fullfact. input_dim = int(sqrt(n_samples)) assert input_dim**2 == n_samples # Check that n_samples is a square number colors = ["b", "r", "g", "o", "y"] inputs = dataset.get_data_by_group(dataset.INPUT_GROUP) outputs = dataset.get_data_by_group(dataset.OUTPUT_GROUP) x = inputs[:input_dim, 0] y = inputs[:input_dim, 0] Z = zeros((input_dim, input_dim)) for i in range(input_dim): Z[i, :] = outputs[input_dim * i : input_dim * (i + 1), 0] fig = plt.figure() cnt = plt.contour(x, y, Z, 50) fig.colorbar(cnt) for index in range(model.n_clusters): samples = nonzero(model.labels == index)[0] plt.scatter(inputs[samples, 0], inputs[samples, 1], color=colors[index]) plt.scatter(1, 1, marker="x") plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_001.png :alt: plot moe :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 140-143 Plot data and predictions from final model ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We construct a refined input space, and compute the model predictions. .. GENERATED FROM PYTHON SOURCE LINES 143-166 .. code-block:: default refinement = 200 fine_x = linspace(x[0], x[-1], refinement) fine_y = linspace(y[0], y[-1], refinement) fine_x, fine_y = meshgrid(fine_x, fine_y) fine_input = {"x": hstack([fine_x.flatten()[:, None], fine_y.flatten()[:, None]])} fine_z = model.predict(fine_input) # Reshape fine_z = fine_z["rosen"].reshape((refinement, refinement)) plt.figure() plt.imshow(Z) plt.colorbar() plt.title("Original data") plt.show() plt.figure() plt.imshow(fine_z) plt.colorbar() plt.title("Predictions") plt.show() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_002.png :alt: Original data :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_002.png :class: sphx-glr-multi-img * .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_003.png :alt: Predictions :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_003.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 167-169 Plot local models ~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 169-179 .. code-block:: default for i in range(model.n_clusters): plt.figure() plt.imshow( model.predict_local_model(fine_input, i)["rosen"].reshape( (refinement, refinement) ) ) plt.colorbar() plt.title(f"Local model {i}") plt.show() .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_004.png :alt: Local model 0 :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_004.png :class: sphx-glr-multi-img * .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_005.png :alt: Local model 1 :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_005.png :class: sphx-glr-multi-img * .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_006.png :alt: Local model 2 :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_moe_006.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 3.298 seconds) .. _sphx_glr_download_examples_mlearning_regression_model_plot_moe.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_moe.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_moe.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_