.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/mlearning/regression_model/plot_rbf_regression.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_mlearning_regression_model_plot_rbf_regression.py: Radial basis function (RBF) regression ====================================== An :class:`.RBFRegressor` is an RBF model based on `SciPy `__. .. seealso:: You can find more information about RBF models on `this wikipedia page `__. .. GENERATED FROM PYTHON SOURCE LINES 32-47 .. code-block:: Python from __future__ import annotations from matplotlib import pyplot as plt from numpy import array from gemseo import configure_logger from gemseo import create_design_space from gemseo import create_discipline from gemseo import sample_disciplines from gemseo.mlearning import create_regression_model from gemseo.mlearning.regression.algos.rbf_settings import RBF configure_logger() .. rst-class:: sphx-glr-script-out .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 48-53 Problem ------- In this example, we represent the function :math:`f(x)=(6x-2)^2\sin(12x-4)` :cite:`forrester2008` by the :class:`.AnalyticDiscipline` .. GENERATED FROM PYTHON SOURCE LINES 53-58 .. code-block:: Python discipline = create_discipline( "AnalyticDiscipline", name="f", expressions={"y": "(6*x-2)**2*sin(12*x-4)"}, ) .. GENERATED FROM PYTHON SOURCE LINES 59-60 and seek to approximate it over the input space .. GENERATED FROM PYTHON SOURCE LINES 60-63 .. code-block:: Python input_space = create_design_space() input_space.add_variable("x", lower_bound=0.0, upper_bound=1.0) .. GENERATED FROM PYTHON SOURCE LINES 64-66 To do this, we create a training dataset with 6 equispaced points: .. GENERATED FROM PYTHON SOURCE LINES 66-70 .. code-block:: Python training_dataset = sample_disciplines( [discipline], input_space, "y", algo_name="PYDOE_FULLFACT", n_samples=6 ) .. rst-class:: sphx-glr-script-out .. code-block:: none WARNING - 20:35:12: No coupling in MDA, switching chain_linearize to True. INFO - 20:35:12: *** Start Sampling execution *** INFO - 20:35:12: Sampling INFO - 20:35:12: Disciplines: f INFO - 20:35:12: MDO formulation: MDF INFO - 20:35:12: Running the algorithm PYDOE_FULLFACT: INFO - 20:35:12: 17%|█▋ | 1/6 [00:00<00:00, 623.22 it/sec] INFO - 20:35:12: 33%|███▎ | 2/6 [00:00<00:00, 1014.34 it/sec] INFO - 20:35:12: 50%|█████ | 3/6 [00:00<00:00, 1321.18 it/sec] INFO - 20:35:12: 67%|██████▋ | 4/6 [00:00<00:00, 1592.67 it/sec] INFO - 20:35:12: 83%|████████▎ | 5/6 [00:00<00:00, 1831.89 it/sec] INFO - 20:35:12: 100%|██████████| 6/6 [00:00<00:00, 2041.85 it/sec] INFO - 20:35:12: *** End Sampling execution *** .. GENERATED FROM PYTHON SOURCE LINES 71-77 Basics ------ Training ~~~~~~~~ Then, we train an RBF regression model from these samples: .. GENERATED FROM PYTHON SOURCE LINES 77-80 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset) model.learn() .. GENERATED FROM PYTHON SOURCE LINES 81-85 Prediction ~~~~~~~~~~ Once it is built, we can predict the output value of :math:`f` at a new input point: .. GENERATED FROM PYTHON SOURCE LINES 85-89 .. code-block:: Python input_value = {"x": array([0.65])} output_value = model.predict(input_value) output_value .. rst-class:: sphx-glr-script-out .. code-block:: none {'y': array([-2.16802353])} .. GENERATED FROM PYTHON SOURCE LINES 90-91 as well as its Jacobian value: .. GENERATED FROM PYTHON SOURCE LINES 91-94 .. code-block:: Python jacobian_value = model.predict_jacobian(input_value) jacobian_value .. rst-class:: sphx-glr-script-out .. code-block:: none {'y': {'x': array([[-45.81825011]])}} .. GENERATED FROM PYTHON SOURCE LINES 95-98 Plotting ~~~~~~~~ You can see that the RBF model is pretty good on the right, but bad on the left: .. GENERATED FROM PYTHON SOURCE LINES 98-110 .. code-block:: Python test_dataset = sample_disciplines( [discipline], input_space, "y", algo_name="PYDOE_FULLFACT", n_samples=100 ) input_data = test_dataset.get_view(variable_names=model.input_names).to_numpy() reference_output_data = test_dataset.get_view(variable_names="y").to_numpy().ravel() predicted_output_data = model.predict(input_data).ravel() plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_001.png :alt: plot rbf regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none WARNING - 20:35:12: No coupling in MDA, switching chain_linearize to True. INFO - 20:35:12: *** Start Sampling execution *** INFO - 20:35:12: Sampling INFO - 20:35:12: Disciplines: f INFO - 20:35:12: MDO formulation: MDF INFO - 20:35:12: Running the algorithm PYDOE_FULLFACT: INFO - 20:35:12: 1%| | 1/100 [00:00<00:00, 3279.36 it/sec] INFO - 20:35:12: 2%|▏ | 2/100 [00:00<00:00, 3173.90 it/sec] INFO - 20:35:12: 3%|▎ | 3/100 [00:00<00:00, 3434.20 it/sec] INFO - 20:35:12: 4%|▍ | 4/100 [00:00<00:00, 3644.84 it/sec] INFO - 20:35:12: 5%|▌ | 5/100 [00:00<00:00, 3802.63 it/sec] INFO - 20:35:12: 6%|▌ | 6/100 [00:00<00:00, 3865.72 it/sec] INFO - 20:35:12: 7%|▋ | 7/100 [00:00<00:00, 3970.27 it/sec] INFO - 20:35:12: 8%|▊ | 8/100 [00:00<00:00, 4056.39 it/sec] INFO - 20:35:12: 9%|▉ | 9/100 [00:00<00:00, 4122.84 it/sec] INFO - 20:35:12: 10%|█ | 10/100 [00:00<00:00, 4186.77 it/sec] INFO - 20:35:12: 11%|█ | 11/100 [00:00<00:00, 4184.79 it/sec] INFO - 20:35:12: 12%|█▏ | 12/100 [00:00<00:00, 4230.97 it/sec] INFO - 20:35:12: 13%|█▎ | 13/100 [00:00<00:00, 4279.90 it/sec] INFO - 20:35:12: 14%|█▍ | 14/100 [00:00<00:00, 4324.34 it/sec] INFO - 20:35:12: 15%|█▌ | 15/100 [00:00<00:00, 4332.96 it/sec] INFO - 20:35:12: 16%|█▌ | 16/100 [00:00<00:00, 4358.85 it/sec] INFO - 20:35:12: 17%|█▋ | 17/100 [00:00<00:00, 4391.40 it/sec] INFO - 20:35:12: 18%|█▊ | 18/100 [00:00<00:00, 4421.26 it/sec] INFO - 20:35:12: 19%|█▉ | 19/100 [00:00<00:00, 4447.09 it/sec] INFO - 20:35:12: 20%|██ | 20/100 [00:00<00:00, 4449.72 it/sec] INFO - 20:35:12: 21%|██ | 21/100 [00:00<00:00, 4466.32 it/sec] INFO - 20:35:12: 22%|██▏ | 22/100 [00:00<00:00, 4489.16 it/sec] INFO - 20:35:12: 23%|██▎ | 23/100 [00:00<00:00, 4511.48 it/sec] INFO - 20:35:12: 24%|██▍ | 24/100 [00:00<00:00, 4532.75 it/sec] INFO - 20:35:12: 25%|██▌ | 25/100 [00:00<00:00, 4536.93 it/sec] INFO - 20:35:12: 26%|██▌ | 26/100 [00:00<00:00, 4550.47 it/sec] INFO - 20:35:12: 27%|██▋ | 27/100 [00:00<00:00, 4565.46 it/sec] INFO - 20:35:12: 28%|██▊ | 28/100 [00:00<00:00, 4583.04 it/sec] INFO - 20:35:12: 29%|██▉ | 29/100 [00:00<00:00, 4601.28 it/sec] INFO - 20:35:12: 30%|███ | 30/100 [00:00<00:00, 4595.66 it/sec] INFO - 20:35:12: 31%|███ | 31/100 [00:00<00:00, 4603.25 it/sec] INFO - 20:35:12: 32%|███▏ | 32/100 [00:00<00:00, 4616.10 it/sec] INFO - 20:35:12: 33%|███▎ | 33/100 [00:00<00:00, 4626.07 it/sec] INFO - 20:35:12: 34%|███▍ | 34/100 [00:00<00:00, 4636.25 it/sec] INFO - 20:35:12: 35%|███▌ | 35/100 [00:00<00:00, 4625.83 it/sec] INFO - 20:35:12: 36%|███▌ | 36/100 [00:00<00:00, 4631.18 it/sec] INFO - 20:35:12: 37%|███▋ | 37/100 [00:00<00:00, 4640.83 it/sec] INFO - 20:35:12: 38%|███▊ | 38/100 [00:00<00:00, 4651.36 it/sec] INFO - 20:35:12: 39%|███▉ | 39/100 [00:00<00:00, 4660.34 it/sec] INFO - 20:35:12: 40%|████ | 40/100 [00:00<00:00, 4650.26 it/sec] INFO - 20:35:12: 41%|████ | 41/100 [00:00<00:00, 4658.32 it/sec] INFO - 20:35:12: 42%|████▏ | 42/100 [00:00<00:00, 4666.88 it/sec] INFO - 20:35:12: 43%|████▎ | 43/100 [00:00<00:00, 4674.83 it/sec] INFO - 20:35:12: 44%|████▍ | 44/100 [00:00<00:00, 4684.11 it/sec] INFO - 20:35:12: 45%|████▌ | 45/100 [00:00<00:00, 4681.03 it/sec] INFO - 20:35:12: 46%|████▌ | 46/100 [00:00<00:00, 4690.47 it/sec] INFO - 20:35:12: 47%|████▋ | 47/100 [00:00<00:00, 4697.88 it/sec] INFO - 20:35:12: 48%|████▊ | 48/100 [00:00<00:00, 4703.89 it/sec] INFO - 20:35:12: 49%|████▉ | 49/100 [00:00<00:00, 4702.67 it/sec] INFO - 20:35:12: 50%|█████ | 50/100 [00:00<00:00, 4705.72 it/sec] INFO - 20:35:12: 51%|█████ | 51/100 [00:00<00:00, 4711.56 it/sec] INFO - 20:35:12: 52%|█████▏ | 52/100 [00:00<00:00, 4719.23 it/sec] INFO - 20:35:12: 53%|█████▎ | 53/100 [00:00<00:00, 4724.32 it/sec] INFO - 20:35:12: 54%|█████▍ | 54/100 [00:00<00:00, 4719.18 it/sec] INFO - 20:35:12: 55%|█████▌ | 55/100 [00:00<00:00, 4719.45 it/sec] INFO - 20:35:12: 56%|█████▌ | 56/100 [00:00<00:00, 4723.89 it/sec] INFO - 20:35:12: 57%|█████▋ | 57/100 [00:00<00:00, 4728.17 it/sec] INFO - 20:35:12: 58%|█████▊ | 58/100 [00:00<00:00, 4732.87 it/sec] INFO - 20:35:12: 59%|█████▉ | 59/100 [00:00<00:00, 4728.46 it/sec] INFO - 20:35:12: 60%|██████ | 60/100 [00:00<00:00, 4732.82 it/sec] INFO - 20:35:12: 61%|██████ | 61/100 [00:00<00:00, 4683.03 it/sec] INFO - 20:35:12: 62%|██████▏ | 62/100 [00:00<00:00, 4684.01 it/sec] INFO - 20:35:12: 63%|██████▎ | 63/100 [00:00<00:00, 4681.97 it/sec] INFO - 20:35:12: 64%|██████▍ | 64/100 [00:00<00:00, 4684.74 it/sec] INFO - 20:35:12: 65%|██████▌ | 65/100 [00:00<00:00, 4689.11 it/sec] INFO - 20:35:12: 66%|██████▌ | 66/100 [00:00<00:00, 4694.72 it/sec] INFO - 20:35:12: 67%|██████▋ | 67/100 [00:00<00:00, 4700.25 it/sec] INFO - 20:35:12: 68%|██████▊ | 68/100 [00:00<00:00, 4700.89 it/sec] INFO - 20:35:12: 69%|██████▉ | 69/100 [00:00<00:00, 4704.81 it/sec] INFO - 20:35:12: 70%|███████ | 70/100 [00:00<00:00, 4710.74 it/sec] INFO - 20:35:12: 71%|███████ | 71/100 [00:00<00:00, 4716.13 it/sec] INFO - 20:35:12: 72%|███████▏ | 72/100 [00:00<00:00, 4719.99 it/sec] INFO - 20:35:12: 73%|███████▎ | 73/100 [00:00<00:00, 4719.02 it/sec] INFO - 20:35:12: 74%|███████▍ | 74/100 [00:00<00:00, 4722.96 it/sec] INFO - 20:35:12: 75%|███████▌ | 75/100 [00:00<00:00, 4727.57 it/sec] INFO - 20:35:12: 76%|███████▌ | 76/100 [00:00<00:00, 4732.85 it/sec] INFO - 20:35:12: 77%|███████▋ | 77/100 [00:00<00:00, 4735.50 it/sec] INFO - 20:35:12: 78%|███████▊ | 78/100 [00:00<00:00, 4730.62 it/sec] INFO - 20:35:12: 79%|███████▉ | 79/100 [00:00<00:00, 4732.76 it/sec] INFO - 20:35:12: 80%|████████ | 80/100 [00:00<00:00, 4736.65 it/sec] INFO - 20:35:12: 81%|████████ | 81/100 [00:00<00:00, 4740.85 it/sec] INFO - 20:35:12: 82%|████████▏ | 82/100 [00:00<00:00, 4744.75 it/sec] INFO - 20:35:12: 83%|████████▎ | 83/100 [00:00<00:00, 4743.52 it/sec] INFO - 20:35:12: 84%|████████▍ | 84/100 [00:00<00:00, 4745.39 it/sec] INFO - 20:35:12: 85%|████████▌ | 85/100 [00:00<00:00, 4748.35 it/sec] INFO - 20:35:12: 86%|████████▌ | 86/100 [00:00<00:00, 4749.94 it/sec] INFO - 20:35:12: 87%|████████▋ | 87/100 [00:00<00:00, 4752.16 it/sec] INFO - 20:35:12: 88%|████████▊ | 88/100 [00:00<00:00, 4749.82 it/sec] INFO - 20:35:12: 89%|████████▉ | 89/100 [00:00<00:00, 4751.45 it/sec] INFO - 20:35:12: 90%|█████████ | 90/100 [00:00<00:00, 4755.33 it/sec] INFO - 20:35:12: 91%|█████████ | 91/100 [00:00<00:00, 4758.59 it/sec] INFO - 20:35:12: 92%|█████████▏| 92/100 [00:00<00:00, 4762.72 it/sec] INFO - 20:35:12: 93%|█████████▎| 93/100 [00:00<00:00, 4761.43 it/sec] INFO - 20:35:12: 94%|█████████▍| 94/100 [00:00<00:00, 4764.01 it/sec] INFO - 20:35:12: 95%|█████████▌| 95/100 [00:00<00:00, 4767.62 it/sec] INFO - 20:35:12: 96%|█████████▌| 96/100 [00:00<00:00, 4771.22 it/sec] INFO - 20:35:12: 97%|█████████▋| 97/100 [00:00<00:00, 4774.98 it/sec] INFO - 20:35:12: 98%|█████████▊| 98/100 [00:00<00:00, 4771.68 it/sec] INFO - 20:35:12: 99%|█████████▉| 99/100 [00:00<00:00, 4769.05 it/sec] INFO - 20:35:12: 100%|██████████| 100/100 [00:00<00:00, 4771.08 it/sec] INFO - 20:35:12: *** End Sampling execution *** .. GENERATED FROM PYTHON SOURCE LINES 111-123 Settings -------- The :class:`.RBFRegressor` has many options defined in the :class:`.RBFRegressor_Settings` Pydantic model. Function ~~~~~~~~ The default RBF is the multiquadratic function :math:`\sqrt{(r/\epsilon)^2 + 1}` depending on a radius :math:`r` representing a distance between two points and an adjustable constant :math:`\epsilon`. The RBF can be changed using the ``function`` option, which can be either an :class:`.RBF`: .. GENERATED FROM PYTHON SOURCE LINES 123-128 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset, function=RBF.GAUSSIAN) model.learn() predicted_output_data_g = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 129-130 or a Python function: .. GENERATED FROM PYTHON SOURCE LINES 130-147 .. code-block:: Python def rbf(self, r: float) -> float: """Evaluate a cubic RBF. An RBF must take 2 arguments, namely ``(self, r)``. Args: r: The radius. Returns: The RBF value. """ return r**3 model = create_regression_model("RBFRegressor", training_dataset, function=rbf) model.learn() predicted_output_data_c = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 148-149 We can see that the predictions are different: .. GENERATED FROM PYTHON SOURCE LINES 149-157 .. code-block:: Python plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_g, label="Regression - Gaussian RBF") plt.plot(input_data.ravel(), predicted_output_data_c, label="Regression - Cubic RBF") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_002.png :alt: plot rbf regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 158-165 Epsilon ~~~~~~~ Some RBFs depend on an ``epsilon`` parameter whose default value is the average distance between input data. This is the case of ``"multiquadric"``, ``"gaussian"`` and ``"inverse"`` RBFs. For example, we can train a first multiquadric RBF model with an ``epsilon`` set to 0.5 .. GENERATED FROM PYTHON SOURCE LINES 165-168 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset, epsilon=0.5) model.learn() predicted_output_data_1 = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 169-170 a second one with an ``epsilon`` set to 1.0: .. GENERATED FROM PYTHON SOURCE LINES 170-173 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset, epsilon=1.0) model.learn() predicted_output_data_2 = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 174-175 and a last one with an ``epsilon`` set to 2.0: .. GENERATED FROM PYTHON SOURCE LINES 175-178 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset, epsilon=2.0) model.learn() predicted_output_data_3 = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 179-180 and see that this parameter represents the regularity of the regression model: .. GENERATED FROM PYTHON SOURCE LINES 180-189 .. code-block:: Python plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_1, label="Regression - Epsilon(0.5)") plt.plot(input_data.ravel(), predicted_output_data_2, label="Regression - Epsilon(1)") plt.plot(input_data.ravel(), predicted_output_data_3, label="Regression - Epsilon(2)") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_003.png :alt: plot rbf regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 190-196 Smooth ~~~~~~ By default, an RBF model interpolates the training points. This is parametrized by the ``smooth`` option which is set to 0. We can increase the smoothness of the model by increasing this value: .. GENERATED FROM PYTHON SOURCE LINES 196-199 .. code-block:: Python model = create_regression_model("RBFRegressor", training_dataset, smooth=0.1) model.learn() predicted_output_data_ = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 200-201 and see that the model is not interpolating: .. GENERATED FROM PYTHON SOURCE LINES 201-208 .. code-block:: Python plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_, label="Regression - Smooth") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_004.png :alt: plot rbf regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 209-216 Thin plate spline (TPS) ----------------------- TPS regression is a specific case of RBF regression where the RBF is the thin plate radial basis function for :math:`r^2\log(r)`. The :class:`.TPSRegressor` class deriving from :class:`.RBFRegressor` implements this case: .. GENERATED FROM PYTHON SOURCE LINES 216-219 .. code-block:: Python model = create_regression_model("TPSRegressor", training_dataset) model.learn() predicted_output_data_ = model.predict(input_data).ravel() .. GENERATED FROM PYTHON SOURCE LINES 220-222 We can see that the difference between this model and the default multiquadric RBF model: .. GENERATED FROM PYTHON SOURCE LINES 222-229 .. code-block:: Python plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_, label="Regression - TPS") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_005.png :alt: plot rbf regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_rbf_regression_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 230-231 The :class:`.TPSRegressor` can be customized with the :class:`.TPSRegressor_Settings`. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.347 seconds) .. _sphx_glr_download_examples_mlearning_regression_model_plot_rbf_regression.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_rbf_regression.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_rbf_regression.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_rbf_regression.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_