.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/mlearning/regression_model/plot_random_forest_regression.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_mlearning_regression_model_plot_random_forest_regression.py: Random forest ============= A :class:`.RandomForestRegressor` is a random forest model based on `scikit-learn `__. .. GENERATED FROM PYTHON SOURCE LINES 28-39 .. code-block:: Python from __future__ import annotations from matplotlib import pyplot as plt from numpy import array from gemseo import create_design_space from gemseo import create_discipline from gemseo import sample_disciplines from gemseo.mlearning import create_regression_model .. GENERATED FROM PYTHON SOURCE LINES 40-45 Problem ------- In this example, we represent the function :math:`f(x)=(6x-2)^2\sin(12x-4)` :cite:`forrester2008` by the :class:`.AnalyticDiscipline` .. GENERATED FROM PYTHON SOURCE LINES 45-50 .. code-block:: Python discipline = create_discipline( "AnalyticDiscipline", name="f", expressions={"y": "(6*x-2)**2*sin(12*x-4)"}, ) .. GENERATED FROM PYTHON SOURCE LINES 51-52 and seek to approximate it over the input space .. GENERATED FROM PYTHON SOURCE LINES 52-55 .. code-block:: Python input_space = create_design_space() input_space.add_variable("x", lower_bound=0.0, upper_bound=1.0) .. GENERATED FROM PYTHON SOURCE LINES 56-58 To do this, we create a training dataset with 6 equispaced points: .. GENERATED FROM PYTHON SOURCE LINES 58-62 .. code-block:: Python training_dataset = sample_disciplines( [discipline], input_space, "y", algo_name="PYDOE_FULLFACT", n_samples=6 ) .. rst-class:: sphx-glr-script-out .. code-block:: none INFO - 16:22:23: *** Start Sampling execution *** INFO - 16:22:23: Sampling INFO - 16:22:23: Disciplines: f INFO - 16:22:23: MDO formulation: MDF INFO - 16:22:23: Running the algorithm PYDOE_FULLFACT: INFO - 16:22:23: 17%|█▋ | 1/6 [00:00<00:00, 670.02 it/sec] INFO - 16:22:23: 33%|███▎ | 2/6 [00:00<00:00, 1089.15 it/sec] INFO - 16:22:23: 50%|█████ | 3/6 [00:00<00:00, 1422.44 it/sec] INFO - 16:22:23: 67%|██████▋ | 4/6 [00:00<00:00, 1700.16 it/sec] INFO - 16:22:23: 83%|████████▎ | 5/6 [00:00<00:00, 1913.81 it/sec] INFO - 16:22:23: 100%|██████████| 6/6 [00:00<00:00, 2055.02 it/sec] INFO - 16:22:23: *** End Sampling execution *** .. GENERATED FROM PYTHON SOURCE LINES 63-69 Basics ------ Training ~~~~~~~~ Then, we train an random forest regression model from these samples: .. GENERATED FROM PYTHON SOURCE LINES 69-72 .. code-block:: Python model = create_regression_model("RandomForestRegressor", training_dataset) model.learn() .. GENERATED FROM PYTHON SOURCE LINES 73-77 Prediction ~~~~~~~~~~ Once it is built, we can predict the output value of :math:`f` at a new input point: .. GENERATED FROM PYTHON SOURCE LINES 77-81 .. code-block:: Python input_value = {"x": array([0.65])} output_value = model.predict(input_value) output_value .. rst-class:: sphx-glr-script-out .. code-block:: none {'y': array([-0.88837697])} .. GENERATED FROM PYTHON SOURCE LINES 82-83 but cannot predict its Jacobian value: .. GENERATED FROM PYTHON SOURCE LINES 83-88 .. code-block:: Python try: model.predict_jacobian(input_value) except NotImplementedError: print("The derivatives are not available for RandomForestRegressor.") .. rst-class:: sphx-glr-script-out .. code-block:: none The derivatives are not available for RandomForestRegressor. .. GENERATED FROM PYTHON SOURCE LINES 89-93 Plotting ~~~~~~~~ You can see that the random forest model is pretty good on the left, but bad on the right: .. GENERATED FROM PYTHON SOURCE LINES 93-105 .. code-block:: Python test_dataset = sample_disciplines( [discipline], input_space, "y", algo_name="PYDOE_FULLFACT", n_samples=100 ) input_data = test_dataset.get_view(variable_names=model.input_names).to_numpy() reference_output_data = test_dataset.get_view(variable_names="y").to_numpy().ravel() predicted_output_data = model.predict(input_data).ravel() plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_001.png :alt: plot random forest regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none INFO - 16:22:23: *** Start Sampling execution *** INFO - 16:22:23: Sampling INFO - 16:22:23: Disciplines: f INFO - 16:22:23: MDO formulation: MDF INFO - 16:22:23: Running the algorithm PYDOE_FULLFACT: INFO - 16:22:23: 1%| | 1/100 [00:00<00:00, 3705.22 it/sec] INFO - 16:22:23: 2%|▏ | 2/100 [00:00<00:00, 3597.17 it/sec] INFO - 16:22:23: 3%|▎ | 3/100 [00:00<00:00, 3593.06 it/sec] INFO - 16:22:23: 4%|▍ | 4/100 [00:00<00:00, 3734.08 it/sec] INFO - 16:22:23: 5%|▌ | 5/100 [00:00<00:00, 3815.78 it/sec] INFO - 16:22:23: 6%|▌ | 6/100 [00:00<00:00, 3894.43 it/sec] INFO - 16:22:23: 7%|▋ | 7/100 [00:00<00:00, 3903.23 it/sec] INFO - 16:22:23: 8%|▊ | 8/100 [00:00<00:00, 3958.29 it/sec] INFO - 16:22:23: 9%|▉ | 9/100 [00:00<00:00, 4001.77 it/sec] INFO - 16:22:23: 10%|█ | 10/100 [00:00<00:00, 4035.70 it/sec] INFO - 16:22:23: 11%|█ | 11/100 [00:00<00:00, 4032.98 it/sec] INFO - 16:22:23: 12%|█▏ | 12/100 [00:00<00:00, 4060.97 it/sec] INFO - 16:22:23: 13%|█▎ | 13/100 [00:00<00:00, 4090.16 it/sec] INFO - 16:22:23: 14%|█▍ | 14/100 [00:00<00:00, 4108.32 it/sec] INFO - 16:22:23: 15%|█▌ | 15/100 [00:00<00:00, 4106.69 it/sec] INFO - 16:22:23: 16%|█▌ | 16/100 [00:00<00:00, 4124.19 it/sec] INFO - 16:22:23: 17%|█▋ | 17/100 [00:00<00:00, 4147.94 it/sec] INFO - 16:22:23: 18%|█▊ | 18/100 [00:00<00:00, 4163.08 it/sec] INFO - 16:22:23: 19%|█▉ | 19/100 [00:00<00:00, 4183.73 it/sec] INFO - 16:22:23: 20%|██ | 20/100 [00:00<00:00, 4183.64 it/sec] INFO - 16:22:23: 21%|██ | 21/100 [00:00<00:00, 4197.30 it/sec] INFO - 16:22:23: 22%|██▏ | 22/100 [00:00<00:00, 4199.46 it/sec] INFO - 16:22:23: 23%|██▎ | 23/100 [00:00<00:00, 4213.72 it/sec] INFO - 16:22:23: 24%|██▍ | 24/100 [00:00<00:00, 4213.44 it/sec] INFO - 16:22:23: 25%|██▌ | 25/100 [00:00<00:00, 4221.66 it/sec] INFO - 16:22:23: 26%|██▌ | 26/100 [00:00<00:00, 4234.53 it/sec] INFO - 16:22:23: 27%|██▋ | 27/100 [00:00<00:00, 4247.00 it/sec] INFO - 16:22:23: 28%|██▊ | 28/100 [00:00<00:00, 4256.32 it/sec] INFO - 16:22:23: 29%|██▉ | 29/100 [00:00<00:00, 4252.52 it/sec] INFO - 16:22:23: 30%|███ | 30/100 [00:00<00:00, 4261.06 it/sec] INFO - 16:22:23: 31%|███ | 31/100 [00:00<00:00, 4270.48 it/sec] INFO - 16:22:23: 32%|███▏ | 32/100 [00:00<00:00, 4222.54 it/sec] INFO - 16:22:23: 33%|███▎ | 33/100 [00:00<00:00, 4214.87 it/sec] INFO - 16:22:23: 34%|███▍ | 34/100 [00:00<00:00, 4222.75 it/sec] INFO - 16:22:23: 35%|███▌ | 35/100 [00:00<00:00, 4231.18 it/sec] INFO - 16:22:23: 36%|███▌ | 36/100 [00:00<00:00, 4240.72 it/sec] INFO - 16:22:23: 37%|███▋ | 37/100 [00:00<00:00, 4241.30 it/sec] INFO - 16:22:23: 38%|███▊ | 38/100 [00:00<00:00, 4246.38 it/sec] INFO - 16:22:23: 39%|███▉ | 39/100 [00:00<00:00, 4249.55 it/sec] INFO - 16:22:23: 40%|████ | 40/100 [00:00<00:00, 4255.15 it/sec] INFO - 16:22:23: 41%|████ | 41/100 [00:00<00:00, 4254.38 it/sec] INFO - 16:22:23: 42%|████▏ | 42/100 [00:00<00:00, 4258.90 it/sec] INFO - 16:22:23: 43%|████▎ | 43/100 [00:00<00:00, 4264.72 it/sec] INFO - 16:22:23: 44%|████▍ | 44/100 [00:00<00:00, 4272.97 it/sec] INFO - 16:22:23: 45%|████▌ | 45/100 [00:00<00:00, 4282.14 it/sec] INFO - 16:22:23: 46%|████▌ | 46/100 [00:00<00:00, 4280.95 it/sec] INFO - 16:22:23: 47%|████▋ | 47/100 [00:00<00:00, 4285.95 it/sec] INFO - 16:22:23: 48%|████▊ | 48/100 [00:00<00:00, 4292.04 it/sec] INFO - 16:22:23: 49%|████▉ | 49/100 [00:00<00:00, 4296.45 it/sec] INFO - 16:22:23: 50%|█████ | 50/100 [00:00<00:00, 4296.39 it/sec] INFO - 16:22:23: 51%|█████ | 51/100 [00:00<00:00, 4299.00 it/sec] INFO - 16:22:23: 52%|█████▏ | 52/100 [00:00<00:00, 4305.42 it/sec] INFO - 16:22:23: 53%|█████▎ | 53/100 [00:00<00:00, 4311.78 it/sec] INFO - 16:22:23: 54%|█████▍ | 54/100 [00:00<00:00, 4317.02 it/sec] INFO - 16:22:23: 55%|█████▌ | 55/100 [00:00<00:00, 4315.13 it/sec] INFO - 16:22:23: 56%|█████▌ | 56/100 [00:00<00:00, 4317.51 it/sec] INFO - 16:22:23: 57%|█████▋ | 57/100 [00:00<00:00, 4319.34 it/sec] INFO - 16:22:23: 58%|█████▊ | 58/100 [00:00<00:00, 4321.41 it/sec] INFO - 16:22:23: 59%|█████▉ | 59/100 [00:00<00:00, 4319.42 it/sec] INFO - 16:22:23: 60%|██████ | 60/100 [00:00<00:00, 4322.54 it/sec] INFO - 16:22:23: 61%|██████ | 61/100 [00:00<00:00, 4327.46 it/sec] INFO - 16:22:23: 62%|██████▏ | 62/100 [00:00<00:00, 4331.95 it/sec] INFO - 16:22:23: 63%|██████▎ | 63/100 [00:00<00:00, 4337.15 it/sec] INFO - 16:22:23: 64%|██████▍ | 64/100 [00:00<00:00, 4335.13 it/sec] INFO - 16:22:23: 65%|██████▌ | 65/100 [00:00<00:00, 4338.75 it/sec] INFO - 16:22:23: 66%|██████▌ | 66/100 [00:00<00:00, 4342.54 it/sec] INFO - 16:22:23: 67%|██████▋ | 67/100 [00:00<00:00, 4345.82 it/sec] INFO - 16:22:23: 68%|██████▊ | 68/100 [00:00<00:00, 4343.25 it/sec] INFO - 16:22:23: 69%|██████▉ | 69/100 [00:00<00:00, 4345.58 it/sec] INFO - 16:22:23: 70%|███████ | 70/100 [00:00<00:00, 4349.91 it/sec] INFO - 16:22:23: 71%|███████ | 71/100 [00:00<00:00, 4353.67 it/sec] INFO - 16:22:23: 72%|███████▏ | 72/100 [00:00<00:00, 4357.53 it/sec] INFO - 16:22:23: 73%|███████▎ | 73/100 [00:00<00:00, 4355.27 it/sec] INFO - 16:22:23: 74%|███████▍ | 74/100 [00:00<00:00, 4356.74 it/sec] INFO - 16:22:23: 75%|███████▌ | 75/100 [00:00<00:00, 4358.05 it/sec] INFO - 16:22:23: 76%|███████▌ | 76/100 [00:00<00:00, 4359.09 it/sec] INFO - 16:22:23: 77%|███████▋ | 77/100 [00:00<00:00, 4357.57 it/sec] INFO - 16:22:23: 78%|███████▊ | 78/100 [00:00<00:00, 4359.34 it/sec] INFO - 16:22:23: 79%|███████▉ | 79/100 [00:00<00:00, 4362.45 it/sec] INFO - 16:22:23: 80%|████████ | 80/100 [00:00<00:00, 4365.88 it/sec] INFO - 16:22:23: 81%|████████ | 81/100 [00:00<00:00, 4368.39 it/sec] INFO - 16:22:23: 82%|████████▏ | 82/100 [00:00<00:00, 4366.63 it/sec] INFO - 16:22:23: 83%|████████▎ | 83/100 [00:00<00:00, 4369.01 it/sec] INFO - 16:22:23: 84%|████████▍ | 84/100 [00:00<00:00, 4372.16 it/sec] INFO - 16:22:23: 85%|████████▌ | 85/100 [00:00<00:00, 4375.72 it/sec] INFO - 16:22:23: 86%|████████▌ | 86/100 [00:00<00:00, 4374.90 it/sec] INFO - 16:22:23: 87%|████████▋ | 87/100 [00:00<00:00, 4376.30 it/sec] INFO - 16:22:23: 88%|████████▊ | 88/100 [00:00<00:00, 4379.28 it/sec] INFO - 16:22:23: 89%|████████▉ | 89/100 [00:00<00:00, 4382.20 it/sec] INFO - 16:22:23: 90%|█████████ | 90/100 [00:00<00:00, 4384.95 it/sec] INFO - 16:22:23: 91%|█████████ | 91/100 [00:00<00:00, 4383.72 it/sec] INFO - 16:22:23: 92%|█████████▏| 92/100 [00:00<00:00, 4384.80 it/sec] INFO - 16:22:23: 93%|█████████▎| 93/100 [00:00<00:00, 4384.83 it/sec] INFO - 16:22:23: 94%|█████████▍| 94/100 [00:00<00:00, 4385.49 it/sec] INFO - 16:22:23: 95%|█████████▌| 95/100 [00:00<00:00, 4383.97 it/sec] INFO - 16:22:23: 96%|█████████▌| 96/100 [00:00<00:00, 4385.53 it/sec] INFO - 16:22:23: 97%|█████████▋| 97/100 [00:00<00:00, 4388.10 it/sec] INFO - 16:22:23: 98%|█████████▊| 98/100 [00:00<00:00, 4390.82 it/sec] INFO - 16:22:23: 99%|█████████▉| 99/100 [00:00<00:00, 4393.52 it/sec] INFO - 16:22:23: 100%|██████████| 100/100 [00:00<00:00, 4342.56 it/sec] INFO - 16:22:23: *** End Sampling execution *** .. GENERATED FROM PYTHON SOURCE LINES 106-113 Settings -------- Number of estimators ~~~~~~~~~~~~~~~~~~~~ The main hyperparameter of random forest regression is the number of trees in the forest (default: 100). Here is a comparison when increasing and decreasing this number: .. GENERATED FROM PYTHON SOURCE LINES 113-131 .. code-block:: Python model = create_regression_model( "RandomForestRegressor", training_dataset, n_estimators=10 ) model.learn() predicted_output_data_1 = model.predict(input_data).ravel() model = create_regression_model( "RandomForestRegressor", training_dataset, n_estimators=1000 ) model.learn() predicted_output_data_2 = model.predict(input_data).ravel() plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_1, label="Regression - 10 trees") plt.plot(input_data.ravel(), predicted_output_data_2, label="Regression - 1000 trees") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_002.png :alt: plot random forest regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 132-142 Others ------ The ``RandomForestRegressor`` class of scikit-learn has a lot of settings (`read more `__), and we have chosen to exhibit only ``n_estimators``. However, any argument of ``RandomForestRegressor`` can be set using the dictionary ``parameters``. For example, we can impose a minimum of two samples per leaf: .. GENERATED FROM PYTHON SOURCE LINES 142-153 .. code-block:: Python model = create_regression_model( "RandomForestRegressor", training_dataset, parameters={"min_samples_leaf": 2} ) model.learn() predicted_output_data_ = model.predict(input_data).ravel() plt.plot(input_data.ravel(), reference_output_data, label="Reference") plt.plot(input_data.ravel(), predicted_output_data, label="Regression - Basics") plt.plot(input_data.ravel(), predicted_output_data_, label="Regression - 2 samples") plt.grid() plt.legend() plt.show() .. image-sg:: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_003.png :alt: plot random forest regression :srcset: /examples/mlearning/regression_model/images/sphx_glr_plot_random_forest_regression_003.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.913 seconds) .. _sphx_glr_download_examples_mlearning_regression_model_plot_random_forest_regression.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_random_forest_regression.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_random_forest_regression.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_random_forest_regression.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_