.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/post_process/plot_post_process_optimization_dataset.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_post_process_plot_post_process_optimization_dataset.py: Post-process an optimization dataset ==================================== .. GENERATED FROM PYTHON SOURCE LINES 22-26 Optimization problems can be exported as an :class:`.OptimizationDataset`, which can later be stored as different file types, such as `csv`. One might later want to recover and visualize the stored data. Fortunately, |g| allows one to use different post-processing algorithms with an :class:`.OptimizationDataset` as input. .. GENERATED FROM PYTHON SOURCE LINES 28-32 In this example, we demonstrate how to use an :class:`.OptimizationDataset` for post-processing. The data used are from an MDO scenario on the :class:`.Power2` problem. The data has been saved in an HDF5 file. The post-processing algorithm used for this example is the :class:`.OptHistoryView`. .. GENERATED FROM PYTHON SOURCE LINES 32-45 .. code-block:: Python from __future__ import annotations import numpy as np from gemseo import execute_post from gemseo.algos.constraint_tolerances import ConstraintTolerances from gemseo.algos.design_space import DesignSpace from gemseo.algos.optimization_problem import OptimizationProblem from gemseo.datasets.optimization_dataset import OptimizationDataset from gemseo.datasets.optimization_metadata import OptimizationMetadata from gemseo.settings.post import OptHistoryView_Settings .. GENERATED FROM PYTHON SOURCE LINES 46-48 First we will recover the use case data from an HDF5 file, and convert it into an :class:`.OptimizationProblem`. .. GENERATED FROM PYTHON SOURCE LINES 48-51 .. code-block:: Python problem = OptimizationProblem.from_hdf("power2_opt_pb.h5") .. GENERATED FROM PYTHON SOURCE LINES 52-53 Now the problem gets converted into an :class:`.OptimizationDataset` .. GENERATED FROM PYTHON SOURCE LINES 53-56 .. code-block:: Python dataset = problem.to_dataset(group_functions=True) .. GENERATED FROM PYTHON SOURCE LINES 57-61 As you can see, the argument `group_functions` must be ``True`` in order to use the post-processing, otherwise, the different functions won't be grouped to their corresponding optimization function (objective, inequality constraints, equality constraints, observables). .. GENERATED FROM PYTHON SOURCE LINES 63-66 Now we can execute the post-processing as usual. The only difference is that, instead of passing a :class:`.BaseScenario` or an HDF5 file as an argument, we pass the :class:`.OptimizationDataset`. .. GENERATED FROM PYTHON SOURCE LINES 66-76 .. code-block:: Python execute_post( dataset, settings_model=OptHistoryView_Settings( save=False, show=True, ), ) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_001.png :alt: Evolution of the optimization variables :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_001.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_002.png :alt: Evolution of the objective value :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_002.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_003.png :alt: Evolution of the distance to the optimum :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_003.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_004.png :alt: Evolution of the inequality constraints :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_004.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_005.png :alt: Evolution of the equality constraints :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_005.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-script-out .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 77-81 The advantage of converting an :class:`.OptimizationProblem` to an :class:`.OptimizationDataset` is that there's no manipulation to be done to post-process. As you can tell, there's no significant advantage with doing this instead of post-processing the problem directly. .. GENERATED FROM PYTHON SOURCE LINES 83-87 The advantage of being able to use an :class:`.OptimizationDataset` is that it allows to use |g| post-processing algorithms using any data. To illustrate this, we recover the data from a csv file instead of an HDF5 file. First, we will save the previous data in a csv file. .. GENERATED FROM PYTHON SOURCE LINES 87-89 .. code-block:: Python dataset.to_csv("results.csv") .. GENERATED FROM PYTHON SOURCE LINES 90-91 We can build directly the :class:`.OptimizationDataset` from the csv file. .. GENERATED FROM PYTHON SOURCE LINES 91-97 .. code-block:: Python recovered_dataset = OptimizationDataset.from_csv("results.csv") print(recovered_dataset.summary) .. rst-class:: sphx-glr-script-out .. code-block:: none OptimizationDataset Class: OptimizationDataset Number of entries: 6 Number of variable identifiers: 5 Variables names and sizes by group: designs: x (3) equality_constraints: eq (1) inequality_constraints: ineq1 (1) and ineq2 (1) objectives: pow2 (1) Number of dimensions (total = 7) by group: designs: 3 equality_constraints: 1 inequality_constraints: 2 objectives: 1 .. GENERATED FROM PYTHON SOURCE LINES 98-103 .. note:: Since the data recovered from the csv comes from an existing :class:`.OptimizationDataset`, the variables are already grouped. Details on how to group the variables in case of importing ungrouped data can be found :ref:`here `. .. GENERATED FROM PYTHON SOURCE LINES 105-111 In order to use an :class:`.OptimizationDataset` we must attribute some optimization metadata to the :class:`.OptimizationDataset`. For this we use the :class:`.OptimizationMetadata` and store it in the attribute :attr:`.misc` of the dataset under the key ``"optimization_metadata"``. Some optimization metadata can be recovered from the dataset itself, but overall, it requires to have knowledge of the problem. .. GENERATED FROM PYTHON SOURCE LINES 114-119 The field `output_names_to_constraint_names` makes reference to the cases where the names of functions were changes for a reason or another (like an offset for example). The argument takes the shape of a dictionary where the keys are the original constraint names and the value a list of associated names. For the use case at hand, there is no name change so the associated constraint names are the names themselves. .. GENERATED FROM PYTHON SOURCE LINES 119-126 .. code-block:: Python output_names_to_constraint_names = {} for constraint_name in ( recovered_dataset.inequality_constraint_names + recovered_dataset.equality_constraint_names ): output_names_to_constraint_names[constraint_name] = constraint_name .. GENERATED FROM PYTHON SOURCE LINES 127-129 The optimum iteration can be retrieved from the dataset by looking for the minimum value of the objective function. .. GENERATED FROM PYTHON SOURCE LINES 129-132 .. code-block:: Python optimum_iteration = recovered_dataset.objective_dataset.idxmin(axis=0).values[0] .. GENERATED FROM PYTHON SOURCE LINES 133-136 The tolerances field is an instance of the :class:`.ConstraintTolerances` model. Which must be instantiated with the corresponding values. In this case the default values are used. .. GENERATED FROM PYTHON SOURCE LINES 136-139 .. code-block:: Python tolerances = ConstraintTolerances() .. GENERATED FROM PYTHON SOURCE LINES 140-143 The last important data to be determined is the point feasibility. This can be predetermined and stored in the csv file. In this case, we determine the feasibility using the tolerances to create a mask. .. GENERATED FROM PYTHON SOURCE LINES 143-156 .. code-block:: Python equality_feasible_mask = ( np.abs(recovered_dataset.equality_constraint_dataset) <= tolerances.equality ).all(axis=1) inequality_feasible_mask = ( np.abs(recovered_dataset.inequality_constraint_dataset) <= tolerances.inequality ).all(axis=1) feasible_iterations = recovered_dataset.index[ equality_feasible_mask & inequality_feasible_mask ].tolist() .. GENERATED FROM PYTHON SOURCE LINES 157-159 With all the optimization metadata ready, we can create the :class:`.OptimizationMetadata` and attribute it to the dataset. .. GENERATED FROM PYTHON SOURCE LINES 159-174 .. code-block:: Python opt_metadata = OptimizationMetadata( objective_name="pow2", standardized_objective_name="pow2", minimize_objective=True, use_standardized_objective=False, # Either True or False according to the user tolerances=ConstraintTolerances(), # Add the corresponding tolerances to the pydantic model output_names_to_constraint_names=output_names_to_constraint_names, feasible_iterations=feasible_iterations, optimum_iteration=optimum_iteration, ) recovered_dataset.misc["optimization_metadata"] = opt_metadata .. GENERATED FROM PYTHON SOURCE LINES 175-179 Given that some post-processing algorithms use the input space of the problem, attributing the input space of the problem to the dataset can be useful. For the :class:`.Power2` problem we know that the input space is :math:`-1.0 < x < 1.0` where `x` has 3 components and has initiated with 1.0. .. GENERATED FROM PYTHON SOURCE LINES 179-186 .. code-block:: Python input_space = DesignSpace() input_space.add_variable("x", 3, lower_bound=-1.0, upper_bound=1.0, value=1.0) recovered_dataset.misc["input_space"] = input_space .. GENERATED FROM PYTHON SOURCE LINES 187-188 With all the optimization metadata gathered, we can execute the post-processing. .. GENERATED FROM PYTHON SOURCE LINES 188-198 .. code-block:: Python execute_post( recovered_dataset, settings_model=OptHistoryView_Settings( save=False, show=True, ), ) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_006.png :alt: Evolution of the optimization variables :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_006.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_007.png :alt: Evolution of the objective value :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_007.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_008.png :alt: Evolution of the distance to the optimum :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_008.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_009.png :alt: Evolution of the inequality constraints :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_009.png :class: sphx-glr-multi-img * .. image-sg:: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_010.png :alt: Evolution of the equality constraints :srcset: /examples/post_process/images/sphx_glr_plot_post_process_optimization_dataset_010.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-script-out .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 199-203 .. warning:: The post-processing algorithm :class:`.GradientSensitivity`, has the option to compute missing gradients. It is not possible to use an OptimizationDataset with that option. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.996 seconds) .. _sphx_glr_download_examples_post_process_plot_post_process_optimization_dataset.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_post_process_optimization_dataset.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_post_process_optimization_dataset.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_post_process_optimization_dataset.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_