.. Copyright 2021 IRT Saint ExupĂ©ry, https://www.irt-saintexupery.com This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. .. Contributors: :author: Matthias De Lozzo .. _simple_opt_example: Tutorial: How to solve an optimization problem ============================================== Although the library **|g|** is dedicated to the :term:MDO, it can also be used for mono-disciplinary optimization problems. This tutorial presents some examples on analytical test cases. 1. Optimization based on a design of experiments ************************************************ Let :math:(P) be a simple optimization problem: .. math:: (P) = \left\{ \begin{aligned} & \underset{x\in\mathbb{N}^2}{\text{minimize}} & & f(x) = x_1 + x_2 \\ & \text{subject to} & & -5 \leq x \leq 5 \end{aligned} \right. In this subsection, we will see how to use **|g|** to solve this problem :math:(P) by means of a Design Of Experiments (DOE) 1.a. Define the objective function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Firstly, by means of the :meth:~gemseo.api.create_discipline API function, we create a :class:~gemseo.core.discipline.MDODiscipline of :class:~gemseo.disciplines.autopy.AutoPyDiscipline type from a python function: .. code:: from gemseo.api import create_discipline def f(x1=0., x2=0.): y = x1 + x2 return y discipline = create_discipline("AutoPyDiscipline", py_func=f) Now, we want to minimize this :class:~gemseo.core.discipline.MDODiscipline over a design of experiments (DOE). 1.b. Define the design space ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For that, by means of the :meth:~gemseo.api.create_design_space API function, we define the :class:~gemseo.algos.design_space.DesignSpace :math:[-5, 5]\times[-5, 5] by using its :meth:~gemseo.algos.design_space.DesignSpace.add_variable method. .. code:: from gemseo.api import create_design_space design_space = create_design_space() design_space.add_variable("x1", 1, l_b=-5, u_b=5, var_type="integer") design_space.add_variable("x2", 1, l_b=-5, u_b=5, var_type="integer") 1.c. Define the DOE scenario ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Then, by means of the :meth:~gemseo.api.create_scenario API function, we define a :class:~gemseo.core.doe_scenario.DOEScenario from the :class:~gemseo.core.discipline.MDODiscipline and the :class:~gemseo.algos.design_space.DesignSpace defined above: .. code:: from gemseo.api import create_scenario scenario = create_scenario( discipline, "DisciplinaryOpt", "y", design_space, scenario_type="DOE" ) 1.d. Execute the DOE scenario ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Lastly, we solve the :class:~gemseo.algos.opt_problem.OptimizationProblem included in the :class:~gemseo.core.doe_scenario.DOEScenario defined above by minimizing the objective function over a design of experiments included in the :class:~gemseo.algos.design_space.DesignSpace. Precisely, we choose a full factorial design _ of size :math:11^2: .. code:: scenario.execute({"algo": "fullfact", "n_samples": 11**2}) The optimum results can be found in the execution log. It is also possible to extract them by invoking the :meth:~gemseo.core.scenario.Scenario.get_optimum method. It returns a dictionary containing the optimum results for the scenario under consideration: .. code:: opt_results = scenario.get_optimum() print("The solution of P is (x*,f(x*)) = ({}, {})".format( opt_results.x_opt, opt_results.f_opt )) which yields: .. code:: bash The solution of P is (x*,f(x*)) = ([-5, -5], -10.0). 2. Optimization based on a quasi-Newton method by means of the library scipy _ ******************************************************************************************************** Let :math:(P) be a simple optimization problem: .. math:: (P) = \left\{ \begin{aligned} & \underset{x}{\text{minimize}} & & f(x) = \sin(x) - \exp(x) \\ & \text{subject to} & & -2 \leq x \leq 2 \end{aligned} \right. In this subsection, we will see how to use **|g|** to solve this problem :math:(P) by means of an optimizer directly used from the library scipy _. 2.a. Define the objective function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Firstly, we create the objective function and its gradient as standard python functions: .. code-block:: python import numpy as np from gemseo.api import create_discipline def g(x=0): y = np.sin(x) - np.exp(x) return y def dgdx(x=0): y = np.cos(x) - np.exp(x) return y 2.b. Minimize the objective function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now, we can to minimize this :class:~gemseo.core.discipline.MDODiscipline over its design space by means of the L-BFGS-B algorithm _ implemented in the function :code:scipy.optimize.fmin_l_bfgs_b. .. code-block:: python from scipy import optimize x_0 = -0.5 * np.ones(1) opt = optimize.fmin_l_bfgs_b(g, x_0, fprime=dgdx, bounds=[(-.2, 2.)]) x_opt, f_opt, _ = opt Then, we can display the solution of our optimization problem with the following code: .. code:: print("The solution of P is (x*,f(x*)) = ({}, {})".format(x_opt[0], f_opt[0])) which gives: .. code:: bash The solution of P is (x*,f(x*)) = (-0.2, -1.01740008). .. seealso:: You can found the scipy implementation of the L-BFGS-B algorithm by clicking here _. 3. Optimization based on a quasi-Newton method by means of the |g| optimization interface ***************************************************************************************************** Let :math:(P) be a simple optimization problem: .. math:: (P) = \left\{ \begin{aligned} & \underset{x}{\text{minimize}} & & f(x) = \sin(x) - \exp(x) \\ & \text{subject to} & & -2 \leq x \leq 2 \end{aligned} \right. In this subsection, we will see how to use **|g|** to solve this problem :math:(P) by means of an optimizer from scipy _ called through the optimization interface of **|g|**. 3.a. Define the objective function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Firstly, by means of the :meth:~gemseo.api.create_discipline API function, we create a :class:~gemseo.core.discipline.MDODiscipline of :class:~gemseo.disciplines.autopy.AutoPyDiscipline type from a python function: .. code-block:: python import numpy as np from gemseo.api import create_discipline def g(x=0): y = np.sin(x) - np.exp(x) return y def dgdx(x=0): y = np.cos(x) - np.exp(x) return y discipline = create_discipline("AutoPyDiscipline", py_func=g, py_jac=dgdx) Now, we can to minimize this :class:~gemseo.core.discipline.MDODiscipline over a design space, by means of a quasi-Newton method from the initial point :math:0.5. 3.b. Define the design space ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For that, by means of the :meth:~gemseo.api.create_design_space API function, we define the :class:~gemseo.algos.design_space.DesignSpace :math:[-2., 2.] with initial value :math:0.5 by using its :meth:~gemseo.algos.design_space.DesignSpace.add_variable method. .. code:: from gemseo.api import create_design_space design_space = create_design_space() design_space.add_variable("x", 1, l_b=-2., u_b=2., value=-0.5 * np.ones(1)) 3.c. Define the optimization problem ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Then, by means of the :meth:~gemseo.api.create_scenario API function, we define a :class:~gemseo.core.mdo_scenario.MDOScenario from the :class:~gemseo.core.discipline.MDODiscipline and the :class:~gemseo.algos.design_space.DesignSpace defined above: .. code:: from gemseo.api import create_scenario scenario = create_scenario( discipline, "DisciplinaryOpt", "y", design_space, scenario_type="MDO" ) 3.d. Execute the optimization problem ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Lastly, we solve the :class:~gemseo.algos.opt_problem.OptimizationProblem included in the :class:~gemseo.core.mdo_scenario.MDOScenario defined above by minimizing the objective function over the :class:~gemseo.algos.design_space.DesignSpace. Precisely, we choose the L-BFGS-B algorithm _ implemented in the function :code:scipy.optimize.fmin_l_bfgs_b and indirectly called by means of the class :class:~gemseo.algos.opt.opt_factory.OptimizersFactory and of its function :meth:~gemseo.algos.driver_factory.DriverFactory.execute: .. code-block:: python scenario.execute({"algo": "L-BFGS-B", "max_iter": 100}) The optimization results are displayed in the log file. They can also be obtained using the following code: .. code:: opt_results = scenario.get_optimum() print("The solution of P is (x*,f(x*)) = ({}, {})".format( opt_results.x_opt, opt_results.f_opt )) which yields: .. code:: The solution of P is (x*,f(x*)) = (-1.29, -1.24). .. seealso:: You can found the scipy _ implementation of the L-BFGS-B algorithm _ algorithm by clicking here _. .. tip:: In order to get the list of available optimization algorithms, use: .. code:: from gemseo.api import get_available_opt_algorithms algo_list = get_available_opt_algorithms() print('Available algorithms: {}'.format(algo_list)) what gives: .. code:: Available algorithms: ['NLOPT_SLSQP', 'L-BFGS-B', 'SLSQP', 'NLOPT_COBYLA', 'NLOPT_BFGS', 'NLOPT_NEWUOA', 'TNC', 'P-L-BFGS-B', 'NLOPT_MMA', 'NLOPT_BOBYQA', 'ODD'] 4. Saving and post-processing ***************************** After the resolution of the :class:~gemseo.algos.opt_problem.OptimizationProblem, we can export the results into a :term:HDF file: .. code:: problem = scenario.formulation.opt_problem problem.export_hdf("my_optim.hdf5") We can also post-process the optimization history by means of the function :meth:~gemseo.api.execute_post, either from the :class:~gemseo.algos.opt_problem.OptimizationProblem: .. code:: from gemseo.api import execute_post execute_post(problem, "OptHistoryView", save=True, file_path="opt_view_with_doe") or from the :term:HDF file created above: .. code:: from gemseo.api import execute_post execute_post("my_optim.hdf5", "OptHistoryView", save=True, file_path="opt_view_from_disk") This command produces a series of PDF files: .. image:: opt_view_from_disk_variables_history.png :scale: 50% .. image:: opt_view_from_disk_obj_history.png :scale: 50% .. image:: opt_view_from_disk_x_xstar_history.png :scale: 50% .. image:: opt_view_from_disk_hessian_approx.png :scale: 50%