hessians module¶
Approximation of the Hessian matrix from an optimization history.
Notations:
\(f\): the function of interest for which to approximate the Hessian matrix,
\(y\): the output value of \(f\),
\(x\in\mathbb{R}^d\): the \(d\) input variables of \(f\),
\(k\): the \(k\)th iteration of the optimization history,
\(K\): the iteration of the optimization history at which to approximate the Hessian matrix,
\(x_k\): the input value at iteration \(k\),
\(\Delta x_k=x_{k+1}x_k\): the variation of \(x\) from iteration \(k\) to iteration \(k+1\),
\(y_k\): the output value at iteration \(k\),
\(\Delta y_k=y_{k+1}y_k\): the variation of the function output from iteration \(k\) to iteration \(k+1\),
\(g_k\): the gradient of \(f\) at \(x_k\),
\(\Delta g_k=g_{k+1}g_k\): the variation of the gradient from iteration \(k\) to iteration \(k+1\),
\(B_k\): the approximation of the Hessian of \(f\) at \(x_k\),
\(H_k\): the inverse of \(B_k\).
Classes:

Hessian matrix approximation with the BFGS algorithm. 

Approximation of the Hessian matrix from an optimization history. 

Least squares approximation of an Hessian matrix from an optimization history. 

Hessian matrix approximation with the Symmetric Rank One (SR1) algorithm. 
 class gemseo.post.core.hessians.BFGSApprox(history)[source]¶
Bases:
gemseo.post.core.hessians.HessianApproximation
Hessian matrix approximation with the BFGS algorithm.
 history¶
The optimization history containing input values, output values and Jacobian values.
 Type
 x_ref¶
The value \(x_K\) of the input variables \(x\) at the iteration \(K\) of the optimization history; this is the point at which the Hessian matrix and its inverse are approximated.
 Type
Optional[ndarray]
 fgrad_ref¶
The value \(g_K\) of the gradient function \(g\) of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 f_ref¶
The value \(y_K\) of the output of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 b_mat_history¶
The history \(B_0,B_1,\ldots,B_K\) of the approximations of the Hessian matrix \(B\).
 Type
List[ndarray]
 h_mat_history¶
The history \(H_0,H_1,\ldots,H_K\) of the approximations of the inverse Hessian matrix \(H\).
 Type
List[ndarray]
Initialize self. See help(type(self)) for accurate signature.
 Parameters
history (Database) – The optimization history containing input values, output values and Jacobian values.
 Return type
None
Methods:
build_approximation
(funcname[, save_diag, ...])Compute \(B\), the approximation of the Hessian matrix.
build_inverse_approximation
(funcname[, ...])Compute \(H\), the approximation of the inverse of the Hessian matrix.
compute_corrections
(x_hist, x_grad_hist)Compute the successive variations of both input variables and gradient.
compute_scaling
(hessk, hessk_dsk, ...)Compute the scaling coefficients \(c_1\) and \(c_2\).
get_s_k_y_k
(x_hist, x_grad_hist, iteration)Compute the variation of the input variables and gradient from an iteration.
get_x_grad_history
(funcname[, first_iter, ...])Return the histories of the inputs and gradient.
iterate_approximation
(hessk, dsk, dyk[, scaling])Update \(B\) from iteration \(k\) to iteration \(k+1\).
iterate_inverse_approximation
(h_mat, s_k, y_k)Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
iterate_s_k_y_k
(x_hist, x_grad_hist)Compute the variations of the input variables and gradient.
rebuild_history
(x_corr, x_0, grad_corr, g_0)Compute the history from the corrections of input variables and gradient.
 build_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, b_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, scaling=False, normalize_design_space=False, design_space=None)¶
Compute \(B\), the approximation of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the Hessian matrix.
save_diag (bool) –
Whether to return the approximations of the Hessian’s diagonal.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
b_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the Hessian matrix.
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to be considered.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the Hessian in
b_mat_history
.By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(B\), the approximation of the Hessian matrix.
The diagonal of \(B\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 build_inverse_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, h_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, factorize=False, scaling=False, angle_tol=1e05, step_tol=10000000000.0, normalize_design_space=False, design_space=None)¶
Compute \(H\), the approximation of the inverse of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the inverse of the Hessian matrix.
save_diag (int) –
Whether to return the list of diagonal approximations.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
h_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the inverse of the Hessian matrix. If None, use \(H_0=\frac{\Delta g_k^T\Delta x_k} {\Delta g_k^T\Delta g_k}I_d\).
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to take.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the inverse Hessian in
h_mat_history
.By default it is set to False.
factorize (bool) –
Whether to factorize the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
angle_tol (float) –
The significativity level for \(\Delta g_k^T\Delta x_k\).
By default it is set to 1e05.
step_tol (float) –
The significativity level for \(\\Delta g_k\_{\infty}\).
By default it is set to 10000000000.0.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(H\), the approximation of the inverse of the Hessian matrix.
The diagonal of \(H\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.The matrix \(H_{1/2}\) such that \(H=H_{1/2}H_{1/2}^T\) if
factorize
isTrue
.\(B\), the approximation of the Hessian matrix.
A matrix \(B_{1/2}\) such that \(B=B_{1/2}B_{1/2}^T\) if
factorize
isTrue
.
 Raises
LinAlgError – When either the inversion of \(H\) fails or the Cholesky decomposition of \(H\) or \(B\) fails.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 static compute_corrections(x_hist, x_grad_hist)¶
Compute the successive variations of both input variables and gradient.
These variations are called corrections.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
 Returns
The successive variations of the input variables.
The successive variations of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 static compute_scaling(hessk, hessk_dsk, dskt_hessk_dsk, dyk, dyt_dsk)¶
Compute the scaling coefficients \(c_1\) and \(c_2\).
\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
hessk_dsk (numpy.ndarray) – The product \(B_k\Delta x_k\).
dskt_hessk_dsk (numpy.ndarray) – The product \(\Delta x_k^T B_k\Delta x_k\).
dyk (numpy.ndarray) – The variation of the gradient \(\Delta g_k\).
dyt_dsk (numpy.ndarray) – The product \(\Delta g_k^T\Delta x_k\).
 Returns
TODO * coeff2: TODO
 Return type
coeff1
 static get_s_k_y_k(x_hist, x_grad_hist, iteration)¶
Compute the variation of the input variables and gradient from an iteration.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
iteration (int) – The optimization iteration at which to compute the variations.
 Returns
The difference between the input variables at iteration
iteration+1
and the input variables at iterationiteration
.The difference between the gradient at iteration
iteration+1
and the gradient at iterationiteration
.
 Raises
ValueError – When the iteration is not stored in the database.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 get_x_grad_history(funcname, first_iter=0, last_iter=0, at_most_niter= 1, func_index=None, normalize_design_space=False, design_space=None)¶
Return the histories of the inputs and gradient.
 Parameters
funcname (str) – The name of the function for which to get the gradient.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered. If 0, consider all the iterations.
By default it is set to 0.
at_most_niter (int) –
The maximum number of iterations to be considered. If 1, consider all the iterations.
By default it is set to 1.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
The history of the input variables.
The history of the gradient.
The length of the history.
The dimension of the input space.
 Raises
ValueError – When either the gradient history contains a single element,
func_index
isNone
while the function output is a vector,func_index
is not an output index, the shape of the history of the input variables is not consistent with the shape of the history of the gradient or the optimization history size is insufficient. Return type
Tuple[numpy.ndarray, numpy.ndarray, int, int]
 static iterate_approximation(hessk, dsk, dyk, scaling=False)¶
Update \(B\) from iteration \(k\) to iteration \(k+1\).
Based on an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
hessk
represents \(B_k\) initially before to be overwritten by \(B_{k+1}\) when passed to this method.See also
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
dsk (numpy.ndarray) – The variation \(\Delta x_k\) of the input variables.
dyk (numpy.ndarray) – The variation \(\Delta g_k\) of the gradient.
scaling (bool) –
Whether to use a scaling stage.
By default it is set to False.
 Return type
None
 static iterate_inverse_approximation(h_mat, s_k, y_k, h_factor=None, b_mat=None, b_factor=None, factorize=False, scaling=False)¶
Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
Use an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
and
\(H_{k+1}=c_1^{1}\Pi_{k+1}H_k\Pi_{k+1}^T +c_2^{1}\frac{\Delta x_k\Delta x_k^T}{\Delta g_k^T\Delta x_k}\)
where:
\(\Pi_{k+1}=I_d\frac{\Delta x_k\Delta g_k^T} {\Delta g_k^T\Delta x_k}\)
and where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
h_mat
andb_mat
represent \(H_k\) and \(B_k\) initially before to be overwritten by \(H_{k+1}\) and \(B_{k+1}\) when passed to this method.See also
 Parameters
h_mat – The approximation \(H_k\) of the inverse of the Hessian matrix at iteration \(k\).
s_k – The variation \(\Delta x_k\) of the input variables.
y_k – The variation \(\Delta g_k\) of the gradient.
h_factor –
The square root of the \(H_k\) at iteration \(k\).
By default it is set to None.
b_mat –
The approximation \(B_k\) of the Hessian matrix at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
b_factor –
The square root of the \(B_k\) at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
factorize –
Whether to update the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling –
do scaling step
By default it is set to False.
 static iterate_s_k_y_k(x_hist, x_grad_hist)[source]¶
Compute the variations of the input variables and gradient.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (ndarray) – The history of the input variables.
x_grad_hist (ndarray) – The history of the gradient.
 Returns
The difference between the input variables at iteration
iteration
and the input variables at iterationiteration+1
.The difference between the gradient at iteration
iteration
and the gradient at iterationiteration+1
.
 Return type
Generator[Tuple[ndarray, ndarray]]
 static rebuild_history(x_corr, x_0, grad_corr, g_0)¶
Compute the history from the corrections of input variables and gradient.
A correction is the variation of a quantity between two successive iterations.
 Parameters
x_corr (numpy.ndarray) – The corrections of the input variables.
x_0 (numpy.ndarray) – The initial values of the input variables.
grad_corr (numpy.ndarray) – The corrections of the gradient.
g_0 (numpy.ndarray) – The initial value of the gradient.
 Returns
The history of the input variables.
The history of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 class gemseo.post.core.hessians.HessianApproximation(history)[source]¶
Bases:
object
Approximation of the Hessian matrix from an optimization history.
 history¶
The optimization history containing input values, output values and Jacobian values.
 Type
 x_ref¶
The value \(x_K\) of the input variables \(x\) at the iteration \(K\) of the optimization history; this is the point at which the Hessian matrix and its inverse are approximated.
 Type
Optional[ndarray]
 fgrad_ref¶
The value \(g_K\) of the gradient function \(g\) of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 f_ref¶
The value \(y_K\) of the output of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 b_mat_history¶
The history \(B_0,B_1,\ldots,B_K\) of the approximations of the Hessian matrix \(B\).
 Type
List[ndarray]
 h_mat_history¶
The history \(H_0,H_1,\ldots,H_K\) of the approximations of the inverse Hessian matrix \(H\).
 Type
List[ndarray]
Initialize self. See help(type(self)) for accurate signature.
 Parameters
history (Database) – The optimization history containing input values, output values and Jacobian values.
 Return type
None
Methods:
build_approximation
(funcname[, save_diag, ...])Compute \(B\), the approximation of the Hessian matrix.
build_inverse_approximation
(funcname[, ...])Compute \(H\), the approximation of the inverse of the Hessian matrix.
compute_corrections
(x_hist, x_grad_hist)Compute the successive variations of both input variables and gradient.
compute_scaling
(hessk, hessk_dsk, ...)Compute the scaling coefficients \(c_1\) and \(c_2\).
get_s_k_y_k
(x_hist, x_grad_hist, iteration)Compute the variation of the input variables and gradient from an iteration.
get_x_grad_history
(funcname[, first_iter, ...])Return the histories of the inputs and gradient.
iterate_approximation
(hessk, dsk, dyk[, scaling])Update \(B\) from iteration \(k\) to iteration \(k+1\).
iterate_inverse_approximation
(h_mat, s_k, y_k)Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
iterate_s_k_y_k
(x_hist, x_grad_hist)Compute the variations of the input variables and gradient.
rebuild_history
(x_corr, x_0, grad_corr, g_0)Compute the history from the corrections of input variables and gradient.
 build_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, b_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, scaling=False, normalize_design_space=False, design_space=None)[source]¶
Compute \(B\), the approximation of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the Hessian matrix.
save_diag (bool) –
Whether to return the approximations of the Hessian’s diagonal.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
b_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the Hessian matrix.
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to be considered.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the Hessian in
b_mat_history
.By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(B\), the approximation of the Hessian matrix.
The diagonal of \(B\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 build_inverse_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, h_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, factorize=False, scaling=False, angle_tol=1e05, step_tol=10000000000.0, normalize_design_space=False, design_space=None)[source]¶
Compute \(H\), the approximation of the inverse of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the inverse of the Hessian matrix.
save_diag (int) –
Whether to return the list of diagonal approximations.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
h_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the inverse of the Hessian matrix. If None, use \(H_0=\frac{\Delta g_k^T\Delta x_k} {\Delta g_k^T\Delta g_k}I_d\).
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to take.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the inverse Hessian in
h_mat_history
.By default it is set to False.
factorize (bool) –
Whether to factorize the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
angle_tol (float) –
The significativity level for \(\Delta g_k^T\Delta x_k\).
By default it is set to 1e05.
step_tol (float) –
The significativity level for \(\\Delta g_k\_{\infty}\).
By default it is set to 10000000000.0.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(H\), the approximation of the inverse of the Hessian matrix.
The diagonal of \(H\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.The matrix \(H_{1/2}\) such that \(H=H_{1/2}H_{1/2}^T\) if
factorize
isTrue
.\(B\), the approximation of the Hessian matrix.
A matrix \(B_{1/2}\) such that \(B=B_{1/2}B_{1/2}^T\) if
factorize
isTrue
.
 Raises
LinAlgError – When either the inversion of \(H\) fails or the Cholesky decomposition of \(H\) or \(B\) fails.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 static compute_corrections(x_hist, x_grad_hist)[source]¶
Compute the successive variations of both input variables and gradient.
These variations are called corrections.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
 Returns
The successive variations of the input variables.
The successive variations of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 static compute_scaling(hessk, hessk_dsk, dskt_hessk_dsk, dyk, dyt_dsk)[source]¶
Compute the scaling coefficients \(c_1\) and \(c_2\).
\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
hessk_dsk (numpy.ndarray) – The product \(B_k\Delta x_k\).
dskt_hessk_dsk (numpy.ndarray) – The product \(\Delta x_k^T B_k\Delta x_k\).
dyk (numpy.ndarray) – The variation of the gradient \(\Delta g_k\).
dyt_dsk (numpy.ndarray) – The product \(\Delta g_k^T\Delta x_k\).
 Returns
TODO * coeff2: TODO
 Return type
coeff1
 static get_s_k_y_k(x_hist, x_grad_hist, iteration)[source]¶
Compute the variation of the input variables and gradient from an iteration.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
iteration (int) – The optimization iteration at which to compute the variations.
 Returns
The difference between the input variables at iteration
iteration+1
and the input variables at iterationiteration
.The difference between the gradient at iteration
iteration+1
and the gradient at iterationiteration
.
 Raises
ValueError – When the iteration is not stored in the database.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 get_x_grad_history(funcname, first_iter=0, last_iter=0, at_most_niter= 1, func_index=None, normalize_design_space=False, design_space=None)[source]¶
Return the histories of the inputs and gradient.
 Parameters
funcname (str) – The name of the function for which to get the gradient.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered. If 0, consider all the iterations.
By default it is set to 0.
at_most_niter (int) –
The maximum number of iterations to be considered. If 1, consider all the iterations.
By default it is set to 1.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
The history of the input variables.
The history of the gradient.
The length of the history.
The dimension of the input space.
 Raises
ValueError – When either the gradient history contains a single element,
func_index
isNone
while the function output is a vector,func_index
is not an output index, the shape of the history of the input variables is not consistent with the shape of the history of the gradient or the optimization history size is insufficient. Return type
Tuple[numpy.ndarray, numpy.ndarray, int, int]
 static iterate_approximation(hessk, dsk, dyk, scaling=False)[source]¶
Update \(B\) from iteration \(k\) to iteration \(k+1\).
Based on an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
hessk
represents \(B_k\) initially before to be overwritten by \(B_{k+1}\) when passed to this method.See also
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
dsk (numpy.ndarray) – The variation \(\Delta x_k\) of the input variables.
dyk (numpy.ndarray) – The variation \(\Delta g_k\) of the gradient.
scaling (bool) –
Whether to use a scaling stage.
By default it is set to False.
 Return type
None
 static iterate_inverse_approximation(h_mat, s_k, y_k, h_factor=None, b_mat=None, b_factor=None, factorize=False, scaling=False)[source]¶
Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
Use an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
and
\(H_{k+1}=c_1^{1}\Pi_{k+1}H_k\Pi_{k+1}^T +c_2^{1}\frac{\Delta x_k\Delta x_k^T}{\Delta g_k^T\Delta x_k}\)
where:
\(\Pi_{k+1}=I_d\frac{\Delta x_k\Delta g_k^T} {\Delta g_k^T\Delta x_k}\)
and where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
h_mat
andb_mat
represent \(H_k\) and \(B_k\) initially before to be overwritten by \(H_{k+1}\) and \(B_{k+1}\) when passed to this method.See also
 Parameters
h_mat – The approximation \(H_k\) of the inverse of the Hessian matrix at iteration \(k\).
s_k – The variation \(\Delta x_k\) of the input variables.
y_k – The variation \(\Delta g_k\) of the gradient.
h_factor –
The square root of the \(H_k\) at iteration \(k\).
By default it is set to None.
b_mat –
The approximation \(B_k\) of the Hessian matrix at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
b_factor –
The square root of the \(B_k\) at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
factorize –
Whether to update the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling –
do scaling step
By default it is set to False.
 static iterate_s_k_y_k(x_hist, x_grad_hist)[source]¶
Compute the variations of the input variables and gradient.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (ndarray) – The history of the input variables.
x_grad_hist (ndarray) – The history of the gradient.
 Returns
The difference between the input variables at iteration
iteration
and the input variables at iterationiteration+1
.The difference between the gradient at iteration
iteration
and the gradient at iterationiteration+1
.
 Return type
Generator[Tuple[ndarray, ndarray]]
 static rebuild_history(x_corr, x_0, grad_corr, g_0)[source]¶
Compute the history from the corrections of input variables and gradient.
A correction is the variation of a quantity between two successive iterations.
 Parameters
x_corr (numpy.ndarray) – The corrections of the input variables.
x_0 (numpy.ndarray) – The initial values of the input variables.
grad_corr (numpy.ndarray) – The corrections of the gradient.
g_0 (numpy.ndarray) – The initial value of the gradient.
 Returns
The history of the input variables.
The history of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 class gemseo.post.core.hessians.LSTSQApprox(history)[source]¶
Bases:
gemseo.post.core.hessians.HessianApproximation
Least squares approximation of an Hessian matrix from an optimization history.
 history¶
The optimization history containing input values, output values and Jacobian values.
 Type
 x_ref¶
The value \(x_K\) of the input variables \(x\) at the iteration \(K\) of the optimization history; this is the point at which the Hessian matrix and its inverse are approximated.
 Type
Optional[ndarray]
 fgrad_ref¶
The value \(g_K\) of the gradient function \(g\) of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 f_ref¶
The value \(y_K\) of the output of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 b_mat_history¶
The history \(B_0,B_1,\ldots,B_K\) of the approximations of the Hessian matrix \(B\).
 Type
List[ndarray]
 h_mat_history¶
The history \(H_0,H_1,\ldots,H_K\) of the approximations of the inverse Hessian matrix \(H\).
 Type
List[ndarray]
Initialize self. See help(type(self)) for accurate signature.
 Parameters
history (Database) – The optimization history containing input values, output values and Jacobian values.
 Return type
None
Methods:
build_approximation
(funcname[, save_diag, ...])Compute \(B\), the approximation of the Hessian matrix.
build_inverse_approximation
(funcname[, ...])Compute \(H\), the approximation of the inverse of the Hessian matrix.
compute_corrections
(x_hist, x_grad_hist)Compute the successive variations of both input variables and gradient.
compute_scaling
(hessk, hessk_dsk, ...)Compute the scaling coefficients \(c_1\) and \(c_2\).
get_s_k_y_k
(x_hist, x_grad_hist, iteration)Compute the variation of the input variables and gradient from an iteration.
get_x_grad_history
(funcname[, first_iter, ...])Return the histories of the inputs and gradient.
iterate_approximation
(hessk, dsk, dyk[, scaling])Update \(B\) from iteration \(k\) to iteration \(k+1\).
iterate_inverse_approximation
(h_mat, s_k, y_k)Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
iterate_s_k_y_k
(x_hist, x_grad_hist)Compute the variations of the input variables and gradient.
rebuild_history
(x_corr, x_0, grad_corr, g_0)Compute the history from the corrections of input variables and gradient.
 build_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, b_mat0=None, at_most_niter= 1, return_x_grad=False, scaling=False, func_index= 1, normalize_design_space=False, design_space=None)[source]¶
Compute \(B\), the approximation of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the Hessian matrix.
save_diag (bool) –
Whether to return the approximations of the Hessian’s diagonal.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
b_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the Hessian matrix.
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to be considered.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
func_index (int) –
The output index of the function to be provided if the function output is a vector.
By default it is set to 1.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(B\), the approximation of the Hessian matrix.
The diagonal of \(B\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 build_inverse_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, h_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, factorize=False, scaling=False, angle_tol=1e05, step_tol=10000000000.0, normalize_design_space=False, design_space=None)¶
Compute \(H\), the approximation of the inverse of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the inverse of the Hessian matrix.
save_diag (int) –
Whether to return the list of diagonal approximations.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
h_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the inverse of the Hessian matrix. If None, use \(H_0=\frac{\Delta g_k^T\Delta x_k} {\Delta g_k^T\Delta g_k}I_d\).
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to take.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the inverse Hessian in
h_mat_history
.By default it is set to False.
factorize (bool) –
Whether to factorize the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
angle_tol (float) –
The significativity level for \(\Delta g_k^T\Delta x_k\).
By default it is set to 1e05.
step_tol (float) –
The significativity level for \(\\Delta g_k\_{\infty}\).
By default it is set to 10000000000.0.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(H\), the approximation of the inverse of the Hessian matrix.
The diagonal of \(H\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.The matrix \(H_{1/2}\) such that \(H=H_{1/2}H_{1/2}^T\) if
factorize
isTrue
.\(B\), the approximation of the Hessian matrix.
A matrix \(B_{1/2}\) such that \(B=B_{1/2}B_{1/2}^T\) if
factorize
isTrue
.
 Raises
LinAlgError – When either the inversion of \(H\) fails or the Cholesky decomposition of \(H\) or \(B\) fails.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 static compute_corrections(x_hist, x_grad_hist)¶
Compute the successive variations of both input variables and gradient.
These variations are called corrections.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
 Returns
The successive variations of the input variables.
The successive variations of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 static compute_scaling(hessk, hessk_dsk, dskt_hessk_dsk, dyk, dyt_dsk)¶
Compute the scaling coefficients \(c_1\) and \(c_2\).
\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
hessk_dsk (numpy.ndarray) – The product \(B_k\Delta x_k\).
dskt_hessk_dsk (numpy.ndarray) – The product \(\Delta x_k^T B_k\Delta x_k\).
dyk (numpy.ndarray) – The variation of the gradient \(\Delta g_k\).
dyt_dsk (numpy.ndarray) – The product \(\Delta g_k^T\Delta x_k\).
 Returns
TODO * coeff2: TODO
 Return type
coeff1
 static get_s_k_y_k(x_hist, x_grad_hist, iteration)¶
Compute the variation of the input variables and gradient from an iteration.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
iteration (int) – The optimization iteration at which to compute the variations.
 Returns
The difference between the input variables at iteration
iteration+1
and the input variables at iterationiteration
.The difference between the gradient at iteration
iteration+1
and the gradient at iterationiteration
.
 Raises
ValueError – When the iteration is not stored in the database.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 get_x_grad_history(funcname, first_iter=0, last_iter=0, at_most_niter= 1, func_index=None, normalize_design_space=False, design_space=None)¶
Return the histories of the inputs and gradient.
 Parameters
funcname (str) – The name of the function for which to get the gradient.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered. If 0, consider all the iterations.
By default it is set to 0.
at_most_niter (int) –
The maximum number of iterations to be considered. If 1, consider all the iterations.
By default it is set to 1.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
The history of the input variables.
The history of the gradient.
The length of the history.
The dimension of the input space.
 Raises
ValueError – When either the gradient history contains a single element,
func_index
isNone
while the function output is a vector,func_index
is not an output index, the shape of the history of the input variables is not consistent with the shape of the history of the gradient or the optimization history size is insufficient. Return type
Tuple[numpy.ndarray, numpy.ndarray, int, int]
 static iterate_approximation(hessk, dsk, dyk, scaling=False)¶
Update \(B\) from iteration \(k\) to iteration \(k+1\).
Based on an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
hessk
represents \(B_k\) initially before to be overwritten by \(B_{k+1}\) when passed to this method.See also
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
dsk (numpy.ndarray) – The variation \(\Delta x_k\) of the input variables.
dyk (numpy.ndarray) – The variation \(\Delta g_k\) of the gradient.
scaling (bool) –
Whether to use a scaling stage.
By default it is set to False.
 Return type
None
 static iterate_inverse_approximation(h_mat, s_k, y_k, h_factor=None, b_mat=None, b_factor=None, factorize=False, scaling=False)¶
Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
Use an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
and
\(H_{k+1}=c_1^{1}\Pi_{k+1}H_k\Pi_{k+1}^T +c_2^{1}\frac{\Delta x_k\Delta x_k^T}{\Delta g_k^T\Delta x_k}\)
where:
\(\Pi_{k+1}=I_d\frac{\Delta x_k\Delta g_k^T} {\Delta g_k^T\Delta x_k}\)
and where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
h_mat
andb_mat
represent \(H_k\) and \(B_k\) initially before to be overwritten by \(H_{k+1}\) and \(B_{k+1}\) when passed to this method.See also
 Parameters
h_mat – The approximation \(H_k\) of the inverse of the Hessian matrix at iteration \(k\).
s_k – The variation \(\Delta x_k\) of the input variables.
y_k – The variation \(\Delta g_k\) of the gradient.
h_factor –
The square root of the \(H_k\) at iteration \(k\).
By default it is set to None.
b_mat –
The approximation \(B_k\) of the Hessian matrix at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
b_factor –
The square root of the \(B_k\) at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
factorize –
Whether to update the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling –
do scaling step
By default it is set to False.
 static iterate_s_k_y_k(x_hist, x_grad_hist)¶
Compute the variations of the input variables and gradient.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (ndarray) – The history of the input variables.
x_grad_hist (ndarray) – The history of the gradient.
 Returns
The difference between the input variables at iteration
iteration
and the input variables at iterationiteration+1
.The difference between the gradient at iteration
iteration
and the gradient at iterationiteration+1
.
 Return type
Generator[Tuple[ndarray, ndarray]]
 static rebuild_history(x_corr, x_0, grad_corr, g_0)¶
Compute the history from the corrections of input variables and gradient.
A correction is the variation of a quantity between two successive iterations.
 Parameters
x_corr (numpy.ndarray) – The corrections of the input variables.
x_0 (numpy.ndarray) – The initial values of the input variables.
grad_corr (numpy.ndarray) – The corrections of the gradient.
g_0 (numpy.ndarray) – The initial value of the gradient.
 Returns
The history of the input variables.
The history of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 class gemseo.post.core.hessians.SR1Approx(history)[source]¶
Bases:
gemseo.post.core.hessians.HessianApproximation
Hessian matrix approximation with the Symmetric Rank One (SR1) algorithm.
The approximation at iteration \(k+1\) is:
\[B_{k+1}=B_k + \frac{(\Delta g_kB_k\Delta x_k)(\Delta g_kB_k\Delta x_k)^T} {(\Delta g_kB_k\Delta x_k)^T\Delta x_k}\]This update from iteration \(k\) to iteration \(k+1\) is applied only if \((\Delta g_kB_k\Delta x_k)^T\Delta x_k \geq \varepsilon\\Delta x_k\\\Delta g_k\\) where \(\varepsilon\) is a small number, e.g. \(10^{8}\).
See also
 history¶
The optimization history containing input values, output values and Jacobian values.
 Type
 x_ref¶
The value \(x_K\) of the input variables \(x\) at the iteration \(K\) of the optimization history; this is the point at which the Hessian matrix and its inverse are approximated.
 Type
Optional[ndarray]
 fgrad_ref¶
The value \(g_K\) of the gradient function \(g\) of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 f_ref¶
The value \(y_K\) of the output of \(f\) at \(x_K\).
 Type
Optional[ndarray]
 b_mat_history¶
The history \(B_0,B_1,\ldots,B_K\) of the approximations of the Hessian matrix \(B\).
 Type
List[ndarray]
 h_mat_history¶
The history \(H_0,H_1,\ldots,H_K\) of the approximations of the inverse Hessian matrix \(H\).
 Type
List[ndarray]
Initialize self. See help(type(self)) for accurate signature.
 Parameters
history (Database) – The optimization history containing input values, output values and Jacobian values.
 Return type
None
Attributes:
Methods:
build_approximation
(funcname[, save_diag, ...])Compute \(B\), the approximation of the Hessian matrix.
build_inverse_approximation
(funcname[, ...])Compute \(H\), the approximation of the inverse of the Hessian matrix.
compute_corrections
(x_hist, x_grad_hist)Compute the successive variations of both input variables and gradient.
compute_scaling
(hessk, hessk_dsk, ...)Compute the scaling coefficients \(c_1\) and \(c_2\).
get_s_k_y_k
(x_hist, x_grad_hist, iteration)Compute the variation of the input variables and gradient from an iteration.
get_x_grad_history
(funcname[, first_iter, ...])Return the histories of the inputs and gradient.
iterate_approximation
(b_mat, s_k, y_k[, scaling])Update \(B\) from iteration \(k\) to iteration \(k+1\).
iterate_inverse_approximation
(h_mat, s_k, y_k)Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
iterate_s_k_y_k
(x_hist, x_grad_hist)Compute the variations of the input variables and gradient.
rebuild_history
(x_corr, x_0, grad_corr, g_0)Compute the history from the corrections of input variables and gradient.
 EPSILON = 1e08¶
 build_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, b_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, scaling=False, normalize_design_space=False, design_space=None)¶
Compute \(B\), the approximation of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the Hessian matrix.
save_diag (bool) –
Whether to return the approximations of the Hessian’s diagonal.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
b_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the Hessian matrix.
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to be considered.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the Hessian in
b_mat_history
.By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(B\), the approximation of the Hessian matrix.
The diagonal of \(B\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 build_inverse_approximation(funcname, save_diag=False, first_iter=0, last_iter= 1, h_mat0=None, at_most_niter= 1, return_x_grad=False, func_index=None, save_matrix=False, factorize=False, scaling=False, angle_tol=1e05, step_tol=10000000000.0, normalize_design_space=False, design_space=None)¶
Compute \(H\), the approximation of the inverse of the Hessian matrix.
 Parameters
funcname (str) – The name of the function for which to approximate the inverse of the Hessian matrix.
save_diag (int) –
Whether to return the list of diagonal approximations.
By default it is set to False.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered.
By default it is set to 1.
h_mat0 (Optional[numpy.ndarray]) –
The initial approximation of the inverse of the Hessian matrix. If None, use \(H_0=\frac{\Delta g_k^T\Delta x_k} {\Delta g_k^T\Delta g_k}I_d\).
By default it is set to None.
at_most_niter (int) –
The maximum number of iterations to take.
By default it is set to 1.
return_x_grad (bool) –
Whether to return the input variables and gradient at the last iteration.
By default it is set to False.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
save_matrix (bool) –
Whether to store the approximations of the inverse Hessian in
h_mat_history
.By default it is set to False.
factorize (bool) –
Whether to factorize the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling (bool) –
do scaling step
By default it is set to False.
angle_tol (float) –
The significativity level for \(\Delta g_k^T\Delta x_k\).
By default it is set to 1e05.
step_tol (float) –
The significativity level for \(\\Delta g_k\_{\infty}\).
By default it is set to 10000000000.0.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
\(H\), the approximation of the inverse of the Hessian matrix.
The diagonal of \(H\).
The history of the input variables if
return_x_grad
isTrue
.The history of the gradient if
return_x_grad
isTrue
.The matrix \(H_{1/2}\) such that \(H=H_{1/2}H_{1/2}^T\) if
factorize
isTrue
.\(B\), the approximation of the Hessian matrix.
A matrix \(B_{1/2}\) such that \(B=B_{1/2}B_{1/2}^T\) if
factorize
isTrue
.
 Raises
LinAlgError – When either the inversion of \(H\) fails or the Cholesky decomposition of \(H\) or \(B\) fails.
 Return type
Tuple[numpy.ndarray, numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray]]
 static compute_corrections(x_hist, x_grad_hist)¶
Compute the successive variations of both input variables and gradient.
These variations are called corrections.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
 Returns
The successive variations of the input variables.
The successive variations of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 static compute_scaling(hessk, hessk_dsk, dskt_hessk_dsk, dyk, dyt_dsk)¶
Compute the scaling coefficients \(c_1\) and \(c_2\).
\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
 Parameters
hessk (numpy.ndarray) – The approximation \(B_k\) of the Hessian matrix at iteration \(k\).
hessk_dsk (numpy.ndarray) – The product \(B_k\Delta x_k\).
dskt_hessk_dsk (numpy.ndarray) – The product \(\Delta x_k^T B_k\Delta x_k\).
dyk (numpy.ndarray) – The variation of the gradient \(\Delta g_k\).
dyt_dsk (numpy.ndarray) – The product \(\Delta g_k^T\Delta x_k\).
 Returns
TODO * coeff2: TODO
 Return type
coeff1
 static get_s_k_y_k(x_hist, x_grad_hist, iteration)¶
Compute the variation of the input variables and gradient from an iteration.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (numpy.ndarray) – The history of the input variables.
x_grad_hist (numpy.ndarray) – The history of the gradient.
iteration (int) – The optimization iteration at which to compute the variations.
 Returns
The difference between the input variables at iteration
iteration+1
and the input variables at iterationiteration
.The difference between the gradient at iteration
iteration+1
and the gradient at iterationiteration
.
 Raises
ValueError – When the iteration is not stored in the database.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]
 get_x_grad_history(funcname, first_iter=0, last_iter=0, at_most_niter= 1, func_index=None, normalize_design_space=False, design_space=None)¶
Return the histories of the inputs and gradient.
 Parameters
funcname (str) – The name of the function for which to get the gradient.
first_iter (int) –
The first iteration of the history to be considered.
By default it is set to 0.
last_iter (int) –
The last iteration of the history to be considered. If 0, consider all the iterations.
By default it is set to 0.
at_most_niter (int) –
The maximum number of iterations to be considered. If 1, consider all the iterations.
By default it is set to 1.
func_index (Optional[int]) –
The output index of the function to be provided if the function output is a vector.
By default it is set to None.
normalize_design_space (bool) –
Whether to scale the input values between 0 and 1 to work in a normalized input space.
By default it is set to False.
design_space (Optional[gemseo.algos.design_space.DesignSpace]) –
The input space used to scale the input values if
normalize_design_space
isTrue
.By default it is set to None.
 Returns
The history of the input variables.
The history of the gradient.
The length of the history.
The dimension of the input space.
 Raises
ValueError – When either the gradient history contains a single element,
func_index
isNone
while the function output is a vector,func_index
is not an output index, the shape of the history of the input variables is not consistent with the shape of the history of the gradient or the optimization history size is insufficient. Return type
Tuple[numpy.ndarray, numpy.ndarray, int, int]
 static iterate_approximation(b_mat, s_k, y_k, scaling=False)[source]¶
Update \(B\) from iteration \(k\) to iteration \(k+1\).
Based on an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
hessk
represents \(B_k\) initially before to be overwritten by \(B_{k+1}\) when passed to this method.See also
 Parameters
scaling –
Whether to use a scaling stage.
By default it is set to False.
 static iterate_inverse_approximation(h_mat, s_k, y_k, h_factor=None, b_mat=None, b_factor=None, factorize=False, scaling=False)¶
Update \(H\) and \(B\) from step \(k\) to step \(k+1\).
Use an iteration of the BFGS algorithm:
\(B_{k+1} = B_k  c_1\frac{B_k\Delta x_k\Delta x_k^TB_k}{\Delta x_k^TB_k\Delta x_k} + c_2\frac{\Delta g_k\Delta g_k^T}{\Delta g_k^T\Delta x_k}\)
and
\(H_{k+1}=c_1^{1}\Pi_{k+1}H_k\Pi_{k+1}^T +c_2^{1}\frac{\Delta x_k\Delta x_k^T}{\Delta g_k^T\Delta x_k}\)
where:
\(\Pi_{k+1}=I_d\frac{\Delta x_k\Delta g_k^T} {\Delta g_k^T\Delta x_k}\)
and where \(c_1=c_2=1\) if
scaling
isFalse
, otherwise:\(c_1=\frac{d1}{\mathrm{Tr}(B_k)\frac{\B_k\Delta x_k\_2^2} {\Delta x_k^T B_k\Delta x_k}}\),
\(c_2=\frac{\Delta g_k^T\Delta x_k}{\\Delta g_k\_2^2}\).
Note
h_mat
andb_mat
represent \(H_k\) and \(B_k\) initially before to be overwritten by \(H_{k+1}\) and \(B_{k+1}\) when passed to this method.See also
 Parameters
h_mat – The approximation \(H_k\) of the inverse of the Hessian matrix at iteration \(k\).
s_k – The variation \(\Delta x_k\) of the input variables.
y_k – The variation \(\Delta g_k\) of the gradient.
h_factor –
The square root of the \(H_k\) at iteration \(k\).
By default it is set to None.
b_mat –
The approximation \(B_k\) of the Hessian matrix at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
b_factor –
The square root of the \(B_k\) at iteration \(k\) if
factorize
isTrue
.By default it is set to None.
factorize –
Whether to update the approximations of the Hessian matrix and its inverse, as \(A=A_{1/2}A_{1/2}^T\) for a matrix \(A\).
By default it is set to False.
scaling –
do scaling step
By default it is set to False.
 static iterate_s_k_y_k(x_hist, x_grad_hist)¶
Compute the variations of the input variables and gradient.
The variations from the iteration \(k\) are defined by:
\(\Delta x_k = x_{k+1}x_k\) for the input variables,
\(\Delta g_k = g_{k+1}  g_k\) for the gradient.
 Parameters
x_hist (ndarray) – The history of the input variables.
x_grad_hist (ndarray) – The history of the gradient.
 Returns
The difference between the input variables at iteration
iteration
and the input variables at iterationiteration+1
.The difference between the gradient at iteration
iteration
and the gradient at iterationiteration+1
.
 Return type
Generator[Tuple[ndarray, ndarray]]
 static rebuild_history(x_corr, x_0, grad_corr, g_0)¶
Compute the history from the corrections of input variables and gradient.
A correction is the variation of a quantity between two successive iterations.
 Parameters
x_corr (numpy.ndarray) – The corrections of the input variables.
x_0 (numpy.ndarray) – The initial values of the input variables.
grad_corr (numpy.ndarray) – The corrections of the gradient.
g_0 (numpy.ndarray) – The initial value of the gradient.
 Returns
The history of the input variables.
The history of the gradient.
 Return type
Tuple[numpy.ndarray, numpy.ndarray]