# Gaussian process regression¶

## Overview¶

The Gaussian process regression (GPR) surrogate discipline expresses the model output as a weighted sum of kernel functions centered on the learning input data:

$y = \mu + w_1\kappa(\|x-x_1\|;\epsilon) + w_2\kappa(\|x-x_2\|;\epsilon) + ... + w_N\kappa(\|x-x_N\|;\epsilon)$

## Details¶

The GPR model relies on the assumption that the original model $$f$$ to replace is an instance of a Gaussian process (GP) with mean $$\mu$$ and covariance $$\sigma^2\kappa(\|x-x'|;\epsilon)$$.

Then, the GP conditioned by the learning set $$(x_i,y_i)_{1\leq i \leq N}$$ is entirely defined by its expectation:

$\hat{f}(x) = \hat{\mu} + w^T k(x)$

and its covariance:

$\hat{c}(x,x') = \hat{\sigma}^2 - k(x)^T K^{-1} k(x')$

where $$[\hat{\mu};\hat{w}]=([1_N~K]^T[1_N~K])^{-1}[1_N~K]^TY$$ with $$K_{ij}=\kappa(\|x_i-x_j\|;\hat{\epsilon})$$, $$k_i(x)=\kappa(\|x-x_i\|;\hat{\epsilon})$$ and $$Y_i=y_i$$.

The correlation length vector $$\epsilon$$ is estimated by numerical non-linear optimization.

## Surrogate model¶

The expectation $$\hat{f}$$ is the GPR surrogate model of $$f$$.

## Error measure¶

The standard deviation $$\hat{s}$$ is a local error measure of $$\hat{f}$$:

$\hat{s}(x):=\sqrt{c(x,x)}$

## Interpolation or regression¶

The GPR surrogate discipline can be regressive or interpolative according to the value of the nugget effect $$\\alpha\geq 0$$ which is a regularization term applied to the correlation matrix $$K$$. When $$\\alpha = 0$$, the surrogate model interpolates the learning data.

## Dependence¶

The GPR model relies on the GaussianProcessRegressor class of the scikit-learn library.

class gemseo.mlearning.regression.gpr.GaussianProcessRegression(data, transformer=None, input_names=None, output_names=None, kernel=None, alpha=1e-10, optimizer='fmin_l_bfgs_b', n_restarts_optimizer=10, random_state=None)[source]

Gaussian process regression

Constructor.

Parameters
• data (Dataset) – learning dataset

• transformer (dict(str)) – transformation strategy for data groups. If None, do not transform data. Default: None.

• input_names (list(str)) – names of the input variables.

• output_names (list(str)) – names of the output variables.

• kernel (openturns.Kernel) – kernel function. If None, use a Matern(2.5). Default: None.

• alpha (float or array) – nugget effect. Default: 1e-10.

• optimizer (str or callable) – optimization algorithm. Default: ‘fmin_l_bfgs_b’.

• n_restarts_optimizer (int) – number of restarts of the optimizer. Default: 10.

• random_state (int) – the seed used to initialize the centers. If None, the random number generator is the RandomState instance used by np.random Default: None.