Calibrate a machine learning algorithm

Calibration of a machine learning algorithm

A machine learning algorithm depends on hyperparameters, e.g. number of clusters for a clustering algorithm, regularization constant for a regression model, kernel for a Gaussian process regression, … Its quality of generalization depends on the values of these hyperparameters. Thus, the hyperparameters minimizing the learning quality measure are rarely those minimizing the generalization one. Classically, the generalization one decreases before growing again as the model becomes more complex, while the learning error keeps decreasing. This phenomenon is called the curse of dimensionality.

In this module, the MLAlgoCalibration class aims to calibrate the hyperparameters in order to minimize this generalization quality measure over a calibration parameter space. This class relies on the MLAlgoAssessor class which is a discipline (MDODiscipline) built from a machine learning algorithm (MLAlgo), a dataset (Dataset), a quality measure (MLQualityMeasure) and various options for data scaling, quality measure and machine learning algorithm. The inputs of this discipline are hyperparameters of the machine learning algorithm while the output is the quality criterion.