calibration module¶
Calibration of a machine learning algorithm¶
A machine learning algorithm depends on hyperparameters, e.g. number of clusters for a clustering algorithm, regularization constant for a regression model, kernel for a Gaussian process regression, … Its quality of generalization depends on the values of these hyperparameters. Thus, the hyperparameters minimizing the learning quality measure are rarely those minimizing the generalization one. Classically, the generalization one decreases before growing again as the model becomes more complex, while the learning error keeps decreasing. This phenomenon is called the curse of dimensionality.
In this module, the MLAlgoCalibration class aims to calibrate the
hyperparameters in order to minimize this generalization quality measure
over a calibration parameter space. This class relies on the
MLAlgoAssessor class which is a discipline
(MDODiscipline)
built from a machine learning algorithm (MLAlgo),
a dataset (Dataset), a quality measure (MLQualityMeasure)
and various options for data scaling, quality measure
and machine learning algorithm. The inputs of this discipline are
hyperparameters of the machine learning algorithm while the output is
the quality criterion.
-
class
gemseo.mlearning.core.calibration.MLAlgoAssessor(algo, dataset, parameters, measure, measure_options=None, transformer=None, **algo_options)[source]¶ Bases:
gemseo.core.discipline.MDODisciplineDiscipline assessing the quality of a machine learning algorithm
Constructor
- Parameters
algo (str) – machine learning algorithm name.
dataset (Dataset) – learning dataset.
parameters (list(str)) – parameters.
measure (MLQualityMeasure) – quality measure.
measure_options (dict) – options of the quality measures.
transformer (dict(str)) – transformation strategy for data groups. If None, do not transform data. Default: None.
algo_options – options of the machine learning algorithm.
-
class
gemseo.mlearning.core.calibration.MLAlgoCalibration(algo, dataset, parameters, calibration_space, measure, measure_options=None, transformer=None, use_doe=True, **algo_options)[source]¶ Bases:
objectCalibration of a machine learning algorithm
Constructor
- Parameters
algo (str) – machine learning algorithm name.
dataset (Dataset) – learning dataset.
parameters (list(str)) – parameters.
calibration_space (DesignSpace) – calibration space.
measure (MLQualityMeasure) – quality measure.
measure_options (dict) – options of the quality measures.
transformer (dict(str)) – transformation strategy for data groups. If None, do not transform data. Default: None.
use_doe (bool) – if True, use a DOEScenario to calibrate the ML algorithm. Otherwise, use a MDOScenario. Default: True.
algo_options – options of the machine learning algorithm.
-
CRITERION= 'criterion'¶
-
property
algos¶ List of trained algorithms.