mse_measure module¶
The mean squared error to measure the quality of a regression algorithm.
The mse_measure
module
implements the concept of mean squared error measures
for machine learning algorithms.
This concept is implemented through the MSEMeasure
class
and overloads the MLErrorMeasure._compute_measure()
method.
The mean squared error (MSE) is defined by
where \(\hat{y}\) are the predictions and \(y\) are the data points.
- class gemseo.mlearning.quality_measures.mse_measure.MSEMeasure(algo, fit_transformers=True)[source]¶
Bases:
MLErrorMeasure
The Mean Squared Error measure for machine learning.
- Parameters:
algo (MLRegressionAlgo) – A machine learning algorithm for regression.
fit_transformers (bool) –
Whether to re-fit the transformers when using resampling techniques. If
False
, use the transformers of the algorithm fitted from the whole learning dataset.By default it is set to True.
- class EvaluationFunctionName(value)¶
Bases:
StrEnum
The name of the function associated with an evaluation method.
- BOOTSTRAP = 'evaluate_bootstrap'¶
- KFOLDS = 'evaluate_kfolds'¶
- LEARN = 'evaluate_learn'¶
- LOO = 'evaluate_loo'¶
- TEST = 'evaluate_test'¶
- class EvaluationMethod(value)¶
Bases:
StrEnum
The evaluation method.
- BOOTSTRAP = 'BOOTSTRAP'¶
The name of the method to evaluate the measure by bootstrap.
- KFOLDS = 'KFOLDS'¶
The name of the method to evaluate the measure by cross-validation.
- LEARN = 'LEARN'¶
The name of the method to evaluate the measure on the learning dataset.
- LOO = 'LOO'¶
The name of the method to evaluate the measure by leave-one-out.
- TEST = 'TEST'¶
The name of the method to evaluate the measure on a test dataset.
- compute_bootstrap_measure(n_replicates=100, samples=None, multioutput=True, seed=None, as_dict=False)¶
Evaluate the quality measure using the bootstrap technique.
- Parameters:
n_replicates (int) –
The number of bootstrap replicates.
By default it is set to 100.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
seed (None | None) – The seed of the pseudo-random number generator. If
None
, an unpredictable generator will be used.as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- compute_cross_validation_measure(n_folds=5, samples=None, multioutput=True, randomize=True, seed=None, as_dict=False)¶
- as_dict: Whether the full quality measure is returned
as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.
- Parameters:
n_folds (int) –
The number of folds.
By default it is set to 5.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
randomize (bool) –
Whether to shuffle the samples before dividing them in folds.
By default it is set to True.
seed (int | None) – The seed of the pseudo-random number generator. If
None
, an unpredictable generator is used.as_dict (bool) –
The description is missing.
By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- compute_learning_measure(samples=None, multioutput=True, as_dict=False)¶
Evaluate the quality measure from the learning dataset.
- Parameters:
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- compute_leave_one_out_measure(samples=None, multioutput=True, as_dict=False)¶
- as_dict: Whether the full quality measure is returned
as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.
- Parameters:
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
The description is missing.
By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- compute_test_measure(test_data, samples=None, multioutput=True, as_dict=False)¶
Evaluate the quality measure using a test dataset.
- Parameters:
test_data (IODataset) – The test dataset.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- evaluate_bootstrap(n_replicates=100, samples=None, multioutput=True, seed=None, as_dict=False)¶
Evaluate the quality measure using the bootstrap technique.
- Parameters:
n_replicates (int) –
The number of bootstrap replicates.
By default it is set to 100.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
seed (None | None) – The seed of the pseudo-random number generator. If
None
, an unpredictable generator will be used.as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- evaluate_kfolds(n_folds=5, samples=None, multioutput=True, randomize=True, seed=None, as_dict=False)¶
- as_dict: Whether the full quality measure is returned
as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.
- Parameters:
n_folds (int) –
The number of folds.
By default it is set to 5.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
randomize (bool) –
Whether to shuffle the samples before dividing them in folds.
By default it is set to True.
seed (int | None) – The seed of the pseudo-random number generator. If
None
, an unpredictable generator is used.as_dict (bool) –
The description is missing.
By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- evaluate_learn(samples=None, multioutput=True, as_dict=False)¶
Evaluate the quality measure from the learning dataset.
- Parameters:
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- evaluate_loo(samples=None, multioutput=True, as_dict=False)¶
- as_dict: Whether the full quality measure is returned
as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.
- Parameters:
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
The description is missing.
By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- evaluate_test(test_data, samples=None, multioutput=True, as_dict=False)¶
Evaluate the quality measure using a test dataset.
- Parameters:
test_data (IODataset) – The test dataset.
samples (Sequence[int] | None) – The indices of the learning samples. If
None
, use the whole learning dataset.multioutput (bool) –
Whether the quality measure is returned for each component of the outputs. Otherwise, the average quality measure.
By default it is set to True.
as_dict (bool) –
Whether the full quality measure is returned as a mapping from
algo.output names
to quality measures. Otherwise, the full quality measure as an array stacking these quality measures according to the order ofalgo.output_names
.By default it is set to False.
- Returns:
The value of the quality measure.
- Return type:
MeasureType
- classmethod is_better(val1, val2)¶
Compare the quality between two values.
This method returns
True
if the first one is better than the second one.For most measures, a smaller value is “better” than a larger one (MSE etc.). But for some, like an R2-measure, higher values are better than smaller ones. This comparison method correctly handles this, regardless of the type of measure.
Examples using MSEMeasure¶
Calibration of a polynomial regression
Machine learning algorithm selection example
MSE example - test-train split
Quality measure for surrogate model comparison