gemseo_benchmark / data_profiles

Hide inherited members

data_profile module

Class to compute data profiles for algorithms comparison.

A data profile is a graphical tool to compare iterative algorithms, e.g. optimization algorithms or root-finding algorithms, on reference problems.

Each of the reference problems must be assigned targets, i.e. values of the objective function or values of the residual norm, ranging from a first acceptable value to the best known value for the problem.

The algorithms will be compared based on the number of targets they reach, cumulated over all the reference problems, relative to the number of problems functions evaluations they make.

The data profile is the empirical cumulated distribution function of the number of functions evaluations made by an algorithm to reach a problem target.

class gemseo_benchmark.data_profiles.data_profile.DataProfile(target_values)

Bases: object

Data profile that compares iterative algorithms on reference problems.

A data profile is an empirical cumulative distribution function of the number of problems functions evaluations required by an algorithm to reach a reference problem target.

Parameters:

target_values (Mapping[str, TargetValues]) – The target values of each of the reference problems.

add_history(problem_name, algorithm_configuration_name, objective_values, infeasibility_measures=None, feasibility_statuses=None)

Add a history of performance values.

Parameters:

• problem_name (str) – The name of the problem.

• algorithm_configuration_name (str) – The name of the algorithm configuration.

• objective_values (Sequence[float]) – A history of objective values. N.B. the value at index i is assumed to have been obtained with i+1 evaluations.

• infeasibility_measures (Sequence[float] | None) – A history of infeasibility measures. If None then measures are set to zero in case of feasibility and set to infinity otherwise.

• feasibility_statuses (Sequence[bool] | None) – A history of (boolean) feasibility statuses. If None then feasibility is always assumed.

Raises:

ValueError – If the problem name is not the name of a reference problem.

Return type:

None

compute_data_profiles(*algo_names)

Compute the data profiles of the required algorithms.

For each algorithm, compute the cumulative distribution function of the number of evaluations required by the algorithm to reach a reference target.

Parameters:

algo_names (str) – The names of the algorithms. If None then all the algorithms are considered.

Returns:

The data profiles.

Return type:

dict[str, list[Number]]

plot(algo_names=None, show=True, file_path=None, markevery=0.1)

Plot the data profiles of the required algorithms.

Parameters:

• algo_names (Iterable[str] | None) – The names of the algorithms. If None then all the algorithms are considered.

• show (bool) –

  If True, show the plot.

  By default it is set to True.

• file_path (str | Path | None) – The path where to save the plot. If None, the plot is not saved.

• markevery (MarkeveryType) –

  The sampling parameter for the markers of the plot. Refer to the Matplotlib documentation.

  By default it is set to 0.1.

Return type:

None

property target_values: dict[str, TargetValues]

The target values of each reference problem.

Target values are a scale of objective function values, ranging from an easily achievable one to the best known value. A data profile is computed by counting the number of targets reached by an algorithm at each iteration.

Raises:

• TypeError – if the target values are not passed as a dictionary.

• ValueError – If the reference problems have different numbers of target values.