gemseo / core / parallel_execution

Hide inherited members

disc_parallel_linearization module

Parallel execution of linearized disciplines.

class gemseo.core.parallel_execution.disc_parallel_linearization.DiscParallelLinearization(disciplines, n_processes=2, use_threading=False, wait_time_between_fork=0.0, exceptions_to_re_raise=(), execute=True)[source]

Bases: CallableParallelExecution

Linearize disciplines in parallel.

  • disciplines (Sequence[MDODiscipline]) – The disciplines to execute.

  • n_processes (int) –

    The maximum simultaneous number of threads, if use_threading is True, or processes otherwise, used to parallelize the execution.

    By default it is set to 2.

  • use_threading (bool) –

    Whether to use threads instead of processes to parallelize the execution. Multiprocessing will copy (serialize) all the disciplines, while threading will share all the memory. This is important to note if you want to execute the same discipline multiple times, in which case you shall use multiprocessing.

    By default it is set to False.

  • wait_time_between_fork (float) –

    The time to wait between two forks of the process/thread.

    By default it is set to 0.0.

  • exceptions_to_re_raise (tuple[type[Exception]]) –

    The exceptions that should be raised again when caught inside a worker. If None, all exceptions coming from workers are caught and the execution is allowed to continue.

    By default it is set to ().

  • execute (bool) –

    Whether to start by executing the discipline with the input data for which to compute the Jacobian; this allows to ensure that the discipline was executed with the right input data; it can be almost free if the corresponding output data have been stored in the cache.

    By default it is set to True.


ValueError – If there are duplicated workers in workers when using multithreading.

class MultiProcessingStartMethod(value)

Bases: StrEnum

The multiprocessing start method.

FORK = 'fork'
FORKSERVER = 'forkserver'
SPAWN = 'spawn'
execute(inputs, exec_callback=None, task_submitted_callback=None)[source]

Execute all the processes.

  • inputs (Sequence[Data | None]) – The input values.

  • exec_callback (Callable[[int, Any], Any] | None) – A callback called with the pair (index, outputs) as arguments when an item is retrieved from the processing. Index is the associated index in inputs of the input used to compute the outputs. If None, no function is called.

  • task_submitted_callback (Callable | None) – A callback function called when all the tasks are submitted, but not done yet. If None, no function is called.


The computed outputs.


TypeError – If the exec_callback is not callable. If the task_submitted_callback is not callable.

Return type:



This class relies on multiprocessing features, it is therefore necessary to protect its execution with an if __name__ == '__main__': statement when working on Windows.

MULTI_PROCESSING_START_METHOD: ClassVar[MultiProcessingStartMethod] = 'fork'

The start method used for multiprocessing.

The default is MultiProcessingStartMethod.SPAWN on Windows, MultiProcessingStartMethod.FORK otherwise.

N_CPUS: Final[int] = 2

The number of CPUs.

inputs: list[Any]

The inputs to be passed to the workers.

n_processes: int

The maximum simultaneous number of threads or processes.

use_threading: bool

Whether to use threads instead of processes to parallelize the execution.

wait_time_between_fork: float

The time to wait between two forks of the process/thread.

workers: Sequence[Callable]

The objects that perform the tasks.