gemseo / core

parallel_execution module

Parallel execution of disciplines and functions using multiprocessing

Classes:

DiscParallelExecution(worker_list[, …])	Execute disciplines in parallel.
DiscParallelLinearization(worker_list[, …])	Linearize disciplines in parallel.
ParallelExecution(worker_list[, …])	Perform a parallel execution of tasks on input values.

class gemseo.core.parallel_execution.DiscParallelExecution(worker_list, n_processes=2, use_threading=False, wait_time_between_fork=0)

Bases: gemseo.core.parallel_execution.ParallelExecution

Execute disciplines in parallel.

Constructor.

Parameters

• worker_list – list of objects that perform the tasks

• n_processes – maximum number of processors on which to run

• use_threading – if True, use Threads instead of processes to parallelize the execution multiprocessing will copy (serialize) all the disciplines, while threading will share all the memory This is important to note if you want to execute the same discipline multiple times, you shall use multiprocessing

• wait_time_between_fork – time waited between two forks of the process /Thread

Attributes:

N_CPUS

Methods:

execute(input_data_list[, exec_callback, …])

Execute all processes.

N_CPUS = 2

execute(input_data_list, exec_callback=None, task_submitted_callback=None)

Execute all processes.

Parameters

• input_data_list – the input values (list or values)

• exec_callback – callback function called with the pair (index, outputs) as arguments when an item is retrieved from the processing, where index is the associated index in input_data_list, of the input used to compute the outputs

• task_submitted_callback – callback function called when all the tasks are submitted, but not yet done

class gemseo.core.parallel_execution.DiscParallelLinearization(worker_list, n_processes=2, use_threading=False, wait_time_between_fork=0)

Bases: gemseo.core.parallel_execution.ParallelExecution

Linearize disciplines in parallel.

Constructor.

Parameters

• worker_list – list of objects that perform the tasks

• n_processes – maximum number of processors on which to run

• use_threading – if True, use Threads instead of processes to parallelize the execution multiprocessing will copy (serialize) all the disciplines, while threading will share all the memory This is important to note if you want to execute the same discipline multiple times, you shall use multiprocessing

• wait_time_between_fork – time waited between two forks of the process /Thread

Attributes:

N_CPUS

Methods:

execute(input_data_list[, exec_callback, …])

Execute all processes.

N_CPUS = 2

execute(input_data_list, exec_callback=None, task_submitted_callback=None)

Execute all processes.

Parameters

• input_data_list – the input values (list or values)

• exec_callback – callback function called with the pair (index, outputs) as arguments when an item is retrieved from the processing, where index is the associated index in input_data_list, of the input used to compute the outputs

• task_submitted_callback – callback function called when all the tasks are submitted, but not yet done

class gemseo.core.parallel_execution.ParallelExecution(worker_list, n_processes=2, use_threading=False, wait_time_between_fork=0)

Bases: object

Perform a parallel execution of tasks on input values.

Input values must be a list of independent pointers.

Constructor.

Parameters

• worker_list – list of objects that perform the tasks

• n_processes – maximum number of processors on which to run

• use_threading – if True, use Threads instead of processes to parallelize the execution multiprocessing will copy (serialize) all the disciplines, while threading will share all the memory This is important to note if you want to execute the same discipline multiple times, you shall use multiprocessing

• wait_time_between_fork – time waited between two forks of the process /Thread

Attributes:

N_CPUS

Methods:

execute(input_data_list[, exec_callback, …])

Execute all processes.

N_CPUS = 2

execute(input_data_list, exec_callback=None, task_submitted_callback=None)

Execute all processes.

Parameters

• input_data_list – the input values (list or values)

• exec_callback – callback function called with the pair (index, outputs) as arguments when an item is retrieved from the processing, where index is the associated index in input_data_list, of the input used to compute the outputs

• task_submitted_callback – callback function called when all the tasks are submitted, but not yet done