PyMpRemoteFunction

The class is specific implementation of RemoteFunction class using Python Multiprocessing backend.

The PyMpRemoteFunction implements internal method _remote() that transmites execution of operations to Python Multiprocessing.

API

class unidist.core.backends.pymp.remote_function.PyMpRemoteFunction(function, num_cpus, num_returns, resources)

The class that implements the interface in RemoteFunction using Python Multiprocessing backend.

Parameters:

  • function (callable) – A function to be called remotely.

  • num_cpus (int) – The number of CPUs to reserve for the remote function.

  • num_returns (int) – The number of ObjectRef-s returned by the remote function invocation.

  • resources (dict) – Custom resources to reserve for the remote function.

_remote(*args, num_cpus=None, num_returns=None, resources=None, **kwargs)

Execute self._remote_function in a worker process.

Parameters:

  • *args (iterable) – Positional arguments to be passed in the self._remote_function.

  • num_cpus (int, optional) – The number of CPUs to reserve for the remote function.

  • num_returns (int, optional) – The number of ObjectRef-s returned by the remote function invocation.

  • resources (dict, optional) – Custom resources to reserve for the remote function.

  • **kwargs (dict) – Keyword arguments to be passed in the self._remote_function.

Returns:

Type of returns depends on num_returns value:

  • if num_returns == 1, ObjectRef will be returned.

  • if num_returns > 1, list of ObjectRef-s will be returned.

  • if num_returns == 0, None will be returned.

Return type:

ObjectRef, list or None