Common interfaces

Functions and classes used by MPI backend modules.

API

class unidist.core.backends.mpi.core.common.Operation

Class that describes supported operations.

EXECUTE

Execute a remote task.

Type:

int, default 1

GET

Return local data to a requester.

Type:

int, default 2

PUT_DATA

Save the data to a local storage.

Type:

int, default 3

PUT_OWNER

Save the data location to a local storage.

Type:

int, default 4

PUT_SHARED_DATA

Save the data into shared memory.

Type:

int, default 5

WAIT

Return readiness signal of a local data to a requester.

Type:

int, default 6

ACTOR_CREATE

Create local actor instance.

Type:

int, default 7

ACTOR_EXECUTE

Execute method of a local actor instance.

Type:

int, default 8

CLEANUP

Cleanup local object storage for out-of-scope IDs.

Type:

int, default 9

TASK_DONE

Increment global task counter.

Type:

int, default 10

GET_TASK_COUNT

Return global task counter to a requester.

Type:

int, default 11

RESERVE_SHARED_MEMORY

Reserve area in shared memory for the data.

Type:

int, default 12

REQUEST_SHARED_DATA

Return the area in shared memory with the requested data.

Type:

int, default 13

CANCEL

Send a message to a worker to exit the event loop.

Type:

int, default 14

READY_TO_SHUTDOWN

Send a message to monitor from a worker, which is ready to shutdown.

Type:

int, default 15

SHUTDOWN

Send a message from monitor to a worker to shutdown.

Type:

int, default 16

class unidist.core.backends.mpi.core.common.MpiDataID(owner_rank, data_number, gc=None)

Class for tracking data IDs of MPI processes.

The class extends unidist.core.backends.common.data_id.DataID functionality, ensuring the uniqueness of the object to correctly construct/reconstruct it. Otherwise, we would get into wrong garbage collection.

Parameters:

  • owner_rank (int) – The rank of the process that owns the data.

  • data_number (int) – Unique data number for the owner process.

  • gc (unidist.core.backends.mpi.core.executor.GarbageCollector or None) – Local garbage collector reference. The actual object is for the data id owner, otherwise, None.

unidist.core.backends.mpi.core.common.get_logger(logger_name, file_name, activate=None)

Configure logger and get it’s instance.

Parameters:

  • logger_name (str) – Name of a logger.

  • file_name (str) – File name.

  • activate (optional, default: None) – Whether to enable logging or not. If None, the value is superseded with MpiLog.

Returns:

A Python logger object.

Return type:

object

unidist.core.backends.mpi.core.common.unwrapped_data_ids_list(args)

Transform all found data ID objects to their underlying value in an iterable object.

Parameters:

args (iterable) – Sequence of data.

Returns:

Transformed list with underlying DataID values.

Return type:

list

unidist.core.backends.mpi.core.common.materialize_data_ids(data_ids, unwrap_data_id_impl, is_pending=False)

Traverse iterable object and materialize all data IDs.

Find all unidist.core.backends.mpi.core.common.MpiDataID instances and call unwrap_data_id_impl on them.

Parameters:

  • data_ids (iterable) – Iterable objects to transform recursively.

  • unwrap_data_id_impl (callable) – Function to get the ID associated data from the local object store if available.

  • is_pending (bool, default: False) – Status of data materialization attempt as a flag.

Returns:

Transformed iterable object (task arguments) and status if all DataID instances were transformed.

Return type:

iterable or bool

unidist.core.backends.mpi.core.common.check_mpich_version(target_version)

Check if the using MPICH version is equal to or greater than the target version.

Parameters:

target_version (str) – Required version of the MPICH library.

Returns:

True ot false.

Return type:

bool

unidist.core.backends.mpi.core.common.is_shared_memory_supported(raise_warning=False)

Check if the unidist on MPI supports shared memory.

Parameters:

raise_warning (bool, default: False) – Whether to raise a warning or not. True is passed only for root process to have the only warning.

Returns:

True or False.

Return type:

bool

Notes

Prior to the MPI 3.0 standard there is no support for shared memory.