Unidist High-Level API

This page contains the API provided by the framework for distributed execution of operations.

init

unidist.api.init()

Initialize an execution backend.

Notes

The concrete execution backend can be set via UNIDIST_BACKEND environment variable or Backend config value. MPI backend is used by default.

is_initialized

unidist.api.is_initialized()

Check if a unidist backend has already been initialized.

Returns:: True or False.
Return type:: bool

remote

unidist.api.remote(*args, **kwargs)

Define a remote function or an actor class.

Parameters:

*args (iterable) – Positional arguments to be passed in a remote function or an actor class.
**kwargs (dict) – Keyword arguments to be passed in a remote function or an actor class.

Return type:

RemoteFunction or ActorClass

get

unidist.api.get(object_refs)

Get a remote object or a list of remote objects from distributed memory.

Parameters:: object_refs (ObjectRef or list) – ObjectRef or a list of ObjectRef-s to get data from.
Returns:: A Python object or a list of Python objects.
Return type:: object

put

unidist.api.put(data)

Put data into distributed memory.

Parameters:: data (object) – Data to be put.
Returns:: ObjectRef matching to data.
Return type:: ObjectRef

wait

unidist.api.wait(object_refs, num_returns=1)

Wait until object_refs are finished.

This method returns two lists. The first list consists of ObjectRef-s that correspond to objects that completed computations. The second list corresponds to the rest of the ObjectRef-s (which may or may not be ready).

Parameters:

object_refs (ObjectRef or list) – ObjectRef or list of ObjectRef-s to be waited.
num_returns (int, default: 1) – The number of ObjectRef-s that should be returned as ready.

Returns:

List of ObjectRef-s that are ready and list of the remaining ObjectRef-s.

Return type:

two lists

is_object_ref

unidist.api.is_object_ref(obj)

Whether an object is ObjectRef or not.

Parameters:: obj (object) – An object to be checked.
Returns:: True if an object is ObjectRef, False otherwise.
Return type:: bool

get_ip

unidist.api.get_ip()

Get node IP address.

Returns:: Node IP address.
Return type:: str

num_cpus

unidist.api.num_cpus()

Get the number of CPUs used by the execution backend.

Return type:: int

cluster_resources

unidist.api.cluster_resources()

Get resources of the cluster.

Returns:: Dictionary with cluster nodes info in the form {“node_ip0”: {“CPU”: x0}, “node_ip1”: {“CPU”: x1}, …}.
Return type:: dict