Backend

The interface defining operations that should be overridden by the concrete backend classes and BackendProxy as well.

API

class unidist.core.base.backend.Backend

An interface that represents the parent class for any backend class.

static cluster_resources()

Get resources of the cluster.

Returns:

Dictionary with cluster nodes info in the form {“node_ip0”: {“CPU”: x0}, “node_ip1”: {“CPU”: x1}, …}.

Return type:

dict

static get(object_refs)

Get a remote object or a list of remote objects from distributed memory.

Parameters:

object_refs (ObjectRef or list) – Object ref or a list of object refs to get data from.

Returns:

A Python object or a list of Python objects.

Return type:

object

Notes

The method of the child class for the concrete backend should pass object_refs specific for the backend.

static get_ip()

Get node IP address.

Returns:

Node IP address.

Return type:

str

static is_initialized()

Check if a unidist backend has already been initialized.

Returns:

True or False.

Return type:

bool

static make_actor(cls, num_cpus, resources)

Define an actor class.

clsobject

Class to be an actor class.

num_cpusint

The number of CPUs to reserve for the lifetime of the actor.

resourcesdict

Custom resources to reserve for the lifetime of the actor.

Return type:

ActorClass

Notes

The method of the child class for the concrete backend should return actor class specific for the backend and arguments to be passed in the actor.

static make_remote_function(function, num_cpus, num_returns, resources)

Define a remote function.

functioncallable

Function to be a remote function.

num_cpusint

The number of CPUs to reserve for the remote function.

num_returnsint

The number of ObjectRef-s returned by the remote function invocation.

resourcesdict

Custom resources to reserve for the remote function.

Return type:

RemoteFunction

Notes

The method of the child class for the concrete backend should return remote function object specific for the backend.

static num_cpus()

Get the number of CPUs used by the execution backend.

Return type:

int

static put(data)

Put data into distributed memory.

Parameters:

data (object) – Data to be put.

Returns:

ObjectRef matching to data.

Return type:

ObjectRef

Notes

The method of the child class for the concrete backend should return object ref specific for the backend.

static shutdown()

Shutdown an execution backend.

static wait(object_refs, num_returns=1)

Wait until object_refs are finished.

This method returns two lists. The first list consists of object ref-s that correspond to objects that completed computations. The second list corresponds to the rest of the object ref-s (which may or may not be ready).

Parameters:

• object_refs (ObjectRef or list) – Object ref or list of object refs to be waited.

• num_returns (int, default: 1) – The number of object refs that should be returned as ready.

Returns:

List of object refs that are ready and list of the remaining object refs.

Return type:

two lists

Notes

The method of the child class for the concrete backend should pass and return object ref-s specific for the backend.

BackendProxy

A singleton class which instance is created during unidist initialiation using init() function. As soon as the instance is created, any operation called by a user can be dispatched to the concrete backend correctly. After an operation is performed by the concrete backend, the result hands over back to this class to postprocess it if necessary and return to the user.

API

class unidist.core.base.backend.BackendProxy(backend_cls)

A class which instance is a proxy object to dispatch operations to the concrete backend.

Parameters:

backend_cls (Backend) – Instance of the concrete backend class.

cluster_resources()

Get resources of the cluster.

Returns:

Dictionary with cluster nodes info in the form {“node_ip0”: {“CPU”: x0}, “node_ip1”: {“CPU”: x1}, …}.

Return type:

dict

get(object_refs)

Get a remote object or a list of remote objects from distributed memory.

Parameters:

object_refs (ObjectRef or list) – ObjectRef or a list of ObjectRef-s to get data from.

Returns:

A Python object or a list of Python objects.

Return type:

object

classmethod get_instance(backend_cls=None)

Get instance of this class.

Parameters:

backend_cls (Backend, optional) – Instance of the concrete backend class.

get_ip()

Get node IP address.

Returns:

Node IP address.

Return type:

str

is_initialized()

Check if a unidist backend has already been initialized.

Returns:

True or False.

Return type:

bool

static is_object_ref(obj)

Whether an object is ObjectRef or not.

Parameters:

obj (object) – An object to be checked.

Returns:

True if an object is ObjectRef, False otherwise.

Return type:

bool

make_actor(cls, num_cpus, resources)

Define an actor class.

clsobject

Class to be an actor class.

num_cpusint

The number of CPUs to reserve for the lifetime of the actor.

resourcesdict

Custom resources to reserve for the lifetime of the actor.

Return type:

ActorClass

make_remote_function(function, num_cpus, num_returns, resources)

Define a remote function.

functioncallable

Function to be a remote function.

num_cpusint

The number of CPUs to reserve for the remote function.

num_returnsint

The number of ObjectRef-s returned by the remote function invocation.

resourcesdict

Custom resources to reserve for the remote function.

Return type:

RemoteFunction

num_cpus()

Get the number of CPUs used by the execution backend.

Return type:

int

put(data)

Put data into distributed memory.

Parameters:

data (object) – Data to be put.

Returns:

ObjectRef matching to data.

Return type:

ObjectRef

shutdown()

Shutdown an execution backend.

wait(object_refs, num_returns=1)

Wait until object_refs are finished.

This method returns two lists. The first list consists of ObjectRef-s that correspond to objects that completed computations. The second list corresponds to the rest of the ObjectRef-s (which may or may not be ready).

Parameters:

• object_refs (ObjectRef or list) – ObjectRef or list of ObjectRef-s to be waited.

• num_returns (int, default: 1) – The number of ObjectRef-s that should be returned as ready.

Returns:

List of ObjectRef-s that are ready and list of the remaining ObjectRef-s.

Return type:

two lists