Installation

There are several ways to install unidist. Most users will want to install with pip or using conda tool, but some users may want to build from the master branch on the GitHub repo. The master branch has the most recent patches, but may be less stable than a release installed from pip or conda.

Installing with pip

Stable version

unidist can be installed with pip on Linux, Windows and MacOS. To install the most recent stable release run the following:

pip install unidist # Install unidist with dependencies for Python Multiprocessing and Python Sequential backends

unidist can also be used with MPI, Dask or Ray execution backend. If you don’t have MPI, Dask or Ray installed, you will need to install unidist with one of the targets:

pip install unidist[all] # Install unidist with dependencies for all the backends
pip install unidist[mpi] # Install unidist with dependencies for MPI backend
pip install unidist[dask] # Install unidist with dependencies for Dask backend
pip install unidist[ray] # Install unidist with dependencies for Ray backend

unidist automatically detects which execution backends are installed and uses that for scheduling computation!

Note

There are different MPI implementations, each of which can be used as a backend in unidist. Mapping unidist[mpi] installs mpi4py package, which is just a Python wrapper for MPI. To enable unidist on MPI execution you need to have a working MPI implementation and certain software installed beforehand. Refer to installation page of the mpi4py documentation for details. Also, you can find some instructions on MPI backend page.

Release candidates

unidist follows Semantic Versioning and before some major or minor releases, we will upload a release candidate to test and check if there are any problems. If you would like to install a pre-release of unidist, run the following:

pip install --pre unidist

These pre-releases are uploaded for dependencies and users to test their existing code to ensure that it still works. If you find something wrong, please raise an issue.

Installing with conda

Using conda-forge channel

unidist releases can be installed using conda from the conda-forge channel. Starting from the first 0.1.0 release it is possible to install unidist with chosen execution backend(s) alongside. Current options are:

Package name in conda-forge	Backend(s)	Supported OSs
unidist	Python Multiprocessing, Python Sequential	Linux, Windows, MacOS
unidist-all	MPI, Dask, Ray, Python Multiprocessing, Python Sequential	Linux, Windows
unidist-mpi	MPI	Linux, Windows, MacOS
unidist-dask	Dask	Linux, Windows, MacOS
unidist-ray	Ray	Linux, Windows

For installing unidist with dependencies for MPI and Dask execution backends into a conda environment the following command should be used:

conda install unidist-mpi unidist-dask -c conda-forge

All set of backends could be available in a conda environment by specifying:

conda install unidist-all -c conda-forge

or explicitly:

conda install unidist-mpi unidist-dask unidist-ray -c conda-forge

Note

There are different MPI implementations, each of which can be used as a backend in unidist. By default, mapping unidist-mpi installs a default MPI implementation, which comes with mpi4py package and is ready to use. The conda dependency solver decides on which MPI implementation is to be installed. If you want to use a specific version of MPI, you can install the core dependencies for MPI backend and the specific version of MPI as conda install unidist-mpi <mpi> as shown in the installation page of mpi4py documentation. That said, it is highly encouraged to use your own MPI binaries as stated in the Using External MPI Libraries section of the conda-forge documentation in order to get ultimate performance.

Using intel channel

Conda intel channel contains a performant MPI implementaion, which can be used in the unidist MPI backend instead of an MPI implementation from conda-forge channel. To install Intel MPI you should use the following:

conda install unidist -c conda-forge
conda install mpi4py -c intel

Installing from the GitHub master branch

If you’d like to try unidist using the most recent updates from the master branch, you can also use pip.

# Install unidist with dependencies for Python Multiprocessing and Python Sequential backends
pip install git+https://github.com/modin-project/unidist
# Install unidist with dependencies for all the backends
pip install git+https://github.com/modin-project/unidist#egg=unidist[all]
# Install unidist with dependencies for MPI backend
pip install git+https://github.com/modin-project/unidist#egg=unidist[mpi]

This will install directly from the repo without you having to manually clone it! Please be aware that these changes have not made it into a release and may not be completely stable.

Building unidist from Source

If you’re planning to contribute to unidist, you need to ensure that you are building unidist from the local repository that you are working of. Occasionally, there are issues in overlapping unidist installs from PyPI and from source. To avoid these issues, we recommend uninstalling unidist before installation from source:

pip uninstall unidist

To build from source, you first must clone the repo. We recommend forking the repository first through the GitHub interface, then cloning as follows:

git clone https://github.com/<your-github-username>/unidist.git

Once cloned, cd into the unidist directory and use pip to install:

cd unidist
# Install unidist with dependencies for Python Multiprocessing and Python Sequential backends
pip install -e .
# Install unidist with dependencies for all the backends
pip install -e .[all]
# Install unidist with dependencies for MPI backend
pip install -e .[mpi]