Why conda?
The Conda software manager is a powerful tool for installing and managing scientific software. It has become somewhat of a standard for setting up reusable and portable software environments that can shared with others and easily recreated across computers. As a developer it is particulary useful for testing sotware in multiple environments before deploying. Some the other advantages of using conda are listed below:
- Create fresh environments to install into (avoid system conflicts).
- Create separate “sandboxed” environments (avoid conflicts among environments).
- Install software locally (easy to remove or update w/o needing administrator).
- Test among different environments (e.g., Py2 and Py3)
- Share environments for others to reproduce.
This post (our goal)
Here I’ve listed instructions for installling a py2/py3 kernel stack on a Linux machine (e.g., HPC cluster) to make both languages available.
Installing conda
I recommend installing miniconda, the bare-bones version of conda that includes only the necessary software to make conda work. Once conda is installed you can use it to install any other software that you want.
I have installed conda many times on many systems, often with multiple kernels, and from this experience I’ve learned a few useful tricks. My installation instructions may differ slightly from others you will find online; this is partly because the conda installation procedure has changed rapidly in recent years, and also because most tutorials only describe the simplest installation approach. My instructions are for the latest conda versions as of late 2018, and are somewhat complex as they are aimed towards users who wish to install multiple kernels.
Conda installation instructions (for Linux)
# For LINUX -- get the latest miniconda3 64-bit installer.
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3-installer.sh
# run batch installer
bash ~/miniconda3-installer.sh -b
# add conda to the PATH variable in your startup file (.bashrc)
echo 'PATH=$HOME/miniconda3/bin:$PATH; export PATH' >> ~/.bashrc
source ~/.bashrc
What is conda
You now have the conda binary installed which can be used to find and install
other software. You also have a default location where your new software will
be installed, which is in your miniconda3/ directory. You also have a default
environment where that software will be grouped into, called base. So far
this environment only has Python3 and a few other Python packages installed.
You can check your conda installation by using the commands conda info, or
see the software installed in your environment with conda list.
Install ipython, jupyter, ipyparallel in base env
conda install ipython jupyter ipyparallel -y
Installing kernels
Here’s another trick. I don’t install much into the base conda environment.
Instead, I create a Python3 and Python2 environment and give explicit names to
each. Then when I install any subsequent software I always designate using the
-n argument to conda install which environment I wish to install into.
Let’s create a Python2 environment and name it py27:
conda create -n py27 python=2 notebook ipykernel -y
source activate py27
ipython kernel install --user --display-name "Python 2.7"
source deactivate
# Installed kernelspec py27 in /home/deren/.local/share/jupyter/kernels/py27
Install software into your Python environments
conda install numpy scipy pandas -n base -y
conda install toytree -c eaton-lab -n base -y
conda install numpy scipy pandas -n py27 -y
conda install toytree -c eaton-lab -n py27 -y
Cleanup
To save space you may want to remove all of the downloaded package files for the software conda installed. You can do this with the command below.
conda clean --all -y
Removing/uninstalling conda
If you decide that you’re not down with conda, or you want to remove all of
your software and start fresh, you can do this easily with conda. The command
to remove your conda environment simply involves removing the directory where
miniconda is installed. Be careful here that you write the correct path name
when using the rm command. If you installed multiple kernels then you will
also want to remove the .conda and .local/share/jupyter/kernels dirs since
these hold the references to those kernel names.
# if you want to remove miniconda
rm -rf ~/miniconda3/
rm -rf ~/.conda
rm -rf ~/.local/share/jupyter/kernels