I Finally Managed to Activate a Conda Environment in My CI Pipeline

You create a conda environment, you activate it, you install some packages, you write some code, you run some test, life is good. Then you decide to set up a continuous integration (CI) pipeline. You find the conda documentation on how to do this and it’s pretty straightforward because the steps are the same as on your laptop: install Ana/Miniconda, create a conda environment from your environment.yml file, activate the environment using conda activate <my_awesome_environment>, and then it happens:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run

  $ conda init <SHELL_NAME>

Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

You think to yourself: “Huh?! What did I do wrong? These are the same steps I execute on my laptop.” You read the error message and realise: “Oh, I have to initialise my shell.” So, you pick the shell used by the CI build server (bash?) and add conda init bash to the set of instructions for the CI build server. Here we go again, install Ana/Miniconda, create a conda environment from your environment.yml file, initialise shell (crucial step it seems), activate the environment using conda activate <my_awesome_environment>, and then this:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run

  $ conda init <SHELL_NAME>

Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

“Again the same error?”. But you realise you were a bit too hasty and didn’t properly read the instructions. It is even indicated in all caps IMPORTANT: You may need to close and restart your shell after running 'conda init'. And the conda init command also gives you this instruction:

==> For changes to take effect, close and re-open your current shell. <==

Okay, we will now restart the shell.

…

But how? Some googling tells you to source the ~/.bashrc file. And obviously you knew this, but kinda forgot. So here we go again, install Ana/Miniconda, create a conda environment from your environment.yml file, initialise the shell with conda init bash, source update the shell with source ~./bashrc, but this already gives you an error:

<some_command>: not found

It seems that there are some other commands in the bashrc file that you’re not supposed to execute, but which are run behind the scenes by the CI server/container itself. Pffff, this becomes quite a lot of work. You desperately try to isolate the conda-init part of the bashrc file and run it directly.

# bashrc file
...

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/root/miniconda/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/root/miniconda/etc/profile.d/conda.sh" ]; then
        . "/root/miniconda/etc/profile.d/conda.sh"
    else
        export PATH="/root/miniconda/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

You try to incorporate this into your CI steps, but it fails, and you try again and again, do some googling, and again. But whatever you do, you keep getting the same bloody error:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run

  $ conda init <SHELL_NAME>

Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

So give give up and switch to using pip with the default Python environment of the CI server/container. The end.

“Wait a minute!” I hear your think, “This article was supposed to solve this issue, was it not?”. You are correct! It will. The trick is to not activate your conda environment. Mind… is… blown!

There is a secret conda command which allows you to use an environment without activating it, namely conda run. So instead of first activating your environment and then running python my_awesome_app.py, you can run it directly with

 conda run -n <my_awesome_environment> python my_awesome_app.py

This will solve all your conda-related CI problems. If you want to know more about the reason why and see the details of my CI setup using AWS CodeBuild, keep reading the following sections. If you just want a working solution, you can stop here 😉.

But whatever you do, spread the word!

Explanation

I cannot give you an in-depth explanation of why certain steps did not work as I don’t know the inner workings of Conda nor of those of the CI server. Also, it did not figure this out myself but I found a working solution on this blog, so credit goes to Itamar Turner-Trauring.

The reason why a plain conda activate did not work is that, and I quote,

Unlike the activate script for the Python virtualenv tool, which just sets an environment variable or two, the conda activation can also activate environment variables set by packages. That means you can’t just emulate it, you need to use Conda’s own activation infrastructure. — Itamar Turner-Trauring

That's why you have to use the conda init command. After doing this, the blog of Itamar discusses the problems of RUN commands in Docker files, which are separate shell sessions, meaning that the conda activate will only work for a single session. The example in this article did not even manage to activate the environment because restarting the shell seemed to be a pain in the ass. Other people ran into the same problem, as seen in this GitHub issue. This can be different with other CI infrastructure. This example used AWS CodeBuild, where build projects are actually Docker containers run on some EC2 instance. But how the details of a build project translates to Docker commands, I do not know.

In conclusion, using the conda activation infrastructure is complex and will depend on what CI tool you use. Using conda run with specifying the conda environment bypasses this and is therefore a CI tool-independent solution.

CI setup using AWS CodeBuild

This section will show the details of my CI setup, the relevant part of the configuration of my AWS CodeBuild project. I mainly followed Conda’s documentation using Conda with Travis CI. Here is how my AWS CodeBuild’s details look like:

{
  "version": "0.2",
  "phases": {
    "pre_build": {
      "commands": [
        ... (non-python stuff)

        "wget [https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh](https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh) -O miniconda.sh -q",
        "bash miniconda.sh -b -p $HOME/miniconda",
        ". \"$HOME/miniconda/etc/profile.d/conda.sh\"",
        "hash -r",
        "conda config --set always_yes yes --set changeps1 no",
        "conda update -q conda",
        "conda info -a",
        "conda env create -q -n ${PY_ENV} -f environment.yml",
      ]
    },
    "build": {
      "commands": [
        "conda run -n ${PY_ENV} python app.py"
      ]
    }
  ...
}

It first installs miniconda, next it makes some preparations for using conda (see documentation), next it creates an environment from the environment.yml file. Here PY_ENV is an environment variable set in my CodeBuild which is a dummy name for my conda environment. And lastly, it executes my app.py file using the conda run with the newly created environment. Note that for AWS CodeBuild I had to change source "$HOME/miniconda/etc/profile.d/conda.sh" (as stated in Conda’s documentation) into. "$HOME/miniconda/etc/profile.d/conda.sh" since the source command is not a POSIX standard.

Final words

Activating a conda environment in a CI Pipeline can be quite difficult, if not impossible for your CI tool. Using conda run with specifying what conda environment to use is CI tool-independent solution for completely bypasses this problem. It is a CI tool-independent way of using conda environments in your pipeline. I hope I have spared you countless hours of frustration with this article. It definitely would have done so for past me.

And done forget to share this valuable knowledge!