Problem with conda environment between build steps

Hey everyone, 

I created a build configuration and can't figure out how to connect the dots here. 

My configuration is supposed to run a python project using conda. It consists of these steps:

1. Command-Line script to activate and update the conda environment for this project (there are three projects and the environment has to be activated accordingly)

2. Python Runner to run pytest

3. Qodana analysis

 

The Commandline script

The command line script runs first and looks like this (some of the things are a bit experimental):

sudo ln -sf bash /bin/sh                #makes sure that the agent uses bash, since the default is dash
ls -l `which sh` # print to confirm
. /opt/miniconda/etc/profile.d/conda.sh # init conda for bash and reload the environment
conda activate <my-environments-name> # activate the environment
conda env update # update the dependencies
conda list # print the packages in this environment

 

The pytest-step

The pytest step runs with the following configuration:

  • Command: pytest
  • Test-reporting: true
  • Python version: <Custom>
  • custom path: /opt/miniconda/bin/python
  • Environment tool: none

Running this step throws an "Module not found"-error for one of the dependencies. But I see in the list in step 1, that the dependency has been installed. 

I also tried to set the python path in the second step to conda instead of python, but this will prevent the step from loading the stuff for test-reporting.

 

What am I missing? I am fairly new to TeamCity (and conda) and could need some help here. What am I doing wrong or what is a proper way to achieve what I want to do? 

Thanks in advance!

0
3 comments

Hello!

Unfortunately, I do not have any practical experience with conda, but I would like to mention that each step runs as a separate subprocess (even though they run in the same directory). When the conda environment is activated, does it add some kind of environment variable? If yes, this may explain why it is not persisted between steps.

As a simple workaround, you could try to invoke your script directly from the command line step.

0

Thank you for your answer!

Wouldn't I lose some of TeamCitys Features (Test Reporting etc.) when running pytest from the same command line script?*

Maybe I could prepapre some basic Docker Containers for each Environment and run the steps inside these... Would just need to find a comfortable way to update their environments from time to time.

 

*Edit: Just tried it and showing test reports in the "Tests"-Tab in TeamCity seems to work fine!

 

Has someone else experience with TeamCity and Conda? I thought that Conda is a fairly popular environment tool.

0

We also have a feature request to support conda in Python runner - please consider voting or commenting on it as you see fit. Do any other package managers work for you, by chance?

0

Please sign in to leave a comment.