I have the following in a CMakeLists.txt file. I'm trying to use cmake to check if a conda environment named myenv is installed on the system and activate that environment. If the environment does not exist, then create the environment and activate it. This assumes that conda is already installed via Anaconda (or Miniconda).
# Create and activate a Python environment.
cmake_minimum_required(VERSION 3.18)
# Define the project
project(MyExample)
# Specify the C++ standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
# Make sure Python is installed
find_package(Python REQUIRED)
# Activate conda environment, assume Anaconda or Miniconda is already installed
if(EXISTS /opt/miniconda3/envs/myenv)
execute_process(COMMAND conda activate myenv)
else()
execute_process(COMMAND conda create --yes --quiet --name myenv python)
execute_process(COMMAND conda activate myenv)
endif()
When I run the above cmake file, I get the error:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
However, conda is installed on my system and I can activate the environment manually in the terminal. Why does the environment not get activated via cmake?
"Why does the environment not get activated via cmake?"
The first issue is in the error message, which indicates that the shell that CMake evaluates the execute_process command under doesn't know what conda activate means.
This could be because conda init <shell> has never been run for the particular shell that CMake uses (bash, perhaps?). This answer has some details on what conda init does. It could also be because the Conda initialization code that defines conda activate only gets loaded in interactive shells - which I wouldn't expect CMake to be using. There might be hacky ways to force execution in an interactive shell (bash -l -c 'conda activate foo'), but that doesn't matter because...
Even if the above were working, the procedures here don't make sense: a Conda environment's activation status is scoped to the shell process. I would expect that the (sub)shell dies with the completion of the execute_process. So, even if the activation worked, it wouldn't persist any further in the CMake script.
Discussion
Generally, this CMake script does not seem like a good approach. Tight-coupling the compilation of code to the existence of a particularly-named Conda environment at the user level seems to go against the spirit of CMake, which aims to automate the discovery of software dependencies so that users don't need commonly hardcoded locations. Perhaps it might be worth reassessing what is trying to be accomplished and the strategy to get there.
For example, on Conda Forge, lots of packages compile with CMake, but there CMake is executed in the context of an already activated environment and itself knows nothing about Conda. This makes it so the CMake code is completely agnostic to how its dependencies are provided, which I regard as cleaner engineering.
Related
I have installed conda environment using mambaforge to use snakemake. I have followed the instructions of Snakemake setup tutorial to install mambaforge. I have created the environment and created a .py file, but when I try to run it, it shows the plain code rather than compiling it. When I went to fix it and to add a Conda environment in my PyCharm interpreter it shows me the following error:
I tried to add the conda executable path by browsing to mambaforge/bin/conda but it still shows me the error.
I also tried to use which conda to find the right path, but rather than giving me a path the following is show, I have tried which conda in conda base environment as well as my setup environment, all it returns is the following:
I am sure that my conda environment is setup as conda --version command shows me the version I have installed.
the conda bin installed with mambaforge is at ~/mambaforge/condabin/conda
I'm not aware of mambaforge conda distribution, I'm using Anaconda / Miniconda distributions with an installed mamba module, so I could create a snakemake environment using terminal (i.e. from command line) and register as Python interpreter in PyCharm Settings as "Conda Environment | Existing environment" (see your first screenshot).
So I suggest you:
check whether "Existing environment" works for you, e.g. is able to find you conda environments (e.g base, or others).
You could set up Anaconda/Miniconda + mamba + configure PyCharm for using Snakemake with the help of my detailed guide that I use for my students' homework on snakemake.
P.S: I always run snakemake scripts using command line, e.g from terminal bundled into PyCharm, or just system terminal app or on remote server. I don't use PyCharm run configurations for that. As for PyCharm, I use it with SnakeCharm plugin to have a good snakemake & python code editing support
i'm having a weird problem...
I can install packages using the built-in package manager in pycharm. But for some reason everytime i use "pip install (xx)" it is installing the packages in a conda env somewhere on my mac...
How can i solve this ?
I've tried the following:
close --> reopen pycharm //
deactivate and activate the venv //
Checked project intepreter is the right one (which it is...)
You're inside the virtual environment venv, while being inside the Conda base environment (note the (venv) and (base) to the left of your prompt). Conda is likely overriding your venv's pip.
My bet as to why this is happening is that, during installation, you set Conda to autostart its base environment whenever a new terminal is open (be it inside PyCharm or not).
You can try to either:
exit Conda (with conda deactivate) and try pip install again (check to see that you're still inside the venv virtual environment).
install the packages directly from PyCharm's GUI - note the small + sign on the bottom-left of the package list. This won't solve the issue related to your terminal, but will function as a workaround for now.
Note that these aren't guaranteed to work, because you may have additional configurations on your system (either installed directly by you, or indirectly by Conda when you installed it).
It often happens that when I create a conda environment, I forgot about activating it, despite the clear conda message. And I end up installing my packages in the conda base environment. (yeah... I'm a bit of a dreamer)
My questions are:
Are there use cases where you create a conda environment but don't activate it right after?
How to create a conda environment and activate it in a single line? (in a Linux prompt shell, and non-interactive)
As a concrete example in answer to your first question, I have used conda as a way to wrap a disposable build environment in some Makefile targets, i.e., I create the environment, and then subsequent commands or targets may make use of the environment via conda run.
Adapting a snippet from one Makefile, you could create a function in a bash startup file:
conda_create_and_run() {
ENV_NAME=$1
CONDA_PY_VER=$2
. ${CONDA_ENV_FILE}
conda config --append envs_dirs ${CONDA_DIR}
conda create -p ${CONDA_DIR}/${ENV_NAME} python=${CONDA_PY_VER} -y
conda activate ${ENV_NAME}
}
Here CONDA_PY_VER is the non-default python version you want the environment to possibly be created with, and CONDA_DIR and CONDA_ENV_FILE are, respectively, the location where conda keeps its environments and the conda environment file you need to source (or have part of your shell init) in order to have the conda commands available.
You would then use it as:
conda_create_and_run myenv 3.8
to create an environment for python3.8 named myenv.
I do conda create --name env then conda activate env.
My prompt now has (env) at the beginning of it. Then I try running python at the prompt and it returns
'python' is not recognized as an internal or external command, operable program or batch file.
The documentation explicitly states
This environment uses the same version of Python that you are currently using because you did not specify a version.
However, if I do conda create --name env python=3.8, my environment correctly runs python 3.8. It seems like if I don't specify a version, my environment is completely empty (which it is when I look in the directory). But the documentation says that it should have the version of python from the base. What am I doing wrong?
I'm using an anaconda prompt in Windows 10.
I think that bit of documentation is outdated and is a hold-over from pre-v4.4, when the recommended practice was to put the base env's bin/ directory on PATH. In Conda v4.4+, the base env is no longer accessible by default when another env is activated.
To have Python in an env, one must explicitly request it to be installed, e.g.,
conda create --name env python
Note, one doesn't have to specify a version.
In the end, this should be seen as an advantage, since it allows users to create non-Python envs and keeps the base env isolated.
So I am used to typing source activate <environment> when starting a python Anaconda environment. That works just fine. But when I create new conda environments I am seeing the message on Ubuntu 16.04 to start the environments with conda activate instead. Besides the errors about how to set up my shell to use conda activate instead, I am still not clear on what is the difference between source activate ... and conda activate ... Is there a reason to change? Does anyone know the difference between these two commands? Thanks.
As of conda 4.4, conda activate is the preferred way to activate an environment. Generally, you won't find too much of a difference between conda activate and the old source activate, except that it's meant to be faster, and work the same across different operating systems (the latter difference makes conda activate a huge improvement IMO).
From the docs, regarding the release of conda version 4.4.0 (released December 2017):
conda activate: The logic and mechanisms underlying environment activation have been reworked. With conda 4.4, conda activate and conda deactivate are now the preferred commands for activating and deactivating environments. You’ll find they are much more snappy than the source activate and source deactivate commands from previous conda versions. The conda activate command also has advantages of (1) being universal across all OSes, shells, and platforms, and (2) not having path collisions with scripts from other packages like python virtualenv’s activate script.
Here is one difference I found. source activate can be used at the beginning of a bash script to load conda environment, whereas conda activate would give me an error:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
This makes a huge difference to me since I often submit bash jobs to cluster and source activate is the only way to change conda environment.
Please correct me if anyone can use conda activate in a bash script.
I am not sure who might find this useful, but if
Your terminal lags due to the addition ">>> conda initialize
" in your .bashrc, then you decide to remove it and add anaconda to the path. If that is the case, then "conda activate env_name"
won't work, but "source activate env_name" will work, and then after
that, you can use either source activate or conda activate. If you
close the shell then to activate the environment again use "source
activate env_name"
FYI, removing ">>> conda initialize >>>" from my .bashrc file has
speedup my terminal and it doesn't lag anymore and I just default in
using "source activate env_name"
I have Ubuntu 20.04, conda version : 4.10.3, and conda-build version
: 3.21.5