Emacs and conda workaround - python

I'm using emacs and anaconda.
I have this in my init.el:
(setenv "WORKON_HOME" "/home/user/anaconda3/envs/")
And conda on my path:
# added by Anaconda3 installer
export PATH="/home/user/anaconda3/bin:$PATH"
but emacs can't find my conda environments, which I understand it is supposed to be able to do..
So, when I run C-c C-p to start a new session, and C-c C-c, it fails to import my packages which are installed in a conda environment, with ModuleNotFoundError.
Since I have added this to my path and it still doesn't work, I am trying to work around this, and still be able to run my conda applications from emacs.
I can open a shell in emacs with M-x shell, then source activate myenv, and run python.
I now want C-c C-c to copy into /this/ shell. How do I mark this shell buffer as a python process to send my file.py's text to on C-c C-c, rather than just a shell shell?
Update1
I've also looked at the following references:
https://emacs.stackexchange.com/questions/20092/using-conda-environments-in-emacs
How does conda-env list / conda info --envs find environments?
But neither package works for me. I still get, when I try:
conda-env-list
*Conda envs*
Produces a blank buffer.
And this for pyvenv-workon:
pyvenv-workon
Work on: (empty)
These environments very much exist, and it makes it impossible to use emacs as a python IDE if I can't run my code.

What I found works for me is to use the conda package from ELPA and set two of its configuration variables to point to my Conda directory. The following snippet does the trick in my .emacs:
(use-package conda
:ensure t
:init
(setq conda-anaconda-home (expand-file-name "~/miniconda3"))
(setq conda-env-home-directory (expand-file-name "~/miniconda3")))
conda-anaconda-home is the equivalent to the ANACONDA_HOME environment variable (i.e. contains all files of your Anaconda installation)
conda-env-home-directory - is the directory where your virtual environments get stored (within the envs subdirectory)
With this configuration I'm able to run M-x conda-env-activate and have access to all previously created envs.

Programs inherit the environment variables from the shell that spawned them. The way conda and virtualenv work is by overriding the shell's PATH variable. They do this so that the OS finds the new version of the app (conda's or virtualenv's) instead of the default one installed with the OS (Macs come with an ancient version of python).
So, what is happening here? If you start Emacs by double clicking on the OS icon it will inherit the default shell environment variables. So when you try to call a library that you installed with conda (or equivalently with virtualenv and pip), because you are using the default OS path, the OS is finding the default version of python (and crucially the default version's libraries). The default version of python is going to respond "I have no idea what library that is."
How to fix? One reliable way is to not start Emacs by double clicking on the OS Icon. Here is what I do most days:
1) start a console/terminal
2) switch to the conda environment `activate py37`
(or with virtualenv: `source .py37dev/bin/activate`)
3) start Emacs from that same shell that has the modified environment variables.
On a Mac its: `/Applications/Emacs.app/Contents/MacOS/Emacs`
(I use a installed version of Emacs on the Mac because the one that
comes with Mac is ancient).
On Linux and Windows the path to EMacs will be different but the idea is the same.
4) start a shell inside Emacs and you should see the shell looks the way it does
in your conda shell (or virtualenv shell)
here it what it looks like for me:
see how the version of python is not the default OS python? Its the one from the virtualenv + pip environment (conda works the exact same way, just the start envirmonment is a different command)

I tested the solutions given in the answers of Wojciech Gac, of Mittenchops and James Anderson.
While solution of James Anderson's solution is by far the easiest, it comes with a few problems:
First, you have to reactivate the environment in each shell process you spawn in emacs. There is also the possibility that emacs has a different pythonpath, therefor reluctantly using system python and not venv python.
The solution with conda.el is somehow weird. In Melpa it is listed as obsolete and with instructions of https://github.com/necaris/conda.el it won't recognize the environments on my certain machine.
On the same machine the solution with pyenv as mentioned in https://emacs.stackexchange.com/a/20093/28567 is working like a charm.
Therefor you only need to install with M-x package-install search for pyenv and then insert following two lines into .emacs:
(setenv "WORKON_HOME" "~/anaconda3/envs") ; /anaconda3 || /miniconda || wathever path your conda installation is located at
(pyvenv-mode 1)

This is my minimal solution to this problem:
create a batch file like this
conda activate <yourEnv>
python -i
set (local) python-shell-interpreter pointing to the batch-file
run-python as always (C-c C-p ...)

I know this is not exactly an answer to your question. But If you JUST want to run your code, open the directory in the terminal where your files are located (e.g. cd /Downloads/Chapter01/yourfile.py) . Then, activate the environment (conda activate *your env*) and pass this command (python yourfile.py). Your code will be executed within that environment by python.
I edit remote files with emacs and run code like this in another terminal window. You may wanna open the two terminal windows side by side while debugging your code.

Related

Understanding Anaconda's warning about adding to PATH variable

I have decided to dabble my toes in using Anaconda in addition to pip, mainly so I can install Github's hub wrapper for git.
When I installed Anaconda, one of the prompts asked if I wanted to add Conda to the PATH variable, and recommended against NOT doing that, so I didn't.
I'd like to continue using Git Bash, as it is the only terminal interface I know how to use, and don't currently want to learn how to use the terminal that Anaconda provides. I did not fully understand the warning's explanation:
Not Recommended. Instead, open Anaconda3 with the Windows Start menu and select "Anaconda (64-bit)". This "add to PATH" option makes ANaconda get found before previously installed software, but may cause problems requiring you to uninstall and reinstall Anaconda.
I'm rather new to developing actual software, and pretty much do everything exclusively in Python in PyCharm on Windows, only occasionally using the terminal. That is changing fast however.
I'd like to know how likely these problems are in my case, under what conditions this will cause issues, and basically whether I should heed the warning or not.
Edit As a point of interest, Anaconda literally disappeared from my machine completely when I added it's directory to PATH. It left behind the terminal, which itself could not find conda when I did conda --version or where conda.
Heed the warning, don't add the anaconda path to %PATH%. There may be commands with similar names. Just use the anaconda cmd line prompt. When I use the anaconda prompt I just use 4 commands
python ...
taskkill /f /im python.exe - kill python programs when they hang up
dir -- same as git bash
cd -- same as git bash
You can always type help to see what other built-in windows commands are available or follow the command name with /? to find out what the parameters are. A bit like man pages on unix.
If you look at the anaconda prompt, what it does is
%windir%\System32\cmd.exe "/K" C:\ProgramData\Anaconda3\Scripts\activate.bat C:\ProgramData\Anaconda3
The problem with just using git bash is it only knows about its own little world and nothing outside it. It is just a way of bringing in *nix tools to Windows for the people who are accustomed to *nix commands can't be bothered to find out what the windows equivalents are (sometimes there aren't any).
Some of the commands (like ln -s) don't work. Since git bash is a *nix prompt, it does not understand windows commands so it won't be able to use this. The only other way is to use the anaconda command prompt. It is a windows command line. If you just wish to type pip and conda commands, the anaconda cmd prompt has been set up to run them. You don't need to learn anything else.

With conda/anaconda should I work in (base) all the time?

This is a total newbie question, but I installed conda along with a bunch of other stuff like Tensorflow with a virtual environment, and now I'm a little confused. If I open a terminal my python is pointing to /usr/bin/python. If I go to the base environment ("conda activate base"), my path points to where I installed anaconda, /usr/local/anaconda2. Does conda expect that I always go into the base environment to use all the anaconda packages, or did I do something weird on my install that my non-environment still points to the original system python? Should I just change my path to point to anaconda? I'm on Ubuntu 18.10.
I am a fan of being explicit -- instead of running python at the command line, I use the full path to the python interpreter when running scripts (e.g. /usr/local/anaconda2/bin/python) -- this way I don't rely on the shell behavior or having to worry about which environment is currently active. Running the interpreter using its full path is no different than just running python while the environment is active.
But that's not everbody's cup of tea... So...
Even if anaconda did not prompt you to update your PATH variable, you can do it explicitly by updating your .bashrc or .bash_profile (or .zshrc or whatever else) by adding this line: PATH=/usr/local/anaconda2/bin${PATH}. Dont forget to re-source the .bashrc or close/reopen your shell for it to take effect.
Conda is capable of creating many virtual environments beyond the base, so if you have more than one and you want to set one as the default, then just update the line above for the appropriate path.

In conda, what is the differece between "base" environment and no environment at all?

The title says it. I installed conda and now all my terminals open in the base environment, "(base)" at the start of my prompts. If I type "conda deactivate" it drops out of base to someplace else, like no environment. How is this different from base?
(This question is a tangent from my other, asking if the expected workflow is for me to stay in base: With conda/anaconda should I work in (base) all the time?)
activating a conda environment is not much more than applying settings to your shell to use a specific python interpreter (and the modules and libs associated to that interpreter)
when you drop out of a conda environment, your shell reverts to the python interpreter determined by your $PATH environment variable -- generally speaking, this default is typically a non-conda environment and is usually the default python installed with the OS (if applicable)
As freude is saying, the obvious way to see this in action is to do which python as you activate/deactivate environments
When playing with python virtual environments in linux (and macOS), it is useful to use the command which python or which pip from the terminal. This command shows the path to the currently used python interpreter - that is the thing, together with the location of site packages, that differs one environment from another. The python environment is nothing else but a directory where you have a copy of your python interpreter and installed libraries. Switching from the (base) to the deactivated (base) implies switching from one python interpreter to another one - that may be checked using which.
In windows, the closest equivalent of which is where.
Short answer: convenience.
When base is activated: check out /anaconda3/bin/ you'll find all the binaries that will be included in the $PATH environment variable (try echo $PATH in your bash shell)
When base is NOT activated: basically you only have conda binary available to use by default. Once again, try echo $PATH in your bash shell to see the difference.
if your already in python interpreter, I usually use this command to show me which path to my Python libraries:
from pprint import pprint
import sys
pprint(sys.path)
Then it shows list of library directories your're working on like this:
['',
'C:\\Program '
'Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\\python310.zip',
'C:\\Program '
'Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\\DLLs',
'C:\\Program '
... etc.
The pprint module is used to make output readable.

PATH not updated correctly from conda activate in VSCode's terminal

I am using VSCodes terminal pane. I activate a conda environment. For some reason, the python command is still set to /usr/bin/python, instead of the correct path to the conda environment.
% conda activate myenv
% which python
/usr/bin/python
The correct anaconda environment directory does seem to be in the $PATH variable, but /usr/bin seems above it in priority.
When I open a standard terminal through the OS, the behavior is as I expect.
% conda activate myenv
% which python
/Users/cpl/anaconda3/envs/myenv/bin/python
Please note: I have already set the VSCode preferences key python.pythonPath to /Users/cpl/anaconda3/envs/myenv/bin/python, and I think that it works correctly. When I run a file through right-clicking and selecting Run Python File In Terminal, the correct python (from the conda environment) is executed. My problem is using the VSCode terminal directly to execute python.
My shell is zsh, and I am using OSX. Any advice?
This behavior is explained in the VSCode docs:
Why are there duplicate paths in the terminal's $PATH environment variable and/or why are they reversed?#
It sounds like VSCode will run your .zshrc twice in MacOS, conflicting with the conda-generated PATH variable definitions.
There are two solutions listed in the link above. The one that works for me is to set the VSCode setting "terminal.integrated.inheritEnv": false. The documentation warns that all of your environmental variables will be stripped if you do this. However, I find I still have my custom variables defined in the .zshrc file.
It is worth noting that recent versions of VSCode will prompt you when it detects you are using a conda environment, and suggests making this change.
I don't use zsh, but I've run into this issue in bash and I believe the cause is the same.
Conda has recently changed the "official" method of activating environments, as described in this issue: https://github.com/Microsoft/vscode-python/issues/1882
Before, you needed to modify your .bashrc/.zshrc to prepend PATH with the directory of conda's activate script, and then activate specific environments by typing source activate name_of_env. VSCode-Python activates conda terminals by sending this command to the shell — with visible echo, like you typed it yourself.
The new method is to source $HOME/anaconda3/etc/profile.d/conda.sh in .bashrc and then activate environments with conda activate name_of_env, which is the behavior you're seeing work correctly in a dedicated terminal. VSCode-Python does not yet support this, and there appear to be issues with cross-platform support on Windows that are complicating the transition.
The best solution for now is to ignore the "correct" method of conda activate and consistently use the older source activate name_of_env, which still works (if your PATH is set to include $HOME/anaconda3/bin).
A dark magic below may work:-
In my Big Sur, I added the following empty entry to my settings.json -- can be found at File(Windows)/Code(Mac)>Preferences>Settings -- click on any link stated "Edit in settings.json"
"terminal.integrated.env.osx": {
"PATH": ""
}
All the best!
If anyone else in the future ends up scratching their heads over this particular problem, I’ve found another culprit:
Terminal>Integrated>Env: Enable Persistent Sessions
I suspect what happens is that after you update system paths, VScode caches the old path in the terminal and persists it. In this case it persists the old python path rather than the new conda one.
Toggling this option off and restarting VSCode clears that cache, and the new path is loaded in. You can also toggle the option back on after you’re done.

How to set the default python path for anaconda on Linux?

I have installed anaconda on a Linux machine. I noticed that after deactivating the anaconda environment with:
source deactivate
When running:
which python
I get:
/home/user/anaconda/bin/python
Instead of
/usr/bin/python
How can I restore this setting?
The comments somewhat cover the answer to the question, but to clarify:
When you installed Anaconda you must have agreed to have it added to your PATH. You'll want to check in your ~/.bash* files and look for any export PATH= lines to check this. So Anaconda is always on your path. The source deactivate command will only deactivate "sub" Conda environments. It will never remove what is called the "root" Conda environment (the one you originally installed). If you don't want Anaconda on your PATH by default then remove it from your ~/.bash* startup files. Then when you want to use Anaconda you'll need to add it to your PATH. Or just add the specific Conda environment you are interested in to your PATH directly, and don't worry about the activate and deactivate scripts. At their core all they do is modify PATH.
I hope that helps clarify things.
Anaconda comes with its own everything, and they ask if you wish to use their software as a default when you install it by adding their bin first to your PATH variable. If you do that, you can only manually remove it later from .bashrc to undo this action.
I chose not to do it, but i made a shell script to start spyder and use the anaconda distribution when i wish to, without altering my PATH by calling spyder like this from the shell script:
PATH=/home/<... path to where i installed anaconda>/bin:$PATH spyder &
This means that i am adding their distribution's bin to the path only for the extent of running that command (spyder), otherwise my environment is unaffected by anaconda.
If i wish to add things to it, i pass an option to the shell when i source it and that triggers these actions:
PATH=/home/<... path to where i installed anaconda>/bin:$PATH
PS1='\[\033[1;34m\](A)\w:\[\033[0m\] '
so that i see (with colors!) that in this terminal i am using an altered PATH, the one with python3 and such from anaconda, etc... When done, i kill the terminal! :)
To get back to the former default, I comment out the entry setting the pathbin the .bashrc file and get these results:
$ which python
/usr/bin/python
$ python -v
python 2.7.*
If you have installed something, you want to use it before older installs, in general.
Your path is well set: your install is searched before system install.
If you want to use system python, run % /usr/bin/python.
If you don't want anaconda, just trash it (or move it somewhere ouside of path).
This may help:
% which -a python.

Categories

Resources