Does `anaconda` create a separate PYTHONPATH variable for each new environment? - python

I am starting to work with the Python Anaconda distribution from Continuum.io to do scipy work.
I have been able to get Anaconda up and running, but I cannot tell whether Anaconda creates a new PYTHONPATH environment variable for each new environment it creates, or whether it relies on the common system PYTHONPATH.
I could not find any information on this in the documentation.
Further, when I did a printenv, I did not see a PYTHONPATH variable in the newly created environment --though I did find a few new anaconda created environment variables.
The best I can find is that Anaconda added some Anaconda directories and the new environment directory to the head of PATH variable --but this does not necessarily isolate the new package from the system environment but it is close.
Does anyone know the answer to this question or found a way to deal with this concern?

Anaconda does not use the PYTHONPATH. One should however note that if the PYTHONPATH is set it could be used to load a library that is not in the anaconda environment. That is why before activating an environment it might be good to do a
unset PYTHONPATH
For instance this PYTHONPATH points to an incorrect pandas lib:
export PYTHONPATH=/home/john/share/usr/anaconda/lib/python
source activate anaconda-2.7
python
>>>> import pandas as pd
/home/john/share/usr/lib/python/pandas-0.12.0-py2.7-linux-x86_64.egg/pandas/hashtable.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/john/share/usr/lib/python/pandas-0.12.0-py2.7-linux-x86_64.egg/pandas/__init__.py", line 6, in <module>
from . import hashtable, tslib, lib
ImportError: /home/john/share/usr/lib/python/pandas-0.12.0-py2.7-linux-x86_64.egg/pandas/hashtable.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
unsetting the PYTHONPATH prevents the wrong pandas lib from being loaded:
unset PYTHONPATH
source activate anaconda-2.7
python
>>>> import pandas as pd
>>>>

No, the only thing that needs to be modified for an Anaconda environment is the PATH (so that it gets the right Python from the environment bin/ directory, or Scripts\ on Windows).
The way Anaconda environments work is that they hard link everything that is installed into the environment. For all intents and purposes, this means that each environment is a completely separate installation of Python and all the packages. By using hard links, this is done efficiently. Thus, there's no need to mess with PYTHONPATH because the Python binary in the environment already searches the site-packages in the environment, and the lib of the environment, and so on.

Related

Path to conda environment as a variable in script

I am using Spyder and I am working with a virtual environment created with conda create, which is located in C:/Users/username/.conda/envs/...
To deal with GDAL proj.db error (proj_create_from_database: Cannot find proj.db) I need to set:
os.environ['PROJ_LIB'] = "C:/Users/username/.conda/envs/test/Library/share/proj"
Because I am not writing the script only for me, I would like to write a line to find a path to the current conda environment and save it as 'conda_env_path' variable (in my case C:/Users/username/.conda/envs/test).
All the answers I found say about finding path to environment in anaconda prompt, not as a line of a script. Thank you.

Tell Atom+Hydrogen to look in virtual enviromnment for packages

I'm completely new to this. I have set up Python3 and Atom and installed Hydrogen for Atom so I can run each line of my code and see the output.
I have set up a virtual environment and added packages to it.
My problem is that inside my Atom .py file, when I say import numpy as np for example, it tells me that the module is not found. So I think it is looking in some default place and not inside my virtual environment. Which makes sense as I don't know how to tell it to look inside the virtual environment.
I know that inside terminal I can load the virtual environment and then call the .py file from there and it will look in the right place. However that is not what I want to do. I want to be able to tell it to look in the virtual environment in the top line of the code and execute using Hydrogen, and then load the packages I want using Hydrogen, and then carry on with each line of code after that using Hydrogen.
Can someone tell me how to tell python to look in a specific virtual environment for the duration of the .py file that is being developed/executed?
For our purpose here virtual environments are just changing the search path of your interpreter.
Therefore if we want to search in a given virtual environment we can just add the path of this environment to our search path, which you can do in python by using
import sys
sys.path.append('/path/to/virtualenv')
The path to your virtual environment depends on how you configured it, but usually they are stored in a subfolder of your home directory called .virtualenvs, so this would probably look like
import sys
sys.path.append('/home/username/.virtualenvs/EnvName/')
# rest of code
Also note that this does not change your system path or pythonpath environment variables, and therefore only lasts for the duration of this python interpreter instance.

Python uses "pip install"ed site-packages over default ones in working directory

I accidentally pip installed a package from my local working directory to my main conda environment which I was using before without the problems mentioned below. Now, when I make changes in files, I must pip install it again to be able to use new functions. Reason for that is, that my PyCharm is now calling the module under site-packages instead of the ones in local directory
Another problem is debugging. As I open an original function file from the module in local working directory and set a breakpoint, it is being ignored and I can't debug because of the reason above.
Can I fix this problem, that it uses the module file in my local working directory and my local changes are directly seen by environment?
Thanks in advance!

Incude FreeCAD in system path for just one conda virtual environment

I want to be able to import FreeCAD into my python scripts, but only in one conda virtual environment. Is there a way to do this without adding FreeCAD to the path at the beginning of each file? I am using Pop!_OS, which should behave like Ubuntu here.
I already found that you can import FreeCAD, but the source I found did so by appending the FreeCAD library location at the beginning of the file: https://www.freecadweb.org/wiki/Embedding_FreeCAD. It looks like you could circumvent this problem by modifying your path variable, and I was able to do so on Windows in my workplace. I just want to do this only for a particular conda virtual environment.
Ideally,
import FreeCAD
will work in a special virtual environment, but not in others.
As mentioned, I got the import statement to work on Windows already by adding FreeCAD's directory to the PATH environment variable. It worked with the default python in command prompt, which should be the anaconda installation, so I think it works in all virtual environments. On Linux, though, I cannot import FreeCAD in python even when I use
PATH=$PATH:/usr/lib/freecad-python3/lib/
which I got from "locate FreeCAD.so" . I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'FreeCAD'
It would be really nice to be able to run the same code on both operating systems, and have the PATH modification confined to one virtual environment.
Conda does not look for packages from the PATH environment. Check this answer for the details. But first check whether your package can be installed using pip or conda.
You might go to the virtualenv site packages dir and add the path to the freecad into easy_install.pth

Unable to load DLL python module in PyCharm. Works fine in IPython

When I use the IPython included with Enthought Python Distribution, I can import the pyvision package just fine. However, when I try to import pyvision inside of PyCharm 1.2.1, I get the following errors
File "C:\Python27\lib\site-packages\pyvision\__init__.py", line 146, in <module>
from pyvision.types.img import Image,OpenCVToNumpy,NumpyToOpenCV
File "C:\Python27\lib\site-packages\pyvision\types\img.py", line 43, in <module>
import numpy
File "C:\Python27\lib\site-packages\numpy\__init__.py", line 142, in <module>
import add_newdocs
File "C:\Python27\lib\site-packages\numpy\add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "C:\Python27\lib\site-packages\numpy\lib\__init__.py", line 13, in <module>
from polynomial import *
File "C:\Python27\lib\site-packages\numpy\lib\polynomial.py", line 17, in <module>
from numpy.linalg import eigvals, lstsq
File "C:\Python27\lib\site-packages\numpy\linalg\__init__.py", line 48, in <module>
from linalg import *
File "C:\Python27\lib\site-packages\numpy\linalg\linalg.py", line 23, in <module>
from numpy.linalg import lapack_lite
ImportError: DLL load failed: The specified module could not be found.
Am I missing some path settings in Windows?
I had the same problem. I'm using Winpython32 and trying to import win32com. Worked everywhere (I tried) except in PyCharm. sys.path and os.environ['PYTHONPATH'] had some extra entries inside Pycharm, but nothing is missing compared to when run elsewhere.
The solution was to start Pycharm within the Winpython console and not using the shortcut.
sys.path and os.environ['PYTHONPATH'] did not change. os.environ['PATH'] had several additional entries set, all related to the python installation. At this point I suspect it has to do with "non-standard" installations. Winpython32 tries to be "portable", while other reports of similar problems are when using Enthought or Python(x,y).
Manually adding:
C:\WinPython-32\python-2.7.6\
C:\WinPython-32\python-2.7.6\DLLs
C:\WinPython-32\python-2.7.6\Scripts
to the system path (the global PATH environment variable in Windows) solved the problem without having to run Pycharm within the Winpython command line. Note: C:\WinPython-32\python-2.7.6\Scripts alone did not solve it.
I've had that problem before, and it seemed to get fixed by repairing Enthought.
EDIT: I just checked, one of my f2py projects was still suffering from this exact error. Repairing Enthought did not work. The solution to my problem actually lay in fixing the Windows path variable. You need to make sure c:\Python27\Scripts (or your equivalent) is in the path. Furthermore, and this is VERY important, make sure each entry in the global and user path environment variables has NO trailing slashes. This breaks the GNU make utility on Windows.
Add to your PATH environment variable
C:\Python27
C:\Python27\DLLs
C:\Python27\Scripts
This is a pretty frustrating bug in PyCharm. Even if you set your virtualenv from within PyCharm, the "python console" defaults to the system python. When you installed PyCharm, presumably you used a win32 python on a 64 bit machine.
Go to file>settings>Build, Execution, Deployment>Console>Python Console and change the Python Interpreter from the system version to your virtualenv.
Of course, PyCharm doesn't immediately refresh it. You have to close your project and reopen it.
To verify this was successful, open the Python Console (Tools>Python Console) and check the very first line of the output: it should point to the python.exe of your virtual environment, not the system python.
I have the same problem (with another package) and I don't want to modify windows path.
I could solve it very roughly in the pycharm console, by creating a python script which updates sys.path and os.environ['PATH'] in file -settings-console-pathon console - starting script.It works only when I use "tools- python shell"
Still it doesn't work in the run options. I even tried to create a virtual environment, add the path to the activate.bat but in pycharm it doesn't work (from command prompt it does) (as suggested in virtualenv that can find relocated libraires (like mysqlclient lib for MySQLdb) )
I explicilty set the path in the python interpreter option but it doesn't work.(as suggested in https://stackoverflow.com/a/24206781/1136458)
I couldn't find the link but I saved this pic - hopefully works for you
I apologize for my explanation being long and probably not the best of clarity, but this is the best I could do to describe my experience.
I was having the same problem after first installation and this is how I solved it:
I had noticed there are some settings, as indicated in other answers, that tell pyCharm which interpreters and environment managers to use and I was sure the problem was with setting these options, but I was not sure how, so I started searching.
Setting interpreter
I had more or less followed the standard tutorial, assuming in some point I would have used newly created virtualenv inside my project folder. I was initially getting the DLL error, but as I made sure the interpreter settings were pointing at a valid executable.
Here it is explained how to do this.
Setting environment
at this point the error disappeared, but I had progressed towards a new error.
The problem at this point I was not being able to import any module as they were not found.
This is because I was working inside the newly created virtualenv, basically a pristine installation, with no modules installed. I am sure there are more evolved solutions (install modules in the virtualenv), but I was just looking for making the code to work, so I set the environment to use my anaconda usual development environment.
For me, with conda on windows (after activating the development environment with conda activate) then (see ..../anaconda-python-where-are-the-virtual-environments-stored for other systems).
where python gives the path to the interpreter
conda info --envs gives me the existing environment that I want to use.
This fixed everything for me.

Categories

Resources