I use cygwin on Windows 10, and want to write Python code
that uses Pandas and other packages contained in my Anaconda base
environment. I can run Python code if I give the absolute
path to the code, but not if bash needs to use PATH
to find the code.
I use these lines in my .bash_profile as suggested
here (search for 'bash_profile'):
. /opt/conda/etc/profile.d/conda.sh
conda activate base
Here's the start of my PATH, showing Anaconda stuff at the front:
$ echo ${PATH:1:160}
cygdrive/c/Users/Glenn/Anaconda3:/cygdrive/c/Users/Glenn/Anaconda3/Library/mingw-w64/bin:/cygdrive/c/Users/Glenn/Anaconda3/Library/usr/bin:/cygdrive/c/Users/Gle...
I can run Anaconda Python at the bash prompt:
$ which python
/cygdrive/c/Users/Glenn/Anaconda3/python
$ python -i
Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>
I can even run a Python program that uses Pandas:
$ pwd
/home/Glenn/bin
$ cat test.py
#!/usr/bin/env python
import pandas as pd
print(pd.Series([1,2]))
$ ./test.py
0 1
1 2
dtype: int64
The problem is that I can't run the program if I rely on the path:
$ which test.py
/home/Glenn/bin/test.py
$ test.py
C:\Users\Glenn\Anaconda3\python.exe: can't open file '/home/Glenn/bin/test.py': [Errno 2] No such file or directory
$
I get the same kind of error message if I invoke test.py from
another directory using an absolute pathname, but not if I invoke
it using a relative pathname.
I'm a longtime Cygwin user and haven't seen this kind of behavior
before, so suspect it has to do with Anaconda.
I can change to any other directory and get the same behavior.
Also, if I put the absolute path to Anaconda's Python in the
shebang line, I get the same behavior.
I've found (and tried) many suggestions for configuring Anaconda in a Cygwin environment, but many of them are 4 or 5 years old.
UPDATE: I was not able to solve this problem, but have switched to the Linux Windows Subsystem and no longer have to deal with the issue.
Related
I am trying to import pyodbc module on a windows computer. It works in the terminal, but not the IDLE. The error message in IDLE is:
Traceback (most recent call last):
File "FilePath/Filename.py", line 3, in <module>
import pyodbc
ImportError: No module named pyodbc
This typically occurs when multiple versions of python are installed with different paths. You can check to see if you have multiple installations by opening up the IDLE terminal and using
import sys
sys.version
sys.path
These commands will print the system PATH and version of the current instance of python. Use this in both IDLE and the command line terminal to see where each differ. Once you know which version is the one you want then just remove the other. You could also remove all python instances and then reinstall a clean python environment but then you would have to re-install all of your modules using pip or easy_install
Open python in cmd (type python and press enter)
Import the module in cmd (type import modulename)
Type modulename.__file__
You will get the path where the module is stored
Copy the corresponding folder
In IDLE, import sys and typing sys.executable to get the paths where it looks for modules to import
Paste your module's folder in the path where IDLE looks for modules.
This method worked for me.
You can pip show after install package and know about location where package installed.
After that check in IDLE sys.path and if directory with package not in sys.path try to add it.
import sys
sys.path.append("/home/dm/.local/lib/python3.6/site-packages")
# or another folder that `pip show` about package.
this happen because of multiple python installed (32bit version, 64bit version) or 3v and 2.7v so to solve this problem you have to invoke the idle for that specific version like this
cd to the dir of the version that the import work fine in cmd in that folder type this command below
pythonw.exe Lib\idlelib\idle.pyw
this command will invoke idle for that version
and the import will work fine
Me too had the same issue while trying to import a module which was successfully imported on terminal and not able to install on IDLE.
How I fixed?
Assuming you know how to execute commands on terminal as well as inside of python interpreter
Open your Terminal & execute the below commands :
:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> import sys
>>> sys.version
'3.6.9 (default, Jan 26 2021, 15:33:00) \n[GCC 8.4.0]'
>>> sys.path
['', '/usr/lib/python36.zip', '/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-
packages', '/usr/lib/python3/dist-packages']
>>>
Now import your module inside of your python3 interpreter.
>>> import nester
>>>
>>> nester.__file__
'/usr/local/lib/python3.6/dist-packages/nester.py'
>>>
Open your IDLE and run the below commands and compare them
Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more
information.
>>> import sys
>>> sys.version
'3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit
(AMD64)]'
>>> sys.path
['','C:\Users\username\AppData\Local\Programs\Python\Python39\Lib\idlelib', 'C:\Users\username\AppData\Local\Programs\Python\Python39\python39.zip', 'C:\Users\username\AppData\Local\Programs\Python\Python39\DLLs', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib', 'C:\Users\username\AppData\Local\Programs\Python\Python39', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib\site-packages']
>>> sys.executable
'C:\Users\username\AppData\Local\Programs\Python\Python39\pythonw.exe'
Now if you compare both outputs from Terminal & IDLE,
Terminal Module location is different from IDLE
I was using Ubuntu 18 terminal on windows machine
So I just copied my file to 'C' directory and ensured its file privileges. That's it.
:~$ cp -p /usr/local/lib/python3.6/dist-packages/nester.py /mnt/c/Users/username/AppData/Local/Programs/Python/Python39/Lib/
It worked!!
I Found the solution. It works for me
The problem is your installation directory does not match with the python version directory.
solution is >>>
type %localappdata% in your search bar then go to this folder.
here select the program folder. then select Programs , Python , Python version , Scripts
copy the location of the Scripts folder
open command prompt and type cd //yourpath (in my case cd C:\Users\3C HOUSE\AppData\Local\Programs\Python\Python37\Scripts)
if you wanna install numpy , now run pip install numpy
When you put your python scripts that have import pandas in the same folder as the site packages like pandas for example and use the same version of python that is used on CMD, it should help run your scripts in IDLE.
Check the path of your code, and that of the module. Copying the module to the path where code is worked for me.
'sys.executable' will give the path where code is stored.
For windows, open command prompt and enter pip show pyodbc to get the path of package and copy the path.
then open idle and run these lines
import sys
sys.path
Match the path from command prompt and the paths mentioned in the list provided by running above lines in IDLE. If the path is not mentioned then run these lines in idle
sys.path.append("Enter the copied path of package here")
After executing these lines, check again by importing the package that if it works for you.
I'm wondering if someone can explain the following strange behavior:
I have a python script using os.environ to pull environment variables for use with pysftp. The variable in question has been called "sftp_wd" and "sftp_wdir" with a value "/etc/<rest of path>" and the effect is the same for both.
>>> import os
>>> os.environ["sftp_wdir"]
This python code in cmd yields the correct value: /etc/...
But when run in git bash the value looks like: <%USERPROFILE%/AppData/Local/Programs/Git>/etc/...
I'm using the same conda environment in both cases and the effect shows up when run in the vscode debugger as well. Additionally if running python with winpty I don't see this effect. I've looked at the documentation on os.environ and for environment variables used by git bash and not found anything to explain this.
printenv also displays the correct value so I'm inclined to believe this is specific to how git bash is initializing python but I can't seem to pin down what's actually happening
Edit: In response to #CharlesDuffy it seems this can be replicated with any environment variable and any value starting with "/"
example I created an environment variable called random with a value of /var/. Results are as follows:
echo
$ echo $random
/var/
printf
$ printf '%q\n' "$random"
/var/
python
$ python
Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)] :: Ana
conda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ['random']
'C:/Program Files/Git/var/'
>>>
Note: The change in the path to the "Git" folder from above is from updating to v2.33.0.2-64bit of Git Bash. The issue is still the same though.
This is behavior of msys, the Windows-compatibility library that Git Bash is compiled with. The documentation covering this behavior is at https://www.msys2.org/docs/filesystem-paths/
An environment variable is available to turn it off:
To turn off automatic path conversion in environment variables entirely, export MSYS2_ENV_CONV_EXCL='*'
To turn it off only for the variable named random: export MSYS2_ENV_CONV_EXCL=random
My default Python binary is set to the one with the Anaconda distribution of Python. This is found at /home/karnivaurus/anaconda/bin/python, and I have made this the default by adding to my .bashrc file the following: export PATH=/home/karnivaurus/anaconda/bin:$PATH.
I also have a Python package called caffe, which is located at /home/karnivaurus/caffe/distribute/python, and I have added this to the package search path by adding to my .bashrc file the following: export PYTHONPATH=${PYTHONPATH}:/home/karnivaurus/caffe/distribute/python.
Now, I have a simple Python file, called test.py, with the following contents:
import caffe
print "Done."
If I run this by entering python test.py into the terminal, it runs fine, printing out "Done.". The problem I am having is when I run this in the PyCharm IDE. In PyCharm, I have set the interpreter to be /home/karnivaurus/anaconda/bin/python. But when I open test.py in PyCharm, and run the file in the IDE, I get the following error:
ImportError: No module named caffe
So my question is: Why can PyCharm not find the caffe module when it runs the Python script, but it can be found when I run the script from the terminal?
There are a few things that can cause this. To debug, please modify your test.py like so:
# Is it the same python interpreter?
import sys
print(sys.executable)
# Is it the same working directory?
import os
print(os.getcwd())
# Are there any discrepancies in sys.path?
# this is the list python searches, sequentially, for import locations
# some environment variables can fcuk with this list
print(sys.path)
import caffe
print "Done."
Try again in both situations to find the discrepancy in the runtime environment.
edit: there was a discrepancy in sys.path caused by PYTHONPATH environment variable. This was set in the shell via .bashrc file, but not set in PyCharm's runtime environment configuration.
For an additional option, you can use pycharm by terminal. And export the corresponding environment paths beforehand. This works for me. And I think it's better than make some changes in the code. You gonna need run the code by terminal after your debugging.
For example, in terminal type:
$ export LD_LIBRARY_PATH=~/build_master_release/lib:/usr/local/cudnn/v5/lib64:~/anaconda2/lib:$LD_LIBRARY_PATH
$ export PYTHONPATH=~/build_master_release/python:$PYTHONPATH
Then run pycharm by charm (pycharm can be soft linked by charm bash):
$ charm
Well this may be a redundant answer, however I think it's important to explicitly called out what causes this error.
It happened to me many times and I got it fixed by making sure that IDE ( pycharm or vscode or any other) is set to same working directory where the code resided.
for example : I have two files train.py and config.py in mlproject/src directory. I'm trying to run import config in train.py
**When run in /mlproject/ directory, I get error when try to import config **
(ml) dude#vscode101:~/mlproject$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'config'
>>>
When run in /mlproject/src/` directory, I'm able to successfully import config
(ml) dude#vscode101:~/mlproject/src$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
>>>
I am trying to import pyodbc module on a windows computer. It works in the terminal, but not the IDLE. The error message in IDLE is:
Traceback (most recent call last):
File "FilePath/Filename.py", line 3, in <module>
import pyodbc
ImportError: No module named pyodbc
This typically occurs when multiple versions of python are installed with different paths. You can check to see if you have multiple installations by opening up the IDLE terminal and using
import sys
sys.version
sys.path
These commands will print the system PATH and version of the current instance of python. Use this in both IDLE and the command line terminal to see where each differ. Once you know which version is the one you want then just remove the other. You could also remove all python instances and then reinstall a clean python environment but then you would have to re-install all of your modules using pip or easy_install
Open python in cmd (type python and press enter)
Import the module in cmd (type import modulename)
Type modulename.__file__
You will get the path where the module is stored
Copy the corresponding folder
In IDLE, import sys and typing sys.executable to get the paths where it looks for modules to import
Paste your module's folder in the path where IDLE looks for modules.
This method worked for me.
You can pip show after install package and know about location where package installed.
After that check in IDLE sys.path and if directory with package not in sys.path try to add it.
import sys
sys.path.append("/home/dm/.local/lib/python3.6/site-packages")
# or another folder that `pip show` about package.
this happen because of multiple python installed (32bit version, 64bit version) or 3v and 2.7v so to solve this problem you have to invoke the idle for that specific version like this
cd to the dir of the version that the import work fine in cmd in that folder type this command below
pythonw.exe Lib\idlelib\idle.pyw
this command will invoke idle for that version
and the import will work fine
Me too had the same issue while trying to import a module which was successfully imported on terminal and not able to install on IDLE.
How I fixed?
Assuming you know how to execute commands on terminal as well as inside of python interpreter
Open your Terminal & execute the below commands :
:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> import sys
>>> sys.version
'3.6.9 (default, Jan 26 2021, 15:33:00) \n[GCC 8.4.0]'
>>> sys.path
['', '/usr/lib/python36.zip', '/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-
packages', '/usr/lib/python3/dist-packages']
>>>
Now import your module inside of your python3 interpreter.
>>> import nester
>>>
>>> nester.__file__
'/usr/local/lib/python3.6/dist-packages/nester.py'
>>>
Open your IDLE and run the below commands and compare them
Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more
information.
>>> import sys
>>> sys.version
'3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit
(AMD64)]'
>>> sys.path
['','C:\Users\username\AppData\Local\Programs\Python\Python39\Lib\idlelib', 'C:\Users\username\AppData\Local\Programs\Python\Python39\python39.zip', 'C:\Users\username\AppData\Local\Programs\Python\Python39\DLLs', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib', 'C:\Users\username\AppData\Local\Programs\Python\Python39', 'C:\Users\username\AppData\Local\Programs\Python\Python39\lib\site-packages']
>>> sys.executable
'C:\Users\username\AppData\Local\Programs\Python\Python39\pythonw.exe'
Now if you compare both outputs from Terminal & IDLE,
Terminal Module location is different from IDLE
I was using Ubuntu 18 terminal on windows machine
So I just copied my file to 'C' directory and ensured its file privileges. That's it.
:~$ cp -p /usr/local/lib/python3.6/dist-packages/nester.py /mnt/c/Users/username/AppData/Local/Programs/Python/Python39/Lib/
It worked!!
I Found the solution. It works for me
The problem is your installation directory does not match with the python version directory.
solution is >>>
type %localappdata% in your search bar then go to this folder.
here select the program folder. then select Programs , Python , Python version , Scripts
copy the location of the Scripts folder
open command prompt and type cd //yourpath (in my case cd C:\Users\3C HOUSE\AppData\Local\Programs\Python\Python37\Scripts)
if you wanna install numpy , now run pip install numpy
When you put your python scripts that have import pandas in the same folder as the site packages like pandas for example and use the same version of python that is used on CMD, it should help run your scripts in IDLE.
Check the path of your code, and that of the module. Copying the module to the path where code is worked for me.
'sys.executable' will give the path where code is stored.
For windows, open command prompt and enter pip show pyodbc to get the path of package and copy the path.
then open idle and run these lines
import sys
sys.path
Match the path from command prompt and the paths mentioned in the list provided by running above lines in IDLE. If the path is not mentioned then run these lines in idle
sys.path.append("Enter the copied path of package here")
After executing these lines, check again by importing the package that if it works for you.
I am confused in trying to understand which python version is used when I run them in interpreter?
That is, how the shell decides which python version to load when I have more than one versions.
I have snapshot of my /usr/bin. It has atleast two different python versions2.5 and 2.6.
I have not specified anything in my .bashrc to choose which one from.
when I execute python command on bash shell, python interpreter opens up, but loads with version 2.6.
How is this detected by shell?
If I want to change versions, how will I do that? That is, I want to change the default version
lastly what kind of files are these in /usr/bin/python. I tried to open them in editor and I see some ###. I have read many posts here but some makes me confusing.
ls /usr/bin/python
python python2.5 python2.6 pythonw pythonw2.6
python-config python2.5-config python2.6-config pythonw2.5
python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
To know which version is used by default, type
which python
To use another one, use the full path when invoking them:
/usr/bin/python/python2.5 myfile.py
/usr/bin/python/python2.6 myfile.py
etc.
If you want to change the default, you can create an alias in ~/.bashrc for example:
my_python='/usr/bin/python/python2.7'
or better change the link direction of /usr/bin/python (or whatever comes from which python). It is a link, so link it to the exact version you need.
ln -s /prefered/path/of/python /usr/bin/python
The files you see in /usr/bin/python are executables. You can do the following to see this:
file /usr/bin/python/python2.5
In my case:
$ file /usr/bin/python
/usr/bin/python: symbolic link to `python2.7'
$ file /usr/bin/python2.7
/usr/bin/python2.7: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0xf66a10f2c444b2329b25ab6790abb7fbb4fe3f78, stripped
python --version
Then head on over to your .bashrc (should be in your home directory) and add:
alias python='[your path]'
for me I have:
alias python='/usr/bin/python2.7'
Just make sure you say:
source ./bashrc
to apply your changes.