I am using Spyder and trying to add /usr/local/lib/python3.7/site-packages to the PYTHONPATH Manager. However, I receive an error informing me "This directory cannot be added to PATH. If you want to set a different Python interpreter, please go to Preferences > Main Interpreter".
However, I have already changed my interpreter to point to /usr/bin/python3
At the moment, I am using the rather annoying work around of putting the following at the top of all my code.
import sys
sys.path.append("/usr/local/lib/python3.7/site-packages")
Typing the following gives me the below. Is there a way which I can even ensure after running pip3 install XXX in the terminal, that the packages are downloaded somewhere such as the below?
for p in sys.path: print(p)
/Users/user
/usr/local/lib/python3.7
/Users/user/opt/anaconda3/lib/python37.zip
/Users/user/opt/anaconda3/lib/python3.7
/Users/user/opt/anaconda3/lib/python3.7/lib-dynload
/Users/user/opt/anaconda3/lib/python3.7/site-packages
/Users/user/opt/anaconda3/lib/python3.7/site-packages/aeosa
/Users/user/opt/anaconda3/lib/python3.7/site-packages/IPython/extensions
/Users/user/.ipython
Alternatively, and preferably, advice on how to add the above site-packages directory to my PATH? I feel I am missing something obvious.
(Spyder maintainer here) We forbid adding site-packages directories through our PYTHONPATH manager because it allows people to mix two different Python versions (which is what you're trying to do by adding your system site-packages to your Anaconda's Python).
And we do that because it usually generates odd errors and segfaults for binary packages such as Numpy, Pandas and Matplotlib, given that binary packages for one Python version are incompatible with packages for another one.
Finally, even though you found a workaround for that (by using sys.path), we strongly suggest you to stop doing that because it'll give you nothing by headaches in the future.
Doing what you are asking isn't the recommended path forward but you can solve the underlying problem in either of the following ways (A or B).
To "ensure pip installs packages to another location which Spyder can see" as the asker guessed in a comment on the accepted answer which got no answer (Method B below) is usually not a good idea. Keeping a clean environment for Spyder will ensure that you can determine requirements (including package version) for each of your projects reliably. Therefore, do the reverse of what you guessed: Ensure Spyder uses the Python interpreter in the environment where pip installed your project's required packages.
A. Change the Python interpreter
Go to Tools, Preferences, and set Python interpreter to the python executable that was used to install the package (If using a virtual environment, it would be your_other_env/bin/python).
Close and reopen Spyder (Spyder says to restart the IPython console, but it may not work in this case and show the error where Spyder cannot restart a kernel it didn't start).
Open Spyder again and run any py file. You will get an error that says to install the spyder-kernels package (for some reason pip 22.0.4 will only install spyder_kernels: This issue is at "spyder-kernels should be spyder_kernels" :edit: but the issue is invalid, so upgrade pip first such as via pip install --upgrade pip in your virtual environment). Take note of the version in the error, since that is the version you need.
If you are using conda or are on Windows the instructions will differ, so see Common Illnesses in the Spyder documentation instead of continuing this step.
source your_other_env/bin/activate
pip install --upgrade pip setuptools
pip install spyder-kernels=...
deactivate
but change ... to the version shown in your Spyder error from step 3. If you installed Spyder with conda as recommended, use the commands from the URL above instead.
I don't recommend Method B, as I've explained. However, it may be useful if you are manually installing Spyder plugins or test suites that apply to all projects but aren't in the requirements.txt or setup.py requirements for your project(s) (and therefore don't affect determining requirements for your users).
B. To "ensure pip installs packages to another location which Spyder can see" you would run "spyder_env/bin/python -m pip install ..." to install the package, where spyder_env is the virtualenv where Spyder is installed (but if Spyder is installed in the system using an installer or linux distro package, you may need to use your system's python such as via python3 -m pip install --user ... where ... is the package name. Always use --user instead of sudo or root to avoid mismatched files caused by mashing together the distro-packaged modules and your manually installed modules).
Related
I've been coding with R for quite a while but I want to start learning and using python more for its machine learning applications. However, I'm quite confused as to how to properly install packages and set up the whole working environment. Unlike R where I suppose most people just use RStudio and directly install packages with install.packages(), there seems to be a variety of ways this can be done in python, including pip install conda install and there is also the issue of doing it in the command prompt or one of the IDEs. I've downloaded python 3.8.5 and anaconda3 and some of my most burning questions right now are:
When to use which command for installing packages? (and also should I always do it in the command prompt aka cmd on windows instead of inside jupyter notebook)
How to navigate the cmd syntax/coding (for example the python documentation for installing packages has this piece of code: py -m pip install "SomeProject" but I am completely unfamiliar with this syntax and how to use it - so in the long run do I also have to learn what goes on in the command prompt or does most of the operations occur in the IDE and I mostly don't have to touch the cmd?)
How to set up a working directory of sorts (like setwd() in R) such that my .ipynb files can be saved to other directories or even better if I can just directly start my IDE from another file destination?
I've tried looking at some online resources but they mostly deal with coding basics and the python language instead of these technical aspects of the set up, so I would greatly appreciate some advice on how to navigate and set up the python working environment in general. Thanks a lot!
Python uses a different way of installing packages. Python has a thing named venv which stands for Virtual Environment. You install all of your packages in venv. Usually for each new project you make a new venv.
By using Anaconda on windows you install everything within the anaconda environment that you have specified.
python -m pip install "modulename" is a command that will install modulename to your default venv. You will be able to use this module when no other venv is specified. Here is the docs page. And here is a tutorial on how to use venv
By default python uses the same directory you have your code in. e.g. C:/Users/me/home/mypythonfile.py will run in C:/Users/me/home/ and will be able to access files in this directory. However you can use ../ to navigate directories or you can specify an absolute path to file you want to open e.g. with open("C:/system32/somesystemfile.sys") as file
Going over the technical differences of conda and pip:
So Conda is a packaging tool and installer that aims to do more than what pip does; handle library dependencies outside of the Python packages as well as the Python packages themselves. Both have many similar functionalities as well, you can install packages or create virtual environments with both.
It is generally advisable to generally have both conda and pip installed since there are some packages which might not be available with conda but with pip and vice versa.
The commands to install in both the ways is easy enough, but one thing to keep in mind is that
conda stores packages in the anaconda/pkgs directory
pip stores it in directory under /usr/local/bin/ for a Unix-based system, or \Program Files\ for Windows
You can use both pip or conda inside the jupyter notebook, it will work just fine, but it may be possible that you get multiple versions of the same package.
Most of the times, you will use cmd only to install a module used in your code, or to create environments, py -m pip install "SomeProject" here basically means that the module "SomeProject" will be downloaded in base env.
You could think of conda as python with a variety of additional functionalities, such as certain pre-installed packages and tools, such as spyder and jupyter. Hence, you must be precise when you say:
I've downloaded python 3.8.5 and anaconda3
Does it mean you installed python in your computer and then also anaconda?
In general, or at least in my opinion, using anaconda has advantages for development, but typically you'll just use a simple python installation in production (if that applies to you).
Anaconda has it's own package registry/repository . When you call conda install <package>, it will search for the package there and install it if available. You would better search it first, for instance matplotlib.
pip is a package manager for the Python Package Index. pip also ships with anaconda. Hence, in an anaconda environment you may install packages from either sources (either using pip install or conda install). For instance, pandas from PyPI and pandas from conda. There is no guarantee that packages exist in both sources. You must either search it first or simply try it.
In your first steps, I would suggest you to stick to only one dev env (either simple python or anaconda, recommend the second). Because that simplifies the question: "which python and which pip is executed in the cmd line?". That said, those commands should work as expected in any terminal, it be a simple cmd or an embedded one like in PyCharm or VS Code.
You could inspect that by running (on windows and linux at least):
which python, which pip.
Honestly, this is a question/answer that falls outside the scope of SO and for more info you would better check official websites, such as for anaconda or search for python vs anaconda blogs.
No matter whatever module I install, in VSCode, it just shows that no module is found.
If the requirement is already satisfied then why still not found? Please let me know...
Edit1:
Here's the pip version which is 3.10 but the version of python it is showing me is 3.9.7
Although you have selected the python310 environment in the third picture, it has no influence at all. Without the activate command such as Activate.ps1, activate.bat, it will automatically search the python and pip depending on the system environment variable of Path.
Obviously, from the second picture, we can know the pip and the python were not in the same environment. Either you mess up the order in the Path, or your python3.9.7 without pip installed.
Such as this can cause the python and pip not in the same environment.
You can move down the python3.9.7 in the Path to avoid it. But it's recommended to create a virtual environment. If you don't know how to create a virtual environment you can refer to here.
You have two versions of python installed, and you have used pip in one of them but not the one you are running.
if you want to ensure that you are running pip for the correct version, you can run:
python -m pip install pynput
I have follow, these steps, but
"apm install linter
Next, we’re going to install a Python Linter package, to help us detect errors in our Python code.
This package is called linter-flake8 and it’s an interface to flake8. To install it, you need to run:
pip install flake8
pip install flake8-docstrings
apm install linter-flake8
You must restart Atom to see the changes"
I have followed those steps and every package with PIP and APM were installed, however, corrections are not made on my python code in ATOM. Is there something else i need to configure or to do appart from steps i mentioned?
2
3
It could be your Environment Variable paths, especially if you have two concurrent versions of Python installed. Check to see which one is at the top of the list and ahead of the one your not currently developing within.
I'll try to give you an example of my installation
Installation path of python in my computer is
C:\Python36-32
So in control panel in system settings in advanced settings in path i add the following paths
C:\Python36-32
C:\Python36-32\Scripts
Now i reopen the cmd and i enter python if it worked you will see the python interpreter line.
To install flake8 system wide run the command
pip install flake8
In atom editor install the package linter-flake8.When package is installed in settings of package in package path add the full path of package for example in my configuration i have in path
C:\Python36-32\Scripts\flake8 without the .exe
Maybe you got installed flake8 on C:\Users\myuser\AppData\Roaming\Python\Python3X\Scripts just like me. If that is the case you only need to add that location to the PATH.
I encountered the following error message, when I was in python mode.
Error (jedi): Failed to start Jedi EPC server.
*** You may need to run "M-x jedi:install-server". ***
This could solve the problem especially if you haven't run the command yet
since Jedi.el installation or update and if the server complains about
Python module imports.
Then I tried M-x jedi:install-server. Then it complained that
Program named "virtualenv" does not exist
Now from the results I got from google, I added the following line to my emacs.d/init.el file, but still I am getting the same error. Please note that I am using Windows 7
(when (memq window-system '(mac ns))
(exec-path-from-shell-initialize))
Trying to set up jedi in emacs under Windows environment has caused me so much time and trouble already, I just want to complete this once and for all.
Ideally, I am looking for an answer that lists out all the steps I need to setup the virtualenv (on Windows 7) in emacs.
I had the same problem but on Windows 10. I got it working by these steps:
In terminal (cmd/powershell/..):
pip install virtualenv
After that, in emacs:
M-x jedi:install-server
And Emacs could install the server.
FYI, I'm running python3.5, pip version 19.2.3, emacs26.2, W10
I think it is clear that virtualenv is missing. So you need to do
just two things. With this way you don't bother with PATH variable and all
that.
install virtualenv however you feel comfortable, (I use conda package manager for everything python related and have the installation instructions for that, feel free to use pip or whatever)
conda activate yourenv
conda install virtualenv
conda list virtualenv #check if virtualenv is actually installed
Find where the virtualenv is located (as I use conda, I know the files are in bin folder in my installation)
and then show jedi where virtualenv is in emacs C-h v jedi:environment-virtualenv "~/path-to-conda-env/bin/virtualenv"). For more help look in the documentation in Configuration or
comment below.
M-x jedi:install-server in emacs
Test on a .py file.
Hello I'm trying to run twisted along with python but python cannot find twisted.
I did run $pip install twisted successfully but it is still not available.
ImportError: No module named twisted.internet.protocol
It seems that most people have $which python at /usr/local/bin/python
but I get /Library/Frameworks/Python.framework/Versions/2.7/bin/python
May this be the issue? If so, how can I change the PATH env?
It is just fine.
Python may be installed in multiple places in your computer.
When you get a new Mac, the default python directory may be
'usr/bin/python2.7'
You may also have a directory
'System/Library/Frameworks/Python.framework/Versions/2.7/bin/python'
The first one is the symlink of the second one.
If you use HomeBrew to install python, you may get a directory in
'usr/local/bin/python2.7'
You may also have a directory as
'Library/Frameworks/Python.framework/Versions/2.7/bin/python'
which is exactly where my directory is.
The difference between the second one and the fourth one, you may find it here
Installing Your Framework
In your question, as you mentioned pip install is successful, but the installed packages still not available. I may guess your pip directory is not in your default python directory, and the packages are installed where your pip directory is. (Please use 'which pip' to check it out)
For example, in my computer, the default pip directory is
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip
though, I have also pip in usr/local/bin.
So, all my packages installed via 'pip install' are stored in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Hope that resolves your doubt. Similar things have happened to me, and it took me a whole night to figure out.
Here is the solution:
Use PYTHONPATH="/usr/local/lib/python2.7/site-packages:$PYTHONPATH" to modify your python directory, or modify your pip directory.
However, I would recommend a better way, use virtualenv. This can isolates Python environments, and can help you easily set up packages for each project.
By the path your giving for OS X python I'm guessing your a rev-or-so old on your OS X (leopard?) so I can't directly compare with my machine.
But, adding packages to the base OS X install is always a touchy thing, one check I would recommend is the permissions on any packages you add. Do a ls -l /Library/Python/2.7/site-packages/ and make sure everything has r rights (and x rights for directories) (I.E. -rwxr-xr-x or drwxr-xr-x).
I had a recent case where a sudo pip wouldn't set user read rights on installed packages, and I believe "No module" was the error I was getting when I tried to use them
Because adding packages is so touchy on OS X, there are tons of guide on the net to doing hand installs of python. The first one I matched on a google is Installing / Updateing Python on OS X (use at your own risk, I personally haven't followed that guide)
(... the 3rd part install system Brew is a very common method for people to do automated installs of python as well)
Okay well in the terminal I finally found out:
open .bash_profile located at your user root (simply do a $cd in terminal to go there) and add where the path is the location of twisted
PYTHONPATH="/usr/local/lib/python2.7/site-packages:$PYTHONPATH"
export PYTHONPATH
I too was getting a ImportError: No module named xxxeven though I did a pip install xxx and pip2 install xxx.
pip2.7 install xxx worked for me. This installed it in the python 2.7 directory.