Develop Reference

Why are python modules not found despite installing them in pip?

I've been trying to use VS Code's python debugger on Linux (mint), which uses debugpy and it keeps giving the error "No module named '_ctypes'". Installing libffi-dev didn't fix it as suggested elsewhere on SO and neither did reinstalling python and python3; so, I tried installing debugpy through pip:
pip install debugpy
Which installs with no issues. However, both python and python3 commands cannot find the module despite the fact that the module is installed (which I can see when I enter pip list)
python -m debugpy
/usr/bin/python: No module named debugpy
python3 -m debugpy
/usr/local/bin/python3: No module named debugpy
So after trying to reinstall pip multiple times, I tried installing through the pip module
python -m pip install debugpy
/usr/bin/python: No module named pip
python3 -m pip install debugpy
/usr/local/bin/python3: No module named pip
So it seems my pip module is also missing too. It may have something to do with my multiple installations of python3 as it seems that there is one in /bin and in /usr/local/bin and the local installation is the one that gets called with the python3 command according to which python3.
This leads to multiple questions:
Should the pip module be installed in python, and if so how do I install it again?
How can I get the pip command to actually install the modules into python?
How can I ensure that there is only one python3 installation in Linux (mint/ubuntu)?
Thank-you. If it helps answer the question, I do not seem to have a PYTHONPATH variable.

i had same issue but it can be fixed by two methods.
reinstall pip for the latest python version you use.
2.use the version default python version which came with your python install

On ubuntu you can try to do sudo apt install python-pip python3-pip. Also there a couple ways you can try what PIP's documentation recommends: Installation
You can try checking sudo update-alternatives --config python maybe you'll see several python installations there.
Also you can check pip contents to find out what python binary it uses:
$ which pip
/usr/local/bin/pip
$ cat /usr/local/bin/pip
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())
#!/usr/bin/python3

I tried installing it with pip and it has worked for me. The problem you might have is that it is not installed in the correct version of python (e.g it installs for python2 instead of python3 or it installs in python3.8 instead of python3.9).
What you can do to avoid this problem is to create a virtual environment, as is explained in this link: https://docs.python.org/3/tutorial/venv.html. If you do that, remember to change the interpreter path in VSCode. At the bottom left of the screen there should be the python version that you are using. If you click on it you see different interpreter versions that you can use. Select 'Enter interpreter path' and manually choose the directory where you have saved the new python virtual environment.

It's difficult to narrow down a solution for apparently you have done a mess... To start, do yourself a favor:
Don't touch your System's Python!
Again, it is difficult for me to understand what is the current state of your Python scene... just promise me you'll fix your System's Python 2/3 (i.e, guarantee pip 2/3 there, using your system's package manager (apt, yum, etc)).
Then, start using some virtual environment manager, nowadays we have Pipenv (https://pipenv.pypa.io/), which can be a bit cumbersome at first but -- trust me -- in ~1 hour you'll love it.
Conda (https://docs.conda.io/) is also a great env manager (or the classics (pyenv, venv, etc...)).
...Just pick one and leave your OS' Python alone. You'll see that your issues will not only get simpler to diagnose but you'll also be able to sleep in peace ;)

How do modules installation work in Python?

[On a mac]
I know I can get packages doing pip install etc.
But I'm not entirely sure how all this works.
Does it matter which folder my terminal is in when I write this command?
What happens if I write it in a specific folder?
Does it matter if I do pip/pip3?
I'm doing a project, which had a requirements file.
So I went to the folder the requirements txt was in and did pip install requirements, but there was a specific tensorflow version, which only works for python 3.7. So I did """python3.7 -m pip install requirements""" and it worked (i'm not sure why). Then I got jupyter with brew and ran a notebook which used one of the modules in the requirements file, but it says there is no such module.
I suspect packages are linked to specific versions of python and I need to be running that version of python with my notebook, but I'm really not sure how. Is there some better way to be setting up my environment than just blindley pip installing stuff in random folders?
I'm sorry if this is not a well formed question, I will fix it if you let me know how.

Yes, there is. Setup an virtual environment.
pip install virtualenv #installs the library
virtualenv mypython #creates the environment
source mypython/bin/activate #activates the environment
Now, install your requirements through pip.
Afterwards, when your work is finished.
Just type deactivate to come out of the virtual environment.

There may be a difference between pip and pip3, depending on what you have installed on your system. pip is likely the pip used for python2 while pip3 is used for python3.
The easiest way to tell is to simply execute python and see what version starts. python will run typically run the older version 2.x python and python3 is required to run python version 3.x. If you install into the python2 environment (using pip install or python -m pip install the libraries will be available to the python version that runs when you execute python. To install them into a python3 environment, use pip3 or python3 -m pip install.
Basically, pip is writing module components into a library path, where import <module> can find them. To do this for ALL users, use python3 or pip3 from the command line. To test it out, or use it on an individual basis, use a virtual environment as #Abhishek Verma said.

how to install flake8 in atom on windows

I have follow, these steps, but
"apm install linter
Next, we’re going to install a Python Linter package, to help us detect errors in our Python code.
This package is called linter-flake8 and it’s an interface to flake8. To install it, you need to run:
pip install flake8
pip install flake8-docstrings
apm install linter-flake8
You must restart Atom to see the changes"
I have followed those steps and every package with PIP and APM were installed, however, corrections are not made on my python code in ATOM. Is there something else i need to configure or to do appart from steps i mentioned?
2
3

It could be your Environment Variable paths, especially if you have two concurrent versions of Python installed. Check to see which one is at the top of the list and ahead of the one your not currently developing within.

I'll try to give you an example of my installation
Installation path of python in my computer is
C:\Python36-32
So in control panel in system settings in advanced settings in path i add the following paths
C:\Python36-32
C:\Python36-32\Scripts
Now i reopen the cmd and i enter python if it worked you will see the python interpreter line.
To install flake8 system wide run the command
pip install flake8
In atom editor install the package linter-flake8.When package is installed in settings of package in package path add the full path of package for example in my configuration i have in path
C:\Python36-32\Scripts\flake8 without the .exe

Maybe you got installed flake8 on C:\Users\myuser\AppData\Roaming\Python\Python3X\Scripts just like me. If that is the case you only need to add that location to the PATH.

pylint doesn't point to virtualenv python

I am pretty new to python and currenty I am trying to use pylint for checking code quality. I am getting a problem. My pylint doesn't point to virtualenv python interpreter. Here is the output that I get when I run pylint --version
$ pylint --version
pylint 0.21.1,
astng 0.20.1, common 0.50.3
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]
In virtualenv I have python 2.7 installed. Will appretiate you help if someone can point me to how to solve that.

A cheap trick is to run (the global) pylint using the virtualenv python. You can do this using python $(which pylint) instead of just pylint. On zsh, you can also do python =pylint.

I am fairly sure that you need to install pylint under your virtual environment and then run that instance of it.
Update - Make life easier:
I would suggest that anybody working a lot in virtual environments create a batch file, (in a known location or on the path), or bash script with something like the following called something like getlint.bat:
pip install pylint
Invoking this after activating the virtual environment will install pylint into that virtual environment. If you are likely to be offline or have a poor internet connection you can, once when you have a good internet connection, (possibly once for each of python 2 & 3):
mkdir C:\Some\Directory\You\Will\Leave\Alone
pip download --dest=C:\Some\Directory\You\Will\Leave\Alone pylint
Which will download pylint and its dependencies to C:\Some\Directory\You\Will\Leave\Alone and you can modify getlint.bat to read:
pip install pylint --find-links=C:\Some\Directory\You\Will\Leave\Alone
It will then use the pre-downloaded versions.

Noufal Ibrahim's answer works if you execute pylint manually.
If you execute pylint from you editor/IDE, you need to configure the plugin correctly.
vim/syntastic
atom/linter-pylint
...
It can get tricky. This may be considered a bug of each IDE/plugin, but that's how it is.
Modifying /usr/bin/pylint to write #!/usr/bin/env python as suggested in another answer fixes this for every use of pylint (manual use, or any editor integration).
However, at least in Debian, using #!/usr/bin/python is a design choice, not a bug. See here for the rationale.
To avoid modifying that system file, one can create a copy of /usr/bin/pylint in /usr/local/bin:
cp /usr/bin/pylint /usr/local/bin/pylint
vi usr/local/bin/pylint # Edit the file to use /usr/bin/env python
This won't be broken by a pylint update, but still infringes Debian's "strongly preferred choice".
This method requires root privileges. An unprivileged user may create an alias
alias pylint='/usr/bin/env python $(which pylint)'.
I always develop in virtualenv and I setup a postmkvirtualenv hook to install pylint and flake8 automatically when creating a virtualenv, so I don't use the versions ditributed by debian anymore.

I ran into this problem, too. My solution was simply to edit the pylint program's shebang, like so... (your path to pylint may be different than mine, though)
$ sudo vim /usr/bin/pylint
Replacing:
#!/usr/bin/python
With:
#!/usr/bin/env python

The issue has been solved on chat (link in comments).
The problem lied in using sudo yum install pylint, because it installed pylint in the global env. The solution was to use the following command:
pip install -i http://f.pypi.python.org/simple pylint
Note the -i usage as the regular index seemed to be broken for the asker.

I know it's been a while since this question was answered, but I just thought I should leave this post here in case someone else runs into the same problem.
If for some reason you need to keep pylint in the global space instead of your virtual environment, you can use the recommendation in here: PyLint + VirtualEnv.
It basically says to configure your pylint using the init-hook and encoding version of a Python program that will use the global pylint and load the rest of the environment.

You can get there by calling the target python interpreter:
./env/bin/python -m pylint ...
# or in an already active env
python -m pylint ...

When you're using Pipenv / virtualenv, install pylint inside the virtualenv:
pipenv install --dev pylint
or, if you don't use Pipenv, install it with pip after you activated your virtualenv:
# activate virtualenv, e.g. `. env/bin/activate`
pip install pylint

I'm using the Syntastic + Pylint combination, and since I have many different virtualenvs that I can work on at any given time, I've created a wrapper over the virtualenv command that, among some other things, installs pylint after all the requirements.
That way, whenever I activate a virtualenv, I'll get its own pylint version.
Hope this helps, and thanks for the tip on deleting the global one from #briford-wylie

Ran into the same problem just today. Continuing on ThorSummoner's answer, when using Pylint with pylint-django inside of a virtual environment such as Pipenv, make sure to call pylint using the target python interpreter (python -m pylint)
A good approach, which will work locally and on your CI as well is to write-down the lint command in the script section of your Pipfile:
[scripts]
lint = "python -m pylint [--options] all-my-modules-names..."
Then calling pylint is as easy as :
pipenv run lint

Python ImportError while module is installed [Ubuntu]

I'd like to make a switch from Windows to Linux (Ubuntu) writing my python programs but I just can't get things to work. Here's the problem: I can see that there are quite the number of modules pre-installed (like numpy, pandas, matplotlib, etc.) in Ubuntu. They sit nicely in the /host/Python27/Lib/site-packages directory. But when I write a test python script and try to execute it, it gives me an ImportError whenever I try to import a module (for instance import numpy as np gives me ImportError: No module named numpy). When I type which python in the commandline I get the /usr/bin/python path. I think I might need to change things related to the python path, but I don't know how to do that.

You can use the following command in your terminal to see what folders are in your PYTHONPATH.
python -c "import sys, pprint; pprint.pprint(sys.path)"
I'm guessing /host/Python27/Lib/site-packages wont be in there (it doesn't sound like a normal python path. How did you install these packages?).
If you want to add folders to your PYTHONPATH then use the following:
export PYTHONPATH=$PYTHONPATH:/host/Python27/Lib/site-packages
Personally here are some recommendations for developing with Python:
Use virtualenv. It is a very powerful tool that creates sandboxed python environments so you can install modules and keep them separate from the main interpreter.
Use pip - When you've created a virtualenv, and activated it you can use pip install to install packages for you. e.g. pip install numpy will install numpy into your virtual environment and will be accessible from only this virtualenv. This means you can also install different versions for testing etc. Very powerful. I would recommend using pip to install your python packages over using ubuntu apt-get install as you are more likely to get the newer versions of modules (apt-get relies on someone packaging the latest versions of your python libraries and may not be available for as many libraries as pip).
When writing python scripts that you will make executable (chmod +x my_python_script.py) make sure you put #!/usr/bin/env python at the top as this will pick up the python interpreter in your virtual environment. If you don't (and put #!/usr/bin/python) then running ./my_python_script.py will always use the system python interpreter.

/host/Python27/Lib/site-packages is not a default python directory on linux installations as far as I am aware.
The normal python installation (and python packages) should be found under /usr/lib or /usr/lib64 depending on your processor architecture.
If you want to check where python is searching in addition to these directories you can use a terminal with the following command:
echo $PYTHONPATH
If the /host/Python27/Lib/site-packages path is not listed, attempt to use the following command and try it again:
export PYTHONPATH=$PYTHONPATH:host/Python27/Lib/site-packages
If this should work and you do not want to write this in a terminal every time you want to use these packages, simply put it into a file called .bashrc in your home folder (normally /home/<username>).

When installing other python libraries, specify the pip version you want to install it to, if it's python2 you use, then enter this syntax:
pip2 install <package>
For python3
pip3 install <package>