Develop Reference

ModuleNotFoundError: No module named 'psycopg2' (python3, mac, pip3 shows psycopg2 successfully installed)

Solved: Learn about virtual environments.
pip install virtualenv
Problem: I get a ModuleNotFoundError for psycopg2 in python3, though it's successfully installed via pip3. (I posted short code to summarize from the terminal, but the errors are of course bugging me with .py scripts I'm trying to run.)
Python3 packages:
macs-MacBook-Air-2% pip3 list
Package Version
---------- -------
pip 19.2.3
psycopg2 2.8.4
setuptools 41.4.0
wheel 0.33.6
Python3 psycopg2 error:
macs-MacBook-Air-2% python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'psycopg2'
I've only done:
pip3 install psycopg2
Extra notes:
As a side note, everything works fine when I run 2.7. (I used pip install psycopg2.)
macs-MacBook-Air-2% python
Python 2.7.10 (default, Oct 6 2017, 22:29:07)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
>>> psycopg2.__version__
'2.8.4 (dt dec pq3 ext lo64)'
>>>
python 2.7 packages:
macs-MacBook-Air-2% pip list
DEPRECATION: ...2.7 end of life notice..
Package Version
-------------------------------------- -----------
altgraph 0.10.2
astroid 1.6.6
...more
psycopg2 2.8.4
...more
zope.interface 4.6.0
I'm very new to coding, and I searched this error. But, I did not find results that made sense to me including both a python3 error + successful python3 installation.

This could be because your default python installation is Python 2. I think you should create a virtual environment and the install psycopg2 on it. This way you will use pip3 and have isolated dependencies that won't generate conflicts with other versions (and maybe corrupt your system):
python3 -m venv ~/.environments/test
source ~/.environments/test/bin/activate
pip install psycopg2

How do I properly install PyGObject? (OSX)

I want to run existing simple examples and write some simple code using GStreamer - specifically, using its Python bindings. I want to install the packages etc to enable that.
Here's an example.
http://brettviren.github.io/pygst-tutorial-org/pygst-tutorial.html
import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst
Gst.init(None)
# ...
my_playbin = Gst.ElementFactory.make("playbin", None)
assert my_playbin
print my_playbin
I can't get PyGObject to work, so I'm stuck right at import gi and can't make any progress beyond the first line.
The platform is MacOS 10.12.6, and Python 3.6.5.
computer:Desktop me$ python3
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gi'
>>>
Okay, let's RTFM.
https://pygobject.readthedocs.io/en/latest/getting_started.html#macosx-getting-started
Seems pretty simple and this should get PyGObject installed, right?
I've already got Homebrew installed, but let's just do it again to be sure.
computer:Desktop me$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
==> This script will install:
/usr/local/bin/brew
[Snip for brevity]
==> Installation successful!
==> Next steps:
- Run `brew help` to get started
- Further documentation:
https://docs.brew.sh
computer:Desktop me$
OK, now let's install pygobject3 and gtk+3
computer:Desktop me$ brew install pygobject3 gtk+3
Updating Homebrew...
Warning: pygobject3 3.32.1 is already installed and up-to-date
To reinstall 3.32.1, run `brew reinstall pygobject3`
Warning: gtk+3 3.24.8 is already installed and up-to-date
To reinstall 3.24.8, run `brew reinstall gtk+3`
computer:Desktop me$
Now let's try Python again:
computer:Desktop me$ python3
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gi'
>>>
So we've followed the instructions and we're still right where we started with no functionality.
Also tried various --with-python3 and --without-python options during the brew install.
computer:Desktop me$ brew install pygobject3 --with-python3 --without-python
Updating Homebrew...
[SNIP FOR BREVITY]
Error: invalid option: --with-python3
All of these options are invalid options, despite being mentioned in various internet threads.
computer:Desktop me$ brew install pygobject3 --with-python#2 gtk+3
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/cask).
No changes to formulae.
[SNIP FOR BREVITY]
Error: invalid option: --with-python#2
Can somebody tell me what I'm missing please?

I know this answer is a bit late, but I figured out why it isn't working and a workaround for those who run into it in the future.
Homebrew will install pygobject3 by symlinking it in from a package install directory, which means that pygobject3 is only accessible from the python binaries installed by homebrew. If I used the python executable at /usr/local/bin/python3, the install location of brew install python, pygobject3 imports just fine.
Obviously, that's suboptimal, and we want to transfer pygobject3 to our preferred installation of python.
I did it like so:
$ cd /usr/local/Cellar/pygobject3/3.34.0/lib/python3.7/site-packages/
$ sudo cp -rf * /usr/local/anaconda3/lib/python3.7/site-packages/
I can now import and use gi:
(base) 2l4#mac105220:/usr/local/anaconda3/lib/python3.7/ > python
Python 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from gi.repository import GLib
>>> loop = GLib.MainLoop()
>>> loop.run()

Upgrade an apt installed Python package in /usr/lib?

Is it possible to upgrade a package installed with apt-get, so located in /usr/lib/ , if such package do have a more recent version in pypi but not within the standard Ubuntu repositories as seen by apt?
I guess it is dangerous as it may break dependencies, but it's just to know.

Yes, it is.
I uninstall Flask
$ sudo apt-get remove python-flask
I don't have it:
$ python
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import flask
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named flask
I install it
$ sudo apt-get install python-flask
$ python
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import flask
>>> flask.__version__
'0.12'
Double-check:
$ pip list -o | grep Flask
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
Flask (0.12.1) - Latest: 0.12.2 [wheel]
Upgrading:
$ sudo pip install --upgrade Flask
...
Successfully installed Flask-0.12.2 Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 itsdangerous-0.24
$ python
Python 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import flask
>>> flask.__version__
'0.12.2'
I see I have an issue with pip check, however:
$ pip list -o | grep Flask
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
Flask (0.12.1) - Latest: 0.12.2 [wheel]
So I must have some links or something broken, but this issue is still alive even if I use apt-get remove. All in all I am able to import newer version of Flask which is what you need I guess.
EDIT
OK, the issue is that pip installs Flask in different location then apt-get. This is pip output:
>>> flask.__file__
'/usr/local/lib/python2.7/dist-packages/flask/__init__.pyc'
And this is apt-get's:
>>> flask.__file__
'/usr/lib/python2.7/dist-packages/flask/__init__.pyc'
Here is a description of how to make pip install you package in a different directory. I have not tested it, however.

Why is the PYTHONPATH different when running Python with and without sudo?

There's something wrong with my OSX system and python that no amount of googling has fixed. I've uninstalled all traces of python except the system python package with OSX that I'm not supposed to uninstall, and then started afresh with a new python from python.org, and installed pip.
Now...not sure if this particular behavior below is part of the issue, but it seems strange to me:
I ran python twice. Once with sudo and once without. Without sudo, I can't access pip. What's going on?
$ sudo /Library/Frameworks/Python.framework/Versions/2.7/bin/python
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pip
However...
$ /Library/Frameworks/Python.framework/Versions/2.7/bin/python
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>>
I've already referred to: sudo python runs old python version
I have nothing in my .bash_profile, or anything in any other profiles.
All I've done is the following:
export PYTHONPATH=/lib/python2.7/site-packages/
ls $PYTHONPATH returns:
_markerlib pip pkg_resources.pyc setuptools-8.0.1.dist-info virtualenv.pyc
easy_install.py pip-1.5.6.dist-info setuptools virtualenv-1.11.6.dist-info virtualenv_support
easy_install.pyc pkg_resources.py setuptools-7.0.dist-info virtualenv.py
which pip returns:
/bin/pip

sudo overrides your export. It's the same Python (as you can easily tell from the version information it prints) but it runs with a different (system default) PYTHONPATH.
This is one of the jobs of sudo; it sanitizes the environment to safe defaults. You may be able to tweak this, but the real question is, what are you trying to accomplish? If you need to run as root with a particular environment, set up a virtualenv and/or write a wrapper script which sets things up before dispatching Python.

What do you get when you compare the output of which pip and sudo which pip?
On my system I get different outputs. If you do, I'm not sure how to fix that, but you could try to force the sudo'd python to look in the correct directory:
import sys
sys.path.insert(0, '/lib/python2.7/site-packages/')
import pip

Python3 has no acces to python2 modules (ubuntu)

Im fairly new to programming and Ubuntu. Yesterday I finally managed to create a dual-boot system, so now I'm running Ubuntu 12.04 LTS.
For a school project, I need to work in Python3 with a module called SPARQLWrapper (https://pypi.python.org/pypi/SPARQLWrapper).
On my freshly installed Ubuntu, I've installed the latest version of Python. When I type "python3" in my terminal, python 3.2.3 starts so thats good.
I installed easy_install (sudo apt-get install python-setuptools), and downloaded and installed the SPARQLWrapper egg file (sudo easy_install SPARQLWrapper-1.5.2-py3.2).
If I run python2 and use "import SPARQLWrapper", it just works. But if I try the same in python3 it gives me the following error:
x#ubuntu:~$ python3
Python 3.2.3 (default, Oct 19 2012, 20:10:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import SPARQLWrapper
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named SPARQLWrapper
So my problem is that python3 isn't able to acces the same modules as my python2. How do I fix this?
Thanks!

To install packages for Python3, you need python3's setuptools.
Following are the steps to be followed to install python3's setuptools and SPARQLWrapper
sudo apt-get install python3-setuptools
sudo easy_install3 pip
pip -V This should show the pip corresponding to your python3 installation.
sudo pip install SPARQLWrapper
After doing the above mentioned steps, I get this
~$ python3
Python 3.3.1 (default, Apr 17 2013, 22:30:32)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import SPARQLWrapper
>>> exit()
~$

Each Python installation has its own modules directory. In addition, Python 3 is not backwards compatible and won't generally run Python 2 code. You'll need to find a Python 3 version of the module you need and install it for Python 3.