Develop Reference

Pycharm: Fails to make venv based on python 3.9.01+ [duplicate]

I setup a new Debian 10 (Buster) instance on AWS EC2, and was able to install a pip3 package that depended on netifaces, but when I came back to it the next day the package is breaking reporting an error in netifaces. If I try to run pip3 install netifaces I get the same error:
~$ pip3 install netifaces
Collecting netifaces
Using cached https://files.pythonhosted.org/packages/0d/18/fd6e9c71a35b67a73160ec80a49da63d1eed2d2055054cc2995714949132/netifaces-0.10.9.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 20, in <module>
from setuptools.dist import Distribution, Feature
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 35, in <module>
from setuptools.depends import Require
File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
from .py33compat import Bytecode
File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 55, in <module>
unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
AttributeError: 'HTMLParser' object has no attribute 'unescape'

HTMLParser().unescape was removed in Python 3.9. Compare the code in Python 3.8 vs Python 3.9.
The error seems to be a bug in setuptools. Try to upgrade setuptools. Or use Python 3.8.

I was facing this issue in PyCharm 2018. Apart from upgrading setuptools as mentioned above, I also had to upgrade to PyCharm 2020.3.4 to solve this issue. Related bug on PyCharm issue tracker: https://youtrack.jetbrains.com/issue/PY-39579
Hope this helps someone avoid spending hours trying to debug this.

I had python3.6 and related packages through deb management.
Needed python3.9 for side project and the solution to fix pip and AttributeError: 'HTMLParser' object has no attribute 'unescape'
was to update pip for python3.9 locally for one user:
python3.9 -m pip install --upgrade pip
now installing python3.9 version of the pip-packages work:
python3.9 -m pip install --target=~/.local/lib/python3.9/site-packages numpy

Downgrading to any older python3 version is not the solution and most of the time upgrading setuptools won't fix the issue. The proper solution that worked for me to work with pip using python3.9 is the following on Ubuntu18:
locate /usr/lib/python3/dist-packages/setuptools/py33compact.py33
and change
# unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape) # comment out this line
unescape = getattr(html, 'unescape', None)
if unescape is None:
# HTMLParser.unescape is deprecated since Python 3.4, and will be removed
# from 3.9.
unescape = html_parser.HTMLParser().unescape

importing pycurl with libcurl4 raises ImportError

I am using Ubuntu 18.04. If I install libcurl4 (instead of libcurl3), when I import pycurl installed with pipenv I get
>>> import pycurl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /home/pietro/envs/try_fabric-kcbGLH3z/lib/python3.6/site-packages/pycurl.cpython-36m-x86_64-linux-gnu.so)
Note that this error is raised only when I use a pycurl installed with pipenv or pip. If I use instead pycurl installed through apt on system python there are any errors...
What can I do to avoid this impasse?

I assume you upgraded your operating system recently.
pip stores a cache of built wheels in your home directory, when upgrading your os it is a good idea to clear this cache as the binaries may now link against incompatible system libraries
You can do this by rm -rf ~/.cache/pip and then recreate your environment

Python SAML authentication

I'm trying to build a SAML authentication mechanism in Python using the OneLogin module, but I can't even get started. Trying from the example code provided in the docs, I can't even load the package.
This works:
import onelogin
but this gets a ModuleNotFoundError:
>>> from onelogin.saml2.auth import OneLogin_Saml2_Auth
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'onelogin.saml2'

I only managed to get this working on a Linux machine, so hope you're not on windows. Here's what worked for me.
I had to do a seperate install of hte xmlsec-library first:
apt-get install xmlsec1 openssl python-lxml libxmlsec1 libxmlsec1-dev
and then I was able to
pip install python3-saml
and there were no complains anymore.

You need to install Xmlsec first and then python3-saml but if you are on Windows, Xmlsec has some issues but as with python 3.6 and below the problem is solved. Use this link to download the wheel file for your python version https://github.com/mehcode/python-xmlsec/releases
Install the wheel file using
pip install <wheel_file_name>
As with Python 3.7, the only way out is to install xmlsec on a Linux machine as it is not yet supported on Windows.

ImportError: No module named 'ase.build'

In ubuntu 16.04 i installed python and modules:
sudo apt install python3 python3-scipy python3-numpy python3-ase
then i try to follow the first tutorial on the ASE homepage. I run python3 in bash terminal, and can import other modules but not ase-build. It looks like this:
>>> from ase.optimize import QuasiNewton
>>> from ase.build import fcc111, add_adsorbate
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'ase.build'
>>>
Using a python script throws an equivalent error.
What could be the problem?
UPDATE & SOLUTION
Seems this was not really even a python problem. I seem to have had some package dependency errors probably due to not running apt update in a long time between program installations. I removed python2.x and python 3.x, then iterated apt update, apt upgrade, apt autoremove, then reinstalled only python3. I installed python3-pip and installed the numpy, scipy, and ase packages using the proper form python3 -m pip install --upgrade <package>. Now everything works as expected.

Check what version of the library you have.
import ase
print(ase.__version__)
If the version is 3.10.0 then that is the problem since the build module appeared (as far as I know) in the 3.11.0 version.

Due to this link - you have installed the 3.9.1.4567-3 version on your computer.
But The asu.build has been added in:
commit 71c9563e423e2add645c26f8d0a722f3db13e135
Author: Jens Jørgen Mortensen
Date: Tue Apr 12 15:40:59 2016 +0200
Move stuff to ase.build module
So, the module asu.build doesn't exist in your version (3.9 has been released in 2015). You have to install the newer version of python3-asu.

Installed Django with Pip but Python cant find it

I am trying to install django, but I am not sure how to proceed. I think I have installed django, but python python doesn't seem to be able to seethe package.
$ sudo pip install django
Requirement already satisfied (use --upgrade to upgrade): django in /usr/local/lib/python2.7/site-packages
Cleaning up...
$ python -c "import sys; sys.path = sys.path[1:]; import django; print(django.__path__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named django
How do I fix this? When I try to run the server I get this error
$ python manage.py runserver
Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management

Your pip is using a different version of Python than your python. Check the output of these commands:
pip -V
python -V
python -c 'import sys; print(sys.path)'
There can be multiple versions of Python and Pip installed in your system.
For example in a Bash shell if you type python + Tab a few times it will show you the available Python binaries on your PATH, for example python2.7, python3.4, and similarly for pip + Tab too.
It depends on your system how to configure correctly so that both python and pip are using the same versions.
It would be best to use virtualenv. You wouldn't have such problems, as in a virtualenv your Python version and Pip version would be well synchronized.
Judging by your command output, pip is using Python 2.7. One quick fix could be to try running Django like this:
python2.7 manage.py runserver
Or, to run the pip version corresponding to your default Python version.
This "quick fix" is a dirty fix. It would be best to use virtualenv.