Develop Reference

Unable to install libraries with pip due to outdated BeautifulSoup

The command
pip install pyteaser
produces this error:
Collecting pyteaser
Using cached https://files.pythonhosted.org/packages/d4/7a/310592c6e7998440e56a8650446ecf3ded076431415c60f0f3b946b54462/pyteaser-2.0.tar.gz
Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from pyteaser) (5.1.0)
Requirement already satisfied: lxml in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pyteaser) (4.4.2)
Collecting cssselect (from pyteaser)
Using cached https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl
Collecting jieba (from pyteaser)
Using cached https://files.pythonhosted.org/packages/b2/c8/e0785494690876adbb5ac364eb148a240b738a614d7b39bf8dba8a7ad066/jieba-0.41.tar.gz
Collecting beautifulsoup (from pyteaser)
Using cached https://files.pythonhosted.org/packages/40/f2/6c9f2f3e696ee6a1fb0e4d7850617e224ed2b0b1e872110abffeca2a09d4/BeautifulSoup-3.2.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pycharm-packaging/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pycharm-packaging/beautifulsoup/
And the command
pip install pytldr
reports this error:
pip install pytldr
Collecting pytldr
Using cached https://files.pythonhosted.org/packages/70/09/02ed27061159e5f6d35abad4ec9ef3cac8e220093d61a2f7a42f53c9cb22/PyTLDR-0.1.4.tar.gz
Requirement already satisfied: numpy>=1.8.0 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pytldr) (1.18.1)
Requirement already satisfied: nltk>=3.0.0 in /home/abhishek/.local/lib/python3.6/site-packages (from pytldr) (3.4.5)
Requirement already satisfied: scipy>=0.13.2 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pytldr) (1.4.1)
Collecting scikit-learn>=0.15.2 (from pytldr)
Using cached https://files.pythonhosted.org/packages/d1/48/e9fa9e252abcd1447eff6f9257636af31758a6e46fd5ce5d3c879f6907cb/scikit_learn-0.22.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting goose-extractor>=1.0.25 (from pytldr)
Using cached https://files.pythonhosted.org/packages/ab/f3/2e89688e4e1d8786e542ea460431f24f1392d885e60303dc20d476a96424/goose-extractor-1.0.25.tar.gz
Collecting networkx>=1.9.1 (from pytldr)
Using cached https://files.pythonhosted.org/packages/41/8f/dd6a8e85946def36e4f2c69c84219af0fa5e832b018c970e92f2ad337e45/networkx-2.4-py3-none-any.whl
Requirement already satisfied: six in /home/abhishek/.local/lib/python3.6/site-packages (from nltk>=3.0.0->pytldr) (1.12.0)
Collecting joblib>=0.11 (from scikit-learn>=0.15.2->pytldr)
Using cached https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl
Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from goose-extractor>=1.0.25->pytldr) (5.1.0)
Requirement already satisfied: lxml in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from goose-extractor>=1.0.25->pytldr) (4.4.2)
Collecting cssselect (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl
Collecting jieba (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/b2/c8/e0785494690876adbb5ac364eb148a240b738a614d7b39bf8dba8a7ad066/jieba-0.41.tar.gz
Collecting beautifulsoup (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/40/f2/6c9f2f3e696ee6a1fb0e4d7850617e224ed2b0b1e872110abffeca2a09d4/BeautifulSoup-3.2.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-9oa07p4e/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-9oa07p4e/beautifulsoup/
Note: you may need to restart the kernel to use updated packages.
EDIT:
After executing the command for BeautifulSoup
pip install beautifulsoup4
the response is:
Requirement already satisfied: beautifulsoup4 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (4.8.2)
Requirement already satisfied: soupsieve>=1.2 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from beautifulsoup4) (1.9.5)
Note: you may need to restart the kernel to use updated packages.

The problem here is that the library is not (yet) ready for Python3, as you can see here in a comment on an issue of this library. And that is the source of your problem. As you can see, there are no issues if you install the library for Python2.7:
$ sudo apt install python-pip # or https://pip.pypa.io/en/stable/installing/
$ python --version
Python 2.7.17
$ python -m pip install pytldr
Successfully installed Pillow-6.2.2 beautifulsoup-3.2.2 cssselect-1.1.0 decorator-4.4.2 goose-extractor-1.0.25 jieba-0.42.1 lxml-4.5.0 networkx-2.2 nltk-3.4.5 numpy-1.16.6 pytldr-0.1.4 scikit-learn-0.20.4 scipy-1.2.3 singledispatch-3.4.0.3 six-1.14.0
Here, you can see that everything worked, because beautifulsoup-3.2.2 version was installed. Trying the same command for Python3 leads to the error you're posted:
$ python3 --version
Python 3.6.9
$ python3 -m pip install pytldr
Collecting beautifulsoup
Using cached BeautifulSoup-3.2.2.tar.gz (32 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-12aluvvq/beautifulsoup/setup.py'"'"'; __file__='"'"'/tmp/pip-install-12aluvvq/beautifulsoup/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-12aluvvq/beautifulsoup/pip-egg-info
cwd: /tmp/pip-install-12aluvvq/beautifulsoup/
Complete output (6 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-12aluvvq/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The error message is very clear:
You're trying to run a very old release of Beautiful Soup under Python 3. This will not work. Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'.
Simply installing BeautifulSoup4 is not solving the problem, because the library still uses code for BeautifulSoup3. That's a dependency of the pytldr package.
Now you have several options, maybe more:
Do not use this library.
Use Python2 (But it's the year 2020).
There is already an open pull request that offers code that allegedly has updated the library to be compliant with Python3. You can build the library from source with code provided there (no guarantee that this will work) or wait until the pull request was approved (or write to the author).
Go through the forks and see if anyone has already got the library up and running for Python3.
Try to update the library yourself and maybe with the help of 2to3.
You can apparently also install this fork directly instead of step 3. Of course, this fork may be out-of-date at some point, or it may be modified in such a way that it is no longer usable. There is also the option of installing a specific commit. But also no guarantee that this will work.

The pyteaser and goose-extractor libraries simply use the BeautifulSoup 3 library, which uses a different name from BeautifulSoup 4. You can't just install BeautifulSoup 4, because pyteaser and goose-extractor need to be updated to use the newer BeautifulSoup library. That's going to be more work, because those 2 projects are not compatible with Python 3 either.
You are basically stuck with Python 2 for these projects unless you are willing to do all the upgrade work yourself or put your trust in incomplete forks:
Unfortunately, even the forks that purport to make pyteaser Python 3 compatible have left in the wrong dependency. The project doesn't actually even import BeautifulSoup anywhere, so you'd have to clone one of the Python 3 forks (like this one) and remove the beautifulsoup entry from the install_requires line.
For PyTLDR, this fork can be installed on Python 3:
pip install git+https://github.com/vinodnimbalkar/PyTLDR.git#egg=pytldr
It dropped goose altogether.
Your mileage may vary, however, since the conversion looks, to me, somewhat haphazard. For Unicode text (the default in Python 3), normalization will never take place (both the if and elif branches test for the same condition now), for example. And while it removed the goose-extractor dependency, the code still tries to import it if you referenced it, which won't work either.
If you wanted to use goose-extractor directly, you can replace with goose3. Note that this also changed the module name, to goose3 (so the PyTLDR fork above could perhaps be fixed by changing the imported module name).

Jnius installation bug, "Unable to determine JDK_HOME"

I tried to install jnius Python module by typing 'pip install jnius' in CMD.
This is the message I got:
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\users\sm\appdata\local\temp\pip-install-vu2sb5\jnius\setup.py", line 111, in <module>
raise Exception('Unable to determine JDK_HOME')
Exception: Unable to determine JDK_HOME
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in
c:\users\sm\appdata\local\temp\pip-install-vu2sb5\jnius\
I restarted CMD, but it didn't help.

I have finally resolved it. I have installed Cython before this happened and I installed both Java JDK and JRE after #Chris comment. And I have also added JAVA_HOME and JDK_HOME to system variables.Thank you #Chris, but it didn't resolve problem completely.
Then I got another error message. There was written that I have to install Microsoft Visual C++ Compiler for Python 2.7 from this page:
https://www.microsoft.com/en-us/download/confirmation.aspx?id=44266
After that I tried to install jnius again but the error message said that permission is denied so I ran CMD as administrator and finally installed jnius by typing pip install jnius.
I spent a lot of time to resolve it and hope that it will help everyone with same or similar problem.

I have resolved the problem in windows os.
1st step:
I have installed cython by using pip install cython
C:\Users>pip install cython
Collecting cython
Downloading Cython-0.29.22-cp38-cp38-win_amd64.whl (1.7 MB)
|████████████████████████████████| 1.7 MB 1.3 MB/s
Installing collected packages: cython
Successfully installed cython-0.29.22
2nd step:
I have installed jnius by using pip install pyjnius command
C:\Users>pip install pyjnius
Collecting pyjnius
Downloading pyjnius-1.3.0-cp38-cp38-win_amd64.whl (226 kB)
|████████████████████████████████| 226 kB 3.2 MB/s
Requirement already satisfied: cython in c:\users\sss\appdata\local\programs\python\python38\lib\site-packages (from pyjnius) (0.29.22)
Requirement already satisfied: six>=1.7.0 in c:\users\sss\appdata\local\programs\python\python38\lib\site-packages (from pyjnius) (1.15.0)
Installing collected packages: pyjnius
Successfully installed pyjnius-1.3.0
I hope this solves your problem.

The thing that worked for me was simply adding JAVA_HOME and JDK_HOME variables.

pip Installing a python module in Python 3, but that module has a dependency package that requires Python 2

so I am new to Python and what is a good way to learn it is to do some personal project :) So I am on my Windows and using virtualenv to contain environment
So, I need to use the Scrappy module:
pip install scrappy
Collecting scrappy
Using cached Scrappy-0.3.0.alpha.4.tar.gz
Collecting guessit (from scrappy)
Using cached guessit-2.1.4.tar.gz
Collecting tvdb_api (from scrappy)
Using cached tvdb_api-2.0.tar.gz
Collecting hachoir-metadata (from scrappy)
Using cached hachoir-metadata-1.3.3.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\USERPH\AppData\Local\Temp\pip-build-__zp7zbj\hachoir-metadata\setup.py", line 65
except OSError, err:
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\USERPH\AppData\Local\Temp\pip-build-__zp7zbj\h
achoir-metadata\
Its because hachoir-metadata is a Python2 dependent package, so I tried installing their Python 3 version
pip install hachoir3
I confirmed that it was installed using
pip list | Select-String hachoir3
so I tried running
pip install scrappy again
but same error.
Any work around to install scrappy? I cannot go down to Python 2.
Thanks!

Scrappy itself contains code that only works in Python 2. I.e. Scrappy cannot be used with Python 3+.

Read the Docs: Distutils2 error during installation

I try to install local instance of Read the Docs on my Win10
When I follow this documentation:
http://docs.readthedocs.io/en/latest/install.html
and type:
pip install -r requirements.txt
I get this error:
Collecting Distutils2==1.0a3 (from -r requirements/pip.txt (line 65))
Using cached Distutils2-1.0a3.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\USER\AppData\Local\Temp\pip-build-xseiavup\Distutils2\setup.py", line 9, in <module>
from distutils2.util import find_packages
File "C:\Users\USER\AppData\Local\Temp\pip-build-xseiavup\Distutils2\distutils2\util.py", line 174
except KeyError, var:
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\USER\AppData\Local\Temp\pip-build-xseiavup\Distutils2\
Anyone know this error?
My pip version: 8.1.2;
python: 3.5

Is it possible you are running the Python 2 pip? The error message clearly indicates that the code is being executed under Python 3 but has Python 2 syntax. Do you get better results with
python -m pip install -r requirements.txt
I wonder? If not then verify that
python
runs version 3.5 ...
Oh. I just checked and the disutils2 page on PyPI says
Distutils2 development is stopped.
tl;dr: keep using setuptools and pip for now, don’t use distutils2.
It looks like there was never a Python 3 version. Looks like you might need to update your code.

PyInstaller won't install, Python 3.6.0a4 and x64 Windows

I have said Python version (from https://www.python.org/downloads/windows/), and x64 Windows 10.
Every time I try to execute "pip install pyinstaller" it crashes with an error:
C:\WINDOWS\system32>pip install pyinstaller
Collecting pyinstaller
Using cached PyInstaller-3.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): setuptools in c:\users\jskurski\appdata\local\programs\python\python36\lib\site-packages (from pyinstaller)
Collecting pefile (from pyinstaller)
Using cached pefile-2016.3.28.tar.gz
Collecting pypiwin32 (from pyinstaller)
Using cached pypiwin32-219.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\jskurski\AppData\Local\Temp\pip-build-y9lsbd5f\pypiwin32\setup.py", line 121
print "Building pywin32", pywin32_version
^
SyntaxError: Missing parentheses in call to 'print'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\jskurski\AppData\Local\Temp\pip-build-y9lsbd5f\pypiwin32\
So, for me it seems there is a version msmatch or something. Unofortunately, I can not figure it out myself.
Any suggestions?
Has anybody sucessfully used PyInstaller with latest 3.6 Python on Windows? Or maybe I should downgrade Python to older version?
edit: tested on another PC (same enviroment) and it was the same.
edit2: seems to work on 3.5.2 version, so it's probably a way to go, for now.

pyinstaller needs pypiwin32 module.
when pip tries to install it, it shows an error because there is no pypiwin32 for python3.6

Case is closed for me, as I downgraded to stable 3.5.2. Probably some inconsistency in that alpha release, which caused this. I just wanted to write a simple GUI Windows program, so I will not investigate further.

You have to install manually pywin32 according to your version of python. The following link you can download.
https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/
Once installed pywin32 with your right version of python. Pyinstaller must be installed

As cdarke pointed out, you are running python 2 code on Python 3.
Try this instead:
pip3 install pyinstaller