I'm wondering how to use BeautifulSoup4 on the Heroku Server.
In my requirements.txt file I have written: "BeautifulSoup==4.9.1",
but I get an Error:
ERROR: Could not find a version that satisfies the requirement BeautifulSoup==4 (from -r /tmp/build_d624b770_/requirements.txt (line 1)) (from versions: 3.2.0, 3.2.1, 3.2.2)
ERROR: No matching distribution found for BeautifulSoup==4 (from -r /tmp/build_d624b770_/requirements.txt (line 1))
So how can I use BeautifulSoup4 then?
Since I cant see your requirements.txt file, from seeing the error I am assuming it has something like BeautifulSoup==4, but the way of installing BeautifulSoup is pip install bs4 since its not like version 4 of BS, its is bs4 itself, so you requirements.txt must contain bs4==0.0.0 or bs4==0.0.1 or just plain bs4 with any other dependencies
Learn more about BS4 releases here
Related
I am trying to import certain packages as I am working with Jupyter notebook files, and most of the packages seem to be missing, even though I have installed them. For example, when I do the command: from bs4 import BeautifulSoup or import requests
I get the error saying ModuleNotFoundError: No module named 'bs4' for the first one and a similar one for importing requests as well. I have tried pip install requests and pip install bs4, but same issue persists. I have installed them on:
"(base) aminnazemzadeh#amins-MacBook-Pro ~ % " which seems to be my home directory, and I also have anaconda3 installed alongside python3. What is the issue that I cannot import these modules.
I am using visual studio if it makes any difference
Once I add :
!pip install requests
!pip install bs4
I get:
/Users/aminnazemzadeh/.zshenv:.:1: no such file or directory: /Users/aminnazemzadeh/.cargo/env
Requirement already satisfied: requests in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (2.28.1)
Requirement already satisfied: charset-normalizer<3,>=2 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (2.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (1.26.11)
Requirement already satisfied: idna<4,>=2.5 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (2022.9.24)
/Users/aminnazemzadeh/.zshenv:.:1: no such file or directory: /Users/aminnazemzadeh/.cargo/env
Requirement already satisfied: bs4 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (0.0.1)
Requirement already satisfied: beautifulsoup4 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from bs4) (4.11.1)
Requirement already satisfied: soupsieve>1.2 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from beautifulsoup4->bs4) (2.3.1)
followed by this warning:
ModuleNotFoundError Traceback (most recent call last)
Cell In[7], line 4
2 get_ipython().system('pip install bs4')
3 from urllib.request import urlopen
----> 4 from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'
Thanks
probably you're installing the packages on an environment other than the one vs code is using. you can try installing the packages directly from your jupyter notebook by running the following code in a notebook cell. the current best practice to be running installs in the notebook is using the magic commands %pip or %conda:
%pip install requests beautifulsoup4
# or
%conda install requests beautifulsoup4
this should install the packages in the same environment that the notebook is running on.
note that you may need to restart the kernel to use the affected packages.
sources:
Jupyter Discourse Forum - Location of libraries or extensions installed in JupyterLab
Jupyter Discourse Forum - Why users can install modules from pip but not from conda?
Installing Beautiful Soup
ps: thanks #wayne for the comments regarding the current best practices for installing on the current running environment.
If you're using conda, you should install via conda whenever possible. When you install via pip, conda loses some of its ability to manage dependency versions.
Try creating a new conda environment, install the needed packages via conda, then set the kernel to your new environment in vscode. Dedicate conda environments to specific projects. It is okay to have a default/generic environment for playing around but not for any significant work as you can easily create errors in your other work if a dependency changes to an incompatible version.
Conda cheet sheet for reference if you need it: https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf
You will need the Jupyter extension in vscode if you do not already have it installed.
You will also need to install the corresponding jupyter package in your conda environment.
I tried running git push heroku master in cmd but got the following error:
ERROR: Could not find a version that satisfies the requirement get (from -r /tmp/build_8801ce40/requirements.txt (line 2)) (from versions: none)
ERROR: No matching distribution found for get (from -r /tmp/build_8801ce40/requirements.txt (line 2))
remote: ! Push rejected, failed to compile Python app.
I have listed in my requirements.txt as below:
Flask==0.11.1
get
gunicorn==19.6.0
post
requests
wikipedia
line-bot-sdk
pandas==0.20.3
googletrans
What is the problem, and how can I fix it?
Your requirements.txt should only contain modules that you need to install. get and post almost certainly don't belong here.
If you are doing something like
from requests import get
you'll need requests in your file. But you don't need get—it comes with requests. That's what the from requests part means.
Try editing your requirements.txt to something more like this:
Flask==0.11.1
gunicorn==19.6.0
requests
wikipedia
line-bot-sdk
pandas==0.20.3
googletrans
Then commit and redeploy.
A good way to make sure that your requirements.txt contains what it should is to use it in development, too. Add new dependencies directly to that file and then pip install -r requirements.txt. In this case, you would have caught the problem earlier.
I have a little (big) problem that I can't solve , I tried to find a solution to it but I can't get anything. Maybe you will see it better than me. it seems it can't import a package, Here is the error :
Build failed: `pip_download_wheels` had stderr output:
ERROR: Could not find a version that satisfies the requirement fr-core-news-md==2.2.5 (from -r requirements.txt (line 38)) (from versions: none)
ERROR: No matching distribution found for fr-core-news-md==2.2.5 (from -r requirements.txt (line 38))
error: `pip_download_wheels` returned code: 1; Error ID: D670E3AB
my main.py file:
import fr_core_news_md
my requirements.txt:
fr-core-news-md==2.2.5
Thank you
Cloud Functions is failing to install the fr-core-news-md dependency because there is no fr-core-news-md on PyPI: https://pypi.org/project/fr-core-news-md/
The command
pip install pyteaser
produces this error:
Collecting pyteaser
Using cached https://files.pythonhosted.org/packages/d4/7a/310592c6e7998440e56a8650446ecf3ded076431415c60f0f3b946b54462/pyteaser-2.0.tar.gz
Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from pyteaser) (5.1.0)
Requirement already satisfied: lxml in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pyteaser) (4.4.2)
Collecting cssselect (from pyteaser)
Using cached https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl
Collecting jieba (from pyteaser)
Using cached https://files.pythonhosted.org/packages/b2/c8/e0785494690876adbb5ac364eb148a240b738a614d7b39bf8dba8a7ad066/jieba-0.41.tar.gz
Collecting beautifulsoup (from pyteaser)
Using cached https://files.pythonhosted.org/packages/40/f2/6c9f2f3e696ee6a1fb0e4d7850617e224ed2b0b1e872110abffeca2a09d4/BeautifulSoup-3.2.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pycharm-packaging/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pycharm-packaging/beautifulsoup/
And the command
pip install pytldr
reports this error:
pip install pytldr
Collecting pytldr
Using cached https://files.pythonhosted.org/packages/70/09/02ed27061159e5f6d35abad4ec9ef3cac8e220093d61a2f7a42f53c9cb22/PyTLDR-0.1.4.tar.gz
Requirement already satisfied: numpy>=1.8.0 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pytldr) (1.18.1)
Requirement already satisfied: nltk>=3.0.0 in /home/abhishek/.local/lib/python3.6/site-packages (from pytldr) (3.4.5)
Requirement already satisfied: scipy>=0.13.2 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from pytldr) (1.4.1)
Collecting scikit-learn>=0.15.2 (from pytldr)
Using cached https://files.pythonhosted.org/packages/d1/48/e9fa9e252abcd1447eff6f9257636af31758a6e46fd5ce5d3c879f6907cb/scikit_learn-0.22.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting goose-extractor>=1.0.25 (from pytldr)
Using cached https://files.pythonhosted.org/packages/ab/f3/2e89688e4e1d8786e542ea460431f24f1392d885e60303dc20d476a96424/goose-extractor-1.0.25.tar.gz
Collecting networkx>=1.9.1 (from pytldr)
Using cached https://files.pythonhosted.org/packages/41/8f/dd6a8e85946def36e4f2c69c84219af0fa5e832b018c970e92f2ad337e45/networkx-2.4-py3-none-any.whl
Requirement already satisfied: six in /home/abhishek/.local/lib/python3.6/site-packages (from nltk>=3.0.0->pytldr) (1.12.0)
Collecting joblib>=0.11 (from scikit-learn>=0.15.2->pytldr)
Using cached https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl
Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from goose-extractor>=1.0.25->pytldr) (5.1.0)
Requirement already satisfied: lxml in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from goose-extractor>=1.0.25->pytldr) (4.4.2)
Collecting cssselect (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl
Collecting jieba (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/b2/c8/e0785494690876adbb5ac364eb148a240b738a614d7b39bf8dba8a7ad066/jieba-0.41.tar.gz
Collecting beautifulsoup (from goose-extractor>=1.0.25->pytldr)
Using cached https://files.pythonhosted.org/packages/40/f2/6c9f2f3e696ee6a1fb0e4d7850617e224ed2b0b1e872110abffeca2a09d4/BeautifulSoup-3.2.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-9oa07p4e/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-9oa07p4e/beautifulsoup/
Note: you may need to restart the kernel to use updated packages.
EDIT:
After executing the command for BeautifulSoup
pip install beautifulsoup4
the response is:
Requirement already satisfied: beautifulsoup4 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (4.8.2)
Requirement already satisfied: soupsieve>=1.2 in /home/abhishek/PycharmProjects/learning-python/venv/lib/python3.6/site-packages (from beautifulsoup4) (1.9.5)
Note: you may need to restart the kernel to use updated packages.
The problem here is that the library is not (yet) ready for Python3, as you can see here in a comment on an issue of this library. And that is the source of your problem. As you can see, there are no issues if you install the library for Python2.7:
$ sudo apt install python-pip # or https://pip.pypa.io/en/stable/installing/
$ python --version
Python 2.7.17
$ python -m pip install pytldr
Successfully installed Pillow-6.2.2 beautifulsoup-3.2.2 cssselect-1.1.0 decorator-4.4.2 goose-extractor-1.0.25 jieba-0.42.1 lxml-4.5.0 networkx-2.2 nltk-3.4.5 numpy-1.16.6 pytldr-0.1.4 scikit-learn-0.20.4 scipy-1.2.3 singledispatch-3.4.0.3 six-1.14.0
Here, you can see that everything worked, because beautifulsoup-3.2.2 version was installed. Trying the same command for Python3 leads to the error you're posted:
$ python3 --version
Python 3.6.9
$ python3 -m pip install pytldr
Collecting beautifulsoup
Using cached BeautifulSoup-3.2.2.tar.gz (32 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-12aluvvq/beautifulsoup/setup.py'"'"'; __file__='"'"'/tmp/pip-install-12aluvvq/beautifulsoup/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-12aluvvq/beautifulsoup/pip-egg-info
cwd: /tmp/pip-install-12aluvvq/beautifulsoup/
Complete output (6 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-12aluvvq/beautifulsoup/setup.py", line 3
"You're trying to run a very old release of Beautiful Soup under Python 3. This will not work."<>"Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'."
^
SyntaxError: invalid syntax
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The error message is very clear:
You're trying to run a very old release of Beautiful Soup under Python 3. This will not work. Please use Beautiful Soup 4, available through the pip package 'beautifulsoup4'.
Simply installing BeautifulSoup4 is not solving the problem, because the library still uses code for BeautifulSoup3. That's a dependency of the pytldr package.
Now you have several options, maybe more:
Do not use this library.
Use Python2 (But it's the year 2020).
There is already an open pull request that offers code that allegedly has updated the library to be compliant with Python3. You can build the library from source with code provided there (no guarantee that this will work) or wait until the pull request was approved (or write to the author).
Go through the forks and see if anyone has already got the library up and running for Python3.
Try to update the library yourself and maybe with the help of 2to3.
You can apparently also install this fork directly instead of step 3. Of course, this fork may be out-of-date at some point, or it may be modified in such a way that it is no longer usable. There is also the option of installing a specific commit. But also no guarantee that this will work.
The pyteaser and goose-extractor libraries simply use the BeautifulSoup 3 library, which uses a different name from BeautifulSoup 4. You can't just install BeautifulSoup 4, because pyteaser and goose-extractor need to be updated to use the newer BeautifulSoup library. That's going to be more work, because those 2 projects are not compatible with Python 3 either.
You are basically stuck with Python 2 for these projects unless you are willing to do all the upgrade work yourself or put your trust in incomplete forks:
Unfortunately, even the forks that purport to make pyteaser Python 3 compatible have left in the wrong dependency. The project doesn't actually even import BeautifulSoup anywhere, so you'd have to clone one of the Python 3 forks (like this one) and remove the beautifulsoup entry from the install_requires line.
For PyTLDR, this fork can be installed on Python 3:
pip install git+https://github.com/vinodnimbalkar/PyTLDR.git#egg=pytldr
It dropped goose altogether.
Your mileage may vary, however, since the conversion looks, to me, somewhat haphazard. For Unicode text (the default in Python 3), normalization will never take place (both the if and elif branches test for the same condition now), for example. And while it removed the goose-extractor dependency, the code still tries to import it if you referenced it, which won't work either.
If you wanted to use goose-extractor directly, you can replace with goose3. Note that this also changed the module name, to goose3 (so the PyTLDR fork above could perhaps be fixed by changing the imported module name).
I am trying to deploy my bot on Heroku through GitHub.
In requirements.txt, I have 'os'. I tried to deploy the bot so Heroku would recognise the Procfile. It gave me this error.
Collecting os (from -r /tmp/build_e656b8b6627c307b02ef77c47e6bb725/requirements.txt (line 3))
Could not find a version that satisfies the requirement os (from -r /tmp/build_e656b8b6627c307b02ef77c47e6bb725/requirements.txt (line 3)) (from versions: )
No matching distribution found for os (from -r /tmp/build_e656b8b6627c307b02ef77c47e6bb725/requirements.txt (line 3))
! Push rejected, failed to compile Python app.
! Push failed
How to fix it?
You don't need to include os in your requirements.txt - it's a built-in module. Just remove it.
You don't need to install/require os. It isn't on the package index, it's a built-in package.
Simply remove the os line from your requirements.txt file to fix this issue.