I've successfully installed BeautifulSoup4 in 2.7 and 3.6 in the terminal, but when I use either Spyder or Jupyter Notebook, I get the following error when trying to import bs4 when using the Python 3.6 kernel
from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'
I've tried the following Terminal commands:
sudo apt-get install python3-bs4
python3.6 -m pip install bs4
py3.6 -m pip install bs4
sudo pip3 install BeautifulSoup4
python3.6 /Users/MY-NAME/anaconda/bin/pip install bs4
NB: I have created separate kernels for the two versions of python:
py27
py3.6
Any other solutions that I haven't tried or anything that I've obviously missed/misunderstood?
Thanks in advance for your help!
I figured out what was going wrong.
So for anyone else running into this problem, I had to do the following:
1. Activate the environment
Activate the environment (i.e. the Kernel for the python version) by using the terminal command:
$ source activate NAME_OF_YOUR_KERNEL
For which I typed in:
$ source activate py3.6
2. Install the target package
Install the target package using the terminal command:
$ conda install TARGET_PACKAGE_NAME
For which I typed in:
$ conda install beautifulsoup4
3. Deactivate your environment
$ source deactivate NAME_OF_YOUR_KERNEL
For which I typed:
$ source deactivate py3.6
I tried to setup Scrapy on Windows 7 by steps described at http://doc.scrapy.org/en/latest/intro/install.html . On my PC was installed Python 3.5.1. Although Scrapy not support this python version it was installed successfully with latest Anaconda but fails to run spider script. I find that Scrapy only works with Python 3.3.+ version so uninstall version 3.5.1, uninstall Anaconda, install python 3.3.5, install pywin32 and install pip. pip fails pip install Scrapy, so I install Anaconda and run conda install -c scrapinghub scrapy Scrapy installed, but I saw that libs installed was for python 3.5 like: scrapy: 1.1.0-py35_0
Now I run the
c:\python\olxscrapy>scrapy crawl OlxCatalogSpider
and get error
File "C:\Anaconda3\lib\site-packages\twisted\internet\stdio.py", line 30, in
module>
from twisted.internet import _win32stdio
ImportError: cannot import name '_win32stdio'
How to make Scrapy run with python 3.3.+
On this blog:
https://blog.scrapinghub.com/2016/05/25/data-extraction-with-scrapy-and-python-3/
it says Scrapy on Python 3 doesn't work in Windows environments yet
Edit:
I recently installed scrapy on Ubuntu for Python 3.5 and received a lot of errors. The errors stopped after: "sudo apt-get install python3.5-dev".
I add the follow package and it works:
pip install twisted-win==0.5.5
Try to create a virtual env:
pip install virtualenv (instalation)
virtualenv -p python3.3.5 envName (creation with specific python version)
source ./envName/bin/activate (activate virtual env)
This way you can guarantee that's the right python version. Also scrapy has some requirements that can't be installed via pip and this may cause your pip install scrapy to fail
So install at your computer:
python-dev
libxslt1-dev
libxslt1.1
libxml2-dev
libxml2
libssl-dev
After this you finaly be able to install scrapy via pip inside your virtual env (probably)
Sry for my poor English isn't my native lang. Hope this work =]
Installation of Scrapy on Windows may facing error while installing Twisted.
Download Twisted according to your Python and windows version on this site http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
Turn to your download folder and pip install <downloaded filename>
pip install scrapy
I am attempting to install Scrapy on my Windows 7 64 bit machine. I started by following the instructions here on Scrapy's documentation.
I got up until the command 'pip install Scrapy'. Everything works except that it cannot find 'libxml2':
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
I then visited this website to get the binaries of libxml2:
ftp://ftp.zlatkovic.com/libxml/64bit/
The instructions for installation of libxml2 are here: https://www.zlatkovic.com/libxml.en.html
They state that you should unzip the binaries and place the contents of the BIN folder in a path such as C:\WINDOWS. I did this. However, after attempting to install Scrapy again, I continue to receive the same error. Is there something I am missing?
Just installled Scrapy on Windows10 x64. It installs only with Python 2.7 on Windows. I used tutotial from here.
Download from the official site and install Python 2.7
Open Command promt (cmd) with full admin permissions and type:
c:\python27\python.exe c:\python27\tools\scripts\win_add2path.py
and press Enter
Install Microsoft Visual C++ for Python 2.7
Install OpenSSL (sorry, can't attach link because of small reputation). Download it to some folder, e.g. C:\Python27\Scripts and execute command from there: pip install pyOpenSSL-16.0.0-py2.py3-none-any.whl (the name of file could not be the same)
Install lxml just like previous file: pip install lxml-3.6.0-cp27-cp27m-win32.whl
Install Pywin32
And finally, install Scrapy - pip install Scrapy
Can recommend to download and install all needed soft for x32 architecture (even if you have x64 system).
It is recommended to use conda to install Scrapy on Windows.
You can download Miniconda from here: http://conda.pydata.org/miniconda.html
Once you have conda installed you can follow this guide to learn about its subcommands: http://conda.pydata.org/docs/test-drive.html
In short, you can use this commands in your terminal:
Create an environment for scrapy: conda create -n scrapyenv python=2
Enable the scrapy environment: activate scrapyenv
Install scrapy: conda install -c scrapinghub scrapy
Edit: Scrapy's conda package has been moved to conda-forge channel. Use the command: conda install -c conda-forge scrapy
Hi this is what I did to get it installed under Win7/Win10:
easy_install lxml
pip install Scrapy
install pywin32 as indicated by the docs
Then, follow the tutorial to be sure everything is working
I have both Python 2.7 and Python 3.5 installed. When I type pip install beautifulsoup4 it tells me that it is already installed in python2.7/site-package directory.
But how do I install it into the python3 dir?
I think pip3 will satisfy your needs, use the below command on the terminal:
pip3 install beautifulsoup4
See doc
Run as root:
apt-get install python3-bs4
#or
pip3 install beautifulsoup4
Afterwards import it like this:
import bs4
If you’re using a recent version of Debian or Ubuntu Linux, you can install Beautiful Soup with the system package manager:
$ apt-get install python-bs4 (for Python 2)
$ apt-get install python3-bs4 (for Python 3)
Afer install import the library
from bs4 import BeautifulSoup
The package is bs4 and can be installed with following command:
python -m pip install bs4
I had some mismatch between Python version and Beautifulsoup. I was installing this project
Th3Jock3R/LinuxGSM-Arma3-Mod-Update
to a linux centos8 dedicated Arma3 server. Python3 and Beautifulsoup4 seem to match.So I updated Python3, removed manually Beautifulsoup files and re-installed it with: sudo yum install python3-beautifulsoup4 (note the number 3). Works. Then pointing directories in Th3Jock3R:s script A3_SERVER_FOLDER = "" and A3_SERVER_DIR = "/home/arma3server{}".format(A3_SERVER_FOLDER) placing and running the script in same folder /home/arma3server with python3 update.py. In this folder is also located new folder called 'modlists' Now the lightness of mod loading blows my mind. -Bob-
If you are on windows, this works for Python3 as well
py -m pip install bs4
I tried importing requests:
import requests
But I get an error:
ImportError: No module named requests
Requests is not a built in module (does not come with the default python installation), so you will have to install it:
OSX/Linux
Python 2: sudo pip install requests
Python 3: sudo pip3 install requests
if you have pip installed (pip is the package installer for python and should come by default with your python installation).
If pip is installed but not in your path you can use python -m pip install requests (or python3 -m pip install requests for python3)
Alternatively you can also use sudo easy_install -U requests if you have easy_install installed.
Linux
Alternatively you can use your systems package manager:
For centos: sudo yum install python-requests
For Debian/Ubuntu Python2: sudo apt-get install python-requests
For Debian/Ubuntu Python3: sudo apt-get install python3-requests
Windows
Use pip install requests (or pip3 install requests for python3) if you have pip installed and Pip.exe added to the Path Environment Variable. If pip is installed but not in your path you can use python -m pip install requests (or python3 -m pip install requests for python3)
Alternatively from a cmd prompt, use > Path\easy_install.exe requests, where Path is your Python*\Scripts folder, if it was installed. (For example: C:\Python32\Scripts)
If you manually want to add a library to a windows machine, you can download the compressed library, uncompress it, and then place it into the Lib\site-packages folder of your python path. (For example: C:\Python27\Lib\site-packages)
From Source (Universal)
For any missing library, the source is usually available at https://pypi.python.org/pypi/. You can download requests here: https://pypi.python.org/pypi/requests
On mac osx and windows, after downloading the source zip, uncompress it and from the termiminal/cmd run python setup.py install from the uncompressed dir.
(source)
It's not obvious to me which version of Python you are using.
If it's Python 3, a solution would be sudo pip3 install requests
To install requests module on Debian/Ubuntu for Python2:
$ sudo apt-get install python-requests
And for Python3 the command is:
$ sudo apt-get install python3-requests
Brew users can use reference below,
command to install requests:
python3 -m pip install requests
Homebrew and Python
pip is the package installer for Python and you need the package requests.
This may be a liittle bit too late but this command can be run even when pip path is not set. I am using Python 3.7 running on Windows 10 and this is the command
py -m pip install requests
and you can also replace 'requests' with any other uninstalled library
If you are using Ubuntu, there is need to install requests
run this command:
pip install requests
if you face permission denied error, use sudo before command:
sudo pip install requests
In my case requests was already installed, but needed an upgrade. The following command did the trick
$ sudo pip install requests --upgrade
On OSX, the command will depend on the flavour of python installation you have.
Python 2.x - Default
sudo pip install requests
Python 3.x
sudo pip3 install requests
On Windows Open Command Line
pip3 install requests
I had the same issue, so I copied the folder named "requests" from https://pypi.python.org/pypi/requests#downloadsrequests download to
"/Library/Python/2.7/site-packages".
Now when you use: import requests, it should work fine.
In the terminal/command-line:
pip install requests
then use it inside your Python script by:
import requests
or else if you want to use pycharm IDE to install a package:
go to setting from File in menu
next go to Python interpreter
click on pip
search for requests package and install it
Adding Third-party Packages to the Application
Follow this link
https://cloud.google.com/appengine/docs/python/tools/libraries27?hl=en#vendoring
step1 : Have a file by named a file named appengine_config.py in the root of your project, then add these lines:
from google.appengine.ext import vendor
Add any libraries installed in the "lib" folder.
vendor.add('lib')
Step 2: create a directory and name it "lib" under root directory of project.
step 3: use pip install -t lib requests
step 4 : deploy to app engine.
Try sudo apt-get install python-requests.
This worked for me.
If you are using anaconda as your python package manager, execute the following:
conda install -c anaconda requests
Installing requests through pip didn't help me.
The only thing that worked for me:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
pip install requests
Facing the same issue but unable to fix it with the above solution, so I tried this way and it worked:-
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py
sudo python2 get-pip.py
python -m pip install requests
For windows just give path as cd and path to the "Scripts" of python and then execute the command easy_install.exe requests.Then try import requests...
I have had this issue a couple times in the past few months. I haven't seen a good solution for fedora systems posted, so here's yet another solution. I'm using RHEL7, and I discovered the following:
If you have urllib3 installed via pip, and requests installed via yum you will have issues, even if you have the correct packages installed. The same will apply if you have urllib3 installed via yum, and requests installed via pip. Here's what I did to fix the issue:
sudo pip uninstall requests
sudo pip uninstall urllib3
sudo yum remove python-urllib3
sudo yum remove python-requests
(confirm that all those libraries have been removed)
sudo yum install python-urllib3
sudo yum install python-requests
Just be aware that this will only work for systems that are running Fedora, Redhat, or CentOS.
Sources:
This very question (in the comments to this answer).
This github issue.
Python Common installation issues
These commands are also useful if Homebrew screws up your path on macOS.
python -m pip install requests
or
python3 -m pip install requests
Multiple versions of Python installed in parallel?
You must make sure your requests module is not being installed in a more recent version of python.
When using python 3.7, run your python file like:
python3 myfile.py
or enter python interactive mode with:
python3
Yes, this works for me. Run your file like this: python3 file.py
I have installed python2.7 and python3.6
Open Command Line to ~/.bash_profile I find that #Setting PATH for Python 3.6 , So
I change the path to PATH="/usr/local/Cellar/python/2.7.13/bin:${PATH}" ,
(please make sure your python2.7's path) ,then save.
It works for me.
if you want request import on windows:
pip install request
then beautifulsoup4 for:
pip3 install beautifulsoup4
Please try the following. If one doesn't work, skip to the next method.
pip install requests
or...
pip3 install requests
or...
python -m pip install requests
or...
python3 -m pip install requests
or...
python -m pip3 install requests
If all of these don't work, please leave a comment!
How does this work? Depending on the operating system you currently use, the pip command may vary or not work on some. These are the commands you may try in order for a fix.
In case you hit pip install requests and had an output massage of Requirement already satisfied but yet you still get the error: ImportError: No module named requests.
This is likely to happen when you find yourself in a different interpreter/virtual environment.
You can copy and append the path of the module into your working environment.
Note: This path usually comes with the message Requirement already satisfied
Before import requests, you should import sys and then append the copied path.
Example:
Command Prompt:
pip install requests
Output:
Requirement already satisfied: requests in /usr/local/lib/python3.9/site-packages
import sys
sys.path.append("/usr/local/lib/python3.9/site-packages")
import requests
I solved this problem.You can try this method.
In this file '.bash_profile', Add codes like alias python=/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Type this command in Command Prompt (Windows) or Terminal (Linux/macOS):
pip install requests
My answer is basically the same as #pi-k. In my case my program worked locally but failed to build on QA servers. (I suspect devops had older versions of the package blocked and my version must have been too out-of-date) I just decided to upgrade everything
$ pip install pip-review
$ pip-review --local --interactive
You get an import error because requests are not a built-in module instead, it is created by someone else and you need to install the requests.
use the following command on your terminal then it will work correctly.
pip install requests
Install python requests library and this error will be solved.
I found that my issue was VSCode was reading from the wrong Python Interpreter. This youtube tutorial solved it for me.
If you are using anaconda
step 1:
where python
step 2:
open anaconda prompt in administrator mode
step 3:
cd <python path>
step 4:
install the package in this location