I'm trying to use scrape on Mac OS X EI, and I have successfully installed scrapy using
pip install scrapy
in the terminal.
After that, I met the above problem when I tried to "import scrapy" in the Pycharm:
Traceback (most recent call last):
File "/Users/ziyuan/PycharmProjects/untitled/en.py", line 1, in <module>
import scrape
File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 48, in <module>
from scrapy.spiders import Spider
File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
from scrapy.http import Request
File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 12, in <module>
from scrapy.http.request.rpc import XmlRpcRequest
File "/Library/Python/2.7/site-packages/scrapy/http/request/rpc.py", line 7, in <module>
from six.moves import xmlrpc_client as xmlrpclib
ImportError: cannot import name xmlrpc_client
And I have tried everything from Scrapy throws ImportError: cannot import name xmlrpc_client
Specifically, when I tried the highest-voted solution, the second line
sudo rm -rf/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
throwed me another problem:
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.py: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.pyc: Operation not permitted
What should I do now? I have searched it in google for a long time and I really have no idea now.
A different way to deal with this problem, rather than highest-voted solution as you mentioned, is using other Python(not the system one) and virtualenv.
But why new Python? Here's an good article for the question. Reasons are easy to find.
And why virtualenv? You can get the point form here.
Furthermore, it's maybe not an good idea to use $ sudo pip (...). Please look at this.
So let's do the recommended way by one of Scrapy's contributor! Look at Mac OS X part: Using Homebrew to install new Python, updated your path variable, ...etc. Although she said using virtualenv is optional, I can only keep ImportError: blahblah.. away in my own virtual environment when using Scrapy.
After trying several solutions and attempting to work around the six 1.4.1 version, I found this worked:
sudo easy_install --upgrade six
Operation not permited means you don't have the permission to remove this. Try sudo rm instead to delete it as a Super User.
Though ideally you shouldn't rm anything from your python, try pip uninstall six instead. However you still need six so you might as well try upgrading it via pip install six --upgrade.
Note that you might have to use pip2 instead of pip because certain systems label it differently.
Related
I'm trying to use w3af to start doing some routine security testing on a webapp that I'm using. Install instructions recommend cloning a git repo, then running the python code and seeing what dependencies are unmet then installing them. My first run yielded:
ModuleNotFoundError: No module named 'ConfigParser
OK, no problem, right?
$ pip install ConfigParser
Collecting ConfigParser
Downloading configparser-5.2.0-py3-none-any.whl (19 kB)
Installing collected packages: ConfigParser
Successfully installed ConfigParser-5.2.0
Mission accomplished, let's try again!
$ ./w3af_console
Traceback (most recent call last):
File "./w3af_console", line 12, in <module>
from w3af.core.controllers.dependency_check.dependency_check import dependency_check
File "/Users/westonx/bin/w3af/w3af/core/controllers/dependency_check/dependency_check.py", line 26, in <module>
from w3af.core.data.db.startup_cfg import StartUpConfig
File "/Users/westonx/bin/w3af/w3af/core/data/db/startup_cfg.py", line 22, in <module>
import ConfigParser
ModuleNotFoundError: No module named 'ConfigParser'
Hmmm. Could swear we took care of that. Let's run pip (maybe pip3?) again to be sure?
$ pip3 install ConfigParser
Requirement already satisfied: ConfigParser in /Users/westonx/.pyenv/versions/3.8.2/lib/python3.8/site-packages (5.2.0)
Seems good. Let's check to see if the import path includes that directory:
$ python -c "import sys; print('\n'.join(sys.path)); import ConfigParser;"
/Users/westonx/.pyenv/versions/3.8.2/lib/python38.zip
/Users/westonx/.pyenv/versions/3.8.2/lib/python3.8
/Users/westonx/.pyenv/versions/3.8.2/lib/python3.8/lib-dynload
/Users/westonx/.pyenv/versions/3.8.2/lib/python3.8/site-packages
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ConfigParser'
So... we know the sys.path includes the directory pip says the module is installed in, but when we import it, python insists it's not there.
configparser-5.2.0.dist-info and configparser.py are indeed in my ~/.pyenv/versions/3.8.2/lib/python3.8/site-packages directory, so it doesn't look like pip telling me something that's not true. But it sure looks like python is.
I'm using pyenv on MacOS 10.14, not sure if that makes a difference. Anyone have ideas of what next steps should be?
ConfigParser is a built in library in Python, but its name was changed to configparser in Python 3. It appears like w3af is still using Python 2 (I found this check that actively states it, but you never even got that far). So, to run this code, you need to run it with Python 2.
There are very many (thousands) of python3 compatibility issues with the w3af codebase. The ConfigParser issue is just the first one you hit - the codebase is full of references to outdated / unmaintained libraries, and has no support for python3 byte strings vs. unicode strings (b"" vs ""), which are used throughout the codebase.
There is a w4af project on Github, where someone has made the effort to port all the w3af code to python3, but you didn't hear that from me.
Recently, (I have no idea when), I installed this package called pymol. Today when I was trying to copy a list of all my install modules, I encountered this error.
~/Projects$ python -c "help('modules')"
Please wait a moment while I gather a list of all available modules...
Error: unable to initalize the pymol.cmd module
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pymol/cmd.py", line 117, in <module>
from chempy import io
ImportError: cannot import name io
I've tried
sudo pip uninstall pymol
This definitely worked... but it also definitely did not work. It removed the .egg, and pip list tells me it's not installed, but the tool is still there. I open a python prompt and import it with no problem. Now my question is, if I just delete the package, is there anything that could go wrong, (WCS I'm thinking kafkaesque broken dependency fixing for the rest of my natural life.) and is there a standardized, best practice to remove this, or any other package manually.
I recently started out with python and i'm right now looking to do an app that shows my Twitter feed. So I downloaded a module(not sure if its called that) called Python-Twitter and installed it. But now everytime I try to import it I just get this error:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import twitter
File "build\bdist.win32\egg\twitter.py", line 38, in <module>
File "C:\Python27\lib\site-packages\requests_oauthlib-0.4.0-py2.7.egg\requests_oauthlib\__init__.py", line 1, in <module>
from .oauth1_auth import OAuth1
File "C:\Python27\lib\site-packages\requests_oauthlib-0.4.0-py2.7.egg\requests_oauthlib\oauth1_auth.py", line 4, in <module>
from requests.utils import to_native_string
ImportError: cannot import name to_native_string
Does anyone know what I might have done wrong with the install or something? Thx
In my case installing requests-oauthlib fixed it
sudo pip install requests-oauthlib
This is because of requests version.
requests.utils.to_native_string is available since requests 2.0.0
So just update requests:
pip install -U requests
See more details here on another thread
This sounds like an issue with your install. The method to_native_string should be defined in the requests\utils.py in your Python install.
I'm on Ubuntu, and installed the Twitter module, and can import it without any errors. When I look in my install, in /usr/lib/python2.7/dist-packages/requests/utils.py, there is a to_native_string method defined there.
The implication from the error you're getting is that in your installation, there is either no utils.py, or, if there is, it does not contain that method.
I would recommend checking your install to see if that is the case. If it is indeed missing, I would recommend axing your install and reinstalling. You might want to try a virtualenv environment instead, so it can act as a sandbox (in that case you can leave your current install as is, as long as it is intact enough to run virtualenv and pip).
If utils.py is actually there and does contain a method with that name, I would recommend running a debugger such as pudb or pdb (pudb isn't built in, but it's more full featured), and step through the import to see if that sheds any additional light.
I am trying to configure svn hook for email notification using mailer.py script from apache, but it always crash on import of svn libs. I try two ways of installing svn and python. I try to install everything manually using command line and use CollableNet installer that have preinstalled Python and all libs I need but result is the same.
Now I try to import one library from hole package that I need using python in command line and I have such response:
>>> import svn.core
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named svn.core
I try to fix it using:
1) import sys; sys.path.append("C:\csvn\lib\svn-python\svn")
and
2) C:\csvn\lib\svn-python\svn>python "\__init__.py" install
But it didn't help.
For now I have no package with setup.py to install it.
I spend a lot of time on this task, and going crazy a little). Please give me any suggestion how to fix it.
I am on Ubuntu 13.04. I get the following error message -
Traceback (most recent call last):
File "analyse.py", line 1, in <module>
from log import shelve
File "/home/shubham/SMART/TaxiData/log.py", line 27, in <module>
from demo import *
File "/home/shubham/SMART/zones/demo.py", line 5, in <module>
from qgis.core import *
ImportError: No module named qgis.core
Actually, everything was working fine till today morning. I guess this might be due to a package update.
I tried looking around on Google but my search was fruitless. So, I will really appreciate any help or pointers you guys can give :)
Thanks.
I solved the problem by completely removing the installation and using the nightly builds at 'deb http://qgis.org/debian-nightly raring main'.
If you have pip installed you could either try pip search qgis or pip freeze. The latter shows a list of all installed python packages to check if you have the package. Maybe try reinstalling qgis ...
Try sudo apt-get install python-qgis . The package does not appear to be available on PyPI, but it was listed as "python-qgis" in the Ubuntu packages.