i have been struggling with an issue for days.
My situation is this: i want to get into hadoop and realtime analytics. As ax example, i want to scrape some prices from a particular website. The script works on my laptop but when i move it to NIFI it starts giving issue. I have a Raspbarian Desktop virtual environment setup on my laptop. I am using NIFI to get data from an ExecuteScript. I have this code which works on my native laptop:
import requests
from bs4 import BeautifulSoup
from html import HTML
page = requests.get('https://web.archive.org/web/20121007172955/https://www.nga.gov/collection/anZ1.htm')
# Create a BeautifulSoup object
soup = BeautifulSoup(page.text, 'html.parser')
As soon as i put it into NIFI, it start complaining about "no module named HTML in line 2. I know this is a module issue but i just don't have the necessary experience to identify the root cause and fix the issue.
Things to note:
I switched form python 2.7 to python 3.5 by using the alternative method.
in the Execute Script component, i set the module directory to the following:
/usr/local/lib/python3.5/dist-packages,/usr/lib/python3/dist-packages
Is there anyone which is familiar with this environment to assist me with the missing modules and how to fix this ?
Related
I am using a particular version of Python (v 2.7.0) that doesn't allow me to use webbrowser or selenium libraries.
It also doesn't recognise os.startfile.
Is there another module to use that allows me to open up a locally addressed html file in my browser?
Since you are using windows you can use os.startfile which will start a file with its associated application.
>>> import os
>>> os.startfile("PATH_TO_HTML_FILE")
I fumbled on a result by treating it as an exe and project. Subprocess module worked. Tested a little further and should work for Jython 2.7.0 also.
import subprocess as sp
sp.Popen(["browser_exe_path","html_path"])
Note - if using brave browser - it's exe wasn't recognised. chrome_proxy.exe worked in it's place.
Happy to learn something via feedback.
The same question has been asked a number of times but I couldn't find the solution.
After I install a package using pip, I am able to import it in python console or python file and it works as expected.
The same package when I try to include in django, it gives import error.
Do I need to modify settings.py file or any requirement that I need to add? I am driving django with the help of virtual env.
Eg:
I am using BeautifulSoup and I am trying to import from bs4 import BeautifulSoup and I am getting error ImportError: No module named 'bs4'
This error only comes in django. I am not able to figure out why this is happening.
Screenshot attached for reference.
1. python console - shows no error
2. django console- import error
I am sorry as it is difficult to read the console but any other thing that I can include which will help me make myself more clear will be appreciated.
You don't show either the code of your site or the command you ran (and the URL you entered, if any) to trigger this issue. There's almost certainly some difference between the Python environment on the command line and that operating in Django.
Are you using virtual environments? If so, dependencies should be separately added to each environment. Were you operating from a different working directory? Python usually has the current directory somewhere in sys.path, so if you changed directories it's possible you made bs4 unavailable that way.
At the interactive Python prompt, try
import bs4
bs4.__file__
That will tell you where bs4 is being imported from, and might therefore give you a clue as to why it's not available to Django.
I'm using netbeans to write a simple python program which I need the requests module for, I've downloaded requests through terminal and it all seems to be fine there but netbeans can't seem to find it.
This is the error that it's throwing up:
import requests
ImportError: No module named requests
I've tried installing the requests library directly into the python folder but the folder won't let me paste anything into it.
There do seem to be answers on the netbeans forums but their server is down so won't let me on their website to my annoyance!
EDIT
I've tried to run python setup.py install as per other answers on the website but had no luck.
EDIT
have tried completely uninstalling python and requests to make sure it wasn't an installation error but still no luck.
This clearly looks like an error of installation of the request module to some other place than where your netbeans expects when running the code.
In your console run
which python
Check if this gives the same path as the one set in your netbeans. You can set your path by adding new platform using Tools > Python Platforms > New:
I would suggest that you learn bit more about sandboxed environments such as virtualenv. This article shows how you can use a virtualenv to install packages and use the same virtualenv for netbeans so that whatever packages you install in the virtualenv will be available in the netbeans for you to use. For this case, it could be requests.
In the end I gave up with requests, as I was using requests to get json data from an API I decided just to go back to the drawing board and start over rather than attempt to fix something that I couldn't work out. I am now using the urllib import and whilst this may not be the most efficient way it works which is the most important thing.
I am trying to authenticate jawbone api in python. In the code there is a line:
import requests
How can I add this. I have very little knowledge on python. Just manipulating the code. Can any one please help? The library is already present in python 3.3
This is only for the users using python33 and in windows platform,,,,
download Requests packages from any site.
Copy the folder Requests from the downloaded package and place it on C://python33/LIB/ folder....
now you are able to import Requests package to your program
I've installed BeautifulSoup for Python 3 (python3-BeautifulSoup-1:3.2.0-4.fc16) on Fedora 16(64bit) using the Add/Remove Software GUI tool. It appears to have installed correctly (yum info python3-BeautifulSoup shows 'Repo: Installed') but when I try to import BeautifulSoup (using import BeautifulSoup statement) in the interactive shell I get an 'ImportError: No module named BeautifulSoup' message.
I've also got BeautifulSoup for Python 2 (python-BeautifulSoup-1:3.2.0-4.fc16) installed h working fine with Python 2.7.2.
The summary appears weird (Monitor filesystem events with Python under Linux) though the descrition makes sense. Where do I start?
I'm new to Python (trying to learn Python3 in my spare time) and I'm not a programmer. I'm trying to learn by writing simple code and following code examples (hence the need for BeautifulSoup.) Please assume minimal Python (and linux) knowledge.
$ repoquery -l python3-BeautifulSoup
/usr/lib/python3.2/site-packages/beautifulsoup4-4.0.0b3-py3.2.egg-info
/usr/lib/python3.2/site-packages/bs4
/usr/lib/python3.2/site-packages/bs4/__init__.py
...