I am building a Python script for processing json data according to some criteria.
Also I have built a custom module which consists of methods for retrieving json data and generation of json file which consist of processed data.
But that module file is stored into S3 bucket and I need to import that module into my script so that I can invoke functions defined in module.
Please suggest me appropriate solution regarding importing python module from external URL
Well, you could download the file using urllib2 and then import it, if the online module is all in one file:
from urllib2 import urlopen
r = urlopen('http://urlHere/fileHere')
f = open('filenameToWrite', 'w')
f.write(r.read())
f.close()
import filenameWithout.PyInIt
Package your module into your favorite extension (tarball, wheel, etc.) using setuptools and then you will be able to install it using pip as bruno by doing something like:
pip install --no-index --trusted-host s3_ip/host --find-links http://s3.com...
Please see my answer to question # 48905127 - believed to be the best solution I found so far and comes with detailed steps and code snippets ready for you to copy and use.
Related
I am trying to use python "requests" package in aws-glue. I tried with making .egg file for that package and it won't work. Also tried the making zip of request package and added to the glue job.
How can I execute below lines of code in AWS-glue?
import requests
print ("Hello")
Is it spark or python shell job? Answer depends on it.
For python shell job:
Use similar to following line in your setup.py file:
install_requires=['requests==2.22.0']
Replace requests with any library you wish to install.
When I try to run pipenv run main.py I am met with the error ImportError: No module named parse.
I've looked around online but all I can find are people not using the correct python version, but I don't think that is the case here.
I first run pipenv --three to build the virtualenv using python 3.6.6. After that succeeds I am met with the previously stated error.
In my __init.py__ file I'm importing parse through from urllib.parse import urlparse. The threads I can find online about the subject seems to be people using the Python 2 import syntax, but that's not the case here as far as I can tell.
Any help would be greatly appreciated...
The problem is that you are only importing that one function urlparse, not the entire package urllib.parse, so you don't have access to that yet. If you need the entire package you should import it with from urllib import parse.
You can also change from urllib.parse import urlparse to from urllib.parse import urlparse as parse if you only need the method, but then you would have clashing name (parse the module and parse the function). This works fine, as you can only access the function parse anyway, but this might get confusing later when you do need the entire package.
I've tried many times but failed to download the NLTK data files through the internal downloader, but finally I figured out the way to download these files through an external downloader.
And then I moved the downloaded files(which are .zip files) into the following path:
C:\Users\Administrator\AppData\Roaming\nltk_data\corpora
and extracted the files here. It should work, but it didn't, which is frustrating. So what's the problem here and how to fix it? Is there a way to install the data files? And thank you.
The best way to download the nltk data when you programming is to use the download. Ex:
import nltk
nltk.download()
Then you can download the data that will be placed in the default folder using this interface:
OR you can set the folder where is your data manually editing the file nltk.data.path. For this use:
import nltk
nltk.data.path.append('YOUR PATH');
You will probably want to forgo manually installing and allow python to do it for you.
If you have already installed NLTK (either pip or conda depending on your Python setup an OS)from your command line or using these directions for windows:
Installation
Then it is really easy. In your python script type:
import nltk
nltk.download()
This will import the necessary functions from NLTK and then download the data associated with the NLTK.
Then you just import what you need from the data files:
from nltk.corpus import "data_name"
And it is ready to use. The reason to download from the script (or by using the command line in administrator mode) is that it stores the data in a centrally accessible location. This allows you to call it as above easily...no chasing down where it lives.
I am using an online spreadsheet app called AirTable and need to be able to access the API using Python.
There is a python interface to the API as outlined on Github:
https://github.com/bayesimpact/airtable-python
I've followed the getting started directions and when I run the code I get the following error:
AttributeError: module 'airtable' has no attribute 'Airtable'
What am I doing wrong?
Temporary fix:
from airtable import airtable
Permanent fix:
1 find your airtable installed path
2 find the ini file
3 Remove the #
4 Specify the class you need to use once imported
from airtable import airtable
class Airtable(object):
pass
May be more class you need to add.
I had this problem as well. The filename which I was writing in was named airtable.py
When I changed the filename to air.py it was able to import airtable without an issue.
First, check that if your project has a file name "airtable.py"
"You should never use the same name for a python file that is for a python library, python confuses its self, deciding where should it import, its priority is to import from the project files first and go to libraries later"
if that does not works then you may have installed the wrong library
pip uninstall airtable
python working library for airtbale is
pip install airtable-python-wrapper
I am trying to authenticate jawbone api in python. In the code there is a line:
import requests
How can I add this. I have very little knowledge on python. Just manipulating the code. Can any one please help? The library is already present in python 3.3
This is only for the users using python33 and in windows platform,,,,
download Requests packages from any site.
Copy the folder Requests from the downloaded package and place it on C://python33/LIB/ folder....
now you are able to import Requests package to your program