How could I make the following call in Python? Pseudocode version:
jsonTwitterResponse = twitter.get(up to max of 3
tweets within 3km of longitude: 7, latitude: 5)
print jsonTwitterResponse
It looks like the geocode API is what I need. I have no idea how to actually code this up though. How would I do the above in actual code?
Here is a sample geocode request:
import urllib, json, pprint
params = urllib.urlencode(dict(q='obama', rpp=10, geocode='37.781157,-122.398720,1mi'))
u = urllib.urlopen('http://search.twitter.com/search.json?' + params)
j = json.load(u)
pprint.pprint(j)
The full Twitter REST API is described here: https://dev.twitter.com/docs/api
Also, Twitter has a location search FAQ that may be of interest.
In addition to Raymond Hettinger's answer, I'd like to mention that you can also use a query like "near:Amsterdam within:5km" if you don't want to work with actual coordinates.
Example: http://search.twitter.com/search?q=near:Amsterdam%20within:5km
I think this method might've been added more recently:
import urllib, json, pprint
params = urllib.urlencode(dict(lat=37.76893497, long=-122.42284884))
u = urllib.urlopen('https://api.twitter.com/1/geo/reverse_geocode.json?' + params)
j = json.load(u)
pprint.pprint(j)
Documentation: https://dev.twitter.com/docs/api/1/get/geo/reverse_geocode
Related
I am trying to save data in the following url as triples into triples store for future query. Here are my code:
import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
import re
url='http://gnafld.net/address/?per_page=10&page=7'
page = requests.get(url)
response = requests.get(url)
response.raise_for_status()
results = re.findall('\"Address ID: (GAACT[0-9]+)\"', response.text)
address1=results[0]
a = "http://gnafld.net/address/"
new_url = a + address1
r = requests.get(new_url).content
print(r)
After I run the code above, I got the answer like:
enter image description here
My question is how to insert the RDF data to a Fuseki Server SPARQL endpoint? I try the code like this:
import rdflib
from rdflib.plugins.stores import sparqlstore
#the following sparql endpoint is provided by the GNAF website
endpoint = 'http://gnafld.net/sparql'
store = sparqlstore.SPARQLUpdateStore(endpoint)
gs=rdflib.ConjunctiveGraph(store)
gs.open((endpoint,endpoint))
for stmt in r:
gs.add(stmt)
But it seems that it does not work. How can I fix this problem? Thanks for your help!
The answer you show in the image is in RDF triple format, it is just not pretty printed.
To store the RDF data in an RDF store you can use RDFlib. Here is an example of how to do that.
If you use Jena Fuseki server you should be able to access it from python just as you access any other SPARQL endpoint from python.
You may want to see my answer to a related SO question as well.
I've setup a code in python to search for tweets using the oauth2 and urllib2 libraries only. (I'm not using any particular twitter library)
I'm able to search for tweets based on keywords. However, I'm getting zero number of tweets when I search for this particular keyword - "Jurgen%20Mayer-Hermann". (this is challenge because my ultimate goal is to search for this keyword only.
On the other hand when I search for the same thing online (twitter interface, I'm getting enough tweets). - https://twitter.com/search?q=Jurgen%20Mayer-Hermann&src=typd
Can someone please see if we can identify the issue?
The code is as follows:
def getfeed(mystr, tweetcount):
url = "https://api.twitter.com/1.1/search/tweets.json?q=" + mystr + "&count=" + tweetcount
parameters = []
response = twitterreq(url, "GET", parameters)
res = json.load(response)
return res
search_str = "Jurgen Mayer-Hermann"
search_str = '%22'+search_str+'%22'
search = search_str.replace(" ","%20")
search = search.replace("#","%23")
tweetcount = str(50)
res = getfeed(search, tweetcount)
When I print the constructed url, I get
https://api.twitter.com/1.1/search/tweets.json?q=%22Jurgen%20Mayer-Hermann%22&count=50
I have actually never worked with the Twitter API, but it looks like the count parameter only applies to searches on timelines as a way to limit the amount of tweets per page of results. In other words, you use it with the GET statuses/home_timeline, GET statuses/mentions, and GET statuses/user_timeline endpoints.
Try without count and see what happens.
Please use urllib.urlencode to encode your query parameters, like so:
import urllib
query = urllib.urlencode({'q': '"Jurgen Mayer-Hermann"', count: 50})
This produces 'q=%22Jurgen+Mayer-Hermann%22&count=50'. Which might bring you more luck...
I wrote the following code:
from hashlib import sha256
from base64 import b64encode
import hmac
import urllib
from time import strftime, gmtime
url = 'http://ecs.amazonaws.com/onca/xml'
AWSAccessKeyId = amazon_settings.amazon_access_key_id
AssociateTag = amazon_settings.amazon_associate_tag
Keywords = urllib.quote_plus('Potter')
Operation = 'ItemSearch'
SearchIndex = 'Books'
Service = 'AWSECommerceService'
Timestamp = urllib.quote_plus(strftime("%Y-%m-%dT%H:%M:%S.000Z", gmtime()))
Version = '2011-08-01'
sign_to = 'GET\necs.amazonaws.com\n/onca/xml\nAWSAccessKeyId=%s&AssociateTag=%s&Keywords=%s&Operation=%s&SearchIndex=%s&Service=%s&Timestamp=%s&Version=%s' % (AWSAccessKeyId, AssociateTag, Keywords, Operation, SearchIndex, Service, Timestamp, Version)
Signature = urllib.quote_plus(b64encode(hmac.new(str(amazon_settings.amazon_secret_access_key), str(sign_to), sha256).digest()))
request = '%s?AWSAccessKeyId=%s&AssociateTag=%s&Keywords=%s&Operation=%s&SearchIndex=%s&Service=%s&Timestamp=%s&Version=%s&Signature=%s' % (url, AWSAccessKeyId, AssociateTag, Keywords, Operation, SearchIndex, Service, Timestamp, Version, Signature)
print request
When i use this code all fine.
But if i try add ItemPage param to sign_to variable and to request variable i get error SignatureDoesNotMatch.
Help me please.
It's actualy not answer to you question, but i recomend you take a look at excellent python wrapper for the Amazon Product Advertising API - python-amazon-product-api
It's hard to find in the documentation, but you have to make sure that your list of Operations are in alphabetical order or else you get a SignatureDoesNotMatch error.
For example, ItemPage must go between AssociateTag and Keywords to be valid.
AWSAccessKeyId
AssociateTag
ItemPage
Keywords
Operation
ResponseGroup
SearchIndex
Service
SignatureVersion
Timestamp
Version
I'm using wikitools package to parse the wikipedia. I just copy this example from documentation. But its not working. When I run this code. I get following error.
Invalid JSON,trying requesting again. Can you please help me ? thanks
from wikitools import wiki
from wikitools import api
# create a Wiki object
site = wiki.Wiki("http://my.wikisite.org/w/api.php")
# define the params for the query
params = {'action':'query', 'titles':'Papori'}
# create the request object
request = api.APIRequest(site, params)
# query the API
result = request.query()
The "http://my.wikisite.org/w/api.php" is only an example, there is no MediaWiki under that domain. Try with "http://en.wikipedia.org/w/api.php" which searches in the English Wikipedia.
Looking for a python script that would simply connect to a web page (maybe some querystring parameters).
I am going to run this script as a batch job in unix.
urllib2 will do what you want and it's pretty simple to use.
import urllib
import urllib2
params = {'param1': 'value1'}
req = urllib2.Request("http://someurl", urllib.urlencode(params))
res = urllib2.urlopen(req)
data = res.read()
It's also nice because it's easy to modify the above code to do all sorts of other things like POST requests, Basic Authentication, etc.
Try this:
aResp = urllib2.urlopen("http://google.com/");
print aResp.read();
If you need your script to actually function as a user of the site (clicking links, etc.) then you're probably looking for the python mechanize library.
Python Mechanize
A simple wget called from a shell script might suffice.
in python 2.7:
import urllib2
params = "key=val&key2=val2" #make sure that it's in GET request format
url = "http://www.example.com"
html = urllib2.urlopen(url+"?"+params).read()
print html
more info at https://docs.python.org/2.7/library/urllib2.html
in python 3.6:
from urllib.request import urlopen
params = "key=val&key2=val2" #make sure that it's in GET request format
url = "http://www.example.com"
html = urlopen(url+"?"+params).read()
print(html)
more info at https://docs.python.org/3.6/library/urllib.request.html
to encode params into GET format:
def myEncode(dictionary):
result = ""
for k in dictionary: #k is the key
result += k+"="+dictionary[k]+"&"
return result[:-1] #all but that last `&`
I'm pretty sure this should work in either python2 or python3...
What are you trying to do? If you're just trying to fetch a web page, cURL is a pre-existing (and very common) tool that does exactly that.
Basic usage is very simple:
curl www.example.com
You might want to simply use httplib from the standard library.
myConnection = httplib.HTTPConnection('http://www.example.com')
you can find the official reference here: http://docs.python.org/library/httplib.html