I want to use the Python 3 module urllib to access an Elasticsearch database at localhost:9200. My script gets a valid request (generated by Kibana) piped to STDIN in JSON format.
Here is what I did:
import json
import sys
import urllib.parse
import urllib.request
er = json.load(sys.stdin)
data = urllib.parse.urlencode(er)
data = data.encode('ascii')
uri = urllib.request.Request('http://localhost:9200/_search', data)
with urllib.request.urlopen(uri) as repsonse:
response.read()
(I understand that my repsonse.read() doesn't make much sense by itself but I just wanted to keep it simple.)
When I execute the script, I get an
HTTP Error 400: Bad request
I am very sure that the JSON data I'm piping to the script is correct, since I had it printed and fed it via curl to Elasticsearch, and got back the documents I expected to get back.
Any ideas where I went wrong? Am I using urllib correctly? Do I maybe mess up the JSON data in the urlencode line? Am I querying Elasticsearch correctly?
Thanks for your help.
With requests you can do one of two things
1) Either you create the string representation of the json object yourself and send it off like so:
payload = {'param': 'value'}
response = requests.post(url, data=json.dumps(payload))
2) Or you have requests do it for you like so:
payload = {'param': 'value'}
response = requests.post(url, json = payload)
So depending on what actually comes out of the sys.stdin call (probably - as Kibana would be sending that if the target was ElasticSearch - a string representation of a json object == equivalent of doing json.dumps on a dictionary), but you might have to adjust a bit depending on the output of sys.stdin.
My guess is that your code could work by just doing so:
import sys
import requests
payload = sys.stdin
response = requests.post('http://localhost:9200/_search', data=payload)
And if you then want to do some work with it in Python, requests has a built in support for this too. You just call this:
json_response = response.json()
Hope this helps you on the right track. For further reading om json.dumps/loads - this answer has some good stuff on it.
For anyone who doesn't want to use requests (for example if you're using IronPython where its not supported):
import urllib2
import json
req = urllib2.Request(url, json.dumps(data), headers={'Content-Type': 'application/json'})
response = urllib2.urlopen(req)
Where 'url' can be something like this (example below is search in index):
http://<elasticsearch-ip>:9200/<index-name>/_search/
Related
When I call the socialcast api from python 2.7 using requests, I get a response "reason" but I don't see that text in my actual API. Here's my code:
import requests
parameters = {"username" = "myUsername", "password" = "myPassword"}
response = requests.get("https://hub.sas.com/api/groups/808/messages.json", parameters)
response.json()
The beginning of the JSON that I'm passing through is this:
{"messages":[{"id":126433,"user":{"id":4468,"name":
So I would expect something else to come back, but what it returns is:
{u'reason': u''}
Is this an error or is there something I'm not understanding?
I solved my problem by using this code:
import requests
from requests.auth import HTTPBasicAuth
r = requests.get('https://hub.sas.com/api/groups/808/messages.json', auth=HTTPBasicAuth('username', 'password'))
data = r.json()
for message in data['messages']:
print(message['user']['name'])
I'm not sure that the from requests.auth import HTTPBasicAuth or the data = r.json() were necessary but it ended up working for me so I left them in there.
I have a this example program, but currently, it doesn't show the post data.
import requests
r = requests.post('https://requestb.in/12p8nqo1',data={'key':'value'})
print(r.text)
. >>> 'ok'
Why doesn't print " key:value "?
Thanks
To access the request data you must deal with the underlying PreparedRequest object, like so:
import requests
r = requests.post('https://requestb.in/12p8nqo1',data={'key':'value'})
print(r.request.body)
I would like to do a HTTP DELETE with python requests module that follows the API below;
https://thingspeak.com/docs/channels#create
DELETE https://api.thingspeak.com/channels/4/feeds
api_key=XXXXXXXXXXXXXXXX
I am using python v2.7 and requests module. My python code looks like this;
def clear(channel_id):
data = {}
data['api_key'] = 'DUCYS8xufsV613VX'
URL_delete = "http://api.thingspeak.com/channels/" + str(channel_id) + "/feeds"
r = requests.delete(URL_delete, data)
The code does not work because requests.delete() can only accept one parameter. How should the correct code look like?
You want
import json
mydata = {}
mydata['api_key'] = "Jsa9i23jka"
r = requests.delete(URL_delete, data=json.dumps(mydata))
You have to use the named input, 'data', and I'm guessing that you actually want JSON dumped, so you have to convert your dictionary, 'mydata' to a json string. You can use json.dumps() for that.
I don't know the API you are using, but by the sound of it you actually want to pass URL parameter, not data, for that you need:
r = requests.delete(URL_delete, params=mydata)
No need to convert mydata dict to a json string.
You can send the data params as #Eugene suggested, but conventionally delete requests only contains url and nothing else. The reason is that a RESTful url should uniquely identify the resource, thereby eliminating the need to provide additional parameters for deletion. On the other hand, if your 'APIKEY' has something to do with authentication, then it should be part of headers instead of request data, something like this.
headers = {'APIKEY': 'xxx'}
response = requests.delete(url, data=json.dumps(payload), headers=headers)
i was trying to use the shorte.st api to automatically create my short links using my python progam, but i really don't know how to use the Apis!
In the dedicated page there is only this code here:
curl H "public-api-token: ---" -X -d "urlToShorten=google.com" PUT http://api.shorte.st/v1/data/url {"status":"ok","shortenedUrl":"http:\/\/sh.st\/XXXX"}
In the public-api-token i have to insert my private token obviously, but since curl is for c (i think) how can i use them with python?
Thanks so much
I prefer to use python lib called requests (http://docs.python-requests.org/en/latest/) for http requests. All you have to do is to send an url that you'd like shorten as data dict and your public api token in headers under the key of "public-api-token". You can find your api token on https://shorte.st/tools/api page. Response content comes as a json encoded string, so you need to decode it to obtain dict object.
import requests
response = requests.put("https://api.shorte.st/v1/data/url", {"urlToShorten":"google.com"}, headers={"public-api-token": "your_api_token"})
print response.content
>>> {"status":"ok","shortenedUrl":"http:\\/\\/sh.st\\/ryHyU"}
import json
decoded_response = json.loads(response.content)
print decoded_response
>>>{u'status': u'ok', u'shortenedUrl': u'http://sh.st/ryHyU'}
And to print out just the created URL use...
import requests
import json
response = requests.put("https://api.shorte.st/v1/data/url", {"urlToShorten":"google.com"}, headers={"public-api-token": "85d3636f48c112de6e413865afc177b5"})
decoded_response = json.loads(response.content)
print(decoded_response['shortenedUrl'])
I'm new to web services and am trying to send the following JSON based request using a python script:
http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx&json={power:290.4,temperature:19.4}
If I paste the above into a browser, it works as expected. However, I am struggling to send the request from Python. The following is what I am trying:
import json
import urllib2
data = {'temperature':'24.3'}
data_json = json.dumps(data)
host = "http://myserver/emoncms2/api/post"
req = urllib2.Request(host, 'GET', data_json, {'content-type': 'application/json'})
response_stream = urllib2.urlopen(req)
json_response = response_stream.read()
How do I add the apikey data into the request?
Thank you!
Instead of using urllib2, you can use requests. This new python lib is really well written and it's easier and more intuitive to use.
To send your json data you can use something like the following code:
import json
import requests
data = {'temperature':'24.3'}
data_json = json.dumps(data)
payload = {'json_payload': data_json, 'apikey': 'YOUR_API_KEY_HERE'}
r = requests.get('http://myserver/emoncms2/api/post', data=payload)
You can then inspect r to obtain an http status code, content, etc
Even though this doesnt exactly answer OPs question, it should be mentioned here that requests module has a json option that can be used like this:
import requests
requests.post(
'http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx',
json={"temperature": "24.3"}
)
which would be equivalent to the curl:
curl 'http://myserver/emoncms2/api/post?apikey=xxxxxxxxxxxxx' \
-H 'Content-Type: application/json' \
--data-binary '{"temperature":"24.3"}'
Maybe the problem is that json.dumps puts " and in the json you put in the url there are no "s.
For example:
data = {'temperature':'24.3'}
print json.dumps(data)
prints:
{"temperature": "24.3"}
and not:
{temperature: 24.3}
like you put in your url.
One way of solving this (which is trouble prone) is to do:
json.dumps(data).replace('"', '')