I have a URL which outputs JSON data when called. I have to check for a specific word in that JSON output. Example below -
r1 = session.get(authurl, headers=headers, timeout=6)
resphead = r1.headers.get('content-type')
if 'application/json' in resphead:
json_data = r1.json()
overallstatus1 = str((json_data['status']))
overallstatus2 = str((json_data['status']['code']))
Sometimes, the output will have to be called using ['status']['code'] and sometimes the code section will not be coming up in the output, i.e. just ['status']. Similarly, i'll have many other status to check which differ in the same way.
What can be done here to read the output even if the keys change.
Kindly clarify.
When you call .json() on your response object it returns a python dict. Therefore you can take advantage of the build-in function get() of the dictionaries that returns a key if it exists in the dictionary, otherwise some pre-specified value (defalt is None)
From the docs:
https://docs.python.org/2/library/stdtypes.html#dict.get
The calls at your code would then be
overallstatus1 = str(json_data.get('status'))
overallstatus2 = str(json_data.get('status', {}).get('code'))
Related
As a Python newbie, I seem to struggle with a use case that I need to implement. The use case is that I need to retrieve some data using a REST call and later on send that data (modified) to another REST API but then with some fields manipulated, for example, rename a property or delete a property.
# get the data
service1 = requests.get(http://myrestservice_1.com)
# modify the data?
# delete or rename a property?
# send the data
service2 = request.post(url=http://myrestservice_2.com, json=service1.json())
Example of the data that I retrieve using a Rest call:
{
"prop1" : "value1",
"prop2" : "value2"
{
How can I, for example, delete or rename the prop2 value from the JSON when it needs to be forwarded the data to another service?
Thanks in advance!
requests.get() returns a Response object, which among other things contains the payload you want to decode. You can either do that yourself:
data = json.loads(service1.text)
or let requests do it for you.
data = service1.json()
Once you have modified data as necessary, you need to re-encode the data structure. Again, you can do so manually
service2 = request.post(url=http://myrestservice_2.com, data=json.dumps(data))
or let requests do it for you.
service2 = request.post(url=http://myrestservice_2.com, json=data)
response = requests.get()
gets you a Response object.
From there,
response.text
gets you a string with the response of the REST api.
response.status_code
gets you the HTTP status code of the call you just did.
And,
response.json()
gets you a Python dict that you can interact with.
I'm trying to learn Python and have following problem:
I get an error while running this as it cannot see the 'name' attribute in data.
It works when I grab one by one items from JSON. However when I want to do it in a loop it fails.
I assume my error is wrong request. That it cannot read JSON correctly and see attributes.
import requests
import json
def main():
req = requests.get('http://pokeapi.co/api/v2/pokemon/')
print("HTTP Status Code: " + str(req.status_code))
print(req.headers)
json_obj = json.loads(req.content)
for i in json_obj['name']:
print(i)
if __name__ == '__main__':
main()
You want to access the name attribute of the results attribute in your json_object like this:
for pokemon in json_obj['results']:
print (pokemon['name'])
I was able to guess that you want to access the results keys because I have looked at the result of
json_obj.keys()
that is
dict_keys(['count', 'previous', 'results', 'next'])
Because all pokemons are saved in a list which is under keyword results, so you firstly need to get that list and then iterate over it.
for result in json_obj['results']:
print(result['name'])
A couple things: as soon mentioned, iterating through json_obj['name'] doesn't really make sense - use json_obj['results'] instead.
Also, you can use req.json() which is a method that comes with the requests library by default. That will turn the response into a dictionary which you can then iterate through as usual (.iteritems() or .items(), depending if you're using Python 2 or 3).
I'm a Python novice, thanks for your patience.
I retrieved a web page, using the requests module. I used Beautiful Soup to harvest a few hundred href objects (links). I used uritools to create an array of full URLs for the target pages I want to download.
I don't want everybody who reads this note to bombard the web server with requests, so I'll show a hypothetical example that is realistic for just 2 hrefs. The array looks like this:
hrefs2 = ['http://ku.edu/pls/WP040?PT001F01=910&pf7331=11',
'http://ku.edu/pls/WP040?PT001F01=910&pf7331=12']
If I were typing these into 100s of lines of code, I understand what to do in order to retrieve each page:
from lxml import html
import requests
url = 'http://ku.edu/pls/WP040/'
payload = {'PT001F01' : '910', 'pf7331' : '11')
r = requests.get(url, params = payload)
Then get the second page
payload = {'PT001F01' : '910', 'pf7331' : '12')
r = requests.get(url, params = payload)
And keep typing in payload objects. Not all of the hrefs I'm dealing with are sequential, not all of the payloads are different simply in the last integer.
I want to automate this and I don't see how to create the payloads from the hrefs2 array.
While fiddling with uritools, I find urisplit which can give me the part I need to parse into a payload:
[urisplit(x)[3] for x in hrefs2]
['PT001F01=910&pf7331=11',
'PT001F01=910&pf7331=12']
Each one of those has to be turned into a payload object and I don't understand what to do.
I'm using Python3 and I used uritools because that appears to be the standards-compliant replacement of urltools.
I fell back on shell script to get pages with wget, which does work, but it is so un-Python-ish that I'm asking here for what to do. I mean, this does work:
import subprocess
for i in hrefs2:
subprocess.call(["wget", i])
You can pass the full url to requests.get() without splitting up the parameters.
>>> requests.get('http://ku.edu/pls/WP040?PT001F01=910&pf7331=12')
<Response [200]>
If for some reason you don't want to do that, you'll need to split up the parameters some how. I'm sure there are better ways to do it, but the first thing that comes to mind is:
a = ['PT001F01=910&pf7331=11',
'PT001F01=910&pf7331=12']
# list to store all url parameters after they're converted to dicts
urldata = []
#iterate over list of params
for param in a:
data = {}
# split the string into key value pairs
for kv in param.split('&'):
# split the pairs up
b = kv.split('=')
# first part is the key, second is the value
data[b[0]] = b[1]
# After converting every kv pair in the parameter, add the result to a list.
urldata.append(data)
You could do this with less code but I wanted to be clear what was going on. I'm sure there is already a module somewhere out there that does this for you too.
I am retrieving a json file with REST API using a GET in python and want to retrieve the json file update some values of it and then update that value. My get is the following:
r1 = requests.get(url)
It returns the whole json file. Now I want to update some values of the json and put it back using PUT REST API call. My put command is the following:
requests.put(url, json=r1)
The fields I want to update are the following:
r1.json()['a']['a'] = 2 // print r1.json()['a']['a'] ->1
r1.json()['a']['b'] = 2 // print r1.json()['a']['b'] ->1
r1.json()['a']['c'] = 2 // print r1.json()['a']['c'] ->1
My question is how can I update correctly the fields of my json?
You should save it as a variable, modify it and then send it back
my_json = r1.json()
my_json['a']['a'] = 2
my_json['a']['b'] = 2
my_json['a']['c'] = 2
requests.put(url, json=my_json)
What you are doing in your code essentialy, is calling .json() which generates a dictionary from the response, you are not really modifying the response object - but really modifying the result of the returned value from the .json() call.
To improve on bluesummer's answer, response.json() is merely a shortcut for json.loads(response.content), and it creates a new dict each time it's called. IOW what this code:
r1.json()['a']['a'] = 2
is
create a dict from r1.content
update this dict
and finally discard it.
which is why you have to keep a reference to the dict (assign it to a variable) if you want to update it.
So I'm trying to learn Python here, and would appreciate any help you guys could give me. I've written a bit of code that asks one of my favorite websites for some information, and the api call returns an answer in a dictionary. In this dictionary is a list. In that list is a dictionary. This seems crazy to me, but hell, I'm a newbie.
I'm trying to assign the answers to variables, but always get various error messages depending on how I write my {},[], or (). Regardless, I can't get it to work. How do I read this return? Thanks in advance.
{
"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true
}
Edited because I put in the wrong sample code.
You need to show your code, but the de-facto way of doing this is by using the requests module, like this:
import requests
url = 'http://www.example.com/api/v1/something'
r = requests.get(url)
data = r.json() # converts the returned json into a Python dictionary
for item in data['answer']:
print(item['widgets'])
Assuming that you are not using the requests library (see Burhan's answer), you would use the json module like so:
data = '{"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true}'
import json
data = json.loads(data)
# Now you can use it as you wish
data['answer'] # and so on...
First I will mention that to access a dictionary value you need to use ["key"] and not {}. see here an Python dictionary syntax.
Here is a step by step walkthrough on how to build and access a similar data structure:
First create the main dictionary:
t1 = {"a":0, "b":1}
you can access each element by:
t1["a"] # it'll return a 0
Now lets add the internal list:
t1["a"] = ["x",7,3.14]
and access it using:
t1["a"][2] # it'll return 3.14
Now creating the internal dictionary:
t1["a"][2] = {'w1':7,'w2':8,'w3':9}
And access:
t1["a"][2]['w3'] # it'll return 9
Hope it helped you.