import requests
import json
r = requests.get("https://api.investing.com/api/search/?t=Equities&q=amd") # i get json text from this api
data = json.loads(r.text)
if data['articles'][0]['exchange'] == 'Sydney': # the error is here KeyError: 'exchange'
print('success')
else:
print('fail')
if i want to get the url '/equities/segue-resources-ltd' by checking if the 'exchange' is 'Sydney' which is stored in this part of the json text, {"id":948190,"url":"/equities/segue-resources-ltd","description":"Segue Resources Ltd","symbol":"AMD","exchange":"Sydney","flag":"AU","type":"Equities"}
If i'm understanding this correctly, the exchange identifier only appears in part of the json response. So, in order to get your result using the same data variable in your question, we can do this:
result = [val["url"] for val in data["quotes"] if val["exchange"] == "Sydney"]
We are using a list comprehension here, where the loop is only going through data["quotes"] instead of the whole json response, and for each item in that json subset, we're returning the value for key == "url" where the exchange == "Sydney". Running the line above should get you:
['/equities/segue-resources-ltd']
As expected. If you aren't comfortable with list comprehensions, the more conventional loop-version of it looks like:
result = []
for val in data["quotes"]:
if val["exchange"] == "Sydney":
result.append(val["url"])
print(result)
KeyError: 'exchange' means that the dictionary data['articles'][0] did not have a key 'exchange'.
Depending on your use case, you may want to iterate over the whole list of articles:
for article in data['articles']:
if 'exchange' in article and article['exchange'] == 'Sydney':
... # Your code here
If you only want to check the first article, then use data['articles'][0].get('exchange'). The dict.get() method will return None if the key is not present instead of throwing a KeyError.
Related
I have a .json file of all of my AWS target groups. This was created using aws elbv2 describe-target-groups. I want to extract every TargetGroupArn from this file and store it into a Python list.
With my current code, I get no output. I can confirm that the dictionary has data in it, but nothing is being appended to the list that I'm trying to create.
import json
from pprint import pprint
with open('target_groups.json') as f:
data = json.load(f)
items = data['TargetGroups']
arn_list = []
for key, val in data.items():
if key == 'TargetGroupArn':
arn_list.append(val)
print(arn_list)
Expected results would be for arn_list to print out looking like this:
[arn:aws:elb:xxxxxxx:targetgroup1, arn:aws:elb:xxxxxxx:targetgroup2, arn:aws:elb:xxxxxxx:targetgroup3]
Change your code to this:
import json
from pprint import pprint
with open('target_groups.json') as f:
data = json.load(f)
arn_list = []
if 'TargetGroups' in data:
items = data['TargetGroups']
for item in items:
if 'TargetGroupArn' in item:
arn_list.append(item['TargetGroupArn'])
print(arn_list)
else:
print('No data')
There are many ways to make this python code more concise. However, I prefer a more wordy style that easier to read.
Also note that this code checks that keys exist so that the code will not stackdump for missing data.
it would be better if you could post the file you are trying to get data from, but this part:
for key, val in data.items():
if key == 'TargetGroupArn':
arn_list.append(val)
need to be changed to:
for key, val in items.items():
if key == 'TargetGroupArn':
arn_list.append(val)
you get data from 'data' and add it to items, but you never actually used it.
give it a shot.
am trying to tell gender by first name.
I been using this code from:
[https://github.com/block8437/gender.py/blob/master/gender.py][1]
import requests, json
def getGenders(names):
url = ""
cnt = 0
if not isinstance(names,list):
names = [names,]
for name in names:
if url == "":
url = "name[0]=" + name
else:
cnt += 1
url = url + "&name[" + str(cnt) + "]=" + name
req = requests.get("https://api.genderize.io?" + url)
results = json.loads(req.text)
retrn = []
for result in results:
if result["gender"] is not None:
retrn.append((result["gender"], result["probability"], result["count"]))
else:
retrn.append((u'None',u'0.0',0.0))
return retrn
Everything was working for 2 days, I have not change anything in the code. I been passing different names in it on and off for 2 days. Suddenly I got this error:
string indices must be integers
on this line:
if result["gender"] is not None:
First, I want to know why this would suddenly happen? Second, How can I fix it?
Iterating through a dictionary iterates through the keys in a dictionary.
results = {"name":"peter","gender":"male","probability":"0.99","count":796}
[result for result in results] # ['count', 'gender', 'name', 'probability']
Iterating through a list iterates through the items of a list.
results = [{"name":"peter","gender":"male","probability":"0.99","count":796}]
[result for result in results] # [{'count': 796, 'gender': 'male', 'name': 'peter', 'probability': '0.99'}]
According to the API description at https://genderize.io/, there are two response formats: a json object for a single name lookup (which json.loads would return as a dictionary), and a list of objects for multiple lookups (which json.loads would return as a list of dictionaries). See the response for
https://api.genderize.io/?name=peter
compared to
https://api.genderize.io/?name[0]=peter
It seems like your error is the result of getting a response in the first format when you are expecting the second. Why might this have changed? Thats a question for the API you are using. It looks to me like your request should always be in the multi-name request format but I can't speak for how they are actually responding.
As for fixing this, you could check the type and wrap naked dictionaries in a list:
retrn = []
if not isinstance(results, list):
results =[results]
for result in results:
if result["gender"] is not None:
retrn.append((result["gender"], result["probability"], result["count"]))
else:
retrn.append((u'None',u'0.0',0.0))
I have just made a program to parse some data from an api. The api gives data back with a JSON format. When I try to parse it it gives me a key error
url = json.loads(r.text)["url"]
KeyError: 'url'
This is the part of the code
url = json.loads(r.text)["url"]
I am trying to get the data in the plain field. Here is the output from the API:
{"updates":[{"id":"a6aa-8bd","description":"Bug fixes and enhancemets","version":"8.1.30","type":"firmware","url":"https://con-man.company.com/api/v1/file-732e844b","updated":"2017-07-25"}]}
You cannot access url since it is inside update (list), therefore you need to Pass index and then key :
One liner:
>>> url = json.loads(r.text)['updates'][0]['url']
'https://con-man.company.com/api/v1/file-732e844b'
Explicit
>>> jobj = json.loads(r.text)
>>> url = jobj['updates'][0]['url']
'https://con-man.company.com/api/v1/file-732e844b'
try this,
url = json.loads(r.text)["updates"][0]["url"]
{
"updates": [
{
"id":"a6aa-8bd",
"description":"Bug fixes and enhancemets",
"version":"8.1.30",
"type":"firmware",
"url":"https://con-man.company.com/api/v1/file-732e844b",
"updated":"2017-07-25"
}
]
}
Try to visualize of your dict, it has only one key "update" in that key value it has another list and into that list, you has another dict
so if in your case
_dict = json.loads(r.text) # read file and load dict
_list = _dict['updates'] # read list inside dict
_dict_1 = _list[0] # read list first value and load dict
url = _dict_1['url'] # read 'url' key from dict
I used this and works now for me.
json_object = json.loads(response.content.decode("utf-8"))['list'][0]['localPercentDynamicObjectsUsed']
I'm trying to parse a JSON response I get from a web API. The problem is that the JSON can have varying levels, which translate into dictionaries of dictionaries, with an occasional list in the mix.
Example (this works):
for r in json_doc['results']:
yield r.get('lastLoginLocation',{}).get('coordinates',{}).get('lat',{})
Can I do the same thing when a list of dictionaries is in there? I'd like it to return the specified key value from the first dictionary in the list if the list is populated, or return '{}' if the list is empty.
Example (this does NOT work)
yield r.get('profile',{}).get('phones',{})[0].get('default',{})
Quite simply in `get('phones') use a list with one empty dict as default, ie:
yield r.get('profile',{}).get('phones',[{}])[0].get('default',{})
Note that this will still break with an IndexError if r["profile"]["phones"] is an empty list. You could get around using a or, ie:
yield (r.get('profile',{}).get('phones',[{}]) or [{}])[0].get('default',{})
but it's getting really messy (and builds two empty dicts and lists for no good reason), so you'd probably be better with more explicit code, cf Pankaj Singhal's answer.
Your approach is quite suboptimal, since a missing profile key in the root dictionary does not terminate the search, but continues further on unnecessarily. There obviously would not be any keys in an empty dict.
You could instead use a try/except:
def get_value(container, keys=None)
if keys is None:
raise ValueError
for r in container:
item = r
for i in keys:
try:
item = item[i]
except (IndexError, KeyError):
yield {}
break
# finished cleanly
else:
yield item
get_value(json_doc['results'], keys=['profile', 'phones', 0, 'default'])
This get_nested helper function might be what you want. I used a similar technique in some XML parsing code in the past. It removes the
implementation from obscuring what your code is actually trying to achieve.
from contextlib import suppress
def get_nested(list_or_dict, keys, default={}):
"""Get value from nested list_or_dict using keys. If the current
level is a dict, lookup the current key. If the current
level is a list, lookup current key in the first element of the
list. Return default for any errors.
"""
def get(item, key):
if hasattr(item, 'get') and key in item:
return item[key]
raise KeyError
for key in keys:
with suppress(KeyError):
list_or_dict = get(list_or_dict, key)
continue
with suppress(IndexError, KeyError):
list_or_dict = get(list_or_dict[0], key)
continue
break
else:
return list_or_dict
return default
Your code to call it would be like this:
for r in json_doc['results']:
yield get_nested(r, ('lastLoginLocation', 'coordinates', 'lat'))
yield get_nested(r, ('profile', 'phones', 'default'))
I'm using the Temboo Twitter API for Python to download tweets. I want to interpret them but am having trouble pulling out certain values. It returns each tweet in JSON. I want to take certain items out of the JSON and pass them over for further use (favorite_count in the example below). print (json.loads(array)) works fine but the following line print (data['favorite_count']) does not and returns and error list indices must be integers, not str. Giving an integer value just returns and out of range index error.
Would really appreciate a solution to extracting a certain section from the JSON list.
homeTimelineResults = homeTimelineChoreo.execute_with_results(homeTimelineInputs)
if __name__ == "__main__":
array = homeTimelineResults.get_Response()
data = json.loads(array)
print (json.loads(array))
print (data['favorite_count'])
From the error you are getting, I would guess that data is a list, not a dictionary. What you could do then is something along these lines:
import collections
homeTimelineResults = homeTimelineChoreo.execute_with_results(homeTimelineInputs)
if __name__ == "__main__":
array = homeTimelineResults.get_Response()
data = json.loads(array)
if data and isinstance(data, collections.Iterable) and not isinstance(data, (str, bytes)):
result = data.pop(0)
print(result['favorite_count'])
Basically we are checking that data is indeed a list or tuple or something you can iterate over (but not a string or a sequence of bytes) and that it is not empty. This is the meaning of the if statement after data = json.loads(array).
If that is indeed the case, we pop the first element and - assuming that it is a dictionary - access its 'favorite_count' key. Of course this assumption is pretty dangerous and one should be a bit more careful and check first :-)