I'm using the Temboo Twitter API for Python to download tweets. I want to interpret them but am having trouble pulling out certain values. It returns each tweet in JSON. I want to take certain items out of the JSON and pass them over for further use (favorite_count in the example below). print (json.loads(array)) works fine but the following line print (data['favorite_count']) does not and returns and error list indices must be integers, not str. Giving an integer value just returns and out of range index error.
Would really appreciate a solution to extracting a certain section from the JSON list.
homeTimelineResults = homeTimelineChoreo.execute_with_results(homeTimelineInputs)
if __name__ == "__main__":
array = homeTimelineResults.get_Response()
data = json.loads(array)
print (json.loads(array))
print (data['favorite_count'])
From the error you are getting, I would guess that data is a list, not a dictionary. What you could do then is something along these lines:
import collections
homeTimelineResults = homeTimelineChoreo.execute_with_results(homeTimelineInputs)
if __name__ == "__main__":
array = homeTimelineResults.get_Response()
data = json.loads(array)
if data and isinstance(data, collections.Iterable) and not isinstance(data, (str, bytes)):
result = data.pop(0)
print(result['favorite_count'])
Basically we are checking that data is indeed a list or tuple or something you can iterate over (but not a string or a sequence of bytes) and that it is not empty. This is the meaning of the if statement after data = json.loads(array).
If that is indeed the case, we pop the first element and - assuming that it is a dictionary - access its 'favorite_count' key. Of course this assumption is pretty dangerous and one should be a bit more careful and check first :-)
Related
I am reading data from nested json with this code:
data = json.loads(json_file.json)
for nodesUni in data["data"]["queryUnits"]['nodes']:
try:
tm = (nodesUni['sql']['busData'][0]['engine']['engType'])
except:
tm = ''
try:
to = (nodesUni['sql']['carData'][0]['engineData']['producer']['engName'])
except:
to = ''
json_output_for_one_GU_owner = {
"EngineType": tm,
"EngineName": to,
}
I am having an issue with None type error (eg. this one doesn't exists at all nodesUni['sql']['busData'][0]['engine']['engType'] cause there are no data, so I am using try/except. But my code is more complex and having a try/except for every value is crazy. Is there any other option how to deal with this?
Error: "TypeError: 'NoneType' object is not subscriptable"
This is non-trivial as your requirement is to traverse the dictionaries without errors, and get an empty string value in the end, all that in a very simple expression like cascading the [] operators.
First method
My approach is to add a hook when loading the json file, so it creates default dictionaries in an infinite way
import collections,json
def superdefaultdict():
return collections.defaultdict(superdefaultdict)
def hook(s):
c = superdefaultdict()
c.update(s)
return(c)
data = json.loads('{"foo":"bar"}',object_hook=hook)
print(data["x"][0]["zzz"]) # doesn't exist
print(data["foo"]) # exists
prints:
defaultdict(<function superdefaultdict at 0x000001ECEFA47160>, {})
bar
when accessing some combination of keys that don't exist (at any level), superdefaultdict recursively creates a defaultdict of itself (this is a nice pattern, you can read more about it in Is there a standard class for an infinitely nested defaultdict?), allowing any number of non-existing key levels.
Now the only drawback is that it returns a defaultdict(<function superdefaultdict at 0x000001ECEFA47160>, {}) which is ugly. So
print(data["x"][0]["zzz"] or "")
prints empty string if the dictionary is empty. That should suffice for your purpose.
Use like that in your context:
def superdefaultdict():
return collections.defaultdict(superdefaultdict)
def hook(s):
c = superdefaultdict()
c.update(s)
return(c)
data = json.loads(json_file.json,object_hook=hook)
for nodesUni in data["data"]["queryUnits"]['nodes']:
tm = nodesUni['sql']['busData'][0]['engine']['engType'] or ""
to = nodesUni['sql']['carData'][0]['engineData']['producer']['engName'] or ""
Drawbacks:
It creates a lot of empty dictionaries in your data object. Shouldn't be a problem (except if you're very low in memory) as the object isn't dumped to a file afterwards (where the non-existent values would appear)
If a value already exists, trying to access it as a dictionary crashes the program
Also if some value is 0 or an empty list, the or operator will pick "". This can be workarounded with another wrapper that tests if the object is an empty superdefaultdict instead. Less elegant but doable.
Second method
Convert the access of your successive dictionaries as a string (for instance just double quote your expression like "['sql']['busData'][0]['engine']['engType']", parse it, and loop on the keys to get the data. If there's an exception, stop and return an empty string.
import json,re,operator
def get(key,data):
key_parts = [x.strip("'") if x.startswith("'") else int(x) for x in re.findall(r"\[([^\]]*)\]",key)]
try:
for k in key_parts:
data = data[k]
return data
except (KeyError,IndexError,TypeError):
return ""
testing with some simple data:
data = json.loads('{"foo":"bar","hello":{"a":12}}')
print(get("['sql']['busData'][0]['engine']['engType']",data))
print(get("['hello']['a']",data))
print(get("['hello']['a']['e']",data))
we get, empty string (some keys are missing), 12 (the path is valid), empty string (we tried to traverse a non-dict existing value).
The syntax could be simplified (ex: "sql"."busData".O."engine"."engType") but would still have to retain a way to differentiate keys (strings) from indices (integers)
The second approach is probably the most flexible one.
import requests
import json
r = requests.get("https://api.investing.com/api/search/?t=Equities&q=amd") # i get json text from this api
data = json.loads(r.text)
if data['articles'][0]['exchange'] == 'Sydney': # the error is here KeyError: 'exchange'
print('success')
else:
print('fail')
if i want to get the url '/equities/segue-resources-ltd' by checking if the 'exchange' is 'Sydney' which is stored in this part of the json text, {"id":948190,"url":"/equities/segue-resources-ltd","description":"Segue Resources Ltd","symbol":"AMD","exchange":"Sydney","flag":"AU","type":"Equities"}
If i'm understanding this correctly, the exchange identifier only appears in part of the json response. So, in order to get your result using the same data variable in your question, we can do this:
result = [val["url"] for val in data["quotes"] if val["exchange"] == "Sydney"]
We are using a list comprehension here, where the loop is only going through data["quotes"] instead of the whole json response, and for each item in that json subset, we're returning the value for key == "url" where the exchange == "Sydney". Running the line above should get you:
['/equities/segue-resources-ltd']
As expected. If you aren't comfortable with list comprehensions, the more conventional loop-version of it looks like:
result = []
for val in data["quotes"]:
if val["exchange"] == "Sydney":
result.append(val["url"])
print(result)
KeyError: 'exchange' means that the dictionary data['articles'][0] did not have a key 'exchange'.
Depending on your use case, you may want to iterate over the whole list of articles:
for article in data['articles']:
if 'exchange' in article and article['exchange'] == 'Sydney':
... # Your code here
If you only want to check the first article, then use data['articles'][0].get('exchange'). The dict.get() method will return None if the key is not present instead of throwing a KeyError.
I would like to create an Alexa skill using Python to use data uploaded by sensors to Thingspeak. The cases where I only use one specific value is quite easy, the response from Thingspeak is the value only. When I want to use several values, in my case to sum up the athmospheric pressure to determine tendencies, teh response is a json object like this:
{"channel":{"id":293367,"name":"Weather Station","description":"My first attempt to build a weather station based on an ESP8266 and some common sensors.","latitude":"51.473509","longitude":"7.355569","field1":"humidity","field2":"pressure","field3":"lux","field4":"rssi","field5":"temp","field6":"uv","field7":"voltage","field8":"radiation","created_at":"2017-06-25T07:35:37Z","updated_at":"2018-08-04T12:11:22Z","elevation":"121","last_entry_id":1812},"feeds":
[{"created_at":"2018-10-21T18:11:45Z","entry_id":1713,"field2":"1025.62"},
{"created_at":"2018-10-21T18:12:05Z","entry_id":1714,"field2":"1025.58"},
{"created_at":"2018-10-21T18:12:25Z","entry_id":1715,"field2":"1025.56"},
{"created_at":"2018-10-21T18:12:45Z","entry_id":1716,"field2":"1025.65"},
{"created_at":"2018-10-21T18:13:05Z","entry_id":1717,"field2":"1025.58"},
{"created_at":"2018-10-21T18:13:25Z","entry_id":1718,"field2":"1025.63"}]
I now started with
f = urllib.urlopen(link) # Get your data
json_object = json.load(f)
for entry in json_object[0]
print entry["field2"]
The json object is a bit recursive, it is a list containing a list with an element with an array as the value.
Now I am not quite sure how to iterate over the values of the key "field2" in the array. I am quite new to Python and also json. Perhaps anyone can help me out?
Thanks in advance!
This has nothing to do with json - once the json string parsed by json.load(), what you get is a plain python object (usually a dict, sometimes a list, rarely - but this would be legal - a string, int, float, boolean or None).
it is a list containing a list with an element with an array as the value.
Actually it's a dict with two keys "channel" and "feeds". The first one has another dict for value, and the second a list of dicts. How to use dicts and lists is extensively documented FWIW
https://docs.python.org/3/tutorial/datastructures.html#dictionaries
https://docs.python.org/3/library/stdtypes.html#mapping-types-dict
https://docs.python.org/3/tutorial/introduction.html#lists
https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range
Here the values you're looking for are stored under the "field2" keys of the dicts in the "feeds" key, so what you want is:
# get the list stored under the "feeds" key
feeds = json_object["feeds"]
# iterate over the list:
for feed in feeds:
# get the value for the "field2" key
print feed["field2"]
You have a dictionary. Use key to access the value
Ex:
json_object = {"channel":{"id":293367,"name":"Weather Station","description":"My first attempt to build a weather station based on an ESP8266 and some common sensors.","latitude":"51.473509","longitude":"7.355569","field1":"humidity","field2":"pressure","field3":"lux","field4":"rssi","field5":"temp","field6":"uv","field7":"voltage","field8":"radiation","created_at":"2017-06-25T07:35:37Z","updated_at":"2018-08-04T12:11:22Z","elevation":"121","last_entry_id":1812},"feeds":
[{"created_at":"2018-10-21T18:11:45Z","entry_id":1713,"field2":"1025.62"},
{"created_at":"2018-10-21T18:12:05Z","entry_id":1714,"field2":"1025.58"},
{"created_at":"2018-10-21T18:12:25Z","entry_id":1715,"field2":"1025.56"},
{"created_at":"2018-10-21T18:12:45Z","entry_id":1716,"field2":"1025.65"},
{"created_at":"2018-10-21T18:13:05Z","entry_id":1717,"field2":"1025.58"},
{"created_at":"2018-10-21T18:13:25Z","entry_id":1718,"field2":"1025.63"}]}
for entry in json_object["feeds"]:
print entry["field2"]
Output:
1025.62
1025.58
1025.56
1025.65
1025.58
1025.63
I just figured it out, it was just like expected.
You have to get the entries array from the dict and than iterate over the list of items and print the value to the key field2.
# Get entries from the response
entries = json_object["feeds"]
# Iterate through each measurement and print value
for entry in entries:
print entry['field2']
I tried to use the openweathermap.org rest API inside python. When I tried to assign a key from the dictionary I created with the JSON data this error occurred.
-list indices must be integers or slices, not str
I'm new to python and I couldn't find a solution to this matter.
The code snip I wrote:
import requests
from pprint import pprint
lokka = str(input("What is the location you need information of?"))
#takes the location as "lokka"
hellload = requests.get("http://api.openweathermap.org/data/2.5/weather?q="+ lokka +"&appid=xxxxxxxxxxxxxxxxx&units=metric")
#the rest api's load will be taken to the account of hellload
jputha = hellload.json()
#json data will be converted to a dictionary
#print (jputha)
#---------------------------------------------------------
#from now onward I'll be kickin the hell out the jsons
long = str(jputha["coord"]["lon"])
lat = str(jputha["coord"]["lat"])
wthr = str(jputha["weather"]["main"])
temp = str(jputha["main"]["temp"])
winspd = str(jputha["wind"]["speed"])
print(long)
print(lat)
print(wthr)
print(temp)
print(winspd)
According to OpenWeatherMap's documentation, the JSON response from the API looks like this:
{"coord":
{"lon":145.77,"lat":-16.92},
"weather":[{"id":803,"main":"Clouds","description":"broken clouds","icon":"04n"}],
"base":"cmc stations",
"main":{"temp":293.25,"pressure":1019,"humidity":83,"temp_min":289.82,"temp_max":295.37},
"wind":{"speed":5.1,"deg":150},
"clouds":{"all":75},
"rain":{"3h":3},
"dt":1435658272,
"sys":{"type":1,"id":8166,"message":0.0166,"country":"AU","sunrise":1435610796,"sunset":1435650870},
"id":2172797,
"name":"Cairns",
"cod":200}
where the weather key contains a list of dicts rather than a dict, so if you simply want the first weather data from the list, you should use [0] to obtain the value of the first index instead:
wthr = str(jputha["weather"][0]["main"])
I have a tuple like: t= ({'count': 5L},)
Here i don't want to use for loop but want to get value as 5.Then how can i do it?
I tried with coverting to string then using JSON.
import json
s = str(t)
d = json.loads(s)
I got error:ValueError: No JSON object could be decoded
And winded up with no result.
I want to get the value of count as integer 5 & store in a variable.
Anyone having any idea?
No need to use Json since it is already your tuple is already a Python data structure.
If you know the index of the item in the tuple, and you know the keyname you can access it directly using:
t = ({'count': 5L},)
value = int(t[0]['count'])