I have searched the web but couldn't find a suitable answer so I will try and ask here.
I am experimenting with a spark core and parsing data through JSON. I have already managed to read the data and print it with the following code:
import urllib, json
from pprint import pprint
url = "https://api.spark.io/v1/devices/mycore/result?access_token=accesstoken"
response = urllib.urlopen(url);
data = json.loads(response.read())
pprint(data)
And now I am trying to print the value I am sending with this code:
data["result"]["data1"]
I found the above in another topic but I am probably to unexperienced to properly apply it to my own code.
This is what python prints:
{u'cmd': u'VarReturn',
u'coreInfo': {u'connected': True,
u'deviceID': u'1111111111111111111',
u'last_app': u'',
u'last_handshake_at': u'2015-03-09T12:28:20.271Z',
u'last_heard': u'2015-03-09T12:56:42.780Z'},
u'name': u'result',
u'result': u'{"data1":2869}'}
the error I get says the following: TypeError: string indices must be integers
I used the example code from this topic:
https://community.spark.io/t/example-logging-and-graphing-data-from-your-spark-core-using-google/2929
I hope I am clear, can anyone enlighten me?
Try to print out data["result"]. From python print you have provided, the output should be '{"data1":2869}', which is another json object.
Try something like this:
import urllib, json
from pprint import pprint
url = "https://api.spark.io/v1/devices/mycore/result?access_token=accesstoken"
response = urllib.urlopen(url);
data = json.loads(response.read())
pprint(data)
new_data = json.loads(data["result"])
print new_data["data1"]
The contents of data["result"] is a unicode string. The string contains something that looks like a JSON doc / Python dictionary (see the single quotes around the whole construction):
>>> data["result"]
u'{"data1":2869}'
Related
I'm trying to do a little code that gets the emails (and other things in the future) from an API. But I'm getting "TypeError: list indices must be integers or slices, not str" and I don't know what to do about it. I've been looking at other questions here but I still don't get it. I might be a bit slow when it comes to this.
I've also been watching some tutorials on the tube, and done the same as them, but still getting different errors. I run Python 3.5.
Here is my code:
from urllib.request import urlopen
import json, re
# Opens the url for the API
url = 'https://jsonplaceholder.typicode.com/posts/1/comments'
r = urlopen(url)
# This should put the response from API in a Dict
result= r.read().decode('utf-8')
data = json.loads(result)
#This shuld get all the names from the the Dict
for name in data['name']: #TypeError here.
print(name)
I know that I could regex the text and get the result that I want.
Code for that:
from urllib.request import urlopen
import re
url = 'https://jsonplaceholder.typicode.com/posts/1/comments'
r = urlopen(url)
result = r.read().decode('utf-8')
f = re.findall('"email": "(\w+\S\w+)', result)
print(f)
But that seems like the wrong way to do this.
Can someone please help me understand what I'm doing wrong here?
data is a list of dicts, that's why you are getting TypeError while iterating on it.
The way to go is something like this:
for item in data: # item is {"name": "foo", "email": "foo#mail..."}
print(item['name'])
print(item['email'])
#PiAreSquared's comment is correct, just a bit more explanation here:
from urllib.request import urlopen
import json, re
# Opens the url for the API
url = 'https://jsonplaceholder.typicode.com/posts/1/comments'
r = urlopen(url)
# This should put the response from API in a Dict
result= r.read().decode('utf-8')
data = json.loads(result)
# your data is a list of elements
# and each element is a dict object, so you can loop over the data
# to get the dict element, and then access the keys and values as you wish
# see below for some example
for element in data: #TypeError here.
name = element['name']
email = element['email']
# if you want to get all names, you should do
names = [element['name'] for element in data]
# same to get all emails
emails = [email['email'] for email in data]
I am unable to parse the JSON data using python.
A webpage url is returning JSON Data
import requests
import json
BASE_URL = "https://www.codechef.com/api/ratings/all"
data = {'page': page, 'sortBy':'global_rank', 'order':'asc', 'itemsPerPage':'40' }
r = requests.get(BASE_URL, data = data)
receivedData = (r.text)
print ((receivedData))
when I printed this, I got large text and when I validated using https://jsonlint.com/ it showed VALID JSON
Later I used
import requests
import json
BASE_URL = "https://www.codechef.com/api/ratings/all"
data = {'page': page, 'sortBy':'global_rank', 'order':'asc', 'itemsPerPage':'40' }
r = requests.get(BASE_URL, data = data)
receivedData = (r.text)
print (json.loads(receivedData))
When I validated the large printed text using https://jsonlint.com/ it showed INVALID JSON
Even if I don't print and directly use the data. It is working properly. So I am sure even internally it is not loading correctly.
is python unable to parse the text to JSON properly?
in short, json.loads converts from a Json (thing, objcet, array, whatever) into a Python object - in this case, a Json Dictionary. When you print that, it will print as a itterative and therefore print with single quotes..
Effectively your code can be expanded:
some_dictionary = json.loads(a_string_which_is_a_json_object)
print(some_dictionary)
to make sure that you're printing json-safe, you would need to re-encode with json.dumps
When you use python's json.loads(text) it returns a python dictionary. When you print that dictionary out it is not in json format.
If you want a json output you should use json.dumps(json_object).
I'm having problems getting data from an HTTP response. The format unfortunately comes back with '\n' attached to all the key/value pairs. JSON says it must be a str and not "bytes".
I have tried a number of fixes so my list of includes might look weird/redundant. Any suggestions would be appreciated.
#!/usr/bin/env python3
import urllib.request
from urllib.request import urlopen
import json
import requests
url = "http://finance.google.com/finance/info?client=ig&q=NASDAQ,AAPL"
response = urlopen(url)
content = response.read()
print(content)
data = json.loads(content)
info = data[0]
print(info)
#got this far - planning to extract "id:" "22144"
When it comes to making requests in Python, I personally like to use the requests library. I find it easier to use.
import json
import requests
r = requests.get('http://finance.google.com/finance/info?client=ig&q=NASDAQ,AAPL')
json_obj = json.loads(r.text[4:])
print(json_obj[0].get('id'))
The above solution prints: 22144
The response data had a couple unnecessary characters at the head, which is why I am only loading the relevant (json) portion of the response: r.text[4:]. This is the reason why you couldn't load it as json initially.
Bytes object has method decode() which converts bytes to string. Checking the response in the browser, seems there are some extra characters at the beginning of the string that needs to be removed (a line feed character, followed by two slashes: '\n//'). To skip the first three characters from the string returned by the decode() method we add [3:] after the method call.
data = json.loads(content.decode()[3:])
print(data[0]['id'])
The output is exactly what you expect:
22144
JSON says it must be a str and not "bytes".
Your content is "bytes", and you can do this as below.
data = json.loads(content.decode())
I was following a tutorial about how to use JSON objects (link: https://www.youtube.com/watch?v=Y5dU2aGHTZg). When they ran the code, they got no errors, but I did. Is it something to do with different Python versions or something?
from urllib.request import urlopen
import json
def printResults(data):
theJSON = json.loads(data)
print (theJSON)
def main():
urlData ="http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_day.geojson"
webUrl = urlopen(urlData)
print(webUrl.getcode())
if (webUrl.getcode()==200):
data = webUrl.read()
printResults(data)
else:
print ("You failed")
main()
The HTTPResponse object returned from urlopen reads bytes data (raw binary data), not str data (textual data), while the json module works with str. You need to know (or inspect the headers to determine) the encoding used for the data received, and decode it appropriately before using json.loads.
Assuming it's UTF-8 (most websites are), you can just change:
data = webUrl.read()
to:
data = webUrl.read().decode('utf-8')
and it should fix your problem.
I think they were using a different version of the urllib
Try with urllib3 and do the import like this:
from urllib import urlopen
Hope this is the fix to your problem
I'm trying to get Twitter API search results for a given hashtag using Python, but I'm having trouble with this "No JSON object could be decoded" error. I had to add the extra % towards the end of the URL to prevent a string formatting error. Could this JSON error be related to the extra %, or is it caused by something else? Any suggestions would be much appreciated.
A snippet:
import simplejson
import urllib2
def search_twitter(quoted_search_term):
url = "http://search.twitter.com/search.json?callback=twitterSearch&q=%%23%s" % quoted_search_term
f = urllib2.urlopen(url)
json = simplejson.load(f)
return json
There were a couple problems with your initial code. First you never read in the content from twitter, just opened the url. Second in the url you set a callback (twitterSearch). What a call back does is wrap the returned json in a function call so in this case it would have been twitterSearch(). This is useful if you want a special function to handle the returned results.
import simplejson
import urllib2
def search_twitter(quoted_search_term):
url = "http://search.twitter.com/search.json?&q=%%23%s" % quoted_search_term
f = urllib2.urlopen(url)
content = f.read()
json = simplejson.loads(content)
return json