I'm trying to print out the data from the sig API but it is giving me an error although the url is correct.
import requests
import json
from json import loads
import pandas as pd
import matplotlib as plt
requests.get("https://api.meetup.com/2/groups?zip=eh1+1af&offset=0&city=Edinburgh&format=json&lon=-3.19000005722&category_id=34&photo-host=public&page=500&radius=25.0&fields=&lat=55.9500007629&order=id&desc=false&sig_id=243750775&sig=9072b77fb34f5b84a392da2505fd946c58e94fe5")
The error is here, apparently ("Invalid syntax");
print json.load(requests.get("https://api.meetup.com/2/groups?zip=eh1+1af&offset=0&city=Edinburgh&format=json&lon=-3.19000005722&category_id=34&photo-host=public&page=500&radius=25.0&fields=&lat=55.9500007629&order=id&desc=false&sig_id=243750775&sig=9072b77fb34f5b84a392da2505fd946c58e94fe5"))
Thank you
Since you're using requests, you should be aware that the Response object has a json method you can call to retrieve JSON from the HTTP response.
r = requests.get(url).json()
type(r)
dict
You can load the JSON response r into a dataframe, if you have to.
df = pd.io.json.json_normalize(r['results'])
Related
I'm trying to get the url of the minecraft skin through the api using python programming but I can't get the url, let's see if someone could
This is the code I'm currently using...
import json
import requests
import base64
response = requests.get(f"https://sessionserver.mojang.com/session/minecraft/profile/11f1cc006cc84499a174bc9b7fa1982a")
id = response.json()["properties"][0]["value"]
####
msg = f"{id}"
msg_bytes = msg.encode('ascii')
base64_bytes = base64.b64decode(msg_bytes)
base64_msg = base64_bytes.decode('ascii')
print(base64_msg)
Thank you very much in advance!
You will have to convert the string to json first.
import requests
import base64
import json
response = requests.get(
f"https://sessionserver.mojang.com/session/minecraft/profile/11f1cc006cc84499a174bc9b7fa1982a"
)
msg = response.json()["properties"][0]["value"]
base64_bytes = base64.b64decode(msg)
print(json.loads(base64_bytes)["textures"]["SKIN"]["url"])
You should always check that your HTTP request succeeds.
This is all you need:
from requests import get as GET
from base64 import b64decode as DECODE
from json import loads as LOADS
URL = 'https://sessionserver.mojang.com/session/minecraft/profile/11f1cc006cc84499a174bc9b7fa1982a'
(response := GET(URL)).raise_for_status()
data = response.json()['properties'][0]['value']
sd = LOADS(DECODE(data))
print(sd['textures']['SKIN']['url'])
Output:
http://textures.minecraft.net/texture/516ca747cee2cf895c02e0b4da4f1fe23495a140326538f960debaaa6fd67045
Note:
This code assumes that the JSON structures are as expected and that the keys always exist
I am following a tutorial an am stuck at parsing the output of requests.get()
My goal is to connect to the API below to pull historical crypto-currency prices and put them into a pandas dataframe for further analysis.
[API: https://www.cryptocompare.com/api/#-api-data-histoday-]
Here's what I have.
import requests
response = requests.get("https://min-api.cryptocompare.com/data/histodayfsym=ETC&tsym=USD&limit=10&aggregate=3&e=CCCAGG")
print(response.text)
Now I want to output into a dataframe...
pd.DataFrame.from_dict(response)
But I get...
PandasError: DataFrame constructor not properly called!
You can use the json package to convert to dict:
import requests
from json import loads
import pandas as pd
response = requests.get("https://min-api.cryptocompare.com/data/histodayfsym=ETC&tsym=USD&limit=10&aggregate=3&e=CCCAGG")
dic = loads(response.text)
print(type(dic))
pd.DataFrame.from_dict(dic)
However as jonrsharpe noted, a much more simple way would be:
import requests
import pandas as pd
response = requests.get("https://min-api.cryptocompare.com/data/histodayfsym=ETC&tsym=USD&limit=10&aggregate=3&e=CCCAGG")
print(type(response.json()))
pd.DataFrame.from_dict(response.json())
I am trying to GET a URL using Python and the response is JSON. However, when I run
import urllib2
response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
html=response.read()
print html
The html is of type str and I am expecting a JSON. Is there any way I can capture the response as JSON or a python dictionary instead of a str.
If the URL is returning valid JSON-encoded data, use the json library to decode that:
import urllib2
import json
response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
data = json.load(response)
print data
import json
import urllib
url = 'http://example.com/file.json'
r = urllib.request.urlopen(url)
data = json.loads(r.read().decode(r.info().get_param('charset') or 'utf-8'))
print(data)
urllib, for Python 3.4
HTTPMessage, returned by r.info()
"""
Return JSON to webpage
Adding to wonderful answer by #Sanal
For Django 3.4
Adding a working url that returns a json (Source: http://www.jsontest.com/#echo)
"""
import json
import urllib
url = 'http://echo.jsontest.com/insert-key-here/insert-value-here/key/value'
respons = urllib.request.urlopen(url)
data = json.loads(respons.read().decode(respons.info().get_param('charset') or 'utf-8'))
return HttpResponse(json.dumps(data), content_type="application/json")
Be careful about the validation and etc, but the straight solution is this:
import json
the_dict = json.load(response)
resource_url = 'http://localhost:8080/service/'
response = json.loads(urllib2.urlopen(resource_url).read())
Python 3 standard library one-liner:
load(urlopen(url))
# imports (place these above the code before running it)
from json import load
from urllib.request import urlopen
url = 'https://jsonplaceholder.typicode.com/todos/1'
you can also get json by using requests as below:
import requests
r = requests.get('http://yoursite.com/your-json-pfile.json')
json_response = r.json()
Though I guess it has already answered I would like to add my little bit in this
import json
import urllib2
class Website(object):
def __init__(self,name):
self.name = name
def dump(self):
self.data= urllib2.urlopen(self.name)
return self.data
def convJSON(self):
data= json.load(self.dump())
print data
domain = Website("https://example.com")
domain.convJSON()
Note : object passed to json.load() should support .read() , therefore urllib2.urlopen(self.name).read() would not work .
Doamin passed should be provided with protocol in this case http
This is another simpler solution to your question
pd.read_json(data)
where data is the str output from the following code
response = urlopen("https://data.nasa.gov/resource/y77d-th95.json")
json_data = response.read().decode('utf-8', 'replace')
None of the provided examples on here worked for me. They were either for Python 2 (uurllib2) or those for Python 3 return the error "ImportError: No module named request". I google the error message and it apparently requires me to install a the module - which is obviously unacceptable for such a simple task.
This code worked for me:
import json,urllib
data = urllib.urlopen("https://api.github.com/users?since=0").read()
d = json.loads(data)
print (d)
I was following a tutorial about how to use JSON objects (link: https://www.youtube.com/watch?v=Y5dU2aGHTZg). When they ran the code, they got no errors, but I did. Is it something to do with different Python versions or something?
from urllib.request import urlopen
import json
def printResults(data):
theJSON = json.loads(data)
print (theJSON)
def main():
urlData ="http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_day.geojson"
webUrl = urlopen(urlData)
print(webUrl.getcode())
if (webUrl.getcode()==200):
data = webUrl.read()
printResults(data)
else:
print ("You failed")
main()
The HTTPResponse object returned from urlopen reads bytes data (raw binary data), not str data (textual data), while the json module works with str. You need to know (or inspect the headers to determine) the encoding used for the data received, and decode it appropriately before using json.loads.
Assuming it's UTF-8 (most websites are), you can just change:
data = webUrl.read()
to:
data = webUrl.read().decode('utf-8')
and it should fix your problem.
I think they were using a different version of the urllib
Try with urllib3 and do the import like this:
from urllib import urlopen
Hope this is the fix to your problem
I am trying to GET a URL using Python and the response is JSON. However, when I run
import urllib2
response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
html=response.read()
print html
The html is of type str and I am expecting a JSON. Is there any way I can capture the response as JSON or a python dictionary instead of a str.
If the URL is returning valid JSON-encoded data, use the json library to decode that:
import urllib2
import json
response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
data = json.load(response)
print data
import json
import urllib
url = 'http://example.com/file.json'
r = urllib.request.urlopen(url)
data = json.loads(r.read().decode(r.info().get_param('charset') or 'utf-8'))
print(data)
urllib, for Python 3.4
HTTPMessage, returned by r.info()
"""
Return JSON to webpage
Adding to wonderful answer by #Sanal
For Django 3.4
Adding a working url that returns a json (Source: http://www.jsontest.com/#echo)
"""
import json
import urllib
url = 'http://echo.jsontest.com/insert-key-here/insert-value-here/key/value'
respons = urllib.request.urlopen(url)
data = json.loads(respons.read().decode(respons.info().get_param('charset') or 'utf-8'))
return HttpResponse(json.dumps(data), content_type="application/json")
Be careful about the validation and etc, but the straight solution is this:
import json
the_dict = json.load(response)
resource_url = 'http://localhost:8080/service/'
response = json.loads(urllib2.urlopen(resource_url).read())
Python 3 standard library one-liner:
load(urlopen(url))
# imports (place these above the code before running it)
from json import load
from urllib.request import urlopen
url = 'https://jsonplaceholder.typicode.com/todos/1'
you can also get json by using requests as below:
import requests
r = requests.get('http://yoursite.com/your-json-pfile.json')
json_response = r.json()
Though I guess it has already answered I would like to add my little bit in this
import json
import urllib2
class Website(object):
def __init__(self,name):
self.name = name
def dump(self):
self.data= urllib2.urlopen(self.name)
return self.data
def convJSON(self):
data= json.load(self.dump())
print data
domain = Website("https://example.com")
domain.convJSON()
Note : object passed to json.load() should support .read() , therefore urllib2.urlopen(self.name).read() would not work .
Doamin passed should be provided with protocol in this case http
This is another simpler solution to your question
pd.read_json(data)
where data is the str output from the following code
response = urlopen("https://data.nasa.gov/resource/y77d-th95.json")
json_data = response.read().decode('utf-8', 'replace')
None of the provided examples on here worked for me. They were either for Python 2 (uurllib2) or those for Python 3 return the error "ImportError: No module named request". I google the error message and it apparently requires me to install a the module - which is obviously unacceptable for such a simple task.
This code worked for me:
import json,urllib
data = urllib.urlopen("https://api.github.com/users?since=0").read()
d = json.loads(data)
print (d)