I have the following code:
import requests
from bs4 import BeautifulSoup
import urllib, json
url = "https://exec85.pythonanywhere.com/static/data_xbox.JSON"
response = urllib.request.urlopen(url)
json_data_xbox = json.loads(response.read())
name_var = "Salah"
rating_var = "96"
if name in json_data_xbox["player"].__str__(): # here I don't know what to write to check if name + rating are matching
print("matching")
else:
print("not matching")
This is the corresponding JSON file:
{"player": [{"name": "Salah", "rating": "96", "rarity": "TOTS", "prices": 5380000}, {"name": "Salah", "rating": "93", "rarity": "FOF PTG", "prices": 956000}]
As you see, I have two entries with the same name but different rating and prices.
I would like to check if two variables can be found within one object in my json dictionary to match the right one.
So in this example I want to check if name_varand rating_var are matching to then get the correct "prices" value.
What would I need to write to get that check done?
You would probably want to loop over each dictionary in the response and check that the key values are equal to your variables. Could do something like this:
json_data_xbox = json.loads(
'{"player": [{"name": "Salah", "rating": "96", "rarity": "TOTS", "prices": 5380000}, {"name": "Salah", "rating": "93", "rarity": "FOF PTG", "prices": 956000}]}')
name = "Salah"
rating_var = "96"
price = ""
for i in json_data_xbox["player"]:
if i["name"] == name and i["rating"] == rating_var:
price = i["prices"]
print(price)
Related
I am really struggling with this one. I'm new to python and I'm trying to extract data from an API.
I have managed to run the script below but I need to amend it to filter on multiple values for one column, lets say England and Scotland. Is there an equivelant to the SQL IN operator e.g. Area_Name IN ('England','Scotland').
from requests import get
from json import dumps
ENDPOINT = "https://api.coronavirus.data.gov.uk/v1/data"
AREA_TYPE = "nation"
AREA_NAME = "england"
filters = [
f"areaType={ AREA_TYPE }",
f"areaName={ AREA_NAME }"
]
structure = {
"date": "date",
"name": "areaName",
"code": "areaCode",
"dailyCases": "newCasesByPublishDate",
}
api_params = {
"filters": str.join(";", filters),
"structure": dumps(structure, separators=(",", ":")),
"latestBy": "cumCasesByPublishDate"
}
formats = [
"json",
"xml",
"csv"
]
for fmt in formats:
api_params["format"] = fmt
response = get(ENDPOINT, params=api_params, timeout=10)
assert response.status_code == 200, f"Failed request for {fmt}: {response.text}"
print(f"{fmt} data:")
print(response.content.decode())
I have tried the script, and dict is the easiest type to handle in this case.
Given your json data output
data = {"length":1,"maxPageLimit":1,"data":[{"date":"2020-09-17","name":"England","code":"E92000001","dailyCases":2788}],"pagination":{"current":"/v1/data?filters=areaType%3Dnation%3BareaName%3Dengland&structure=%7B%22date%22%3A%22date%22%2C%22name%22%3A%22areaName%22%2C%22code%22%3A%22areaCode%22%2C%22dailyCases%22%3A%22newCasesByPublishDate%22%7D&latestBy=cumCasesByPublishDate&format=json&page=1","next":null,"previous":null,"first":"/v1/data?filters=areaType%3Dnation%3BareaName%3Dengland&structure=%7B%22date%22%3A%22date%22%2C%22name%22%3A%22areaName%22%2C%22code%22%3A%22areaCode%22%2C%22dailyCases%22%3A%22newCasesByPublishDate%22%7D&latestBy=cumCasesByPublishDate&format=json&page=1","last":"/v1/data?filters=areaType%3Dnation%3BareaName%3Dengland&structure=%7B%22date%22%3A%22date%22%2C%22name%22%3A%22areaName%22%2C%22code%22%3A%22areaCode%22%2C%22dailyCases%22%3A%22newCasesByPublishDate%22%7D&latestBy=cumCasesByPublishDate&format=json&page=1"}}
You can try something like this:
countries = ['England', 'France', 'Whatever']
return [country for country in data where country['name'] in countries]
I presume the data list is the only interesting key in the data dict since all others do not have any meaningful values.
A part of json file's content as below:
{"ID": "PK45", "People": "Kate", "Date": "2020-01-05"}, {"ID": "OI85", "People": "John", "Date": "2020-01-18" }, {"ID": "CN658", "People": "Pevo", "Date": "2020-02-01" }
It has multiple portions containing "ID", "People" and "Date".
What I want to do is to retrieve John's ID (in this case "OI85". )
If the key is unique, I can use:
data_content = json.loads(data)
ID = data_content['ID']
But there are multiple similar portions. So I can only locate the "John" first:
with open("C:\\the_file.json") as data_file:
data = data_file.read()
where = data.find('John')
where_1 = data[where - 20 : where]
ID = where_1[where_1.find('ID')+3:where_1.find('ID')+7]
print (ID)
It looked clumsy.
What will be the smart json way to retrieve the specific data from multiple similar portions in a .json file?
Thank you.
Iterate on the list of dicts until you find the right one:
import json
data = '[{"ID": "PK45", "People": "Kate", "Date": "2020-01-05"}, {"ID": "OI85", "People": "John", "Date": "2020-01-18" }, {"ID": "CN658", "People": "Pevo", "Date": "2020-02-01" }]'
data_content = json.loads(data)
def find_id_by_name(name, data):
for d in data:
if d['People'] == name:
return d["ID"]
else:
raise ValueError('Name not found')
print(find_id_by_name('John', data_content))
# OI85
print(find_id_by_name('Jane', data_content))
# ... ValueError: Name not found
If you have to do many such searches, it may be worth creating another dict from your data to associate IDs to names:
ids_by_name = {d['People']:d['ID'] for d in data_content}
print(ids_by_name['John'])
# OI85
You probably should use the json module, which makes the task trivial:
import json
with open('data.json') as f:
records = json.load(f)
for record in records:
if record['People'] == 'John':
print(record['ID'])
break
I am a beginner to python and scripting so I am unfamiliar with how json innately works, but this is the problem I have. I wrote a script which took values of the "location" variable from the json file I was reading and used googlemaps API to find the country this location was in. However, as some of these locations are repeat, and I did not want to repeatedly check the same location over and over. I stored all the values retrieved from the location variable in a list, then converted the list into a set to get rid of duplicates.
My question is this: once I have retrieved the country data (I have the data stored in a list), how can I add this country data to my original json file?
For instance, these are a few of my tests from the json file I am reading.
{"login": "hi", "name": "hello", "location": "Seoul, South Korea"}
{"login": "hi", "name": "hello", "location": null}
{"login": "hi", "name": "hello", "location": "Berlin, Germany"}
{"login": "hi", "name": "hello", "location": "Pittsburgh, PA"}
{"login": "hi", "name": "hello", "location": "London"}
{"login": "hi", "name": "hello", "location": "Tokyo, Japan"}
input = codecs.open(inputFile, 'r', 'utf8')
for line in input.readlines():
temp = json.loads(line)
if (temp['location'] != None): #some locations are Null
locationList.append(temp['location'])
input.close()
locationList = list(set(locationList))
print(locationList)
#getting location data, storing it in countryList variable
for uniqueLocation in locationList:
geocodeResult = gm.geocode(uniqueLocation)[0] #getting information about each location
geoObject = geocodeResult['address_components'] #gettnig just the address components
for item in geoObject: #iterating in object
if item['types'][0] == 'country': #if first element of this item is country
countryName = item['long_name'] #then retrieve long_name from this item
countryList.append(countryName)
print(countryList)
Check this out:
How to append data to a json file?
I'm trying to call API's from various cryptocurrency exchanges in PYTHON.
This is the API JSON-string that gets returned by the following URL (https://api.mintpal.com/v1/market/stats/uro/BTC)
[
{
"market_id": "210",
"coin": "Uro",
"code": "URO",
"exchange": "BTC",
"last_price": "0.00399700",
"yesterday_price": "0.00353011",
"change": "+13.23",
"24hhigh": "0.00450000",
"24hlow": "0.00353010",
"24hvol": "6.561",
"top_bid": "0.00374001",
"top_ask": "0.00399700"
}
]
I'm interested in getting the "Last Price", I print it using the following code.
import urllib2
import json
url = 'https://api.mintpal.com/v1/market/stats/URO/BTC'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
for item in URO_data:
URO_last_price = item['last_price']
print URO_last_price
So far so good. However, I'm trying to do the same thing for the Bittrex exchange using the following URL (https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-uro)
The JSON String returned looks as follows:
{
"success": true,
"message": "",
"result": [
{
"MarketName": "BTC-URO",
"High": 0.00479981,
"Low": 0.00353505,
"Volume": 30375.93454693,
"Last": 0.00391656,
"BaseVolume": 120.61056568,
"TimeStamp": "2014-07-29T17:54:35.897",
"Bid": 0.00393012,
"Ask": 0.00395967,
"OpenBuyOrders": 182,
"OpenSellOrders": 182,
"PrevDay": 0.00367999,
"Created": "2014-05-15T05:46:29.917"
}
]
}
This JSON string has a different structure then the one before, and I can't use my first code to get the value "LAST". However, I can work around it by printing 'string index', but that's not a solution.
url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
URO_String = str(URO_data)
last_price = URO_String[79:89]
URO_LastPrice = float(last_price)
print last_price
I want to get the value of "Last" in the second JSON string.
The Json is best thought of as a bunch of layers, and you need to get the information you need by going layer by layer.
The first layer you need to get is "result" with:
URO_data["result"]
REsult is an array, so you need to get the first item in the array by index
URO_data["result"][0]
From that object, you need to get the Last entry
URO_data["result"][0]["Last"]
That should work, but I haven't tested it.
You can access it as bellow,
>>>
>>> url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
>>> json_obj = urllib2.urlopen(url)
>>> URO_data = json.load(json_obj)
>>> URO_data['result']['Last']
0.00396129
>>>
I was wondering how I could import a JSON file, and then save that to an ordered CSV file, with header row and the applicable data below.
Here's what the JSON file looks like:
[
{
"firstName": "Nicolas Alexis Julio",
"lastName": "N'Koulou N'Doubena",
"nickname": "N. N'Koulou",
"nationality": "Cameroon",
"age": 24
},
{
"firstName": "Alexandre Dimitri",
"lastName": "Song-Billong",
"nickname": "A. Song",
"nationality": "Cameroon",
"age": 26,
etc. etc. + } ]
Note there are multiple 'keys' (firstName, lastName, nickname, etc.). I would like to create a CSV file with those as the header, then the applicable info beneath in rows, with each row having a player's information.
Here's the script I have so far for Python:
import urllib2
import json
import csv
writefilerows = csv.writer(open('WCData_Rows.csv',"wb+"))
api_key = "xxxx"
url = "http://worldcup.kimonolabs.com/api/players?apikey=" + api_key + "&limit=1000"
json_obj = urllib2.urlopen(url)
readable_json = json.load(json_obj)
list_of_attributes = readable_json[0].keys()
print list_of_attributes
writefilerows.writerow(list_of_attributes)
for x in readable_json:
writefilerows.writerow(x[list_of_attributes])
But when I run that, I get a "TypeError: unhashable type:'list'" error. I am still learning Python (obviously I suppose). I have looked around online (found this) and can't seem to figure out how to do it without explicitly stating what key I want to print...I don't want to have to list each one individually...
Thank you for any help/ideas! Please let me know if I can clarify or provide more information.
Your TypeError is occuring because you are trying to index a dictionary, x with a list, list_of_attributes with x[list_of_attributes]. This is not how python works. In this case you are iterating readable_json which appears it will return a dictionary with each iteration. There is no need pull values out of this data in order to write them out.
The DictWriter should give you what your looking for.
import csv
[...]
def encode_dict(d, out_encoding="utf8"):
'''Encode dictionary to desired encoding, assumes incoming data in unicode'''
encoded_d = {}
for k, v in d.iteritems():
k = k.encode(out_encoding)
v = unicode(v).encode(out_encoding)
encoded_d[k] = v
return encoded_d
list_of_attributes = readable_json[0].keys()
# sort fields in desired order
list_of_attributes.sort()
with open('WCData_Rows.csv',"wb+") as csv_out:
writer = csv.DictWriter(csv_out, fieldnames=list_of_attributes)
writer.writeheader()
for data in readable_json:
writer.writerow(encode_dict(data))
Note:
This assumes that each entry in readable_json has the same fields.
Maybe pandas could do this - but I newer tried to read JSON
import pandas as pd
df = pd.read_json( ... )
df.to_csv( ... )
pandas.DataFrame.to_csv
pandas.io.json.read_json
EDIT:
data = ''' [
{
"firstName": "Nicolas Alexis Julio",
"lastName": "N'Koulou N'Doubena",
"nickname": "N. N'Koulou",
"nationality": "Cameroon",
"age": 24
},
{
"firstName": "Alexandre Dimitri",
"lastName": "Song-Billong",
"nickname": "A. Song",
"nationality": "Cameroon",
"age": 26,
}
]'''
import pandas as pd
df = pd.read_json(data)
print df
df.to_csv('results.csv')
result:
age firstName lastName nationality nickname
0 24 Nicolas Alexis Julio N'Koulou N'Doubena Cameroon N. N'Koulou
1 26 Alexandre Dimitri Song-Billong Cameroon A. Song
With pandas you can save it in csv, excel, etc (and maybe even directly in database).
And you can do some operations on data in table and show it as graph.