Related
My Code:
import requests
import json
web_page = requests.get("http://api.bart.gov/api/etd.aspx?cmd=etd&orig=mont&key=MW9S-E7SL-26DU-VV8V&json=y")
response = web_page.text
parsed_json = json.loads(response)
#print(parsed_json)
print(parsed_json['root']['date'])
print(parsed_json['root']['time'])
print(parsed_json['root']['station']['name'])
How to extract value of destination and minutes from below in Python.
[{'name': 'Montgomery St.', 'abbr': 'MONT', 'etd': [{'destination': 'Daly City', 'abbreviation': 'DALY', 'limited': '0', 'estimate': [{'minutes': '39', 'platform': '1', 'direction': 'South', 'length': '10', 'color': 'WHITE', 'hexcolor': '#ffffff', 'bikeflag': '1', 'delay': '220'}]}, {'destination': 'SF Airport', 'abbreviation': 'SFIA', 'limited': '0', 'estimate': [{'minutes': '16', 'platform': '1', 'direction': 'South', 'length': '10', 'color': 'YELLOW', 'hexcolor': '#ffff33', 'bikeflag': '1', 'delay': '132'}, {'minutes': '26', 'platform': '1', 'direction': 'South', 'length': '10', 'color': 'BLUE', 'hexcolor': '#0099cc', 'bikeflag': '1', 'delay': '69'}]}]}]
Try this:
json_obj = {'name': 'Montgomery St.', 'abbr': 'MONT', 'etd': [{'destination': 'Antioch', 'abbreviation': 'ANTC', 'limited': '0', 'estimate': [{'minutes': '1', 'platform': '2', 'direction': 'North', 'length': '10', 'color': 'YELLOW', 'hexcolor': '#ffff33', 'bikeflag': '1', 'delay': '254'}]},
{'destination': 'Daly City', 'abbreviation': 'DALY', 'limited': '0', 'estimate': [{'minutes': '39', 'platform': '1', 'direction': 'South', 'length': '0', 'color': 'BLUE', 'hexcolor': '#0099cc', 'bikeflag': '1', 'delay': '0'}]},
{'destination': 'SF Airport', 'abbreviation': 'SFIA', 'limited': '0', 'estimate': [{'minutes': '38', 'platform': '1', 'direction': 'South', 'length': '10', 'color': 'YELLOW', 'hexcolor': '#ffff33', 'bikeflag': '1', 'delay': '0'}]}]}
for item in json_obj['etd']:
dest = item['destination']
minute = item['estimate'][0]['minutes']
print(dest, minute)
Output:
Antioch 1
Daly City 39
SF Airport 38
The problem is in parsed_json['root']['station']['name']. parsed_json['root']['station'] is a list, not a dict, so it doesn't have name key. You need to use index 0 or iterate over it
for station in parsed_json['root']['station']:
for etd in station['etd']:
for estimate in etd['estimate']:
print(etd['destination'], estimate['minutes'])
Output
Daly City 35
SF Airport 16
SF Airport 26
Try this to get json data:
import json
# some JSON:
json_data= {'destination': 'Daly City', 'abbreviation': 'DALY', 'limited': '0', 'estimate': [{'minutes': '39', 'platform': '1', 'direction': 'South', 'length': '0', 'color': 'BLUE', 'hexcolor': '#0099cc', 'bikeflag': '1', 'delay': '0'}]}
# parse json_data:
data = json.dumps(json_data)
extract_json = json.loads(data)
print("Destination: "+extract_json["destination"])
print("Minutes: "+extract_json["estimate"][0]["minutes"])
Output:
Destination: Daly City
Minutes: 39
Assuming the data is in d_MONT:
d_MONT = {'name': 'Montgomery St.', 'abbr': 'MONT', 'etd': [{'destination': 'Antioch', 'abbreviation': 'ANTC', 'limited': '0', 'estimate': [{'minutes': '1', 'platform': '2', 'direction': 'North', 'length': '10', 'color': 'YELLOW', 'hexcolor': '#ffff33', 'bikeflag': '1', 'delay': '254'}]},
{'destination': 'Daly City', 'abbreviation': 'DALY', 'limited': '0', 'estimate': [{'minutes': '39', 'platform': '1', 'direction': 'South', 'length': '0', 'color': 'BLUE', 'hexcolor': '#0099cc', 'bikeflag': '1', 'delay': '0'}]},
{'destination': 'SF Airport', 'abbreviation': 'SFIA', 'limited': '0', 'estimate': [{'minutes': '38', 'platform': '1', 'direction': 'South', 'length': '10', 'color': 'YELLOW', 'hexcolor': '#ffff33', 'bikeflag': '1', 'delay': '0'}]}]}
This will find the next train to destinationRequired:
destinationList = d_MONT['etd']
destinationRequired = 'Daly City'
for destinationDict in destinationList:
if destinationDict['destination'] == destinationRequired:
earliest = None
for estimate in destinationDict['estimate']:
if earliest is None or estimate['minutes'] < eariest:
earliest = estimate['minutes']
print("Next train to {0}: {1} minutes".format(destinationRequired, earliest))
break
else:
print("No trains to {0}".format(destinationRequired))
Note there are more Pythonic ways to do this, and the code example above does not follow PEP8, but I think it is important you understand the basic logic of how to do what you want rather than a complex Python one-liner.
You do not document the JSON object format, so I don't think it is safe to assume the list of trains to destination will be in order, therefore the safest is to step through each one and find the earliest. It isn't even clear if more than one train will ever be returned in the list, in which case a simple [0] would be sufficient rather than stepping through each one.
[
{'Year': 1901,
'Category': 'Chemistry',
'Prize': 'The Nobel Prize in Chemistry 1901',
'Motivation': '"in recognition of the extraordinary services he has rendered by the discovery of the laws of chemical dynamics and osmotic pressure in solutions"',
'Prize Share': '1/1',
'Laureate ID': 160,
'Laureate Type': 'Individual',
'Full Name': "Jacobus Henricus van 't Hoff",
'Birth Date': '1852-08-30',
'Birth City': 'Rotterdam',
'Birth Country': 'Netherlands',
'Sex': 'Male',
'Organization Name': 'Berlin University',
'Organization City': 'Berlin',
'Organization Country': 'Germany',
'Death Date': '1911-03-01',
'Death City': 'Berlin',
'Death Country': 'Germany'},
{'Year': 1901,
'Category': 'Literature',
'Prize': 'The Nobel Prize in Literature 1901',
'Motivation': '"in special recognition of his poetic composition, which gives evidence of lofty idealism, artistic perfection and a rare combination of the qualities of both heart and intellect"',
'Prize Share': '1/1',
'Laureate ID': 569,
'Laureate Type': 'Individual',
'Full Name': 'Sully Prudhomme',
'Birth Date': '1839-03-16',
'Birth City': 'Paris',
'Birth Country': 'France',
'Sex': 'Male',
'Organization Name': '',
'Organization City': '',
'Organization Country': '',
'Death Date': '1907-09-07',
'Death City': 'Châtenay',
'Death Country': 'France'}
]
If you want to find, how many person belong to same birth country only from given list of dict, you can use the following code :
from collections import Counter
li = [each['Birth City'] for each in val if each['Birth City']]
print(dict(Counter(li)))
OUTPUT
{'Rotterdam': 1, 'Paris': 1}
I am new in python and tried to understand the working with dictionaries operations but stuck in between.
I have data like below:
[{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'}, {'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'}, {'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'}, {'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'}, {'mesure':'30', 'name': 'chennai', 'age': '25', 'class':'kg3'}, {'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'}, {'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'}{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'}, {'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}]
I want to convert it in format like:
[{"Name": "mumbai",
"data": [{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'},
{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'},
{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'}]}
{"Name": "hyd",
"data":[{'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},
{'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'},
{'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'}]}
{"Name": "pune",
"data":[{'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'},
{'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'},
{'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}]}]
I Tried:
def dir_data(data):
main_list = []
main_dir = []
for i in data:
names = i["name"]
main_dir.append({"name": names, "data": i})
print(main_dir)
if __name__== "__main__":
data = [{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'}, {'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'}, {'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'}, {'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'}, {'mesure':'30', 'name': 'chennai', 'age': '25', 'class':'kg3'}, {'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'}, {'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'}{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'}, {'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}]
dir_data(data)
I tried above code but couldn't get exact output so please guide me to get it....
Thank you
def dir_data(data):
items = []
names = []
for i in data:
if i['name'] not in names:
items.append({"Name": i['name'], "data": [i]})
names.append(i['name'])
else:
iname = names.index(i['name'])
items[iname]['data'].append(i)
return items
data = [{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'},
{'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},
{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'},
{'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'},
{'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'},
{'mesure':'30', 'name': 'chennai', 'age': '25', 'class':'kg3'},
{'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'},
{'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'},
{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'},
{'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}
]
print(dir_data(data))
Try that one.
I can see the code you have written seems to be working but, you haven't returning the function, comma missing in data and also there is some mistakes in the way of call the function.
Just call the function like this,
def dir_data(data):
main_list = []
main_dir = []
for i in data:
names = i["name"]
main_dir.append({"name": names, "data": i})
return(main_dir)
data = [{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'}, {'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'}, {'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'}, {'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'}, {'mesure':'30', 'name': 'chennai', 'age': '25', 'class':'kg3'}, {'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'}, {'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'}, {'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}]
dir_data(data)
You can get desired solution by using below code
test_data = [{'mesure': '10', 'name': 'mumbai', 'age': '15', 'class': 'kg1'}, {'mesure': '20', 'name': 'hyd', 'age': '20', 'class': 'kg2'}, {'mesure': '11', 'name': 'mumbai', 'age': '145', 'class': 'kg6'}, {'mesure': '21', 'name': 'hyd', 'age': '20', 'class': 'kg2'}, {'mesure': '40', 'name': 'pune', 'age': '30', 'class': 'kg4'}, {'mesure': '30', 'name': 'chennai', 'age': '25', 'class': 'kg3'}, {'mesure': '41', 'name': 'pune', 'age': '30', 'class': 'kg7'}, {'mesure': '22', 'name': 'hyd', 'age': '20', 'class': 'kg2'}, {'mesure': '12', 'name': 'mumbai', 'age': '40', 'class': 'kg7'}, {'mesure': '46', 'name': 'pune', 'age': '30', 'class': 'kg8'}]
dic = dict()
for i in test_data:
dic.setdefault(i['name'].title(),[]).append(i)
result = [{"name":k ,"data":v} for k,v in dic.items()]
Output
[{'data': [{'class': 'kg4', 'age': '30', 'name': 'pune', 'mesure': '40'},
{'class': 'kg7', 'age': '30', 'name': 'pune', 'mesure': '41'},
{'class': 'kg8', 'age': '30', 'name': 'pune', 'mesure': '46'}], 'name': 'Pune'},
{'data': [{'class': 'kg3', 'age': '25', 'name': 'chennai', 'mesure': '30'}], 'name': 'Chennai'},
{ 'data': [{'class': 'kg2', 'age': '20', 'name': 'hyd', 'mesure': '20'},
{'class': 'kg2', 'age': '20', 'name': 'hyd', 'mesure': '21'},
{'class': 'kg2', 'age': '20', 'name': 'hyd', 'mesure': '22'}], 'name': 'Hyd'},
{
'data': [{'class': 'kg1', 'age': '15', 'name': 'mumbai', 'mesure': '10'},
{'class': 'kg6', 'age': '145', 'name': 'mumbai', 'mesure': '11'},
{'class': 'kg7', 'age': '40', 'name': 'mumbai', 'mesure': '12'}], 'name': 'Mumbai'}]
Try this:
import json
data = [{'mesure':'10', 'name': 'mumbai', 'age': '15', 'class':'kg1'}, {'mesure':'20', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'11', 'name': 'mumbai', 'age': '145', 'class':'kg6'}, {'mesure':'21', 'name': 'hyd', 'age': '20', 'class':'kg2'}, {'mesure':'40', 'name': 'pune', 'age': '30', 'class':'kg4'}, {'mesure':'30', 'name': 'chennai', 'age': '25', 'class':'kg3'}, {'mesure':'41', 'name': 'pune', 'age': '30', 'class':'kg7'}, {'mesure':'22', 'name': 'hyd', 'age': '20', 'class':'kg2'},{'mesure':'12', 'name': 'mumbai', 'age': '40', 'class':'kg7'}, {'mesure':'46', 'name': 'pune', 'age': '30', 'class':'kg8'}]
def dir_data(data):
# set guarantees the uniqueness of each name
names = set([item['name'] for item in data])
main_dir = []
# collect the data for each name
for name in names:
name_data = [d for d in data if d['name']==name]
main_dir.append({"Name":name,"data":name_data})
return json.dumps(main_dir)
Below is the solution which will give you the exact result as described by you:
def checkKey(dict, key):
if key in dict:
return True
else:
return False
def dir_data(data):
for item in test:
if checkKey(tem_dict, item['name']):
tem_dict[item['name']].append(item)
else:
tem_dict[item['name']] = []
tem_dict[item['name']].append(item)
res_dict = {}
res = []
for item in tem_dict:
print item
res_dict['Name'] = item
res_dict['data'] = tem_dict[item]
res.append(res_dict)
res_dict = {}
return res
let me know if this works for you or not.
How can I convert a pandas dataframe to a dict using unique column values as the keys for the dictionary? In this case I want to use unique username's as the key.
Here is my progress so far based on information found on here and online.
My test dataframe:
import pandas
import pprint
df = pandas.DataFrame({
'username': ['Kevin', 'John', 'Kevin', 'John', 'Leslie', 'John'],
'sport': ['Soccer', 'Football', 'Racing', 'Tennis', 'Baseball', 'Bowling'],
'age': ['51','32','20','19','34','27'],
'team': ['Cowboyws', 'Packers', 'Sonics', 'Raiders', 'Wolves', 'Lakers']
})
I can create a dictionary by doing this:
dct = df.to_dict(orient='records')
pprint.pprint(dct, indent=4)
>>>>[{'age': '51', 'sport': 'Soccer', 'team': 'Cowboyws', 'username': 'Kevin'},
{'age': '32', 'sport': 'Football', 'team': 'Packers', 'username': 'John'},
{'age': '20', 'sport': 'Racing', 'team': 'Sonics', 'username': 'Kevin'},
{'age': '19', 'sport': 'Tennis', 'team': 'Raiders', 'username': 'John'},
{'age': '34', 'sport': 'Baseball', 'team': 'Wolves', 'username': 'Leslie'},
{'age': '27', 'sport': 'Bowling', 'team': 'Lakers', 'username': 'John'}]
I tried using the groupby and apply method which got me closer but it converts all the values to lists. I want them to remain as dictionaries so i can retain the each value's key:
result = df.groupby('username').apply(lambda x: x.values.tolist()).to_dict()
pprint.pprint(result, indent=4)
{ 'John': [ ['32', 'Football', 'Packers', 'John'],
['19', 'Tennis', 'Raiders', 'John'],
['27', 'Bowling', 'Lakers', 'John']],
'Kevin': [ ['51', 'Soccer', 'Cowboyws', 'Kevin'],
['20', 'Racing', 'Sonics', 'Kevin']],
'Leslie': [['34', 'Baseball', 'Wolves', 'Leslie']]}
This is the desired result I want:
{
'John': [{'age': '32', 'sport': 'Football', 'team': 'Packers', 'username': 'John'},
{'age': '19', 'sport': 'Tennis', 'team': 'Raiders', 'username': 'John'},
{'age': '27', 'sport': 'Bowling', 'team': 'Lakers', 'username': 'John'}],
'Kevin': [{'age': '51', 'sport': 'Soccer', 'team': 'Cowboyws', 'username': 'Kevin'},
{'age': '20', 'sport': 'Racing', 'team': 'Sonics', 'username': 'Kevin'}],
'Leslie': [{'age': '34', 'sport': 'Baseball', 'team': 'Wolves', 'username': 'Leslie'}]
}
Use groupby and apply. Inside the apply, call to_dict with the "records" orient (similar to what you've figured out already).
df.groupby('username').apply(lambda x: x.to_dict(orient='r')).to_dict()
I prefer using for loop here , also you may want to drop the username columns , since it is redundant
d = {x: y.drop('username',1).to_dict('r') for x , y in df.groupby('username')}
d
Out[212]:
{'John': [{'age': '32', 'sport': 'Football', 'team': 'Packers'},
{'age': '19', 'sport': 'Tennis', 'team': 'Raiders'},
{'age': '27', 'sport': 'Bowling', 'team': 'Lakers'}],
'Kevin': [{'age': '51', 'sport': 'Soccer', 'team': 'Cowboyws'},
{'age': '20', 'sport': 'Racing', 'team': 'Sonics'}],
'Leslie': [{'age': '34', 'sport': 'Baseball', 'team': 'Wolves'}]}
I'd like to store the following data in a data dictionary so that I can easily export it to a CSV file. The problem is that the columns for each school id are not always in the same order:
text = """
school id= 28392
name|year|degree|age|race
Susan A. Smith|2007|PhD|27|white
Fred Collins|2006|PhD|26|hispanic
Amber Real|2007|MBA|28|white
Mike Lee|2003|PhD|27|white
school id= 273533123
name|year|age|race|degree
John B. Black|2003|27|hispanic|MBA
Steven Smith|2005|28|black|PhD
Jacob Waters|2003|25|hispanic|MBA
school id = 3452332
name|year|race|age|degree
Peter Hintze|2002|white|27|Bachelors
Ann Graden|2004|black|25|MBA
Bryan Stewart|2004|white|28|PhD
"""
I'd like to be able to eventually output all data to a CSV file with the following headings:
school id|year|name|age|race|degree
Can I do this in Python?
This actually seems pretty easy. Process the file into a data structure, then export it into a csv.
school = None
headers = None
data = {}
for line in text.splitlines():
if line.startswith("school id"):
school = line.split('=')[1].strip()
headers = None
continue
if school is not None and headers is None:
headers = line.split('|')
continue
if school is not None and headers is not None and line:
if not school in data:
data[school] = []
datum = dict(zip(headers, line.split('|')))
data[school].append(datum)
In [29]: data
Out[29]:
{'273533123': [{'age': '27',
'degree': 'MBA',
'name': 'John B. Black',
'race': 'hispanic',
'year': '2003'},
{'age': '28',
'degree': 'PhD',
'name': 'Steven Smith',
'race': 'black',
'year': '2005'},
{'age': '25',
'degree': 'MBA',
'name': 'Jacob Waters',
'race': 'hispanic',
'year': '2003'}],
'28392': [{'age': '27',
'degree': 'PhD',
'name': 'Susan A. Smith',
'race': 'white',
'year': '2007'},
{'age': '26',
'degree': 'PhD',
'name': 'Fred Collins',
'race': 'hispanic',
'year': '2006'},
{'age': '28',
'degree': 'MBA',
'name': 'Amber Real',
'race': 'white',
'year': '2007'},
{'age': '27',
'degree': 'PhD',
'name': 'Mike Lee',
'race': 'white',
'year': '2003'}],
'3452332': [{'age': '27',
'degree': 'Bachelors',
'name': 'Peter Hintze',
'race': 'white',
'year': '2002'},
{'age': '25',
'degree': 'MBA',
'name': 'Ann Graden',
'race': 'black',
'year': '2004'},
{'age': '28',
'degree': 'PhD',
'name': 'Bryan Stewart',
'race': 'white',
'year': '2004'}]}