Develop Reference

Need help translating a nested dictionary into a pandas dataframe

Looking into translating the following nested dictionary which is an API pull from Yelp into a pandas dataframe to run visualization on:
Top 50 Pizzerias in Chicago
{'businesses': [{'alias': 'pequods-pizzeria-chicago',
'categories': [{'alias': 'pizza', 'title': 'Pizza'}],
'coordinates': {'latitude': 41.92187, 'longitude': -87.664486},
'display_phone': '(773) 327-1512',
'distance': 2158.7084581522413,
'id': 'DXwSYgiXqIVNdO9dazel6w',
'image_url': 'https://s3-media1.fl.yelpcdn.com/bphoto/8QJUNblfCI0EDhOjuIWJ4A/o.jpg',
'is_closed': False,
'location': {'address1': '2207 N Clybourn Ave',
'address2': '',
'address3': '',
'city': 'Chicago',
'country': 'US',
'display_address': ['2207 N Clybourn Ave',
'Chicago, IL 60614'],
'state': 'IL',
'zip_code': '60614'},
'name': "Pequod's Pizzeria",
'phone': '+17733271512',
'price': '$$',
'rating': 4.0,
'review_count': 6586,
'transactions': ['restaurant_reservation', 'delivery'],
'url': 'https://www.yelp.com/biz/pequods-pizzeria-chicago?adjust_creative=wt2WY5Ii_urZB8YeHggW2g&utm_campaign=yelp_api_v3&utm_medium=api_v3_business_search&utm_source=wt2WY5Ii_urZB8YeHggW2g'},
{'alias': 'lou-malnatis-pizzeria-chicago',
'categories': [{'alias': 'pizza', 'title': 'Pizza'},
{'alias': 'italian', 'title': 'Italian'},
{'alias': 'sandwiches', 'title': 'Sandwiches'}],
'coordinates': {'latitude': 41.890357,
'longitude': -87.633704},
'display_phone': '(312) 828-9800',
'distance': 4000.9990531720227,
'id': '8vFJH_paXsMocmEO_KAa3w',
'image_url': 'https://s3-media3.fl.yelpcdn.com/bphoto/9FiL-9Pbytyg6usOE02lYg/o.jpg',
'is_closed': False,
'location': {'address1': '439 N Wells St',
'address2': '',
'address3': '',
'city': 'Chicago',
'country': 'US',
'display_address': ['439 N Wells St',
'Chicago, IL 60654'],
'state': 'IL',
'zip_code': '60654'},
'name': "Lou Malnati's Pizzeria",
'phone': '+13128289800',
'price': '$$',
'rating': 4.0,
'review_count': 6368,
'transactions': ['pickup', 'delivery'],
'url': 'https://www.yelp.com/biz/lou-malnatis-pizzeria-chicago?adjust_creative=wt2WY5Ii_urZB8YeHggW2g&utm_campaign=yelp_api_v3&utm_medium=api_v3_business_search&utm_source=wt2WY5Ii_urZB8YeHggW2g'},
....]
I've tried the below and iterations of it but haven't had any luck.
df = pd.DataFrame.from_dict(topresponse)
Im really new to coding so any advice would be helpful

response["businesses"] is a list of records, so:
df = pd.DataFrame.from_records(response["businesses"])

How to Arrange a List of Dictionaries in Python [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
data=[{'address': 'High Tech Campus 60', 'beta': 1.406659, 'ceo': 'Mr. Richard Clemmer', 'changes': -3.9400024, 'cik': '0001413447', 'city': 'Eindhoven', 'companyName': 'NXP Semiconductors N.V.', 'country': 'NL', 'currency': 'USD', ...}]
I have a dictionary.
Need to receive a list of dictionaries comma separated: [{},{},..]
How do I add them in a loop?
I tried to use append:
data_list.append(data.copy())
But it returns smth different: [[{...}]]
How do I get a list of such format:
[{'address': 'High Tech Campus 60', 'beta': 1.406659, 'ceo': 'Mr. Richard Clemmer', 'changes': -3.9400024, 'cik': '0001413447', 'city': 'Eindhoven', 'companyName': 'NXP Semiconductors N.V.', 'country': 'NL', 'currency': 'USD', ...}, {'address': '41st, 1155 Rene-Leve...W Flr 4000', 'beta': 2.219123, 'ceo': 'Mr. Klaus Paulini', 'changes': -0.00999999, 'cik': '0001113423', 'city': 'MONTREAL', 'companyName': 'Aeterna Zentaris Inc.', 'country': 'CA', 'currency': 'USD', ...}, {'address': '125 Summer Street', 'beta': 0.0, 'ceo': 'Dr. Jean-Pierre Som...ossi Ph.D.', 'changes': 2.5800018, 'cik': '0001593899', 'city': 'Boston', 'companyName': 'Atea Pharmaceuticals, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '401 Charmany Dr', 'beta': 1.073689, 'ceo': 'Mr. Corey Chambas', 'changes': 0.0, 'cik': '0001521951', 'city': 'Madison', 'companyName': 'First Business Finan...ices, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '490 Arsenal Way', 'beta': 0.0, 'ceo': 'Mr. Marc A. Cohen', 'changes': -0.9699974, 'cik': '0001662579', 'city': 'Watertown', 'companyName': 'C4 Therapeutics, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': 'General-Guisan-Strasse 6', 'beta': 1.629418, 'ceo': 'Mr. Carlos Creus Moreira', 'changes': -0.09000015, 'cik': '0001738699', 'city': 'Zug', 'companyName': 'WISeKey Internationa...Holding AG', 'country': 'CH', 'currency': 'USD', ...}, {'address': '508 W Wall St Ste 800', 'beta': 1.7762, 'ceo': 'Mr. Stephen Jumper', 'changes': -0.04999995, 'cik': '0000799165', 'city': 'Midland', 'companyName': 'Dawson Geophysical Company', 'country': 'US', 'currency': 'USD', ...}, {'address': '955 Perimeter Road', 'beta': 0.0, 'ceo': 'Mr. Ravi Vig', 'changes': -1.2900009, 'cik': '0000866291', 'city': 'Manchester', 'companyName': 'Allegro MicroSystems, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '490 Lapp Rd', 'beta': 1.138646, 'ceo': 'Ms. Geraldine Henwood', 'changes': -0.04999995, 'cik': '0001588972', 'city': 'Malvern', 'companyName': 'Recro Pharma, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '5 Haplada Street, PO Box 5011', 'beta': 1.396288, 'ceo': 'Mr. Guy Bernstein', 'changes': -0.9300003, 'cik': '0000876779', 'city': 'OR YEHUDA', 'companyName': 'Magic Software Enter...rises Ltd.', 'country': 'IL', 'currency': 'USD', ...}, {'address': '111 West 33rd Street', 'beta': 0.0, 'ceo': 'Mr. Richard Gumer', 'changes': -0.20249999, 'cik': '0001823323', 'city': 'New York', 'companyName': 'KL Acquisition Corp', 'country': 'US', 'currency': 'USD', ...}, {'address': '2 Canal Park Ste 4', 'beta': 1.907176, 'ceo': 'Mr. Langley Steinert', 'changes': -1.4399986, 'cik': '0001494259', 'city': 'Cambridge', 'companyName': 'CarGurus, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '119 Standard St', 'beta': 1.592636, 'ceo': 'Mr. Ethan Brown', 'changes': -3.859993, 'cik': '0001655210', 'city': 'El Segundo', 'companyName': 'Beyond Meat, Inc.', 'country': 'US', 'currency': 'USD', ...}, {'address': '3854 American Way Ste A', 'beta': 0.502729, 'ceo': 'Mr. Paul Kusserow', 'changes': -1.5899963, 'cik': '0000896262', 'city': 'Baton Rouge', 'companyName': 'Amedisys, Inc.', 'country': 'US', 'currency': 'USD', ...}, ...]
Ok, it looks like initially I have not a dictionary but a list of dictionaries from one element. So how do I add another dictionary to the list after comma?
Upd: I managed to receive a list of dictionaries. It appeared it's not fully correct as some rows include additional fields. The list looks like this:'currency': 'USD', ...}, 'code', 'status', {'address': '5 ...
How can I validate a list of dictionaries and make sure every dictionary matches predefined list of columns.
enter code here

data_list.append(data[0].copy())
You could also do
data_list = data_list + data

Json to Pandas, include "Parents"

With a list of 150+ Neighborhoods , I am using Foursquare API to retrieve nearby venues at 500m radius of a given Neighbourhood. Each Neighbourhood is expected to return 10-20 nearby venues.
Refer to snippet of json result as returned by Foursquare.
With results['response']['groups'][0]['items'], I able to retrieve the nearby venues information and make it a Table as below. However results['response']['groups'][0]['items'] does not have the Neighbourhood ( under headerFullLocation in json) of associated venues.
Q: How can I link the Neighbourhood(headerFullLocation) to its associated nearby venue and add it as a column to table below? Thanks for the advice.
{'suggestedFilters': {'header': 'Tap to show:',
'filters': [{'name': 'Open now', 'key': 'openNow'}]},
'headerLocation': 'Alexandra Park',
'headerFullLocation': 'Alexandra Park, Toronto',**
'headerLocationGranularity': 'neighborhood',
'totalResults': 138,
'suggestedBounds': {'ne': {'lat': 43.6545000045, 'lng': -79.39379244047241},
'sw': {'lat': 43.645499995499996, 'lng': -79.4062075595276}},
'groups': [{'type': 'Recommended Places',
'name': 'recommended',
'items': [{'reasons': {'count': 0,
'items': [{'summary': 'This spot is popular',
'type': 'general',
'reasonName': 'globalInteractionReason'}]},
'venue': {'id': '5644dbaa498e7f7534154326',
'**name': 'Maker Pizza',**
'contact': {},
'location': {'address': '59 Cameron St',
'lat': 43.6504011331197,
'lng': -79.39804047841302,
'labeledLatLngs': [{'label': 'display',
'lat': 43.6504011331197,
'lng': -79.39804047841302}],
'distance': 164,
'postalCode': 'M5T 2H1',
'cc': 'CA',
'city': 'Toronto',
'state': 'ON',
'country': 'Canada',
'formattedAddress': ['59 Cameron St', 'Toronto ON M5T 2H1', 'Canada']},
'categories': [{'id': '4bf58dd8d48988d1ca941735',
'name': 'Pizza Place',
'pluralName': 'Pizza Places',
'shortName': 'Pizza',
'icon': {'prefix': 'https://ss3.4sqi.net/img/categories_v2/food/pizza_',
'suffix': '.png'},
'primary': True}],
'verified': False,
'stats': {'tipCount': 0,
'usersCount': 0,
'checkinsCount': 0,
'visitsCount': 0},
'beenHere': {'count': 0,
'lastCheckinExpiredAt': 0,
'marked': False,
'unconfirmedCount': 0},
'photos': {'count': 0, 'groups': []},
'hereNow': {'count': 0, 'summary': 'Nobody here', 'groups': []}},

Why don't you just do venues['Neighbourhood'] = response['headerFullLocation']. I am assuming, you send a separate request for each neigbhourhood and plan to concatenate multiple venue dataframes in the end.

how to find the no. of person from a particular country from the below code?

[
{'Year': 1901,
'Category': 'Chemistry',
'Prize': 'The Nobel Prize in Chemistry 1901',
'Motivation': '"in recognition of the extraordinary services he has rendered by the discovery of the laws of chemical dynamics and osmotic pressure in solutions"',
'Prize Share': '1/1',
'Laureate ID': 160,
'Laureate Type': 'Individual',
'Full Name': "Jacobus Henricus van 't Hoff",
'Birth Date': '1852-08-30',
'Birth City': 'Rotterdam',
'Birth Country': 'Netherlands',
'Sex': 'Male',
'Organization Name': 'Berlin University',
'Organization City': 'Berlin',
'Organization Country': 'Germany',
'Death Date': '1911-03-01',
'Death City': 'Berlin',
'Death Country': 'Germany'},
{'Year': 1901,
'Category': 'Literature',
'Prize': 'The Nobel Prize in Literature 1901',
'Motivation': '"in special recognition of his poetic composition, which gives evidence of lofty idealism, artistic perfection and a rare combination of the qualities of both heart and intellect"',
'Prize Share': '1/1',
'Laureate ID': 569,
'Laureate Type': 'Individual',
'Full Name': 'Sully Prudhomme',
'Birth Date': '1839-03-16',
'Birth City': 'Paris',
'Birth Country': 'France',
'Sex': 'Male',
'Organization Name': '',
'Organization City': '',
'Organization Country': '',
'Death Date': '1907-09-07',
'Death City': 'Châtenay',
'Death Country': 'France'}
]

If you want to find, how many person belong to same birth country only from given list of dict, you can use the following code :
from collections import Counter
li = [each['Birth City'] for each in val if each['Birth City']]
print(dict(Counter(li)))
OUTPUT
{'Rotterdam': 1, 'Paris': 1}

How to use a for loop to iterate through a list of dictionaries, select a key, and append the values to a new list

I'm a beginner and this is a basic question. I need to use a for loop to iterate through a list of dictionaries, and for a certain key in each of the dictionaries, append the value to a new list. The original list is a list of cities, with each dictionary in the list representing a city and containing information about the city. I need the loop to pick out the 'Population' key in each city dictionary, and append the value to a new list called city_populations.
I've only managed to append the population from one of the dictionaries to the list. Having trouble getting it to iterate through the list of dictionaries and append each population from each city. Here are three things I've tried:
Attempt 1:
city_populations = []
for city in cities:
city_populations.append(cities[0]['Population'])
city_populations
Attempt 2:
city_populations = []
for city in cities:
city_populations.append(cities[index]['Population'])
city_populations
Attempt 3:
city_populations = []
for city in cities:
index = 0
city_populations.append(cities[index]['Population'])
index =+ 1
city_populations
Here is the list of cities:
[{'City': 'Buenos Aires',
'Country': 'Argentina',
'Population': 2891000,
'Area': 4758},
{'City': 'Toronto', 'Country': 'Canada', 'Population': 2800000, 'Area': 2731},
{'City': 'Pyeongchang',
'Country': 'South Korea',
'Population': 2581000,
'Area': 3194},
{'City': 'Marakesh', 'Country': 'Morocco', 'Population': 928850, 'Area': 200},
{'City': 'Albuquerque',
'Country': 'New Mexico',
'Population': 559277,
'Area': 491},
{'City': 'Los Cabos',
'Country': 'Mexico',
'Population': 287651,
'Area': 3750},
{'City': 'Greenville', 'Country': 'USA', 'Population': 84554, 'Area': 68},
{'City': 'Archipelago Sea',
'Country': 'Finland',
'Population': 60000,
'Area': 8300},
{'City': 'Walla Walla Valley',
'Country': 'USA',
'Population': 32237,
'Area': 33},
{'City': 'Salina Island', 'Country': 'Italy', 'Population': 4000, 'Area': 27},
{'City': 'Solta', 'Country': 'Croatia', 'Population': 1700, 'Area': 59},
{'City': 'Iguazu Falls',
'Country': 'Argentina',
'Population': 0,
'Area': 672}]
How could I achieve what I am trying to do here? Thanks for your help.

Your problem is that you are not actually using the different values in the dictionary as you iterate through it. Try this:
city_populations = []
for city in cities:
city_populations.append(city['Population'])
city_populations
Your 3rd attempt is actually almost there. It would work if you just put index = 0 outside of the loop.

Stop worrying about indexes. You want every Population field of each dictionary of the list of dictionaries.
Use a list comprehension:
city_populations = [city["Population"] for city in cities]

Here is my solution plus some bonus code:
cities = [{'City': 'Buenos Aires',
'Country': 'Argentina',
'Population': 2891000,
'Area': 4758},
{'City': 'Toronto', 'Country': 'Canada', 'Population': 2800000, 'Area': 2731},
{'City': 'Pyeongchang',
'Country': 'South Korea',
'Population': 2581000,
'Area': 3194},
{'City': 'Marakesh', 'Country': 'Morocco', 'Population': 928850, 'Area': 200},
{'City': 'Albuquerque',
'Country': 'New Mexico',
'Population': 559277,
'Area': 491},
{'City': 'Los Cabos',
'Country': 'Mexico',
'Population': 287651,
'Area': 3750},
{'City': 'Greenville', 'Country': 'USA', 'Population': 84554, 'Area': 68},
{'City': 'Archipelago Sea',
'Country': 'Finland',
'Population': 60000,
'Area': 8300},
{'City': 'Walla Walla Valley',
'Country': 'USA',
'Population': 32237,
'Area': 33},
{'City': 'Salina Island', 'Country': 'Italy', 'Population': 4000, 'Area': 27},
{'City': 'Solta', 'Country': 'Croatia', 'Population': 1700, 'Area': 59},
{'City': 'Iguazu Falls',
'Country': 'Argentina',
'Population': 0,
'Area': 672}]
# This is the specific solution to your problem
city_populations = []
for city in cities:
city_populations.append(city['Population'])
print(city_populations)
# In order to better understand what is happening please try also this code
for city in cities:
print (city)
print (city['Population'])
# A more pythonic and elegant way to solve the problem is using list comprehension
city_populations = [city["Population"] for city in cities]
print(city_populations)
# If you want to be able to access specific keys / values you can use items()
for key, value in cities[0].items():
print ("Key: " + key)
print("Value: " + str(value))