This is my data set, this is the column I separated from the csv file.
0 [{'id': 16, 'name': 'Animation'}, {'id': 35, '...
1 [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...
2 [{'id': 10749, 'name': 'Romance'}, {'id': 35, ...
3 [{'id': 35, 'name': 'Comedy'}, {'id': 18, 'nam...
4 [{'id': 35, 'name': 'Comedy'}]
How to get just a list with the content ['Animation', 'Adventure', 'Romance', 'Comedy', 'Comedy'] as output?
I guess you want to see something like that.
list_of_items = [[{'id': 16, 'name': 'Animation'}, {'id': 16, 'name': 'Animation2'}],[{'id': 16, 'name': 'Animation3'}, {'id': 16, 'name': 'Animation4'}]]
output_list = []
for item in list_of_items:
for dict in item:
output_list.append(dict['name'])
Output:
>>> print(output_list)
['Animation', 'Animation2', 'Animation3', 'Animation4']
I don't know if you made a typo but you have some errors with the ' in what you wrote.
But nevertheless from what I can see you have a list with dictionaries. So we loop through that list to access each dictionary and select what in the dictionary we want and append it to the list you created:
d = [{'id': 10749, 'name': 'Romance'}, {'id': 35, 'name': 'Comedy'}]
list_1 = []
for el in d:
list_1.append(el['name'])
print(list_1)
The output will be: ['Romance', 'Comedy']
It's unclear if you have a list of lists or just one list.
For a single list you can use a list comprehension:
dict_list = [{'id': 10749, 'name': 'Romance'}, {'id': 35, 'name': 'Comedy'}]
[dict_item['name'] for dict_item in dict_list]
Otherwise, you can unnest the first list and then do a list comprehension
dict_list = [[{'id': 1, 'name': 'Animation'}, {'id': 2, 'name': 'Comedy'}],[{'id': 3, 'name': 'Romance'}, {'id': 4, 'name': 'Comedy'}]]
[dict_item['name'] for dict_item in [dict_item for sublist in dict_list for dict_item in sublist]]
Related
I am new to python and learning how to use a dictionary comprehension. I have a movie cast dictionary that I would like to filter on a specific value using the dictionary comprehension technique. I was able to get it work but for some reason I get empty dictionaries added as well if the condition is not met. Why does it do it? And how can I ensure these are not included?
movie_cast = [{'id': 90633,'name': 'Gal Gadot','cast_id': 0, 'order': 0},
{'id': 62064, 'name': 'Chris Pine','cast_id': 15, 'order': 1},
{'id': 41091, 'name': 'Kristen Wiig', 'cast_id': 12,'order': 2},
{'id': 41092, 'name': 'Pedro Pascal', 'cast_id': 13, 'order': 3},
{'id': 32, 'name': 'Robin Wright', 'cast_id': 78, 'order': 4}]
limit = 1
cast_limit = []
for dict in movie_cast:
d = {key:value for (key,value) in dict.items() if dict['order'] < limit}
cast_limit.append(d)
print(cast_limit)
current_result = [{'id': 90633,'name': 'Gal Gadot','cast_id': 0, 'order': 0},
{'id': 62064, 'name': 'Chris Pine','cast_id': 15, 'order': 1},{},{},{}]
desired_result = [{'id': 90633,'name': 'Gal Gadot','cast_id': 0, 'order': 0},
{'id': 62064, 'name': 'Chris Pine','cast_id': 15, 'order': 1}]
Try with this (you need a list comprehension, not a dict comprehension):
cast_limit = [dct for dct in movie_cast if dct['order'] < limit]
I.e., you need to filter out elements of the list, not elements of a dict.
I have a large data set like ~30000 records. I would like to extract words like "Animation", "Comedy", "Family". It is successful for me to extract the words out and delete the id, however I do not know how to stack the words back according to their row.
My code currently:
import ast, json
import pandas as pd
from csv import reader
file_name = 'xx.csv'
data = []
with open(file_name, 'r', encoding= 'unicode_escape') as read_obj:
csv_reader = reader(read_obj)
headings = next(csv_reader)
for i in csv_reader:
data.extend(ast.literal_eval(i[7]))
df = pd.DataFrame(data)
del df["id"]
print(df)
And it would produce result:
name
0 Animation
1 Comedy
2 Family
3 Adventure
4 Fantasy
...
40060 Drama
40061 Thriller
40062 Action
40063 Drama
40064 Thriller
The large data set is in csv format, but the cell should be in json formatting.
Sample data:
[{'id': 16, 'name': 'Animation'}, {'id': 35, 'name': 'Comedy'}, {'id': 10751, 'name': 'Family'}]
[{'id': 12, 'name': 'Adventure'}, {'id': 14, 'name': 'Fantasy'}, {'id': 10751, 'name': 'Family'}]
[{'id': 10749, 'name': 'Romance'}, {'id': 35, 'name': 'Comedy'}]
[{'id': 35, 'name': 'Comedy'}, {'id': 18, 'name': 'Drama'}, {'id': 10749, 'name': 'Romance'}]
[{'id': 35, 'name': 'Comedy'}]
[{'id': 28, 'name': 'Action'}, {'id': 80, 'name': 'Crime'}, {'id': 18, 'name': 'Drama'}, {'id': 53, 'name': 'Thriller'}]
[{'id': 28, 'name': 'Action'}, {'id': 80, 'name': 'Crime'}, {'id': 18, 'name': 'Drama'}, {'id': 53, 'name': 'Thriller'}]
[{'id': 28, 'name': 'Action'}, {'id': 80, 'name': 'Crime'}, {'id': 18, 'name': 'Drama'}, {'id': 53, 'name': 'Thriller'}]
[{'id': 35, 'name': 'Comedy'}, {'id': 10749, 'name': 'Romance'}]
[{'id': 28, 'name': 'Action'}, {'id': 12, 'name': 'Adventure'}, {'id': 18, 'name': 'Drama'}, {'id': 10751, 'name': 'Family'}]
I think this does everything you need:
import json
import pandas as pd
df = pd.read_csv(file_name, encoding='unicode_escape', usecols=['name'])
result = df.to_json(orient='records')
parsed = json.loads(result)
json.dumps(parsed, indent=4)
I would like to add an id key to a list of dictionaries, where each id represents the enumerated nested dictionary.
Current list of dictionaries:
current_list_d = [{'id': 0, 'name': 'Paco', 'age': 18} #all id's are 0
{'id': 0, 'name': 'John', 'age': 20}
{'id': 0, 'name': 'Claire', 'age': 22}]
Desired output:
output_list_d = [{'id': 1, 'name': 'Paco', 'age': 18} #id's are counted/enumerated
{'id': 2, 'name': 'John', 'age': 20}
{'id': 3, 'name': 'Claire', 'age': 22}]
My code:
for d in current_list_d:
d["id"]+=1
You could use a simple for loop with enumerate and update in-place the id keys in the dictionaries:
for new_id, d in enumerate(current_list_d, start=1):
d['id'] = new_id
current_list_d
[{'id': 1, 'name': 'Paco', 'age': 18},
{'id': 2, 'name': 'John', 'age': 20},
{'id': 3, 'name': 'Claire', 'age': 22}]
You can use a variable.
id_val = 1
for dict in current_list_d :
dict["id"] = id_val
id_val+=1
I have a list of dictionaries:
movies['genres'].head()
where each line looks like:
0 [{'id': 16, 'name': 'Animation'}, {'id': 35, 'name': 'Comedy'}, {'id': 10751, 'name': 'Family'}]
1 [{'id': 12, 'name': 'Adventure'}, {'id': 14, 'name': 'Fantasy'}, {'id': 10751, 'name': 'Family'}]
2 [{'id': 10749, 'name': 'Romance'}, {'id': 35, 'name': 'Comedy'}]
3 [{'id': 35, 'name': 'Comedy'}, {'id': 18, 'name': 'Drama'}, {'id': 10749, 'name': 'Romance'}]
4 [{'id': 35, 'name': 'Comedy'}]
Name: genres, dtype: object
I would like to save it in a data frame where one column is 'id' and the rows are the id values and another column 'name' where the rows are the name values. I tried with:
pd.DataFrame(movies['genres'])
However when I ran it I obtained:
genres
0 [{'id': 16, 'name': 'Animation'}, {'id': 35, 'name': 'Comedy'}, {'id': 10751, 'name': 'Family'}]
1 [{'id': 12, 'name': 'Adventure'}, {'id': 14, 'name': 'Fantasy'}, {'id': 10751, 'name': 'Family'}]
2 [{'id': 10749, 'name': 'Romance'}, {'id': 35, 'name': 'Comedy'}]
Could you help me?
Regards
You should use the command .from_dict() as described here
df = pd.DataFrame.from_dict(movies["genres"])
I have an unknown number of lists of product results as dictionary entries that all have the same keys. I'd like to generate a new list of products that appear in all of the old lists.
'what products are available in all cities?'
given:
list1 = [{'id': 1, 'name': 'bat', 'price': 20.00}, {'id': 2, 'name': 'ball', 'price': 12.00}, {'id': 3, 'name': 'brick', 'price': 19.00}]
list2 = [{'id': 1, 'name': 'bat', 'price': 18.00}, {'id': 3, 'name': 'brick', 'price': 11.00}, {'id': 2, 'name': 'ball', 'price': 17.00}]
list3 = [{'id': 1, 'name': 'bat', 'price': 16.00}, {'id': 4, 'name': 'boat', 'price': 10.00}, {'id': 3, 'name': 'brick', 'price': 15.00}]
list4 = [{'id': 1, 'name': 'bat', 'price': 14.00}, {'id': 2, 'name': 'ball', 'price': 9.00}, {'id': 3, 'name': 'brick', 'price': 13.00}]
list...
I want a list of dicts in which the 'id' exists in all of the old lists:
result_list = [{'id': 1, 'name': 'bat}, {'id': 3, 'name': 'brick}]
The values that aren't constant for a given 'id' can be discarded, but the values that are the same for a given 'id' must be in the results list.
If I know how many lists I've got, I can do:
results_list = []
for dict in list1:
if any(dict['id'] == d['id'] for d in list2):
if any(dict['id'] == d['id'] for d in list3):
if any(dict['id'] == d['id'] for d in list4):
results_list.append(dict)
How can I do this if I don't know how many lists I've got?
Put the ids into sets and then take the intersection of the sets.
list1 = [{'id': 1, 'name': 'steve'}, {'id': 2, 'name': 'john'}, {'id': 3, 'name': 'mary'}]
list2 = [{'id': 1, 'name': 'jake'}, {'id': 3, 'name': 'tara'}, {'id': 2, 'name': 'bill'}]
list3 = [{'id': 1, 'name': 'peter'}, {'id': 4, 'name': 'rick'}, {'id': 3, 'name': 'marci'}]
list4 = [{'id': 1, 'name': 'susan'}, {'id': 2, 'name': 'evan'}, {'id': 3, 'name': 'tom'}]
lists = [list1, list2, list3, list4]
sets = [set(x['id'] for x in lst) for lst in lists]
intersection = set.intersection(*sets)
print(intersection)
Result:
{1, 3}
Note that we call the class method set.intersection rather than the instance method set().intersection, since the latter takes intersections of its arguments with the empty set set(), and of course the intersection of anything with the empty set is empty.
If you want to turn this back into a list of dicts, you can do:
result = [{'id': i, 'name': None} for i in intersection]
print(result)
Result:
[{'id': 1, 'name': None}, {'id': 3, 'name': None}]
Now, if you also want to hold onto those attributes which are the same for all instances of a given id, you'll want to do something like this:
list1 = [{'id': 1, 'name': 'bat', 'price': 20.00}, {'id': 2, 'name': 'ball', 'price': 12.00}, {'id': 3, 'name': 'brick', 'price': 19.00}]
list2 = [{'id': 1, 'name': 'bat', 'price': 18.00}, {'id': 3, 'name': 'brick', 'price': 11.00}, {'id': 2, 'name': 'ball', 'price': 17.00}]
list3 = [{'id': 1, 'name': 'bat', 'price': 16.00}, {'id': 4, 'name': 'boat', 'price': 10.00}, {'id': 3, 'name': 'brick', 'price': 15.00}]
list4 = [{'id': 1, 'name': 'bat', 'price': 14.00}, {'id': 2, 'name': 'ball', 'price': 9.00}, {'id': 3, 'name': 'brick', 'price': 13.00}]
lists = [list1, list2, list3, list4]
sets = [set(x['id'] for x in lst) for lst in lists]
intersection = set.intersection(*sets)
all_keys = set(lists[0][0].keys())
result = []
for ident in intersection:
res = [dic for lst in lists
for dic in lst
if dic['id'] == ident]
replicated_keys = []
for key in all_keys:
if len(set(dic[key] for dic in res)) == 1:
replicated_keys.append(key)
result.append({key: res[0][key] for key in replicated_keys})
print(result)
Result:
[{'id': 1, 'name': 'bat'}, {'id': 3, 'name': 'brick'}]
What we do here is:
Look at each id in intersection and grab each dict corresponding to that id.
Find which keys have the same value in all of those dicts (one of which is guaranteed to be id).
Put those key-value pairs into result
This code assumes that:
Each dict in list1, list2, ... will have the same keys. If this assumption is false, let me know - it shouldn't be difficult to relax.