Related
I want to loop through my data and population my dictonairies with 'event' value and their corresponding 'xCordAdjusted' and 'yCordAdjusted'
Dataframe:
season period teamCode event goal xCord xCordAdjusted yCord yCordAdjusted shotType playerPositionThatDidEvent playerNumThatDidEvent shooterPlayerId shooterName shooterLeftRight
2014 1 MTL MISS 0 61 61 29 29 WRIST C 51 8471976.0 David Desharnais L
2014 1 TOR SHOT 0 -54 54 29 -29 BACK C 42 8475098.0 Tyler Bozak R
2014 1 TOR SHOT 0 -40 40 32 -32 WRIST D 46 8471392.0 Roman Polak R
My work:
league_data = {};
league_data['SHOT'] = {};
league_data['SHOT']['x'] = [];
league_data['SHOT']['y'] = [];
league_data['GOAL'] = {};
league_data['GOAL']['x'] = [];
league_data['GOAL']['y'] = [];
league_data['MISS'] = {};
league_data['MISS']['x'] = [];
league_data['MISS']['y'] = [];
event_types = ['SHOT','GOAL','MISS']
for data in season_df:
for event in event_types:
if data in event_types:
if 'x' in range(0,100):
league_data[event]['x'].append(['xCordAdjusted'])
league_data[event]['y'].append(['yCordAdjusted'])
league_data
Output:
{'SHOT': {'x': [], 'y': []},
'GOAL': {'x': [], 'y': []},
'MISS': {'x': [], 'y': []}}
You can extract the desired information directly from the DataFrame in a vectorized fashion, instead of looping over it repeatedly:
league_data = {
'SHOT': {},
'GOAL': {},
'MISS': {},
}
for event in event_types:
mask = (season_df['event'] == event) & season_df['xCord'].between(0, 100)
x_adjusted = season_df.loc[mask, 'xCordAdjusted'].tolist()
y_adjusted = season_df.loc[mask, 'yCordAdjusted'].tolist()
league_data[event]['x'] = x_adjusted
league_data[event]['y'] = y_adjusted
gives
{'GOAL': {'x': [], 'y': []},
'MISS': {'x': [61], 'y': [-29]},
'SHOT': {'x': [], 'y': []}
}
Note that I adjusted the range condition since your original code if 'x' in range(0,100) doesn't do what you intend because it doesn't reference your DataFrame at all.
for data in season_df: iterate on columns, not rows.
Instead, use for index, row in season_df.iterrows()
However, iteration on rows is quite slow, so if your data is quite big, you can utilize vectorization.
Also, your code looks not working as you expected.. like if 'x' in range(0, 100). I re-code it on my assumption, try this.
for event in event_types:
matched_df = season_df[season_df['event'] == event]
x_matched_list = matched_df[(0 <= matched_df['xCordAdjusted']) & (matched_df['xCordAdjusted'] <= 100)]['xCordAdjusted'].tolist()
league_data[event]['x'] = x_matched_list # or extend
y_matched_list = matched_df[(0 <= matched_df['yCordAdjusted']) & (matched_df['yCordAdjusted'] <= 100)]['yCordAdjusted'].tolist()
league_data[event]['y'] = y_matched_list # or extend
But be careful with possibility of length 'xCordAdjusted' not matching with 'yCordAdjusted'
I'm having some difficulty properly boxing the returned solution to the min_cals <= sum(calories) <= max_cals. There are combinations that yield solutions where sum(cals) < min_cals. In addition to remaining within the caloric range the solution should yield a list where the sum(cost approaches as closely to budget as possible without exceeding the budget limit. Here's some re-contextualized code. I could really use a hand:
menu = [
{'name':'Cheese Pizza Slice', 'calories': 700, 'cost': 4},
{'name':'House Salad', 'calories': 100, 'cost': 8.5},
{'name':'Grilled Shrimp', 'calories': 400, 'cost': 15},
{'name':'Beef Brisket', 'calories': 400, 'cost': 12},
{'name':'Soda', 'calories': 100, 'cost': 1},
{'name':'Cake', 'calories': 300, 'cost': 3},
]
def menu_recommendation(menu, min_cal, max_cal, budget):
menu = [item for item in menu if item['calories'] <= max_cal and item['cost'] <= budget]
if len(menu) == 0: return []
return min((
[item] + menu_recommendation(menu, min_cal - item['calories'], max_cal - item['calories'], budget - item['cost'])
for item in menu
), key=
lambda recommendations: [budget - sum(item['cost'] for item in recommendations) and min_cal <= sum(item['calories'] for item in recommendations) <= max_cal, -sum(item['calories'] for item in recommendations)]
)
recommendation = menu_recommendation(menu, 1000, 1200, 15)
total_cost = sum(item['cost'] for item in recommendation)
total_cals = sum(item['calories'] for item in recommendation)
print(f'recommendation: {recommendation}')
print(f'total cost: {total_cost}')
print(f'total calories: {total_cals}')
for example, the following returns a solution with a total calorie count of 700, which is below the 1000 minimum.
recommendation = menu_recommendation(menu, 1000, 1200, 15)
We can work up something recursive, probably.
def smallest_combo(lst, m, n, z):
# filter list to remove elements we can't use next without breaking the rules
lst = [dct for dct in lst if m <= dct['x'] <= n and dct['y'] <= z]
# recursive base case
if len(lst) == 0:
return []
# go through our list of eligibles
# simulate 'what would the best possibility be if we picked that one to go with next'
# then of those results select the one with the sum('y') closest to z
# (breaking ties with the largest sum('x'))
return min((
[dct] + smallest_combo(lst, m - dct['x'], n - dct['x'], z - dct['y'])
for dct in lst
), key=
lambda com: [z - sum(d['y'] for d in com), -sum(d['x'] for d in com)]
)
inp = [{'name': 'item1', 'x': 600, 'y': 5},
{'name': 'item2', 'x': 200, 'y': 8},
{'name': 'item3', 'x': 500, 'y': 12.5},
{'name': 'item4', 'x': 0, 'y': 1.5},
{'name': 'item5', 'x': 100, 'y': 1}]
print(smallest_combo(inp, 500, 1500, 25))
# [{'name': 'item3', 'x': 500, 'y': 12.5}, {'name': 'item3', 'x': 500, 'y': 12.5}]
There would be a number of ways to speed this up. First by making a recursive cache, and second by taking a dynamic programming approach instead (i.e. start at the bottom instead of at the top).
Here is a dynamic programming solution that builds up a data structure showing all of the (calorie, cost) options we can wind up with along with one item each. We look for the best one meeting the criteria, then find what recommendation that is.
def menu_recommendation(menu, min_cal, max_cal, budget):
# This finds the best possible solution in pseudo-polynomial time.
recommendation_tree = {(0, 0.0): None}
for item in menu:
# This tree will wind up being the old plus new entries from adding this item.
new_recommendation_tree = {}
for key in recommendation_tree.keys():
calories, cost = key
new_recommendation_tree[key] = recommendation_tree[key]
new_key = (calories + item['calories'], cost + item['cost'])
if new_key not in recommendation_tree and new_key[0] <= max_cal:
# This is a new calorie/cost combination to look at.
new_recommendation_tree[new_key] = item
# And now save the work.
recommendation_tree = new_recommendation_tree
# Now we look for the best combination.
best = None
for key in recommendation_tree:
calories, cost = key
# By construction, we know that calories <= max_cal
if min_cal <= calories:
if best is None or abs(budget - cost) < abs(budget - best[1]):
# We improved!
best = key
if best is None:
return None
else:
# We need to follow the tree back to the root to find the recommendation
calories, cost = best
item = recommendation_tree[best]
answer = []
while item is not None:
# This item is part of the menu.
answer.append(item)
# And now find where we were before adding this item.
calories = calories - item['calories']
cost = cost - item['cost']
best = (calories, cost)
item = recommendation_tree[best]
return answer
I came up with this, it's basically a knapsack but it removes recursively dishes from the menu if they are not suitable for the recommendation:
menu = [
{'name':'Cheese Pizza Slice', 'calories': 700, 'cost': 4},
{'name':'House Salad', 'calories': 100, 'cost': 8.5},
{'name':'Grilled Shrimp', 'calories': 400, 'cost': 15},
{'name':'Beef Brisket', 'calories': 400, 'cost': 12},
{'name':'Soda', 'calories': 100, 'cost': 1},
{'name':'Cake', 'calories': 300, 'cost': 3},
]
def get_price(recommendation):
return sum(dish["cost"] for dish in recommendation)
def get_calories(recommendation):
return sum(dish["calories"] for dish in recommendation)
def menu_recommendation(menu, min_cal, max_cal, budget):
sorted_menu = sorted(menu, key=lambda dish: dish["cost"], reverse=True)
recommendation = []
for dish in sorted_menu:
if dish["cost"] + get_price(recommendation) <= budget:
recommendation.append(dish)
if recommendation:
if get_calories(recommendation) < min_cal:
sorted_menu.remove(min(recommendation, key=lambda dish: dish["calories"]/dish["cost"]))
return menu_recommendation(sorted_menu, min_cal, max_cal, budget)
if get_calories(recommendation) > max_cal:
sorted_menu.remove(max(recommendation, key=lambda dish: dish["calories"]/dish["cost"]))
return menu_recommendation(sorted_menu, min_cal, max_cal, budget)
return recommendation
recommendation = menu_recommendation(menu, 500, 800, 15)
total_cost = sum(item['cost'] for item in recommendation)
total_cals = sum(item['calories'] for item in recommendation)
print(f'recommendation: {recommendation}')
print(f'total cost: {total_cost}')
It removes elements according to the calorie/cost rate, because it's the cost to which is applied the knapsack.
Please let me know if you have any question.
I believe I have solved the issue of making recommendations that were outside of the scoped min_cal / max_cal boundaries, but I still feel like there could be a solution that more closely approaches budget.
Here is my updated code:
menu = [
{'name':'Cheese Pizza Slice', 'calories': 700, 'cost': 4},
{'name':'House Salad', 'calories': 100, 'cost': 8.5},
{'name':'Grilled Shrimp', 'calories': 400, 'cost': 15},
{'name':'Beef Brisket', 'calories': 400, 'cost': 12},
{'name':'Soda', 'calories': 100, 'cost': 1},
{'name':'Cake', 'calories': 300, 'cost': 3},
]
def menu_recommendation(menu, min_cal, max_cal, budget):
menu = [item for item in menu if item['calories'] <= max_cal and item['cost'] <= budget]
if len(menu) == 0: return []
return min(([item] + menu_recommendation(menu, min_cal - item['calories'], max_cal - item['calories'], budget - item['cost'])
for item in menu
), key=
lambda recommendations: [budget - sum(item['cost'] for item in recommendations)
and min_cal - sum(item['calories'] for item in recommendations) >= 0
and max_cal - sum(item['calories'] for item in recommendations) >= 0,
-sum(item['calories'] for item in recommendations)]
)
recommendation = menu_recommendation(menu, 1000, 1200, 15)
total_cost = sum(item['cost'] for item in recommendation)
total_cals = sum(item['calories'] for item in recommendation)
print(f'recommendation: {recommendation}')
print(f'total cost: {total_cost}')
print(f'total calories: {total_cals}')
If anyone has any improvements I'd love to hear them!
I'm trying to add on values to a key after making a dictionary.
This is what I have so far:
movie_list = "movies.txt" # using a file that contains this order on first line: Title, year, genre, director, actor
in_file = open(movie_list, 'r')
in_file.readline()
def list_maker(in_file):
movie1 = str(input("Enter in a movie: "))
movie2 = str(input("Enter in another movie: "))
d = {}
for line in in_file:
l = line.split(",")
title_year = (l[0], l[1]) # only then making the tuple ('Title', 'year')
for i in range(4, len(l)):
d = {title_year: l[i]}
if movie1 or movie2 == l[0]:
print(d.values())
The output I get it:
Enter in a movie: 13 B
Enter in another movie: 1920
{('13 B', '(2009)'): 'R. Madhavan'}
{('13 B', '(2009)'): 'Neetu Chandra'}
{('13 B', '(2009)'): 'Poonam Dhillon\n'}
{('1920', '(2008)'): 'Rajneesh Duggal'}
{('1920', '(2008)'): 'Adah Sharma'}
{('1920', '(2008)'): 'Anjori Alagh\n'}
{('1942 A Love Story', '(1994)'): 'Anil Kapoor'}
{('1942 A Love Story', '(1994)'): 'Manisha Koirala'}
{('1942 A Love Story', '(1994)'): 'Jackie Shroff\n'}
.... so on and so forth. I get the whole list of movies.
How would I go about doing so if I wanted to enter in those two movies (any 2 movies as a union of the values to the key (movie1, movie2) )?
Example:
{('13 B', '(2009)'): 'R. Madhavan', 'Neetu Chandra', 'Poonam Dhillon'}
{('1920', '(2008)'): 'Rajneesh Duggal', 'Adah Sharma', 'Anjori Alagh'}
Sorry if the output isn't completely what you want, but here's how you should do it:
d = {}
for line in in_file:
l = line.split(",")
title_year = (l[0], l[1])
people = []
for i in range(4, len(l)):
people.append(l[i]) # we append items to the list...
d = {title_year: people} # ...and then make the dict so that the list is in it.
if movie1 or movie2 == l[0]:
print(d.values())
Basically, what we are doing here is that we are making a list, and then setting the list to a key inside of the dict.
Say I have a file with the following:
/* Full name: abc */
.....
.....(.....)
.....(".....) ;
/* .....
/* .....
..... : "....."
}
"....., .....
Car : true ;
House : true ;
....
....
Age : 33
....
/* Full name: xyz */
....
....
Car : true ;
....
....
Age : 56
....
I am only interested in full name, car, house and age of each person. There are many other lines of data with different format between the variable/attritbute that I am interested.
My code so far:
import re
initial_val = {'House': 'false', 'Car': 'false'}
with open('input.txt') as f:
records = []
current_record = None
for line in f:
if not line.strip():
continue
elif current_record is None:
people_name = re.search('.+Full name ?: (.+) ', line)
if people_name:
current_record = dict(initial_val, Name = people_name.group(1))
else:
continue
elif current_record is not None:
house = re.search(' *(House) ?: ?([a-z]+)', line)
if house:
current_record['House'] = house.group(2)
car = re.search(' *(Car) ?: ?([a-z]+)', line)
if car:
current_record['Car'] = car.group(2)
people_name = re.search('.+Full name ?: (.+) ', line)
if people_name:
records.append(current_record)
current_record = dict(initial_val, Name = people_name.group(1))
print records
What I get:
[{'Name': 'abc', 'House': 'true', 'Car': 'true'}]
My question:
How am I suppose to extract the data and store it in a dictionary like:
{'abc': {'Car': true, 'House': true, 'Age': 33}, 'xyz':{'Car': true, 'House': false, 'Age': 56}}
My purpose:
check whether each person has car, house and age, if no then return false
The I could print them in a table like this:
Name Car House Age
abc true true 33
xyz true false 56
Note that I am using Python 2.7 and I do not know what is the actual value of each variable/attribute (Eg. abc, true, true, 33) of each person.
What is the best solution to my question? Thanks.
Well, you just have to keep track of the current record:
def parse_name(line):
# first remove the initial '/* ' and final ' */'
stripped_line = line.strip('/* ')
return stripped_line.split(':')[-1]
WANTED_KEYS = ('Car', 'Age', 'House')
# default values for when the lines are not present for a record
INITIAL_VAL = {'Car': False, 'House': False, Age: -1}
with open('the_filename') as f:
records = []
current_record = None
for line in f:
if not line.strip():
# skip empty lines
continue
elif current_record is None:
# first record in the file
if line.startswith('/*'):
current_record = dict(INITIAL_VAL, name=parse_name(line))
else:
# this should probably be an error in the file contents
continue
elif line.startswith('/*'):
# this means that the current record finished, and a new one is starting
records.append(current_record)
current_record = dict(INITIAL_VAL, name=parse_name(line))
else:
key, val = line.split(':')
if key.strip() in WANTED_KEYS:
# we want to keep track of this field
current_record[key.strip()] = val.strip()
# otherwise just ignore the line
print('Name\tCar\tHouse\tAge')
for record in records:
print(record['name'], record['Car'], record['House'], record['Age'], sep='\t')
Note that for Age you may want to convert it to an integer using int:
if key == 'Age':
current_record['Age'] = int(val)
The above code produces a list of dictionaries, but it is easy enough to convert it to a dictionary of dicts:
new_records = {r['name']: dict(r) for r in records}
for val in new_records.values():
del val['name']
After this new_records will be something like:
{'abc': {'Car': True, 'House': True, Age: 20}, ...}
If you have other lines with a different format in between the interesting ones you can simply write a function that returns True or False depending on whether the line is in the format you require and use it to filter the lines of the file:
def is_interesting_line(line):
if line.startswith('/*'):
return True
elif ':' in line:
return True
for line in filter(is_interesting_line, f):
# code as before
Change is_interesting_line to suit your needs. In the end, if you have to handle several different formats etc. maybe using a regex would be better, in that case you could do something like:
import re
LINE_REGEX = re.compile(r'(/\*.*\*/)|(\w+\s*:.*)| <other stuff>')
def is_interesting_line(line):
return LINE_REGEX.match(line) is not None
If you want you can obtain fancier formatting for the table, but you probably first need to determine the maximum length of the name etc. or you can use something like tabulate to do that for you.
For example something like (not tested):
max_name_length = max(max(len(r['name']) for r in records), 4)
format_string = '{:<{}}\t{:<{}}\t{}\t{}'
print(format_string.format('Name', max_name_length, 'Car', 5, 'House', 'Age'))
for record in records:
print(format_string.format(record['name'], max_name_length, record['Car'], 5, record['House'], record['Age']))
I have been racking my brain on this for hours now. I'm trying to replace the offense number which is 1-30 to its corresponding offense type i.e. stealing, embezzlement, Burglary, etc. and then sort that into a list.
Here is a sample of the output I currently have:
offense # : Victim Total
1 189
10 712
11 1844
12 184
13 147
14 4364
15 595
16 175
17 387
18 2893
2 597
20 661
Here is what code I have thus far. The offense_map dictionary is what I would like to use to replace the 1-30 in the output to the offense type. Then sort the list in descending order from the largest victim count (right column) to the least. I am working with ~100,000 rows of data so efficiency is important for this program.
from collections import Counter
incidents_f = open('incidents.csv', mode = "r")
crime_dict = dict()
for line in incidents_f:
line_1st = line.strip().split(",")
if line_1st[0].upper() != "REPORT_NO":
report_no = line_1st[0]
offense = line_1st[3]
zip_code = line_1st[4]
if len(zip_code) < 5:
zip_code = "99999"
if report_no in crime_dict:
crime_dict[report_no].append(zip_code).append(offense)
else:
crime_dict[report_no] = [zip_code]+[offense]
#close File
incidents_f.close
details_f = open('details.csv',mode = 'r')
for line in details_f:
line_1st = line.strip().split(",")
if line_1st[0].upper() != "REPORT_NO":
report_no = line_1st[0]
involvement = line_1st[1]
if involvement.upper() != 'VIC':
continue
else:
crime_dict[report_no].append(involvement.upper())
#close File
details_f.close
offense_map = {'1':'Homicide','2':'Rape','3':'Robbery','4':'Assault','5':'Burglary','6':'Stealing','7':'Auto Theft','8':'Non Agg Assault','9':'Arson','10':'Forgery','11':'Fraud','12':'Embezzlement','13':'Stolen Property','14':'Property Damage','15':'Weapons Law Violation','16':'Prostitution','17':'Sex Offense Other','18':'Possession/Sale/Dist','20':'Family Offense','21':'DUI','22':'Liquor Law Violation','24':'Disorderly','25':'Loitering','26':'Misc Violation','29':'Missing/Runaway','30':'Casualty/Suicide'}
victims_by_offense = {}
for k, v in crime_dict.items():
zip = v[1]
if zip not in victims_by_offense.keys():
victims_by_offense[zip] = 0
victims_by_offense[zip] += v[0:].count('VIC')
for zip in sorted(victims_by_offense.keys()):
print(zip, victims_by_offense[zip])
To get a list of keys in victims_by_offense in descending order of Victim Total:
victims_by_offense = {'1': 189, '10': 712, '11': 1844, '12': 184, '13': 147, '14': 4364, '15': 595, '16': 175, '17': 387, '18': 2893, '2': 597, '20': 661}
sorted_keys = sorted(victims_by_offense, key=victims_by_offense.get, reverse=True)
Then
for zip in sorted_keys:
print(offense_map[zip], victims_by_offense[zip])
I get
('Property Damage', 4364)
('Possession/Sale/Dist', 2893)
('Fraud', 1844)
('Forgery', 712)
('Family Offense', 661)
('Rape', 597)
('Weapons Law Violation', 595)
('Sex Offense Other', 387)
('Homicide', 189)
('Embezzlement', 184)
('Prostitution', 175)
('Stolen Property', 147)
('Homicide', 189)
('Embezzlement', 184)
('Prostitution', 175)
('Stolen Property', 147)
I tweaked your code a bit to use csv.reader objects instead of stripping and splitting yourself, as well as changed your data structure to be
crimes = {report_no: {'offense': offense_number,
'zip': zip_code,
'victims': victim_count},
...}
but I think it works much better this way.
import csv
import itemgetter
crimes = dict()
# build `crimes` dict with zero-count victims
with open("incidents.csv") as f:
reader = csv.reader(f)
headers = next(reader)
for report_no, _, _, offense, zip_code, *_ in reader:
if len(zip_code) < 5:
zip_code = "99999"
report = (zip_code, offense)
crimes[report_no] = {'offense': offense,
'zip': zip_code,
'victims': 0}
# parse victims information
with open("details.csv") as f:
reader = csv.reader(f)
headers = next(reader)
for report_no, involvement, *_ in reader:
if involvement.upper() == "VIC":
crimes[report_no]['victims'] += 1
offense_map = {'1':'Homicide',
'2':'Rape',
'3':'Robbery',
'4':'Assault',
'5':'Burglary',
'6':'Stealing',
'7':'Auto Theft',
'8':'Non Agg Assault',
'9':'Arson',
'10':'Forgery',
'11':'Fraud',
'12':'Embezzlement',
'13':'Stolen Property',
'14':'Property Damage',
'15':'Weapons Law Violation',
'16':'Prostitution',
'17':'Sex Offense Other',
'18':'Possession/Sale/Dist',
'20':'Family Offense',
'21':'DUI',
'22':'Liquor Law Violation',
'24':'Disorderly',
'25':'Loitering',
'26':'Misc Violation',
'29':'Missing/Runaway',
'30':'Casualty/Suicide'}
counts = {k: 0 for k in offense_map.values()}
# start counting crimes by victim count (by name, not number)
for crime_info in crimes.values()
try:
offense_no = crime_info['offense']
offense_name = offense_map[offense_no]
counts[offense_name] += crime_info['victims']
except KeyError:
# we couldn't map that
print("No such offense: {}".format(crime_info['offense']))
# sort by value
for k,v in sorted(counts.items(), key=operator.itemgetter(1), reverse=True):
print(k, v)