Related
I am working on a personal project that analyzes hockey player shot data. One thing that I would like to investigate is the effects of different game-states (5v5, power play, short handed). The problem that I have is that I am not sure how to structure this part of my program.
My initial thought is to define a dictionary with 3600 sub-dictionaries as follows:
game_state = {}
game_state[time] = {}
game_state[time][home] = []
game_state[time][away] = []
I can then use the time for each shot event that I am interested in to lookup each teams' game-state. This, however, seems like an inefficient way of doing things.
As I am writing this question up it occurs to me that most of the game is 5v5 for both teams. Perhaps I could set up a similar dictionary but only use the times that the game-state is not 5v5 to generate the keys, and then when looking up play data assume that no entry means a 5v5 game-state.
My Question: Is there something better suited than a dictionary for this kind of application?
Edit:
To #Karl Knechtel's point, I do not need to save any of this information beyond one iteration of a for loop in my main file. In the main file I loop through game_data (a pickled JSON file) and collect x, y coordinates for all shots to later be binned and plotted. I am trying to refine the shot data to consider only a specific game state by introducing an additional check into my data parsing loop.
This sounds like the perfect use case for a relational database like SQLite or Postgres. Without getting too much into the nitty gritties, you could define a relation called Shot with time as a primary key. This would allow you to also look up more interesting questions like "How many shots are made short handed when it's a powerplay?" You could potentially also have a relation called Game which allows you to know which shots happened in which game.
If you want a less labor intensive solution, I think grouping the data into a class would be good idea. For example,
class Shot:
def __init__(self, time: int, short_hand: bool, num_players: int):
self.time = time
self.short_hand = short_hand
self.num_players = num_players
You could then have a dictionary that maps time to Shot instances.
shots: dict[int, Shot] = {}
shots[100] = Shot(100, False, 10)
shots[150] = Shot(150, True, 9)
...
NB: I highly suggest the first option since it sounds like it will be more useful for your case.
I am making a Django web Application and i am facing a problem in comparing two different lists in different functions
def test(request,slug):
n=request.user
choice=TestOptions.objects.filter(choice=slug).first()
que=questions.objects.filter(Subject=choice)
question=[]
un=['a','b','c','d','e']
for q in que:
if q not in question:
question.append(q)
else:
continue
sampling = random.sample(question, 5)
print("--------------SamPLING111")
print(sampling)
print("--------------SamPLING111")
correctAnswers=[]
for j in sampling:
correctAnswers.append(j.answer)
marks(correctAnswers)
d = dict(zip(un,sampling))
return render(request,"code/test.html",{'questions':d})
def acceptAnswer(request):
answers=[]
if request.method=="POST":
answers.append(request.POST['a'])
answers.append(request.POST['b'])
answers.append(request.POST['c'])
answers.append(request.POST['d'])
answers.append(request.POST['e'])
score(answers)
return render(request,"code/dub.html")
def marks(correct):
list1=[]
l1=correct
def score(and):
list2=[]
l2=ans
function test is passing a list and function acceptAnswer is passing another list my job is to compare those two lists
how can I compare l1 and l2?
I am not 100 percent what you are trying to do with these lists, but in order to compare them I would just return them. Here is a quick example:
def marks(correct):
list1 = []
l1 = correct
return l1
def score(answer):
list2 = []
l2 = answer
return l2
numbers = [1,2,3]
numbers2 = [1,2,3]
numbers3 = [3,4,5]
print(marks(numbers) == score(numbers2)) # True
print(marks(numbers2) == score(numbers3)) # False
Hopefully this helps!
Rather than continue with comments I figured I'd elaborate in an answer though it isn't an exact answer to your question I think it is the real answer.
You really have two issues. One is a design issue ie how to make your program work correctly and the other is an implementation issue about the scope of variables and how to deal with it.
I can see in your profile you're a university student and given the nature of the code it seems very likely you're writing your web app for the purposes of learning maybe even an assignment.
If you were doing this outside of a university I'd expect you were seeking practitioner type skills in which case I'd suggest the answer would be to design your application the way Django expects you to, which I would say would translate into storing state information in a database.
If this is a lab however you may not have covered databases yet. Labs sometimes have students doing silly things because they can't teach everything at the same time. So your Prof may not expect you to use a database yet.
Inside a web application you have to consider that the web is request response and that you can get requests from a lot of different sources so you have state management concerns that classical desktop applications don't have. Who is supposed to see these tests and who is supposed to see the marks and what is the order these things happen? Should anyone be able to create a test? Should anyone be able to take a test? You might not care yet, eventually you'll want to care about sessions. If people are taking their own tests you could store data in a user session but then other people wouldn't see those tests. Generally the correct way to store this sort of state is in a database where you can access it according to what you know about the current request. If this is some sort of basic intro app your Prof may be happy with you doing something kludgy for now.
This appears simple, but I can't find a good solution.
It's the old 'pass by reference'/ 'pass by value' / 'pass by object reference' problem. I understand what is happening, but I can't find a good work around.
I am aware of solutions for small problems, but my state is very large and extremely expensive to save/ recalculate. Given these constraints, I can't find a solution.
Here is some simple pseudocode to illustrate what I would like to do (if Python would let me pass by reference):
class A:
def __init__(self,x):
self.g=x
self.changes=[]
def change_something(self,what,new): # I want to pass 'what' by reference
old=what # and then de-reference it here to read the value
self.changes.append([what,old]) # store a reference
what=new # dereference and change the value
def undo_changes():
for c in self.changes:
c[0]=c[1] # dereference and restore old value
Edit: Adding some more pseudocode to show how I would like the use the above
test=A(1) # initialise test.g as 1
print(test.g)
out: 1
test.change_something(test.g,2)
# if my imaginary code above functioned as described in its comments,
# this would change test.g to 2 and store the old value in the list of changes
print(test.g)
out: 2
test.undo_changes()
print(test.g)
out: 1
Obviously the above code doesnt work in python due to being 'pass by object reference'. Also I'd like to be able to undo a single change, not just all of them as in the code above.
The thing is... I can't seem to find a good work around. There are solutions out there like these:
Do/Undo using command pattern in Python
making undo in python
Which involve storing a stack of commands. 'Undo' then involves removing the last command and then re-building the final state by taking the initial state and re-applying everything but the last command. My state is too large for this to be feasible, the issues are:
The state is very large. Saving it entirely is prohibitively expensive.
'Do' operations are costly (making recalculating from a saved state infeasible).
Do operations are also non-deterministic, relying on random input.
Undo operations are very frequent
I have one idea, which is to ensure that EVERYTHING is stored in lists, and writing my code so that everything is stored, read from and written to these lists. Then in the code above I can pass the list name and list index every time I want to read/write a variable.
Essentially this amounts to building my own memory architecture and C-style pointer system within Python!
This works, but seems a little... ridiculous? Surely there is a better way?
Please check if it helps....
class A:
def __init__(self,x):
self.g=x
self.changes={}
self.changes[str(x)] = {'init':x, 'old':x, 'new':x} #or make a key by your choice(immutable)
def change_something(self,what,new): # I want to pass 'what' by reference
self.changes[what]['new'] = new #add changed value to your dict
what=new # dereference and change the value
def undo_changes():
what = self.changes[what]['old'] #retrieve/changed to the old value
self.changes[what]['new'] = self.changes[what]['old'] #change latest new val to old val as you reverted your changes
for each change you can update the change_dictionary. Onlhy thing you have to figure out is "how to create entry for what as a key in self.change dictionary", I just made it str(x), just check the type(what) and how to make it a key in your case.
Okay so I have come up with an answer... but it's ugly! I doubt it's the best solution. It uses exec() which I am told is bad practice and to be avoided if at all possible. EDIT: see below!
Old code using exec():
class A:
def __init__(self,x):
self.old=0
self.g=x
self.h=x*10
self.changes=[]
def change_something(self,what,new):
whatstr='self.'+what
exec('self.old='+whatstr)
self.changes.append([what,self.old])
exec(whatstr+'=new')
def undo_changes(self):
for c in self.changes:
exec('self.'+c[0]+'=c[1]')
def undo_last_change(self):
c = self.changes[-1]
exec('self.'+c[0]+'=c[1]')
self.changes.pop()
Thanks to barny, here's a much nicer version using getattr and setattr:
class A:
def __init__(self,x):
self.g=x
self.h=x*10
self.changes=[]
def change_something(self,what,new):
self.changes.append([what,getattr(self,what)])
setattr(self,what,new)
def undo_changes(self):
for c in self.changes:
setattr(self,c[0],c[1])
def undo_last_change(self):
c = self.changes[-1]
setattr(self,c[0],c[1])
self.changes.pop()
To demonstrate, the input:
print("demonstrate changing one value")
b=A(1)
print('g=',b.g)
b.change_something('g',2)
print('g=',b.g)
b.undo_changes()
print('g=',b.g)
print("\ndemonstrate changing two values and undoing both")
b.change_something('h',3)
b.change_something('g',4)
print('g=', b.g, 'h=',b.h)
b.undo_changes()
print('g=', b.g, 'h=',b.h)
print("\ndemonstrate changing two values and undoing one")
b.change_something('h',30)
b.change_something('g',40)
print('g=', b.g, 'h=',b.h)
b.undo_last_change()
print('g=', b.g, 'h=',b.h)
returns:
demonstrate changing one value
g= 1
g= 2
g= 1
demonstrate changing two values and undoing both
g= 4 h= 3
g= 1 h= 10
demonstrate changing two values and undoing one
g= 40 h= 30
g= 1 h= 30
EDIT 2: Actually... after further testing, my initial version with exec() has some advantages over the second. If the class contains a second class, or list, or whatever, the exec() version has no trouble updating a list within a class within a class, however the second version will fail.
originally I had this awful piece of code with two for i in. To sum up, the idea is to get a list of AMI's ID of some AMIS with a particular tag. To do so, I iterate over list (wrong!) and I get a list of IDS.
def aws_get_images_exceptions(connection, exceptions):
list_exceptions = []
for ex in exceptions:
images = connection.get_all_images(filters={ "tag-key":"Name","tag-value":ex})
for ami in images:
list_exceptions.append(ami.id)
return list_exceptions
Over each iteration I get the complete list of "exceptions" based on the fixed tag Role that I provide to function and then I append the right AMIS.
Ok, I was able to remove the second for i in with the next code:
def aws_get_images_exceptions(connection, exceptions):
for ex in exceptions:
images = connection.get_all_images(filters={ "tag-key":"Name","tag-value":ex})
list_exceptions = list(map(lambda x: x.id, images)) + list_exceptions
return list_exceptions
I've checked and I get the same amount of AMIS, same IDS, etc. But I don't get the way to do it in a functional way removing the very first for. As you notice I do a manual call to get_all_images because I iterate over a list of "exceptions" which I give to the function.
Do you know how I can remove this first for i in?
Thanks to all!
you don't need to remove for-loops (especially with lambdas) because it is normal and readable enough,
however it will be better to extract images fetching to separate function using generator
def get_images(connection, exceptions):
for ex in exceptions:
images = connection.get_all_images(filters={"tag-key": "Name", "tag-value": ex})
# in Python 3.3+ instead of loop use
# yield from images
for image in images:
yield image
and after that initial function will be
def aws_get_images_exceptions(connection, exceptions):
images = get_images(connection, exceptions)
return [ami.id for ami in images]
and i don't think it can be written fully in functional way since it looks like you are making calls to some sort of database (and this is a side effect)
Further reading
about yield keyword and generators
docs,
PEP-255 where yield keyword was introduced,
epic Stack Overflow thread.
P. S.
get_images can also be re-written without using generator
def get_images(connection, exceptions):
images = []
for ex in exceptions:
images += connection.get_all_images(filters={"tag-key": "Name", "tag-value": ex})
return images
but generators are truly awesome and you will see how powerful they are.
It's certainly possible, simply by nesting fors in your list comprehension:
def aws_get_images_exceptions(connection, exceptions):
list_exceptions = [ami.id for ex in exceptions for ami in connection.get_all_images(filters={ "tag-key":"Name","tag-value":ex})]
return list_exceptions
However, that doesn't mean it's good code. I find your original function much easier to read than my own abomination here.
Functional style doesn't necessarily mean writing a one-line list comprehension to do everything. The essence of what you want is to iterate over a bunch of streams; you have a list of exceptions, each of which produces a list of images to iterate over. You can use natural Python syntax handle this just as cleanly.
def aws_get_images_exceptions(connection, exceptions):
for ex in exceptions:
filters = {"tag-key": "Name", "tag-value": ex}
for ami in connection.get_all_images(filters=filters):
yield ami.id
Don't worry about defining the variable filters; temporary variables are often used (via let expressions) even in pure functional languages. Otherwise, you don't have any real side effects (other than the unavoidable database query).
Notice the only significant change I've made to your code is to yield each id, rather than append it to a list that gets returned. The caller can decide whether to create a list from the resulting generator.
ids = list(aws_get_images_exceptions(connection, exceptions))
On every Tkinter notebook tab, there is a list of checkbuttons and the variables get saved to their corresponding v[ ] (i.e. cb.append(Checkbuttons(.., variables = v[x],..)).
For now, I am encountering this error:
File "/home/pass/OptionsInterface.py", line 27, in __init__
self.ntbk_render(f = self.f1, ntbkLabel="Options",cb = optsCb, msg = optMsg)
File "/home/pass/OptionsInterface.py", line 59, in ntbk_render
text = msg[x][1], command = self.cb_check(v, opt)))
File "/home/pass/OptionsInterface.py", line 46, in cb_check
opt[ix]=(v[ix].get())
IndexError: list assignment index out of range
And I think the error is coming here. I don't know how to access the values of the checkbutton variables.
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
opt[ix]=(v[ix].get())
print opt
Here are some snippets:
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
opt[ix]=(v[ix].get())
print opt
def ntbk_render(self, f=None, ntbkLabel="", cb = [], msg = []):
v = []
opt = []
msg = get_thug_args(word = ntbkLabel, argList = msg) #Allows to get the equivalent list (2d array)
#to serve as texts for their corresponding checkboxes
for x in range(len(msg)):
v.append(IntVar())
off_value = 0
on_value = 1
cb.append(Checkbutton(f, variable = v[x], onvalue = on_value, offvalue = off_value,
text = msg[x][1], command = self.cb_check(v, opt)))
cb[x].grid(row=self.rowTracker + x, column=0, sticky='w')
opt.append(off_value)
cb[-1].deselect()
After solving the error, I want to get all the values of the checkbutton variables of each tab after pressing the button Ok at the bottom. Any tips on how to do it will help!
Alright, so there’s a bit more (… alright, maybe a little more than a bit…) here than I intended, but I’ll leave it on the assumption that you’ll simply take away from it what you need or find of value.
The short answer is that when your Checkbutton calls cb_check, it’s passing the arguments like this:
cb_check(self = self, v = v, cb = opt, opt = [])
I think it’s pretty obvious why you’re getting an IndexError when we write it out like this: you’re using the length of your opt list for indexes to use on the empty list that the function uses when opt is not supplied; in other words, if you have 5 options, the it will try accessing indices [0…4] on empty list [] (obviously, it stops as soon as it fails to access Index 0). Your function doesn’t know that the thing you’re passing it are called v and opt: it simply takes some random references you give it and places them in the order of the positional arguments, filling in keyword arguments in order after that, and then fills out the rest of the keyword arguments with whatever defaults you told it to use.
Semi-Quick Aside:
When trying to fix an error, if I have no idea what went wrong, I would start by inserting a print statement right before it breaks with all the references that are involved in the broken line, which will often tell you what references do not contain the values you thought they had. If this looks fine, then I would step in further and further, checking any lookups/function returns for errors. For example:
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
print(ix, opt, v) ## First check, for sanity’s sake
print(v[ix]) ## Second Check if I still can’t figure it out, but
## this is a lookup, not an assignment, so it
## shouldn’t be the problem
print(v[ix].get()) ## Third Check, again, not an assignment
print(opt[ix]) ## “opt[ix]={something}” is an assignment, so this is
## (logically) where it’s breaking. Here we’re only
## doing a lookup, so we’ll get a normal IndexError
## instead (it won’t say “assignment”)
opt[ix]=(v[ix].get()) ##point in code where IndexError was raised
The simple fix would be to change the Checkbutton command to “lambda: self.cb_check(v,cb,opt)” or more explicitly (so we can do a sanity check) “lambda: self.cb_check(v = v, cb = cb, opt = opt).” (I’ll further mention that you can change “lambda:” to “lambda v = v, cb = cb, opt = opt:” to further ensure that you’ll forever be referring to the same lists, but this should be irrelevant, especially because of changes I’ll suggest below)
[The rest of this is: First Section- an explicit explanation of what your code is doing and a critique of it; second section- an alternative approach to how you have this laid out. As mentioned, the above fixes your problem, so the rest of this is simply an exercise in improvement]
In regards to your reference names-
There’s an old adage “Code is read much more often than it is written,” and part of the Zen of Python says: “Explicit is better than Implicit.[…] Readability counts.” So don’t be afraid to type a little bit more to make it easier to see what’s going on (same logic applies to explicitly passing variables to cb_check in the solution above). v can be varis; cb can be cbuttons; ix would be better (in my opinion) as ind or just plain index; f (in ntkb_render) should probably be parent or master.
Imports-
It looks like you’re either doing star (*) imports for tkinter, or explicitly importing parts of it. I’m going to discourage you from doing either of these things for two reasons. The first is the same reason as above: if only a few extra keystrokes makes it easier to see where everything came from, then it’s worth it in the long run. If you need to go through your code later to find every single tkinter Widget/Var/etc, then simply searching “tk” is a lot easier than searching “Frame” then “Checkbutton” then IntVar and so on. Secondly, imports occasionally clash: so if- for example- you do
import time ## or from time import time, even
from datetime import time
life may get kinda hairy for you. So it would be better to “import tkinter as tk” (for example) than the way you are currently doing it.
cb_check-
There are a few things I’ll point out about this function:
1) v, cb, and opt are all required for the function to work correctly; if the empty list reference is used instead, then it’s going to fail unless you created 0 Checkbuttons (because there wouldn’t be anything to iterate over in the “for loop”; regardless, this doesn’t seem like it should ever happen). What this means is that they’re better off simply being positional arguments (no default value). Had you written them this way, the function would have given you an error stating that you weren’t giving it enough information to work with rather than a semi-arbitrary “IndexError.”
2) Because you supply the function with all the information it needs, there is no practical reason (based on the code supplied, at any rate) as to why the function needs to be a method of some object.
3) This function is being called each time you select a Checkbutton, but reupdates the recorded values (in opt) of all the Checkbuttons (instead of just the one that was selected).
4) The opt list is technically redundant: you already have a reference to a list of all the IntVars (v), which are updated/maintained in real time without you having to do anything; it is basically just as easy to perform v[ix].get() as it is to do opt[ix]: in exchange for the “.get()” call when you eventually need the value you have to include a whole extra function and run it repeatedly to make sure your opt list is up to date. To complicate matters further, there’s an argument for v also being redundant, but we’ll get to that later.
And as an extra note: I’m not sure why you wrapped the integer value of your IntVar (v[ix].get()) with parentheses; they seem extraneous, but I don’t know if you’re trying to cast the value in the same manner as C/Java/etc.
ntbk_render-
Again, notice that this function is given nearly everything it needs to be executed, and therefore feels less like a method than a stand-alone function (at this moment; again, we’ll get to this at the end). The way it’s setup also means that it requires all of that information, so they would better off as positional argument as above.
The cb reference-
Unlike v and opt, the cb reference can be supplied to the function. If we follow cb along its path through the code, we’ll find out that its length must always be equal to v and opt. Assumedly, the reason we may want to pass cb to this method but not v or opt is because we only care about the reference to cb in the rest of our code. However, notice that cb is always an empty iterable with an append method (seems safe to assume it will always be an empty list). So either we should be testing to make sure that it’s empty before we start doing anything with it (because it will break our code if it isn’t), or we should just create it at the same time that we’re creating v and opt. Not knowing how your code is set up, I personally would think it would be easiest to initialize it alongside the other two and then simply return it at the end of the method (putting “return cb” at the end of this function and “cb=[whatever].ntbk_render(f = someframe, ntbklabel = “somethug”, msg = argList)”). Getting back to the redundancy of opt and v (point 4 in cb_check), since we’re keeping all the Checkbuttons around in cb, we can use them to access their IntVars when we need to.
msg-
You pass msg to the function, and then use it to the value of “argList” in get_thug_args and replace it with the result. I think it would make more sense to call the keyword that you pass the ntbk_render “argList” because that’s what it is going to be used for, and then simply let msg be the returned value of get_thug_args. (The same line of thought applies to the keyword “ntbkLabel”, for the record)
Iterating-
I’m not sure if using an Index Reference (x) is just a habit picked up from more rigid programing languages like C and Java, but iterating is probably one of my favorite advantages (subjective, I know) that Python has over those types of languages. Instead of using x, to get your option out of msg, you can simply step through each individual option inside of msg. The only place that we run into insurmountable problems is when we use self.rowTracker (which, on the subject, is not updated in your code; we’ll fix that for now, but as before, we’ll be dealing with that later). What we can do to amend this is utilize the enumerate function built into Python; this will create a tuple containing the current index followed by the value at the iterated index.
Furthermore, because you’re keeping everything in lists, you have to keep going back to the index of the list to get the reference. Instead, simply create references to the things (datatypes/objects) you are creating and then add the references to the lists afterwards.
Below is an adjustment to the code thus far based on most of the things I noted above:
import tkinter as tk ## tk now refers to the instance of the tkinter module that we imported
def ntbk_render(self, parent, word, argList):
cbuttons=list() ## The use of “list()” here is purely personal preference; feel free to
## continue with brackets
msg = get_thug_args(word = word, argList=argList) ## returns a 2d array [ [{some value},
## checkbutton text,…], …]
for x,option in enumerate(msg):
## Each iteration automatically does x=current index, option=msg[current_index]
variable = tk.IntVar()
## off and on values for Checkbuttons are 0 and 1 respectively by default, so it’s
## redundant at the moment to assign them
chbutton=tk.Checkbutton(parent, variable=variable, text=option[1])
chbutton.variable = variable ## rather than carrying the variable references around,
## I’m just going to tack them onto the checkbutton they
## belong to
chbutton.grid(row = self.rowTracker + x, column=0, sticky=’w’)
chbutton.deselect()
cbuttons.append(chbutton)
self.rowTracker += len(msg) ## Updating the rowTracker
return cbuttons
def get_options(self, cbuttons):
## I’m going to keep this new function fairly simple for clarity’s sake
values=[]
for chbutton in cbuttons:
value=chbutton.variable.get() ## It is for this purpose that we made
## chbutton.variable=variable above
values.append(value)
return values
Yes, parts of this are a bit more verbose, but any mistakes in the code are going to be much easier to spot because everything is explicit.
Further Refinement
The last thing I’ll touch on- without going into too much detail because I can’t be sure how much of this was new information for you- is my earlier complaints about how you were passing references around. Now, we already got rid of a lot of complexity by reducing the important parts down to just the list of Checkbuttons (cbuttons), but there are still a few references being passed that we may not need. Rather than dive into a lot more explanation, consider that each of these Notebook Tabs are their own objects and therefore could do their own work: so instead of having your program add options to each tab and carry around all the values to the options, you could relegate that work to the tab itself and then tell it how or what options to add and ask it for its options and values when you need them (instead of doing all that work in the main program).