Related
The below block of code works as intended
patient = Patient()
patient.log = []
patient.id = "xyz"
patient.example_attribute = []
ALL_PATIENT_LOG = []
def update(patient, attribute, argument, replace = False):
now = datetime.now()
entry = {"argument": argument, "attribute": attribute, "time": now.strftime("%H:%M on %d/%m/%y")}
patient.log.append(entry)
But when I add the following to the end of update
entry["patient_id"] = patient.id # we also need to include the patient's ID for the global log
ALL_PATIENT_LOG.append(entry)
if replace:
patient.__dict__[attribute] = [argument]
else:
patient.__dict__[attribute].append(argument)
the patient.log entry is changed such that it also contains the patient id
This violates my understanding of how python works, as I thought that later lines cannot affect how earlier lines are executed. What have I misunderstood which prevents me from understanding why this executes the way it does? How could I change this code to get the desired behaviour? (two different log entries depending on whether its being appended to the patient log or the ALL_PATIENT_LOG)
The line
patient.log.append(entry)
Appends the entry dictionary to the patient.log, it does not "add" the elements of the entry dictionary to the list. So when you in the next line call
entry["patient_id"] = patient.id
You are changing the entry dictionary, and since the patient.log is referencing to that dictionary, it will have the updated information when you look up that list.
A way to solve this, is to create a copy of the dictionary, instead of a reference, also see this post :
patient.log.append(entry.copy())
As that post also mentions, make sure you understand how copy works, as it might not always work as intended. Especially if there are references within the object you copy.
I'm using an unstable python library that's undergoing changes to the API in its various git submodules. Every few weeks, some member or method's location will get changed or renamed. For example, the expression
vk.rinkront[0].asyncamera.printout()
that worked a few weeks back, now has the asyncamera member located in
vk.toplink.enabledata.tx[0].asyncamera[0].print()
I manage to figure out the method's new location by greping, git diffing, ipython autocomplete, and generally bumbling my way around. This process is painful, because the repository has been quite abstracted and the member names observable from the python shell do not necessarily appear in the git submodule code.
Is there a python routine that performs a graph traversal of the object hierarchy while checking for keywords (e.g. print ) ?
(if no, I'll hack some bfs/dfs that checks the child node type before pushing/appending the __dict__, array, dir(). etc... contents onto a queue/stack. But some one must have come across this problem before and come up with a more elegant solution.)
-------EDITS----------
to sum up, is there an existing library such that the following code would work?
import unstable_library
import object_inspecter.search
vk = unstable_library.create() #initializing vk
object_inspecter.search(vk,"print") #searching for lost method
Oy, I feel for you... You probably should just lock down on a version, use it until the API becomes stable, and then upgrade later. That's why complex projects focus on dependency version control.
There is a way, but it's not really standard. The dir() built-in function returns a list of string names of all attributes and methods of a class. With a little bit a wrangling, you could write a script that recursively digs down.
The ultimate issue, though, is that you're going to run into infinite recursive loops when you try to investigate class members. You'll need to add in smarts to either recognize the pattern or limit the depth of your searching.
you can use fairly simple graph traversal to do this
def find_thing(ob,pattern,seen=set(),name=None,curDepth=0,maxDepth=99,isDict=True):
if(ob is None):
return []
if id(ob) in seen:
return []
seen.add(id(ob))
name = str(name or getattr(ob,"__name__",str(ob)))
base_case = check_base_cases(ob,name,pattern)
if base_case is not None:
return base_case
if(curDepth>=maxDepth):
return []
return recursive_step(ob,pattern,name=name,curDepth=curDepth,isDict=isDict)
now just define your two steps (base_case and recursive_step)... something like
def check_base_cases(ob,name,pattern):
if isinstance(ob,str):
if re.match(pattern,ob):
return [ob]
else:
return []
if isinstance(ob,(int,float,long)):
if ob == pattern or str(ob) == pattern:
return [ob]
else:
return []
if isinstance(ob,types.FunctionType):
if re.match(pattern,name):
return [name]
else:
return []
def recursive_step(ob,pattern,name,curDepth,isDict=True):
matches = []
if isinstance(ob,(list,tuple)):
for i,sub_ob in enumerate(ob):
matches.extend(find_thing(sub_ob,pattern,name='%s[%s]'%(name,i),curDepth=curDepth+1))
return matches
if isinstance(ob,dict):
for key,item in ob.items():
if re.match(pattern,str(key)):
matches.append('%s.%s'%(name,key) if not isDict else '%s["%s"]'%(name,key))
else:
matches.extend(find_thing(item,pattern,
name='%s["%s"]'%(name,key) if isDict else '%s.%s'%(name,key),
curDepth=curDepth+1))
return matches
else:
data = dict([(x,getattr(ob,x)) for x in dir(ob) if not x.startswith("_")])
return find_thing(data,pattern,name=name,curDepth=curDepth+1,isDict=False)
finally you can test it like so
print(find_thing(vk,".*print.*"))
I used the following Example
class vk(object):
class enabledata:
class camera:
class object2:
def printX(*args):
pass
asynccamera = [object2]
tx = [camera]
toplink = {'enabledata':enabledata}
running the following
print( find_thing(vk,'.*print.*') )
#['vk.toplink["enabledata"].camera.asynccamera[0].printX']
This appears simple, but I can't find a good solution.
It's the old 'pass by reference'/ 'pass by value' / 'pass by object reference' problem. I understand what is happening, but I can't find a good work around.
I am aware of solutions for small problems, but my state is very large and extremely expensive to save/ recalculate. Given these constraints, I can't find a solution.
Here is some simple pseudocode to illustrate what I would like to do (if Python would let me pass by reference):
class A:
def __init__(self,x):
self.g=x
self.changes=[]
def change_something(self,what,new): # I want to pass 'what' by reference
old=what # and then de-reference it here to read the value
self.changes.append([what,old]) # store a reference
what=new # dereference and change the value
def undo_changes():
for c in self.changes:
c[0]=c[1] # dereference and restore old value
Edit: Adding some more pseudocode to show how I would like the use the above
test=A(1) # initialise test.g as 1
print(test.g)
out: 1
test.change_something(test.g,2)
# if my imaginary code above functioned as described in its comments,
# this would change test.g to 2 and store the old value in the list of changes
print(test.g)
out: 2
test.undo_changes()
print(test.g)
out: 1
Obviously the above code doesnt work in python due to being 'pass by object reference'. Also I'd like to be able to undo a single change, not just all of them as in the code above.
The thing is... I can't seem to find a good work around. There are solutions out there like these:
Do/Undo using command pattern in Python
making undo in python
Which involve storing a stack of commands. 'Undo' then involves removing the last command and then re-building the final state by taking the initial state and re-applying everything but the last command. My state is too large for this to be feasible, the issues are:
The state is very large. Saving it entirely is prohibitively expensive.
'Do' operations are costly (making recalculating from a saved state infeasible).
Do operations are also non-deterministic, relying on random input.
Undo operations are very frequent
I have one idea, which is to ensure that EVERYTHING is stored in lists, and writing my code so that everything is stored, read from and written to these lists. Then in the code above I can pass the list name and list index every time I want to read/write a variable.
Essentially this amounts to building my own memory architecture and C-style pointer system within Python!
This works, but seems a little... ridiculous? Surely there is a better way?
Please check if it helps....
class A:
def __init__(self,x):
self.g=x
self.changes={}
self.changes[str(x)] = {'init':x, 'old':x, 'new':x} #or make a key by your choice(immutable)
def change_something(self,what,new): # I want to pass 'what' by reference
self.changes[what]['new'] = new #add changed value to your dict
what=new # dereference and change the value
def undo_changes():
what = self.changes[what]['old'] #retrieve/changed to the old value
self.changes[what]['new'] = self.changes[what]['old'] #change latest new val to old val as you reverted your changes
for each change you can update the change_dictionary. Onlhy thing you have to figure out is "how to create entry for what as a key in self.change dictionary", I just made it str(x), just check the type(what) and how to make it a key in your case.
Okay so I have come up with an answer... but it's ugly! I doubt it's the best solution. It uses exec() which I am told is bad practice and to be avoided if at all possible. EDIT: see below!
Old code using exec():
class A:
def __init__(self,x):
self.old=0
self.g=x
self.h=x*10
self.changes=[]
def change_something(self,what,new):
whatstr='self.'+what
exec('self.old='+whatstr)
self.changes.append([what,self.old])
exec(whatstr+'=new')
def undo_changes(self):
for c in self.changes:
exec('self.'+c[0]+'=c[1]')
def undo_last_change(self):
c = self.changes[-1]
exec('self.'+c[0]+'=c[1]')
self.changes.pop()
Thanks to barny, here's a much nicer version using getattr and setattr:
class A:
def __init__(self,x):
self.g=x
self.h=x*10
self.changes=[]
def change_something(self,what,new):
self.changes.append([what,getattr(self,what)])
setattr(self,what,new)
def undo_changes(self):
for c in self.changes:
setattr(self,c[0],c[1])
def undo_last_change(self):
c = self.changes[-1]
setattr(self,c[0],c[1])
self.changes.pop()
To demonstrate, the input:
print("demonstrate changing one value")
b=A(1)
print('g=',b.g)
b.change_something('g',2)
print('g=',b.g)
b.undo_changes()
print('g=',b.g)
print("\ndemonstrate changing two values and undoing both")
b.change_something('h',3)
b.change_something('g',4)
print('g=', b.g, 'h=',b.h)
b.undo_changes()
print('g=', b.g, 'h=',b.h)
print("\ndemonstrate changing two values and undoing one")
b.change_something('h',30)
b.change_something('g',40)
print('g=', b.g, 'h=',b.h)
b.undo_last_change()
print('g=', b.g, 'h=',b.h)
returns:
demonstrate changing one value
g= 1
g= 2
g= 1
demonstrate changing two values and undoing both
g= 4 h= 3
g= 1 h= 10
demonstrate changing two values and undoing one
g= 40 h= 30
g= 1 h= 30
EDIT 2: Actually... after further testing, my initial version with exec() has some advantages over the second. If the class contains a second class, or list, or whatever, the exec() version has no trouble updating a list within a class within a class, however the second version will fail.
On every Tkinter notebook tab, there is a list of checkbuttons and the variables get saved to their corresponding v[ ] (i.e. cb.append(Checkbuttons(.., variables = v[x],..)).
For now, I am encountering this error:
File "/home/pass/OptionsInterface.py", line 27, in __init__
self.ntbk_render(f = self.f1, ntbkLabel="Options",cb = optsCb, msg = optMsg)
File "/home/pass/OptionsInterface.py", line 59, in ntbk_render
text = msg[x][1], command = self.cb_check(v, opt)))
File "/home/pass/OptionsInterface.py", line 46, in cb_check
opt[ix]=(v[ix].get())
IndexError: list assignment index out of range
And I think the error is coming here. I don't know how to access the values of the checkbutton variables.
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
opt[ix]=(v[ix].get())
print opt
Here are some snippets:
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
opt[ix]=(v[ix].get())
print opt
def ntbk_render(self, f=None, ntbkLabel="", cb = [], msg = []):
v = []
opt = []
msg = get_thug_args(word = ntbkLabel, argList = msg) #Allows to get the equivalent list (2d array)
#to serve as texts for their corresponding checkboxes
for x in range(len(msg)):
v.append(IntVar())
off_value = 0
on_value = 1
cb.append(Checkbutton(f, variable = v[x], onvalue = on_value, offvalue = off_value,
text = msg[x][1], command = self.cb_check(v, opt)))
cb[x].grid(row=self.rowTracker + x, column=0, sticky='w')
opt.append(off_value)
cb[-1].deselect()
After solving the error, I want to get all the values of the checkbutton variables of each tab after pressing the button Ok at the bottom. Any tips on how to do it will help!
Alright, so there’s a bit more (… alright, maybe a little more than a bit…) here than I intended, but I’ll leave it on the assumption that you’ll simply take away from it what you need or find of value.
The short answer is that when your Checkbutton calls cb_check, it’s passing the arguments like this:
cb_check(self = self, v = v, cb = opt, opt = [])
I think it’s pretty obvious why you’re getting an IndexError when we write it out like this: you’re using the length of your opt list for indexes to use on the empty list that the function uses when opt is not supplied; in other words, if you have 5 options, the it will try accessing indices [0…4] on empty list [] (obviously, it stops as soon as it fails to access Index 0). Your function doesn’t know that the thing you’re passing it are called v and opt: it simply takes some random references you give it and places them in the order of the positional arguments, filling in keyword arguments in order after that, and then fills out the rest of the keyword arguments with whatever defaults you told it to use.
Semi-Quick Aside:
When trying to fix an error, if I have no idea what went wrong, I would start by inserting a print statement right before it breaks with all the references that are involved in the broken line, which will often tell you what references do not contain the values you thought they had. If this looks fine, then I would step in further and further, checking any lookups/function returns for errors. For example:
def cb_check(self, v = [], cb = [], opt = []):
for ix in range(len(cb)):
print(ix, opt, v) ## First check, for sanity’s sake
print(v[ix]) ## Second Check if I still can’t figure it out, but
## this is a lookup, not an assignment, so it
## shouldn’t be the problem
print(v[ix].get()) ## Third Check, again, not an assignment
print(opt[ix]) ## “opt[ix]={something}” is an assignment, so this is
## (logically) where it’s breaking. Here we’re only
## doing a lookup, so we’ll get a normal IndexError
## instead (it won’t say “assignment”)
opt[ix]=(v[ix].get()) ##point in code where IndexError was raised
The simple fix would be to change the Checkbutton command to “lambda: self.cb_check(v,cb,opt)” or more explicitly (so we can do a sanity check) “lambda: self.cb_check(v = v, cb = cb, opt = opt).” (I’ll further mention that you can change “lambda:” to “lambda v = v, cb = cb, opt = opt:” to further ensure that you’ll forever be referring to the same lists, but this should be irrelevant, especially because of changes I’ll suggest below)
[The rest of this is: First Section- an explicit explanation of what your code is doing and a critique of it; second section- an alternative approach to how you have this laid out. As mentioned, the above fixes your problem, so the rest of this is simply an exercise in improvement]
In regards to your reference names-
There’s an old adage “Code is read much more often than it is written,” and part of the Zen of Python says: “Explicit is better than Implicit.[…] Readability counts.” So don’t be afraid to type a little bit more to make it easier to see what’s going on (same logic applies to explicitly passing variables to cb_check in the solution above). v can be varis; cb can be cbuttons; ix would be better (in my opinion) as ind or just plain index; f (in ntkb_render) should probably be parent or master.
Imports-
It looks like you’re either doing star (*) imports for tkinter, or explicitly importing parts of it. I’m going to discourage you from doing either of these things for two reasons. The first is the same reason as above: if only a few extra keystrokes makes it easier to see where everything came from, then it’s worth it in the long run. If you need to go through your code later to find every single tkinter Widget/Var/etc, then simply searching “tk” is a lot easier than searching “Frame” then “Checkbutton” then IntVar and so on. Secondly, imports occasionally clash: so if- for example- you do
import time ## or from time import time, even
from datetime import time
life may get kinda hairy for you. So it would be better to “import tkinter as tk” (for example) than the way you are currently doing it.
cb_check-
There are a few things I’ll point out about this function:
1) v, cb, and opt are all required for the function to work correctly; if the empty list reference is used instead, then it’s going to fail unless you created 0 Checkbuttons (because there wouldn’t be anything to iterate over in the “for loop”; regardless, this doesn’t seem like it should ever happen). What this means is that they’re better off simply being positional arguments (no default value). Had you written them this way, the function would have given you an error stating that you weren’t giving it enough information to work with rather than a semi-arbitrary “IndexError.”
2) Because you supply the function with all the information it needs, there is no practical reason (based on the code supplied, at any rate) as to why the function needs to be a method of some object.
3) This function is being called each time you select a Checkbutton, but reupdates the recorded values (in opt) of all the Checkbuttons (instead of just the one that was selected).
4) The opt list is technically redundant: you already have a reference to a list of all the IntVars (v), which are updated/maintained in real time without you having to do anything; it is basically just as easy to perform v[ix].get() as it is to do opt[ix]: in exchange for the “.get()” call when you eventually need the value you have to include a whole extra function and run it repeatedly to make sure your opt list is up to date. To complicate matters further, there’s an argument for v also being redundant, but we’ll get to that later.
And as an extra note: I’m not sure why you wrapped the integer value of your IntVar (v[ix].get()) with parentheses; they seem extraneous, but I don’t know if you’re trying to cast the value in the same manner as C/Java/etc.
ntbk_render-
Again, notice that this function is given nearly everything it needs to be executed, and therefore feels less like a method than a stand-alone function (at this moment; again, we’ll get to this at the end). The way it’s setup also means that it requires all of that information, so they would better off as positional argument as above.
The cb reference-
Unlike v and opt, the cb reference can be supplied to the function. If we follow cb along its path through the code, we’ll find out that its length must always be equal to v and opt. Assumedly, the reason we may want to pass cb to this method but not v or opt is because we only care about the reference to cb in the rest of our code. However, notice that cb is always an empty iterable with an append method (seems safe to assume it will always be an empty list). So either we should be testing to make sure that it’s empty before we start doing anything with it (because it will break our code if it isn’t), or we should just create it at the same time that we’re creating v and opt. Not knowing how your code is set up, I personally would think it would be easiest to initialize it alongside the other two and then simply return it at the end of the method (putting “return cb” at the end of this function and “cb=[whatever].ntbk_render(f = someframe, ntbklabel = “somethug”, msg = argList)”). Getting back to the redundancy of opt and v (point 4 in cb_check), since we’re keeping all the Checkbuttons around in cb, we can use them to access their IntVars when we need to.
msg-
You pass msg to the function, and then use it to the value of “argList” in get_thug_args and replace it with the result. I think it would make more sense to call the keyword that you pass the ntbk_render “argList” because that’s what it is going to be used for, and then simply let msg be the returned value of get_thug_args. (The same line of thought applies to the keyword “ntbkLabel”, for the record)
Iterating-
I’m not sure if using an Index Reference (x) is just a habit picked up from more rigid programing languages like C and Java, but iterating is probably one of my favorite advantages (subjective, I know) that Python has over those types of languages. Instead of using x, to get your option out of msg, you can simply step through each individual option inside of msg. The only place that we run into insurmountable problems is when we use self.rowTracker (which, on the subject, is not updated in your code; we’ll fix that for now, but as before, we’ll be dealing with that later). What we can do to amend this is utilize the enumerate function built into Python; this will create a tuple containing the current index followed by the value at the iterated index.
Furthermore, because you’re keeping everything in lists, you have to keep going back to the index of the list to get the reference. Instead, simply create references to the things (datatypes/objects) you are creating and then add the references to the lists afterwards.
Below is an adjustment to the code thus far based on most of the things I noted above:
import tkinter as tk ## tk now refers to the instance of the tkinter module that we imported
def ntbk_render(self, parent, word, argList):
cbuttons=list() ## The use of “list()” here is purely personal preference; feel free to
## continue with brackets
msg = get_thug_args(word = word, argList=argList) ## returns a 2d array [ [{some value},
## checkbutton text,…], …]
for x,option in enumerate(msg):
## Each iteration automatically does x=current index, option=msg[current_index]
variable = tk.IntVar()
## off and on values for Checkbuttons are 0 and 1 respectively by default, so it’s
## redundant at the moment to assign them
chbutton=tk.Checkbutton(parent, variable=variable, text=option[1])
chbutton.variable = variable ## rather than carrying the variable references around,
## I’m just going to tack them onto the checkbutton they
## belong to
chbutton.grid(row = self.rowTracker + x, column=0, sticky=’w’)
chbutton.deselect()
cbuttons.append(chbutton)
self.rowTracker += len(msg) ## Updating the rowTracker
return cbuttons
def get_options(self, cbuttons):
## I’m going to keep this new function fairly simple for clarity’s sake
values=[]
for chbutton in cbuttons:
value=chbutton.variable.get() ## It is for this purpose that we made
## chbutton.variable=variable above
values.append(value)
return values
Yes, parts of this are a bit more verbose, but any mistakes in the code are going to be much easier to spot because everything is explicit.
Further Refinement
The last thing I’ll touch on- without going into too much detail because I can’t be sure how much of this was new information for you- is my earlier complaints about how you were passing references around. Now, we already got rid of a lot of complexity by reducing the important parts down to just the list of Checkbuttons (cbuttons), but there are still a few references being passed that we may not need. Rather than dive into a lot more explanation, consider that each of these Notebook Tabs are their own objects and therefore could do their own work: so instead of having your program add options to each tab and carry around all the values to the options, you could relegate that work to the tab itself and then tell it how or what options to add and ask it for its options and values when you need them (instead of doing all that work in the main program).
I've got a situation where a module needs to do some simple, but slightly time consuming initialization. One of the end conditions is a pair of lists which will be filled out by the initialization; what's bugging me is that there's a conflict between the role of the lists - which are basically intended to be constants - and the need to actually initialize them.
I feel uneasy writing code like this:
CONSTANT_LIST = []
DIFFERENT_LIST = []
for item in get_some_data_slowly():
if meets_criteria_one(item):
CONSTANT_LIST.append(item)
continue
if meets_criteria_two(item):
DIFFERENT_LIST.append(item)
Since the casual reader will see those lists in the position usually occupied by constants and may expect them to be empty.
OTOH, I'd be OK with the same under-the-hood facts if I could write this as a list comprehension:
CONSTANT_LIST = [i for i in some_data() if criterion(i)]
and so on... except that I need two lists drawn from the same (slightly time consuming) source, so two list comprehensions will make the code noticeably slower.
To make it worse, the application is such that hiding the constants behind methods:
__private_const_list = None
__other_private_list = None
def public_constant_list():
if __private_const_list: return __private_const_list
# or do the slow thing now and fill out both lists...
# etc
def public_other_const_list():
# same thing
is kind of silly since the likely use frequency is basically 1 per session.
As you can see it's not a rocket science issue but my Python sense is not tingling at all. Whats the appropriate pythonic pattern here?
The loop is quite clear. Don't obfuscate it by being too clever. Just use comments to help explain
CONSTANT_LIST = [] # Put a comment here to tell the reader that these
DIFFERENT_LIST = [] # are constants that are filled in elsewhere
"""
Here is an example of what CONSTANT_LIST looks like
...
Here is an example of what DIFFERENT_LIST looks like
...
"""
for item in get_some_data_slowly():
if meets_criteria_one(item):
CONSTANT_LIST.append(item)
elif meets_criteria_two(item):
DIFFERENT_LIST.append(item)
Maybe use elif instead of continue/if