Something is bugging me.
I'm following along with this beginner tutorial for django (cs50) and at some point we receive a string back from a form submission and want to add it to a list:
https://www.youtube.com/watch?v=w8q0C-C1js4&list=PLhQjrBD2T380xvFSUmToMMzERZ3qB5Ueu&t=5777s
def add(request):
if 'tasklist' not in request.session:
request.session['tasklist'] = []
if request.method == 'POST':
form_data = NewTaskForm(request.POST)
if form_data.is_valid():
task = form_data.cleaned_data['task']
request.session['tasklist'] += [task]
return HttpResponseRedirect(reverse('tasks:index'))
I've checked the type of request.session['tasklist']and python shows it's a list.
The task variable is a string.
So why doesn't request.session['tasklist'].append(task) work properly? I can see it being added to the list via some print statements but then it is 'forgotten again' - it doesn't seem to be permanently added to the tasklist.
Why do we use this request.session['tasklist'] += [task] instead?
The only thing I could find is https://ogirardot.wordpress.com/2010/09/17/append-objects-in-request-session-in-django/ but that refers to a site that no longer exists.
The code works fine, but I'm trying to understand why you need to use a different operation and can't / shouldn't use the append method.
Thanks.
The reason why it does not work is because django does not see that you have changed anything in the session by using the append() method on a list that is in the session.
What you are doing here is essentially pulling out the reference to the list and making changes to it without the session backend knowing anything about it. An other way to explain:
The append() method is on the list itself not on the session object
When you call append() on the list you are only talking to the list and the list's parent (the session) has no idea what you guys are doing
When you however do an assignment on the session itself session['whatever'] = 'something' then it knows that something is up and changes are made
So the key here is that you need to operate on the session object directly if you want your changes to be updated automatically
Django only thinks it needs to save a changed session item if the item got reassigned to the session. See here: django session base code the __setitem__ method containing a self.modified = True statement.
The session['list'] += [new_element] adds a new list item (mutates the list stored in the session, so the list reference stays the same) and then gets it reassigned to the session again -> thus triggering first a __getitem__ call -> then your += / __iadd__ runs on the value read -> then a __setitem__ call is made (with the list ref. passed to it). You can see it in the django codebase that it marks the session after each __setitem__ call as modified.
The session['list'] = session['list'] + [new_item] mode of doing the same does create a new list every time it's run so its a bit less efficient, but you should not store hundreds of items in the session anyway. So you're probably fine. This also works exactly as above.
However if you use sub-keys in the session like session['list']['x'] = 'whatever' the session will not see itself as modified so you need to mark it as by request.session.modified = True
Short answer: It's about how Python chooses to implement the dict data structure.
Long answer:
Let's start by saying that request.session is a dictionary.
Quoting Django's documentation, "By default, Django only saves to the session database when the session has been modified – that is if any of its dictionary values have been assigned or deleted". Link
So, the problem is that the session database is not being modified by
request.session['tasklist'].append(task)
Seeing the related parts Django's Session base code (as posted by #Csaba K. in an answer), the variable self.modified is to be set True when setitem dunder method is called.
Now, at this step the problem seems like the setitem dunder method is not being called with request.session['tasklist'].append(task) but with request.session['tasklist'] += [task] it gets called. It is not due to if the reference of request.session['tasklist'] is changing or not as pointed out by another answer, because the reference to the underlying list remains the same.
To confirm, let's create a custom dictionary which extends the Python dict, and print something when setitem dunder method is called.
class MyDict(dict):
def __init__(self, globalVar):
super().__init__()
self.globalVar = globalVar
def __setitem__(self, key, value):
super().__setitem__(key, value)
print("Called Set item when: ", end="")
myDict = MyDict(0)
print("Creating Dict")
print("-----")
myDict["y"] = []
print("Adding a new key-value pair")
print("-----")
myDict["y"] += ["x"]
print(" using +=")
print("-----")
myDict["y"].append("x")
print("append")
print("-----")
myDict["y"].extend(["x"])
print("extend")
print("-----")
myDict["y"] = myDict["y"] + ["x"]
print(" using +",)
print("-----")
It prints:
Creating Dict
-----
Called Set item when: Adding a new key-value pair
-----
Called Set item when: using +=
-----
append
-----
extend
-----
Called Set item when: using +
-----
As we can see, setitem dunder method is called and in turn self.modified is set true only when adding a new key-value pair, or using += or using +, but not when initializing, appending or extending an iterable (in this case a list). Now, the operator + and += do very different things in Python, as explained in the other answer. += behaves more like the append method but in this case, I guess it's more about how Python chooses to implement the dict data structure rather than how +, += and append behave on lists.
I found this while doing some more searching:
https://code.djangoproject.com/wiki/NewbieMistakes
Scroll to 'Appending to a list in session doesn't work'
Again, it is a very dated entry but still seems to hold true.
Not completely satisfied because this does not answer the question as to 'why' this doesn't work, but at the very least confirms 'something's up' and you should probably still use the recommendations there.
(if anyone out there can actually explain this in a more verbose manner then I'd be happy to hear it)
Related
I've seen several answers about how to check if list is empty, but didn't find excatly what i need.
shortly - in python, I need a way to check if list is full, and then empty it, but i need that the check will start just after i fill the list.
I'm define list by call my class - Packet()
class Packet(object):
"""description of class"""
def __init__(self):
self.newPacket = []
newPacket = Packet()
I have menu, which one of the options is to call function in the class to fill the list.
but, if the function get chose again, i need to empty the instance, and start a new one.
I've tried to do that:
if newPacket:
del newPacket
newPacket.MakePacket()
but this don't let me start the list by call the function..
if i disable the
if newPacket:
del newPacket
the function works just fine.
You appear to be confusing a particular Packet instance that you have created and chosen to name newPacket, with its attribute of the same name. Rather than delete the instance, or even delete the list, it sounds like you want to empty the list. Because you've given two different things the same name, the list in question is now accessible from your command-line as newPacket.newPacket (although which the object itself likes to refer to it, in its own methods, as self.newPacket).
So. When you del newPacket, you are removing the reference to the object newPacket from the current workspace. The interpreter will then raise a NameError if you try to do anything with that symbol, such as newPacket.MakePacket() - because that variable no longer exists in the current workspace.
If you want to implement Packet methods that count the items in the self.newPacket list attribute, or empty it, you could say:
class Packet(object):
# ...
def count( self ):
return len( self.newPacket )
def clear( self ):
del self.newPacket[:]
That incidentally illustrates one way of emptying a list, while retaining a reference to the now-empty list: del myList[:]
values = input("Enter a list (U CAN ALSO CREATE AN EMPTY LIST. IF YOU WANT THEN SIMPLY PRESS 'ENTER'). If not then write e.g. ABC or 1, 2 or 2019 to insert: ")
list = list(values)
if list == []:
print("Your list", list, "is an empty list!")
else:
print("Your list", list, "is not an empty list.")
Firstly, it'll take an input from user. Secondly, after giving the input it'll check the given input. Finally, after the checking process completion it'll give a msg to the user (will show a msg if the given input empty or not? Whatever the scenario is).
I am quite surprised at the behavior of this code, which is inside a function:
for user in full_details["users"]:
user = collections.defaultdict(lambda: False, user)
if user["comments"]:
user["comments"] = [comment.__dict__ for comment in user["comments"]]
print("just converted user comments to dict objects")
print(user["comments"])
print("printing full details")
print(full_details)
My understanding was that if I modified a dictionary or a list, that modification applied to the object and would remain. However, when I change user["comments"] for each user in full_details["users"] within my if I am not then seeing those same changes again reflected in full_details just immediately after. Why is that? I thought that whenever you create a new list and assign it to a passed-in parameter, that new list will persist outside the function.
My trouble is that the change made here does not persist:
user["comments"] = [comment.__dict__ for comment in user["comments"]]
Also, full_details is a default_dict:
full_details = collections.defaultdict(lambda: False, thread_details.__dict__)
You are assigning "user" twice. First in the for statement. Then in the first line of the body of the for loop, you are creating a new object and also assigning it to "user". At this point, you have lost your reference to the original object.
This might not be its intended use, but I would like to know how to use a multidimensional manager.list(). I can create on just fine, something like this:
from multiprocessing import manager
test = manager.list(manager.list())
however when ever I try to access the first element of the test list it returns the element's value and not its proxy object
test[0] # returns [] and not the proxy, since I think python is running __getitem__.
Is there anyway for me to get around this and use the manager.list() in this way?
The multiprocessing documentation has a note on this:
Note
Modifications to mutable values or items in dict and list proxies will
not be propagated through the manager, because the proxy has no way of
knowing when its values or items are modified. To modify such an item,
you can re-assign the modified object to the container proxy:
# create a list proxy and append a mutable object (a dictionary)
lproxy = manager.list()
lproxy.append({})
# now mutate the dictionary
d = lproxy[0]
d['a'] = 1
d['b'] = 2
# at this point, the changes to d are not yet synced, but by
# reassigning the dictionary, the proxy is notified of the change
lproxy[0] = d
So, the only way to use a multidimensional list is to actually reassign any changes you make to the second dimension of the list back to the top-level list, so instead of:
test[0][0] = 1
You do:
tmp = test[0]
tmp[0] = 1
test[0] = tmp
Not the most pleasant way to do things, but you can probably write some helper functions to make it a bit more tolerable.
Edit:
It seems the reason that you get a plain list back when you append a ListProxy to another ListProxy is because of how pickling Proxies works. BaseProxy.__reduce__ creates a RebuildProxy object, which what actually is used to unpickle the Proxy. RebuildProxy looks like this:
def RebuildProxy(func, token, serializer, kwds):
'''
Function used for unpickling proxy objects.
If possible the shared object is returned, or otherwise a proxy for it.
'''
server = getattr(process.current_process(), '_manager_server', None)
if server and server.address == token.address:
return server.id_to_obj[token.id][0]
else:
incref = (
kwds.pop('incref', True) and
not getattr(process.current_process(), '_inheriting', False)
)
return func(token, serializer, incref=incref, **kwds)
As the docstring says, if the unpickling is occuring inside a manager server, the actual shared object is created, rather than the Proxy to it. This is probably a bug, and there is actually one filed against this behavior already.
This is something I have been trying to solve for 3 days now and I just can't get my head around it why this is not working.
I have a method that creates a new version of an Object. It used to work, that you would pass in the sou obj. and this would be the source from which a new version is created. You can also pass in a destination, which is not really important in this example. Now I wanted to add locking to this method as we want to add multiple users. So I want to be sure that I always have the most current object from which I create a new one. So I added a line that would just get the newest object. If there is no newer object in the database it would be the same anyway.
def createRevision(request, what, sou, destination=None, ignore = [], **args):
...
if "initial" not in args.keys():
source = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
print "------------"
print source == sou
print "------------"
# This outputs True
else:
source = sou
further down in the method I do something like
source.current = False
source.save()
Basically the idea is that I pass in BaseItem and if I don't specify the "initial" keyword then I get the current item from that project with the same ppk (Which is a special random pk in conduction with current). I do this just to be on the save side, that I really have the most current object. And if it is the initial version I just use that one, as there can not be another version.
So now the problem is, that everything works fine if I use sou in this method. I can save it etc .. but as soon as I use source and initial is not in the args it just doesn't save it. The print statement tells me they are the same. Everything I print after the save tells me it has been saved but it just doesn't do it.
source.current = False
source.save()
print "SAVED !!!!"
print source.pk
print source.current
rofl = get_object_or_404(BaseItem, pk=source.pk, project=sou.project)
print rofl.pk
print source.current
outputs the same pk and the same current value but somehow it is not properly saved. As soon as I look into django admin or do a select current = True.
I really don't know what to do anymore.
Why does it work without a problem if I pass in the object into the method but starts to fail when I get the exact same object in the method?
Of course I call the method with the same object:
x = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
createRevision(request, "", x)
Thank you pztrick for the hint with the caches. I finally solved it. So the problem was that I was doing:
x = get_object_or_404(BaseItem, ppk=sou.ppk, project=sou.project, current=True)
createRevision(request, "", x)
# .... loads of lines of code
unlock(x)
unlock is a method I wrote that just sets a timestamp so I know no other user is editing it. So now the problem was that I was saving x in createRevision with all the correct data but of course unlock(x) still had a reference to an "old" not updated object and of course was saving it again. Hence it was overwriting my changes in createRevision.
Thank you again to everyone who helped with this.
I think you may be running afoul of model manager caching which is intended to limit database queries. However, by invoking the .all() method on the model manager you force it to hit the databse again.
So, try this: Replace your argument from the BaseItem class to the model manager's .all() QuerySet:
source = get_object_or_404(BaseItem.objects.all(), ppk=sou.ppk, project=sou.project, current=True)
# ...
rofl = get_object_or_404(BaseItem.objects.all(), pk=source.pk, project=sou.project)
get_object_or_404 supports mode classes, model managers, or QuerySets as the first parameter so this is valid.
I need magic tool, that helps me to understand where one my problem variable is changed in the code.
I know about perfect tool:
pdb.set_trace()
and I need something similar format, but about only one variable changing history.
For example, my current problem is strange value of context['request'] variable inside Django's tag template definition method. The value is string '<<request>>' and I don't understand where it modified from Django's Request object. I can't debug it, because problem is appearing not so often, but permanently. I only see it in error emails and I can't call it specially. The perfect solution will be to create a log with variable's assignment and any modifications.
I'm not really familiar with django, so your mileage may vary. In general, you can override the __setitem__ method on objects to capture item assignment. However, this doesn't work on dictionaries, only on user-created classes, so first of all it depends on what this context object is.
As I get from a short look at the Django docs, it's indeed not a regular dict, so you can try something like this:
def log_setitem(obj):
class Logged(obj.__class__):
def __setitem__(self, item, val):
print "setting", item, "to", val, "on", self
super(Logged, self).__setitem__(item, val)
obj.__class__ = Logged
d = {}
try:
log_setitem(d) # throws an error
except:
print "doesn't work"
class Dict2(dict):
pass
d2 = Dict2()
log_setitem(d2) # this works
d2["hello"] = "world" # prints the log message before assigning
Even if this works, it of course only works if the assignment actually happens through the "standard" way, i.e. somewhere in the code there's a call like context['request'] = "something".
Might be worth a try, but I can't promise you anything.