I'm trying to remove null and empty keys from my python object by calling a method from a module.
from file2 import remove_nulls
# initialize object and set attributes
obj.remove_nulls()
In my remove_nulls() method, if I print the resultant object, I can observe that null and empty keys are removed, but after returning from the function, the final object still has the null and empty keys.
def remove_null(feature):
return json.dumps(del_none(feature.__dict__.copy()))
def del_none(d):
for key, value in list(d.items()):
if value is None:
del d[key]
elif isinstance(value, dict):
del_none(value)
return d
Can someone help to fine where it went wrong?
Just too many copies...:
def remove_null(feature):
return json.dumps(del_none(feature.__dict__.copy()))
that applies del_none on a copy of your object, dumps the proper "cleaned" object, then returns, leaving your object untouched (since you created a copy). You probably need to do just:
def remove_null(feature):
return json.dumps(del_none(feature.__dict__))
The confusion probably comes from the need to copy the dictionary keys+values to avoid removing items on a dict while iterating on it (which was probably handled here, at the wrong level, then handled at the proper level, but the first copy was not removed)
Related
Something is bugging me.
I'm following along with this beginner tutorial for django (cs50) and at some point we receive a string back from a form submission and want to add it to a list:
https://www.youtube.com/watch?v=w8q0C-C1js4&list=PLhQjrBD2T380xvFSUmToMMzERZ3qB5Ueu&t=5777s
def add(request):
if 'tasklist' not in request.session:
request.session['tasklist'] = []
if request.method == 'POST':
form_data = NewTaskForm(request.POST)
if form_data.is_valid():
task = form_data.cleaned_data['task']
request.session['tasklist'] += [task]
return HttpResponseRedirect(reverse('tasks:index'))
I've checked the type of request.session['tasklist']and python shows it's a list.
The task variable is a string.
So why doesn't request.session['tasklist'].append(task) work properly? I can see it being added to the list via some print statements but then it is 'forgotten again' - it doesn't seem to be permanently added to the tasklist.
Why do we use this request.session['tasklist'] += [task] instead?
The only thing I could find is https://ogirardot.wordpress.com/2010/09/17/append-objects-in-request-session-in-django/ but that refers to a site that no longer exists.
The code works fine, but I'm trying to understand why you need to use a different operation and can't / shouldn't use the append method.
Thanks.
The reason why it does not work is because django does not see that you have changed anything in the session by using the append() method on a list that is in the session.
What you are doing here is essentially pulling out the reference to the list and making changes to it without the session backend knowing anything about it. An other way to explain:
The append() method is on the list itself not on the session object
When you call append() on the list you are only talking to the list and the list's parent (the session) has no idea what you guys are doing
When you however do an assignment on the session itself session['whatever'] = 'something' then it knows that something is up and changes are made
So the key here is that you need to operate on the session object directly if you want your changes to be updated automatically
Django only thinks it needs to save a changed session item if the item got reassigned to the session. See here: django session base code the __setitem__ method containing a self.modified = True statement.
The session['list'] += [new_element] adds a new list item (mutates the list stored in the session, so the list reference stays the same) and then gets it reassigned to the session again -> thus triggering first a __getitem__ call -> then your += / __iadd__ runs on the value read -> then a __setitem__ call is made (with the list ref. passed to it). You can see it in the django codebase that it marks the session after each __setitem__ call as modified.
The session['list'] = session['list'] + [new_item] mode of doing the same does create a new list every time it's run so its a bit less efficient, but you should not store hundreds of items in the session anyway. So you're probably fine. This also works exactly as above.
However if you use sub-keys in the session like session['list']['x'] = 'whatever' the session will not see itself as modified so you need to mark it as by request.session.modified = True
Short answer: It's about how Python chooses to implement the dict data structure.
Long answer:
Let's start by saying that request.session is a dictionary.
Quoting Django's documentation, "By default, Django only saves to the session database when the session has been modified – that is if any of its dictionary values have been assigned or deleted". Link
So, the problem is that the session database is not being modified by
request.session['tasklist'].append(task)
Seeing the related parts Django's Session base code (as posted by #Csaba K. in an answer), the variable self.modified is to be set True when setitem dunder method is called.
Now, at this step the problem seems like the setitem dunder method is not being called with request.session['tasklist'].append(task) but with request.session['tasklist'] += [task] it gets called. It is not due to if the reference of request.session['tasklist'] is changing or not as pointed out by another answer, because the reference to the underlying list remains the same.
To confirm, let's create a custom dictionary which extends the Python dict, and print something when setitem dunder method is called.
class MyDict(dict):
def __init__(self, globalVar):
super().__init__()
self.globalVar = globalVar
def __setitem__(self, key, value):
super().__setitem__(key, value)
print("Called Set item when: ", end="")
myDict = MyDict(0)
print("Creating Dict")
print("-----")
myDict["y"] = []
print("Adding a new key-value pair")
print("-----")
myDict["y"] += ["x"]
print(" using +=")
print("-----")
myDict["y"].append("x")
print("append")
print("-----")
myDict["y"].extend(["x"])
print("extend")
print("-----")
myDict["y"] = myDict["y"] + ["x"]
print(" using +",)
print("-----")
It prints:
Creating Dict
-----
Called Set item when: Adding a new key-value pair
-----
Called Set item when: using +=
-----
append
-----
extend
-----
Called Set item when: using +
-----
As we can see, setitem dunder method is called and in turn self.modified is set true only when adding a new key-value pair, or using += or using +, but not when initializing, appending or extending an iterable (in this case a list). Now, the operator + and += do very different things in Python, as explained in the other answer. += behaves more like the append method but in this case, I guess it's more about how Python chooses to implement the dict data structure rather than how +, += and append behave on lists.
I found this while doing some more searching:
https://code.djangoproject.com/wiki/NewbieMistakes
Scroll to 'Appending to a list in session doesn't work'
Again, it is a very dated entry but still seems to hold true.
Not completely satisfied because this does not answer the question as to 'why' this doesn't work, but at the very least confirms 'something's up' and you should probably still use the recommendations there.
(if anyone out there can actually explain this in a more verbose manner then I'd be happy to hear it)
I have a view that simply checks for a key in the session store and if its present it will delete it and if its not present it should pass, it may be worth noting that the key store is holding ids of model instances.
def RefreshInstances(request):
if request.session['instances']:
del request.session['instances']
else:
pass
return redirect('/')
This works and achieves its goal deleting the instances, however if the key store is empty I get a key error rather than the code just being passed?
Can anyone shed any light?
Thanks in advance.
Accessing keys of dicts (or dictlikes) that don't exist do raise KeyError.
You can explicitly check for the key:
if 'instances' in request.session:
# ...
or you can use the .get() method, which returns a default value (None by default) if the key does not exist – this is also handy because falsy values such as 0, '', False, [] etc. pass the test:
if request.session.get('instances'):
del request.session['instances']
... but for deletion, simply use .pop() with a default and without an if:
request.session.pop('instances', None) # Remove `instances` if it's there, do nothing otherwise.
This can be condensed into a single line:
request.session.pop('instances', None)
If you use request.session['instances'], you perform a lookup. If the key (here 'instances') is not available, it will raise a KeyError. The error is thus thrown before the expression's truthiness is evaluated by the if statement.
It is however better to just use .pop(..) here:
request.session.pop('instances', None)
This will remove the key if it is available, and otherwise do nothing. It will return the value that was associated with the 'instances' key, given such value exists, and return None otherwise.
I'm working with nested dictionaries on Python (2.7) obtained from YAML objects and I have a couple of questions that I've been trying to get an answer to by reading, but have not been successful. I'm somewhat new to Python.
One of the simplest functions is one that reads the whole dictionary and outputs a list of all the keys that exist in it. I use an underscore at the beginning since this function is later used by others within a class.
class Myclass(object):
#staticmethod
def _get_key_list(d,keylist):
for key,value in d.iteritems():
keylist.append(key)
if isinstance(value,dict):
Myclass._get_key_list(d.get(key),keylist)
return list(set(keylist))
def diff(self,dict2):
keylist = []
all_keys1 = self._get_key_list(self.d,keylist)
all_keys2 = self._get_key_list(dict2,keylist)
... # More code
Question 1: Is this a correct way to do this? I am not sure whether it's good practice to use a static method for this reason. Since self._get_key_list(d,keylist) is recursive, I dont want "self" to be the first argument once the function is recursively called, which is what would happen for a regular instance method.
I have a bunch of static methods that I'm using, but I've read in a lot of places thay they could perhaps not be good practice when used a lot. I also thought I could make them module functions, but I wanted them to be tied to the class.
Question 2: Instead of passing the argument keylist to self._get_key_list(d,keylist), how can I initialize an empty list inside the recursive function and update it? Initializing it inside would reset it to [] every time.
I would eliminate keylist as an explicit argument:
def _get_keys(d):
keyset = set()
for key, value in d.iteritems():
keylist.add(key)
if isinstance(value, dict):
keylist.update(_get_key_list(value))
return keyset
Let the caller convert the set to a list if they really need a list, rather than an iterable.
Often, there is little reason to declare something as a static method rather than a function outside the class.
If you are concerned about efficiency (e.g., getting lots of repeat keys from a dict), you can go back to threading a single set/list through the calls as an explicit argument, but don't make it optional; just require that the initial caller supply the set/list to update. To emphasize that the second argument will be mutated, just return None when the function returns.
def _get_keys(d, result):
for key, value in d.iteritems():
result.add(key)
if isinstance(value, dict):
_get_keys(value, result)
result = set()
_get_keys(d1, result)
_get_keys(d2, result)
# etc
There's no good reason to make a recursive function in a class a static method unless it is meant to be invoked outside the context of an instance.
To initialize a parameter, we usually assign to it a default value in the parameter list, but in case it needs to be a mutable object such as an empty list in this case, you need to default it to None and the initialize it inside the function, so that the list reference won't get reused in the next call:
class Myclass(object):
def _get_key_list(self, d, keylist=None):
if keylist is None:
keylist = []
for key, value in d.iteritems():
keylist.append(key)
if isinstance(value, dict):
self._get_key_list(d.get(key), keylist)
return list(set(keylist))
def diff(self, dict2):
all_keys1 = self._get_key_list(self.d)
all_keys2 = self._get_key_list(dict2)
... # More code
I have a dictionary defined as
BREED_CLASS = {
'bdcl': ['Border Collie', BreedBdcl, 'BreedBdcl'],
}
and a model defined as
class Dog(models.Model):
breed_code = models.CharField(max_length=4, default='')
I'm trying to use the breed_code as a key to access the items in the dictionary, for example like this
return BREED_CLASS[instance.breed_code][0]
but that raises a KeyError, with exception value '' aka the default value.
I've tried doing something like
bc = instance.breed_code
and that correctly returns the value saved to that instantiated object ('bdcl'). But when I put that into the dictionary access as so
return BREED_CLASS[bc][0]
it gets me the same exact KeyError with the same exact exception value. How do I figure this out?
The CharField class seems to be using the __str__ magic method to return a string when you print it, giving you the illusion it is a string but it's actually an object. Your dictionary uses the actual string for storing the value but when you use BREED_CLASS[instance.breed_code] you are passing it the object. Converting your object to a string should fix it. Try this:
BREED_CLASS[str(instance.breed_code)][0]
So I figured out a workaround that gets me what I want, but I have no idea why it works.
This by itself does not work
return BREED_CLASS[instance.breed_code][2]
But having this for loop that iterates through the keys does
for key in BREED_CLASS:
if key == instance.breed_code:
return BREED_CLASS[instance.breed_code][2]
Notice that both return calls are identical, with instance.breed_code being used for the dict key index.
Does anyone have an answer as for why this is?
I have a dictionary of methods in Python that is inside of a definition. This def is called outside of the class using an object. Whenever the def is called I am returned the results of the last item in the dictionary. In this case I am returned the results of def spc_summary:.
def setStyles(self):
# Assign function to file
functions = {
'app_server.php':self.app_server(),
'dcrm2nlyte.php':self.dcrm2nlyte(),
'export.php':self.export(),
'host_server.php':self.host_server(),
'spc.php':self.spc(),
'spc_approved.php':self.spc_approved(),
'spc_approved_by_dc.php':self.spc_approved_by_dc(),
'spc_by_dc.php':self.spc_by_dc(),
'spc_complete.php':self.spc_complete(),
'spc_summary.php':self.spc_summary()
}
filename = self.phpfile
functions.get(filename)
Can someone please explain what is happening here? Let me know if more detail is required. Thanks!
Let me add some detail:
The functions.get(filename) is retreiving the last dictionary item reguardless of what filename is. I have done this => functions('spc.php') and it still returned results for `def spc_summary'. And those def's should not have the same results.
Your function dict seems to be doing the wrong thing. While defining your dict you are mapping the keys to the function result instead of the function object. If you map it to the function object the function gets called when you invoke functions.get(filename)()
Your dict should probably be like below:
functions = {
'app_server.php':self.app_server,
'dcrm2nlyte.php':self.dcrm2nlyte,
'export.php':self.export,
'host_server.php':self.host_server,
'spc.php':self.spc,
'spc_approved.php':self.spc_approved,
'spc_approved_by_dc.php':self.spc_approved_by_dc,
'spc_by_dc.php':self.spc_by_dc,
'spc_complete.php':self.spc_complete,
'spc_summary.php':self.spc_summary
}
Dictionaries are unordered, so the last object returned from iterating over a dict will probably not be the last item inserted into that dict.
functions.get(filename) is going to take the current value of filename and look it up in functions. filename gets its value from self.phpfile, so in your example self.phpfile must be set to 'spc_summary.php'.