The way to limit the returned set of results from a Django queryset is done via an array slice. For example, to get the first 5 people:
People.objects.all()[0:5]
Or, to get them ordered by name:
People.objects.order_by(name)[0:5]
Or ordered by name, but only those over 65:
People.objects.order_by(name).filter(age__gt=65)[0:5]
In fact the only activity I can think of on a query set that doesn't have a function is limiting.
What I'd like to know is, it there a method (internal, documented or otherwise) that can be called on a QuerySet that acts as a limit or slice?
If not, what is the best way about doing this?
Notes:
Yes, this is probably a bad idea, no I'm not super keen on implementing it, but if there were a good reason for it to be done, could it be?
Yes, I'm aware slices are executed lazily, that's not what I'm asking.
This is not a duplicate of this question, as the accepted answer says:
Do results[:max_count] in a view, after .order_by().
Reviewing the code of Django Querysets, its not as blackboxy as it seemed
def __getitem__(self, k):
"""
Retrieves an item or slice from the set of results.
"""
# ... trimmed ...
if isinstance(k, slice):
qs = self._clone()
if k.start is not None:
start = int(k.start)
else:
start = None
if k.stop is not None:
stop = int(k.stop)
else:
stop = None
qs.query.set_limits(start, stop)
return list(qs)[::k.step] if k.step else qs
qs = self._clone()
qs.query.set_limits(k, k + 1)
return list(qs)[0]
The key line is here:
qs.query.set_limits(start, stop)
The reason the slice is lazy is because it just takes the start and stop values and passes them to another method.
Which corresponds to a call to the sql.Query object here:
def set_limits(self, low=None, high=None):
So it is possible (although probably not recommended) to slice a Queryset like so:
people = People.objects.order_by(name).filter(age__gt=65) # unevaluated
people.query.set_limits(start, stop) # still unevaluated
for person in people: # now its evaluated
person.do_the_thing()
Related
I have a series of functions in a module which are starting to become quite repetitive. Each function extracts a list, and has an optional boolean argument to sort the list before returning it. Feels like there ought to be a way to inherit the sorting from a parent function?
def get_electrical_equipment(sort_by_name = False):
elements = DB.FilteredElementCollector(revit.doc)\
.OfCategory(DB.BuiltInCategory.OST_ElectricalEquipment)\
.WhereElementIsNotElementType()\
.ToElements()
if sort_by_name: elements.sort(key=lambda x: x.Name)
return elements
def get_panel_schedules(sort_by_name = False):
elements = DB.FilteredElementCollector(revit.doc)\
.WherePasses(DB.ElementClassFilter(DB.Electrical.PanelScheduleView))\
.WhereElementIsNotElementType()\
.ToElements()
if sort_by_name: elements.sort(key=lambda x: x.Name)
return elements
def get_panel_schedule_sheet_instances(sort_by_name = False):
elements = DB.FilteredElementCollector(revit.doc)\
.OfClass(DB.Electrical.PanelScheduleSheetInstance)\
.ToElements()
if sort_by_name: elements.sort(key=lambda x: x.Name)
return elements
First of all, I think you can completely eliminate the call to ToElements. It is a waste of memory and computation time, as I have pointed out about 500 times in the past in the Revit API discussion forum and in The Building Coder, e.g., in How to Distinguish Redundant Rooms. Now, to address your question, you can simply implement a common method get_elements_of_category_and_class taking a category and a class argument. Pass in either one or the other or both and execute OfClass and OfCategory checks on the filtered element collector, either one or the other or both, skipping evaluation of null-valued arguments.
I have a custom class,
class A:
def __init__(self, a, b):
self.a = a
self.b = b
The class is not iterable or indexable or anything like that. If at all possible, I would like to keep it that way. Is it possible to have something like the following work?
>>> x = A(1, 2)
>>> min(x)
1
>>> max(x)
2
What got me thinking about this is that min and max are listed as "Common Sequence Operations" in the docs. Since range is considered to be a sequence type by the very same docs, I was thinking that there must be some sort of optimization that is possible for range, and that perhaps I could take advantage of it.
Perhaps there is a magic method that I am not aware of that would enable this?
Yes. When min takes one arguments it assumes it to be an iterable, iterates over it and takes the minimum value. So,
class A:
def __init__(self, a, b):
self.a = a
self.b = b
def __iter__(self):
yield self.a
yield self.b
Should work.
Additional Note: If you don't want to use __iter__, I don't know of way to do that. You probably want to create your own min function, that calls some _min_ method if there is one in the argument it is passed to and calls the old min else.
oldmin = min
def min(*args):
if len(args) == 1 and hasattr(args[0], '_min_'):
return args[0]._min_()
else:
return oldmin(*args)
There are no __min__ and __max__ special methods*. This is kind of a shame since range has seen some pretty nice optimizations in Python 3. You can do this:
>>> 1000000000000 in range(1000000000000)
False
But don't try this unless you want to wait a long time:
>>> max(range(1000000000000))
However creating your own min/max functions is a pretty good idea, as suggested by Lærne.
Here is how I would do it. UPDATE: removed the dunder name __min__ in favor of _min, as recommended by PEP 8:
Never invent such names; only use them as documented
Code:
from functools import wraps
oldmin = min
#wraps(oldmin)
def min(*args, **kwargs)
try:
v = oldmin(*args, **kwargs)
except Exception as err:
err = err
try:
arg, = args
v = arg._min()
except (AttributeError, ValueError):
raise err
try:
return v
except NameError:
raise ValueError('Something weird happened.')
I think this way is maybe a little bit better because it handles some corner cases the other answer hasn't considered.
Note that an iterable object with a _min method will still be consumed by oldmin as per usual, but the return value is overridden by the special method.
HOWEVER, if the _min method requires the iterator to still be available for consumption, this will need to be tweaked because the iterator is getting consumed by oldmin first.
Note also that if the __min method is simply implemented by calling oldmin, things will still work fine (even though the iterator was consumed; this is because oldmin raises a ValueError in this case).
* Such methods are often called "magic", but this is not the preferred terminology.
Since range is considered to be a sequence type by the very same docs, I was thinking that there must be some sort of optimization that is possible for range, and that perhaps I could take advantage of it.
There's no optimization going on for ranges and there are no specialized magic methods for min/max.
If you peek at the implementation for min/max you'll see that after some argument parsing is done, a call to iter(obj) (i.e obj.__iter__()) is made to grab an iterator:
it = PyObject_GetIter(v);
if (it == NULL) {
return NULL;
}
then calls to next(it) (i.e it.__next__) are performed in a loop to grab values for comparisons:
while (( item = PyIter_Next(it) )) {
/* Find min/max */
Is it possible to have something like the following work?
No, if you want to use the built-in min* the only option you have is implementing the iterator protocol.
*By patching min, you can of-course, make it do anything you want. Obviously at the cost of operating in Pythonland. If, though, you think you can utilize some optimizations, I'd suggest you create a min method rather than re-defining the built-in min.
In addition, if you only have ints as instance variables and you don't mind a different call, you can always use vars to grab the instance.__dict__ and then supply it's .values() to min:
>>> x = A(20, 4)
>>> min(vars(x).values())
4
On Codewars.com I encountered the following task:
Create a function add that adds numbers together when called in succession. So add(1) should return 1, add(1)(2) should return 1+2, ...
While I'm familiar with the basics of Python, I've never encountered a function that is able to be called in such succession, i.e. a function f(x) that can be called as f(x)(y)(z).... Thus far, I'm not even sure how to interpret this notation.
As a mathematician, I'd suspect that f(x)(y) is a function that assigns to every x a function g_{x} and then returns g_{x}(y) and likewise for f(x)(y)(z).
Should this interpretation be correct, Python would allow me to dynamically create functions which seems very interesting to me. I've searched the web for the past hour, but wasn't able to find a lead in the right direction. Since I don't know how this programming concept is called, however, this may not be too surprising.
How do you call this concept and where can I read more about it?
I don't know whether this is function chaining as much as it's callable chaining, but, since functions are callables I guess there's no harm done. Either way, there's two ways I can think of doing this:
Sub-classing int and defining __call__:
The first way would be with a custom int subclass that defines __call__ which returns a new instance of itself with the updated value:
class CustomInt(int):
def __call__(self, v):
return CustomInt(self + v)
Function add can now be defined to return a CustomInt instance, which, as a callable that returns an updated value of itself, can be called in succession:
>>> def add(v):
... return CustomInt(v)
>>> add(1)
1
>>> add(1)(2)
3
>>> add(1)(2)(3)(44) # and so on..
50
In addition, as an int subclass, the returned value retains the __repr__ and __str__ behavior of ints. For more complex operations though, you should define other dunders appropriately.
As #Caridorc noted in a comment, add could also be simply written as:
add = CustomInt
Renaming the class to add instead of CustomInt also works similarly.
Define a closure, requires extra call to yield value:
The only other way I can think of involves a nested function that requires an extra empty argument call in order to return the result. I'm not using nonlocal and opt for attaching attributes to the function objects to make it portable between Pythons:
def add(v):
def _inner_adder(val=None):
"""
if val is None we return _inner_adder.v
else we increment and return ourselves
"""
if val is None:
return _inner_adder.v
_inner_adder.v += val
return _inner_adder
_inner_adder.v = v # save value
return _inner_adder
This continuously returns itself (_inner_adder) which, if a val is supplied, increments it (_inner_adder += val) and if not, returns the value as it is. Like I mentioned, it requires an extra () call in order to return the incremented value:
>>> add(1)(2)()
3
>>> add(1)(2)(3)() # and so on..
6
You can hate me, but here is a one-liner :)
add = lambda v: type("", (int,), {"__call__": lambda self, v: self.__class__(self + v)})(v)
Edit: Ok, how this works? The code is identical to answer of #Jim, but everything happens on a single line.
type can be used to construct new types: type(name, bases, dict) -> a new type. For name we provide empty string, as name is not really needed in this case. For bases (tuple) we provide an (int,), which is identical to inheriting int. dict are the class attributes, where we attach the __call__ lambda.
self.__class__(self + v) is identical to return CustomInt(self + v)
The new type is constructed and returned within the outer lambda.
If you want to define a function to be called multiple times, first you need to return a callable object each time (for example a function) otherwise you have to create your own object by defining a __call__ attribute, in order for it to be callable.
The next point is that you need to preserve all the arguments, which in this case means you might want to use Coroutines or a recursive function. But note that Coroutines are much more optimized/flexible than recursive functions, specially for such tasks.
Here is a sample function using Coroutines, that preserves the latest state of itself. Note that it can't be called multiple times since the return value is an integer which is not callable, but you might think about turning this into your expected object ;-).
def add():
current = yield
while True:
value = yield current
current = value + current
it = add()
next(it)
print(it.send(10))
print(it.send(2))
print(it.send(4))
10
12
16
Simply:
class add(int):
def __call__(self, n):
return add(self + n)
If you are willing to accept an additional () in order to retrieve the result you can use functools.partial:
from functools import partial
def add(*args, result=0):
return partial(add, result=sum(args)+result) if args else result
For example:
>>> add(1)
functools.partial(<function add at 0x7ffbcf3ff430>, result=1)
>>> add(1)(2)
functools.partial(<function add at 0x7ffbcf3ff430>, result=3)
>>> add(1)(2)()
3
This also allows specifying multiple numbers at once:
>>> add(1, 2, 3)(4, 5)(6)()
21
If you want to restrict it to a single number you can do the following:
def add(x=None, *, result=0):
return partial(add, result=x+result) if x is not None else result
If you want add(x)(y)(z) to readily return the result and be further callable then sub-classing int is the way to go.
The pythonic way to do this would be to use dynamic arguments:
def add(*args):
return sum(args)
This is not the answer you're looking for, and you may know this, but I thought I would give it anyway because if someone was wondering about doing this not out of curiosity but for work. They should probably have the "right thing to do" answer.
New python users often get tripped up by mutable argument defaults. What are the gotchas and other issues of using this 'feature' on purpose, for example, to get tweakable defaults at runtime that continue to display properly in function signatures via help()?
class MutableString (str):
def __init__ (self, value):
self.value = value
def __str__ (self):
return self.value
def __repr__ (self):
return "'" + self.value + "'"
defaultAnimal = MutableString('elephant')
def getAnimal (species=defaultAnimal):
'Return the given animal, or the mutable default.'
return species
And in use:
>>> help(getAnimal)
getAnimal(species='elephant')
Return the given animal, or the mutable default.
>>> print getAnimal()
elephant
>>> defaultAnimal.value = 'kangaroo'
>>> help(getAnimal)
getAnimal(species='kangaroo')
Return the given animal, or the mutable default.
>>> print getAnimal()
kangaroo
First, read Why are default values shared between objects. That doesn't answer your question, but it provides some background.
There are different valid uses for this feature, but they pretty much all share something in common: the default value is a transparent, simple, obviously-mutable, built-in type. Memoization caches, accumulators for recursive calls, optional output variables, etc. all look like this. So, experienced Python developers will usually spot one of these use cases—if they see memocache={} or accum=[], they'll know what to expect. But your code will not look like a use for mutable default values at all, which will be as misleading to experts as it is to novices.
Another problem is that your function looks like it's returning a string, but it's lying:
>>> print getAnimal()
kangaroo
>>> print getAnimal()[0]
e
Of course the problem here is that you've implemented MutableString wrong, not that it's impossible to implement… but still, this should show why trying to "trick" the interpreter and your users tends to open the door to unexpected bugs.
--
The obvious way to handle it is to store the changing default in a module, function, or (if it's a method) instance attribute, and use None as a default value. Or, if None is a valid value, use some other sentinel:
defaultAnimal = 'elephant'
def getAnimal (species=None):
if species is None:
return defaultAnimal
return species
Note that this is pretty much exactly what the FAQ suggests. Even if you inherently have a mutable value, you should do this dance to get around the problem. So you definitely shouldn't create a mutable value out of an inherently immutable one to create the problem.
Yes, this means that help(getAnimal) doesn't show the current default. But nobody will expect it to.
They will probably expect you to tell them that the default value is a hook, of course, but that's a job for a docstring:
defaultAnimal = 'elephant'
def getAnimal (species=None):
"""getAnimal([species]) -> species
If the optional species parameter is left off, a default animal will be
returned. Normally this is 'elephant', but this can be configured by setting
foo.defaultAnimal to another value.
"""
if species is None:
return defaultAnimal
return species
The only useful use I've seen for it is as a cache:
def fibo(n, cache={}):
if n < 2:
return 1
else:
if n in cache:
return cache[n]
else:
fibo_n = fibo(n-1) + fibo(n-2) # you can still hit maximum recursion depth
cache[n] = fibo_n
return fibo_n
...but then it's cleaner to use the #lru_cache decorator.
#lru_cache
def fibo(n):
if n < 2:
return 1
else:
return fibo(n-1) + fibo(n-2)
I'm implementing some basic data structures in preparation for an exam and have come across the following issue. I want to implement an unsorted linked list, and have already implemented a pop() method, however I don't know, either syntactically or conceptually, how to make a function sometimes take an argument, sometimes not take an argument. I hope that makes sense.
def pop(self):
current = self.head
found = False
endOfList = None
while current != None and not found:
if current.getNext() == None:
found = True
endOfList = current.getData()
self.remove(endOfList)
self.count = self.count - 1
else:
current = current.getNext()
return endOfList
I want to know how to make the statement unsortedList.pop(3) valid, 3 being just an example and unsortedList being a new instance of the class.
The basic syntax (and a common use case) for using a parameter with a default value looks like this:
def pop(self, index=None):
if index is not None:
#Do whatever your default behaviour should be
You then just have to identify how you want your behaviour to change based on the argument. I am just guessing that the argument should specify the index of the element that should be pop'ed from the list.
If that is the case you can directly use a valid default value instead of None e.g. 0
def pop(self, index=0):
First, add a parameter with a default value to the function:
def pop(self, item=None):
Now, in the code, if item is None:, you can do the "no param" thing; otherwise, use item. Whether you want to switch at the top, or lower down in the logic, depends on your logic. In this case, item is None probably means "match the first item", so you probably want a single loop that checks item is None or current.data == item:.
Sometimes you'll want to do this for a parameter that can legitimately be None, in which case you need to pick a different sentinel. There are a few questions around here (and blog posts elsewhere) on the pros and cons of different choices. But here's one way:
class LinkedList(object):
_sentinel = object()
def pop(self, item=_sentinel):
Unless it's valid for someone to use the private _sentinel class member of LinkedList as a list item, this works. (If that is valid—e.g., because you're building a debugger out of these things—you have to get even trickier.)
The terminology on this is a bit tricky. Quoting the docs:
When one or more top-level parameters have the form parameter = expression, the function is said to have “default parameter values.”
To understand this: "Parameters" (or "formal parameters") are the things the function is defined to take; "arguments" are things passed to the function in a call expression; "parameter values" (or "actual parameters", but this just makes things more confusing) are the values the function body receives. So, it's technically incorrect to refer to either "default parameters" or "parameters with default arguments", but both are quite common, because even experts find this stuff confusing. (If you're curious, or just not confused yet, see function definitions and calls in the reference documentation for full details.)
Is your exam using Python specifically? If not, you may want to look into function overloading. Python doesn't support this feature, but many other languages do, and is a very common approach to solving this kind of problem.
In Python, you can get a lot of mileage out of using parameters with default values (as Michael Mauderer's example points out).
def pop(self, index=None):
prev = None
current = self.head
if current is None:
raise IndexError("can't pop from empty list")
if index is None:
index = 0 # the first item by default (counting from head)
if index < 0:
index += self.count
if not (0 <= index < self.count):
raise IndexError("index out of range")
i = 0
while i != index:
i += 1
prev = current
current = current.getNext()
assert current is not None # never happens if list is self-consistent
assert i == index
value = current.getData()
self.remove(current, prev)
##self.count -= 1 # this should be in self.remove()
return value