The following is totally bogus code. But let's say you needed to do some extra side effecting function calls (for debugging to logs)? How would you put that in?
[ i for i in range(10) ]
Or does one always have to rewrite as a normal for loop?
list=[]
for i in range(10):
otherStuff()
list.append(i)
In C, there is a comma operator for such things...
Plainly, don't use side-effects in list comprehensions. It makes your code incredibly unclear to the next person who has to maintain it, even if you understand it perfectly. List comprehensions are a succinct way of creating a list, not a way to call a function n times.
For further reading, see the question Is it Pythonic to use list comprehensions for just side effects?
In other words, you should use an explicit for loop for that.
You need to include a call to your side-effect-having code somewhere in your value expression, but you need to ignore that value.
or is one possible choice for this. Just make sure that your side-effect function returns a "Falsey" value (False, None, 0, etc.), and put your debug call in the left-hand side of the or.
def debug_func(i):
print i, i**3
return None
whole_numbers = [ debug_func(i) or i for i in range(10) ]
print whole_numbers
As an alternative, your function could be an identity function, always returning its sole argument:
def debug_func(i):
print i, i**3
return i
# Production code:
whole_numbers = [i for i in range(10)]
# Debug code
whole_numbers = [debug_func(i) for i in range(10)]
Here's one option that doesn't require anything about what your function returns:
[(myfunc(), i)[1] for i in range(10)]
You can also do more than one function at a time:
[(myfunc(), myfunc2(), i)[-1] for i in range(10)]
Related
I have a list of objects and they have a method called process. In Python 2 one could do this
map(lambda x: x.process, my_object_list)
In Python 3 this will not work because map doesn't call the function until the iterable is traversed. One could do this:
list(map(lambda x: x.process(), my_object_list))
But then you waste memory with a throwaway list (an issue if the list is big). I could also use a 2-line explicit loop. But this pattern is so common for me that I don't want to, or think I should need to, write a loop every time.
Is there a more idiomatic way to do this in Python 3?
Don't use map or a list comprehension where simple for loop will do:
for x in list_of_objs:
x.process()
It's not significantly longer than any function you might use to abstract it, but it is significantly clearer.
Of course, if process returns a useful value, then by all means, use a list comprehension.
results = [x.process() for x in list_of_objs]
or map:
results = list(map(lambda x: x.process(), list_of_objs))
There is a function available that makes map a little less clunky, especially if you would reuse the caller:
from operator import methodcaller
processor = methodcaller('process')
results = list(map(processor, list_of_objs))
more_results = list(map(processor, another_list_of_objs))
If you are looking for a good name for a function to wrap the loop, Haskell has a nice convention: a function name ending with an underscore discards its "return value". (Actually, it discards the result of a monadic action, but I'd rather ignore that distinction for the purposes of this answer.)
def map_(f, *args):
for f_args in zip(*args):
f(*f_args)
# Compare:
map(f, [1,2,3]) # -- return value of [f(1), f(2), f(3)] is ignored
map_(f, [1,2,3]) # list of return values is never built
Since you're looking for a Pythonic solution, why would even bother trying to adapt map(lambda x: x.process, my_object_list) for Python 3 ?
Isn't a simple for loop enough ?
for x in my_object_list:
x.process()
I mean, this is concise, readable and avoid creating an unnecessary list if you don't need return values.
Think about a function that I'm calling for its side effects, not return values (like printing to screen, updating GUI, printing to a file, etc.).
def fun_with_side_effects(x):
...side effects...
return y
Now, is it Pythonic to use list comprehensions to call this func:
[fun_with_side_effects(x) for x in y if (...conditions...)]
Note that I don't save the list anywhere
Or should I call this func like this:
for x in y:
if (...conditions...):
fun_with_side_effects(x)
Which is better and why?
It is very anti-Pythonic to do so, and any seasoned Pythonista will give you hell over it. The intermediate list is thrown away after it is created, and it could potentially be very, very large, and therefore expensive to create.
You shouldn't use a list comprehension, because as people have said that will build a large temporary list that you don't need. The following two methods are equivalent:
consume(side_effects(x) for x in xs)
for x in xs:
side_effects(x)
with the definition of consume from the itertools man page:
def consume(iterator, n=None):
"Advance the iterator n-steps ahead. If n is none, consume entirely."
# Use functions that consume iterators at C speed.
if n is None:
# feed the entire iterator into a zero-length deque
collections.deque(iterator, maxlen=0)
else:
# advance to the empty slice starting at position n
next(islice(iterator, n, n), None)
Of course, the latter is clearer and easier to understand.
List comprehensions are for creating lists. And unless you are actually creating a list, you should not use list comprehensions.
So I would got for the second option, just iterating over the list and then call the function when the conditions apply.
Second is better.
Think of the person who would need to understand your code. You can get bad karma easily with the first :)
You could go middle between the two by using filter(). Consider the example:
y=[1,2,3,4,5,6]
def func(x):
print "call with %r"%x
for x in filter(lambda x: x>3, y):
func(x)
Depends on your goal.
If you are trying to do some operation on each object in a list, the second approach should be adopted.
If you are trying to generate a list from another list, you may use list comprehension.
Explicit is better than implicit.
Simple is better than complex. (Python Zen)
You can do
for z in (fun_with_side_effects(x) for x in y if (...conditions...)): pass
but it's not very pretty.
Using a list comprehension for its side effects is ugly, non-Pythonic, inefficient, and I wouldn't do it. I would use a for loop instead, because a for loop signals a procedural style in which side-effects are important.
But, if you absolutely insist on using a list comprehension for its side effects, you should avoid the inefficiency by using a generator expression instead. If you absolutely insist on this style, do one of these two:
any(fun_with_side_effects(x) and False for x in y if (...conditions...))
or:
all(fun_with_side_effects(x) or True for x in y if (...conditions...))
These are generator expressions, and they do not generate a random list that gets tossed out. I think the all form is perhaps slightly more clear, though I think both of them are confusing and shouldn't be used.
I think this is ugly and I wouldn't actually do it in code. But if you insist on implementing your loops in this fashion, that's how I would do it.
I tend to feel that list comprehensions and their ilk should signal an attempt to use something at least faintly resembling a functional style. Putting things with side effects that break that assumption will cause people to have to read your code more carefully, and I think that's a bad thing.
Consider the following code:
a = [... for i in input]
i = a.index(f(a))
I'm wondering whether I could be able to do an one-liner. Obvious try is:
i = [... for i in input].index(f([... for i in input]))
But this solution requires list to be generated twice.
Not-so-obvious try is:
i = [ a.index(f(a)) for a in [[... for i in input],] ]
Which does the trick, but makes code really weird.
This leads me to idea that probably there is possibility to somehow use list, generated by list comprehension, in its own method call. Something like (both not working, obviously):
i = [... for i in input].index(f(_))
# or
i = [... for i in input].index(f(self))
Can it be done?
As you are doing a recursion task, based on what your function does you can mix your function with a list comprehension or a generator expression.
for example consider the following code :
>>> f=lambda x:next(i for i in x if i.startswith('a'))
>>>
>>> a=['df','sr','t','aaf','ar','trf']
>>> a.index(f(a))
3
You can mix like following using enumerate :
>>> next(i for i,j in enumerate(a) if j.startswith('a'))
3
So its all based on your function that how you can put its structure within a list comprehension or a generator expression,and then apply some changes on it and use python tools based on your needs (in this case we used enumerate).
One way that avoids repeating the list-comprehension is to create an anonymous function and calling it directly (untested):
i = (lambda a: a.index(f(a)))([... for i in input])
This is still a bit ugly, though. Your first example that used the temporary variable a is much clearer. Writing one-liners as an exercise is fun, but it is usually not the best way for writing maintainable code.
I know I’ve seen (perhaps exclusively in other languages) where you can use for loops in function arguments. I forget what it was called, but in an attempt to make my code smaller I want to try it. For those of you who don't know what I'm talking about, it goes something like this:
math.sum(for i in range(5)) # Just an example; code will probably not work
Or something like that? I'm not sure how it works yet, but I intend to learn. I know there is a name for this sort of thing, but I've forgotten what it is. Could anyone give me some pointers, or am I insane and this doesn't exist in python?
A "for loop as an expression" is usually called a "comprehension", at least in Haskell, Python, and other languages inspired by them.
Read List Comprehensions in the tutorial for an introduction to the idea. There are also set comprehensions and dict comprehensions, which are pretty obvious once you get list comprehensions.
Then there are generator expressions, which are a bit trickier—but a lot cooler. You're not going to understand those until you first read Iterators, and then Generators, and then Generator Expressions is the very next section.
It still probably won't be clear why generator expressions are cool, but David Beazley explains that masterfully.
To translate your code to real code, all you need is:
math.sum(i for i in range(5))
However, what you're asking for is "all of the elements of range(5), which you can do a lot more easily like this:
math.sum(range(5))
Why? Because a range is already an iterable object.* If it weren't, you couldn't use it in a for loop in the first place, by definition.
Unless you have either some expression to perform on each element, or an if clause to filter the loop, or multiple for clauses for nested looping, comprehensions don't buy you anything. So, here's some more useful examples:
math.sum(i*i for i in range(5))
math.sum(i for i in range(5) if i%3 != 0)
math.sum(j for i in range(5) for j in range(i))
* Technically speaking, you're asking for an iterator over all of the elements in range(5), not just any iterable over them. For a for loop it doesn't matter, but if you need something that you can call next on manually, have it remember its current position, etc., it does. In that case, what you want is iter(range(5)).
The fact that your comprehension happens to be a function argument is almost completely irrelevant here. You can use them anywhere you can use an expression:
squares_to_5 = (i*i for i in range(5)) # often useful
for square in (i*i for i in range(5)): # silly, but legal
However, notice that generator expressions need to be put inside parentheses. In the special case where a generator expression is the only argument to a function, so it's already in parentheses, you can leave the extra parentheses off.
You're thinking of list comprehensions and generator expressions.
This would work in Python with only a slight modification:
sum(i for i in range(5))
This is the seminal work on generators: http://www.dabeaz.com/generators/
Technically speaking they are completely unrelated to the fact that you're using them as function arguments:
x = (i for i in range(5))
evens = [i for i in range(100) if i % 2 == 0]
even_squares = [i**2 for i in evens]
I want to keep astructure saved over iterations of a function, in order to keep data to use it for later purpose.
There is a simple way to do it, in using a global variable.
The thing is, anyone could access the variable.
So, is there a way to create a variable accessing only by the function, but that would not be erased when running this function?
Ex :
mylist = []
def test():
global mylist
if mylist:
my_list.append(1)
other_stuff, using my_list
Is exactly what I want to do, but my_list could be accessed by anyone else.
(ok I know, the example is completely dumb)
Sorry for the formulation, but I could not come with something simpler
EDIT : Ok, so with so different (and all interesting) solutions. I'll quickly explain the idea of what I want to do :)
Let's imagine I want to calculate is a number is a prime. This is usually computationnaly coslty.
So I got a is_prime(value) function, returning False or True.
value can be any number, but there are chances (in my case) that value takes several times the same value (litteraly ^^).
In this case, I could use a (not to long ) list of primes I have already found to quickly check and save computations.
But my prime_list is no use in a function returning true/false.
So here is my question :).
Hope to clarify some minds (including me! ).
Here is a case where a list as the default value of a parameter can come in handy:
def test(myList=[]):
myList.append(1)
other_stuff, using myList
I leave the if myList off, since it will exist (it was created when test was defined). (Plus, if myList tests if the list is non-empty, not defined).
myList, as a function parameter, is local to test. When test is defined, it is initialized to a list. The value of the list persists for the life of your program.
Compare to the standard advice of
def test(myList = None):
if myList is None:
myList = []
which is necessary when you want a fresh empty list inside test at each call if no list is provided by the call.
You can add a "static variable" to the function as follows:
>>> def fun(x):
... fun.counter += x
... print fun.counter
>>> fun.counter = 0
>>> fun(1)
1
>>> fun(5)
6
One solution is to wrap it in another function:
def make_tester():
mylist = []
def test():
if mylist:
my_list.append(1)
# other_stuff, using my_list
return test
test = make_tester()
This incurs a small overhead in that you have to first call tester to create the function that will be called later. OTOH, it has the possible benefit that you can establish multiple contexts, each with their own instance of mylist:
test_a = make_tester()
test_b = make_tester()