I know how to use both for loops and if statements on separate lines, such as:
>>> a = [2,3,4,5,6,7,8,9,0]
... xyz = [0,12,4,6,242,7,9]
... for x in xyz:
... if x in a:
... print(x)
0,4,6,7,9
And I know I can use a list comprehension to combine these when the statements are simple, such as:
print([x for x in xyz if x in a])
But what I can't find is a good example anywhere (to copy and learn from) demonstrating a complex set of commands (not just "print x") that occur following a combination of a for loop and some if statements. Something that I would expect looks like:
for x in xyz if x not in a:
print(x...)
Is this just not the way python is supposed to work?
You can use generator expressions like this:
gen = (x for x in xyz if x not in a)
for x in gen:
print(x)
As per The Zen of Python (if you are wondering whether your code is "Pythonic", that's the place to go):
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Flat is better than nested.
Readability counts.
The Pythonic way of getting the sorted intersection of two sets is:
>>> sorted(set(a).intersection(xyz))
[0, 4, 6, 7, 9]
Or those elements that are xyz but not in a:
>>> sorted(set(xyz).difference(a))
[12, 242]
But for a more complicated loop you may want to flatten it by iterating over a well-named generator expression and/or calling out to a well-named function. Trying to fit everything on one line is rarely "Pythonic".
Update following additional comments on your question and the accepted answer
I'm not sure what you are trying to do with enumerate, but if a is a dictionary, you probably want to use the keys, like this:
>>> a = {
... 2: 'Turtle Doves',
... 3: 'French Hens',
... 4: 'Colly Birds',
... 5: 'Gold Rings',
... 6: 'Geese-a-Laying',
... 7: 'Swans-a-Swimming',
... 8: 'Maids-a-Milking',
... 9: 'Ladies Dancing',
... 0: 'Camel Books',
... }
>>>
>>> xyz = [0, 12, 4, 6, 242, 7, 9]
>>>
>>> known_things = sorted(set(a.iterkeys()).intersection(xyz))
>>> unknown_things = sorted(set(xyz).difference(a.iterkeys()))
>>>
>>> for thing in known_things:
... print 'I know about', a[thing]
...
I know about Camel Books
I know about Colly Birds
I know about Geese-a-Laying
I know about Swans-a-Swimming
I know about Ladies Dancing
>>> print '...but...'
...but...
>>>
>>> for thing in unknown_things:
... print "I don't know what happened on the {0}th day of Christmas".format(thing)
...
I don't know what happened on the 12th day of Christmas
I don't know what happened on the 242th day of Christmas
The following is a simplification/one liner from the accepted answer:
a = [2,3,4,5,6,7,8,9,0]
xyz = [0,12,4,6,242,7,9]
for x in (x for x in xyz if x not in a):
print(x)
12
242
Notice that the generator was kept inline. This was tested on python2.7 and python3.6 (notice the parens in the print ;) )
It is honestly cumbersome even so: the x is mentioned four times.
I personally think this is the prettiest version:
a = [2,3,4,5,6,7,8,9,0]
xyz = [0,12,4,6,242,7,9]
for x in filter(lambda w: w in a, xyz):
print x
Edit
if you are very keen on avoiding to use lambda you can use partial function application and use the operator module (that provides functions of most operators).
https://docs.python.org/2/library/operator.html#module-operator
from operator import contains
from functools import partial
print(list(filter(partial(contains, a), xyz)))
I would probably use:
for x in xyz:
if x not in a:
print(x...)
a = [2,3,4,5,6,7,8,9,0]
xyz = [0,12,4,6,242,7,9]
set(a) & set(xyz)
set([0, 9, 4, 6, 7])
You can use generators too, if generator expressions become too involved or complex:
def gen():
for x in xyz:
if x in a:
yield x
for x in gen():
print x
I liked Alex's answer, because a filter is exactly an if applied to a list, so if you want to explore a subset of a list given a condition, this seems to be the most natural way
mylist = [1,2,3,4,5]
another_list = [2,3,4]
wanted = lambda x:x in another_list
for x in filter(wanted, mylist):
print(x)
this method is useful for the separation of concerns, if the condition function changes, the only code to fiddle with is the function itself
mylist = [1,2,3,4,5]
wanted = lambda x:(x**0.5) > 10**0.3
for x in filter(wanted, mylist):
print(x)
The generator method seems better when you don't want members of the list, but a modification of said members, which seems more fit to a generator
mylist = [1,2,3,4,5]
wanted = lambda x:(x**0.5) > 10**0.3
generator = (x**0.5 for x in mylist if wanted(x))
for x in generator:
print(x)
Also, filters work with generators, although in this case it isn't efficient
mylist = [1,2,3,4,5]
wanted = lambda x:(x**0.5) > 10**0.3
generator = (x**0.9 for x in mylist)
for x in filter(wanted, generator):
print(x)
But of course, it would still be nice to write like this:
mylist = [1,2,3,4,5]
wanted = lambda x:(x**0.5) > 10**0.3
# for x in filter(wanted, mylist):
for x in mylist if wanted(x):
print(x)
Use intersection or intersection_update
intersection :
a = [2,3,4,5,6,7,8,9,0]
xyz = [0,12,4,6,242,7,9]
ans = sorted(set(a).intersection(set(xyz)))
intersection_update:
a = [2,3,4,5,6,7,8,9,0]
xyz = [0,12,4,6,242,7,9]
b = set(a)
b.intersection_update(xyz)
then b is your answer
A simple way to find unique common elements of lists a and b:
a = [1,2,3]
b = [3,6,2]
for both in set(a) & set(b):
print(both)
based on the article here: https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a
I used the following code for the same reason and it worked just fine:
an_array = [x for x in xyz if x not in a]
This line is a part of the program! this means that XYZ is an array which is to be defined and assigned previously, and also the variable a
Using generator expressions (which is recommended in the selected answer) makes some difficulties because the result is not an array
Related
There is a formatting question, that I would like some additional insight/understanding on. I have the following code:
In[1]:
mylist = [[99]]
for [x] in mylist:
print(x)
Out[1]:
99
My main question is regarding the []s around the x in the second line. So I have never used []s before when describing the 'x' variable of a loop. Since my output is 99 and not [99], it looks like the []s are going to ask the loop to extract the number from its additional set of brackets.
Question Updated Based on Responses:
If I change the code to remove the brackets:
In[1]:
mylist = [[99]]
for x in mylist:
print(x)
Out[1]:
[99]
I get [99] instead of 99. However, if I do the following:
In[1]:
mylist = [[99,100]]
for x,y in mylist:
print(x)
print(y)
Out[1]:
99
100
This example above doesn't require an additional set of []s around x,y and produces a bracket-less answer in the output, unlike the previous two examples, which requires a [] to produce a bracket-less answer.
I realize this is an odd and fairly silly question since I would never construct a single element list like this. I simply saw this being casually used in an answer elsewhere online (with no explanation on it unfortunately). Being a newcomer, I'm just curious to broaden my understanding of the language.
Thanks in advance.
When you do this:
>>> myList = [[99]]
>>> for x in myList:
print x
Python interprets that as "print each element in this iterable".
When you do this:
>>> myList = [[99,100], [99,101], [99, 102]]
>>> for x in myList:
print x
Python still interprets that as "print each element in this iterable" so that you get this:
[99, 100]
[99, 101]
[99, 102]
But, if you do this:
>>> myList = [[99,100], [99,101], [99, 102]]
>>> for x, y in myList:
print x, y
Python will "unpack" the values for you from each element in the iterable and assign them to x and y. If you want to do that above for only the case of myList = [[99]], Python requires the for [x] in myList syntax so that it unpacks the single value from the list.
The ability to "unpack" an iterable is very powerful in Python. You can assign variables on the fly by unpacking an iterable. In your case, you can imagine having to assign lat lon as variables maybe or something else. You can also unpack values into the args of a function.
In Python 3 you can also do something like this:
x = [1,2,3,4]
first, *rest = x
print (first) # 1
print (rest) # [2,3,4]
For the second example, with lists of length 2, to unpack both values you can do
In[1]:
myList = [[99,100]]
for x, y in myList:
print(x)
print(y)
Out[1]:
99
100
As per Python documentation, the square brackets are essentially ignored in assignments and loop constructs.
So as a complement to user2774697's answer, for x, y in L is equivalent to for [x, y] in L which is equivalent to for ([x, y]) in L and is also equivalent to for (x, y) in L.
The thing that the square brackets do different than a bare round parenthesis is that it enforces an unpacking behavior, it requires the elements in L to be iterable.
When working with a function that returns multiple values with a tuple, I will often find myself using the following idiom to unpack the results from inside a list comprehension.
fiz, buz = zip(*[f(x) for x in input])
Most of the time this works fine, but it throws a ValueError: need more than 0 values to unpack if input is empty. The two ways I can think of to get around this are
fiz = []
buz = []
for x in input:
a, b = f(x)
fiz.append(a)
buz.append(b)
and
if input:
fiz, buz = zip(*[f(x) for x in input])
else:
fiz, buz = [], []
but neither of these feels especially Pythonic—the former is overly verbose and the latter doesn't work if input is a generator rather than a list (in addition to requiring an if/else where I feel like one really shouldn't be needed).
Is there a good simple way to do this? I've mostly been working in Python 2.7 recently, but would also be interested in knowing any Python 3 solutions if they are different.
If f = lambda x: (x,x**2) then this works
x,y = zip(*map(f,input)) if len(input) else ((),())
If input=[], x=() and y=().
If input=[2], x=(2,) and y=(4,)
If input=[2,3], x=(2,3) and y=(4,9)
They're tuples (not lists), but thats thats pretty easy to change.
I would consider using collections.namedtuple() for this sort of thing. I believe the named tuples are deemed more pythonic, and should avoid the need for complicated list comprehensions and zipping / unpacking.
From the documentation:
>>> p = Point(11, y=22) # instantiate with positional or keyword arguments
>>> p[0] + p[1] # indexable like the plain tuple (11, 22)
33
>>> x, y = p # unpack like a regular tuple
>>> x, y
(11, 22)
>>> p.x + p.y # fields also accessible by name
33
>>> p # readable __repr__ with a name=value style
Point(x=11, y=22)
You could use:
fiz = []
buz = []
results = [fiz, buz]
for x in input:
list(map(lambda res, val: res.append(val), results, f(x)))
print(results)
Note about list(map(...)): in Python3, map returns a generator, so we must use it if we want the lambda to be executed.list does it.
(adapted from my answer to Pythonic way to append output of function to several lists, where you could find other ideas.)
Imagine a function:
def Intersects (x, list1, list2, list3):
if x in list1 and x in list2 and x in list3: return True
return False
There has to be a better way to do this but I can't figure it out. How can I do this? (performance is important)
EDIT: I've come across a related, however harder problem this time. This time I've got 3 unrelated integers and I need to test if they intersect too.
Like:
1, 2, 3 <-- elements to look for
if InAll ((1, 2, 3)) ...
but I am not looking for a tuple, instead I am looking for just integers. How do I unpack the tuple and do the same search?
If you want to be able to give it any number of lists (not just three), you might like:
def inAll(x, *lsts):
return all((x in l) for l in lsts)
If you are checking memberships in these lists many times (on many xs, that is), you'll want to make each into a set before you start looping through the xs.
From your most recent edit, it looks like you also want to be able to pass multiple items in x. Code for that would be:
def inAll(xs, *lsts):
return all(all(x in l for x in xs) for l in lsts)
def intersects(x,L1,L2,L3):
if not isinstance(x,(list,tuple)):
x = [x]
return set(x).intersection(L1).intersection(L2).intersection(L3) == set(x)
should work for lots of stuff
>>> l1 = range(10)
>>> l2 = range(5,10)
>>> l3 = range(2,7)
>>> def intersects(x,L1,L2,L3):
... if not isinstance(x,(list,tuple)):
... x = [x]
... return set(x).intersection(L1).intersection(L2).intersection(L3) == set(x)
...
>>> intersects(6,l1,l2,l3)
True
>>> intersects((5,6),l1,l2,l3)
True
>>> intersects((5,6,7),l1,l2,l3)
False
The best way to do something like this is to use the set...probably it will be more efficient even with the conversion time:
def InAll(element_list, *sets):
#calculate the intersection between all the sets
intersection = reduce( lambda i , j: i & j, sets )
#check if every element is in the intersection
return all( i in intersection for i in element_list )
sets = [ set(range(5)), set(range(10)) ]
InAll( [9], *sets )
#False
sets = [ set(range(5)), set(range(10)) ]
InAll( [1,3], *sets )
#True
It's better to convert the list into set beforehand, but it's easy to insert the conversion conde inside the function (but if you have a lot of these controls, keep a copy of the set)
I have a list of strings. I have a function that given a string returns 0 or 1. How can I delete all strings in the list for which the function returns 0?
[x for x in lst if fn(x) != 0]
This is a "list comprehension", one of Python's nicest pieces of syntactical sugar that often takes lines of code in other languages and additional variable declarations, etc.
See:
http://docs.python.org/tutorial/datastructures.html#list-comprehensions
I would use a generator expression over a list comprehension to avoid a potentially large, intermediate list.
result = (x for x in l if f(x))
# print it, or something
print list(result)
Like a list comprehension, this will not modify your original list, in place.
edit: see the bottom for the best answer.
If you need to mutate an existing list, for example because you have another reference to it somewhere else, you'll need to actually remove the values from the list.
I'm not aware of any such function in Python, but something like this would work (untested code):
def cull_list(lst, pred):
"""Removes all values from ``lst`` which for which ``pred(v)`` is false."""
def remove_all(v):
"""Remove all instances of ``v`` from ``lst``"""
try:
while True:
lst.remove(v)
except ValueError:
pass
values = set(lst)
for v in values:
if not pred(v):
remove_all(v)
A probably more-efficient alternative that may look a bit too much like C code for some people's taste:
def efficient_cull_list(lst, pred):
end = len(lst)
i = 0
while i < end:
if not pred(lst[i]):
del lst[i]
end -= 1
else:
i += 1
edit...: as Aaron pointed out in the comments, this can be done much more cleanly with something like
def reversed_cull_list(lst, pred):
for i in range(len(lst) - 1, -1, -1):
if not pred(lst[i]):
del lst[i]
...edit
The trick with these routines is that using a function like enumerate, as suggested by (an) other responder(s), will not take into account the fact that elements of the list have been removed. The only way (that I know of) to do that is to just track the index manually instead of allowing python to do the iteration. There's bound to be a speed compromise there, so it may end up being better just to do something like
lst[:] = (v for v in lst if pred(v))
Actually, now that I think of it, this is by far the most sensible way to do an 'in-place' filter on a list. The generator's values are iterated before filling lst's elements with them, so there are no index conflict issues. If you want to make this more explicit just do
lst[:] = [v for v in lst if pred(v)]
I don't think it will make much difference in this case, in terms of efficiency.
Either of these last two approaches will, if I understand correctly how they actually work, make an extra copy of the list, so one of the bona fide in-place solutions mentioned above would be better if you're dealing with some "huge tracts of land."
>>> s = [1, 2, 3, 4, 5, 6]
>>> def f(x):
... if x<=2: return 0
... else: return 1
>>> for n,x in enumerate(s):
... if f(x) == 0: s[n]=None
>>> s=filter(None,s)
>>> s
[3, 4, 5, 6]
With a generator expression:
alist[:] = (item for item in alist if afunction(item))
Functional:
alist[:] = filter(afunction, alist)
or:
import itertools
alist[:] = itertools.ifilter(afunction, alist)
All equivalent.
You can also use a list comprehension:
alist = [item for item in alist if afunction(item)]
An in-place modification:
import collections
indexes_to_delete= collections.deque(
idx
for idx, item in enumerate(alist)
if afunction(item))
while indexes_to_delete:
del alist[indexes_to_delete.pop()]
How to run an operation on a collection in Python and collect the results?
So if I have a list of 100 numbers, and I want to run a function like this for each of them:
Operation ( originalElement, anotherVar ) # returns new number.
and collect the result like so:
result = another list...
How do I do it? Maybe using lambdas?
List comprehensions. In Python they look something like:
a = [f(x) for x in bar]
Where f(x) is some function and bar is a sequence.
You can define f(x) as a partially applied function with a construct like:
def foo(x):
return lambda f: f*x
Which will return a function that multiplies the parameter by x. A trivial example of this type of construct used in a list comprehension looks like:
>>> def foo (x):
... return lambda f: f*x
...
>>> a=[1,2,3]
>>> fn_foo = foo(5)
>>> [fn_foo (y) for y in a]
[5, 10, 15]
Although I don't imagine using this sort of construct in any but fairly esoteric cases. Python is not a true functional language, so it has less scope to do clever tricks with higher order functions than (say) Haskell. You may find applications for this type of construct, but it's not really that pythonic. You could achieve a simple transformation with something like:
>>> y=5
>>> a=[1,2,3]
>>> [x*y for x in a]
[5, 10, 15]
Another (somewhat depreciated) method of doing this is:
def kevin(v):
return v*v
vals = range(0,100)
map(kevin,vals)
List comprehensions, generator expressions, reduce function.