Related
I have the following lists:
a = [ 1, 6, 76, 15, 46, 55, 47, 15, 72, 58, ..] # there could be more than 10 elements in each
b = [17, 48, 22, 7, 35, 19, 91, 85, 49, 35, ..]
c = [46, 8, 53, 49, 28, 82, 30, 86, 57, 9, ..]
d = [82, 12, 24, 60, 66, 17, 13, 69, 28, 99, ..]
e = [ 1, 53, 17, 82, 21, 20, 88, 10, 82, 41, ..]
I want to write a function which takes any number of those list (could be all, could be only a and c for example) as its argument and selects the leftmost unique 10 elements equally from every list. For example, I will show in pictures with.
The initial data we have (length of 10 assumption).
We look at the first elements of every row and see a and e have same values. We randomly select let's say e, remove that element and shift it to the left and get this
Here we see that there is again overlap, 17 is appearing already and we shift e one more time
Again similar problem and we shift it one last time
Finally, we can select the first two elements of each list and there will be no duplicates
[1, 6, 17, 48, 46, 8, 82, 12, 21, 53]
It could be that more than one list could have identical values, same rules should apply.
I came with this which and for solving randomness I decided to shuffle the list before using it:
def prepare_unique_array(
arrays: list = [],
max_length: int = 10,
slice_number: int = 2
):
unique_array = []
for array in arrays:
for i in range(slice_number):
while not len(unique_array) == max_length:
if array[i] not in unique_array:
unique_array.append(array[i])
else:
while array[i+1] in unique_array:
i += 1
unique_array.append(array[i+1])
return unique_array
Which gives the desired result given those initial values, but anything changes and it does not work.
maybe there is a numpy approach which does it faster and easier as well.
I will appreciate any guide/help
Using cycle and iter to pick one element from each iterable, alternately:
from itertools import cycle
def uniques_evenly(n, *iterables):
its = cycle(iter(seq) for seq in iterables)
seen = set()
it = next(its)
for _ in range(n):
x = next(it)
while x in seen:
x = next(it) # pick next unique number
seen.add(x)
yield x
it = next(its) # switch to next iterator
Note that this will crash if one of the iterators is too short.
Testing:
a = [ 1, 6, 76, 15, 46, 55, 47, 15, 72, 58, 37756, 712, 666]
b = [17, 48, 22, 7, 35, 19, 91, 85, 49, 35, 42]
c = [46, 8, 53, 49, 28, 82, 30, 86, 57, 9]
d = [82, 12, 24, 60, 66, 17, 13, 69, 28, 99]
e = [ 1, 53, 17, 82, 21, 20, 88, 10, 82, 41, 216]
print( list(uniques_evenly(10, a,b,c,d,e)) )
# [1, 17, 46, 82, 53, 6, 48, 8, 12, 21]
Explanations
We use iter() to transform a list into an iterator. An iterator is something that "consumes" values and returns them, at every call of next(). For instance:
l = [3, 4, 7] # l is a list
i = iter(l) # i is an iterator
print( next(i) )
# 3
print( next(i) )
# 4
print( next(i) )
# 7
print( next(i) )
# raises exception StopIteration
Then, we use itertools.cycle to alternate between the five iterators. cycle returns an infinite iterator that cycles between the items in the list we gave it. We gave it a list of five iterators:
its = cycle(iter(seq) for seq in iterables)
This is the same thing as if we had written:
its = cycle([iter(a), iter(b), iter(c), iter(d), iter(e)]
Here is a demonstration with only two iterators instead of 5:
a = ['hello', 'world']
b = [5, 12]
its = cycle([iter(a), iter(b)])
it = next(its) # it is now the iterator on a
print( next(it) ) # 'hello'
it = next(its) # it is now the iterator on b
print( next(it) ) # 5
it = next(its) # it cycles back to a
print( next(it) ) # 'world'
it = next(its) # it is now b
print( next(it) ) # 12
it = next(its) # it is a
print( next(it) ) # raises exception StopIteration
So this is essentially what happens in uniques_evenly.
In addition, we use a set seen to remember the elements we have already seen. Sets are cool because testing for membership is a fast operation: if x in seen: is a constant-time operation, it doesn't matter how big seen is.
Now it is one of the five iterators, say the iterator on list d. We pick the next element in d with x = next(it); then we run the loop:
while x in seen:
x = next(it)
Which means: if x is an element already seen previously, then pick the next element in d. Keep picking the next element until you find an element that wasn't seen previously. Once we've finally picked an element x that hasn't been seen previously, we add it to set seen so it won't be picked again in the future, and then:
yield x
This is a bit special. It means uniques_evenly is not a function; if it was a function, we would have used the keyword return and not yield. Instead, uniques_evenly is a generator. Instead of using this syntax with yield, the other alternative would have been to declare a list result, initially empty, then instead of yield x, I would have written result.append(x); then at the very end of the function, return result. Then uniques_evenly would have been a function that returns a list.
The difference between a function that returns a list, and a generator, is a bit subtle. Basically a generator behaves like a lazy list, whose elements are computed and produced only when needed.
When testing my code, I immediately converted the generator to a list anyway:
print( list(uniques_evenly(10, a,b,c,d,e)) )
So the difference doesn't matter. If you're more comfortable with having a variable result and using result.append(x) instead of yield x, then returning result at the end of the function, you can do it this way instead.
I am trying to check whether any of two values at the same index for two lists of numbers is at least sixty. Here is my code:
def list_list(x,y):
z=zip(x,y)
l=[list(x) for x in z]
print(l)
for el in l:
print(el)
if el[0]>=60 or el[1]>=60:
el.append('pass')
return el
else:
el.append('fail')
return el
print(list_list(x=[28, 59, 22, 5],
y=[59, 85, 55, 60]))
When I run this code it only returns [28,59,'fail'] other remaining lists are not returned. It should also return [59, 85, 'pass'], [22, 55, 'fail'] and [5, 60, 'pass'].
Why does my function stop after the first result?
There are a couple of issues with your code. The most blatant one is that when you say return, your function returns for good. None of the following lines will be executed once a function encounters a return statement.
In addition, there's no need to build the list l from the return value of zip, you can iterate over the tuples generated by zip directly.
There are two ways to prevent your function from prematurely returning. The first is to build a result-list to which you append the intermediary results and return the result-list at the end:
def list_list(x,y):
result = [] # make an empty list to store results
for a,b in zip(x,y):
# decide pass or fail
if a >= 60 or b >= 60:
word = 'pass'
else:
word = 'fail'
# append to result list
result.append([a, b, word])
# return result list
return result
Alternatively, you can construct a generator that spits out the intermediary results as they are computed:
def list_list(x,y):
for a,b in zip(x,y):
# decide pass or fail
if a >= 60 or b >= 60:
word = 'pass'
else:
word = 'fail'
# yield intermediary result
yield [a, b, word]
print(list(list_list([28, 59, 22, 5], [59, 85, 55, 60]))) # [[28, 59, 'fail'], [59, 85, 'pass'], [22, 55, 'fail'], [5, 60, 'pass']]
Of course, all of this could be done with a list comprehension:
>>> x=[28, 59, 22, 5]
>>> y=[59, 85, 55, 60]
>>> [[a,b,'pass'] if a >= 60 or b >= 60 else [a,b,'fail'] for a,b in zip(x,y)]
[[28, 59, 'fail'], [59, 85, 'pass'], [22, 55, 'fail'], [5, 60, 'pass']]
Final notes:
in Python2, consider using itertools.izip for memory efficiency instead of zip because you don't need the complete list of tuples from zip.
Consider using a better function name. list_list is telling me nothing about what the function is supposed to do. How about atleast_sixty_pairs or something like that?
When you say return in your function, you are quitting the function. If you go to a party and return, you aren't still at the party. You could change return to yield in order to create a generator function, but what you probably want is to create your own list:
def list_list(x,y):
z=zip(x,y)
l=[list(x) for x in z]
returnlist = []
print(l)
for el in l:
print(el)
if el[0]>=60 or el[1]>=60:
el.append('pass')
returnlist.append(el)
else:
el.append('fail')
returnlist.append(el)
return returnlist
print(list_list(x=[28, 59, 22, 5],
y=[59, 85, 55, 60]))
I had a job interview today. During it I was asked to write down an algorithm that will reverse a list. First I offered the answer using the reversed() method:
x = [1,2,3,4,5]
y = reversed(x)
for i in y:
print i
The senior developer conducting the interview asked me if I know another way, upon which I wrote down the other known method with slicing:
x = [1,2,3,4,5]
y = x[::-1]
Unfortunately he was unhappy with this solution as well and asked me to think of another one. After few minutes I said I could not come up with a better one. He said that this was not good enough for their standards.
I am perfectly fine with his opinion and have no problem practicing more on my code. My question is, what is a better solution that I am not aware of, if there is one. Could there be some other more 'programmer' way...Only other thing that comes to mind is recursion, however I thought of it only after the interview was already done.
Thanks.
Both your answers are good in terms of python so the interviewer must have been asking you to implement your own method:
Using recursion:
def recur_rev(l):
return recur_rev(l[1:]) + l[:1] if l else l
Or a list comp and range starting at the length of l -1 and going in reverse:
l = list(range(100))
print([l[ind] for ind in range(len(l)-1,-1,-1)])
Using itertools.count:
from itertools import count
cn = count(len(l) -1, -1)
print([l[next(cn)] for ele in l])
For efficiency use a generator expression:
rev = (l[next(cn)] for ele in l)
for ele in rev:
print(ele)
Or using map:
print(list(map(l.__getitem__,range(len(l)-1,-1,-1)))) # list needed for python3
[99, 98, 97, 96, 95, 94, 93, 92, 91, 90, 89, 88, 87, 86, 85, 84, 83, 82, 81, 80, 79, 78, 77, 76, 75, 74, 73, 72, 71, 70, 69, 68, 67, 66, 65, 64, 63, 62, 61, 60, 59, 58, 57, 56, 55, 54, 53, 52, 51, 50, 49, 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
without the list call on map we will get a map object we can iterate over in python3, you can use itertools.imap in python2 to achieve a similar result
I guess your interviewer wanted to hear something like this:
A straightforward way to revert a list is to find out its length, then iterate from len-1 downto 0 and append the i-th element to the result. This approach works fine with real lists, but how can we revert a generator, i.e. something whose length is not known and cannot be found? (Good pythonistas use generators and yields instead of lists and returns).
Here's where we need a data structure known as "stack". A stack is like, well, a stack of things, like plates, put on the top of each other.
Whatever you put last comes out first, whatever was put first comes out last. So, our strategy will be like this:
while there are items, put each item onto the stack
while the stack is not empty, pop items off the stack and return them
Stacks can be programmed in python using lists, where .append puts an item on the top of the stack, and .pop removes the last thing off the stack:
def reverse(it):
stack = []
for item in it:
stack.append(item)
while stack:
yield stack.pop()
Of course, Python already has a built-in stack, and it's the stack of call frames. We can use that instead of the simulated one above, which leads us to the following recursive solution:
def reverse(it):
head = next(it)
for item in reverse(it):
yield item
yield head
in python3, this is even more elegant:
def reverse(it):
head = next(it)
yield from reverse(it)
yield head
Again, this works with arbitrary iterables, whose length is unknown at the call time.
One simple algorithm that could be easily ported to other languages would be:
x = [1,2,3,4,5]
y = []
for i in len(x):
y.append(x[len(x) - 1 - i])
Or using an iterator. This could be easily ported to be used with a chained list where you don't know the length:
x = [1,2,3,4,5]
y = []
x_iterator = iter(x)
try:
while (1):
y.insert(0, x_iterator.next())
except:
print y
Or a more pythonic version with insert:
x = [1,2,3,4,5]
y = []
for elem in x:
y.insert(0, elem)
I'm assuming your interviewer didn't want you to use built-in Python methods, but an algorithm that's more language-agnostic. Something like:
lst = [1,2,3,4,5]
lst2 = []
while len(lst) > 0:
lst2.append(lst.pop())
or
lst2 = [lst.pop() for _ in lst[:]]
How about something like that:
x = [1, 2, 3, 4, 5]
for i in xrange(len(x) / 2):
x[i], x[-i - 1] = x[-i - 1], x[i]
print(x)
The idea is to swap array elements from opposite directions
I like this way:
def reverse(arr):
for i in range(len(arr) / 2):
arr[-i-1], arr[i] = arr[i], arr[-i-1]
Basically iterating through the first half of the array and swapping i and len(i)-i-1.
def reverse(text):
rev = []
for let in range(len(text),0,-1):
rev.append(text[let-1])
return ''.join(rev)
First off, there isn't a question if you break this problem into three steps: 1) create some_list 2) create random_list and 3) run a straight forward list comprehension over the two lists (i.e. [(x, y, f(y, m)) for x in l for y in m]. Because of the constraints of some APIs I am working with, I want be able to write the code with two lines: 1) create some_list 2) run a list comprehension which creates the second second list and allows for some operation/function/method against the internally generated list itself.
This is what I would like to be able to do:
import random
[(x, y, not random_list.index(y)) for x in some_list
for y in random.sample([1,2,3,4,5], (random.choice([1,2,3]))) as random_list]
I know the as random_list doesn't work here. I know if I create a list based on random.sample([1,2,3,4,5], random.choice([1,2,3])) before I code the list comprehension, there isn't any problem here.
Writing out the question leads to some answers. Here is the specific answer to the specific problem given above:
[(x, y, not i) for x in some_list
for i, y in enumerate(random.sample([1,2,3,4,5], random.choice([1,2,3])))]
My basic question is: if a list is created inside a list comprehension, can that list itself be referenced within the list comprehension? Or do I need to create some kind of wrapper?
def wrapper():
f = lambda i, l: 42
l = random_list()
return [(i, f(i, l)) for i in l]
[(x, i, v) for x in some_list for i, v in wrapper()]
I think I have largely answered my question, but it seemed like a good one worth writing up. If there are other thoughts, comments, that would be very useful.
#DSM valid question. Here is the particular (django) code:
PlayerPosition.objects.bulk_create([
PlayerPosition(
player=player,
position_id=position_id,
primary=not index
) for player in Player.objects.all()
for index, position_id in enumerate(random_position_list())
])
Essentially you want to create a list, bind it to a name, and then use the name inside a list comprehension. What prevents binding a name in an expression is that assignments are statements. However, you can (ab)use lambdas to get what you want:
(lambda random_list: [
(x, y, not random_list.index(y))
for x in some_list
for y in random_list])(
random_list=random.sample([1, 2, 3, 4, 5], random.choice([1, 2, 3])))
That works in that it's one line of code, but I'm not sure if it works within your other constraints, which are at this point enigmatic and unclear.
EDIT: If you can make a helper function then you should almost certainly take that approach. I would do it differently than yours, though. I'd make a generic one like this:
def iter_with_list(l):
for element in l:
yield (l, element)
Using it like so:
[(x, y, not random_list.index(y))
for x in some_list
for random_list, y in iter_with_list(random.sample([1,2,3,4,5], random.choice([1, 2, 3])))]
That way you can still keep all the logic in the comprehension without spreading it all over the place. This is just another trick to bind a name as part of an expression - in this case by repeatedly offering it up to be bound for each iteration of the comprehension - but it's a lot more elegant than the previous one.
My basic question is: if a list is created inside a list comprehension, can that list itself be referenced within the list comprehension?
Short answer is, yes it can. You can have as many list comprehensions inside list comprehensions as you like, but things will get quite unreadable. Demonstration:
>>> [var for var in [val for val in range(1, 100)]]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
>>>
If you want to combine two lists, as tuples:
You can do something like this:
>>> from random import randint
>>> zip([x for x in xrange(0, 10)], [randint(0, 100) for _ in xrange(10)])
[(0, 56), (1, 60), (2, 7), (3, 29), (4, 85), (5, 76), (6, 95), (7, 91), (8, 40), (9, 4)]
You can even apply operations on the values of list comprehensions (but it gets really unreadable then):
>>> weird_list = [val for val in [var for var in xrange(0, 100) if not var % 2] if not val % 3]
>>> print weird_list
[0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60, 66, 72, 78, 84, 90, 96]
With regard to your comment:
>>> another_weird_list = [(var, 42) for var in xrange(5)]
>>> another_weird_list
[(0, 42), (1, 42), (2, 42), (3, 42), (4, 42)]
No need for a lambda.
In response to your second comment:
Initially:
arbitrary_list = [x for x in range(10)]
f = lambda x: [var + 1 for var in x]
no_list = [(var, f(arbitrary_list)) for var in arbitrary_list]
Then:
no_list = [(var, [var + 1 for var in [x for x in range(10)]]) for var in [x for x in range(10)]]
One liner. You can still make an arbitrary_list if you want.
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
All:
def a(p):
return p+1
def gen(func, k=100):
l= []
for x in range(k):
temp = ("%s_with_parameter_%s" %(func.__name__, x), lambda: func(x))
# maybe this will be more clear to explain my quetion:
# i want to get list/dict which can bind self-defined string and function together
l.append(temp)
return l
l = gen(a, 100)
for x in range(len(l)):
l[x][1]()
100
100
100
100
100
100
100
100
...I suppose output will be a 1 to 101 print out, but it shows a 100 list.
May I get help for this snippet here?
Thanks!
As other answers have noted, the lambdas are using the last value of x because they're closed over it and so see any changes made to it. The trick is to bind them to the value.
You can do this by writing them as
lambda x=x: func(x)
This binds the value of x that is current when the lambda is created to the default parameter p which means that the lambda is no longer a closure over x and is not affected by any future changes in its value. You would change the way you call it to not pass in an argument that you don't do anything with:
for x in range(len(l)):
l[x][1]()
Now, the lambda uses the default value which is bound to what you want.
If you do actually need to pass a value in, then you can just stick the default parameter that is used for binding purposes on after the 'real' parameter:
lambda p, x=x: func(p, x)
The standard trick is to write your function gen() as:
def gen(func, k=100):
l= []
for x in range(k):
l.append(lambda x=x: func(x))
return l
Note the parameter to the lambda expression. This enforces that a new x is created for each lambda. Otherwise, all use the same x from the enclosing scope.
Maybe with this slight variant of your code you'll understand what happens.
def a(p):
return p+1
def gen(func, k=100):
l= []
for x in range(k):
l.append(lambda p: func(x))
x = 77
return l
l = gen(a, 100)
for x in range(len(l)):
print l[x](10)
Now you always get 78 when calling lx. The problem is you always have the same variable x in the closure, not it's value at the moment of the definition of the lambda.
I believe you probably want something like Haskell curryfication. In python you can use functools.partial for that purpose. I would write your code as below. Notice I have removed the dummy parameter.
import functools
def a(p):
return p+1
def gen(func, k=100):
l= []
for x in range(k):
l.append(functools.partial(func, x))
return l
l = gen(a, 100)
for x in range(len(l)):
print l[x]()
Now let's introduce back the dummy parameter:
import functools
def a(p):
return p+1
def gen(func, k=100):
fn = lambda x, p: func(x)
l= []
for x in range(k):
l.append(functools.partial(fn, x))
return l
l = gen(a, 100)
for x in range(len(l)):
print l[x](10)
Edit: I much prefer the solution of Sven Marnach over mine :-)
In your code, it looks like the value used in a() is the last value known for x (which is 100) while you're p variable is actually dummy (like you mention).
This is because the lambda function is "resolved" when you call it.
It means that calling l[x](10) would be "equivalent" to: (this is not correct, it's to explain)
lambda 10: func(100)
You can use the yield statement to create a generator, I think is the most elegant solution:
>>> def a(p):
... return p+1
...
>>> def gen(func, k=100):
... for x in range(k):
... yield lambda :func(x)
...
>>> for item in gen(a, 100):
... item()
...
1
2
3
4
(...)
100
>>>
But as you can see it goes only until 100 because of the range function:
>>> range(100)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 2
2, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 4
2, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 6
2, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 8
2, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
>>>
You can use gen(a, 101) to solve this:
>>> for item in gen(a, 101):
... item()
...
1
2
3
4
5
(...)
101
>>>