Develop Reference

How to use `filter` with multiple iterables, as is supported by `map`?

filter accepts only one iterable, whereas map accepts a variadic number of iterables. For example, I can exhaust map(operator.add, [1, 2, 3, 4], [1, 2, 2, 4]) to get [2, 4, 5, 8].
I'm looking for a similar mechanism for filter, accepting any predicate and a variable number of iterables. Exhausting filter(operator.eq, [1, 2, 3, 4], [1, 2, 2, 4]) causes a TypeError about how filter only accepts 1 iterable, not 2.
My expected output for that particular case is ([1, 2, 4], [1, 2, 4]), i.e the pairwise elements that don't satisfy operator.eq are removed.
Here's what I have so far (eager version supporting only 2 iterables instead of N):
from typing import TypeVar, Callable, Iterable
A = TypeVar("A")
B = TypeVar("B")
def filter_(predicate: Callable[[A, B], bool], iterable1: Iterable[A], iterable2: Iterable[B]) -> (Iterable[A], Iterable[B]):
filtered_iterable1 = []
filtered_iterable2 = []
for value1, value2 in zip(iterable1, iterable2):
if predicate(value1, value2):
filtered_iterable1.append(value1)
filtered_iterable2.append(value2)
return filtered_iterable1, filtered_iterable2
However my goal is to 1) be able to support N iterables and 2) to have filter_ be lazy instead of eager, as is with filter.

Unfortunately there's no equivalent to starmap like starfilter, so the equivalent I can think of is:
[i for i in zip(*lists) if predicate(*i)]
lists here being something like ([..], [..]). This results in:
[(1, 1), (2, 2), (4, 4)]
To turn this back into separate lists, use tuple(map(list, zip(*result))):
([1, 2, 4], [1, 2, 4])
So, putting it together:
predicate = operator.eq
lists = [1, 2, 3, 4], [1, 2, 2, 4]
result = tuple(map(list, zip(*(i for i in zip(*lists) if predicate(*i)))))

Your answer is in your implementation. Map accepts a function taking multiple lists which must match the number of arguments. Filter takes a single list to filter, so the difference in not just semantic - it makes sense for filter to only take a single list. In your case the list is indeed the zip, and that is what you implement. What you are missing is a nifty way to unpair the paired results:
>>> r1, r2 = zip(*filter(lambda x: predicate(*x), zip([1, 2, 3, 4, 5], [1, 1, 3, 3, 5)))
>>> r1
(1, 3, 5)
>>> r2
(1, 3, 5)

How about:
def filter_(predicate, *iterables):
for t in zip(*iterables):
if predicate(*t):
yield t
print(list(filter_(operator.eq, [1, 2, 3, 4], [1, 2, 2, 4])))
It is lazy, it outputs [(1, 1), (2, 2), (4, 4)] for your test case, and no, you can't have ([1, 2, 4], [1, 2, 4]) as a result in a lazy way. To convert from [(1, 1), (2, 2), (4, 4)] to ([1, 2, 4], [1, 2, 4]) you could use: zip(*filter_(operator.eq, [1, 2, 3, 4], [1, 2, 2, 4])) but then of course you lose the laziness.

Calculating array combinations using recursion

I wish to input an n*m array and the resulting output be an array containing the different combinations of the row elements.
Here's an example to clarify (albeit an extremely simple case):
I wish to input an array of the following shape:
[[1, 2, 3]
[2, 5, 6]]
And wish to receive the following output:
[[1,2], [1,5], [1,6], [2,5], [2,6], [3,2], [3,5], [3,6]]
As you can see [2,2] is not included because of repetition.
I can write quick and dirty code containing nested for loops when the input dimensions are know a priori:
A = [[1, 2, 3], [2, 5, 6]]
m = len(A[0])
for i in range(0, m):
for j in range(0, m):
if A[0][i]!=A[1][j]: #check and eliminate repetition
combined.append([A[0][i],A[1][j])
choice_num.append([i+1, j+1]) #See (**) below
I would really like to know how to implement this as a recursive function so given some input n-D array, A, one can simply call it as:
recursive_looper(A)
(**) Another feature that I would like is for the function to output the column number corresponding to the element used in the combination so we get two outputs:
element values: [[1,2], [1,5], [1,6], [2,5], [2,6], [3,2], [3,5], [3,6]]
element position: [[1,1], [1,2], [1,3], [2,2], [2,3], [3,1], [3,2], [3,3]]
Any tips or suggestions would be greatly appreciated!!
Edit: I am open to any solution that can achieve the desired output. Recursion was simply the first thing that came to mind.
Edit 2 (Extended capabilities): This code must not be restricted to a specific list input shape but be extensible to any array of shape (n,m).
I'll provide an example for where the code breaks down. The work-around was implementing n-1 conditional statements, which I would like to avoid because the array shape must be known a priori.
A = [[2, 4, 1, 11, 3], [3, 2, 1, 4, 11], [2, 3, 4, 17, 13]]
If I do not make any modifications to your indexing/filter I receive the following output for the 'filtered' list:
#[[2, 3, 2], [2, 3, 3], [2, 3, 4], [2, 3, 17], [2, 3, 13], [2, 1, 2], ..., [3, 11, 13]]
Immediately I notice that it only compared element position 0 with position 1 for 'likeness', hence why the first combination contains two 2's.
I can make a modification to the Index grabber and filter loop which looks like so:
for i in range(0, len(projects_master)-2):
indexes = [idx for idx, t in enumerate(prod) if t[i] == t[i+1] or t[i]==t[i+2] or t[i+1] == t[i+2] ]
res = []
for i in range(0, len(A)-2):
res.append(list(filter( lambda v: v[i] != v[i+1] and v[i] != v[i+2] and v[i+1] != v[i+2], prod)))
result = [list(t) for t in res[0]]
This does give the correct output, but like I said, I needed to write out n-1 t[i] and v[i] conditions. How can this be done automatically?
EDIT 3 - FINAL
Thanks a bunch to those who provided different approaches to help me achieve the same end goal. I took some insight from each and wrote something that makes sense to me and seems to function well for any input. The code which filters duplicates and removes them from the combinations is shown below:
ind_remove = []
for i in range(0, len(prod)):
if len(prod[i]) != len(set(prod[i])):
ind_remove.append(i)
adder=0
for i in ind_remove:
del prod[i-adder]
adder=adder+1 #takes into account change in indices after an element is deleted.

You can use itertools.product to generate the required combinations, which works like a cartesion product between two sets to generate the combinations.
So it the lists have been [[1, 2], [3, 4]], the cartesian product within the sublists will be
[[1, 3], [1, 4], [2, 3], [2, 4]]
from itertools import product
a = [[1, 2, 3], [2, 5, 6]]
# Generate all possible products, *a gives you two lists
prod = list(product(*a))
#[(1, 2), (1, 5), (1, 6), (2, 2), (2, 5), (2, 6), (3, 2), (3, 5), (3, 6)]
#Get the list of duplicate indexes
indexes = [idx for idx, t in enumerate(prod) if t[0] == t[1] ]
print(indexes)
#[3]
#Remove tuples who are duplicates
res = list(filter( lambda v: v[0] != v[1], prod))
print(res)
#[(1, 2), (1, 5), (1, 6), (2, 5), (2, 6), (3, 2), (3, 5), (3, 6)]
#Convert the tuples to list
result = [list(t) for t in res]
print(result)
#[[1, 2], [1, 5], [1, 6], [2, 5], [2, 6], [3, 2], [3, 5], [3, 6]]

You can use a function that iterates over the items of the first list of the given list of lists and merge each item with the combinations from the recursive calls:
def nonrepetitive_product(lists):
if not lists:
yield []
return
first, *rest = lists
combinations = list(nonrepetitive_product(rest))
for item in first:
for combination in combinations:
if item not in combination:
yield [item, *combination]
so that given:
l = [[1, 2, 3], [2, 5, 6]]
list(nonrepetitive_product(l)) returns:
[[1, 2], [1, 5], [1, 6], [2, 5], [2, 6], [3, 2], [3, 5], [3, 6]]

If you want the positions and values for any number of rows, you'd be better off using itertools.product and enumerate together. Filtering is a little tricky, but it can be done:
import itertools
A = [[1, 2, 3], [2, 5, 6], [7, 8, 3]]
prod = itertools.product(*map(enumerate, A)) # yields ((i,x),(j,y),(k,z),...) nested tuples
transposed = ([*zip(*pairs)] for pairs in prod) # yields ((i,j,k,...), (x,y,z,...)) 2-tuples
filtered = [(ijk, xyz) for ijk, xyz in transposed if len(xyz) == len(set(xyz))] # filter dupes
indexes, values = zip(*filtered) # you might find `filtered` more useful than separate lists

Using list comprehension, tuples and itertools.groupby

This has been giving me some trouble for a while, maybe I've got tunneled vision. Given a list of integers, generate a new list where every group of adjacent duplicates has been turned into a tuple.
For example, given the list: [1, 2, 3, 3, 4, 5, 5, 5, 6]
The generated list contains: [1, 2, (3, 3), 4, (5, 5, 5), 6]
I'd like to achieve this using list comprehension.
numbers = [1, 2, 3, 3, 4, 5, 5, 5, 6]
it = itertools.groupby(numbers)
numbers = [tuple(group) if len(tuple(group)) > 1 else key for key, group in it]
The result I'm expecting:
[1, 2, (3, 3), 4, (5, 5, 5), 6]
The result I'm getting:
[1, 2, (), 4, (), 6]
The inserted tuples are empty, apparently - but at the same time they're not, since they would have had to have had more than one element in them to get inserted in the first place. What's going on? I'm new to python, and even after exhausting all the keywords I can think of I still haven't been able to find a similar question online. I'm sure it's something simple and I just can't see it. Any help is appreciated.

If you want to do list comprehension
>>>l = [1, 2, 3, 3, 4, 5, 5, 5, 6]
>>>[k[0] if len(k) == 1 else tuple(k) for k in [list(j) for i,j in itertools.groupby(l)]]
[1, 2, (3, 3), 4, (5, 5, 5), 6]

The problem is that the group variable is an iterator that only can be iterated once. It appears empty after exhausting. You need to store the intermediate group temporarily. One way to go is using nested generators/comprehesions as itzmeontv suggested, or to use a mapping function:
def make_group(group):
group = tuple(group)
if len(group) == 1:
return group[0]
return group
numbers = [make_group(group) for key, group in itertools.group_by(numbers)]

You may try this one
a = [1, 2, 3, 3, 4, 5, 5, 5, 6]
[(i,)*a.count(i) if a.count(i)>1 else i for i in set(a)]
output:
[1, 2, (3, 3), 4, (5, 5, 5), 6]

How to swap elements in tuple?

I have a very specific problem where I need to know how to swap elements in a list or tuple.
I have one list that is called board state and I know the elements that need to be swapped. How do I swap them? In java with two-dimensional arrays, I could easily do the standard swap technique but here it says tuple assignment is not possible.
Here is my code:
board_state = [(0, 1, 2), (3, 4, 5), (6, 7, 8)]
new = [1, 1] # [row, column] The '4' element here needs to be swapped with original
original = [2, 1] # [row, column] The '7' element here needs to be swapped with new
Result should be:
board_state = [(0, 1, 2), (3, 7, 5), (6, 4, 8)]
How do I swap?

Tuples, like strings, are immutable: it is not possible to assign to the individual items of a tuple.
Lists are mutable, so convert your board_state to a list of lists:
>>> board_state = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
And then use the standard Python idiom for swapping two elements in a list:
>>> board_state[1][1], board_state[2][1] = board_state[2][1], board_state[1][1]
>>> board_state
[[0, 1, 2], [3, 7, 5], [6, 4, 8]]

Getting two-tuples out of a list

I just extracted some data from a list using python but think it's overcomplicated and unpythonic and there's probably a much better way to do this. I'm actually pretty sure I saw this somewhere in the standard library docs but my brain refuses to tell me where.
So here it goes:
Input:
x = range(8) # any even sequence
Output:
[[0, 1], [2, 3], [4, 5], [6, 7]]
My take:
[ [x[i], x[i+1]] for i in range(len(x))[::2] ]

Tuples?
In Python 2.n
>>> zip(*2*[iter(x)])
[(0, 1), (2, 3), (4, 5), (6, 7)]
In Python 3.n
zip() behaves slightly differently...
>> zip(*2*[iter(x)])
<zip object at 0x285c582c>
>>> list(zip(*2*[iter(x)])])
[(0, 1), (2, 3), (4, 5), (6, 7)]
Lists?
The implementation is the same in Python 2 and 3...
>>> [[i,j] for i,j in zip(*2*[iter(x)])]
[[0, 1], [2, 3], [4, 5], [6, 7]]
Or, alternatively:
>>> [list(t) for t in zip(*2*[iter(x)])]
[[0, 1], [2, 3], [4, 5], [6, 7]]
The latter is more useful if you want to split into lists of 3 or more elements, without spelling it out, such as:
>>> [list(t) for t in zip(*4*[iter(x)])]
[[0, 1, 2, 3], [4, 5, 6, 7]]
If zip(*2*[iter(x)]) looks a little odd to you (and it did to me the first time I saw it!), take a look at How does zip(*[iter(s)]*n) work in Python?.
See also this pairwise implementation, which I think is pretty neat.

If you want tuples instead of lists you can try:
>>> zip(range(0, 8, 2), range(1, 8, 2))
[(0, 1), (2, 3), (4, 5), (6, 7)]

Input:
x = range(8) # any even sequence
Solution:
output = []
for i, j in zip(*[iter(x)]*2):
output.append( [i, j] )
Output:
print output
[[0, 1], [2, 3], [4, 5], [6, 7]]

You can rewrite it a bit:
>>> l = range(8)
>>> [[l[i], l[i+1]] for i in xrange(0, len(l), 2)]
[[0, 1], [2, 3], [4, 5], [6, 7]]
For some list tasks you can use itertools, but I'm pretty sure there's no helper function for this one.