Develop Reference

Pairwise circular Python 'for' loop

Is there a nice Pythonic way to loop over a list, retuning a pair of elements? The last element should be paired with the first.
So for instance, if I have the list [1, 2, 3], I would like to get the following pairs:
1 - 2
2 - 3
3 - 1

A Pythonic way to access a list pairwise is: zip(L, L[1:]). To connect the last item to the first one:
>>> L = [1, 2, 3]
>>> zip(L, L[1:] + L[:1])
[(1, 2), (2, 3), (3, 1)]

I would use a deque with zip to achieve this.
>>> from collections import deque
>>>
>>> l = [1,2,3]
>>> d = deque(l)
>>> d.rotate(-1)
>>> zip(l, d)
[(1, 2), (2, 3), (3, 1)]

I'd use a slight modification to the pairwise recipe from the itertools documentation:
def pairwise_circle(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ... (s<last>,s0)"
a, b = itertools.tee(iterable)
first_value = next(b, None)
return itertools.zip_longest(a, b,fillvalue=first_value)
This will simply keep a reference to the first value and when the second iterator is exhausted, zip_longest will fill the last place with the first value.
(Also note that it works with iterators like generators as well as iterables like lists/tuples.)
Note that #Barry's solution is very similar to this but a bit easier to understand in my opinion and easier to extend beyond one element.

I would pair itertools.cycle with zip:
import itertools
def circular_pairwise(l):
second = itertools.cycle(l)
next(second)
return zip(l, second)
cycle returns an iterable that yields the values of its argument in order, looping from the last value to the first.
We skip the first value, so it starts at position 1 (rather than 0).
Next, we zip it with the original, unmutated list. zip is good, because it stops when any of its argument iterables are exhausted.
Doing it this way avoids the creation of any intermediate lists: cycle holds a reference to the original, but doesn't copy it. zip operates in the same way.
It's important to note that this will break if the input is an iterator, such as a file, (or a map or zip in python-3), as advancing in one place (through next(second)) will automatically advance the iterator in all the others. This is easily solved using itertools.tee, which produces two independently operating iterators over the original iterable:
def circular_pairwise(it):
first, snd = itertools.tee(it)
second = itertools.cycle(snd)
next(second)
return zip(first, second)
tee can use large amounts of additional storage, for example, if one of the returned iterators is used up before the other is touched, but as we only ever have one step difference, the additional storage is minimal.

There are more efficient ways (that don't built temporary lists), but I think this is the most concise:
> l = [1,2,3]
> zip(l, (l+l)[1:])
[(1, 2), (2, 3), (3, 1)]

Pairwise circular Python 'for' loop
If you like the accepted answer,
zip(L, L[1:] + L[:1])
you can go much more memory light with semantically the same code using itertools:
from itertools import islice, chain #, izip as zip # uncomment if Python 2
And this barely materializes anything in memory beyond the original list (assuming the list is relatively large):
zip(l, chain(islice(l, 1, None), islice(l, None, 1)))
To use, just consume (for example, with a list):
>>> list(zip(l, chain(islice(l, 1, None), islice(l, None, 1))))
[(1, 2), (2, 3), (3, 1)]
This can be made extensible to any width:
def cyclical_window(l, width=2):
return zip(*[chain(islice(l, i, None), islice(l, None, i)) for i in range(width)])
and usage:
>>> l = [1, 2, 3, 4, 5]
>>> cyclical_window(l)
<itertools.izip object at 0x112E7D28>
>>> list(cyclical_window(l))
[(1, 2), (2, 3), (3, 4), (4, 5), (5, 1)]
>>> list(cyclical_window(l, 4))
[(1, 2, 3, 4), (2, 3, 4, 5), (3, 4, 5, 1), (4, 5, 1, 2), (5, 1, 2, 3)]
Unlimited generation with itertools.tee with cycle
You can also use tee to avoid making a redundant cycle object:
from itertools import cycle, tee
ic1, ic2 = tee(cycle(l))
next(ic2) # must still queue up the next item
and now:
>>> [(next(ic1), next(ic2)) for _ in range(10)]
[(1, 2), (2, 3), (3, 1), (1, 2), (2, 3), (3, 1), (1, 2), (2, 3), (3, 1), (1, 2)]
This is incredibly efficient, an expected usage of iter with next, and elegant usage of cycle, tee, and zip.
Don't pass cycle directly to list unless you have saved your work and have time for your computer to creep to a halt as you max out its memory - if you're lucky, after a while your OS will kill the process before it crashes your computer.
Pure Python Builtin Functions
Finally, no standard lib imports, but this only works for up to the length of original list (IndexError otherwise.)
>>> [(l[i], l[i - len(l) + 1]) for i in range(len(l))]
[(1, 2), (2, 3), (3, 1)]
You can continue this with modulo:
>>> len_l = len(l)
>>> [(l[i % len_l], l[(i + 1) % len_l]) for i in range(10)]
[(1, 2), (2, 3), (3, 1), (1, 2), (2, 3), (3, 1), (1, 2), (2, 3), (3, 1), (1, 2)]

I would use a list comprehension, and take advantage of the fact that l[-1] is the last element.
>>> l = [1,2,3]
>>> [(l[i-1],l[i]) for i in range(len(l))]
[(3, 1), (1, 2), (2, 3)]
You don't need a temporary list that way.

Amazing how many different ways there are to solve this problem.
Here's one more. You can use the pairwise recipe but instead of zipping with b, chain it with the first element that you already popped off. Don't need to cycle when we just need a single extra value:
from itertools import chain, izip, tee
def pairwise_circle(iterable):
a, b = tee(iterable)
first = next(b, None)
return izip(a, chain(b, (first,)))

I like a solution that does not modify the original list and does not copy the list to temporary storage:
def circular(a_list):
for index in range(len(a_list) - 1):
yield a_list[index], a_list[index + 1]
yield a_list[-1], a_list[0]
for x in circular([1, 2, 3]):
print x
Output:
(1, 2)
(2, 3)
(3, 1)
I can imagine this being used on some very large in-memory data.

This one will work even if the list l has consumed most of the system's memory. (If something guarantees this case to be impossible, then zip as posted by chepner is fine)
l.append( l[0] )
for i in range( len(l)-1):
pair = l[i],l[i+1]
# stuff involving pair
del l[-1]
or more generalizably (works for any offset n i.e. l[ (i+n)%len(l) ] )
for i in range( len(l)):
pair = l[i], l[ (i+1)%len(l) ]
# stuff
provided you are on a system with decently fast modulo division (i.e. not some pea-brained embedded system).
There seems to be a often-held belief that indexing a list with an integer subscript is un-pythonic and best avoided. Why?

This is my solution, and it looks Pythonic enough to me:
l = [1,2,3]
for n,v in enumerate(l):
try:
print(v,l[n+1])
except IndexError:
print(v,l[0])
prints:
1 2
2 3
3 1
The generator function version:
def f(iterable):
for n,v in enumerate(iterable):
try:
yield(v,iterable[n+1])
except IndexError:
yield(v,iterable[0])
>>> list(f([1,2,3]))
[(1, 2), (2, 3), (3, 1)]

How about this?
li = li+[li[0]]
pairwise = [(li[i],li[i+1]) for i in range(len(li)-1)]

from itertools import izip, chain, islice
itr = izip(l, chain(islice(l, 1, None), islice(l, 1)))
(As above with #j-f-sebastian's "zip" answer, but using itertools.)
NB: EDITED given helpful nudge from #200_success. previously was:
itr = izip(l, chain(l[1:], l[:1]))

If you don't want to consume too much memory, you can try my solution:
[(l[i], l[(i+1) % len(l)]) for i, v in enumerate(l)]
It's a little slower, but consume less memory.

Starting in Python 3.10, the new pairwise function provides a way to create sliding pairs of consecutive elements:
from itertools import pairwise
# l = [1, 2, 3]
list(pairwise(l + l[:1]))
# [(1, 2), (2, 3), (3, 1)]
or simply pairwise(l + l[:1]) if you don't need the result as a list.
Note that we pairwise on the list appended with its head (l + l[:1]) so that rolling pairs are circular (i.e. so that we also include the (3, 1) pair):
list(pairwise(l)) # [(1, 2), (2, 3)]
l + l[:1] # [1, 2, 3, 1]

Just another try
>>> L = [1,2,3]
>>> zip(L,L[1:]) + [(L[-1],L[0])]
[(1, 2), (2, 3), (3, 1)]

L = [1, 2, 3]
a = zip(L, L[1:]+L[:1])
for i in a:
b = list(i)
print b

this seems like combinations would do the job.
from itertools import combinations
x=combinations([1,2,3],2)
this would yield a generator. this can then be iterated over as such
for i in x:
print i
the results would look something like
(1, 2)
(1, 3)
(2, 3)

Unexpected behavor zipping an iterator with a sequence

While trying to solve a particular code golf question, I came across a particular scenario, which I was having difficulty in understanding the behavior.
The scenario was, ziping an iterator with a sequence, and after the transpose operation, the iterator was one past the expected element.
>>> l = range(10)
>>> it = iter(l)
>>> zip(it, range(5))
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)]
>>> next(it) #expecting 5 here
6
Am I missing something obvious?
Note Please provide credible references for answers that may not be obvious

I suspect that, the 5 is consumed when zip tried to zip the next items. Zip stops when one of its arg is "empty":
>>> l = range(10)
>>> it = iter(l)
>>> zip(range(5),it)
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)]
>>> it.next()
5
By reversing the order, zip knows that it can stop and do not consume the next item from it

If you want references you can check the izip documentation. It gives an equivalent implementation:
def izip(*iterables):
iterators = map(iter, iterables)
while iterators:
yield tuple(map(next, iterators))
Since list(izip(*args)) is expected to have same behavior as zip(*args), the result you got is actually the logical behavior.

Removing specific tuples from List

I've got a list
a = [(1,2),(1,4),(2,6),(1,8),(3,6),(1,10),(1,6)]
If I say that:
for x in a:
if x[0]==1:
print x
I get the expected result : (1,2) (1,4) (1,8) (1,10) (1,6)
However I want to remove all the occurrences of all the tuples in the format (1,x),So
for x in a:
if x[0]==1:
a.remove(x)
I thought that all the occurences should be removed.However when i say
Print a
I get [(1,4),(2,6),(3,6),(1,6)]
Not all the tuples were removed. How do I do it.??
Thanks

I'd use list comprehension:
def removeTuplesWithOne(lst):
return [x for x in lst if x[0] != 1]
a = removeTuplesWithOne([(1,2),(1,4),(2,6),(1,8),(3,6),(1,10),(1,6)])
For me it's more pythonic than built-in filter function.
P.S. This function does not change your original list, it creates new one. If your original list is huge, i'd probably use generator expression like so:
def removeTuplesWithOne(lst):
return (x for x in lst if x[0] != 1)

This isn't the same approach as yours but should work
a = filter(lambda x: x[0] != 1, a)

You can use list comprehension like this, to filter out the items which have 1 as the first element.
>>> original = [(1, 2), (1, 4), (2, 6), (1, 8), (3, 6), (1, 10), (1, 6)]
>>> [item for item in original if item[0] != 1]
[(2, 6), (3, 6)]
This creates a new list, rather than modifying the existing one. 99% of the time, this will be fine, but if you need to modify the original list, you can do that by assigning back:
original[:] = [item for item in original if item[0] != 1]
Here we use slice assignment, which works by replacing every item from the start to the end of the original list (the [:]) with the items from the list comprehension. If you just used normal assignment, you would just change what the name original pointed to, not actually modify the list itself.

You can do it with a generator expression if you're dealing with huge amounts of data:
a = [(1,2),(1,4),(2,6),(1,8),(3,6),(1,10),(1,6)]
# create a generator
a = ((x,y) for x, y in a if x == 1)
# simply convert it to a list if you need to...
>>> print list(a)
[(1, 2), (1, 4), (1, 8), (1, 10), (1, 6)]

python - how do you store elements in a tuple

how to store elements in a tuple?
I did this:
for i in range (0, capacity):
for elements in self.table[i]:
# STORE THE ALL THE ELEMENTS IN THE TUPLE

tuples are immutable. You can create a tuple. But you cannot store elements to an already created tuple. To create (or convert) your list of elements to a tuple. Just tuple() it.
t = tuple([1,2,3])
In your case it is
t = tuple(self.table[:capacity])

Since you haven't told us - we can only guess what table looks like
If it is a list of lists for example, you can do this to get a tuple of tuples
>>> table =[[1,2,3],[4,5,6],[7,8,9]]
>>> tuple(map(tuple, table))
((1, 2, 3), (4, 5, 6), (7, 8, 9))
>>> capacity=2
>>> tuple(map(tuple, table[:capacity]))
((1, 2, 3), (4, 5, 6))

It's as easy as t = tuple(self.table[i])

I think this is what you want.
x = []
for i in range (0, capacity):
for elements in self.table[i]:
x.append(elements)
t = tuple(x)

How does zip(*[iter(s)]*n) work in Python?

s = [1,2,3,4,5,6,7,8,9]
n = 3
list(zip(*[iter(s)]*n)) # returns [(1,2,3),(4,5,6),(7,8,9)]
How does zip(*[iter(s)]*n) work? What would it look like if it was written with more verbose code?

iter() is an iterator over a sequence. [x] * n produces a list containing n quantity of x, i.e. a list of length n, where each element is x. *arg unpacks a sequence into arguments for a function call. Therefore you're passing the same iterator 3 times to zip(), and it pulls an item from the iterator each time.
x = iter([1,2,3,4,5,6,7,8,9])
print(list(zip(x, x, x)))

The other great answers and comments explain well the roles of argument unpacking and zip().
As Ignacio and ujukatzel say, you pass to zip() three references to the same iterator and zip() makes 3-tuples of the integers—in order—from each reference to the iterator:
1,2,3,4,5,6,7,8,9 1,2,3,4,5,6,7,8,9 1,2,3,4,5,6,7,8,9
^ ^ ^
^ ^ ^
^ ^ ^
And since you ask for a more verbose code sample:
chunk_size = 3
L = [1,2,3,4,5,6,7,8,9]
# iterate over L in steps of 3
for start in range(0,len(L),chunk_size): # xrange() in 2.x; range() in 3.x
end = start + chunk_size
print L[start:end] # three-item chunks
Following the values of start and end:
[0:3) #[1,2,3]
[3:6) #[4,5,6]
[6:9) #[7,8,9]
FWIW, you can get the same result with map() with an initial argument of None:
>>> map(None,*[iter(s)]*3)
[(1, 2, 3), (4, 5, 6), (7, 8, 9)]
For more on zip() and map(): http://muffinresearch.co.uk/archives/2007/10/16/python-transposing-lists-with-map-and-zip/

I think one thing that's missed in all the answers (probably obvious to those familiar with iterators) but not so obvious to others is -
Since we have the same iterator, it gets consumed and the remaining elements are used by the zip. So if we simply used the list and not the iter
eg.
l = range(9)
zip(*([l]*3)) # note: not an iter here, the lists are not emptied as we iterate
# output
[(0, 0, 0), (1, 1, 1), (2, 2, 2), (3, 3, 3), (4, 4, 4), (5, 5, 5), (6, 6, 6), (7, 7, 7), (8, 8, 8)]
Using iterator, pops the values and only keeps remaining available, so for zip once 0 is consumed 1 is available and then 2 and so on. A very subtle thing, but quite clever!!!

iter(s) returns an iterator for s.
[iter(s)]*n makes a list of n times the same iterator for s.
So, when doing zip(*[iter(s)]*n), it extracts an item from all the three iterators from the list in order. Since all the iterators are the same object, it just groups the list in chunks of n.

One word of advice for using zip this way. It will truncate your list if it's length is not evenly divisible. To work around this you could either use itertools.izip_longest if you can accept fill values. Or you could use something like this:
def n_split(iterable, n):
num_extra = len(iterable) % n
zipped = zip(*[iter(iterable)] * n)
return zipped if not num_extra else zipped + [iterable[-num_extra:], ]
Usage:
for ints in n_split(range(1,12), 3):
print ', '.join([str(i) for i in ints])
Prints:
1, 2, 3
4, 5, 6
7, 8, 9
10, 11

Unwinding layers of "cleverness", you may find this equivalent spelling easier to follow:
x = iter(s)
for a, b, c in zip(*([x] * n)):
print(a, b, c)
which is, in turn, equivalent to the even less-clever:
x = iter(accounts_iter)
for a, b, c in zip(x, x, x):
print(a, b, c)
Now it should start to become clear. There is only a single iterator object, x. On each iteration, zip(), under the covers, calls next(x) 3 times, once for each iterator object passed to it. But it's the same iterator object here each time. So it delivers the first 3 next(x) results, and leaves the shared iterator object waiting to deliver its 4th result next. Lather, rinse, repeat.
BTW, I suspect you're parsing *([iter(x)]*n) incorrectly in your head. The trailing *n happens first, and then the prefix * is applied to the n-element list *n created. f(*iterable) is a shortcut for calling f() with a variable number of arguments, one for each object iterable delivers.

I needed to break down each partial step to really internalize how it is working. My notes from the REPL:
>>> # refresher on using list multiples to repeat item
>>> lst = list(range(15))
>>> lst
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
>>> # lst id value
>>> id(lst)
139755081359872
>>> [id(x) for x in [lst]*3]
[139755081359872, 139755081359872, 139755081359872]
# replacing lst with an iterator of lst
# It's the same iterator three times
>>> [id(x) for x in [iter(lst)]*3 ]
[139755085005296, 139755085005296, 139755085005296]
# without starred expression zip would only see single n-item list.
>>> print([iter(lst)]*3)
[<list_iterator object at 0x7f1b440837c0>, <list_iterator object at 0x7f1b440837c0>, <list_iterator object at 0x7f1b440837c0>]
# Must use starred expression to expand n arguments
>>> print(*[iter(lst)]*3)
<list_iterator object at 0x7f1b4418b1f0> <list_iterator object at 0x7f1b4418b1f0> <list_iterator object at 0x7f1b4418b1f0>
# by repeating the same iterator, n-times,
# each pass of zip will call the same iterator.__next__() n times
# this is equivalent to manually calling __next__() until complete
>>> iter_lst = iter(lst)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
(0, 1, 2)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
(3, 4, 5)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
(6, 7, 8)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
(9, 10, 11)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
(12, 13, 14)
>>> ((iter_lst.__next__(), iter_lst.__next__(), iter_lst.__next__()))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
# all together now!
# continuing with same iterator multiple times in list
>>> print(*[iter(lst)]*3)
<list_iterator object at 0x7f1b4418b1f0> <list_iterator object at 0x7f1b4418b1f0> <list_iterator object at 0x7f1b4418b1f0>
>>> zip(*[iter(lst)]*3)
<zip object at 0x7f1b43f14e00>
>>> list(zip(*[iter(lst)]*3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 10, 11), (12, 13, 14)]
# NOTE: must use list multiples. Explicit listing creates 3 unique iterators
>>> [iter(lst)]*3 == [iter(lst), iter(lst), iter(lst)]
False
>>> list(zip(*[[iter(lst), iter(lst), iter(lst)]))
[(0, 0, 0), (1, 1, 1), (2, 2, 2), (3, 3, 3), ....

It is probably easier to see what is happening in python interpreter or ipython with n = 2:
In [35]: [iter("ABCDEFGH")]*2
Out[35]: [<iterator at 0x6be4128>, <iterator at 0x6be4128>]
So, we have a list of two iterators which are pointing to the same iterator object. Remember that iter on a object returns an iterator object and in this scenario, it is the same iterator twice due to the *2 python syntactic sugar. Iterators also run only once.
Further, zip takes any number of iterables (sequences are iterables) and creates tuple from i'th element of each of the input sequences. Since both iterators are identical in our case, zip moves the same iterator twice for each 2-element tuple of output.
In [41]: help(zip)
Help on built-in function zip in module __builtin__:
zip(...)
zip(seq1 [, seq2 [...]]) -> [(seq1[0], seq2[0] ...), (...)]
Return a list of tuples, where each tuple contains the i-th element
from each of the argument sequences. The returned list is truncated
in length to the length of the shortest argument sequence.
The unpacking (*) operator ensures that the iterators run to exhaustion which in this case is until there is not enough input to create a 2-element tuple.
This can be extended to any value of n and zip(*[iter(s)]*n) works as described.

x = [1,2,3,4,5,6,7,8,9]
zip(*[iter(x)] * 3)
is the same as:
x = [1,2,3,4,5,6,7,8,9]
iter_var = iter(x)
zip(iter_var,iter_var,iter_var)
Each time zip() gets the next value in iter_var it moves to the next value of x.
Try running next(iter_var) to see how this works.