I want to generate or return an append-accumulated list from a given list (or iterator). For a list like [1, 2, 3, 4], I would like to get, [1], [1, 2], [1, 2, 3] and [1, 2, 3, 4]. Like so:
>>> def my_accumulate(iterable):
... grow = []
... for each in iterable:
... grow.append(each)
... yield grow
...
>>> for x in my_accumulate(some_list):
... print x # or something more useful
...
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
This works but is there an operation I could use with itertools.accumulate to facilitate this? (I'm on Python2 but the pure-python implementation/equivalent has been provided in the docs.)
Another problem I have with my_accumulate is that it doesn't work well with list(), it outputs the entire some_list for each element in the list:
>>> my_accumulate(some_list)
<generator object my_accumulate at 0x0000000002EC3A68>
>>> list(my_accumulate(some_list))
[[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
Option 1:
I wrote my own appending accumulator function to use with itertools.accumulate but considering the LoC and final useful-ness, it seems like a waste of effort, with my_accumulate being more useful, (though may fail in case of empty iterables and consumes more memory since grow keeps growing):
>>> def app_acc(first, second):
... if isinstance(first, list):
... first.append(second)
... else:
... first = [first, second]
... return first
...
>>> for x in accumulate(some_list, app_acc):
... print x
...
1
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
>>> list(accumulate(some_list, app_acc)) # same problem again with list
[1, [1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
(and the first returned elem is not a list, just a single item)
Option 2: Figured it would be easier to just do incremental slicing but using the ugly iterate over list length method:
>>> for i in xrange(len(some_list)): # the ugly iterate over list length method
... print some_list[:i+1]
...
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
The easiest way to use accumulate is to make each item in the iterable a list with a single item and then the default function works as expected:
from itertools import accumulate
acc = accumulate([el] for el in range(1, 5))
res = list(acc)
# [[1], [1, 2], [1, 2, 3], [1, 2, 3, 4]]
Related
Let's use a simple example: say I have a list of lists
ll = [[1, 2], [1, 3], [2, 3], [1, 2, 3], [2, 3, 4]]
and I want to find all longest lists, which means all lists that maximize the len function. Of course we can do
def func(x):
return len(x)
maxlen = func(max(ll, key=lambda x: func(x)))
res = [l for l in ll if func(l) == maxlen]
print(res)
Output
[[1, 2, 3], [2, 3, 4]]
But I wonder if there are more efficient way to do this, especially when the function is very expensive or the list is very long. Any suggestions?
From a computer science/algorithms perspective, this is a very classical "reduce" problem.
so, pseudocode. It's honestly very straightforward.
metric():= a mapping from elements to non-negative numbers
winner = []
maxmetric = 0
for element in ll:
if metric(element) larger than maxmetric:
winner = [ element ]
maxmetric = metric(element)
else if metric(element) equal to maxmetric:
append element to winner
when the function is very expensive
Note that you do compute func(x) for each element twice, first there
maxlen = func(max(ll, key=lambda x: func(x)))
then there
res = [l for l in ll if func(l) == maxlen]
so it would be beneficial to store what was already computed. functools.lru_cache allow that easily just replace
def func(x):
return len(x)
using
import functools
#functools.lru_cache(maxsize=None)
def func(x):
return len(x)
However, beware as due to way how data are stored argument(s) must be hashable, so in your example you would first need convert list e.g. to tuples i.e.
ll = [(1, 2), (1, 3), (2, 3), (1, 2, 3), (2, 3, 4)]
See descripiton in docs for further discussion
Is not OK use dictionary like below, (this is O(n))
ll = [[1, 2], [1, 3], [2, 3], [1, 2, 3], [2, 3, 4]]
from collections import defaultdict
dct = defaultdict(list)
for l in ll:
dct[len(l)].append(l)
dct[max(dct)]
Output:
[[1, 2, 3], [2, 3, 4]]
>>> dct
defaultdict(list, {2: [[1, 2], [1, 3], [2, 3]], 3: [[1, 2, 3], [2, 3, 4]]})
OR use setdefault and without defaultdict like below:
ll = [[1, 2], [1, 3], [2, 3], [1, 2, 3], [2, 3, 4]]
dct = {}
for l in ll:
dct.setdefault(len(l), []).append(l)
Output:
>>> dct
{2: [[1, 2], [1, 3], [2, 3]], 3: [[1, 2, 3], [2, 3, 4]]}
I have the foll. list in python:
[1, 2, 3, 4]
Is there an python itertools function that results in foll:
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
This is trivial without itertools:
def fall(it):
ls = []
for x in it:
ls.append(x)
yield ls
for x in fall(xrange(20)):
print x
Note that this works with any iterable, not just a list.
If you still want itertools, something like this should work (py3):
for x in itertools.accumulate(map(lambda x: [x], it)):
print(x)
Again, it's lazy and works with any iterable.
There isn't anything in itertools, that I can think of, but this should work:
def incremental(L):
for i in range(1, len(L)+1):
yield L[:i]
Output:
In [53]: print(*incremental([1, 2, 3, 4]), sep='\n')
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
This can be written as one-liner using list comprehension:
>>> [ list[:x+1] for x in range(len(list)) ]
[[1], [1, 2], [1, 2, 3], [1, 2, 3, 4]]
If it is must to use itertools, you may use itertools.islice as:
from itertools import islice
my_list = [1, 2, 3, 4]
for i in range(1, len(my_list)+1):
print list(islice(my_list, i))
However there is absolutely no need to use itertools here. You may achieve this via simple list slicing as:
for i in range(len(my_list)):
print my_list[:i+1]
Both of the above solutions will print the result as:
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
One line solution, using partial and islice,
from itertools import islice
from functools import partial
my_list = [1, 2, 3, 4]
[list(l) for l in map(partial(islice, my_list), range(1,len(my_list)+1))]
you get,
[[1], [1, 2], [1, 2, 3], [1, 2, 3, 4]]
in other words,
from itertools import islice
from functools import partial
my_list = [1, 2, 3, 4]
p = partial(islice, my_list)
for i in range(1,5):
print(list(p(i)))
you get,
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
You don't need itertools, just use a map:
>>> l = [1, 2, 3, 4]
>>> for sub_list in map(lambda index: l[:index + 1], range(len(l))):
... print sub_list
Consider the following function to permute numbers in a list:
def permute(numbers, N=0):
# base case
if N == len(numbers):
print numbers
return
for i in range(len(numbers)-N):
# swapping relevant elements
element=numbers.pop(N+i)
numbers.insert(N,element)
# recursive call
permute(numbers, N+1)
# swapping back relevant elements when backtracking
element=numbers.pop(N)
numbers.insert(N+i,element)
numbers=[1,2,3]
permute(numbers)
Why is it that when I run the above code, it produces the correct output:
[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 1, 2]
[3, 2, 1]
but when I'm trying to accumulate the result to a list:
def permute(numbers, permutations, N=0):
# base case
if N == len(numbers):
print numbers
permutations.append(numbers)
return
for i in range(len(numbers)-N):
# swapping relevant elements
element=numbers.pop(N+i)
numbers.insert(N,element)
# recursive call
permute(numbers, permutations, N+1)
# swapping back relevant elements when backtracking
element=numbers.pop(N)
numbers.insert(N+i,element)
numbers=[1,2,3]
permutations=[]
permute(numbers, permutations)
print "-----------"
for p in permutations:
print p
The output is this:
[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 1, 2]
[3, 2, 1]
-----------
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3]
Not quite what I expected...
All of the items you're appending to permutations are actually the same list. Changing one changes all of the others. The easiest fix for this is to make a copy of the list before appending it, so future changes to numbers has no effect on the already-appended results.
def permute(numbers, permutations, N=0):
# base case
if N == len(numbers):
print numbers
permutations.append(numbers[:])
return
Result:
[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 1, 2]
[3, 2, 1]
---------
[1, 2, 3]
[1, 3, 2]
[2, 1, 3]
[2, 3, 1]
[3, 1, 2]
[3, 2, 1]
I'm a Python beginner and I'm currently going through Zed Shaw's course "Learn Python the Hardway"
So, in exercise 32 we are told:
How do you make a 2-dimensional (2D) list?
That's a list in a list like this: [[1,2,3],[4,5,6]]
I did this:
# Extra 1
global_list = [[1, 2, 3]]
inside_list = []
for i in global_list[0]:
inside_list.append(i)
global_list.append(inside_list)
print(global_list)
But I’m not entirely convinced that's the correct way. My question is: Is there a way to get the same result without ever leaving the for i in.... loop?
I also tried this, to no avail.
global_list = [[1, 2, 3]]
inside_list = []
for i in global_list[0]:
inside_list.append(i)
global_list.append(inside_list)
print(global_list)
Thanks in advance for your answers.
Lists can be appended and inserted into a list just like any other object, e.g:
outer_list = []
print(outer_list)
inner_list1 = [1, 2, 3]
outer_list.append(inner_list1)
print(outer_list)
inner_list2 = [4, 5, 6]
outer_list.append(inner_list2)
print(outer_list)
I am not sure if you already went over list comprehension. However, one nice way of doing what you are doing is:
>>> global_list = [[1,2,3]]
>>> global_list.append([i + 3 for i in global_list[0]])
>>> print global_list
[[1, 2, 4], [4, 5, 6]]
The question was "How do you make a 2-dimensional (2D) list?". The answer given was "That's a list in a list like this: [[1,2,3],[4,5,6]]". Literally, that's the answer:
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> print a
[[1, 2, 3], [4, 5, 6]]
You can also do this:
>>> a = [[1, 2, 3]]
>>> a.append([4, 5, 6])
>>> a
[[1, 2, 3], [4, 5, 6]]
You don't need a for loop to append a list inside another list:
>>> a = [[1, 2, 3]]
>>> a[0]
[1, 2, 3]
>>> a.append(a[0])
>>> a
[[1, 2, 3], [1, 2, 3]]
However, this makes the second element of the list of lists the same as the first, so if you change one you change the other:
>>> a[0] is a[1]
True
>>> a[0][0] = 4
>>> a
[[4, 2, 3], [4, 2, 3]]
What you can do to make a copy of the list is list(a[0]):
>>> a = [[1, 2, 3]]
>>> a[0]
[1, 2, 3]
>>> a[0] is a[0]
True
>>> list(a[0])
[1, 2, 3]
>>> a[0] is list(a[0])
False
>>> a.append(list(a[0]))
>>> a
[[1, 2, 3], [1, 2, 3]]
>>> a[0] is a[1]
False
>>> a[0][0] = 4
>>> a
[[4, 2, 3], [1, 2, 3]]
I'm having trouble wrapping my head around dealing with lists within lists in python (I am rather new at programming).
Now how would I access each nested list position and then manipulate the position of the nested list's values to, for example, change their order while maintaining the position of the nested list in the main list.
For example:
if I wanted to go from:
aList = [
[1,2,3,4,3,2],
[2,3,4,5,4,3],
[2,1,2,3,4,3]
]
to here:
new_aList = [
[2,3,4,3,2,1],
[3,4,5,4,3,2],
[3,4,3,2,1,2]
]
I get how to access each list, but I'm experiencing a block as to how I would change the position of the values of the nested lists.
Thanks for any help!
Use a list comprehension to get the new elements, and optionally slice-assign to replace the existing elements in the list.
new_aList = [list(reversed(x)) for x in aList]
aList[:] = [list(reversed(x)) for x in aList]
You could reverse each item like this
>>> aList = [
... [1,2,3,4,3,2],
... [2,3,4,5,4,3],
... [2,1,2,3,4,3]
... ]
>>> aList[0].reverse()
>>> aList[1].reverse()
>>> aList[2].reverse()
>>> aList
[[2, 3, 4, 3, 2, 1], [3, 4, 5, 4, 3, 2], [3, 4, 3, 2, 1, 2]]
But in general it's better to use a loop since aList could have lots of items
>>> aList = [
... [1,2,3,4,3,2],
... [2,3,4,5,4,3],
... [2,1,2,3,4,3]
... ]
>>> for item in aList:
... item.reverse()
...
>>> aList
[[2, 3, 4, 3, 2, 1], [3, 4, 5, 4, 3, 2], [3, 4, 3, 2, 1, 2]]
Both of those methods will modify aList in place, so the unmodified version is destroyed. Heres how you could create a new list and leave aList unchanged
>>> aList = [
... [1,2,3,4,3,2],
... [2,3,4,5,4,3],
... [2,1,2,3,4,3]
... ]
>>> new_aList = []
>>> for item in aList:
... new_aList.append(list(reversed(item)))
...
>>> new_aList
[[2, 3, 4, 3, 2, 1], [3, 4, 5, 4, 3, 2], [3, 4, 3, 2, 1, 2]]
another way to reverse a list is to use this extended slice trick. The -1 means step through the list in steps of -1 ie. backwards.
>>> new_aList = []
>>> for item in aList:
... new_aList.append(item[::-1])
...
>>> new_aList
[[2, 3, 4, 3, 2, 1], [3, 4, 5, 4, 3, 2], [3, 4, 3, 2, 1, 2]]
Instead of explicitly making an empty list and appending to it, it is more usual to write a loop like this as a list comprehension
>>> new_aList = [item[::-1] for item in aList]
>>> new_aList
[[2, 3, 4, 3, 2, 1], [3, 4, 5, 4, 3, 2], [3, 4, 3, 2, 1, 2]]
bList = []
for l in aList:
l.reverse()
bList.append(l)
For more fine-grained manipulation of lists, the position of values can be swapped using tuple unpacking (which neatly avoids having to use temporary variables):
aList[0][1], aList[0][3] = aList[0][3], aList[0][1]
After this operation, the second and fourth values would change places, and so aList[0] would look like this:
[1, 4, 3, 2, 3, 2]