Using information from the previous step in generators? - python

I have a generator which tries to mimic realtime. This generator makes sure that the user has no access to the future but only to current time.
To simplify my case i use this generator
def generator(n):
for x in range(n):
yield [[x],[x+3]]
if run for n = 5 the generator returns:
[[0], [3]]
[[1], [4]]
[[2], [5]]
[[3], [6]]
[[4], [7]]
I want to be able to combine the elements of each generator iteration with the elements of previous generator iteration to compute sum() of inner lists
case 1:
sum([0]), sum([3])
case 2:
sum([0,1]), sum([3,4])
case 3:
sum([0,1,2]), sum([3,4,5])
...
case LAST
sum([0,1,2,3,4]), sum([3,4,5,6,7])
I don't see how this can be achieved by using:
for x in generator(5):
do sum operation
the values that I will use in the for loop will be gone for the next iteration.
Please do not focus on the numbers and the results, but mainly on the logic and algorithm behind a possible solution. In this case for me it is important to be able to preserve the situation where access to future data is not allowed, only data from the past can be used for calculations, however the data that I consider from the past is already gone when the for iteration is ended!
Any solution? Suggestions?
Thanks in advance!

l1, l2 = [], []
for x1, x2 in generator(5):
l1.extend(x1)
l2.extend(x2)
print sum(l1), sum(l2)
Seems pretty straightforward. It's not like the generator can force you to forget what it gave you. If the operation you want to do is something as simple as sum, you don't even need to keep all the old data, just its sum.
sum1 = sum2 = 0
for x1, x2 in generator(5)
sum1 += x1[0]
sum2 += x2[0]
If the number of lists you need is dynamic, that's easy to handle:
lists = [[] for _ in xrange(numberoflists)]
for subtuple in generator(5):
for element, sublist in zip(subtuple, lists):
sublist.extend(element)
do_whatever_with(map(sum, lists))

I hope I understood this correctly, but are you trying to do something like this?
>>> tmp = [[],[]]
>>> for x in generator(5):
tmp[0] += x[0]
tmp[1] += x[1]
print sum(tmp[0]),sum(tmp[1])
0, 3
1, 7
3, 12
6, 18
10, 25
>>> tmp
[[0, 1, 2, 3, 4], [3, 4, 5, 6, 7]]
Or if you don't want to save the entire list:
>>> tmp = [0,0]
>>> for x in generator(5):
tmp[0] += x[0][0]
tmp[1] += x[1][0]
print tmp[0],tmp[1]
0, 3
1, 7
3, 12
6, 18
10, 25

Related

How to add a element into array from location L to R?

Suppose I have an array A of size N and want to perform Q queries and each query is the type -
"X L R" where X is an element that I want to add in array A from location L to R.
Mathematical format like-
N=5
A=[1 4 3 2 4]
Q=2
X L R=[[5 1 2],
[-5 1 3]]
my algorithm for this problem is like this-
#python code
N=int(input())
A=list(map(int,input().split()))
Q=int(input())
L=[list(map(int,input().split())) for i in range(Q)]
for i in range(Q):
for j in range(L[i][1]-1,L[i][2]):
A[j]=A[j]+L[i][0]
but my code takes a long time hence I want to reduce the time of this code.
How I can reduce the time of this code anyone has any idea please let me know.
Thank you.
What you basically need is a difference array. That would allow you to get Q update queries done in O(1) time each, plus O(N) complexity to recreate the original array.
n = 5
a = [1, 4, 3, 2, 4]
q = 2
xlr = [[5, 1, 2], [-5, 1, 3]]
for item in xlr:
# create the list to be inserted first
to_add = [item[0]]*(item[2]-item[1])
a[item[1]:item[1]] = to_add
print(a)
Inside the for loop, I first create the list that needs to be inserted at the desired location. After that, I set the list that needs to be inserted at the position identified by the element xlr[i][1].
Generated output - [1, -5, -5, 5, 4, 3, 2, 4]

How to parse these operations through lists?

Program description:
Program accepts a list l containing other lists. Output l where lists with length greater than 3 will be changed accordingly: the element with index 3 is going to be a sum of removed elements (from third to the end).
My solution:
l = [[1,2], [3,4,4,3,1], [4,1,4,5]]
s = 0
for i in range(len(l)-1):
if len(l[i]) > 3:
for j in range(3,len(l[i])-1):
s += l[i][j]
l[i].remove(l[i][j])
l[i].insert(len(l[i]),s)
l
Test:
Input: [[1,2], [3,4,4,3,1], [4,1,4,5]]
Expected Output: [[1, 2], [3, 4, 8], [4, 1, 9]]
Program run:
Input: [[1,2], [3,4,4,3,1], [4,1,4,5]]
Output: [[1, 2], [4, 4, 3, 1, 3], [4, 1, 4, 5]]
Question: I don't understand what can be the source of the problem in this case, why should it add some additional numbers to the end, instead of summ. I will appreciate any help.
remove is the wrong function. You should use del instead. Read the documentation to understand why.
And another bug you have is that you do not reset s. It should be set to 0 in the outer for loop.
But you're making it too complicated. I think it's better to show how you can do it really easy.
for e in l: # No need for range. Just iterate over each element
if len(e) > 3:
e[2]=sum(e[2:]) # Sum all the elements
del(e[3:]) # And remove
Or if you want it as a list comprehension that creates a new list and does not alter the old:
[e[0:2] + [sum(e[2:])] if len(e)>3 else e for e in l]
First of all, remove() is the wrong method, as it deletes by value, not index:
Python list method remove() searches for the given element in the list
and removes the first matching element.
You'd want to use del or pop().
Second of all, you're not slicing all of the elements from the end of the list, but only one value.
remove is reason why your code is not working. (as mentioned by Mat-KH in the other answer)
You can use list comprehension and lambda function to make it a two liner.
func = lambda x: x if len(x) < 3 else x[:2] + [sum(x[2:])]
l = [func(x) for x in l]

Storing the result of a for loop in a array

I am trying to save the result of a nested for loop in a list in python. can someone tell me how to do it?
V is an array containing [1, 2, 3]
while n is the length = 3
and sq is the matrix containing swaps.
i have tried many approaches but whenever i return the result it only gives me one element of the list.
any help would be appreciated. Thankyou
def Permute1(sq,v,n):
for i in range(n):
for j in range(n):
if (sq[i,j]==1):
temp=v[i]
v[i]=v[j]
v[j]=temp
print(v)
results:
[1, 2, 3]
[2, 1, 3]
[3, 1, 2]
[3, 1, 2]
[3, 2, 1]
[3, 2, 1]
I'm not sure what is the utility of d = vhere.
To swap two elements in Python, I recommend:
v[i], v[j] = v[j], v[i]
Declaring an empty list before for loops and append the values (like AkshayNevrekar said) can also be useful, depending on what you need as a result.
def Permute1(sq,v,n):
result=[]
for i in range(n):
for j in range(n):
if (sq[i,j]==1):
temp=v[i]
v[i]=v[j]
v[j]=temp
result += [v]
print(result)
return result
No tested but may be it can help.
Check yield.
You can use this to generate all the permutations and process each one of them & store them in a list.

Most Pythonic way to iteratively build up a list? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I was trying to do something in Python that uses the following general procedure, and I want to know what the best way to approch this is.
First, an initialization step:
Create an item M.
Create a list L and add M to L.
Second, loop through the following:
Create a new item by modifying the last item added to L.
Add the new item to L.
As a simple example, say I want to create a list of lists where the nth list contains the numbers from 1 to n. I could use the following (silly) procedure.
Initially M is [1] and L=[[1]].
Next, modify [1] by adding 2 to it to create the new item [1,2], then add [1,2] to L so L=[[1],[1,2]].
Next, modify [1,2] by adding 3 to it to create the new item [1,2,3], then add [1,2,3] to L so L=[[1],[1,2],[1,2,3]].
Next, modify [1,2,3] by adding 4 to it to create the new item [1,2,3,4], then add [1,2,3,4] to L so L=[[1],[1,2],[1,2,3],[1,2,3,4]].
etc.
I tried a few things, but most of them would modify not just the last item added but also items added to L in previous steps. For the particular problem I was interested in, I did manage to find a solution that behaves properly (at least for small cases), but it seems inelegant, I’m not sure why it works when other things didn’t, and I’m not even confident that it would still behave as desired for large cases. I’m also not confident that I could adapt my approach to similar problems. It's not a case of me not understanding the problem, since I've coded the same thing in other programming languages without issues.
So I’m wondering how more experienced Python programmers would handle this general task.
(I’m omitting my own code in part because I’m new here and I haven’t figured out how to enter it on stackoverflow, but also because it's long-ish and I don’t want help with the particular problem, but rather with how to handle the more general procedure I described above.)
When adding a list object M to another list, you are only adding a reference; continuing to manipulate the list M means you will see those changes reflected through the other reference(s) too:
>>> M = []
>>> resultlist = []
>>> resultlist.append(M)
>>> M is resultlist[0]
True
>>> M.append(1)
>>> resultlist[0]
[1]
>>> M
[1]
Note that M is resultlist[0] is True; it is the same object.
You'd add a copy of M instead:
resultlist.append(M[:])
The whole slice here ([:] means to slice from start to end) creates a new list with a shallow copy of the contents of M.
The generic way to build produce a series L from a continuously altered starting point M is to use a generator function. Your simple add the next number to M series could be implemented as:
def growing_sequence():
M = []
counter = 0
while True:
M.append(counter)
counter += 1
yield M[:]
This will yield ever longer lists each time you iterate, on demand:
>>> gen = growing_sequence()
>>> next(gen)
[0]
>>> next(gen)
[0, 1]
>>> for i, lst in enumerate(gen):
... print i, lst
... if i == 2: break
...
0 [0, 1, 2]
1 [0, 1, 2, 3]
2 [0, 1, 2, 3, 4]
You can do:
M=[1]
L=[M]
for e in range(5):
li=L[-1][:]
li.append(li[-1]+1)
L.append(li)
Or more tersely:
for e in range(5):
L.append(L[-1][:]+[L[-1][-1]+1])
I think that the best way to do this is with a generator. That way, you don't have to deal with list.append, deep-copying lists or any of that nonsense.
def my_generator(max):
for n in range(max+1):
yield list(range(n+1))
Then, you just have to list-ify it:
>>> list(my_generator(5))
[[0], [0,1], [0,1,2], [0,1,2,3], [0,1,2,3,4], [0,1,2,3,4,5]]
This approach is also more flexible if you wanted to make it an infinite generator. Simply switch the for loop for a while true.
This will be based on iterate from Haskell.
iterate :: (a -> a) -> a -> [a]
iterate f x returns an infinite list of repeated applications of f to x:
iterate f x == [x, f x, f (f x), ...]
In Python:
def iterate(f, x):
while True:
yield x
x = f(x)
Example usage:
>>> import itertools.islice
>>> def take(n, iterable):
... return list(islice(iterable, n))
>>> take(4, iterate(lambda x: x + [len(x) + 1], [1]))
[[1], [1, 2], [1, 2, 3], [1, 2, 3, 4]]
To produce a finite list, the type signature (again starting in Haskell just for clarity) could be infiniteFinitely :: (a -> Maybe a) -> a -> [a].
If we were to use list in place of Maybe in Python:
from itertools import takewhile
def iterateFinitely(f, x):
return map(lambda a: a[0], takewhile(len, iterate(lambda y: f(y[0]), [x])))
Example usage:
>>> list(iterateFinitely(lambda x: [x / 2] if x else [], 20))
[20, 10, 5, 2, 1, 0]
Since ending with a falsy value is probably pretty common, you might also add a version of this function that does that.
def iterateUntilFalsy(f, x):
return iterateFinitely(lambda y: [f(y)] if y else [], x)
Example usage:
>>> list(iterateUntilFalsy(lambda x: x / 2, 20))
[20, 10, 5, 2, 1, 0]
>>> list(iterateUntilFalsy(lambda x: x[1:], [1,2,3,4]))
[[1, 2, 3, 4], [2, 3, 4], [3, 4], [4], []]
Try this:
M = [1]
L = [M]
for _ in xrange(3):
L += [L[-1] + [L[-1][-1] + 1]]
After the above code is executed, L will contain [[1], [1, 2], [1, 2, 3], [1, 2, 3, 4]]. Explanation:
The first two lines simply seed the iteration with initial values
The for line states how many loops we want to perform after the initial value has been set, 3 in this case. I'm using _ as the iteration variable because we're not interested in its value, we just want to do a certain number of loops
Now for the interesting part; and remember that in Python a negative index in a list starts counting from the end, so an index of -1 points to the last element.
This: L += … updates the list, appending a new sublist at the end as many times as specified in the loop
This: [L[-1] + …] creates a new sublist by taking the last sublist and adding a new element at the end
And finally this: [L[-1][-1] + 1] obtains the previous last element in the last sublist, adds one to it and returns a single-element list to be concatenated at the end of the previous expression

Splitting Up Lists of Lists by Length in Python

Given the following problem, what is the most efficient (or reasonably efficient) way to do this in Python:
Problem. Given a list of lists,
L = [list_0, list_1, list_2, list_3, ..., list_n]
where len(list_i) <= 3, let's say, for each list inside of L. How can we split up L into L_1, L_2, L_3, where L_1 has only length 1 lists, L_2 has only length 2 lists, and L_3 has only length 3 lists?
Potential Solutions. Here's the best I could do; I've also included a sample set here as well. It runs in around 8.6 seconds on my PC.
import time
# These 4 lines make a large sample list-of-list to test on.
asc_sample0 = [[i] for i in range(500)]
asc_sample1 = [[i,j] for i in range(500) for j in range(20)]
asc_sample2 = [[i,j,k] for i in range(20) for j in range(10) for k in range(20)]
asc_sample = asc_sample0 + asc_sample1 + asc_sample2
start = time.clock()
cells0 = [i for i in asc if len(i) == 1]
cells1 = [i for i in asc if len(i) == 2]
cells2 = [i for i in asc if len(i) == 3]
print time.clock() - start
I also attempted to "pop" elements off and append to lists cells0, etc., but this took significantly longer. I also attempted to append and then remove that element so I could get through in one loop which worked okay when there were, say, 10^10 lists of size 1, but only a few of size 2 and 3, but, in general, it was not efficient.
I'd mostly appreciate some neat ideas. I know that one of the answers will most likely be "Write this in C", but for now I'd just like to look at Python solutions for this.
An old fashioned solution might work better here:
cells0, cells1, cells2 = [], [], []
for lst in asc_sample:
n = len(lst)
if n == 1:
cells0.append(lst)
elif n == 2:
cells1.append(lst)
else:
cells2.append(lst)
This is definitely one of the best because it runs in parallel. Another thing that you should look at though is the itertools.groupby and the built-in filter method.
result = dict()
for lst in L:
result.setdefault(len(lst), []).append(lst)
print result
Output
{
1: [[0], [1], [2], [3]],
2: [[0, 0], [0, 1], [0, 2]],
3: [[0, 0, 0], [0, 0, 1], [0, 0, 2]]
}
Indexing a list/tuple should be faster than doing key lookups. This is about 30% faster than the version given in the question
cells = [],[],[],[] # first list here isn't used, but it's handy for the second version
for i in asc:
cells[len(i)].append(i)
Slightly faster again by extracting the append methods (On larger lists this is almost twice as fast as the OP)
cells = [],[],[],[]
appends = [x.append for x in cells]
for i in asc:
appends[len(i)](i)

Categories

Resources