Develop Reference

How to manipulate a list with recursive function?

I'm trying to manipulate a given list in an unusual way (at least for me).
Basically, I have the list a (also image 1), it has the first index as principal. Now, I want to iterate through the other indexes and if a certain value match with one of those in the first index, I want to insert the sublist of this index inside the first one.
I don't know if I was clear enough, but the goal should be the list b (also image 2). I think a recursive function should be used here, but I don't know how. Do you guys think it's possible?
Original list:
a = [[1,2,3],[2,5],[6,3],[10,5]]
Expected Output:
b = [[1,2,[2,5,[10,5]],3,[6,3]]]

You could use a dictionary to record where the first occurrence of each number is found, recording the list in which it was found, and at which index. If then a list is found that has a value that was already encountered, the recorded list can be mutated having the matching list inserted. If there was no match (which is the case for the very first list [1,2,3]), then this list is just appended to the result.
Because insertion into a list will impact other insertion points, I suggest to first collect the insertion actions, and then apply them in reversed order:
Here is the code for that:
def solve(a):
dct = {}
result = []
insertions = []
for lst in a:
found = None
for i, val in enumerate(lst):
if val in dct:
found = val
else:
dct[val] = [lst, i]
if found is None:
result.append(lst)
else:
insertions.append((*dct[found], lst))
for target, i, lst in reversed(insertions):
target.insert(i + 1, lst)
return result
# Example run:
a = [[1,2,3],[2,5],[6,3],[10,5]]
print(solve(a))
Output:
[[1, 2, [2, 5, [10, 5]], 3, [6, 3]]]

Simple way of excluding an element from a calculation on a list?

For example I want to check the correlation coefficient between two lists like:
r = np.corrcoef(list25, list26)[0,1]
but I want to exclude -1's in the lists from the calculation. Is there a simple one-liner way of doing this instead of making a new copies of the lists and iterating through to remove all -1's and such?

There is a one liner solution. It's creating a new list without the ones. It can be done using List Comprehension:
new_list = [x for x in old_list if x != -1]
it basically copies everything that matches the condition from the old list to the new list.
So, for your example:
r = np.corrcoef([x for x in list25 if x != -1], [x for x in list26 if x != -1])[0,1]

Use a generator
def greater_neg_1(items):
for item in items:
if item>-1:
yield item
Usage:
>>> L = [1,-1,2,3,4,-1,4]
>>> list(greater_neg_1(L))
[1, 2, 3, 4, 4]
or:
r = np.corrcoef(greater_neg_1(list25), greater_neg_1(list26))[0,1]
Won't require any extra memory.

If you actually want to remove the -1 from the lists:
while -1 in list25: list25.remove(-1)

Test to check if a list is present inside a python list of lists

Basically, I need to get few permutations of a list. So, the method that I have used is to shuffle the list string randomly to get the permutation and add it to a list, while adding I check if there exists the same permutation in the list. I am not able to implement the check. Here is the code that I have written.
list = [x for x in range(0,max)]
totalperm = 10
perms = []
while(len(perms) <> totalperm):
random.shuffle(list)
if list not in perms:
perms.append(list)
Please let me know what am I missing here.

Use python's builtin set to prevent duplicates:
perms = set()
while len(perms) != totalperm:
random.shuffle(lst)
perms.add(tuple(lst))

When you're shuffling the list, you're modifying it in place. Later, you add a reference to it to your list of perms. Next time through your loop, you shuffle the list in place again. If you look at perms, it will contain n references to your original list.
You probably want to do something like this instead:
shuffled = list[:]
random.shuffle(shuffled)
if shuffled not in perms:
perms.append(shuffled)

This doesn't work because you're working with a reference to the single list the whole time, so it is always in the list after the first time you append it.
Consider:
perms = []
items = [1, 2, 3]
random.shuffle(mylist) # perhaps items is now [2, 1, 3]
perms.append(items) # perms is now [[2, 1, 3]]
random.shuffle(mylist) # perhaps items is now [1, 3, 2]
perms.append(items) # now what is perms?
The answer is, perms has two references to a single list, so it is [1, 3, 2], [1, 3, 2]].
In your code, items and perms[0] are the same object and have the same value. So when you ask if items is in perms the answer is always yes. Because you just put it in there!
The solution is simple: add a copy of the list.
perms.append(items[:])
By the way, you should probably use a set rather than a list for storing your permutations. It's much faster to check membership of a set and attempts to add duplicates are simply ignored. The only downside is that you can't put lists in a set because lists are mutable and therefore unhashable -- fortunately, you can just convert them to tuples, which are immutable and hashable. Then you have:
items = range(0, max) # Python 3: items = list(range(0, max))
perms = set()
nump = 10
while len(perms) < nump:
random.shuffle(items)
perms.add(tuple(items))

[1,2,3] in [[1,2,3], [4,5,6]] == True
Works for me. Is this all of your code?

Your code is going into an infinite loop because you change list in place. Then you test to see if it is in perms, big surprise, it is! So it can never append perms.
Repeat ad nauseum.
Here's how I did it:
import random
max = 4
list = [x for x in range(0, max)]
totalperm = 10
perms = []
while(len(perms) < totalperm):
copiedlist = list[:]
random.shuffle(copiedlist)
if copiedlist not in perms:
perms.append(copiedlist)
print perms
else:
print "It's already there, egads!"

how to safely remove elements from a list in Python

I loop through a list and remove the elements that satisfy my condition. But why doesn't this work, as noted below? Thank you.
>>> a=[ i for i in range(4)]
>>> a
[0, 1, 2, 3]
>>> for e in a:
... if (e > 1) and (e < 4):
... a.remove(e)
...
>>> a
[0, 1, 3]
>>> a=[ i for i in range(4)]
>>> for e in a:
... if (e > -1) and (e < 3):
... a.remove(e)
...
>>> a
[1, 3]

You cannot change something while you're iterating it. The results are weird and counter-intuitive, and nearly never what you want. In fact, many collections explicitly disallow this (e.g. sets and dicts).
Instead, iterate over a copy (for e in a[:]: ...) or, instead of modifying an existing list, filter it to get a new list containing the items you want ([e for e in a if ...]). Note that in many cases, you don't have to iterate again to filter, just merge the filtering with the generation of the data.

Why don't you just do this initially in the list comprehension? E.g.
[i for i in range(4) if i <= 1 or i >= 4]
You can also use this to construct a new list from the existing list, e.g.
[x for x in a if x <= 1 or x >= 4]

The idea of filtering is a good one, however it misses the point which is that some lists may be very large and the number of elements to remove may be very small.
In which case the answer is to remember the list indexes of the elements to remove and then iterate through the list of indexes, sorted from largest to smallest, removing the elements.

The easiest way to visualize it is to think of the iteration working on list-offsets instead of the actual items - do something to the first item, then the second item, then the third item, until it runs out of items. If you change the number of items in the list, it changes the offsets of all the remaining items in the list:
lst = [1,2,3,4]
for item in lst:
if item==2:
lst.remove(item)
else:
print item
print lst
results in
1
4
[1,3,4]
which makes sense if you step through it like so:
[1,2,3,4]
^
first item is not 2, so print it -> 1
[1,2,3,4]
^
second item is 2, so remove it
[1,3,4]
^
third item is 4, so print it -> 4
The only real solution is do not change the number of items in the list while you are iterating over it. Copy the items you want to keep to a new list, or keep track of the values you want to remove and do the remove-by-value in a separate pass.

It is not safe to remove elements from a list while iterating though it. For that exists the filter function. It takes a function(that admits one argument) and an iterable(in this case your list). It returns a new iterable of the same type(list again here) with the elements where the function applied to that element returned True:
In your case you can use a lambda function like this:
a = filter(lambda x: x > 1 and x < 4, range(4))
or if you have the list already:
a = range(4)
a = filter(lambda x: x > 1 and x < 4, a)
remember that if you are using python3 it will return an iterator and not a list.

python: list assignment index out of range

for row in c:
for i in range(len(row)):
if i not in keep:
del row[i]
i am getting this error on the last line:
IndexError: list assignment index out of range
i dont understand how it can be out of range if it exists! please help

If row is a list, then don't forget that deleting an element of a list will move all following elements back one place to fill the gap, so all of the later indices into the list will be off-by-one (and each time you delete another element, the error in each index grows by one). There are a few ways to avoid this problem -- to give one, try iterating through the list backwards.
To respond to how to iterate backwards, you could try using the extra parameters you can pass to range. The first two parameters give the range to iterate over (the lower bound being inclusive, and the upper bound exclusive), and the third parameter is the step:
>>> range(5)
[0, 1, 2, 3, 4]
>>> range(0, 5)
[0, 1, 2, 3, 4]
>>> range(3, 5)
[3, 4]
>>> range(3, 5, -1)
[]
>>> range(5, 3, -1)
[5, 4]
So, in your case, it seems you'd want:
range(len(row) - 1, -1, -1)
Or the easier to read (thanks to viraptor):
reversed(range(len(row))
Alternatively, you could try using list comprehensions (I'm assuming c is a list):
for row_number, row in enumerate(c):
c[row_number] = [x for i, x in enumerate(row) if i in keep]

Maybe you can write it like this
for row in c:
row[:] = [x for i,x in enumerate(row) if i in keep]

You should not change a list while you are iterating over it to prevent such errors.

The index existed at the start of the loop, but once you have deleted an element, the list is shorter, and does not contain the same number of elements. Thus, your indices may be wrong. If you delete from the end of the list forward, that won't invalidate list indices.
But I'm surprised about this pattern; perhaps there is a better way to solve what you're looking for? Maybe set subtraction or something similar would do what you want to do without modifying a list dozens or hundreds of times.

A more pythonic solution would be to use filter():
>>> keep = {1,3,5}
>>> row = [1, 2, 3, 4]
>>> list(filter(keep.__contains__, row))
[1, 3]
As a filtering function we can use keep.__contains__ which corresponds to the in operator of the set. Because of this we only get items from the row which are in keep.
Note: Use row[:] = filter(keep.__contains__, row) for in-place update of row or a list comprehension: c = [filter(keep.__contains__, row) for row in c]