Develop Reference

How to 'format' a list in python [duplicate]

I'm trying to write a program that removes duplicates from a list, but my program keeps throwing the error "list index out of range" on line 5, if n/(sequence[k]) == 1:. I can't figure this out. Am I right in thinking that the possible values of "k" are 0, 1, and 2? How is "sequence" with any of those as the index outside of the possible index range?
def remove_duplicates(sequence):
new_list = sequence
for n in sequence:
for k in range(len(sequence)):
if n/(sequence[k]) == 1:
new_list.remove(sequence[k])
print new_list
remove_duplicates([1,2,3])

I strongly suggest Akavall's answer:
list(set(your_list))
As to why you get out of range errors: Python passes by reference, that is sequence and new_list still point at the same memory location. Changing new_list also changes sequence.
And finally, you are comparing items with themselves, and then remove them. So basically even if you used a copy of sequence, like:
new_list = list(sequence)
or
new_list = sequence[:]
It would return an empty list.

Your error is concurrent modification of the list:
for k in range(len(sequence)):
if n/(sequence[k]) == 1:
new_list.remove(sequence[k])
It may seem removing from new_list shouldn't effect sequence, but you did new_list = sequence at the beginning of the function. This means new_list actually literally is sequence, perhaps what you meant is new_list=list(sequence), to copy the list?
If you accept that they are the same list, the error is obvious. When you remove items, the length, and the indexes change.
P.S. As mentioned in a comment by #Akavall, all you need is:
sequence=list(set(sequence))
To make sequence contain no dupes. Another option, if you need to preserve ordering, is:
from collections import OrderedDict
sequence=list(OrderedDict.fromkeys(sequence))

If you don't like list(set(your_list)) because it's not guaranteed to preserved order, you could grab the OrderedSet recipe and then do:
from ordered_set import OrderedSet
foo = list("face a dead cabbage")
print foo
print list(set(foo)) # Order might change
print list(OrderedSet(foo)) # Order preserved

# like #Akavall suggested
def remove_duplicates(sequence):
# returns unsorted unique list
return list(set(sequence))
# create a list, if ele from input not in that list, append.
def remove_duplicates(sequence):
lst = []
for i in sequence:
if i not in lst:
lst.append(i)
# returns unsorted unique list
return lst

Organize/Formatting the python code to one line

Is there any way to rewrite the below python code in one line
for i in range(len(main_list)):
if main_list[i] != []:
for j in range(len(main_list[i])):
main_list[i][j][6]=main_list[i][j][6].strftime('%Y-%m-%d')
something like below,
[main_list[i][j][6]=main_list[i][j][6].strftime('%Y-%m-%d') for i in range(len(main_list)) if main_list[i] != [] for j in range(len(main_list[i]))]
I got SyntaxError for this.
Actually, i'm trying to storing all the values fetched from table into one list. Since the table contains date method/datatype, my requirement needs to convert it to string as i faced with malformed string error.
So my approach is to convert that element of list from datetime.date() to str. And i got it working. Just wanted it to work with one line

Use the explicit for loop. There's no better option.
A list comprehension is used to create a new list, not to modify certain elements of an existing list.
You may be able to update values via a list comprehension, e.g. [L.__setitem__(i, 'some_value') for i in range(len(L))], but this is not recommended as you are using a side-effect and in the process creating a list of None values which you then discard.
You could also write a convoluted list comprehension with a ternary statement indicating when you meet the 6th element in a 3rd nested sublist. But this will make your code difficult to maintain.
In short, use the for loop.

You're getting a syntax error because you're not allowed to perform assignments within a list comprehension. Python forbids assignments because it is discouraging over complex list comprehensions in favour of for loops.
Obviously you shouldn't do this on one line, but this is how to do it:
import datetime
# Example from your comment:
type1 = "some type"
main_list = [[], [],
[[1, 2, 3, datetime.date(2016, 8, 18), type1],
[3, 4, 5, datetime.date(2016, 8, 18), type1]], [], []]
def fmt_times(lst):
"""Format the fourth value of each element of each non-empty sublist"""
for i in range(len(lst)):
if lst[i] != []:
for j in range(len(lst[i])):
lst[i][j][3] = lst[i][j][3].strftime('%Y-%m-%d')
return lst
def fmt_times_one_line(main_list):
"""Format the fourth value of each element of each non-empty sublist"""
return [[] if main_list[i] == [] else [[main_list[i][j][k] if k != 3 else main_list[i][j][k].strftime('%Y-%m-%d') for k in range(len(main_list[i][j]))] for j in range(len(main_list[i])) ] for i in range(len(main_list))]
import copy
# Deep copy needed because fmt_times modifies the sublists.
assert fmt_times(copy.deepcopy(main_list)) == fmt_times_one_line(main_list)
The list comprehension is a functional thing. If you know how map() works in python or javascript then it's the same thing. In a map() or comprehension we generally don't mutate the data we're mapping over (and python discourages attempting it) so instead we recreate the entire object, substituting only the values we wanted to modify.

One line?
main_list = convert_list(main_list)
You will have to put a few more lines somewhere else though:
def convert_list(main_list):
for i, ml in enumerate(main_list):
if isinstance(ml, list) and len(ml) > 0:
main_list[i] = convert_list(ml)
elif isinstance(ml, datetime.date):
main_list[i] = ml.strftime('%Y-%m-%d')
return main_list
You might be able to whack this together with a list comprehension but it's a terrible idea (for reasons better explained in the other answer).

Remove duplicates from list python

I'm trying to write a program that removes duplicates from a list, but my program keeps throwing the error "list index out of range" on line 5, if n/(sequence[k]) == 1:. I can't figure this out. Am I right in thinking that the possible values of "k" are 0, 1, and 2? How is "sequence" with any of those as the index outside of the possible index range?
def remove_duplicates(sequence):
new_list = sequence
for n in sequence:
for k in range(len(sequence)):
if n/(sequence[k]) == 1:
new_list.remove(sequence[k])
print new_list
remove_duplicates([1,2,3])

I strongly suggest Akavall's answer:
list(set(your_list))
As to why you get out of range errors: Python passes by reference, that is sequence and new_list still point at the same memory location. Changing new_list also changes sequence.
And finally, you are comparing items with themselves, and then remove them. So basically even if you used a copy of sequence, like:
new_list = list(sequence)
or
new_list = sequence[:]
It would return an empty list.

Your error is concurrent modification of the list:
for k in range(len(sequence)):
if n/(sequence[k]) == 1:
new_list.remove(sequence[k])
It may seem removing from new_list shouldn't effect sequence, but you did new_list = sequence at the beginning of the function. This means new_list actually literally is sequence, perhaps what you meant is new_list=list(sequence), to copy the list?
If you accept that they are the same list, the error is obvious. When you remove items, the length, and the indexes change.
P.S. As mentioned in a comment by #Akavall, all you need is:
sequence=list(set(sequence))
To make sequence contain no dupes. Another option, if you need to preserve ordering, is:
from collections import OrderedDict
sequence=list(OrderedDict.fromkeys(sequence))

If you don't like list(set(your_list)) because it's not guaranteed to preserved order, you could grab the OrderedSet recipe and then do:
from ordered_set import OrderedSet
foo = list("face a dead cabbage")
print foo
print list(set(foo)) # Order might change
print list(OrderedSet(foo)) # Order preserved

# like #Akavall suggested
def remove_duplicates(sequence):
# returns unsorted unique list
return list(set(sequence))
# create a list, if ele from input not in that list, append.
def remove_duplicates(sequence):
lst = []
for i in sequence:
if i not in lst:
lst.append(i)
# returns unsorted unique list
return lst

Accessing elements of a list

I have a list of strings, and calling a function on each string which returns a string. The thing I want is to update the string in the list. How can I do that?
for i in list:
func(i)
The function func() returns a string. i want to update the list with this string. How can it be done?

If you need to update your list in place (not create a new list to replace it), you'll need to get indexes that corresponds to each item you get from your loop. The easiest way to do that is to use the built-in enumerate function:
for index, item in enumerate(lst):
lst[index] = func(item)

You can reconstruct the list with list comprehension like this
list_of_strings = [func(str_obj) for str_obj in list_of_strings]
Or, you can use the builtin map function like this
list_of_strings = map(func, list_of_strings)
Note : If you are using Python 3.x, then you need to convert the map object to a list, explicitly, like this
list_of_strings = list(map(func, list_of_strings))
Note 1: You don't have to worry about the old list and its memory. When you make the variable list_of_strings refer a new list by assigning to it, the reference count of the old list reduces by 1. And when the reference count drops to 0, it will be automatically garbage collected.

First, don't call your lists list (that's the built-in list constructor).
The most Pythonic way of doing what you want is a list comprehension:
lst = [func(i) for i in lst]
or you can create a new list:
lst2 = []
for i in lst:
lst2.append(func(i))
and you can even mutate the list in place
for n, i in enumerate(lst):
lst[n] = func(i)
Note: most programmers will be confused by calling the list item i in the loop above since i is normally used as a loop index counter, I'm just using it here for consistency.
You should get used to the first version though, it's much easier to understand when you come back to the code six months from now.
Later you might also want to use a generator...
g = (func(i) for i in lst)
lst = list(g)

You can use map() to do that.
map(func, list)

append/extend list in loop

I would like to extend a list while looping over it:
for idx in xrange(len(a_list)):
item = a_list[idx]
a_list.extend(fun(item))
(fun is a function that returns a list.)
Question:
Is this already the best way to do it, or is something nicer and more compact possible?
Remarks:
from matplotlib.cbook import flatten
a_list.extend(flatten(fun(item) for item in a_list))
should work but I do not want my code to depend on matplotlib.
for item in a_list:
a_list.extend(fun(item))
would be nice enough for my taste but seems to cause an infinite loop.
Context:
I have have a large number of nodes (in a dict) and some of them are special because they are on the boundary.
'a_list' contains the keys of these special/boundary nodes. Sometimes nodes are added and then every new node that is on the boundary needs to be added to 'a_list'. The new boundary nodes can be determined by the old boundary nodes (expresses here by 'fun') and every boundary node can add several new nodes.

Have you tried list comprehensions? This would work by creating a separate list in memory, then assigning it to your original list once the comprehension is complete. Basically its the same as your second example, but instead of importing a flattening function, it flattens it through stacked list comprehensions. [edit Matthias: changed + to +=]
a_list += [x for lst in [fun(item) for item in a_list] for x in lst]
EDIT: To explain what going on.
So the first thing that will happen is this part in the middle of the above code:
[fun(item) for item in a_list]
This will apply fun to every item in a_list and add it to a new list. Problem is, because fun(item) returns a list, now we have a list of lists. So we run a second (stacked) list comprehension to loop through all the lists in our new list that we just created in the original comprehension:
for lst in [fun(item) for item in a_list]
This will allow us to loop through all the lists in order. So then:
[x for lst in [fun(item) for item in a_list] for x in lst]
This means take every x (that is, every item) in every lst (all the lists we created in our original comprehension) and add it to a new list.
Hope this is clearer. If not, I'm always willing to elaborate further.

Using itertools, it can be written as:
import itertools
a_list += itertools.chain(* itertools.imap(fun, a_list))
or, if you're aiming for code golf:
a_list += sum(map(fun, a_list), [])
Alternatively, just write it out:
new_elements = map(fun, a_list) # itertools.imap in Python 2.x
for ne in new_elements:
a_list.extend(ne)

As you want to extend the list, but loop only over the original list, you can loop over a copy instead of the original:
for item in a_list[:]:
a_list.extend(fun(item))

Using generator
original_list = [1, 2]
original_list.extend((x for x in original_list[:]))
# [1, 2, 1, 2]