I refer to list operations:
L = myList + otherList
L = myList.append([5])
L = myList.extend(otherList)
I am curious if there are efficiency differences among these operations.
These are totally different operations.
They have different purposes, so efficiency wouldn't matter. append is used to append a single value to a list, extend is used for multiple values, and the addition is for when you don't want to modify the original list, but to have another list with the extra values added on.
>>> lst = [1, 2, 3]
>>> lst2 = [5, 6]
>>> lst.append(4) # appending
>>> lst
[1, 2, 3, 4]
>>> lst.extend(lst2) # extending
>>> lst
[1, 2, 3, 4, 5, 6]
>>> lst + lst2 # addition
[1, 2, 3, 4, 5, 6, 5, 6]
Also note that list.append and list.extend operate in-place, so assigning the result to a variable will make that variable hold the value None.
Your example here is sort of misleading in the case of append.
>>> l1 = [1,2,3,4]
>>> l1.append([5])
>>> l1
[1, 2, 3, 4, [5]]
Append takes a single item and appends it to the end of the existing list. By passing in an iterable to append, you're adding another list (in this case) within a list.
extend takes an iterable and essentially calls append for each item in the iterable`, adding the items onto the end of the existing list.
The mylist + otherlist is the only interesting case here, as the result of using the + operator creates a new list, using more memory.
Timing them answers your question about efficiency in regards of speed:
import timeit
def first():
mylist + otherlist
def second():
mylist.append(otherlist)
def third():
mylist.extend(otherlist)
for test in (first, second, third):
mylist = [1, 2, 3, 4]
otherlist = [5]
print "%s: %f" % (test, timeit.timeit(test, number=1000000))
On my machine the result was:
<function first at 0x10ff3ba28>: 0.320835
<function second at 0x10ff3baa0>: 0.275077
<function third at 0x10ff3bb18>: 0.284508
Showing that the first example was clearly slowest.
Related
pset = []
def powe(a):
powehelp(a,0)
def powehelp(a, ind):
if len(a) == ind:
pset.append(a)
return
powehelp(a[:], ind+1)
a.pop(ind)
powehelp(a[:],ind)
powe([1,2,3])
print(pset)
This code creates its subset, and in this code I want to know why I can't use powehelp(a, ind+1) instead of powehelp(a[:], ind+1)?
I know that a[:] means get all the values of list a.
When using a, the result is [[], [], [], []].
Your powehelp function uses pop, which means it alters the list it is given.
If you pass a into it, it is a that gets altered, and evidently ends up empty.
a[:] creates a copy of a. If you pass a[:] into powehelp, your original list a is unaffected.
The given a is a list in itself, and so appending a list to a list would make a nested list.
a = [1, 2, 3]
b = []
b.append(a)
b
[[1, 2, 3]]
When you pset.append(a) you insert the list a into the endtail of the list pset so you result is a nested list. If I understand your requirement correctly, you're looking to concatenate your lists, as in:
c = [4, 5, 6]
a + c
[1, 2, 3, 4, 5, 6]
vals= [1]
for j in xrange(i):
vals.append([k for k in f(vals[j])])
This loop appends values to itself over a loop. If I compress this into a list comprehension, it doesn't work because it doesn't "dynamically" extend vals using itself on each iteration -- it processes vals as it is originally framed.
Is there a way to do a one line list comprehension that dynamically appends like this? Based on my research, it looks like maybe I am looking for a reduce function? (the equivalent of a fold)
You can indeed use reduce for this, using the initial list as the third parameter.
>>> def f(lst):
... return [x+1 for x in lst] + [len(lst)]
>>> reduce(lambda lst, i: lst + [f(lst[i])], range(5), [[1]])
[[1], [2, 1], [3, 2, 2], [4, 3, 3, 3], [5, 4, 4, 4, 4], [6, 5, 5, 5, 5, 5]]
(Note that the initial list should probably be [[1]], not [1], otherwise you are passing a number to f in the first iteration, but a list in all following iterations.)
Also note that concerning performance your original loop is probably a bit faster, as the reduce basically has to create two new lists in each iteration, while you just have to append to a list. Personally, I would go with a variation of the loop, removing the (probably useless) inner list comprehension and using [-1] to make clear that you are always using the previous result.
vals = [[1]]
for _ in xrange(n):
vals.append(f(vals[-1]))
Is it possible to pass a slice of a list into a function and modify the list via the slice?
This doesn't seem to work:
def foo(a_list):
a_list[0]='abc'
x=[1,2,3,4]
foo(x[0:2])
I want x to now be x=['abc',2,3,4]
No. A "list slice" in the sense you describe is nothing but a new list. When you do x[0:2], the resulting list does not "know" that it was created as a slice of another list. It's just a new list.
What you could do is pass in the slice object itself, separately from the list:
def foo(a_list, a_slice):
a_list[a_slice]='abc'
>>> foo(x, slice(0, 2))
>>> x
['a', 'b', 'c', 3, 4]
No! Slices create copies of lists (see the documentation). You can do this:
>>> x = [1, 2, 3, 4]
>>> x[1:3] = [7, 8, 9]
>>> X
[1, 7, 8, 9, 4]
But, when you get a new list from a slicing operation, it's a copy, and thus changes to it won't affect the original:
>>> x = [1, 2, 3, 4]
>>> y = x[1:3]
>>> y[0] = 5
>>> y
[5, 3]
>>> x
[1, 2, 3, 4]
def foo(a_list):
a_list[0]='abc'
return a_list
x=[1,2,3,4]
foo(x) #it returns x=['abc',2,3,4]
Assumably you're trying to pass in x[0:2], rather than x[0,2], but the reason it doesn't work is because when you create a slice you are creating a subarray copy of x.
You are not operating on the same instance of that array, what you are doing is passing an entirely new array. Passing in 'x' alone would work, but passing in x[0:2] would not unless you specifically wrote it as x[0:2] = foo(x[0:2]) and had foo() return a_list.
As brenns10 explained, slices create a copy (even in python 3.0) of your origional data.
You could do something like the following:
def foo(x):
x[0] = 'abc'
return x
x = [0, 1, 2, 3]
x[0:2] = foo(x[0:2]) # x = ['abc', 1, 2, 3]
While this gives you the desired outcome, it doesn't exactly work as you would want. This could be problematic if needing to perform large slices as you'd have to perform a lot of copying.
I hope to write the join_lists function to take an arbitrary number of lists and concatenate them. For example, if the inputs are
m = [1, 2, 3]
n = [4, 5, 6]
o = [7, 8, 9]
then we I call print join_lists(m, n, o), it will return [1, 2, 3, 4, 5, 6, 7, 8, 9]. I realize I should use *args as the argument in join_lists, but not sure how to concatenate an arbitrary number of lists. Thanks.
Although you can use something which invokes __add__ sequentially, that is very much the wrong thing (for starters you end up creating as many new lists as there are lists in your input, which ends up having quadratic complexity).
The standard tool is itertools.chain:
def concatenate(*lists):
return itertools.chain(*lists)
or
def concatenate(*lists):
return itertools.chain.from_iterable(lists)
This will return a generator which yields each element of the lists in sequence. If you need it as a list, use list: list(itertools.chain.from_iterable(lists))
If you insist on doing this "by hand", then use extend:
def concatenate(*lists):
newlist = []
for l in lists: newlist.extend(l)
return newlist
Actually, don't use extend like that - it's still inefficient, because it has to keep extending the original list. The "right" way (it's still really the wrong way):
def concatenate(*lists):
lengths = map(len,lists)
newlen = sum(lengths)
newlist = [None]*newlen
start = 0
end = 0
for l,n in zip(lists,lengths):
end+=n
newlist[start:end] = list
start+=n
return newlist
http://ideone.com/Mi3UyL
You'll note that this still ends up doing as many copy operations as there are total slots in the lists. So, this isn't any better than using list(chain.from_iterable(lists)), and is probably worse, because list can make use of optimisations at the C level.
Finally, here's a version using extend (suboptimal) in one line, using reduce:
concatenate = lambda *lists: reduce((lambda a,b: a.extend(b) or a),lists,[])
One way would be this (using reduce) because I currently feel functional:
import operator
from functools import reduce
def concatenate(*lists):
return reduce(operator.add, lists)
However, a better functional method is given in Marcin's answer:
from itertools import chain
def concatenate(*lists):
return chain(*lists)
although you might as well use itertools.chain(*iterable_of_lists) directly.
A procedural way:
def concatenate(*lists):
new_list = []
for i in lists:
new_list.extend(i)
return new_list
A golfed version: j=lambda*x:sum(x,[]) (do not actually use this).
You can use sum() with an empty list as the start argument:
def join_lists(*lists):
return sum(lists, [])
For example:
>>> join_lists([1, 2, 3], [4, 5, 6])
[1, 2, 3, 4, 5, 6]
Another way:
>>> m = [1, 2, 3]
>>> n = [4, 5, 6]
>>> o = [7, 8, 9]
>>> p = []
>>> for (i, j, k) in (m, n, o):
... p.append(i)
... p.append(j)
... p.append(k)
...
>>> p
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
This seems to work just fine:
def join_lists(*args):
output = []
for lst in args:
output += lst
return output
It returns a new list with all the items of the previous lists. Is using + not appropriate for this kind of list processing?
Or you could be logical instead, making a variable (here 'z') equal to the first list passed to the 'join_lists' function
then assigning the items in the list (not the list itself) to a new list to which you'll then be able add the elements of the other lists:
m = [1, 2, 3]
n = [4, 5, 6]
o = [7, 8, 9]
def join_lists(*x):
z = [x[0]]
for i in range(len(z)):
new_list = z[i]
for item in x:
if item != z:
new_list += (item)
return new_list
then
print (join_lists(m, n ,o)
would output:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
For an assignment we were asked to create a function that would reverse all the elements in an arbitrarily nested list. So inputs to the function should return something like this:
>>> seq = [1,[2,[3]]]
>>> print arb_reverse(seq)
[[[3],2],1]
>>> seq = [9,[[],[0,1,[[],[2,[[],3]]]],[],[[[4],5]]]]
>>> print arb_reverse(seq)
[[[[5,[4]]],[],[[[[3,[]],2],[]],1,0],[]],9]
I came up with a recursive solution which works well:
def arb_reverse(seq):
result = []
for element in reversed(seq):
if not is_list(element):
result.append(element)
else:
result.append(arb_reverse(element))
return result
But for a bit of a personal challenge I wanted to create a solution without the use of recursion. One version of this attempt resulted in some curious behavior which I am not understanding. For clarification, I was NOT expecting this version to work properly but the resulting input mutation does not make sense. Here is the iterative version in question:
def arb_reverse(seq):
elements = list(seq) #so input is not mutated, also tried seq[:] just to be thorough
result = []
while elements:
item = elements.pop()
if isinstance(item, list):
item.reverse() #this operation seems to be the culprit
elements += item
else:
result.append(item)
return result
This returns a flattened semi-reversed list (somewhat expected), but the interesting part is what it does to the input (not expected)...
>>> a = [1, [2, [3]]]
>>> arb_reverse(a)
[2, 3, 1]
>>> a
[1, [[3], 2]]
>>> p = [1, [2, 3, [4, [5, 6]]]]
>>> print arb_reverse(p)
[2, 3, 4, 5, 6, 1]
>>> print p
[1, [[[6, 5], 4], 3, 2]]
I was under the impression that by passing the values contained in the input to a variable using list() or input[:] as i did with elements, that I would avoid mutating the input. However, a few print statements later revealed that the reverse method had a hand in mutating the original list. Why is that?
The list() call is making a new list with shallow-copied lists from the original.
Try this (stolen from here):
from copy import deepcopy
listB = deepcopy(listA)
Try running the following code through this tool http://people.csail.mit.edu/pgbovine/python/tutor.html
o1 = [1, 2, 3]
o2 = [4, 5, 6]
l1 = [o1, o2]
l2 = list(l1)
l2[0].reverse()
print l2
print l1
Specifically look at what happens when l2[0].reverse() is called.
You'll see that when you call list() to create a copy of the list, the lists still reference the same objects.