Develop Reference

Why python call by reference is so unruly

arr = [1]
def f1(lst):
lst.append(2)
print(lst)
lst = 2
print(lst)
f1(arr)
print(arr) # [1,2]
why python call by reference parameter does not change to value?
what does lst variable indeicating when do "lst = 2"
(not connected to arr?)

Assigning lst = 2 doesn't affect the value of arr. In fact, python doesn't do "call by reference" at all.

Annotating your code with comments that might help clear it up:
arr = [1]
def f1(lst):
# lst and arr both refer to the same [1] list at this point.
# Two different and independent names for the same object.
lst.append(2) # appends 2 to [1], aka lst, aka arr
print(lst) # lst/arr is now [1, 2]
lst = 2 # reassign the name 'lst' to the value 2!
# At this point, lst refers to 2 instead of [1, 2].
# lst and arr are no longer connected.
# arr is still [1, 2] even though lst is 2.
print(lst) # indeed, lst is now 2
# but arr is still [1, 2], as seen below:
f1(arr)
print(arr) # [1,2]
https://nedbatchelder.com/text/names.html is highly recommended reading on this topic! The main thing to understand is that lst and arr are just different names that at different points in the code might refer to the same value or different values.
When you call lst.append, you are modifying the value that lst is a name for, which arr also happens to be a name for. When you say lst = 2, you are rebinding the name lst, but you are not modifying the value that it previously referred to (and to which arr still refers).

How to parse these operations through lists?

Program description:
Program accepts a list l containing other lists. Output l where lists with length greater than 3 will be changed accordingly: the element with index 3 is going to be a sum of removed elements (from third to the end).
My solution:
l = [[1,2], [3,4,4,3,1], [4,1,4,5]]
s = 0
for i in range(len(l)-1):
if len(l[i]) > 3:
for j in range(3,len(l[i])-1):
s += l[i][j]
l[i].remove(l[i][j])
l[i].insert(len(l[i]),s)
l
Test:
Input: [[1,2], [3,4,4,3,1], [4,1,4,5]]
Expected Output: [[1, 2], [3, 4, 8], [4, 1, 9]]
Program run:
Input: [[1,2], [3,4,4,3,1], [4,1,4,5]]
Output: [[1, 2], [4, 4, 3, 1, 3], [4, 1, 4, 5]]
Question: I don't understand what can be the source of the problem in this case, why should it add some additional numbers to the end, instead of summ. I will appreciate any help.

remove is the wrong function. You should use del instead. Read the documentation to understand why.
And another bug you have is that you do not reset s. It should be set to 0 in the outer for loop.
But you're making it too complicated. I think it's better to show how you can do it really easy.
for e in l: # No need for range. Just iterate over each element
if len(e) > 3:
e[2]=sum(e[2:]) # Sum all the elements
del(e[3:]) # And remove
Or if you want it as a list comprehension that creates a new list and does not alter the old:
[e[0:2] + [sum(e[2:])] if len(e)>3 else e for e in l]

First of all, remove() is the wrong method, as it deletes by value, not index:
Python list method remove() searches for the given element in the list
and removes the first matching element.
You'd want to use del or pop().
Second of all, you're not slicing all of the elements from the end of the list, but only one value.

remove is reason why your code is not working. (as mentioned by Mat-KH in the other answer)
You can use list comprehension and lambda function to make it a two liner.
func = lambda x: x if len(x) < 3 else x[:2] + [sum(x[2:])]
l = [func(x) for x in l]

Get complement (opposite) of list slice

Is there syntax to get the elements of a list not within a given slice?
Given the slice [1:4] it's easy to get those elements:
>>> l = [1,2,3,4,5]
>>> l[1:4]
[2, 3, 4]
If I want the rest of the list I can do:
>>> l[:1] + l[4:]
[1, 5]
Is there an even more succinct way to do this? I realize that I may be being too needy because this is already very concise.
EDIT: I do not think that this is a duplicate of Invert slice in python because I do not wish to modify my original list.

If you want to modify the list in-place, you can delete the slice:
>>> l = [1, 2, 3, 4, 5]
>>> del l[1:4]
>>> l
[1, 5]
Otherwise your originally suggestion would be the most succinct way. There isn't a way to get the opposite of a list slice using a single slice statement.

Clearly the best solution to create a class to encapsulate some magical behavior that occurs when you use 'c' as the step value. Clearly.
class SuperList(list):
def __getitem__(self, val):
if type(val) is slice and val.step == 'c':
copy = self[:]
copy[val.start:val.stop] = []
return copy
return super(SuperList, self).__getitem__(val)
l = SuperList([1,2,3,4,5])
print l[1:4:'c'] # [1, 5]

[x for i, x in enumerate(l) if i not in range(1, 4)]
Which is less concise. So the answer to your question is no, you can't do it more concisely.

I was looking for some solution for this problem that would allow for proper handling of the step parameter as well.
None of the proposed solution was really viable, so I ended up writing my own:
def complement_slice(items, slice_):
to_exclude = set(range(len(items))[slice_])
step = slice_.step if slice_.step else 1
result = [
item for i, item in enumerate(items) if i not in to_exclude]
if step > 0:
return result
else:
return result[::-1]
ll = [x + 1 for x in range(5)]
# [1, 2, 3, 4, 5]
sl = slice(1, 4)
ll[sl]
# [2, 3, 4]
complement_slice(ll, sl)
# [1, 5]
To the best of my knowledge, it does handle all the corner cases as well, including steps, both positive and negative, as well as repeating values.
I wanted to write it as a generator, but I got annoyed by checking all corner cases for positive/negative/None values for all parameters.
In principle, that is possible, of course.

You can use list comprehension with loop
l = [i for i in l if i not in l[1:4]]

python: list assignment index out of range

for row in c:
for i in range(len(row)):
if i not in keep:
del row[i]
i am getting this error on the last line:
IndexError: list assignment index out of range
i dont understand how it can be out of range if it exists! please help

If row is a list, then don't forget that deleting an element of a list will move all following elements back one place to fill the gap, so all of the later indices into the list will be off-by-one (and each time you delete another element, the error in each index grows by one). There are a few ways to avoid this problem -- to give one, try iterating through the list backwards.
To respond to how to iterate backwards, you could try using the extra parameters you can pass to range. The first two parameters give the range to iterate over (the lower bound being inclusive, and the upper bound exclusive), and the third parameter is the step:
>>> range(5)
[0, 1, 2, 3, 4]
>>> range(0, 5)
[0, 1, 2, 3, 4]
>>> range(3, 5)
[3, 4]
>>> range(3, 5, -1)
[]
>>> range(5, 3, -1)
[5, 4]
So, in your case, it seems you'd want:
range(len(row) - 1, -1, -1)
Or the easier to read (thanks to viraptor):
reversed(range(len(row))
Alternatively, you could try using list comprehensions (I'm assuming c is a list):
for row_number, row in enumerate(c):
c[row_number] = [x for i, x in enumerate(row) if i in keep]

Maybe you can write it like this
for row in c:
row[:] = [x for i,x in enumerate(row) if i in keep]

You should not change a list while you are iterating over it to prevent such errors.

The index existed at the start of the loop, but once you have deleted an element, the list is shorter, and does not contain the same number of elements. Thus, your indices may be wrong. If you delete from the end of the list forward, that won't invalidate list indices.
But I'm surprised about this pattern; perhaps there is a better way to solve what you're looking for? Maybe set subtraction or something similar would do what you want to do without modifying a list dozens or hundreds of times.

A more pythonic solution would be to use filter():
>>> keep = {1,3,5}
>>> row = [1, 2, 3, 4]
>>> list(filter(keep.__contains__, row))
[1, 3]
As a filtering function we can use keep.__contains__ which corresponds to the in operator of the set. Because of this we only get items from the row which are in keep.
Note: Use row[:] = filter(keep.__contains__, row) for in-place update of row or a list comprehension: c = [filter(keep.__contains__, row) for row in c]

Deleting multiple elements from a list

Is it possible to delete multiple elements from a list at the same time? If I want to delete elements at index 0 and 2, and try something like del somelist[0], followed by del somelist[2], the second statement will actually delete somelist[3].
I suppose I could always delete the higher numbered elements first but I'm hoping there is a better way.

For some reason I don't like any of the answers here.
Yes, they work, but strictly speaking most of them aren't deleting elements in a list, are they? (But making a copy and then replacing the original one with the edited copy).
Why not just delete the higher index first?
Is there a reason for this?
I would just do:
for i in sorted(indices, reverse=True):
del somelist[i]
If you really don't want to delete items backwards, then I guess you should just deincrement the indices values which are greater than the last deleted index (can't really use the same index since you're having a different list) or use a copy of the list (which wouldn't be 'deleting' but replacing the original with an edited copy).
Am I missing something here, any reason to NOT delete in the reverse order?

You can use enumerate and remove the values whose index matches the indices you want to remove:
indices = 0, 2
somelist = [i for j, i in enumerate(somelist) if j not in indices]

If you're deleting multiple non-adjacent items, then what you describe is the best way (and yes, be sure to start from the highest index).
If your items are adjacent, you can use the slice assignment syntax:
a[2:10] = []

You can use numpy.delete as follows:
import numpy as np
a = ['a', 'l', 3.14, 42, 'u']
I = [0, 2]
np.delete(a, I).tolist()
# Returns: ['l', '42', 'u']
If you don't mind ending up with a numpy array at the end, you can leave out the .tolist(). You should see some pretty major speed improvements, too, making this a more scalable solution. I haven't benchmarked it, but numpy operations are compiled code written in either C or Fortran.

As a specialisation of Greg's answer, you can even use extended slice syntax. eg. If you wanted to delete items 0 and 2:
>>> a= [0, 1, 2, 3, 4]
>>> del a[0:3:2]
>>> a
[1, 3, 4]
This doesn't cover any arbitrary selection, of course, but it can certainly work for deleting any two items.

As a function:
def multi_delete(list_, *args):
indexes = sorted(list(args), reverse=True)
for index in indexes:
del list_[index]
return list_
Runs in n log(n) time, which should make it the fastest correct solution yet.

So, you essentially want to delete multiple elements in one pass? In that case, the position of the next element to delete will be offset by however many were deleted previously.
Our goal is to delete all the vowels, which are precomputed to be indices 1, 4, and 7. Note that its important the to_delete indices are in ascending order, otherwise it won't work.
to_delete = [1, 4, 7]
target = list("hello world")
for offset, index in enumerate(to_delete):
index -= offset
del target[index]
It'd be a more complicated if you wanted to delete the elements in any order. IMO, sorting to_delete might be easier than figuring out when you should or shouldn't subtract from index.

I'm a total beginner in Python, and my programming at the moment is crude and dirty to say the least, but my solution was to use a combination of the basic commands I learnt in early tutorials:
some_list = [1,2,3,4,5,6,7,8,10]
rem = [0,5,7]
for i in rem:
some_list[i] = '!' # mark for deletion
for i in range(0, some_list.count('!')):
some_list.remove('!') # remove
print some_list
Obviously, because of having to choose a "mark-for-deletion" character, this has its limitations.
As for the performance as the size of the list scales, I'm sure that my solution is sub-optimal. However, it's straightforward, which I hope appeals to other beginners, and will work in simple cases where some_list is of a well-known format, e.g., always numeric...

Here is an alternative, that does not use enumerate() to create tuples (as in SilentGhost's original answer).
This seems more readable to me. (Maybe I'd feel differently if I was in the habit of using enumerate.) CAVEAT: I have not tested performance of the two approaches.
# Returns a new list. "lst" is not modified.
def delete_by_indices(lst, indices):
indices_as_set = set(indices)
return [ lst[i] for i in xrange(len(lst)) if i not in indices_as_set ]
NOTE: Python 2.7 syntax. For Python 3, xrange => range.
Usage:
lst = [ 11*x for x in xrange(10) ]
somelist = delete_by_indices( lst, [0, 4, 5])
somelist:
[11, 22, 33, 66, 77, 88, 99]
--- BONUS ---
Delete multiple values from a list. That is, we have the values we want to delete:
# Returns a new list. "lst" is not modified.
def delete__by_values(lst, values):
values_as_set = set(values)
return [ x for x in lst if x not in values_as_set ]
Usage:
somelist = delete__by_values( lst, [0, 44, 55] )
somelist:
[11, 22, 33, 66, 77, 88, 99]
This is the same answer as before, but this time we supplied the VALUES to be deleted [0, 44, 55].

An alternative list comprehension method that uses list index values:
stuff = ['a', 'b', 'c', 'd', 'e', 'f', 'woof']
index = [0, 3, 6]
new = [i for i in stuff if stuff.index(i) not in index]
This returns:
['b', 'c', 'e', 'f']

here is another method which removes the elements in place. also if your list is really long, it is faster.
>>> a = range(10)
>>> remove = [0,4,5]
>>> from collections import deque
>>> deque((list.pop(a, i) for i in sorted(remove, reverse=True)), maxlen=0)
>>> timeit.timeit('[i for j, i in enumerate(a) if j not in remove]', setup='import random;remove=[random.randrange(100000) for i in range(100)]; a = range(100000)', number=1)
0.1704120635986328
>>> timeit.timeit('deque((list.pop(a, i) for i in sorted(remove, reverse=True)), maxlen=0)', setup='from collections import deque;import random;remove=[random.randrange(100000) for i in range(100)]; a = range(100000)', number=1)
0.004853963851928711

This has been mentioned, but somehow nobody managed to actually get it right.
On O(n) solution would be:
indices = {0, 2}
somelist = [i for j, i in enumerate(somelist) if j not in indices]
This is really close to SilentGhost's version, but adds two braces.

l = ['a','b','a','c','a','d']
to_remove = [1, 3]
[l[i] for i in range(0, len(l)) if i not in to_remove])
It's basically the same as the top voted answer, just a different way of writing it. Note that using l.index() is not a good idea, because it can't handle duplicated elements in a list.

Remove method will causes a lot of shift of list elements. I think is better to make a copy:
...
new_list = []
for el in obj.my_list:
if condition_is_true(el):
new_list.append(el)
del obj.my_list
obj.my_list = new_list
...

technically, the answer is NO it is not possible to delete two objects AT THE SAME TIME. However, it IS possible to delete two objects in one line of beautiful python.
del (foo['bar'],foo['baz'])
will recusrively delete foo['bar'], then foo['baz']

we can do this by use of a for loop iterating over the indexes after sorting the index list in descending order
mylist=[66.25, 333, 1, 4, 6, 7, 8, 56, 8769, 65]
indexes = 4,6
indexes = sorted(indexes, reverse=True)
for i in index:
mylist.pop(i)
print mylist

For the indices 0 and 2 from listA:
for x in (2,0): listA.pop(x)
For some random indices to remove from listA:
indices=(5,3,2,7,0)
for x in sorted(indices)[::-1]: listA.pop(x)

I wanted to a way to compare the different solutions that made it easy to turn the knobs.
First I generated my data:
import random
N = 16 * 1024
x = range(N)
random.shuffle(x)
y = random.sample(range(N), N / 10)
Then I defined my functions:
def list_set(value_list, index_list):
index_list = set(index_list)
result = [value for index, value in enumerate(value_list) if index not in index_list]
return result
def list_del(value_list, index_list):
for index in sorted(index_list, reverse=True):
del(value_list[index])
def list_pop(value_list, index_list):
for index in sorted(index_list, reverse=True):
value_list.pop(index)
Then I used timeit to compare the solutions:
import timeit
from collections import OrderedDict
M = 1000
setup = 'from __main__ import x, y, list_set, list_del, list_pop'
statement_dict = OrderedDict([
('overhead', 'a = x[:]'),
('set', 'a = x[:]; list_set(a, y)'),
('del', 'a = x[:]; list_del(a, y)'),
('pop', 'a = x[:]; list_pop(a, y)'),
])
overhead = None
result_dict = OrderedDict()
for name, statement in statement_dict.iteritems():
result = timeit.timeit(statement, number=M, setup=setup)
if overhead is None:
overhead = result
else:
result = result - overhead
result_dict[name] = result
for name, result in result_dict.iteritems():
print "%s = %7.3f" % (name, result)
Output
set = 1.711
del = 3.450
pop = 3.618
So the generator with the indices in a set was the winner. And del is slightly faster then pop.

You can use this logic:
my_list = ['word','yes','no','nice']
c=[b for i,b in enumerate(my_list) if not i in (0,2,3)]
print c

Another implementation of the idea of removing from the highest index.
for i in range(len(yourlist)-1, -1, -1):
del yourlist(i)

You may want to simply use np.delete:
list_indices = [0, 2]
original_list = [0, 1, 2, 3]
new_list = np.delete(original_list, list_indices)
Output
array([1, 3])
Here, the first argument is the original list, the second is the index or a list of indices you want to delete.
There is a third argument which you can use in the case of having ndarrays: axis (0 for rows and 1 for columns in case of ndarrays).

I can actually think of two ways to do it:
slice the list like (this deletes the 1st,3rd and 8th elements)
somelist = somelist[1:2]+somelist[3:7]+somelist[8:]
do that in place, but one at a time:
somelist.pop(2)
somelist.pop(0)

You can do that way on a dict, not on a list. In a list elements are in sequence. In a dict they depend only on the index.
Simple code just to explain it by doing:
>>> lst = ['a','b','c']
>>> dct = {0: 'a', 1: 'b', 2:'c'}
>>> lst[0]
'a'
>>> dct[0]
'a'
>>> del lst[0]
>>> del dct[0]
>>> lst[0]
'b'
>>> dct[0]
Traceback (most recent call last):
File "<pyshell#19>", line 1, in <module>
dct[0]
KeyError: 0
>>> dct[1]
'b'
>>> lst[1]
'c'
A way to "convert" a list in a dict is:
>>> dct = {}
>>> for i in xrange(0,len(lst)): dct[i] = lst[i]
The inverse is:
lst = [dct[i] for i in sorted(dct.keys())]
Anyway I think it's better to start deleting from the higher index as you said.

To generalize the comment from #sth. Item deletion in any class, that implements abc.MutableSequence, and in list in particular, is done via __delitem__ magic method. This method works similar to __getitem__, meaning it can accept either an integer or a slice. Here is an example:
class MyList(list):
def __delitem__(self, item):
if isinstance(item, slice):
for i in range(*item.indices(len(self))):
self[i] = 'null'
else:
self[item] = 'null'
l = MyList(range(10))
print(l)
del l[5:8]
print(l)
This will output
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 'null', 'null', 'null', 8, 9]

Importing it only for this reason might be overkill, but if you happen to be using pandas anyway, then the solution is simple and straightforward:
import pandas as pd
stuff = pd.Series(['a','b','a','c','a','d'])
less_stuff = stuff[stuff != 'a'] # define any condition here
# results ['b','c','d']

some_list.remove(some_list[max(i, j)])
Avoids sorting cost and having to explicitly copy list.

None of the answers offered so far performs the deletion in place in O(n) on the length of the list for an arbitrary number of indices to delete, so here's my version:
def multi_delete(the_list, indices):
assert type(indices) in {set, frozenset}, "indices must be a set or frozenset"
offset = 0
for i in range(len(the_list)):
if i in indices:
offset += 1
elif offset:
the_list[i - offset] = the_list[i]
if offset:
del the_list[-offset:]
# Example:
a = [0, 1, 2, 3, 4, 5, 6, 7]
multi_delete(a, {1, 2, 4, 6, 7})
print(a) # prints [0, 3, 5]

I tested the suggested solutions with perfplot and found that NumPy's
np.delete(lst, remove_ids)
is the fastest solution if the list is longer than about 100 entries. Before that, all solutions are around 10^-5 seconds. The list comprehension seems simple enough then:
out = [item for i, item in enumerate(lst) if i not in remove_ids]
Code to reproduce the plot:
import perfplot
import random
import numpy as np
import copy
def setup(n):
lst = list(range(n))
random.shuffle(lst)
# //10 = 10%
remove_ids = random.sample(range(n), n // 10)
return lst, remove_ids
def if_comprehension(lst, remove_ids):
return [item for i, item in enumerate(lst) if i not in remove_ids]
def del_list_inplace(lst, remove_ids):
out = copy.deepcopy(lst)
for i in sorted(remove_ids, reverse=True):
del out[i]
return out
def del_list_numpy(lst, remove_ids):
return np.delete(lst, remove_ids)
b = perfplot.bench(
setup=setup,
kernels=[if_comprehension, del_list_numpy, del_list_inplace],
n_range=[2**k for k in range(20)],
)
b.save("out.png")
b.show()

How about one of these (I'm very new to Python, but they seem ok):
ocean_basin = ['a', 'Atlantic', 'Pacific', 'Indian', 'a', 'a', 'a']
for i in range(1, (ocean_basin.count('a') + 1)):
ocean_basin.remove('a')
print(ocean_basin)
['Atlantic', 'Pacific', 'Indian']
ob = ['a', 'b', 4, 5,'Atlantic', 'Pacific', 'Indian', 'a', 'a', 4, 'a']
remove = ('a', 'b', 4, 5)
ob = [i for i in ob if i not in (remove)]
print(ob)
['Atlantic', 'Pacific', 'Indian']

I put it all together into a list_diff function that simply takes two lists as inputs and returns their difference, while preserving the original order of the first list.
def list_diff(list_a, list_b, verbose=False):
# returns a difference of list_a and list_b,
# preserving the original order, unlike set-based solutions
# get indices of elements to be excluded from list_a
excl_ind = [i for i, x in enumerate(list_a) if x in list_b]
if verbose:
print(excl_ind)
# filter out the excluded indices, producing a new list
new_list = [i for i in list_a if list_a.index(i) not in excl_ind]
if verbose:
print(new_list)
return(new_list)
Sample usage:
my_list = ['a', 'b', 'c', 'd', 'e', 'f', 'woof']
# index = [0, 3, 6]
# define excluded names list
excl_names_list = ['woof', 'c']
list_diff(my_list, excl_names_list)
>> ['a', 'b', 'd', 'e', 'f']