I'm using Python 3.5.
As part of a problem, I'm trying to design a function that takes a list as input and reverts it. So if x = [a, b, c] the function would make x = [c, b, a].
The problem is, I'm not allowed to use any built-in functions, and it has got me stuck. My initial thought was the following loop inside a function:
for revert in range(1, len(x) + 1):
y.append(x[-revert])
And it works. But the problem is I'm using len(x), which I believe is a built-in function, correct?
So I searched around and have made the following very simple code:
y = x[::-1]
Which does exactly what I wanted, but it just seems almost too simple/easy and I'm not sure whether "::" counts as a function.
So I was wondering if anyone had any hints/ideas how to manually design said function? It just seems really hard when you can't use any built-in functions and it has me stuck for quite some time now.
range and len are both built-in functions. Since list methods are accepted, you could do this with insert. It is reeaallyy slow* but it does the job for small lists without using any built-ins:
def rev(l):
r = []
for i in l:
r.insert(0, i)
return r
By continuously inserting at the zero-th position you end up with a reversed version of the input list:
>>> print(rev([1, 2, 3, 4]))
[4, 3, 2, 1]
Doing:
def rev(l):
return l[::-1]
could also be considered a solution. ::-1 (:: has a different result) isn't a function (it's a slice) and [] is, again, a list method. Also, contrasting insert, it is faster and way more readable; just make sure you're able to understand and explain it. A nice explanation of how it works can be found in this S.O answer.
*Reeaaalllyyyy slow, see juanpa.arrivillaga's answer for cool plot and append with pop and take a look at in-place reverse on lists as done in Yoav Glazner's answer.
:: is not a function, it's a python literal. as well as []
How to check if ::, [] are functions or not. Simple,
import dis
a = [1,2]
dis.dis(compile('a[::-1]', '', 'eval'))
1 0 LOAD_NAME 0 (a)
3 LOAD_CONST 0 (None)
6 LOAD_CONST 0 (None)
9 LOAD_CONST 2 (-1)
12 BUILD_SLICE 3
15 BINARY_SUBSCR
16 RETURN_VALUE
If ::,[] were functions, you should find a label CALL_FUNCTION among python instructions executed by a[::-1] statement. So, they aren't.
Look how python instructions looks like when you call a function, lets say list() function
>>> dis.dis(compile('list()', '', 'eval'))
1 0 LOAD_NAME 0 (list)
3 CALL_FUNCTION 0
6 RETURN_VALUE
So, basically
def rev(f):
return f[::-1]
works fine. But, I think you should do something like Jim suggested in his answer if your question is a homework or sent by you teacher. But, you can add this quickest way as a side note.
If you teacher complains about [::-1] notation, show him the example I gave you.
Another way ( just for completeness :) )
def another_reverse(lst):
new_lst = lst.copy() # make a copy if you don't want to ruin lst...
new_lst.reverse() # notice! this will reverse it in place
return new_lst
Here's a solution that doesn't use built-in functions but relies on list methods. It reverse in-place, as implied by your specification:
>>> x = [1,2,3,4]
>>> def reverse(seq):
... temp = []
... while seq:
... temp.append(seq.pop())
... seq[:] = temp
...
>>> reverse(x)
>>> x
[4, 3, 2, 1]
>>>
ETA
Jim, your answer using insert at position 0 was driving me nuts! That solution is quadratic time! You can use append and pop with a temporary list to achieve linear time using simple list methods. See (reverse is in blue, rev is green):
If it feels a little bit like "cheating" using seq[:] = temp, we could always loop over temp and append every item into seq and the time complexity would still be linear but probably slower since it isn't using the C-based internals.
Your example that works:
y = x[::-1]
uses Python slices notation which is not a function in the sense that I assume you're requesting. Essentially :: acts as a separator. A more verbose version of your code would be:
y = x[len(x):None:-1]
or
y = x[start:end:step]
I probably wouldn't be complaining that python makes your life really, really easily.
Edit to be super pedantic. Someone could argue that calling [] at all is using an inbuilt python function because it's really syntactical sugar for the method __getitem__().
x.__getitem__(0) == x[0]
And using :: does make use of the slice() object.
x.__getitem__(slice(len(x), None, -1) == x[::-1]
But... if you were to argue this, anything you write in python would be using inbuilt python functions.
Another way for completeness, range() takes an optional step parameter that will allow you to step backwards through the list:
def reverse_list(l):
return [l[i] for i in range(len(l)-1, -1, -1)]
The most pythonic and efficient way to achieve this is by list slicing. And, since you mentioned you do not need any inbuilt function, it completely suffice your requirement. For example:
>>> def reverse_list(list_obj):
... return list_obj[::-1]
...
>>> reverse_list([1, 3, 5 , 3, 7])
[7, 3, 5, 3, 1]
Just iterate the list from right to left to get the items..
a = [1,2,3,4]
def reverse_the_list(a):
reversed_list = []
for i in range(0, len(a)):
reversed_list.append(a[len(a) - i - 1])
return reversed_list
new_list = reverse_the_list(a)
print new_list
Related
If we take b = [1,2,3] and if we try doing: b+=(4,)
It returns b = [1,2,3,4], but if we try doing b = b + (4,) it doesn't work.
b = [1,2,3]
b+=(4,) # Prints out b = [1,2,3,4]
b = b + (4,) # Gives an error saying you can't add tuples and lists
I expected b+=(4,) to fail as you can't add a list and a tuple, but it worked. So I tried b = b + (4,) expecting to get the same result, but it didn't work.
The problem with "why" questions is that usually they can mean multiple different things. I will try to answer each one I think you might have in mind.
"Why is it possible for it to work differently?" which is answered by e.g. this. Basically, += tries to use different methods of the object: __iadd__ (which is only checked on the left-hand side), vs __add__ and __radd__ ("reverse add", checked on the right-hand side if the left-hand side doesn't have __add__) for +.
"What exactly does each version do?" In short, the list.__iadd__ method does the same thing as list.extend (but because of the language design, there is still an assignment back).
This also means for example that
>>> a = [1,2,3]
>>> b = a
>>> a += [4] # uses the .extend logic, so it is still the same object
>>> b # therefore a and b are still the same list, and b has the `4` added
[1, 2, 3, 4]
>>> b = b + [5] # makes a new list and assigns back to b
>>> a # so now a is a separate list and does not have the `5`
[1, 2, 3, 4]
+, of course, creates a new object, but explicitly requires another list instead of trying to pull elements out of a different sequence.
"Why is it useful for += to do this? It's more efficient; the extend method doesn't have to create a new object. Of course, this has some surprising effects sometimes (like above), and generally Python is not really about efficiency, but these decisions were made a long time ago.
"What is the reason not to allow adding lists and tuples with +?" See here (thanks, #splash58); one idea is that (tuple + list) should produce the same type as (list + tuple), and it's not clear which type the result should be. += doesn't have this problem, because a += b obviously should not change the type of a.
They are not equivalent:
b += (4,)
is shorthand for:
b.extend((4,))
while + concatenates lists, so by:
b = b + (4,)
you're trying to concatenate a tuple to a list
When you do this:
b += (4,)
is converted to this:
b.__iadd__((4,))
Under the hood it calls b.extend((4,)), extend accepts an iterator and this why this also work:
b = [1,2,3]
b += range(2) # prints [1, 2, 3, 0, 1]
but when you do this:
b = b + (4,)
is converted to this:
b = b.__add__((4,))
accept only list object.
From the official docs, for mutable sequence types both:
s += t
s.extend(t)
are defined as:
extends s with the contents of t
Which is different than being defined as:
s = s + t # not equivalent in Python!
This also means any sequence type will work for t, including a tuple like in your example.
But it also works for ranges and generators! For instance, you can also do:
s += range(3)
The "augmented" assignment operators like += were introduced in Python 2.0, which was released in October 2000. The design and rationale are described in PEP 203. One of the declared goals of these operators was the support of in-place operations. Writing
a = [1, 2, 3]
a += [4, 5, 6]
is supposed to update the list a in place. This matters if there are other references to the list a, e.g. when a was received as a function argument.
However, the operation can't always happen in place, since many Python types, including integers and strings, are immutable, so e.g. i += 1 for an integer i can't possibly operate in place.
In summary, augmented assignment operators were supposed to work in place when possible, and create a new object otherwise. To facilitate these design goals, the expression x += y was specified to behave as follows:
If x.__iadd__ is defined, x.__iadd__(y) is evaluated.
Otherwise, if x.__add__ is implemented x.__add__(y) is evaluated.
Otherwise, if y.__radd__ is implemented y.__radd__(x) is evaluated.
Otherwise raise an error.
The first result obtained by this process will be assigned back to x (unless that result is the NotImplemented singleton, in which case the lookup continues with the next step).
This process allows types that support in-place modification to implement __iadd__(). Types that don't support in-place modification don't need to add any new magic methods, since Python will automatically fall back to essentially x = x + y.
So let's finally come to your actual question – why you can add a tuple to a list with an augmented assignment operator. From memory, the history of this was roughly like this: The list.__iadd__() method was implemented to simply call the already existing list.extend() method in Python 2.0. When iterators were introduced in Python 2.1, the list.extend() method was updated to accept arbitrary iterators. The end result of these changes was that my_list += my_tuple worked starting from Python 2.1. The list.__add__() method, however, was never supposed to support arbitrary iterators as the right-hand argument – this was considered inappropriate for a strongly typed language.
I personally think the implementation of augmented operators ended up being a bit too complex in Python. It has many surprising side effects, e.g. this code:
t = ([42], [43])
t[0] += [44]
The second line raises TypeError: 'tuple' object does not support item assignment, but the operation is successfully performed anyway – t will be ([42, 44], [43]) after executing the line that raises the error.
Most people would expect X += Y to be equivalent to X = X + Y. Indeed, the Python Pocket Reference (4th ed) by Mark Lutz says on page 57 "The following two formats are roughly equivalent: X = X + Y , X += Y". However, the people who specified Python did not make them equivalent. Possibly that was a mistake which will result in hours of debugging time by frustrated programmers for as long as Python remains in use, but it's now just the way Python is. If X is a mutable sequence type, X += Y is equivalent to X.extend( Y ) and not to X = X + Y.
As it's explained here, if array doesn't implement __iadd__ method, the b+=(4,) would be just a shorthanded of b = b + (4,) but obviously it's not, so array does implement __iadd__ method. Apparently the implementation of __iadd__ method is something like this:
def __iadd__(self, x):
self.extend(x)
However we know that the above code is not the actual implementation of __iadd__ method but we can assume and accept that there's something like extend method, which accepts tupple inputs.
So I just came across what seems to me like a strange Python feature and wanted some clarification about it.
The following array manipulation somewhat makes sense:
p = [1,2,3]
p[3:] = [4]
p = [1,2,3,4]
I imagine it is actually just appending this value to the end, correct?
Why can I do this, however?
p[20:22] = [5,6]
p = [1,2,3,4,5,6]
And even more so this:
p[20:100] = [7,8]
p = [1,2,3,4,5,6,7,8]
This just seems like wrong logic. It seems like this should throw an error!
Any explanation?
-Is it just a weird thing Python does?
-Is there a purpose to it?
-Or am I thinking about this the wrong way?
Part of question regarding out-of-range indices
Slice logic automatically clips the indices to the length of the sequence.
Allowing slice indices to extend past end points was done for convenience. It would be a pain to have to range check every expression and then adjust the limits manually, so Python does it for you.
Consider the use case of wanting to display no more than the first 50 characters of a text message.
The easy way (what Python does now):
preview = msg[:50]
Or the hard way (do the limit checks yourself):
n = len(msg)
preview = msg[:50] if n > 50 else msg
Manually implementing that logic for adjustment of end points would be easy to forget, would be easy to get wrong (updating the 50 in two places), would be wordy, and would be slow. Python moves that logic to its internals where it is succint, automatic, fast, and correct. This is one of the reasons I love Python :-)
Part of question regarding assignments length mismatch from input length
The OP also wanted to know the rationale for allowing assignments such as p[20:100] = [7,8] where the assignment target has a different length (80) than the replacement data length (2).
It's easiest to see the motivation by an analogy with strings. Consider, "five little monkeys".replace("little", "humongous"). Note that the target "little" has only six letters and "humongous" has nine. We can do the same with lists:
>>> s = list("five little monkeys")
>>> i = s.index('l')
>>> n = len('little')
>>> s[i : i+n ] = list("humongous")
>>> ''.join(s)
'five humongous monkeys'
This all comes down to convenience.
Prior to the introduction of the copy() and clear() methods, these used to be popular idioms:
s[:] = [] # clear a list
t = u[:] # copy a list
Even now, we use this to update lists when filtering:
s[:] = [x for x in s if not math.isnan(x)] # filter-out NaN values
Hope these practical examples give a good perspective on why slicing works as it does.
The documentation has your answer:
s[i:j]: slice of s from i to j (note (4))
(4) The slice of s from i to j is defined as the sequence of items
with index k such that i <= k < j. If i or j is greater than
len(s), use len(s). If i is omitted or None, use 0. If j
is omitted or None, use len(s). If i is greater than or equal to
j, the slice is empty.
The documentation of IndexError confirms this behavior:
exception IndexError
Raised when a sequence subscript is out of range. (Slice indices are silently truncated to fall in the allowed range; if an index is
not an integer, TypeError is raised.)
Essentially, stuff like p[20:100] is being reduced to p[len(p):len(p]. p[len(p):len(p] is an empty slice at the end of the list, and assigning a list to it will modify the end of the list to contain said list. Thus, it works like appending/extending the original list.
This behavior is the same as what happens when you assign a list to an empty slice anywhere in the original list. For example:
In [1]: p = [1, 2, 3, 4]
In [2]: p[2:2] = [42, 42, 42]
In [3]: p
Out[3]: [1, 2, 42, 42, 42, 3, 4]
I am trying to sort a list, move all 0 to the end of list.
example: [0,1,0,2,3,0,4]->[1,2,3,4,0,0,0]
and I see someone code it in 1 line
list.sort(cmp=lambda a,b:-1 if b==0 else 0)
But I don't understand what inside the parentheses mean.
Could anyone tell me? Thank you.
Preface:
Sort a list according to the normal comparison:
some_list.sort()
Supply a custom comparator:
some_list.sort(cmp=my_comparator)
A lambda function:
x = lambda a, b: a - b
# is roughly the same as
def x(a, b):
return a - b
An if-else-expression:
value = truthy_case if condition else otherwise
# is roughly the same as
if condition:
value = truthy_case
else:
value = otherwise
The line list.sort(cmp=lambda a,b:-1 if b==0 else 0) itself:
Now, the condition in the comparator is whether b==0, if so indicate that b has a bigger value than a (the sign of the result is negative), otherwise indicate that the values compare the same (the sign is zero).
Whilst Python's list.sort() is stable, this code is not sane, because the comparator needs to test a, too, not only b. A proper implementation would use the key argument:
some_list.sort(key=lambda a: 0 if a == 0 else -1)
Fixed list.sort(cmp=...) implementation:
If you want to use list.sort(cmp=...) (you don't) or if you are just curious, this is a sane implementation:
some_list.sort(cmp=lambda a, b: 0 if a == b else
+1 if a == 0 else
-1 if b == 0 else 0)
But notice:
In Py3.0, the cmp parameter was removed entirely (as part of a larger effort to simplify and unify the language, eliminating the conflict between rich comparisons and the __cmp__ methods).
An alternative:
Sorting a list is in O(𝘯 log 𝘯). I do not know if for this simple problem the code runs faster, but I wouldn't think so. An O(𝘯) solution is filtering:
new_list = [x for x in some_list if x != 0]
new_list.extend([0] * (len(some_list) - len(new_list)))
The difference will probably only matter for quite long lists, though.
>>> sorted(l, key=lambda x:str(x) if x == 0 else x)
[1, 3, 4, 8, 0, 0, 0]
Guess what's happening here? I am exploiting the fact that, as a preference, python will pick up integers first, then strings. SO I converted 0 into '0'.
Here's the proof.
>>> ll = [3,2,3, '1', '3', '0']
>>> sorted(ll)
[2, 3, 3, '0', '1', '3']
You should answer yourself and this is plan:
The ternary expression description is available here:
https://docs.python.org/3/reference/expressions.html?highlight=ternary%20operator#conditional-expressions
You can find a lot of expression description in that document:
https://docs.python.org/3/reference/expressions.html
Q: What does lambda mean?
Please spend just 5 days and read a Tutorial about Python language, which is a fork of the original Gvinno Van Rossum book.
https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions
I know that it's possible to convert generators into lists at a "low-level" (eg. list(i for i in xrange(10))), but is it possible to do the reverse without iterating through the list first (eg. (i for i in range(10)))?
Edit: removed the word cast for clarity in what I'm trying to achieve.
Edit 2: Actually, I think I may have misunderstood generators at a fundamental level. That'll teach me to not post SO questions before my morning coffee!
Try this: an_iterator = iter(a_list) ... docs here. Is that what you want?
You can take a list out of an iterator by using the built-in function list(...) and an iterator out of a list by using iter(...):
mylist = list(myiterator)
myiterator = iter(mylist)
Indeed, your syntax is an iterator:
iter_10 = (i for i in range(10))
instead of using [...] which gives a list.
Have a look at this answer Hidden features of Python
Indeed it's possible a list possesses the iterator interface:
list_iter = list.__iter__() # or iter(list), returns list iterator
list_iter.__next__() # list_iter.next() for python 2.x
So,
lst_iter = iter(lst)
does the trick.
Though it makes more sense to use comprehensions and make a generator out of it: e.g.
lst_iter_gt_10 = (item for item in lst if item > 10)
I'm not sure you mean, but what you typed is valid Python code:
>>> x = (i for i in range(10))
>>> x
<generator object at 0xb7f05f6c>
>>> for i in x: print i
0
1
2
3
4
5
6
7
8
9
next was key:
next(iter([]), None)
where you can replace None with whatever default you want
Like the title says, is there any way to name variables (i.e., lists) used within a nested list comprehension in Python?
I could come up with a fitting example, but I think the question is clear enough.
Here is an example of pseudo code:
[... [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension] ... [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension] ...]
Is there any way to avoid the repetition here and simply add a variable for this temporary list only for use within the list comprehension?
CLARIFICATION:
The list comprehension is already working fine, so it's not a question of 'can it be done with a list comprehension'. And it is quicker than it's original form of a for statement too, so it's not one of those 'for statements vs list comprehensions' questions either. It is simply a question of making the list comprehension more readable by making variable names for variables internal to the list comprehension alone. Just googling around I haven't really found any answer. I found this and this, but that's not really what I am after.
Based on my understanding of what you want to do, No you cannot do it.
You cannot carry out assignments in list comprehensions because a list comprehension is essentially of the form
[expression(x, y) for x in expression_that_creates_a_container
for y in some_other_expression_that_creates_a_container(x)
if predicate(y, x)]
Granted there are a few other cases but they're all about like that. Note that nowhere does there exist room for a statement which is what a name assignment is. So you cannot assign to a name in the context of a list comprehension except by using the for my_variable in syntax.
If you have the list comprehension working, you could post it and see if it can be simplified. Solutions based on itertools are often a good alternative to burly list comprehensions.
I think I understand exactly what you meant, and I came up with a "partial solution" to this problem. The solution works fine, but is not efficent.
Let me explain with an example:
I was just trying to solve a Pythagorean triplet which sum was 1000. The python code to solve it is just:
def pythagoreanTriplet(sum):
for a in xrange(1, sum/2):
for b in xrange(1, sum/3):
c = sum - a - b
if c > 0 and c**2 == a**2 + b**2:
return a, b, c
But I wanted to code it in a functional programming-like style:
def pythagoreanTriplet2(sum):
return next((a, b, sum-a-b) for a in xrange(1, sum/2) for b in xrange(1, sum/3) if (sum-a-b) > 0 and (sum-a-b)**2 == a**2 + b**2)
As can be seen in the code, I calc 3 times (sum-a-b), and I wanted to store the result in an internal varible to avoid redundant calculation. The only way I found to do that was by adding another loop with a single value to declare an internal variable:
def pythagoreanTriplet3(sum):
return next((a, b, c) for a in xrange(1, sum/2) for b in xrange(1, sum/3) for c in [sum-a-b] if c > 0 and c**2 == a**2 + b**2)
It works fine... but as I said at the begin of the post, is not an efficent method. Comparing the 3 methods with cProfile, the time required for each method is the next one:
First method: 0.077 seconds
Secnd method: 0.087 seconds
Third method: 0.109 seconds
Some people could classify the following as a "hack", but it is definitely useful in some cases.
f = lambda i,j: int(i==j) #A dummy function (here Kronecker's delta)
a = tuple(tuple(i + (2+f_ij)*j + (i + (1+f_ij)*j)**2
for j in range(4)
for f_ij in (f(i,j),) ) #"Assign" value f(i,j) to f_ij.
for i in range(4) )
print(a)
#Output: ((0, 3, 8, 15), (2, 13, 14, 23), (6, 13, 44, 33), (12, 21, 32, 93))
This approach is particularly convenient if the function f is costly to evaluate. Because it is somewhat unusual, it may be a good idea to document the "assignment" line, as I did above.
I'm just gonna go out on a limb here, because I have no idea what you really are trying to do. I'm just going to guess that you are trying to shoehorn more than you should be into a single expression. Don't do that, just assign subexpressions to variables:
sublist = [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension]
composedlist = [... sublist ... sublist ...]
This feature was added in Python 3.8 (see PEP 572), it's called "assignment expressions" and the operator is := .
Examples from the documentation:
results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]
stuff = [[y := f(x), x/y] for x in range(5)]