One of my coworkers was using the builtin max function (on Python 2.7), and he found a weird behavior.
By mistake, instead of using the keyword argument key (as in key=lambda n: n) to pre-sort the list passed as a parameter, he did:
>>> max([1,2,3,3], lambda n : n)
[1, 2, 3, 3]
He was doing what in the documentation is explained as:
If two or more positional arguments are provided, the largest of the positional arguments is returned., so now I'm curious about why this happens:
>>> (lambda n:n) < []
True
>>> def hello():
... pass
...
>>> hello < []
True
>>> len(hello)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'function' has no len()
I know it's not a big deal, but I'd appreciate if any of the stackoverflowers could explain how those comparisons are internally made (or point me into a direction where I can find that information). :-)
Thank you in advance!
Python 2 orders objects of different types rather arbitrarily. It did this to make lists always sortable, whatever the contents. Which direction that comparison comes out as is really not of importance, just that one always wins. As it happens, the C implementation falls back to comparing type names; lambda's type name is function, which sorts before list.
In Python 3, your code would raise an exception instead:
>>> (lambda n: n) < []
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: function() < list()
because, as you found out, supporting arbitrary comparisons mostly leads to hard-to-crack bugs.
Everything in Python (2) can be compared, but some are fairly nonsensical, as you've seen.
>>> (lambda n:n) < []
True
Python 3 resolves this, and produces exceptions instead.
Related
I've learned about iterators and such and discovered this quite interesting way of getting the first element in a list that a condition is applied (and also with default value in case we don't find it):
first_occurence = next((x for x in range(1,10) if x > 5), None)
For me, it seems a very useful, clear way of obtaining the result.
But since I've never seen that in production code, and since next is a little more "low-level" in the python structure I was wondering if that could be bad practice for some reason. Is that the case? and why?
It's fine. It's efficient, it's fairly readable, etc.
If you're expecting a result, or None is a possible result (so using None as a placeholder makes it hard to figure out if you got a result or got the default) it may be better to use the EAFP form rather than providing a default, catching the StopIteration it raises if no item is found, or just letting it bubble up if the problem is from the caller's input not meeting specs (so it's up to them to handle it). It looks even cleaner at point of use that way:
first_occurence = next(x for x in range(1,10) if x > 5)
Alternatively, when None is a valid result, you can use an explicit sentinel object that's guaranteed unique like so:
sentinel = object() # An anonymous object you construct can't possibly appear in the input
first_occurence = next((x for x in range(1,10) if x > 5), sentinel)
if first_occurence is not sentinel: # Compare with is for performance and to avoid broken __eq__ comparing equal to sentinel
A common use case for this one of these constructs to replace a call to any when you not only need to know if any item passed the test, but which item (any can only return True or False, so it's unsuited to finding which item passed).
We can wrap it up in a function to provide an even nicer interface:
_raise = object()
# can pass either an iterable or an iterator
def first(iterable, condition, *, default=_raise, exctype=None):
"""Get the first value from `iterable` which meets `condition`.
Will consume elements from the iterable.
default -> if no element meets the condition, return this instead.
exctype -> if no element meets the condition and there is no default,
raise this kind of exception rather than `StopIteration`.
(It will be chained from the original `StopIteration`.)
"""
try:
# `iter` is idempotent; this makes sure we have an iterator
return next(filter(condition, iter(iterable)))
except StopIteration as e:
if default is not _raise:
return default
if exctype:
raise exctype() from e
raise
Let's test it:
>>> first(range(10), lambda x: x > 5)
6
>>> first(range(10), lambda x: x > 11)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in first
StopIteration
>>> first(range(10), lambda x: x > 11, exctype=ValueError)
Traceback (most recent call last):
File "<stdin>", line 4, in first
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in first
ValueError
>>> first(range(10), lambda x: x > 11, default=None)
>>>
Consider the following code simulation a non-homogenous poisson process. The focus is on the Non_hom_poisson(T,n,N) function.
def lam_t(T): #calculates lam(t) at given t
Lambda=[]
t=T
for i in range(len(T)):
Lambda.append(0.8+0.4*np.sin(0.2*np.pi*t[i]))
return Lambda
def thinning(max_min,Lam_t,lam_min,times,subset_length):
#thins max-min to lam(t)-lam_min
t=0
J=0
I=0
S=[]
path=2
t_J=np.arange(0,602,2)
while (J<300):
if (path==2):
unif=np.random.uniform(0,1)
X=(-1/float(max_min[J]))*np.log(unif)
path=3
if (path==3):
if ((t+X)>t_J[J+1]):
if ((J+2)>=300):
return S;
X=(X-t_J[J+1]+t)*max_min[J+1]/float(max_min[J+2])
t=t_J[J+1]
J+=1
path=3
else:
t+=(X)
U=np.random.uniform(0,1)
L_t=0.8+0.4*np.sin(0.2*(np.pi)*t)
top_prob=float(L_t-lam_min[J])
bottom_prob=float(max_min[J])
prob=top_prob/float(bottom_prob)
if (U<=(prob)):
I+=1
S.append(float(t))
path=2
if (t>600):
break
return S;
def mod_lam(t,lam):
interval=float(np.mod(t,10))
J=np.arange(2,12,2)
for i in range(len(J)):
if (interval<J[i]):
return float(lam[i])
return float(lam[i])
def Non_hom_poisson(T,n,N):
time=np.arange(0.1,10.1,0.1)
Lambda_t=lam_t(time)
max_lam=[max(Lambda_t[x:(x+19)]) for x in range(0,len(time),20)]
min_lam=[min(Lambda_t[x:(x+19)]) for x in range(0,len(time),20)]
max_min_lam=[(max_lam[x]-min_lam[x]) for x in range(len(max_lam))]
max_min_lam=np.tile(max_min_lam,60)
min_lam=np.tile(min_lam,60)
poisson_min=[np.random.poisson(float(min_lam[0]))]
i=0
while (poisson_min[i]<600):
y=float(mod_lam(poisson_min[i],min_lam))
x=-1/float(y)*np.log(np.random.uniform(0,1))
poisson_min.append(float(x)+float(poisson_min[i]))
i+=1
thinned=thinning(max_min_lam,Lambda_t,min_lam,time,20)
superposition=np.append(thinned,poisson_min)
return np.sort(superposition)
NH=Non_hom_poisson(600,5,1)
print(NH)
I'm getting the following error - "'int' object is not callable" - when max_lam=[max(Lambda_t[x:(x+19)]) for x in range(0,len(time),20)] is called. Any suggestion?
There are three function calls in that line: range(), max() and len(). Those are python builtins.
But neither of those identifiers are reserved words. So what's probably happened is that you have used one of those identifiers as a variable name in the global scope. Since both len() and range() are called with no error in the lam_t() function, it must be max that is an integer.
Which means that an assignment like this has been executed in the global scope.
max = 100
The code included in the question does not contain any such assignment, but the error message indicates that max points to an integer. You can reproduce the error in the python repl.
>>> max = 1
>>> max(1, 3, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
>>> del max # fix it
>>> max(1, 3, 2)
3
It's good practice to avoid using sum, len, list, max, min, int, hash, dir etc. as variable names.
It's a rich cause of confusing bugs if you are using something like a jupyter notebook where it's common practice to define lots of variables in the global namespace and then forget about it.
alist = []
def show(*args, **kwargs):
alist.append(*args, **kwargs)
print(alist)
>>> show('tiger')
['tiger']
>>> show('tiger','cat')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in show
TypeError: append() takes exactly one argument (2 given)
>>> show('tiger','cat', {'name':'tom'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in show
TypeError: append() takes exactly one argument (3 given)
Since the method append of alist only accepts one argument, why not detect a syntax error on the line alist.append(*args, **kwargs) in the definition of the method show?
It's not a syntax error because the syntax is perfectly fine and that function may or may not raise an error depending on how you call it.
The way you're calling it:
alist = []
def show(*args, **kwargs):
alist.append(*args, **kwargs)
print(alist)
>>> show('tiger')
['tiger']
>>> show('tiger','cat')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in show
TypeError: append() takes exactly one argument (2 given)
A different way:
alist = []
def show(*args, **kwargs):
alist.append(*args, **kwargs)
print(alist)
>>> show('tiger')
['tiger', 'tiger']
>>> class L: pass
...
>>> alist = L()
>>> alist.append = print
>>> show('tiger','cat')
tiger cat
<__main__.L object at 0x000000A45DBCC048>
Python objects are strongly typed. The names that bind to them are not. Nor are function arguments. Given Python's dynamic nature it would be extremely difficult to statically predict what type a variable at a given source location will be at execution time, so the general rule is that Python doesn't bother trying.
In your specific example, alist is not in the local scope. Therefore it can be modified after your function definition was executed and the changes will be visible to your function, cf. code snippets below.
So, in accord with the general rule: predicting whether or not alist will be a list when you call .append? Near-impossible. In particular, the interpreter cannot predict that this will be an error.
Here is some code just to drive home the point that static type checking is by all practical means impossible in Python. It uses non-local variables as in your example.
funcs = []
for a in [1, "x", [2]]:
def b():
def f():
print(a)
return f
funcs.append(b())
for f in funcs:
f()
Output:
[2] # value of a at definition time (of f): 1
[2] # value of a at definition time (of f): 'x'
[2] # value of a at definition time (of f): [2]
And similarly for non-global non-local variables:
funcs = []
for a in [1, "x", [2]]:
def b(a):
def f():
print(a)
a = a+a
return f
funcs.append(b(a))
for f in funcs:
f()
Output:
2 # value of a at definition time (of f): 1
xx # value of a at definition time (of f): 'x'
[2, 2] # value of a at definition time (of f): [2]
It's not a syntax error because it's resolved at runtime. Syntax errors are caught initially during parsing. Things like unmatched brackets, undefined variable names, missing arguments (this is not a missing argument *args means any number of arguments).
show has no way of knowing what you'll pass it at runtime and since you are expanding your args variable inside show, there could be any number of arguments coming in and it's valid syntax! list.append takes one argument! One tuple, one list, one int, string, custom class etc. etc. What you are passing it is some number elements depending on input. If you remove the * it's all dandy as its one element e.g. alist.append(args).
All this means that your show function is faulty. It is equipped to handle args only when its of length 1. If its 0 you also get a TypeError at the point append is called. If its more than that its broken, but you wont know until you run it with the bad input.
You could loop over the elements in args (and kwargs) and add them one by one.
alist = []
def show(*args, **kwargs):
for a in args:
alist.append(a)
for kv in kwargs.items():
alist.append(kv)
print(alist)
I have a function that either returns a tuple or None. How is the Caller supposed to handle that condition?
def nontest():
return None
x,y = nontest()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not iterable
EAFP:
try:
x,y = nontest()
except TypeError:
# do the None-thing here or pass
or without try-except:
res = nontest()
if res is None:
....
else:
x, y = res
How about:
x,y = nontest() or (None,None)
If nontest returns a two-item tuple like it should, then x and y are assigned to the items in the tuple. Otherwise, x and y are each assigned to none. Downside to this is that you can't run special code if nontest comes back empty (the above answers can help you if that is your goal). Upside is that it is clean and easy to read/maintain.
If you can change the function itself, it's probably a better idea to make it raise a relevant exception instead of returning None to signal an error condition. The caller should then just try/except that.
If the None isn't signalling an error condition, you'll want to rethink your semantics altogether.
Given the following Python code:
def avg(a):
if len(a):
return sum(a) / len(a)
What is the language defined behavior of avg when the length of a is zero or is its behavior unspecified by the language and thus should not be counted upon in Python code?
The default return value is None.
From the documentation on Calls:
A call always returns some value, possibly None, unless it raises an exception. How this value is computed depends on the type of the callable object.
If len(a) is 0, that will be treated as a False-like value, and your return statement won't be reached. When the flow of control drops out of the bottom of a function with no explicit return statement being reached, Python functions implicitly return None:
>>> print(avg([]))
None
If len(a) is not defined - in other words, if the object has no __len__() method - you'll get a TypeError:
>>> print(avg(False))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in avg
TypeError: object of type 'bool' has no len()