How to write conditions in a function (k_over_iq)?
dt_for_all_days_np=a numpy array of numbers.
def k_over_iq(dt):
if dt !=0:
return 0.7*(1-e**(-0.01*dt**2.4))
else:
return 1
k_over_iq_i=k_over_iq(dt_for_all_days_np)
I get the following error:
if dt !=0: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
if dt != 0: won't work if dt is a numpy array. The if tries to get a single boolean out of the array, and as the error message warns, that's ambiguous: is array([True, False]) True or False?
To get around that in a vectorized fashion, two common ways are to use where or to use boolean indexing to patch.
Method #1, np.where
>>> dt = np.array([0,1,2,3])
>>> np.where(dt != 0, 0.7*(1-np.exp(-0.01*dt**2.4)), 1)
array([ 1. , 0.00696512, 0.03598813, 0.09124601])
This uses the function whenever dt != 0:
>>> dt != 0
array([False, True, True, True], dtype=bool)
and 1 otherwise.
Method #2: boolean indexing to patch
Compute the function everywhere and then fix the values that are wrong.
>>> b = 0.7*(1-np.exp(-0.01*dt**2.4))
>>> b
array([ 0. , 0.00696512, 0.03598813, 0.09124601])
>>> b[dt == 0] = 1
>>> b
array([ 1. , 0.00696512, 0.03598813, 0.09124601])
Indentation is the problem in your function.
I recommend you reading this: Python: Myths about Indentation
def k_over_iq(dt):
if dt !=0:
return 0.7*(1-e**(-0.01*dt**2.4))
else:
return 1
k_over_iq_i=k_over_iq(dt_for_all_days_np)
Related
a < b < c is an chained expression in Python, and it looks like it works on objects with appropriate comparison operators defined, but it doesn't work on numpy arrays. Why?
import numpy as np
class ContrarianContainer(object):
def __init__(self, x):
self.x = x
def __le__(self, y):
return not self.x <= y
def __lt__(self, y):
return not self.x < y
def __ge__(self, y):
return not self.x >= y
def __gt__(self, y):
return not self.x > y
def __eq__(self, y):
return not self.x == y
def __ne__(self, y):
return not self.x != y
numlist = np.array([1,2,3,4])
for n in numlist:
print 0 < n < 3.5
for n in numlist:
print 0 > ContrarianContainer(n) > 3.5
print 0 < numlist < 3.5
this prints:
True
True
True
False
True
True
True
False
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-187-277da7750148> in <module>()
4 for n in numlist:
5 print 0 < n < 3.5
----> 6 print 0 < numlist < 3.5
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
0 < numlist < 3.5
Is equivalent to:
(0 < numlist) and (numlist < 3.5)
except that numlist is only evaluated once.
The implicit and between the two results is causing the error
So the docs say:
Formally, if a, b, c, ..., y, z are expressions and op1, op2, ..., opN are comparison operators, then a op1 b op2 c ... y opN z is equivalent to a op1 b and b op2 c and ... y opN z, except that each expression is evaluated at most once.
and
(but in both cases z is not evaluated at all when x < y is found to be false).
For a scalar
In [20]: x=5
In [21]: 0<x<10
Out[21]: True
In [22]: 0<x and x<10
Out[22]: True
But with an array
In [24]: x=np.array([4,5,6])
In [25]: 0<x and x<10
...
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
This ValueError arises when a numpy boolean is used in a context that expects a scalar boolean.
In [26]: (0<x)
Out[26]: array([ True, True, True], dtype=bool)
In [30]: np.array([True, False]) or True
...
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
In [33]: if np.array([True, False]): print('yes')
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
It evaluates the 0<x, but doesn't even get to evaluating the x<10, because it can't use the resulting boolean array in a or/and context. numpy has defined | and &, but not or or and.
In [34]: (0<x) & x<10
Out[34]: array([ True, True, True], dtype=bool)
When we use 0 < x <10 we are implicitly expecting to evaluate a vectorized version of the scalar chained expression.
In [35]: f = np.vectorize(lambda x: 0<x<10, otypes=[bool])
In [36]: f(x)
Out[36]: array([ True, True, True], dtype=bool)
In [37]: f([-1,5,11])
Out[37]: array([False, True, False], dtype=bool)
Note that attempting to apply chaining to a list doesn't even get past the first <:
In [39]: 0 < [-1,5,11]
TypeError: unorderable types: int() < list()
This set of expressions indicates that the & operator has precedence over the < operator:
In [44]: 0 < x & x<10
ValueError ...
In [45]: (0 < x) & x<10
Out[45]: array([ True, True, True], dtype=bool)
In [46]: 0 < x & (x<10)
Out[46]: array([False, True, False], dtype=bool)
In [47]: 0 < (x & x)<10
ValueError...
So the safe version is (0 < x) & (x<10), making sure that all < are evaluated before the &.
edit
Here's a further example that confirms the short-cut and evaluation:
In [53]: x=2
In [54]: 3<x<np.arange(4)
Out[54]: False
In [55]: 1<x<np.arange(4)
Out[55]: array([False, False, False, True])
When 3<x is False, it returns that, without further evaluation.
When it is True, it goes on to evaluate x<np.arange(4), returning a 4 element boolean.
Or with a list that doesn't support < at all:
In [56]: 3<x<[1,2,3]
Out[56]: False
In [57]: 1<x<[1,2,3]
Traceback (most recent call last):
File "<ipython-input-57-e7430e03ad55>", line 1, in <module>
1<x<[1,2,3]
TypeError: '<' not supported between instances of 'int' and 'list'
My code should stop when all the values of my array are greater than 100. I am using np.all() to evaluate the condition. It looks like the condition is met, but np.all() seems to be incorrectly evaluating. What am I missing? Python 2.7.8 on OS 10.8.5.
(Pdb) ratio_J_a
array([ 250.44244741, 186.92848637, 202.67726408, 143.01112845,
132.95878384, 176.49130164, 178.9892571 , 118.07516559,
205.59639112, 183.64142204])
(Pdb) np.all(ratio_J_a) > 100.
False
(Pdb) np.all(ratio_J_a) < 100.
True
I think you should have:
In [5]: import numpy as np
In [6]: ratio_J_a = np.array([ 250.44244741, 186.92848637, 202.67726408, 143.01112845,
...: 132.95878384, 176.49130164, 178.9892571 , 118.07516559,
...: 205.59639112, 183.64142204])
In [7]: print(np.all(ratio_J_a > 100.))
True
In [8]: print(np.all(ratio_J_a < 100.))
False
And just in case, you are actually wondering why you got this:
(Pdb) np.all(ratio_J_a) > 100.
False
(Pdb) np.all(ratio_J_a) < 100.
True
The reason is that np.all(ratio_J_a) evaluates to true, which is treated as one, e.g.
In [17]: "{}".format(np.all(ratio_J_a))
Out[17]: 'True'
In [18]: "{:d}".format(np.all(ratio_J_a))
Out[18]: '1'
Thus, in your case you are actually doing this:
(Pdb) 1 > 100.
False
(Pdb) 1 < 100.
True
numpy.all tests whether all array elements along a given axis evaluate to True.
>>> import numpy as np
>>> ratio_J_a = np.array([
... 250.44244741, 186.92848637, 202.67726408, 143.01112845,
... 132.95878384, 176.49130164, 178.9892571 , 118.07516559,
... 205.59639112, 183.64142204
... ])
>>> np.all(ratio_J_a > 100)
True
>>> np.all(ratio_J_a < 100)
False
Why you got wrong result:
np.all(ratio_J_a) is evaluated as True because non-zero numbers are treated as a truth value. And True is equal to 1. 1 > 100 is False, 1 < 100 is True.
I went onto one line of code, which I want to convert from numpy syntax to regular python 2.7 syntax:
SunAz=SunAz + (SunAz < 0) * 360
source:
https://github.com/Sandia-Labs/PVLIB_Python/blob/master/pvlib/pvl_ephemeris.py#L147
If we pretend that the numpy array is one dimensional, can it be translated to regular python 2.7 syntax like so:
newSunAz = []
for item in SunAz:
if item < 0:
newItem = item + item*360
newSunAz.append(newItem)
else:
newSunAz.append(item)
??
Thank you for the help.
I'm not sure that this would be the translation. The line
SunAz=SunAz + (SunAz < 0) * 360
(SunAz < 0) creates a boolean array, True where the angle is negative, and false otherwise. Multiplying False by a constant gives 0, and True is interpreted to be a 1. This line actually says, "shift the angle by 360 degrees if negative, otherwise leave it be".
So a more literal translation would be the following:
SunAz = [angle + 360 if angle < 0 else angle for angle in SunAz]
Try this:
new_sun_az = [i+360 if i > 0 else i for i in sun_az]
The main difference is that most operators are applied to the list object in plain Python lists and return a single result, while they return a numpy array where each item is the result of the operation applied to the corresponding item on the original array for numpy arrays.
>>> import numpy as np
>>> plainlist = range(5)
>>> plainlist
[0, 1, 2, 3, 4]
>>> plainlist > 5 # single result
True
>>> nparray = np.array(plainlist)
>>> nparray
array([0, 1, 2, 3, 4])
>>> nparray > 5 # array of results
array([False, False, False, False, False], dtype=bool)
>>>
[update]
Mike's answer is right. My original answer was:
new_sun_az = [i+i*360 if i > 0 else i for i in sun_az]
I have a big data set of floating point numbers. I iterate through them and evaluate np.log(x) for each of them.
I get
RuntimeWarning: divide by zero encountered in log
I would like to get around this and return 0 if this error occurs.
I am thinking of defining a new function:
def safe_ln(x):
#returns: ln(x) but replaces -inf with 0
l = np.log(x)
#if l = -inf:
l = 0
return l
Basically,I need a way of testing that the output is -inf but I don't know how to proceed.
Thank you for your help!
You are using a np function, so I can safely guess that you are working on a numpy array?
Then the most efficient way to do this is to use the where function instead of a for loop
myarray= np.random.randint(10,size=10)
result = np.where(myarray>0, np.log(myarray), 0)
otherwise you can simply use the log function and then patch the hole:
myarray= np.random.randint(10,size=10)
result = np.log(myarray)
result[result==-np.inf]=0
The np.log function return correctly -inf when used on a value of 0, so are you sure that you want to return a 0? if somewhere you have to revert to the original value, you are going to experience some problem, changing zeros into ones...
Since the log for x=0 is minus infinite, I'd simply check if the input value is zero and return whatever you want there:
def safe_ln(x):
if x <= 0:
return 0
return math.log(x)
EDIT: small edit: you should check for all values smaller than or equal to 0.
EDIT 2: np.log is of course a function to calculate on a numpy array, for single values you should use math.log. This is how the above function looks with numpy:
def safe_ln(x, minval=0.0000000001):
return np.log(x.clip(min=minval))
You can do this.
def safe_ln(x):
try:
l = np.log(x)
except ZeroDivisionError:
l = 0
return l
I like to use sys.float_info.min as follows:
>>> import numpy as np
>>> import sys
>>> arr = np.linspace(0.0, 1.0, 3)
>>> print(arr)
[0. 0.5 1. ]
>>> arr[arr < sys.float_info.min] = sys.float_info.min
>>> print(arr)
[2.22507386e-308 5.00000000e-001 1.00000000e+000]
>>> np.log10(arr)
array([-3.07652656e+02, -3.01029996e-01, 0.00000000e+00])
Other answers have also introduced small positive values, but I prefer to use the smallest possible value to make the approximation more accurate.
The answer given by Enrico is nice, but both solutions result in a warning:
RuntimeWarning: divide by zero encountered in log
As an alternative, we can still use the where function but only execute the main computation where it is appropriate:
# alternative implementation -- a bit more typing but avoids warnings.
loc = np.where(myarray>0)
result2 = np.zeros_like(myarray, dtype=float)
result2[loc] =np.log(myarray[loc])
# answer from Enrico...
myarray= np.random.randint(10,size=10)
result = np.where(myarray>0, np.log(myarray), 0)
# check it is giving right solution:
print(np.allclose(result, result2))
My use case was for division, but the principle is clearly the same:
x = np.random.randint(10, size=10)
divisor = np.ones(10,)
divisor[3] = 0 # make one divisor invalid
y = np.zeros_like(divisor, dtype=float)
loc = np.where(divisor>0) # (or !=0 if your data could have -ve values)
y[loc] = x[loc] / divisor[loc]
use exception handling:
In [27]: def safe_ln(x):
try:
return math.log(x)
except ValueError: # np.log(x) might raise some other error though
return float("-inf")
....:
In [28]: safe_ln(0)
Out[28]: -inf
In [29]: safe_ln(1)
Out[29]: 0.0
In [30]: safe_ln(-100)
Out[30]: -inf
you could do:
def safe_ln(x):
#returns: ln(x) but replaces -inf with 0
try:
l = np.log(x)
except RunTimeWarning:
l = 0
return l
For those looking for a np.log solution that intakes a np.ndarray and nudges up only zero values:
import sys
import numpy as np
def smarter_nextafter(x: np.ndarray) -> np.ndarray:
safe_x = np.where(x != 0, x, np.nextafter(x, 1))
return np.log(safe_x)
def clip_usage(x: np.ndarray, safe_min: float | None = None) -> np.ndarray:
# Inspiration: https://stackoverflow.com/a/13497931/
clipped_x = x.clip(min=safe_min or np.finfo(x.dtype).min)
return np.log(clipped_x)
def inplace_usage(x: np.ndarray, safe_min: float | None = None) -> np.ndarray:
# Inspiration: https://stackoverflow.com/a/62292638/
x[x == 0] = safe_min or np.finfo(x.dtype).min
return np.log(x)
Or if you don't mind nudging all values and like bad big-O runtimes:
def brute_nextafter(x: np.ndarray) -> np.ndarray:
# Just for reference, don't use this
while not x.all():
x = np.nextafter(x, 1)
return np.log(x)
>>> import numpy
>>> numpy.array([2]) > 1
array([ True], dtype=bool)
>>> numpy.array([2]).any() > 1
False
Shouldn't any() test all elements of the array and return True?
It does return True. But (True > 1) == False. While the first part is 2 > 1 which of course is True.
As others posted, you probably want:
(numpy.array([2]) > 1).any()
Perhaps you are confusing it with this
>>> (numpy.array([2]) > 1).any()
True