I keep getting
Exception: 'numpy.float64' object is not callable
when trying to minimize a function.
I can call the function I'm trying to minimize as
def testLLCalc():
mmc = MortalityModelCalibrator()
a = mmc.log_likelihood(2000, np.array([[0.6, 0.2, 0.8]]))
but when I try and minimize it by doing
x0 = np.array([0, 0, 0])
res = minimize(-a[0], x0)
I get the exception above. Any help would be appreciated. Full traceback is:
Error
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\unittest\case.py", line 59, in testPartExecutor
yield
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\unittest\case.py", line 601, in run
testMethod()
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "C:\Users\Matt\Documents\PyCharmProjects\Mortality\src\PennanenMortalityModel_test.py", line 57, in testLLCalc
res = minimize(-a[0], x0)
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\_minimize.py", line 444, in minimize
return _minimize_bfgs(fun, x0, args, jac, callback, **options)
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\optimize.py", line 913, in _minimize_bfgs
gfk = myfprime(x0)
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\optimize.py", line 292, in function_wrapper
return function(*(wrapper_args + args))
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\optimize.py", line 688, in approx_fprime
return _approx_fprime_helper(xk, f, epsilon, args=args)
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\optimize.py", line 622, in _approx_fprime_helper
f0 = f(*((xk,) + args))
File "C:\Program Files (x86)\JetBrains\WinPython-64bit-3.5.3.0Qt5\python-3.5.3.amd64\lib\site-packages\scipy\optimize\optimize.py", line 292, in function_wrapper
return function(*(wrapper_args + args))
Exception: 'numpy.float64' object is not callable
scipy's minimize expects a callable function as first argument.
As you did not show your complete code it's just a guessing-game here, but this
res = minimize(-a[0], x0)
has to mean that the first element of a should be a function.
Seeing this line:
a = mmc.log_likelihood(2000, np.array([[0.6, 0.2, 0.8]]))
it does not look like that as probably a scalar is returned.
The effect is simple: scipy want's to call this given function with some argument (x0 at the beginning), but calls some numpy-array value with some argument in your case (which is not valid of course).
Review the docs:
minimize(fun, x0, args=(),...
fun : callable
Objective function.
x0 : ndarray
Initial guess.
args : tuple, optional
Extra arguments passed to the objective function and its derivatives
Do you know what a 'callable' is? It's a function (or equivalent), something can be 'called' with fun(x0, arg0, arg1, ...).
The error tells us that -a[0] is an element of a numpy array, a.
It's not clear whether you are trying to minimize this function, or whether this is part of using minimize. It can't be source of a, because it doesn't return anything.
def testLLCalc():
mmc = MortalityModelCalibrator()
a = mmc.log_likelihood(2000, np.array([[0.6, 0.2, 0.8]]))
# return a ????
So - review your understanding of basic Python, especially the idea of a 'callable'. And run some the minimize examples, to get a better feel for how to use this function.
Related
So I was working on a project and I came across this error related to sympy:
Traceback (most recent call last):
File "c:\Users\Andres\OneDrive - Centre d'Estudis Monlau\z.ottro\final program try 1\testing area 2.py", line 4, in <module>
Ymin = minimum(root(x,3), x, domain=Interval(-7,7))
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\calculus\util.py", line 837, in minimum
return function_range(f, symbol, domain).inf
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\calculus\util.py", line 220, in function_range range_int += Interval(vals.inf, vals.sup, left_open, right_open)
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\sets\sets.py", line 274, in inf
return self._inf
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\sets\sets.py", line 1910, in _inf
return Min(*self)
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\functions\elementary\miscellaneous.py", line 391, in __new__
args = frozenset(cls._new_args_filter(args))
File "C:\Users\Andres\AppData\Local\Programs\Python\Python39\lib\site-packages\sympy\functions\elementary\miscellaneous.py", line 564, in _new_args_filter
raise ValueError("The argument '%s' is not comparable." % arg)
ValueError: The argument '(-7)**(1/3)' is not comparable.
I was able to isolate where happened and recreate the error with these lines:
from sympy import *
x = Symbol("x")
Ymin = minimum(root(x,3), x, domain=Interval(-7,7))
I'm not undrestanding why it's happening, as when is root(x,2) it doasn't have this problem. Does anyone have a clue?
Okey i found an answer online posted by user6655984(stackoverflow) and i wanted to share it online:
expression = root(Abs(x), 3)*sign(x)
def integrate_ode(ode,u0,T):
sol=solve_ivp(ode,T,u0)
return sol.y[:,-1]
def phase(ode,u0):
return ode(0,u0)[0] #dx/dt(0)
def shooting(ode,U):
print(len(U))
u0=U[0:-1]
#print(u0)
T=U[-1]
#print(T)
G=np.concatenate((integrate_ode(ode,u0,T)-u0,phase(ode,u0)))
return G
To get the solutions I'm using:
fsolve(lambda U:shooting(ode1,U),...(initial conditions))
I'm getting this error:
IndexError: invalid index to scalar variable.
Here is my full traceback:
Traceback (most recent call last):
"...Python\workweek16.py", line 62, in <module>
G=fsolve(lambda U:shooting(ode1,U),(0,0))
File "C:\Users\tmara\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\scipy\optimize\minpack.py", line 160, in fsolve
res = _root_hybr(func, x0, args, jac=fprime, **options)
File "C:\Users\tmara\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\scipy\optimize\minpack.py", line 226, in _root_hybr
shape, dtype = _check_func('fsolve', 'func', func, x0, args, n, (n,))
File "C:\Users\tmara\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\scipy\optimize\minpack.py", line 24, in _check_func
res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
File "...Python\workweek16.py", line 62, in <lambda>
G=fsolve(lambda U:shooting(ode1,U),(0,0))
File "...Python\workweek16.py", line 51, in shooting
G=np.concatenate((integrate_ode(ode,u0,T)-u0,phase(ode,u0)))
File "...Python\workweek16.py", line 40, in integrate_ode
sol=solve_ivp(ode,T,u0)
File "C:\Users\tmara\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\scipy\integrate\_ivp\ivp.py", line 508, in solve_ivp
t0, tf = float(t_span[0]), float(t_span[1])
IndexError: invalid index to scalar variable.
Does anyone know what to do?
The t_span argument for solve_ivp needs to actually be a time span. So replace T there with [0,T].
I'm trying to implement LSH in pyspark, my implementation perfectly works for small sets of documents but, when the set is quite huge, I obtain this error:
AttributeError: Can't pickle local object '__hash_family__.<locals>.hash_member'
And then:
19/11/21 17:59:40 ERROR TaskSetManager: Task 0 in stage 3.0 failed 1 times; aborting job
Traceback (most recent call last):
File "/Users/<my_home_dir>/PycharmProjects/data_mining/hw_2/ex_3/main_kijiji.py", line 62, in <module>
lsh = signatures.reduce(lambda x, y: __update_hash_table__(x[0], x[1], lsh_b, lsh_r) +
File "/Library/Python/3.7/site-packages/pyspark/rdd.py", line 844, in reduce
vals = self.mapPartitions(func).collect()
File "/Library/Python/3.7/site-packages/pyspark/rdd.py", line 816, in collect
sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
File "/Library/Python/3.7/site-packages/py4j/java_gateway.py", line 1257, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/Library/Python/3.7/site-packages/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/Library/Python/3.7/site-packages/py4j/protocol.py", line 328, in get_return_value
format(target_id, ".", name), value)
The error is generated by this line of code:
lsh = signatures.reduce(lambda x, y: __update_hash_table__(x[0], x[1], hash_tables, lsh_b, lsh_r) +
__update_hash_table__(y[0], y[1], hash_tables, lsh_b, lsh_r)).cache()
where hash_tables is a list generated in this way:
hash_tables = [[__hash_family__(i, lsh_num_hashes), {}] for i in range(lsh_b)]
The function hash_family is the following:
def __hash_family__(i, resultSize=20):
maxLen = 10 # how long can our i be (in decimal)
salt = str(i).zfill(maxLen)[-maxLen:]
def hash_member(x):
return hashlib.sha1((x + salt).encode()).digest()[-resultSize:]
return hash_member
And this is the function update_hash_table:
def __update_hash_table__(doc_id, sig, hash_tables, lsh_b, lsh_r):
for b in range(lsh_b):
start_row = b * lsh_r
end_row = start_row + lsh_r
band = str(sig[start_row:end_row])
bucket_idx = hash_tables[b][0](''.join(band))
try:
hash_tables[b][1][bucket_idx].append(doc_id)
except KeyError:
hash_tables[b][1][bucket_idx] = [doc_id]
return hash_tables
I even tried to generate the hash_tables directly in the file that contains the definition of update_hash_table or to generate the tables inside the function, but I always get the pickling error, how can I rewrite my code to store in the variable lsh the result of that reduce operation?
I know I could collect signatures and transform it in a list from the rdd, but it would be very expensive, can I still execute this operation without increasing too much the execution time?
I have a funtion defined in a class
class someClass:
def objFunction(self, weights):
return [self.obj1(weights), self.obj2(weights), self.obj3(weights)]
def asf(self, f):
def obj(x):
return np.max(np.array(f(x[0],x[1],x[2])))+0.00001*np.sum(f(x[0],x[1],x[2]))
res=minimize(obj,
[0.3,0.3,0.4], method='SLSQP'
,jac=ad.gh(obj)[0],options = {'disp':True, 'ftol': 1e-20,
'maxiter': 1000})
return res
where obj1, obj2 and obj3 are some objective functions to optimized. I am running this method making an object separately:
newObj = SomeClass()
newObj.objFunction(weights)
This works fine and give expected results. But when I used the same method inside another method in the class it returns the above mentioned error. This is how I am doing:
a = someClass()
a.asf(a.objFunction(weights)
It throws this:
Traceback (most recent call last):
File "D:/*******.py", line 332, in <module>
print(investment.asf(obj1(w),ref,ideal,nadir, rho))
File "*******.py", line 313, in asf
,options = {'disp':True, 'ftol': 1e-20, 'maxiter': 1000})
File "C:\Users\*****\Downloads\WinPython-64bit-3.5.1.2\python-3.5.1.amd64\lib\site-packages\scipy\optimize\_minimize.py", line 455, in minimize
constraints, callback=callback, **options)
File "C:\Users\*****\Downloads\WinPython-64bit-3.5.1.2\python-3.5.1.amd64\lib\site-packages\scipy\optimize\slsqp.py", line 363, in _minimize_slsqp
fx = func(x)
File "C:\Users\*******\Downloads\WinPython-64bit-3.5.1.2\python-3.5.1.amd64\lib\site-packages\scipy\optimize\optimize.py", line 289, in function_wrapper
return function(*(wrapper_args + args))
File "D:********.py", line 305, in obj
return np.max(np.array(f(x[0], x[1], x[2], x[3])))+rho*np.sum(f(x[0], x[1], x[2], x[3]))
TypeError: 'list' object is not callable
I think I am doing some OOP (object oriented programming) error in coding because I am not good at it. Any suggestions for this ? Thanks
a.objFunction(weights) returns a list, that's clear from the definition.
a.asf expects one argument called f, which in the definition gets used like:
f(x[0],x[1],x[2])
So you are giving a.asf a list and trying to call it like a function.
I'm having trouble creating a dictionary for the constraints using scipy.optimize.basinhopping. I'm able to get my code to run (without constraints), but the answer doesn't make sense because I need to enforce some constraints. For now, I'm only trying to get one constraint working but for the final solution I need to figure out how to implement several constraints. The code I have now is:
x0 = [f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11]
args = arg1,arg2,arg3,arg4
def func(x,*args)
#Do some math
return result
#This is where I need help most
cons = {'type':'ineq','fun': lambda x: x[5]-x[4]}
minimizer_kwargs = {"method":"COBYLA","args":"args","constraints":"cons"}
ret = scipy.optimize.basinhopping(func,x0,minimizer_kwargs=minimizer_kwargs)
But get this error when trying to run it:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 601, in runfile
execfile(filename, namespace)
File "C:\Python27\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 66, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "C:/Python27/Scripts/SpectralResearch/mainScripts/main.py", line 121, in <module>
ret = scipy.optimize.basinhopping(func,x0,minimizer_kwargs=minimizer_kwargs)
File "C:\Python27\lib\site-packages\scipy\optimize\_basinhopping.py", line 605, in basinhopping
accept_tests, disp=disp)
File "C:\Python27\lib\site-packages\scipy\optimize\_basinhopping.py", line 72, in __init__
minres = minimizer(self.x)
File "C:\Python27\lib\site-packages\scipy\optimize\_basinhopping.py", line 279, in __call__
return self.minimizer(self.func, x0, **self.kwargs)
File "C:\Python27\lib\site-packages\scipy\optimize\_minimize.py", line 432, in minimize
return _minimize_cobyla(fun, x0, args, constraints, **options)
File "C:\Python27\lib\site-packages\scipy\optimize\cobyla.py", line 218, in _minimize_cobyla
raise TypeError('Constraints must be defined using a '
TypeError: Constraints must be defined using a dictionary.
Essentially I need to enforce the constraint that certain variables are greater than others. I've been looking at the documentation([1],[2]) and articles, but haven't found anything that works. Any ideas what I could be doing wrong?
minimizer_kwargs = {"method":"COBYLA","args":args,"constraints":cons}
You passed the strings "args" and "cons" instead of the variables with those names.