I was studying the AdaDelta optimization algorithm so I tried to implement it in Python, but there is something wrong with my code, since I get the following error:
AttributeError: 'numpy.ndarray' object has no attribute 'sqrt'
I did not find something about what is causing that error. According to the message, it's because of this line of code:
rms_grad = np.sqrt(self.e_grad + epsilon)
This line is similar to this equation:
RMS[g]t=√E[g^2]t+ϵ
I got the core equations of the algorithm in this article: http://ruder.io/optimizing-gradient-descent/index.html#adadelta
Just one more detail: I'm initializing the E[g^2]t matrix like this:
self.e_grad = (1 - mu)*np.square(nabla)
Where nabla is the gradient. Similar to this equation:
E[g2]t = γE[g2]t−1 + (1−γ)g2t
(the first term is equal to zero in the first iteration, just like the line of code above)
So I want to know if I'm initializing the E matrix the wrong way or I'm doing the square root inappropriately. I tried to use the pow() function but it doesn't work. If anyone could help me with this I would be very grateful, I'm trying this for weeks.
Additional details requested by andersource:
Here is the entire source code on github: https://github.com/pedrovbeltran/neural-networks-and-deep-learning/blob/experimental/modified-networks/network2_with_adadelta.py .
I think the problem is that self.e_grad_w is an ndarray of shape (2,) which further contains two additional ndarrays with 2d shapes, instead of directly containing data. This seems to be initialized in e_grad_initializer, in which nabla_w has the same structure. I didn't track where this comes from all the way back, but I believe once you fix this issue the problem will be resolved.
Related
I am trying to solve a MILP problem using Python with Gurobi Solver. When I solve the model with the constraint (attached below), it appears the error like this: "AttributeError: 'gurobipy.QuadExpr' object has no attribute 'getVar'".
Could you help me to fix this error? Thank you in advance!
mdl.addConstrs((t[i,k] * X[i,j,k] - te1[i] <= 5) >> (z1[i,k] == 1) for i,j,k in arcos if i != 0 and i != 23)
where: t[i,j]: a continuous variable;
X[i,j,k], z1[i,k]: binary variables;
te1[i]: a parameter
The Gurobi documentation demonstrates how to add indicator constraints: Model.addGenConstrIndicator()
And the documentation for Model.addConstrs() also has an example for adding multiple indicator constraints in one call:
model.addConstrs((x[i] == 1) >> (y[i] + z[i] <= 5) for i in range(5))
In general, you need to define a binary variable that serves as the indicator. Your constraints seem to be the other way around, with a condition being fulfilled resulting in whether an indicator variable is set.
This is indeed not a very informative error message. Developers are notorious for not paying much attention to formulating good, meaningful error messages. They should. A better error message could have prevented this post.
Now to the underlying issue. An indicator constraint has the following structure:
binary variable = 0 ==> linear constraint
or
binary variable = 1 ==> linear constraint
You need to reformulate things to fit into this scheme. Or use a big-M formulation.
I was wondering if I could solve the problem described at https://github.com/cair/pyTsetlinMachine/issues/6 "myself".
I added a running example to the github issue.
The following error occurs
File "/mnt/c/ProjectsGit/BreastCancerDemo_pkl.py", line 42, in <module>
tm2.set_state(state)
File "/home/unix/miniconda3/lib/python3.8/site-packages/pyTsetlinMachine/tm.py", line 351, in set_state
for i in range(self.number_of_classes):
AttributeError: 'MultiClassTsetlinMachine' object has no attribute 'number_of_classes'
The class 'MultiClassTsetlinMachine' seems to have such as attribute?
https://github.com/cair/pyTsetlinMachine/blob/0c1ff1d43e1dd466ae0e41d50a4bde94bb36fedc/pyTsetlinMachine/tm.py#L396
Is it the case that the attribute 'number_of_classes' is not stored? but could be recovered by the dimension of the vector?
Any ideas? Hints?
You can see in the source code that the attribute gets set (the first time) you run fit, and does not get stored or retrieved in get_state or set_state. The code in that link is self.number_of_classes = int(np.max(Y) + 1), so I guess Y is supposed to be numerically encoded and one-dimensional. So yes, setting it yourself (not the dimension of the vector, but the number of unique values) should be enough. You could pickle that number together with the state, if that's more convenient.
That should at least be a good start, but I'm not familiar enough with the package to know if everything will work correctly from there.
I am encountering some problems when translating the following code from MATLAB to Python:
Matlab code snippet:
x=M_test %M_test is a 1x3 array that holds the adjustment points for the function
y=R_test %R_test is also a 1x3 array
>> M_test=[0.513,7.521,13.781]
>> R_test=[2.39,3.77,6.86]
expo3= #(b,x) b(1).*(exp(-(b(2)./x).^b(3)));
NRCF_expo3= #(b) norm(y-expo3(b,x));
B0_expo3=[fcm28;1;1];
B_expo3=fminsearch(NRCF_expo3,B0_expo3);
Data_raw.fcm_expo3=(expo3(B_expo3,Data_raw.M));
The translated (python) code:
expo3=lambda x,M_test: x[0]*(1-exp(-1*(x[1]/M_test)**x[2]))
NRCF_expo3=lambda R_test,x,M_test: np.linalg.norm(R_test-expo3,ax=1)
B_expo3=scipy.optimize.fmin(func=NRCF_expo3,x0=[fcm28,1,1],args=(x,))
For clarity, the object function 'expo3' wants to go through the adjustment points defined by M_test.
'NRCF_expo3' is the function that wants to be minimised, which is basically the error between R_test and the drawn exponential function.
When I run the code, I obtain the following error message:
B_expo3=scipy.optimize.fmin(func=NRCF_expo3,x0=[fcm28,1,1]),args=(x,))
NameError: name 'x' is not defined
There are a lot of similar questions that I have perused.
If I delete the 'args' from the optimization function, as numpy/scipy analog of matlab's fminsearch
seems to indicate it is not necessary, I obtain the error:
line 327, in function_wrapper
return function(*(wrapper_args+args))
TypeError: <lambda>() missing 2 required positional arguments: 'x' and 'M_test'
There are a lot of other modifications that I have tried, following examples like Using scipy to minimize a function that also takes non variational parameters or those found in Open source examples, but nothing works for me.
I expect this is probably quite obvious, but I am very new to Python and I feel like I am looking for a needle in a haystack. What am I not seeing?
Any help would be really appreciated. I can also provide more code, if that is necessary.
I think you shouldn't use lambdas in your code, make instead a single target function with your three parameters (see PEP8). There is a lot of missing information in you post, but for what I can infer, you want something like this:
from scipy.optimize import fmin
# Define parameters
M_TEST = np.array([0.513, 7.521, 13.781])
X_ARR = np.array([2.39,3.77,6.86])
X0 = np.array([10, 1, 1]) # whatever your variable fcm28 is
def nrcf_exp3(r_test, m_test, x):
expo3 = x[0] * (1 - np.exp(-(x[1] / m_test) ** x[2]))
return np.linalg.norm(r_test - expo3)
fmin(nrcf_exp3, X0, args=(M_TEST, X_ARR))
I'm having some issues with SymPy's current assumptions.
Look at this thread. One of the hints said to use the assume module (reference here).
I tried doing the following computation $\lim_{x \to \infty} \frac{\ln{x}}{x^k}$. I want to evaluate this limit for $k >0$.
So I tried this:
with assuming(k>0):
limit((log(x))/(x**k),x,oo)
I also tried this:
eval(limit((log(x))/(x**k),x,oo),k>0)
But regardless, I get this error:
NotImplementedError: Result depends on the sign of -sign(k)
In the case of
with assume(k>0):
limit((log(x))/(x**k),x,oo)
I get this error:
TypeError: 'module' object is not callable
Any ideas what I'm doing wrong?
This seems to work. The first answer in the thread that you linked says that "The assumption system of SymPy is kind of a mess right now". I'm not sure if that has changed since then.
k = Symbol('k', positive=True)
print limit((log(x))/(x**k),x,oo)
Using the scipy.optimize.minimize() function I went trough different results using different methods for the same objective function. To evaluate the goodness-of-fit I use to look at the reduced chi squared as a first criterion. After some time I ended with this useful guide http://newville.github.io/lmfit-py/fitting.html#Minimizer where it is specified that the reduced chi squared is set as attribute of the Minimizer object returned from the minimize() function. But if I do
minobj = scipy.optimize.minimize(...)
minobj.redchi
I get
AttributeError: redchi
Meanwhile minobj.message and minobj.success are correctly displayed.
Any guess?
The documentation is a little misleading --- if you look at lmfit/minimizer.py, and do a string search for "redchi" in the entire file, it only appears once, and that is in the leastsq() method. So basically, it only calculates the reduced chi squared for least-squares fitting.
If you're feeling up to it, you could add redchi to the other methods in the appropriate places, fork the lmfit github repo, and commit your changes.
In addition to Ashwin's answer, you could always just use:
result = lmfit.minimize(...)
x2 = result.chisqr
nfree = result.nfree
red_x2 = x2/nfree