im new into Python and i try to figure out how everythings work. I have a little problem with the minimize function of the scipy.optimize package. I try to minimize a given function with some start values but python gives me very high parameter values.
This ist my simple code:
import numpy as np
from scipy.optimize import minimize
global array
y_wert = np.array([1,2,3,4,5,6,7,8])
global x_wert
x_wert = np.array([1,2,3,4,5,6,7,8])
def Test(x):
Summe = 0
for i in range(0,len(y_wert)):
Summe = Summe + (y_wert[i] - (x[0]*x_wert[i]+x[1]))
return(Summe)
x_0 = [1,0]
xopt = minimize(Test,x_0, method='nelder-mead',options={'xatol': 1e-8, 'disp': True})
print(xopt)
If i run this script the best given parameters are:
[1.02325529e+44, 9.52347084e+40]
which really doesnt solve this problem. Ive also try some slightly different startvalues but that doesnt solve my problems.
Can anyone give me a clue as to where my mistake lies?
Thanks a lot for your help!
Your test function is effectively a straight line with negative gradient so there is no minimum, it's an infinitely decreasing function, that explains your large results, try something like x squared instead
Related
I am implementing Andrew Ng's Machine Learning course on Python, but I got stuck because the scipy's optimize functions keep giving me a hard time by not working/giving me dimension errors
The goal is to find the minimum of the cost function (a scalar function that takes theta (dimension (1,401)), X (dimension (5000,401)), and y (dimension (5000,1)) as inputs). I have defined such cost function and its gradient wrt parameters. When running one of the optimize functions (I have tried fmin_tnc, minimize, Nelder-Mead and others, all not working), either they run for ages or keep giving me errors saying that the array dimension is wrong, or that they find a division by 0... errors that I am not able to spot.
weirdest thing is that this problem has popped up at first when I was doing exercise 2 on logistic regression, and then magically disappeared without me changing anything. Now, Implementing multi-classification logistic regression, it has appeared again, and it won't fix even though I have literally copied and pasted the code of exercise 2!
The code is the following:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy.io import loadmat
import scipy.misc
import matplotlib.cm as cm
from scipy.optimize import minimize,fmin_tnc
import random
def sigmoid(z):
return 1/(1+np.exp(-z))
def J(theta,X,y):
theta_t=np.transpose(theta)
prod=np.matmul(X,theta_t)
sigm=sigmoid(prod)
vec=y*np.log(sigm)+(1-y)*np.log(1-sigm)
return -np.sum(vec)/len(y)
def grad(theta,X,y):
theta_t=np.transpose(theta)
prod=np.matmul(X,theta_t)
sigm=sigmoid(prod)
one=sigm-y
return np.matmul(np.transpose(one),X)/len(y)
data=loadmat('/home/marco/Desktop/MLang/mlex3/ex3/ex3data1.mat')
X,y = data['X'],data['y']
X=np.column_stack((np.ones(len(X[:,0])),X))
initial_theta=np.zeros((1,len(X[0,:])))
res=fmin_tnc(func=J, x0=initial_theta.flatten(), args=(X,y.flatten()), fprime=grad)
theta_opt=res[0]
Instead of returning the value of theta that minimizes the function as theta_opt, it says:
/home/marco/anaconda3/lib/python3.6/site-packages ipykernel_launcher.py:8: RuntimeWarning: divide by zero encountered in log
I have no clue where this divide by zero occurs, given that there is literally no division in the whole code, except for the division by len(y), which is 5000, and the division in the sigmoid function (1/(1+exp(-z)), which can never be 0!
Any suggestions?
I am trying to solve a system of simultaneous equations as follows:
"145.0x/21025 = -0.334"
"(-48.402x-96.650y+96.650z)/21025 = -0.334"
"(-48.402x+132.070y+35.214z)/21025 = -0.334"
"sqrt(x^2+y^2+z^2) = 145.0"
I am using the following Python script:
from scipy.optimize import root
from numpy import sqrt
from sys import argv, stdout
initGuesses = eval(argv[1])
equations = argv[2:]
def f(variables):
x,y,z = variables
results = []
for eqn in equations:
results.append(eval(eqn))
return results
solution = root(f, initGuesses, method="lm")
stdout.write(str(solution["x"][0]) + "," + str(solution["x"][1]) + "," + str(solution["x"][2]))
stdout.flush()
The program is called as follows:
python3 SolvePosition3D.py "(1,1,1)" "(145.0*x+0.0*y+0.0*z)/21025.0+0.334" "(-48.402*x-96.650*y+96.650*z)/21025+0.334" "(-48.402*x+132.070*y+35.214*z)/21025+0.334" "sqrt(x**2+y**2+z**2)-145.0"
And I am receiving the following output:
48.2699997956,35.4758788666,132.042180583
This solution is wrong; the correct solution is roughly
-48,-35,-132
which is the same numbers but * -1.
The answer returned by the program satisfies the final equation, but violates all others.
Does anyone know why this is happening? Getting the correct solutions to these equations (and many others like them) is vitally important to my current project.
I was able to run the code via adding
from numpy import sqrt
from scipy.optimize import root
and switching to regular old prints for the solution.
Your starting guess seems to be wrong. Starting from (1, 1, 1), the root finder converges to 48.2699997956,35.4758788666,132.042180583, as you said. If you plug in (-1,-1,-1), you get -48.2649482763,-35.4698607274,-132.050694891 as expected.
As for why it does that, well, nonlinear systems are just hard to solve like that. Most algorithms tend to find solutions deterministically based on the starting point. If you need to try multiple starting points, try a grid-based search.
I want to minimize a function in order to obtain some parameters' value of : a,e,I,Omega,om,tp.
I use this "module" : docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html.
My function has 13 parameters:
I imported : from scipy.optimize import minimize. Then I try to minimize it.And the error occurs:
Would someone help to solve this problem?
PS: I started python one week ago that may explain this syntax of the program, however I'm willing to improve myself.
from numpy import *
import numpy as np
import scipy as sp
from scipy.optimize import minimize
import matplotlib.pyplot as plt
from pylab import *
from os import chdir
chdir("/Users/benjaminjaillant/Desktop")
def Chi_VLT(a,e,I,tp,Omega,om,Mbh,R0,Vr_bh,alpha_bh,V_alp_bh,delta_bh,V_del_bh):
return sum(((Vr_etoile(t_vr_VLT*365*24*3600,a,e,I,tp,om,Mbh,Vr_bh)/1000)-vr_VLT)**2/vr_error_VLT**2) + sum(((alpha_etoile_IR(t_orbit_VLT*365*24*3600,a,e,I,tp,Omega,om,Mbh,alpha_bh,V_alp_bh,R0)*206264806.246)-Ra_VLT)**2/Ra_error_VLT**2) + sum(((delta_etoile_IR(t_orbit_VLT*365*24*3600,a,e,I,tp,Omega,om,Mbh,delta_bh,V_del_bh,R0)*206264806.246)-Dec_VLT)**2/Dec_error_VLT**2)
x0 = [1.5e14,0.8,2.5,63.10e9,4,1,8.5e36,2.5e20,2000,1.3e-8,-10e-18,2e-9,1.5e-17]
res = minimize(Chi_VLT, x0 , method='nelder-mead',options={'xtol': 1e-4,'maxiter':50 ,'disp': True})
print res.message
print res.x
I guess you are messing with whole thing here.
your function scipy.optimize.minimize takes two required positional argument,
fun and x0.
you need ndarray as x0
In your case your fun Chi_VLT requires 13 arguments, you need to pass that using args=(tuple, containing, 13, items)
Then only you will be able to minimize your fun.
The minimize routine expects an ndarray, say guess, as initial function arguments and accepts an additional tuple of arguments x0 which might constitute the coefficients in your cost function. If you rewrite CHI_VLT to take as first arg an ndarray and then the remaining arguments accordingly it should work.
res = minimize(CHI_VLT, guess, args=x0,...)
how can I minimize a function (uncostrained), respect a[0] and a[1]?
example (this is a simple example for I uderstand scipy, numpy and py):
import numpy as np
from scipy.integrate import *
from scipy.optimize import *
def function(a):
return(quad(lambda t: ((np.cos(a[0]))*(np.sin(a[1]))*t),0,3))
i tried:
l=np.array([0.1,0.2])
res=minimize(function,l, method='nelder-mead',options={'xtol': 1e-8, 'disp': True})
but I get errors.
I get the results in matlab.
any idea ?
thanks in advance
This is just a guess, because you haven't included enough information in the question for anyone to really know what the problem is. Whenever you ask a question about code that generates an error, always include the complete error message in the question. Ideally, you should include a minimal, complete and verifiable example that we can run to reproduce the problem. Currently, you define function, but later you use the undefined function chirplet. That makes it a little bit harder for anyone to understand your problem.
Having said that...
scipy.integrate.quad returns two values: the estimate of the integral, and an estimate of the absolute error of the integral. It looks like you haven't taken this into account in function. Try something like this:
def function(a):
intgrl, abserr = quad(lambda t: np.cos(a[0])*np.sin(a[1])*t, 0, 3)
return intgrl
I am fairly new to python and trying to transfer some code from matlab to python. I am trying to optimize a function in python using fmin_bfgs. I always try to vectorize the code when possible, but I ran into the following problem that I can't figure out. Here is a test example.
from pylab import *
from scipy.optimize import fmin_bfgs
## Create some linear data
L=linspace(0,10,100).reshape(100,1)
n=L.shape[0]
M=2*L+5
L=hstack((ones((n,1)),L))
m=L.shape[0]
## Define sum of squared errors as non-vectorized and vectorized
def Cost(theta,X,Y):
return 1.0/(2.0*m)*sum((theta[0]+theta[1]*X[:,1:2]-Y)**2)
def CostVec(theta,X,Y):
err=X.dot(theta)-Y
resid=err**2
return 1.0/(2.0*m)*sum(resid)
## Initialize the theta
theta=array([[0.0], [0.0]])
## Run the minimization on the two functions
print fmin_bfgs(Cost, x0=theta,args=(L,M))
print fmin_bfgs(CostVec, x0=theta,args=(L,M))
The first answer, with the unvectorized function, gives the correct answer which is just the vector [5, 2]. But, the the second answer, using the vectorizied form of the cost function returns roughly [15,0]. I have figured out the 15 doesn't appear from nowhere as it is 2 times the mean of the data plus the intercept, i.e., $2\times 5+5$. Any help is greatly appreciated.