I am trying to find a solution to the following system where f and g are R^2 -> R^2 functions:
f(x1,x2) = (y1,y2)
g(y1,y2) = (x1,x2)
I tried solving it using scipy.optimize.fsolve as follows:
def eqm(vars):
x1,x2,y1,y2 = vars
eq1 = f([x1, x2])[0] - y1
eq2 = f([x1, x2])[1] - y2
eq3 = g([y1, y2])[0] - x1
eq4 = g([y1, y2])[1] - x2
return [eq1, eq2, eq3, eq4]
fsolve(eqm, x0 = [1,0.5,1,0.5])
Although it is returning an output, it does not seem to be a correct one as it does not seem to satisfy the two conditions, and seems to vary a lot with the x0 specified. Also getting a warning:
'The iteration is not making good progress, as measured by the improvement from the last ten iterations.' I do know for a fact that a unique solution exists, which I have obtained algebraically.
Not sure what is going on and if there is a simpler way of solving it, especially using just two equations instead of splitting up into 4. Something like:
def equations(vars):
X,Y = vars
eq1 = f(X)-Y
eq2 = g(Y)-X
return [eq1, eq2]
fsolve(equations, x0 =[[1,0.5],[1,0.5]])
Suggestions on other modules e.g. sympy are also welcome!
First, I recommend working with numpy arrays since manipulating these is simpler than lists.
I've slighlty rewritten your code:
import scipy.optimize as opt
def f(x):
return x
def g(x):
return x
def func(vars):
input = np.array(vars)
eq1 = f(input[:2]) - input[2:]
eq2 = g(input[2:]) - input[:2]
return np.concatenate([eq1, eq2])
root = opt.fsolve(func, [1, 1, 0., 1.2])
print(root)
print(func(root)) # should be close to zeros
What you have should work correctly, so I believe there is something wrong with the equations you're using. If you provide those, I can try to see what may be wrong.
This seems to be more of a problem of numerical mathematics than Python coding. Your functions may have "ugly" behavior around the solution, may be strongly non-linear or contain singularities. We cannot help further without seeing the functions. One thing you might try is to instead solve a system
g(f(x)) - x = 0
and simplify g(f(x)) as much as possible analytically. Then calculate y = f(x) after solving the equation.
Related
I am working with a fairly complex objective function that I am minimizing by varying 4 parameters. A while ago I decided to use the Python framework Mystic, which seamlessly allows me to use penalties for complex inequalities (which I need).
However, Mystic has a less-than-obvious way to assign hard constraints (not inequalities and not bound constraints, linear inequalities between parameters only) and even less obvious way to handle them.
All my 4 parameters have finite lower and upper bounds. I would like to add a linear inequality as a hard constraint like this:
def constraint(x): # needs to be <= 0
return x[0] - 3.0*x[2]
But if I try to use Mystic in this way:
from mystic.solvers import fmin_powell
xopt = fmin_powell(OF, x0=x0, bounds=bounds, constraints=constraint)
Then Mystic insists in calling the objective function to resolve the constraints first and then proceed with the actual optimization; since the objective function value has no impact nor any effect on the constraint function as defined above then I am not sure why this is happening. The constraint function defined above simply tells Mystic that a region of the hyperparameters search space should be off limits.
I have scoured pretty much all the examples in the Mystic folder and I stumbled across an alternative way to define a hard constraint: use a penalty function and then call a magic method "as_constraint" to "convert it" to a constraint. Unfortunately, all those examples go pretty much this way:
from mystic.solvers import fmin_powell
from mystic.constraints import as_constraint
from mystic.penalty import quadratic_inequality
def penalty_function(x): # <= 0.0
return x[0] - 3.0*x[2]
#quadratic_inequality(penalty_function)
def penalty(x):
return 0.0
solver = as_constraint(penalty)
result = fmin_powell(OF, x0=x0, bounds=bounds, penalty=penalty)
There is this magic line:
solver = as_constraint(penalty)
That I can't see what it's doing - the solver variable is never used again.
So, for the question: is there any way to define linear inequalities in Mystic that do not involve an expensive pre-solve of the constraints but simply tell Mystic to exclude certain regions of the search space?
Thank you in advance for any suggestion.
Andrea.
What mystic does is map the space it searches, so you are optimizing over a "kernel transformed" space (to use machine learning jargon). You can think of the constraints as applying an operator, if you know that that means. So, y = f(x) under some constraints x' = c(x) becomes y = f(c(x)). This is why the optimizer evaluates the constraints before evaluating the objective.
So you can build a constraint like this:
>>> import mystic.symbolic as ms
>>> equation = 'x1 - 3*a*x2 <= 0'
>>> eqn = ms.simplify(equation, locals=dict(a=1), all=True)
>>> print(eqn)
x1 <= 3*x2
>>> c = ms.generate_constraint(ms.generate_solvers(eqn, nvars=3))
>>> c([1,2,3])
[1, 2, 3]
>>> c([0,100,-100])
[0, -300.0, -100]
Or if you have more than one:
>>> equation = '''
... x1 > x2 * x0
... x0 + x1 < 10
... x1 + x2 > 5
... '''
>>> eqn = ms.simplify(equation, all=True)
>>> print(eqn)
x1 > -x2 + 5
x0 < -x1 + 10
x1 > x0*x2
>>> import mystic.constraints as mc
>>> c = ms.generate_constraint(ms.generate_solvers(eqn), join=mc.and_)
>>> c([1,2,3])
[1, 3.000000000000004, 3]
I've defined the following function as a method of approximating an integral using Boole's Rule:
def integrate_boole(f,l,r,N):
h=((r-l)/N)
xN = np.linspace(l,r,N+1)
fN = f(xN)
return ((2*h)/45)*(7*fN[0]+32*(np.sum(fN[1:-2:2]))+12*(np.sum(fN[2:-3:4]))+14*(np.sum(fN[4:-5]))+7*fN[-1])
I used the function to get the value of the integral for sin(x)dx between 0 and pi (where N=8) and assigned it to a variable sine_int.
The answer given was 1.3938101893248442
After doing the original equation (see here) out by hand I realised this answer was quite inaccurate.
The sums of fN are giving incorrect values, but I'm not sure why. For example, np.sum(fN[4:-5]) is going to 0.
Is there a better way of coding the sums involved, or is there an error in my parameters that's causing the calculations to be inaccurate?
Thanks in advance.
EDIT
I should have made it clearer that this is supposed to be a composite version of the rule, i.e. approximating over N points where N is divisible by 4. So the typical 5 points with 4 intervals isn't going to cut it here, unfortunately. I would copy the equation I'm using into here, but I don't have an image of it and LaTex isn't an option. It should/might be clear from the code I have after return.
From a quick inspection looks like the term multiplying f(x_4) should be 32, not 14:
def integrate_boole(f,l,r,N):
h=((r-l)/N)
xN = np.linspace(l,r,N+1)
fN = f(xN)
return ((2*h)/45)*(7*fN[0]+32*(np.sum(fN[1:-2:2]))+
12*(np.sum(fN[2:-3:4]))+32*(np.sum(fN[4:-5]))+7*fN[-1])
First, one of your coefficients was wrong as pointed out by #nixon. Then, I think you do not really understand how the Boole's rule works - It approximates the integral of a function only using 5 points of the function. Hence, the terms like np.sum(fN[1:-2:2]) makes no sense. You only need five points, which you can obtain with xN = np.linspace(l,r,5). Your h is simply the distance between 2 of the contiguos points h = xN[1] - xN[0]. And then, easy peasy:
import numpy as np
def integrate_boole(f,l,r):
xN = np.linspace(l,r,5)
h = xN[1] - xN[0]
fN = f(xN)
return ((2*h)/45)*(7*fN[0]+32*fN[1]+12*fN[2]+32*fN[3]+7*fN[4])
def f(x):
return np.sin(x)
I = integrate_boole(f, 0, np.pi)
print(I) # Outputs 1.99857...
I'm not sure what you're hoping your code does w.r.t. Boole's rule. Why are you summing over samples of the function (i.e. np.sum(fN[2:-3:4]))? I think your N parameter is also not well defined and I'm not sure what it's supposed to represent. Maybe you're using another rule I'm not familiar with: I'll let you decide.
Regardless, here's an implementation of Boole's rule as Wikipedia defines it. Variables map to the Wikipedia version you linked:
def integ_boole(func, left, right):
h = (right - left) / 4
x1 = left
x2 = left + h
x3 = left + 2*h
x4 = left + 3*h
x5 = right # or left + 4h
result = (2*h / 45) * (7*func(x1) + 32*func(x2) + 12*func(x3) + 32*func(x4) + 7*func(x5))
return result
then, to test:
import numpy as np
print(integ_boole(np.sin, 0, np.pi))
outputs 1.9985707318238357, which is extremely close to the correct answer of 2.
HTH.
I need to solve a large system of non-linear equations (static truss system).
The equations are derived from nodes (xyz) and their constraints (position, forces).
For now we are using Mathematica for this task, but we would like to migrate to Python.
But with Mathematica (or EES (engineering equation solver) or SymPy) it is pretty convenient. I throw a bunch of stuff in like node positions or forces on nodes and it does some magic and creates the equations by itself combining the input and solve them.
If I want to use scipy.optimize.root I have to somehow get the equations.
scipy.optimize.root and scipy.optimize.fsolve need the equations in the following format:
def func(x):
out = [x[0]*cos(x[1]) - 4],
x[1]*x[0] - x[1] - 5)
return out
But in my case there will be up to 5000 equations that define the system.
One thing that came to my mind would be to use eval() and fiddle the equations to a string somehow.
In the end I would like to have an object-oriented approach where a node or a constraint knows how to transform itself to an equation.
A very simple skeleton could be
n = Node(5, 2, 6)
n.to_equation()
f = ForceConstraint(1, 2, 3)
f.to_equation()
and this would somehow transform to equations like
x[0] - 5,
x[1] - 2,
x[2] - 6,
x[2] ** 2 - x[1] * x[0] # and some non-linear stuff
to describe the overall system.
Basically there should be some magic part that looks at the matching parts of the equations and constraints.
E.g. look through all info you have on Node1's x direction and merge that into equations, or search for all info you have on the forces in y direction on Node2.
Is scipy the correct tool for the job at all?
Does someone have a good idea how to do this?
I think you might be interested in symfit. It's a package I wrote to connect scipy with sympy.
I'm not exactly sure what your specific equations are, but any expression you can write in sympy can in principle be fed to symfit. For example, for your simple example above you could write:
from symfit import parameters, variables, Fit
import numpy as np
x0, x1, x2 = parameters('x0, x1, x2')
y0, y1, y2, y3 = variables('y0, y1, y2, y3')
model_dict = {
y0: x0 - 5,
y1: x1 - 2,
y2: x2 - 6,
y3: x2 ** 2 - x1 * x0
}
fit = Fit(model_dict, y0=np.array(0.0), y1=np.array(0.0), y2=np.array(0.0), y3=np.array(0.0))
fit_result = fit.execute()
print(fit_result)
Variable and Parameter objects in symfit are just sympy Symbol subclasses, so you can do all the sympy manipulations on these expressions that you want. For example, you could have defined your Node as
>>> x, x_0 = symbols('x, x_0')
>>> Node = x - x_0
And then make the lines of your model by repeatedly applying e.g.
>>> Node.subs({x: x1, x_0: 2})
x1 - 2
Lastly you add your constraint and presto: fittable model! Check the docs for more info or ask me any follow up question.
I have just read Using adaptive time step for scipy.integrate.ode when solving ODE systems
.
My code below works fine but the results it produces when solving more complicated equations rather than the one I have provided in the example below, the differential equations seem inaccurate. Is there a way to change this code so that it automatically adapts the time-step according to speci ed absolute and
relative error tolerances? eg. 10^-8?
from scipy.integrate import ode
initials = [0.5,0.2]
integration_range = (0, 30)
def f(t,X):
x,y = X[0],X[1]
dxdt = x**2 + y
dydt = y**2 + x
return [dxdt,dydt]
X_solutions = []
t_solutions = []
def solution_getter(t,X):
t_solutions.append(t)
X_solutions.append(X.copy())
backend = "dopri5"
ode_solver = ode(f).set_integrator(backend)
ode_solver.set_solout(solution_getter)
ode_solver.set_initial_value(y=initials, t=0)
ode_solver.integrate(integration_range[1])
You could set the values of rtol and atol in the set_integrator call, see https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.ode.html.
The default values provide a medium accuracy that is good enough for graphics, but may not be enough for other purposes.
Suppose I have a function f(x) defined between a and b. This function can have many zeros, but also many asymptotes. I need to retrieve all the zeros of this function. What is the best way to do it?
Actually, my strategy is the following:
I evaluate my function on a given number of points
I detect whether there is a change of sign
I find the zero between the points that are changing sign
I verify if the zero found is really a zero, or if this is an asymptote
U = numpy.linspace(a, b, 100) # evaluate function at 100 different points
c = f(U)
s = numpy.sign(c)
for i in range(100-1):
if s[i] + s[i+1] == 0: # oposite signs
u = scipy.optimize.brentq(f, U[i], U[i+1])
z = f(u)
if numpy.isnan(z) or abs(z) > 1e-3:
continue
print('found zero at {}'.format(u))
This algorithm seems to work, except I see two potential problems:
It will not detect a zero that doesn't cross the x axis (for example, in a function like f(x) = x**2) However, I don't think it can occur with the function I'm evaluating.
If the discretization points are too far, there could be more that one zero between them, and the algorithm could fail finding them.
Do you have a better strategy (still efficient) to find all the zeros of a function?
I don't think it's important for the question, but for those who are curious, I'm dealing with characteristic equations of wave propagation in optical fiber. The function looks like (where V and ell are previously defined, and ell is an positive integer):
def f(u):
w = numpy.sqrt(V**2 - u**2)
jl = scipy.special.jn(ell, u)
jl1 = scipy.special.jnjn(ell-1, u)
kl = scipy.special.jnkn(ell, w)
kl1 = scipy.special.jnkn(ell-1, w)
return jl / (u*jl1) + kl / (w*kl1)
Why are you limited to numpy? Scipy has a package that does exactly what you want:
http://docs.scipy.org/doc/scipy/reference/optimize.nonlin.html
One lesson I've learned: numerical programming is hard, so don't do it :)
Anyway, if you're dead set on building the algorithm yourself, the doc page on scipy I linked (takes forever to load, btw) gives you a list of algorithms to start with. One method that I've used before is to discretize the function to the degree that is necessary for your problem. (That is, tune \delta x so that it is much smaller than the characteristic size in your problem.) This lets you look for features of the function (like changes in sign). AND, you can compute the derivative of a line segment (probably since kindergarten) pretty easily, so your discretized function has a well-defined first derivative. Because you've tuned the dx to be smaller than the characteristic size, you're guaranteed not to miss any features of the function that are important for your problem.
If you want to know what "characteristic size" means, look for some parameter of your function with units of length or 1/length. That is, for some function f(x), assume x has units of length and f has no units. Then look for the things that multiply x. For example, if you want to discretize cos(\pi x), the parameter that multiplies x (if x has units of length) must have units of 1/length. So the characteristic size of cos(\pi x) is 1/\pi. If you make your discretization much smaller than this, you won't have any issues. To be sure, this trick won't always work, so you may need to do some tinkering.
I found out it's relatively easy to implement your own root finder using the scipy.optimize.fsolve.
Idea: Find any zeroes from interval (start, stop) and stepsize step by calling the fsolve repeatedly with changing x0. Use relatively small stepsize to find all the roots.
Can only search for zeroes in one dimension (other dimensions must be fixed). If you have other needs, I would recommend using sympy for calculating the analytical solution.
Note: It may not always find all the zeroes, but I saw it giving relatively good results. I put the code also to a gist, which I will update if needed.
import numpy as np
import scipy
from scipy.optimize import fsolve
from matplotlib import pyplot as plt
# Defined below
r = RootFinder(1, 20, 0.01)
args = (90, 5)
roots = r.find(f, *args)
print("Roots: ", roots)
# plot results
u = np.linspace(1, 20, num=600)
fig, ax = plt.subplots()
ax.plot(u, f(u, *args))
ax.scatter(roots, f(np.array(roots), *args), color="r", s=10)
ax.grid(color="grey", ls="--", lw=0.5)
plt.show()
Example output:
Roots: [ 2.84599497 8.82720551 12.38857782 15.74736542 19.02545276]
zoom-in:
RootFinder definition
import numpy as np
import scipy
from scipy.optimize import fsolve
from matplotlib import pyplot as plt
class RootFinder:
def __init__(self, start, stop, step=0.01, root_dtype="float64", xtol=1e-9):
self.start = start
self.stop = stop
self.step = step
self.xtol = xtol
self.roots = np.array([], dtype=root_dtype)
def add_to_roots(self, x):
if (x < self.start) or (x > self.stop):
return # outside range
if any(abs(self.roots - x) < self.xtol):
return # root already found.
self.roots = np.append(self.roots, x)
def find(self, f, *args):
current = self.start
for x0 in np.arange(self.start, self.stop + self.step, self.step):
if x0 < current:
continue
x = self.find_root(f, x0, *args)
if x is None: # no root found.
continue
current = x
self.add_to_roots(x)
return self.roots
def find_root(self, f, x0, *args):
x, _, ier, _ = fsolve(f, x0=x0, args=args, full_output=True, xtol=self.xtol)
if ier == 1:
return x[0]
return None
Test function
The scipy.special.jnjn does not exist anymore, but I created similar test function for the case.
def f(u, V=90, ell=5):
w = np.sqrt(V ** 2 - u ** 2)
jl = scipy.special.jn(ell, u)
jl1 = scipy.special.yn(ell - 1, u)
kl = scipy.special.kn(ell, w)
kl1 = scipy.special.kn(ell - 1, w)
return jl / (u * jl1) + kl / (w * kl1)
The main problem I see with this is if you can actually find all roots --- as have already been mentioned in comments, this is not always possible. If you are sure that your function is not completely pathological (sin(1/x) was already mentioned), the next one is what's your tolerance to missing a root or several of them. Put differently, it's about to what length you are prepared to go to make sure you did not miss any --- to the best of my knowledge, there is no general method to isolate all the roots for you, so you'll have to do it yourself. What you show is a reasonable first step already. A couple of comments:
Brent's method is indeed a good choice here.
First of all, deal with the divergencies. Since in your function you have Bessels in the denominators, you can first solve for their roots -- better look them up in e.g., Abramovitch and Stegun (Mathworld link). This will be a better than using an ad hoc grid you're using.
What you can do, once you've found two roots or divergencies, x_1 and x_2, run the search again in the interval [x_1+epsilon, x_2-epsilon]. Continue until no more roots are found (Brent's method is guaranteed to converge to a root, provided there is one).
If you cannot enumerate all the divergencies, you might want to be a little more careful in verifying a candidate is indeed a divergency: given x don't just check that f(x) is large, check that, e.g. |f(x-epsilon/2)| > |f(x-epsilon)| for several values of epsilon (1e-8, 1e-9, 1e-10, something like that).
If you want to make sure you don't have roots which simply touch zero, look for the extrema of the function, and for each extremum, x_e, check the value of f(x_e).
I've also encountered this problem to solve equations like f(z)=0 where f was an holomorphic function. I wanted to be sure not to miss any zero and finally developed an algorithm which is based on the argument principle.
It helps to find the exact number of zeros lying in a complex domain. Once you know the number of zeros, it is easier to find them. There are however two concerns which must be taken into account :
Take care about multiplicity : when solving (z-1)^2 = 0, you'll get two zeros as z=1 is counting twice
If the function is meromorphic (thus contains poles), each pole reduce the number of zero and break the attempt to count them.