Why does the following code return a ValueError?
from scipy.optimize import fsolve
import numpy as np
def f(p,a=0):
x,y = p
return (np.dot(x,y)-a,np.outer(x,y)-np.ones((3,3)),x+y-np.array([1,2,3]))
x,y = fsolve(f,(np.ones(3),np.ones(3)),9)
ValueError: setting an array element with a sequence.
The basic problem here is that your function f does not satisfy the criteria required for fsolve to work. These criteria are described in the documentation - although arguably not very clearly.
The particular things that you need to be aware of are:
the input to the function that will be solved for must be an n-dimensional vector (referred to in the docs as ndarray), such that the value of x you want is the solution to f(x, *args) = 0.
the output of f must be the same shape as the x input to f.
Currently, your function takes a 2 member tuple of 1x3-arrays (in p) and a fixed scalar offset (in a). It returns a 3 member tuple of types (scalar,3x3 array, 1x3 array)
As you can see, neither condition 1 nor 2 is met.
It is hard to advise you on exactly how to fix this without being exactly sure of the equation you are trying to solve. It seems you are trying to solve some particular equation f(x,y,a) = 0 for x and y with x0 = (1,1,1) and y0 = (1,1,1) and a = 9 as a fixed value. You might be able to do this by passing in x and y concatenated (e.g. pass in p0 = (1,1,1,1,1,1) and in the function use x=p[:3] and y = p[3:] but then you must modify your function to output x and y concatenated into a 6-dimensional vector similarly. This depends on the exact function your are solving for and I can't work this out from the output of your existing f (i.e based on a dot product, outer product and sum based tuple).
Note that arguments that you don't pass in the vector (e.g. a in your case) will be treated as fixed values and won't be varied as part of the optimisation or returned as part of any solution.
Note for those who like the full story...
As the docs say:
fsolve is a wrapper around MINPACK’s hybrd and hybrj algorithms.
If we look at the MINPACK hybrd documentation, the conditions for the input and output vectors are more clearly stated. See the relevant bits below (I've cut some stuff out for clarity - indicated with ... - and added the comment to show that the input and output must be the same shape - indicated with <--)
1 Purpose.
The purpose of HYBRD is to find a zero of a system of N non-
linear functions in N variables by a modification of the Powell
hybrid method. The user must provide a subroutine which calcu-
lates the functions. The Jacobian is then calculated by a for-
ward-difference approximation.
2 Subroutine and type statements.
SUBROUTINE HYBRD(FCN,N,X, ...
...
FCN is the name of the user-supplied subroutine which calculates
the functions. FCN must be declared in an EXTERNAL statement
in the user calling program, and should be written as follows.
SUBROUTINE FCN(N,X,FVEC,IFLAG)
INTEGER N,IFLAG
DOUBLE PRECISION X(N),FVEC(N) <-- input X is an array length N, so is output FVEC
----------
CALCULATE THE FUNCTIONS AT X AND
RETURN THIS VECTOR IN FVEC.
----------
RETURN
END
N is a positive integer input variable set to the number of
functions and variables.
X is an array of length N. On input X must contain an initial
estimate of the solution vector. On output X contains the
final estimate of the solution vector.
Related
Please consider the following python code
import matplotlib.pyplot as plt
import numpy as np
#create some data to plot.
dt = 0.001
t = np.arange(0.0,100,dt)
r = np.exp(-t[:1000]/0.05)
x = np.random.randn(len(t))
s = np.convolve(x,r)[:len(x)]*dt
The code compiles and runs and I largely understand what it is doing. However, I am confused about the code '[:len(x)]' is actually doing. If I truncate 's' to 'np.convolve(x,r)*dt', the code fails to compile and there is an error message from 'base.py' as follows:
"raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (100000,) and (100999,)"
What is '[:len(x)]' actually doing and is there something in the language documentation that gives some examples of this sort of context ?
Thanks.
All the objects are of type 'ndarray'.
t is length 100000
t is of shape (100000,)
r is length 1000
r is of shape (1000,)
x is length 100000
x is of shape (100000,)
s is length 100999
s is of shape (100999,)
If we read the docs for np.convolve, we see that with the default parameters, it returns an array that is one shorter than the sum of the lengths of the input array. That is if you call np.convolve(a, b), and len(a) = A and len(b) = B, the output is length A + B - 1.
This is because a convolution can be interpreted as integrating the product of two functions, with one of the functions shifted relative to the other. By default, np.convolve calculates this convolution for all points at which these functions overlap, so the length of the output is approximately the sum of the lengths of the input functions. In your case, x has length 100,000, and r has length 1,000, so the output length is 100,000 + 1,000 - 1 = 100,999.
You can change this behaviour with the mode parameter, so that np.convolve truncates the output automatically, but neither of the alternate options seem to match your use case. You could try supplying mode = same, which ensures the output is the same length as the longest input, and see what happens for your own interest though.
Since t - length 100,000 - and s need to be the same length so you can plot (I assume) s(t), you need to truncate the output s to a length of 100,000 to match.
This is what the notation [:len(x)] does. This is called "slice" notation, and the gist is that A[start:stop] allows you to select the subset of values in A from start (inclusive) to stop (exclusive). If you don't supply a start or end, it defaults to the start or end of the array respectively. So [:len(x)] picks from 0 to len(x) (exclusive) which gives you an array of length len(x). This ensures len(s) = len(x).
Assume I know that the function f returns a vector of unknown dimension. The function returns value of the form: np.array(list). Can I get the dimension of the return value without evaluating the function?
Consider for instance this function:
def f(x):
f1 = x**2
f2 = x
return np.array([f1,f2]);
I want some something like: dim(f) that gives the answer 2.
Right now I get around the problem by evaluating the function for a random number and then find the dimension: len(f(np.random.rand()))
but I want to avoid the evaluation of the function.
npis off course numpy
This is not possible. The output dimension of the function is dependent on that of its input, it's not static. You can reason about it yourself, in your example the function has an output dimension of [2, *x.ndim]. Sure, if x is a single scalar the dimension will be two, but if x is an array the dimension is gonna be different, et cetera.
In short, if you know your function always takes a scalar or some other value with static dimensions, just put its output dimension in the doc-string. If you take arguments that can be any dimension, you won't get around evaluating the function to find out the dimension of its result.
What Koen Dercksen said is correct, there is no way to determine the dimension of the result without evaluating the function. However, if you do evaluate the function, you should use the .shape property instead of len()
For example:
def f(x):
f1 = x**2
f2 = x
return np.array([f1,f2]);
len = f(0).shape[0]
Also generating a new random number for the function call is unnecessary, you can pass a constant as the parameter and the result will be the same.
I am working with sagemath which uses the python language. In the polynomial ring over the integer ring, I want to define a set whose elements have degree are less than a given number and the absolute value of coefficients are less than a given number.
How to achieve this? For a polynomial, I have defined the degree function
and the max_coefficient function already.
For example,
(x^3-3*x-5).degree(x) will return 3
max_coefficient(x^3-3*x-5) will return 5
The following are my codes.
R=Polynomialring(ZZ,x)
def A(deg_bound,coefficient_bound):
S=set()
for poly in R:
if poly.degree(x)<=deg_bound and max_coefficient(poly)<=coefficient_bound:
S=S.add(poly)
return S
But sagemath tells me I can't do for in the polynomial ring.
It is rather unclear what sort of objects are in R, but you seemto know how to manipulate them...
There is one error that may be causing you trouble:
S = S.add(poly)
is first adding poly to S, then assigning None to S, which is rather unfortunate.
try to replace it with:
S.add(poly)
which accumulates distinct objects into S
Assume we have a function with unknown formula, given few inputs and results of this function, how can we get the function's formula.
For example we have inputs x and y and result r in format (x,y,r)
[ (2,4,8) , (3,6,18) ]
And the desired function can be
f(x,y) = x * y
As you post the question, the problem is too generic. If you want to find any formula mapping the given inputs to the given result, there are simply too many possible formulas. In order to make sense of this, you need to somehow restrict the set of functions to consider. For example you could say that you're only interested in polynomial solutions, i.e. where
r = sum a_ij * x^i * y^j for i from 0 to n and j from 0 to n - i
then you have a system of equations, with the a_ij as parameters to solve for. The higher the degree n the more such parameters you'd have to find, so the more input-output combinations you'd need to know. Variations of this use rational functions (so you divide by another polynomial), or allow some trigonometric functions, or something like that.
If your setup were particularly easy, you'd have just linear equations, i.e. r = a*x + b*y + c. As you can see, even that has three parameters a,b,c so you can't uniquely find all three of them just given the two inputs you provided in your question. And even then the result would not be the r = x*y you were aiming for, since that's technically of degree 2.
If you want to point out that r = x*y is a particularly simple formula, and you would like to look for simple formulas, then one approach would be enumerating formulas in order of increasing complexity. But if you do this without parameters (since ugly parameters will make a simple formula like a*x + b*y + c appear complex), then it's hard to guilde this enumeration towards the one you want, so you'd really have to enumerate all possible formulas, which will become infeasible very quickly.
I am trying to optimize (minimize) a two dimensional function E(n,k) defined as follows:
error=lambda x,y,w: (math.log(abs(Tformulated(x,y,w))) - math.log(abs(Tw[w])))**2 + (math.atan2(Tformulated(x,y,w).imag,Tformulated(x,y,w).real) - math.atan2(Tw[w].imag,Tw[w].real))**2
where Tformulated is obtained as follows :
def Tformulated(n,k,w):
z=1j
L=1
C=0.1
RC=(w*L)/C
n1=complex(1,0)
n3=complex(1,0)
n2=complex(n,k)
FP=1/(1-(((n2-n1)/(n2+n1))*((n2-n3)/(n2+n3))*math.exp(-2*z*n2*RC)))
Tform=((2*n2*(n1+n3))/((n2+n1)*(n2+n3)))*(math.exp(-z*(n2-n1)*RC))*FP
return Tform
and Tw is a list previously calculated having complex valued elements.
What I am exactly trying to do is for each value of w (used in "error x,y,w ....") I want to minimize the function "error" for the values of x & y. w ranges from 1 to 2048. So, it is basically a 2D minimization problem. I have tried programming on my part (though I am getting stuck at what method to use and how to use it); my code is as follows :
temp=[]
i=range(5)
retval = fmin_powell(error , x ,y, args=(i) , maxiter=100 ,maxfun=100)
temp.append(retval)
I am not sure even if fmin_powell is the correct way to go.
Here's a simplest example:
from scipy.optimize import fmin
def minf(x):
return x[0]**2 + (x[1]-1.)**2
print fmin(minf,[1,2])
[out]:
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 44
Function evaluations: 82
[ -1.61979362e-05 9.99980073e-01]
A possible gotcha here is that the minimization routines are expecting a list as an argument. See the docs for all the gory details. Not sure if you can minimize complex-valued functions directly, you might need to consider the real and imaginary parts separately.