If I want to print some variable for debugging in theano, it is easy, just write
x2 = printing.Print('x is: ')(x), and then use x2 instead of x in the following computations.
But what if I want to print some expression of x, for example x+y. How can I do it?
If I write z = printing.Print('x+y is: ')(x+y) then I will need to insert z into the computation graph later, what is the recommended way to do it?
The result of a print operation must be reachable (via some path in the computation graph) from an output of the Theano function. If you want to print an expression that is not used then a simple solution is to just include the result of that expression in the outputs of the Theano function.
Suppose you are interested in x*y but would like to print x+y, then
x = theano.tensor.scalar()
y = theano.tensor.scalar()
z = printing.Print('x+y is: ')(x+y)
f1 = theano.function([x, y], [x * y]
f2 = theano.function([x, y], [z]
f3 = theano.function([x, y], [x * y, z]
f1 will fail to print x+y because z is not reachable from an output of the function; f2 will print x+y but will not compute x*y; f3 will do both.
This is a nasty hack, but I've resorted to things like:
x = 1e-11 * Print("mean of x")(x.mean()) + x
If you make it 0 * Print(...) then it gets optimised away.
Related
My first py file is the function that I want to find the roots, like this:
def myfun(unknowns,a,b):
x = unknowns[0]
y = unknowns[1]
eq1 = a*y+b
eq2 = x**b
z = x*y + y/x
return eq1, eq2
And my second one is to find the value of x and y from a starting point, given the parameter value of a and b:
a = 3
b = 2
x0 = 1
y0 = 1
x, y = scipy.optimize.fsolve(myfun, (x0,y0), args= (a,b))
My question is: I actually need the value of z after plugging in the result of found x and y, and I don't want to repeat again z = x*y + y/x + ..., which in my real case it's a middle step variable without an explicit expression.
However, I cannot replace the last line of fun with return eq1, eq2, z, since fslove only find the roots of eq1 and eq2.
The only solution now is to rewrite this function and let it return z, and plug in x and y to get z.
Is there a good solution to this problem?
I believe that's the wrong approach. Since you have z as a direct function of x and y, then what you need is to retrieve those two values. In the listed case, it's easy enough: given b you can derive x as the inverse of eqn2; also given a, you can invert eqn1 to get y.
For clarity, I'm changing the names of your return variables:
ret1, ret2 = scipy.optimize.fsolve(myfun, (x0,y0), args= (a,b))
Now, invert the two functions:
# eq2 = x**b
x = ret2**(1/b)
# eq1 = a*y+b
y = (ret1 - b) / a
... and finally ...
z = x*y + y/x
Note that you should remove the z computation from your function, as it serves no purpose.
I'm trying to make some basic substitutions but SymPy doesn't want to help me out
x, y, z, k = symbols("x y z k", positive=True, real=True)
exp = x**4 + x**3 + x**2 + x
what_im_expecting = simplify(y**(Rational(1/4)) + y**(Rational(3/4)) + sqrt(y) + y)
what_i_actually_get = exp.subs(x**4,y)
exp, what_i_actually_get, what_im_expecting
returns
x + y**(Rational(3, 4)) + sqrt(y) + y
Can anyone help me out?
a more complex example:
The method subs can be trusted to replace the terms that exactly match the given "old" expression, which x**4 here. The replacement of other things related to x**4 is not so certain. (There are many open issues with subs: some say it substitutes too much, some say too little.) There is some substitution logic specific to powers, but x by itself is not formally a power, so it escapes that logic. A workaround: temporarily replace x by x**1, preventing automatic evaluation of that power to x.
x1 = sp.Pow(x, 1, evaluate=False)
subbed = exp.subs(x, x1).subs(x**4, y).subs(x1, x)
Now subbed is y**(3/4) + y**(1/4) + sqrt(y) + y.
But, don't expect human-like ingenuity from subs. With the same workaround, trying to do subs(x**4 - 1, y) results in x**3 + x**2 + x + y + 1: nothing like sqrt(y+1), etc, appears. It's better to substitute in the most direct way possible:
subs(x, (y+1)**Rational(1, 4))
Then you don't need any workarounds.
Pytorch version 0.3.1
EDIT: I am rewriting this question to be simpler, as I've narrowed down the bug.
I have some variables:
x = ag.Variable(torch.ones(1, 1), requires_grad = True)
y = ag.Variable(torch.ones(1, 1), requires_grad = True)
z = ag.Variable(torch.ones(1, 1), requires_grad = True)
I then create a variable representing their concatenation:
w = torch.cat([x, y, z])
f = x + y + z
Then I try to take derivatives:
ag.grad(f, x, retain_graph=True, create_graph=True)
This is fine and returns 1, as expected. Same for y and z.
However,
ag.grad(f, w, retain_graph=True, create_graph=True)
Returns an error: RuntimeError: differentiated input is unreachable
Of course that makes sense - w is not explicitly used in the declaration of f. However, I’d like a behavior where one line of code can generate something like [1; 1; 1] as output.
Let’s say I wanted to conveniently batch my variables together, and then take the gradient of the whole shebang at once, rather than processing variables independently (which can make bookkeeping a nightmare). Is there any way to get the outcome I desire?
Does something like this work or you want to keep f = x + y + z?
w = torch.cat([x, y, z])
f = w[0] + w[1] + w[2]
print (ag.grad(f, w, retain_graph=True, create_graph=True))
# output (tensor([[ 1.],[ 1.],[ 1.]]),)
I'm trying to compute higher derivatives of f(y(x)) w.r.t to x using sympys diff.
from sympy import *
from IPython.display import display
init_printing(use_latex=True)
x = symbols('x')
f, y = symbols('f, y', cls=Function)
d2 = diff(f(y(x)),x,2)
print(d2)
print(d2.doit())
Sympy returns :
Derivative(y(x), x)**2*Derivative(f(y(x)), y(x), y(x)) + Derivative(y(x), x, x)*Subs(Derivative(f(_xi_1), _xi_1), (_xi_1,), (y(x),))
Derivative(f(y(x)), y(x))*Derivative(y(x), x, x) + 2*Derivative(y(x), x)**2*Derivative(f(y(x)), y(x), y(x))
Latex image: Sympy result.
While the first result seems to be correct, I do not understand the factor 2 in the second expression after the doit() operation.
It looks like you stumbled upon a bug, which was just fixed a few weeks ago.
You can test this by substituting f, y, and x with some functions or values respectively (append to your code):
f_ex = Lambda(x, x**2)
y_ex = Lambda(x, sin(x))
x_ex = 2
substitutions = [ (f,f_ex), (y,y_ex), (x,x_ex) ]
print( d2.subs(substitutions).doit().n() ) #-1.30728724172722
print( d2.doit().subs(substitutions).doit().n() ) #-0.960930862590836
The printed values should be the same.
The issue can be further isolated to:
print((Derivative(f(y(x)), x, x)))
print((Derivative(f(y(x)), y(x), y(x))).doit())
Here, a plain doit simply adds a factor of 2, which is obviously wrong.
I have following test program. My query is two folded: (1) Some how the solution is giving zero and (2) Is it appropriate to use this x2= np.where(x > y, 1, x) kind of conditions on variables ? Are there any constrained optimization routines in Scipy ?
a = 13.235
b = 70.678
def system(X, a,b):
x=X[0]
y=X[1]
x2= np.where(x > y, 1, x)
f=np.zeros(3)
f[0] = 2*x2 - y - a
f[1] = 3*x2 + 2*y- b
return (X)
func= lambda X: system(X, a, b)
guess=[5,5]
sol = optimize.root(func,guess)
print(sol)
edit: (2a) Here with x2= np.where(x > y, 1, x) condition, two equations becomes one equation.
(2b) In another variation requirement is: x2= np.where(x > y, x^2, x^3). Let me comments on these two as well. Thanks !
First up, your system function is an identity, since you return X instead of return f. The return should be the same shape as the X so you had better have
f = np.array([2*x2 - y - a, 3*x2 + 2*y- b])
Next the function, as written has a discontinuity where x=y, and this is causing there to be be a problem for the initial guess of (5,5). Setting the initial guess to (5,6) allows for the the solution [13.87828571, 14.52157143] to be found rapidly.
With the second example, again using an initial guess of [5,5] causes problems of discontinuity, using [5,6] gives a solution of [ 2.40313743, 14.52157143].
Here is my code:
import numpy as np
from scipy import optimize
def system(X, a=13.235, b=70.678):
x = np.where(X[0] > X[1], X[0]**2, X[0]**3)
y=X[1]
return np.array( [2*x - y - a, 3*x + 2*y - b])
guess = [5,6]
sol = optimize.root(system, guess)
print(sol)