This code to calculate the trace of a matrix (based on an example in the Theano "loop" tutorial) works fine:
import numpy as np
import theano as th
import theano.tensor as T
floatX = 'float32'
X = T.matrix()
results = th.scan(lambda i,j,t_f : T.cast(X[i,j] + t_f, floatX),
sequences=[T.arange(X.shape[0]), T.arange(X.shape[1])],
outputs_info=np.asarray(0., dtype=floatX))[0]
result = results[-1]
compute_trace = th.function([X], result)
x = np.eye(5, dtype=floatX)
x[0] = np.arange(5, dtype=floatX)
print compute_trace(x)
But if I remove the cast operation from the lambda function like this:
lambda i,j,t_f : X[i,j] + t_f
The following error message is produced:
ValueError: When compiling the inner function of scan the following error has been encountered: The initial state (outputs_info in scan nomenclature) of variable IncSubtensor{Set;:int64:}.0 (argument number 2) has dtype float32, while the result of the inner function (fn) has dtype float64. This can happen if the inner function of scan results in an upcast or downcast.
Why so? X and outputs_info are explicitly set to float32. How does the result of adding them get to be float64?
This is a very late answer, but we're working on a fork of Theano called Aesara, and, since people still run into problems like this, it seems worthwhile to provide a public explanation.
That said, the issue is X = T.matrix(). T.matrix creates a float64 matrix when theano.config.floatX == "float64" (the default), and the result is an upcast to float64 for the sum in the body of the scan's loop function.
If X = T.fmatrix() is used, a float32 matrix is created instead and the problem is no longer present; otherwise, as mentioned in the comments, one can also set theano.config.floatX to "float32".
Related
I am using the method of lines with 'solve_ivp` to solve a nonlinear PDE:
#njit(fastmath=True,error_model="numpy",cache=True)
def thinFilmEq(t,h,dx,Ma,phiFun,tempFun):
phi = phiFun(h)
temperature = tempFun(h)
hxx = (np.roll(h,1) - 2*h + np.roll(h,-1))/dx**2 # use np.roll as I'm implementing periodic BC
p = phi - hxx
px = (np.roll(p,-1) - np.roll(p,1))/(2*dx)
Tx = (np.roll(temperature,-1) - np.roll(temperature,1))/(2*dx)
flux = h**3*px/3 + Ma*h**2*Tx/2
dhdt = (np.roll(flux,-1) - np.roll(flux,1))/(2*dx)
return dhdt
I get the following error: TypingError: non-precise type pyobject
[1] During: typing of argument at C:/Users/yhcha/method_of_lines/test_01_thinFilmEq.py (28) I suspect it is due to phiFun and tempFun. They are functions which I supply at the time of calling. I make the functions arguments to the dhdt function just to keep things more general. When I try to remove phiFun and tempFun and explicitly give the function form inside thinFilmEq, the error goes away.
Then, I see the following error TypingError: Use of unsupported NumPy function 'numpy.roll' or unsupported use of the function. I thought maybe np.roll is not supported although it is included in the official website. I tried to 'enlarge' the array to somehow manually apply the same thing as np.roll when dealing with the finite difference for periodic BC:
def augment(x):
x2 = np.empty(len(x)+2)
x2[1:-1] = x
x2[0] = x[-1]
x2[-1] = x[0]
return x2
H = augment(x)
hx = (H[2:]-[H:-2])/dx # use this instead of hx=(roll(h,-1)-roll(h,1))/dx
My questions are:
It seems that I can get numba to work, at the expense of making the codes less generally (cannot supply an arbitrary function like phiFun and elegant (e.g. cannot use a one-liner with np.roll). Are there ways to get around it or is it just the price I need to pay when using numba to 'compile' the code?
The original version without numba is close to 10x slower than the Matlab version I coded, and the numba version is still around 3-4 times slower than Matlab. I don't really expect scipy to outperform Matlab, but are there other ways to speedup the code to bridge the gap?
I'm trying to implement some algorithm on python. For the sake of documentation and clear understanding of the flow details I use sympy. As it turned out it fails on computation of an inverted float matrix.
So I'm getting
TypeError Traceback (most recent call last)
<ipython-input-20-c2193b2ae217> in <module>()
10 np.linalg.inv(xx)
11 symInv = lambdify(X0,X0Inv)
---> 12 symInv(xx)
/opt/anaconda3/lib/python3.6/site-packages/numpy/__init__.py in <lambda>(X0)
TypeError: ufunc 'bitwise_xor' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
If the matrix is integer it works fine:
import numpy as np
from sympy import *
init_printing()
X0 = MatrixSymbol('X0',2,2)
xx = np.random.rand(4,4)
#xx = np.random.randint(10,size=(4,4)) # this line makes it workable
X0Inv = X0**-1
np.linalg.inv(xx)
symInv = lambdify(X0,X0Inv)
symInv(xx)
Link to a live version of the code
If anybody knows any workaround it would be great if you could share. Thanks in advance.
UPDATE. As it is pointed out by #hpaulj and #tel the issue is how lambdify translates ** to numpy code for matrix symbols: by some reason it tries to XOR elements. I will try to find an easy way to alter this behavior. Any help/hints are appreciated.
As hpaulj points out, the error seems to stem from a conversion of ** to ^ that happens in lambdify, for some reason.
You can fix the error that you're getting by using np.power instead of **:
import numpy as np
from sympy import MatrixSymbol, lambdify
X0 = MatrixSymbol('X0',2,2)
xx = np.random.rand(4,4)
X0Inv = np.power(X0, -1)
symInv = lambdify(X0,X0Inv)
print('matrix xx')
print(xx, end='\n\n')
print('result of symInv(xx)')
print(symInv(xx), end='\n\n')
Output:
matrix xx
[[0.4514882 0.84588859 0.02431252 0.25468078]
[0.46767727 0.85748153 0.51207567 0.59636962]
[0.84557537 0.38459205 0.76814414 0.96624407]
[0.0933803 0.43467119 0.77823338 0.58770188]]
result of symInv(xx)
[[2.214897321138516, 1.1821887747951494], [2.1382266426713077, 1.1662058776397513]]
However, as you have it set up symInv doesn't produce the matrix inverse, but instead only does the element-wise exponentiation of each value in xx. In other words, symInv(xx)[i,j] == xx[i,j]**-1. This code shows the difference between element-wise exponentiation and the true inverse.
print('result of xx**-1')
print(xx**-1, end='\n\n')
print('result of np.linalg.inv(xx)')
print(np.linalg.inv(xx))
Output:
result of xx**-1
[[ 2.21489732 1.18218877 41.13107402 3.92648394]
[ 2.13822664 1.16620588 1.95283638 1.67681243]
[ 1.18262669 2.60015778 1.301839 1.0349352 ]
[10.7088969 2.30058954 1.28496159 1.70154295]]
result of np.linalg.inv(xx)
[[-118.7558445 171.37619558 -20.37188041 -88.94733652]
[ -0.56274492 2.49107626 -1.00812489 -0.62648633]
[-160.35674704 230.3266324 -28.87548299 -116.75862026]
[ 231.62940572 -334.07044947 42.21936405 170.90926978]]
Edit: workaround
I'm 95% sure that what you've run into is a bug in the Sympy code. It seems that X0^-1 was valid syntax for Sympy Matrix objects at some point, but no longer. However, it seems that someone forgot to tell that to whomever maintains the lambdify code, since it still translates every matrix exponentiation into the carrot ^ syntax.
So what you should do is submit an issue on the Sympy github. Just post your code and the error it produces, and ask if that's the intended behavior. In the meantime, here's a filthy hack to work around the problem:
import numpy as np
from sympy import MatrixSymbol, lambdify
class XormulArray(np.ndarray):
def __new__(cls, input_array):
return np.asarray(input_array).view(cls)
def __xor__(self, other):
return np.linalg.matrix_power(self, other)
X0 = MatrixSymbol('X0',2,2)
xx = np.random.rand(4,4)
X0Inv = X0.inv()
symInv = lambdify(X0,X0Inv,'numpy')
print('result of symInv(XormulArray(xx))')
print(symInv(XormulArray(xx)), end='\n\n')
print('result of np.linalg.inv(xx)')
print(np.linalg.inv(xx))
Output:
result of symInv(XormulArray(xx))
[[ 3.50382881 -3.84573344 3.29173896 -2.01224981]
[-1.88719742 1.86688465 0.3277883 0.0319487 ]
[-3.77627792 4.30823019 -5.53247103 5.53412775]
[ 3.89620805 -3.30073088 4.27921307 -4.68944191]]
result of np.linalg.inv(xx)
[[ 3.50382881 -3.84573344 3.29173896 -2.01224981]
[-1.88719742 1.86688465 0.3277883 0.0319487 ]
[-3.77627792 4.30823019 -5.53247103 5.53412775]
[ 3.89620805 -3.30073088 4.27921307 -4.68944191]]
Basically, you'll have to cast all of your arrays to the thin wrapper type XormulArray right before you pass them into symInv. This hack is not best practice for a bunch of reasons (including the fact that it apparently breaks the (2,2) shape restriction you placed on X0), but it'll probably be the best you can do until the Sympy codebase is fixed.
I'm trying to calculate mse using tensorflow and compare the result with sklearn.metrics.mean_squared_error method.
def mse(y,y_hat):
return tf.reduce_mean(tf.squared_difference(y, y_hat)).eval()
compute_mse = lambda vector1, vector2: mse(vector1,vector2)
My test loop
for n in [1,5,10,10**3]:
elems = [np.arange(n),np.arange(n,0,-1), np.zeros(n),
np.ones(n),np.random.random(n),np.random.randint(100,size=n)]
for el in elems:
for el_2 in elems:
true_mse = np.array(mean_squared_error(el,el_2))
my_mse = compute_mse(el,el_2)
if not np.allclose(true_mse,my_mse):
print('Wrong result:')
print("All tests passed")
But my tf function always return either 0 or 1.
Could you please point me where I'm wrong.
UPD
Thanks to #apnorton for pointing on issue with types.
def mse(y,y_hat):
y_ = tf.Variable(y, tf.float64)
y_hat_ = tf.Variable(y_hat, tf.float64)
return tf.reduce_mean(tf.squared_difference(y_, y_hat_).eval()
If you print all the outputs of your tf function, you'll see it doesn't return only 1s and 0s, but it does only return integers. This is because the values of elems are all of type numpy.int32. The sklearn function appears to cast these as floats when doing the mean step, while the Tensor Flow approach does not.
To see a fixed variant, consider changing the compute_mse line to:
my_mse = compute_mse(el.astype(float),el_2.astype(float))
Edit: In response to the question in the comments, I'd avoid creating a variable only for the purpose of a cast. Instead, I'd recommend using the tf.to_float method:
def mse(y,y_hat):
return tf.reduce_mean(tf.squared_difference(tf.to_float(y), tf.to_float(y_hat))).eval()
I've encountered an overflow warning as a result of multiplying the output of Numpy products that I'm looking to understand. A simplified version of their actual use within my larger project is detailed below:
import numpy as np
class MyClass(object):
def __init__(self,
array_1,
array_2):
# Assigning arrays to be used in later methods
self.array_1 = array_1
self.array_2 = array_2
# Assigning some scaling factors to be used in later methods.
self.value_1 = np.prod(self.array_1.shape)
self.value_2 = np.prod(self.array_2.shape)
print("Numpy Product Assignment: {0}, {1}".format(self.value_1, self.value_2))
# Alternative assignment of scaling factors
self.alt_value_1 = self.array_1.shape[0] * self.array_1.shape[1]
self.alt_value_2 = self.array_2.shape[0] * self.array_2.shape[1]
print("Regular Product Assignment: {0}, {1}".format(self.alt_value_1, self.alt_value_2))
pass
def mymethod(self):
print("Direct Multiplication: {0}".format(80160 * 262144))
print("Numpy Product Multiplication: {0}".format(self.value_1 * self.value_2))
print("Regular Product Multiplcation {0}".format(self.alt_value_1 * self.alt_value_2))
if __name__ == '__main__':
test_array_1 = np.zeros([512, 512], dtype=complex)
test_array_2 = np.zeros([1002, 80], dtype=complex)
test_class = MyClass(test_array_1, test_array_2)
test_class.mymethod()
Including the use with the class structure for completeness, though heavily edited down to the bare minimum. If I run this code (on Python 3.6.0) I get the following output:
C:/somepath/scratch.py:247: RuntimeWarning: overflow encountered in long_scalars
print("Numpy Product Multiplication: {0}".format(self.value_1 * self.value_2))
Numpy Product Assignment: 262144, 80160
Regular Product Assignment: 262144, 80160
Direct Multiplication: 21013463040
Numpy Product Multiplication: -461373440
Regular Product Multiplcation 21013463040
Process finished with exit code 0
Clearly I can get around the problem using the regular multiplication, but I would like to understand why there is a problem and if it can be fixed as is. I think there is some dtype=X subtlety that I have missed, so my question is what is causing these overflow errors?
This looks like an overflow caused by 32 integers. You can convert your values to 64 bit like this:
self.value_1 = np.prod(self.array_1.shape, dtype=np.int64)
self.value_2 = np.prod(self.array_2.shape, dtype=np.int64)
Numpy automatically selects a 32 bit integer type if the values used for array construction are small enough. During multiplication they are not automatically cast to 64 bit.
I need to create a some fixed length(length equal to size of some other tensor vector which is passed) zero vector in theano.
def some_fun(self, y)
x_h = T.fvector('x_h')
ret = T.alloc(0, x_h)
vec_h = theano.function(inputs=[x_h], outputs=ret)
vec=vec_h(y.shape[0])
vec[T.arange(y.shape[0]),y]=1
When I am running this I am getting error "Shape arguments to Alloc must be integers, but argument 0 is not for apply node: x_h"
It may very big mistake, as I am new to theano.
Thanks
Have you tried theano.tensor.zeros_like? It seems like that should be a shortcut to what you're trying to do.
Then, when you get
"TypeError: 'TensorVariable' object does not support item assignment"
you can replace the line vec[T.arange(y.shape[0]),y]=1 by using theano.tensor.set_subtensor instead.