I have a #tf.function decorated function. Inside the function, I would like to draw from a distribution and compute some value several times (let's call this proceduce f(x)).
How can I do this in Tensorflow 2.0? I can't use numpy arrays as I would like to use the #tf.function decorator.
A numpy implementation would look like:
reps = 4
store = np.zeros((n, reps))
for i in range(reps):
store[:, i] = f(x) #f(x) is shape (n,)
The goal would then be to compute the row means of store.
This should be easy but I haven't been able to work out how to do it!
Something like this maybe:
import tensorflow as tf
def f():
return tf.random.normal((10,))
#tf.function
def store_this():
reps = 4
n = 10
store = tf.zeros((n, reps))
values = [f() for _ in range(reps)]
indices = tf.stack([tf.tile(tf.range(n), multiples=[reps]), tf.repeat(tf.range(reps), repeats=n)], axis=-1)
return tf.tensor_scatter_nd_update(store, indices, tf.reshape(values, [-1]))
store_this()
If f takes a one-dimensional tensor as input, this is a shorter alternative:
#tf.function
def f(x):
return tf.random.normal((10,))
x = tf.constant([1.0, 2.0])
reps = 4
def store_this(fp, x, reps):
return tf.transpose(tf.map_fn(fp, tf.tile(tf.expand_dims(x, 0),[reps,1])))
store_this(f, x, reps)
Related
I need help to compute a mathematical expression using only numpy operations. The expression I want to compute is the following :
Where : x is an (N, S) array and f is a numpy function (that can work with broadcastable arrays e.g np.maximum, np.sum, np.prod, ...). If that is of importance, in my case f is a symetric function.
So far my code looks like this:
s = 0
for xp in x: # Loop over N...
s += np.sum(np.prod(f(xp, x), axis=1))
And still has loop that I'd like to get rid of.
Typically N is "large" (around 30k) but S is small (less than 20) so if anyone can find a trick to only loop over S this would still be a major improvement.
I belive the problem is easy by N-plicating the array but one of size (32768, 32768, 20) requires 150Go of RAM that I don't have. However, (32768, 32768) fits in memory though I would appreciate a solution that does not allocate such array.
Maybe a use of np.einsum with well-chosen arrays is possible?
Thanks for your replies. If any information is missing let me know!
Have a nice day !
Edit 1 :
Form of f I'm interested in includes (for now) : f(x, y) = |x - y|, f(x, y) = |x - y|^2, f(x, y) = 2 - max(x, y).
Your loop is very efficient. Some possible ways are
Method-1 (looping over S)
import numpy as np
def f(x,y):
return np.abs(x-y)
N = 200
S = 20
x_data = random.rand(N,S) #(i,s)
y_data = random.rand(N,S) #(i',s)
product = f(broadcast_to(x_data[:,0][...,None],(N,N)) ,broadcast_to(y_data[:,0][...,None],(N,N)).T)
for i in range(1,S):
product *= f(broadcast_to(x_data[:,i][...,None],(N,N)) ,broadcast_to(y_data[:,i][...,None],(N,N)).T)
sum = np.sum(product)
Method-2 (dispatching S number of blocks)
import numpy as np
def f(x,y):
x1 = np.broadcast_to(x[:,None,...],(x.shape[0],y.shape[0],x.shape[1]))
y1 = np.broadcast_to(y[None,...],(x.shape[0],y.shape[0],x.shape[1]))
return np.abs(x1-y1)
def f1(x1,y1):
return np.abs(x1-y1)
N = 5000
S = 20
x_data = np.random.rand(N,S) #(i,s)
y_data = np.random.rand(N,S) #(i',s)
def fun_new(x_data1,y_data1):
s = 0
pp =np.split(x_data1,S,axis=0)
for xp in pp:
s += np.sum(np.prod(f(xp, y_data1), axis=2))
return s
def fun_op(x_data1,y_data1):
s = 0
for xp in x_data1: # Loop over N...
s += np.sum(np.prod(f1(xp, y_data1), axis=1))
return s
fun_new(x_data,y_data)
I am trying to use scipy.odeint() method in order to solve an second order partial derivative function.
I can do that for a single value of constant k, which is a constant of the function I have.
But I want to try this solution for many values of k.
To do so, I included the values that I want in a list k, and going through a loop I want to plug in these values for the final solution as arguments.
However, I am getting an error
error: Extra arguments must be in a tuple
import numpy as np
from scipy.integrate import odeint
### Code with a single value of K.THAT WORKS FINE!!!! ###
k = 1 #attributes to be changed
t = [0.1,0.2,0.3] #Data
init = [45,0] #initial values
#Function to apply an integration
def f(init, t, args=(k,)):
dOdt = init[1]
dwdt = -np.cos(init[0]) + k*dOdt
return [dOdt, dwdt]
#integrating function that returns a list of 2D numpy arrays
zCH = odeint(f,init,t)
################################################################
### Code that DOES NOT WORK!###
k = [1,2,3] #attributes to be changed
t = [0.1,0.2,0.3] #Data
init = [45,0] #initial values
#Function to apply an integration
def f(init, t, args=(k,)):
dOdt = init[1]
dwdt = -np.cos(init[0]) + k*dOdt
return [dOdt, dwdt]
solutions = []
for i in k:
#integrating function that returns a list of 2D numpy arrays
zCH = odeint(f,init,t,(k[i-1]))
solutions.append(zCH)```
It has to do with the way you are passing k into your function f().
The following changes the value of k on each iteration
k_list = [1,2,3] #attributes to be changed
t = [0.1,0.2,0.3] #Data
init = [45,0] #initial values
#Function to apply an integration
def f(init, t, args=(k,)):
dOdt = init[1]
dwdt = -np.cos(init[0]) + k*dOdt
return [dOdt, dwdt]
solutions = []
for k in k_list:
#integrating function that returns a list of 2D numpy arrays
zCH = odeint(f, init, t)
solutions.append(zCH)
I have the following function:
k=np.linspace(0,5,100)
def f(x,y):
m=k
return sum(np.sin(m-x)*np.exp(-y**2))
I would like to obtain a 2D grid of values of f(x,y) evaluated on these two arrays:
x=np.linspace(0,4,30)
y=np.linspace(0,2,70)
Is there a way of calculation faster than a double "for" cycle like this one?
matrix=np.zeros((len(x),len(y)))
for i in range(len(x)):
for j in range(len(y)):
matrix[i,j]=f(x[i],y[j])
z=matrix.T
I tried to use the "numpy meshgrid" function in this way:
xx,yy=np.meshgrid(x, y)
z=f(xx,yy)
however I got the following error message:
ValueError: operands could not be broadcast together with shapes (100,) (70,30).
Here's a numpy approach. If we start with your original array setup,
k = np.linspace(0,5,100)
x = np.linspace(0,4,30)
y = np.linspace(0,2,70)
then
matrix = np.sin(k[:,np.newaxis] - x).sum(axis = 0)[:,np.newaxis]*np.exp(-y**2)
returns the same (30,70) "matrix" calculated with the double "for" cycle.
For reference, https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html outlines the broadcasting rules in numpy and https://www.numpy.org/devdocs/user/theory.broadcasting.html
gives a nice illustration of using these rules.
k=np.linspace(0,5,100)
x=np.linspace(0,4,30)
y=np.linspace(0,2,70)
def f(x,y):
## m=k
return sum(np.sin(k-x)*np.exp(-y**2))
# original
def g():
m = np.zeros((len(x),len(y)))
for i in range(len(x)):
for j in range(len(y)):
m[i,j]=f(x[i],y[j])
return m.T
# k.shape, x.shape, y.shape -> (100,), (30,), (70,)
# sine of each k minus each x
q = np.sin(k[:,None]-x) # q.shape -> (100,30)
# [sine of each k minus each x] times [e to the negative (y squared)]
r = np.exp(-y**2) # r.shape --> (70,)
s = q[...,None] * r # s.shape --> (100,30,70)
t = s.sum(0)
v = t.T # v.shape -> (70,30)
assert np.all(np.isclose(v,g()))
assert np.all(v == g())
Broadcasting
I am trying to create function using numpy something like f=(x-a1)^2+(y-a2)^2+a3
Where a1,a2,a3 are random generated numbers and x,y are parameters.
But I cant work with it, I want to find f(0,0) where [0,0] is [x,y] and [a1,a2,a3] were set before,but my code doesnt work.
And then I want to convert this function to tensorflow tensor.Here is my code, string with "##" dont work.
import tensorflow as tf
from random import random, seed
import numpy as np
def mypolyval(x, min_point, min_value):
res = min_value
for i in range(len(min_point)):
res += (x[i] - min_point[i]) ** 2
return res
class FunGen:
def __init__(self, dim, n):
self.dim = dim
self.n = n
self.functions = []
self.x = []
def c2(self):
seed(10)
for _ in range(self.n):
min_point = [random() for _ in range(self.dim)]
min_value = random()
f = np.vectorize(mypolyval, excluded=['x'])
##print(f(x=np.array([0, 0]), min_point=min_point, min_value=min_value))
self.functions.append((f, min_point, min_value))
return self.functions
functions = FunGen(2, 1).c2()
for i in functions:
print(type(i[0]))
f=i[0]
## print(f(x=[0, 0], min_value=i[1], min_point=i[2]))
##a=tf.convert_to_tensor(f,dtype=np.float32)
The problem is not related to tensorflow. The line
min_point = [random() for _ in range(self.dim)]
creates a list and lists don't have the .size() property.
You can turn it into a numpy array using min_point = np.array([random() for _ in range(self.dim)])), then the .size() will work.
Or if you want to stick to the list use for i in range(len(min_point)), which calculates the length of a list.
Also you need to add min_point and min_value them to the excluded list:
from random import random, seed
import numpy as np
def mypolyval(x, min_point, min_value):
print('x', x)
print('min_point', min_point)
print('min_value', min_value)
res = min_value
for i in range(len(min_point)):
res += (x[i] - min_point[i]) ** 2
return res
class FunGen:
def __init__(self, dim, n):
self.dim = dim
self.n = n
self.functions = []
self.x = []
def c2(self):
seed(10)
for _ in range(self.n):
min_point = [random() for _ in range(self.dim)]
min_value = random()
f = np.vectorize(mypolyval, excluded=['x', 'min_point', 'min_value'])
#print(f(x=[0, 0], min_value=min_value, min_point=min_point))
self.functions.append((f, min_point, min_value))
return self.functions
functions = FunGen(2, 1).c2()
for i in functions:
print(type(i[0]))
print(i)
f=i[0]
print(f(x=[0, 0], min_value=i[2], min_point=i[1]))
To create a TensorFlow function from a numpy function, you should use tf.py_func:
Wraps a python function and uses it as a TensorFlow op.
From the TensorFlow API:
def my_func(x):
# x will be a numpy array with the contents of the placeholder below
return np.sinh(x)
inp = tf.placeholder(tf.float32)
y = tf.py_func(my_func, [inp], tf.float32)
I'm trying to implement the Hinge loss function in Python and faced with some misleadings.
In some sources that I used to read (for example, "Regression Analysis in Python"under Luca Massoron) states that Hinge sometimes calls as Softmax function.
But for me it is kind of strange because, Hinge:
and Softmax is just exponential function like:
I made that function in Python (for Softmax) this way:
def softmax(x):
e_x = np.exp(x - np.max(x))
return e_x/e_x.sum(axis=0)
Have two questions:
Can I use that softmax function like an equivalent to hinge function?
If not, how can hinge be implemented in Python?
Thanks.
Can I use that softmax function like an equivalent to hinge function?
no - they are not equivalent.
a hinge function is a loss function and do not provide well-calibrated probabilities, whereas softmax is a mapping function (one that maps a set of scores into a distribution, one that sums to one).
If not, how can hinge be implemented in Python?
this following snippet captures the essence of hinge loss functions:
import numpy as np
import matplotlib.pyplot as plt
xmin, xmax = -1, 2
xx = np.linspace(xmin, xmax, 100)
plt.plot(xx, np.where(xx < 1, 1 - xx, 0), label="Hinge loss")
you can also implement softmax functions in pure python :)
import numpy as np
import math as math
def sofyMax(data):
# pure python
# math:: $rezult(powe,sumColumn) = \dfrac{powe(data)}{sumColumn(powe(data))}$
def powe(data):
outp = [[] for _ in range(len(data))]
for column in range(len(data[0])):
r = 0
for row in data:
outp[r]+=[math.exp(row[column])]
r+=1
return outp
def sumColumn(data):
outps = []
for column in range(len(data[0])):
total = 0
for row in data:
total+=row[column]
outps += [total]
return outps
def rezult(data,sumcolumn):
outp = [[] for _ in range(len(data))]
l = 0
for row in data:
for c,s in zip(row,sumcolumn) :
outp[l] += [c/s]
l+=1
return outp
et1 = powe(data)
et2 = sumColumn(et1)
return rezult(et1,et2)
data = np.random.randn(10,5)
(np.exp(data)/np.sum(np.exp(data),axis=0)) == (np.array(sofyMax(data)))