I have a function that takes a [32, 32, 3] tensor, and outputs a [256,256,3] tensor.
Specifically, the function interprets the smaller array as if it was a .svg file, and 'renders' it to a 256x256 array as a canvas using this algorithm
For an explanation of WHY I would want to do this, see This question
The function behaves exactly as intended, until I try to include it in the training loop of a GAN. The current error I'm seeing is:
NotImplementedError: Cannot convert a symbolic Tensor (mul:0) to a numpy array.
A lot of other answers to similar errors seem to boil down to "You need to re-write the function using tensorflow, not numpy"
Here's the working code using numpy - is it possible to re-write it to exclusively use tensorflow functions?
def convert_to_bitmap(input_tensor, target, j):
#implied conversion to nparray - the tensorflow docs seem to indicate this is okay, but the error is thrown here when training
array = input_tensor
outputArray = target
output = target
for i in range(32):
col = float(array[i,0,j])
if ((float(array[i,0,0]))+(float(array[i,0,1]))+(float(array[i,0,2]))/3)< 0:
continue
#slice only the red channel from the i line, multiply by 255
red_array = array[i,:,0]*255
#slice only the green channel, multiply by 255
green_array = array[i,:,1]*255
#combine and flatten them
combined_array = np.dstack((red_array, green_array)).flatten()
#remove the first two and last two indices of the combined array
index = [0,1,62,63]
clipped_array = np.delete(combined_array,index)
#filter array to remove values less than 0
filtered = clipped_array > 0
filtered_array = clipped_array[filtered]
#check array has an even number of values, delete the last index if it doesn't
if len(filtered_array) % 2 == 0:
pass
else:
filtered_array = np.delete(filtered_array,-1)
#convert into a set of tuples
l = filtered_array.tolist()
t = list(zip(l, l[1:] + l[:1]))
if not t:
continue
output = fill_polygon(t, outputArray, col)
return(output)
The 'fill polygon' function is copied from the 'mahotas' library:
def fill_polygon(polygon, canvas, color):
if not len(polygon):
return
min_y = min(int(y) for y,x in polygon)
max_y = max(int(y) for y,x in polygon)
polygon = [(float(y),float(x)) for y,x in polygon]
if max_y < canvas.shape[0]:
max_y += 1
for y in range(min_y, max_y):
nodes = []
j = -1
for i,p in enumerate(polygon):
pj = polygon[j]
if p[0] < y and pj[0] >= y or pj[0] < y and p[0] >= y:
dy = pj[0] - p[0]
if dy:
nodes.append( (p[1] + (y-p[0])/(pj[0]-p[0])*(pj[1]-p[1])) )
elif p[0] == y:
nodes.append(p[1])
j = i
nodes.sort()
for n,nn in zip(nodes[::2],nodes[1::2]):
nn += 1
canvas[y, int(n):int(nn)] = color
return(canvas)
NOTE: I'm not trying to get someone to convert the whole thing for me! There are some functions that are pretty obvious (tf.stack instead of np.dstack), but others that I don't even know how to start, like the last few lines of the fill_polygon function above.
Yes you can actually do this, you can use a python function in sth called tf.pyfunc. Its a python wrapper but its extremely slow in comparison to plain tensorflow. However, tensorflow and Cuda for example are so damn fast because they use stuff like vectorization, meaning you can rewrite a lot , really many of the loops in terms of mathematical tensor operations which are very fast.
In general:
If you want to use custom code as a custom layer, i would recommend you to rethink the algebra behind those loops and try to express them somehow different. If its just preprocessing before the training is going to start, you can use tensorflow but doing the same with numpy and other libraries is easier.
To your main question: Yes its possible, but better dont use loops. Tensorflow has a build-in loop optimizer but then you have to use tf.while() and thats anyoing (maybe just for me). I just blinked over your code, but it looks like you should be able to vectorize it quite good using the standard tensorflow vocabulary. If you want it fast, i mean really fast with GPU support write all in tensorflow, but nothing like 50/50 with tf.convert_to_tensor(), because than its going to be slow again. because than you switch between GPU and CPU and plain Python interpreter and the tensorflow low level API. Hope i could help you at least a bit
This code 'works', in that it only uses tensorflow functions, and does allow the model to train when used in a training loop:
def convert_image (x):
#split off the first column of the generator output, and store it for later (remove the 'colours' column)
colours_column = tf.slice(img_to_convert, tf.constant([0,0,0], dtype=tf.int32), tf.constant([32,1,3], dtype=tf.int32))
#split off the rest of the data, only keeping R + G, and discarding B
image_data_red = tf.slice(img_to_convert, tf.constant([0,1,0], dtype=tf.int32), tf.constant([32,31,1], dtype=tf.int32))
image_data_green = tf.slice(img_to_convert, tf.constant([0,1,1], dtype=tf.int32), tf.constant([32, 31,1], dtype=tf.int32))
#roll each row by 1 position, and make two more 2D tensors
rolled_red = tf.roll(image_data_red, shift=-1, axis=0)
rolled_green = tf.roll(image_data_green, shift=-1, axis=0)
#remove all values where either the red OR green channels are 0
zeroes = tf.constant(0, dtype=tf.float32)
#this is for the 'count_nonzero' command
boolean_red_data = tf.not_equal(image_data_red, zeroes)
boolean_green_data = tf.not_equal(image_data_green, zeroes)
initial_data_mask = tf.logical_and(boolean_red_data, boolean_green_data)
#count non-zero values per row and flatten it
count = tf.math.count_nonzero(initial_data_mask, 1)
count_flat = tf.reshape(count, [-1])
flat_red = tf.reshape(image_data_red, [-1])
flat_green = tf.reshape(image_data_green, [-1])
boolean_red = tf.math.logical_not(tf.equal(flat_red, tf.zeros_like(flat_red)))
boolean_green = tf.math.logical_not(tf.equal(flat_green, tf.zeros_like(flat_red)))
mask = tf.logical_and(boolean_red, boolean_green)
flat_red_without_zero = tf.boolean_mask(flat_red, mask)
flat_green_without_zero = tf.boolean_mask(flat_green, mask)
# create a ragged tensor
X0_ragged = tf.RaggedTensor.from_row_lengths(values=flat_red_without_zero, row_lengths=count_flat)
Y0_ragged = tf.RaggedTensor.from_row_lengths(values=flat_green_without_zero, row_lengths=count_flat)
#do the same for the rolled version
rolled_data_mask = tf.roll(initial_data_mask, shift=-1, axis=1)
flat_rolled_red = tf.reshape(rolled_red, [-1])
flat_rolled_green = tf.reshape(rolled_green, [-1])
#from SO "shift zeros to the end"
boolean_rolled_red = tf.math.logical_not(tf.equal(flat_rolled_red, tf.zeros_like(flat_rolled_red)))
boolean_rolled_green = tf.math.logical_not(tf.equal(flat_rolled_green, tf.zeros_like(flat_rolled_red)))
rolled_mask = tf.logical_and(boolean_rolled_red, boolean_rolled_green)
flat_rolled_red_without_zero = tf.boolean_mask(flat_rolled_red, rolled_mask)
flat_rolled_green_without_zero = tf.boolean_mask(flat_rolled_green, rolled_mask)
# create a ragged tensor
X1_ragged = tf.RaggedTensor.from_row_lengths(values=flat_rolled_red_without_zero, row_lengths=count_flat)
Y1_ragged = tf.RaggedTensor.from_row_lengths(values=flat_rolled_green_without_zero, row_lengths=count_flat)
#available outputs for future use are:
X0 = X0_ragged.to_tensor(default_value=0.)
Y0 = Y0_ragged.to_tensor(default_value=0.)
X1 = X1_ragged.to_tensor(default_value=0.)
Y1 = Y1_ragged.to_tensor(default_value=0.)
#Example tensor cel (replace with (x))
P = tf.cast(x, dtype=tf.float32)
#split out P.x and P.y, and fill a ragged tensor to the same shape as Rx
Px_value = tf.cast(x, dtype=tf.float32) - tf.cast((tf.math.floor(x/255)*255), dtype=tf.float32)
Py_value = tf.cast(tf.math.floor(x/255), dtype=tf.float32)
Px = tf.squeeze(tf.ones_like(X0)*Px_value)
Py = tf.squeeze(tf.ones_like(Y0)*Py_value)
#for each pair of values (Y0, Y1, make a vector, and check to see if it crosses the y-value (Py) either up or down
a = tf.math.less(Y0, Py)
b = tf.math.greater_equal(Y1, Py)
c = tf.logical_and(a, b)
d = tf.math.greater_equal(Y0, Py)
e = tf.math.less(Y1, Py)
f = tf.logical_and(d, e)
g = tf.logical_or(c, f)
#Makes boolean bitwise mask
#calculate the intersection of the line with the y-value, assuming it intersects
#P.x <= (G.x - R.x) * (P.y - R.y) / (G.y - R.y + R.x) - use tf.divide_no_nan for safe divide
h = tf.math.less(Px,(tf.math.divide_no_nan(((X1-X0)*(Py-Y0)),(Y1-Y0+X0))))
#combine using AND with the mask above
i = tf.logical_and(g,h)
#tf.count_nonzero
#reshape to make a column tensor with the same dimensions as the colours
#divide by 2 using tf.floor_mod (returns remainder of division - any remainder means the value is odd, and hence the point is IN the polygon)
final_count = tf.cast((tf.math.count_nonzero(i, 1)), dtype=tf.int32)
twos = tf.ones_like(final_count, dtype=tf.int32)*tf.constant([2], dtype=tf.int32)
divide = tf.cast(tf.math.floormod(final_count, twos), dtype=tf.int32)
index = tf.cast(tf.range(0,32, delta=1), dtype=tf.int32)
clipped_index = divide*index
sort = tf.sort(clipped_index)
reverse = tf.reverse(sort, [-1])
value = tf.slice(reverse, [0], [1])
pair = tf.constant([0], dtype=tf.int32)
slice_tensor = tf.reshape(tf.stack([value, pair, pair], axis=0),[-1])
output_colour = tf.slice(colours_column, slice_tensor, [1,1,3])
return output_colour
This is where the 'convert image' function is applied using tf.vectorize_map:
def convert_images(image_to_convert):
global img_to_convert
img_to_convert = image_to_convert
process_list = tf.reshape((tf.range(0,65536, delta=1, dtype=tf.int32)), [65536, 1])
output_line = tf.vectorized_map(convert_image, process_list)
output_line_squeezed = tf.squeeze(output_line)
output_reshape = (tf.reshape(output_line_squeezed, [256,256,3])/127.5)-1
output = tf.expand_dims(output_reshape, axis=0)
return output
It is PAINFULLY slow, though - It does not appear to be using the GPU, and looks to be single threaded as well.
I'm adding it as an answer to my own question because is clearly IS possible to do this numpy function entirely in tensorflow - it just probably shouldn't be done like this.
In the following code snippet I intend to do the following:
(1) Multiply each element of the identity by the d optimization variable.
(2) Sum a vector of ones to a CVXPY affine expression, which is also a vector of 24 elements.
(3) Create a constraint which compares two vectors element-wise.
import numpy as np
import cvxpy as cp
weights = cp.Variable(5)
d = cp.Variable(1)
meas = np.random.rand(8, 3)
det = np.random.rand(24, 5)
dm = d * np.eye(3) # (1)
beh = np.ones([24, 1]) + cp.reshape((dm # meas.T).T, [24, 1]) # (2)
constrs = [beh == det # weights] #(3)
My questions are:
Q1: Did I code what I wanted?
Q2: At (2), I get the following error:
/usr/lib/python3.8/site-packages/cvxpy/utilities/shape.py in sum_shapes(shapes)
45 # Only allow broadcasting for 0D arrays or summation of scalars.
46 if shape != t and len(squeezed(shape)) != 0 and len(squeezed(t)) != 0:
---> 47 raise ValueError(
48 "Cannot broadcast dimensions " +
49 len(shapes)*" %s" % tuple(shapes))
ValueError: Cannot broadcast dimensions (24, 1) [24, 1]
What exactly does this mean, and how do I fix it?
Q3: When I do det # weights, at (3), I get an Expression(AFFINE, UNKNOWN, (24,)). In the constraint, I'll compare it with beh, which I'm guessing will be an Expression(AFFINE, UNKNOWN, (24, 1)). Will this comparison also bring an issue?
When I started using cvxpy, I also had some trouble making dimensions fit. In my experience, it is a good idea to use arrays with as few dimensions as possible. So if you have a 2-dimensional array where 1 of the dimensions only has length 1, see if you can reduce the dimension. (see below)
The problem in (2) is solved when you changed the brackets you use when reshaping the cvxpy expression to (24,1), like this:
beh = np.ones([24, 1]) + cp.reshape((dm # meas.T).T, (24, 1)) # (2)
You could also avoid your problem by simply doing:
beh = 1 + cp.reshape((dm # meas.T).T, (24, 1)) # (2)
which will do the same: add 1 to each entry of the cvxpy array.
After this is done, you will have a problem with your final line: "ValueError: Cannot broadcast dimensions (24, 1) (24,)"
This can be remedied by making beh of the dimension (24, ) too (reduce the dimension will solve your problems here, as mentioned before). The full working code would be:
import numpy as np
import cvxpy as cp
weights = cp.Variable(5)
d = cp.Variable(1)
meas = np.random.rand(8, 3)
det = np.random.rand(24, 5)
dm = d * np.eye(3) # (1)
beh = 1 + cp.reshape((dm # meas.T).T, (24, )) # (2)
constrs = [beh == det # weights] #(3)
Hope this helps!
I find myself reshaping 1D vectors way to many times. I wonder if this is because I'm doing something wrong, or because it is an inherit fault of numpy.
Why can't numpy infer that when he gets an object of shape (400,) to transform it to (400,1) ? And why do so many numpy operations result in removing the axis completely?
e.g.
def predict(Theta1, Theta2, X):
m = X.shape[0]
X = np.c_[np.ones(m), X]
hidden = sigmoid(X # Theta1.T)
hidden = np.c_[np.ones(m), hidden]
output = sigmoid(hidden # Theta2.T)
result = np.argmax(output, axis=1) + 1 # removes the 2nd axis - (400,)
return result.reshape((-1, 1)) # re-add the axis - (400,1)
pred = predict(Theta1, Theta2, X)
print(np.mean(pred == y))
If I don't reshape the result in the last row, I get funky behavior when comparing pred (400,) and y (400,1).
you can use
np.array_split(data, s)
knowing that new dimensions will have length of s (data shape is s * s)
The new numpy version (1.22) now added an optional keepdims to argmax. Source: here.
I need to use this loss function for a CNN the list_distance and list_residual are output tensors from hidden layers which are important to compute the loss, but when i execute the code it gives me back this error
TypeError: Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.
Is there another way to iterate over tensors without the use of the costruct
x in X or convert it in a numpy array or using the backend function of keras?
def DBL(y_true, y_pred, list_distances, list_residual, l=0.65):
prob_dist = []
Li = []
# mean of the images power spectrum
S = np.sum([np.power(np.abs(fp.fft2(residual)), 2)
for residual in list_residual], axis=0) / K.shape(list_residual)[0]
# log-ratio between the geometric and arithmetic of S
R = np.log10((scistats.gmean(S) / np.mean(S)))
for c_i, dis_i in enumerate(list_distances):
prob_dist.append([
np.exp(-dis_i) / sum([np.exp(-dis_j) if c_j != c_i else 0 for c_j, dis_j in enumerate(list_distances)])
])
for count, _ in enumerate(prob_dist):
Li.append(
-1 * np.log10(sum([p_j for c_j, p_j in enumerate(prob_dist[count])
if y_pred[count] == 1 and count != c_j])))
L0 = np.sum(Li)
return L0 - l * R
You need to define a custom function to feed into tf.map_fn() - Tensorflow dox
Mapper functions map (funnily enough) the existing object (tensor) into a new one using a function you define.
They apply the custom function to every element in the object, without all the mucking about with for loops.
For instance (non tested code, may not run - on my phone atm):
def custom(a):
b = a + 1
return b
original = np.array([2,2,2])
mapped = tf.map_fn(custom, original)
# mapped == [3, 3, 3] ... hopefully
Tensorflow examples all use lambda functions, so you might need to define your functions like that if the above doesn’t work. Tensorflow example:
elems = np.array([1, 2, 3, 4, 5, 6])
squares = map_fn(lambda x: x * x, elems)
# squares == [1, 4, 9, 16, 25, 36]
Edit:
As an aside, map functions are much easier to parallelise than for loops - it is assumed that each element of an object is processed uniquely - so you can see a performance uplift by using them.
Edit 2:
For the "reduce sum, but not on this index" part, I would heavily recommend you start looking back at matrix operations... As mentioned, map functions work element-wise - they are not aware of other elements. A reduce function is what you want, but even they are finiky when you try and do "not this index" sums... also tensorflow is built around matrix ops... Not the MapReduce paradigm.
Something along these lines might help:
sess = tf.Session()
var = np.ones([3, 3, 3]) * 5
zero_identity = tf.linalg.set_diag(
var, tf.zeros(var.shape[0:-1], dtype=tf.float64)
)
exp_one = tf.exp(var)
exp_two = tf.exp(zero_identity)
summed = tf.reduce_sum(exp_two, axis = [0,1])
final = exp_one / summed
print("input matrix: \n", var, "\n")
print("Identities of the matrix to Zero: \n", zero_identity.eval(session=sess), "\n")
print("Exponential Values numerator: \n", exp_one.eval(session=sess), "\n")
print("Exponential Values to Sum: \n", exp_two.eval(session=sess), "\n")
print("Summed values for zero identity matrix\n ... along axis [0,1]: \n", summed.eval(session=sess), "\n")
print("Output:\n", final.eval(session=sess), "\n")
I have a array of ones in tensorflow and I want to update its values based on another array in a for loop. Here is the code:
def get_weights(labels, class_ratio=0.5):
weights = tf.ones_like(labels, dtype=tf.float64))
pos_num = class_ratio * 100
neg_num = 100 - class_ratio * 100
for i in range(labels.shape[0]):
if labels[i] == 0:
weights[i].assign(pos_num/neg_num)
else:
weights[i].assign(neg_num)
return weights
an then I have this code to call the above function:
with tf.Graph().as_default():
labels = tf.placeholder(tf.int32, (5,))
example_weights = get_weights(labels, class_ratio=0.1)
with tf.Session() as sess:
np_labels = np.random.randint(0, 2, 5)
np_weights = sess.run(example_weights, feed_dict={labels: np_labels})
print("Labels: %r" % (np_labels,))
print("Weights: %r" % (np_weights,))
but when I run it, it gives me this error:
ValueError: Sliced assignment is only supported for variables
How can I assign/update values of an array in tensorflow?
A tf.Tensor in TensorFlow is a read-only value—in fact, a symbolic expression for computing a read-only value—so you cannot in general assign values to it. (The main exceptions are tf.Variable objects.) This means that you are encourage to use "functional" operations to define your tensor. For example, there are several ways to generate the weights tensor functionally:
Since weights is defined as an element-wise transformation of labels, you can use tf.map_fn() to create a new tensor (containing a tf.cond() to replace the if statement) by mapping a function across it:
def get_weights(labels, class_ratio=0.5):
pos_num = tf.constant(class_ratio * 100)
neg_num = tf.constant(100 - class_ratio * 100)
def compute_weight(x):
return tf.cond(tf.equal(x, 0), lambda: pos_num / neg_num, lambda: neg_num)
return tf.map_fn(compute_weight, labels, dtype=tf.float32)
This version allows you to apply an arbitrarily complicated function to each element of labels.
However, since the function is simple, cheap to compute, and representable using simple TensorFlow ops, you can avoid using tf.map_fn() and instead use tf.where():
def get_weights(labels, class_ratio=0.5):
pos_num = tf.fill(tf.shape(labels), class_ratio * 100)
neg_num = tf.fill(tf.shape(labels), 100 - class_ratio * 100)
return tf.where(tf.equal(labels, 0), pos_num / neg_num, neg_num)
(You could also use tf.where() instead of tf.cond() in the tf.map_fn() version.)