Purpose：
I want to get label name directly in tensorflow-serving when prediction, my question is how to transfer pred = tf.nn.sigmoid(last_layer_output) to real label name?
Question Description：
I know how to do it with multi-classes issue:
CLASSES = tf.constant(['a', 'b', 'c', 'd', 'e'])
pred = tf.nn.softmax(last_layer_output)
# pred pretty similar to:
pred = [[0, 1, 0, 0, 0],
[0, 0, 0, 1, 0],
[0, 0, 0, 0, 1]]
classes_value = tf.argmax(pred, 1)
classes_name = tf.gather(CLASSES, classes_value)
# classes_name: [b'b' b'd' b'e']
# batch_size = 3
So classes_name is what I want, I could use it design signature in tensorflow-serving, when I prediction, I could get final labels.
But how to do like this in multi-label issue?
e.g.
CLASSES = tf.constant(['a', 'b', 'c', 'd', 'e'])
pred = tf.nn.sigmoid(last_layer_output)
pred = tf.round(pred)
# pred pretty similar to:
pred = [[0, 1, 1, 0, 1], # 'b','c','e'
[1, 0, 0, 1, 0], # 'a','d'
[1, 1, 1, 0, 1]] # 'a','b','c','e'
How could I transfer pred to labels name? I can't do it after sees.run() or using other api like numpy because this is for tensorflow-serving, I think I need to do it using tf method.
Any suggestions are appreciated !
You should first define given probabilities of many classes, which classes you want to return. For example if the probability of this class is above 0.5.
Then you can use tf.where with proper condition to get indices and then same tf.gather to get classes.
Like this:
indicies = tf.where(tf.greater(pred, 0.5))
classes = tf.gather(CLASSES, indicies[:, 1])
Then you need to re-organize it using indicies[:, 0] that tells you which example the class from.
Also, you must understand that correct form of the answer is SparseTensor, which are not very supported by serving and etc. So you may want to return two tensors [strings and indicators which strings are for which example] and deal with on your client side.
Related
I have a CV tracking algorithm that gives me the 2D coordinates of the centroid of the object of interest (a red ball) in real time. I want to use a Kalman Filter to obtain the predicted coordinates of the ball in the next frame (future).
The thing is that I don't know if I should:
Predict (state k), Correct (state k), and then Predict again (state k+1).
Correct (state k), and then Predict (state k+1).
Predict (state k), Correct (state k).
The first two approaches gave me decent results. However, the results obtained in the last approach were practically the same as the mesures (I guess this is because I am not doing a prediction for the next future state k+1).
What is the proper way to obtain the predicted coordinates of the ball in the following frame (future state k+1) using a Kalman Filter?
Code used:
Initialization of Kalman filter:
kf = cv2.KalmanFilter(4, 2) #position x,y and velocity x,y
kf.measurementMatrix = np.array([[1, 0, 0, 0], [0, 1, 0, 0]], np.float32)
kf.transitionMatrix = np.array([[1, 0, 1, 0], [0, 1, 0, 1], [0, 0, 1, 0], [0, 0, 0, 1]], np.float32)
kf.processNoiseCov =1*np.array([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]], np.float32)
kf.measurementNoiseCov = 1*np.array([[1, 0], [0, 1]], np.float32)
First approach:
def Estimate(kf,x,y):
predicted = kf.predict()
measured = np.array([[np.float32(coordX)], [np.float32(coordY)]])
estimate=kf.correct(measured)
predicted = kf.predict()
return predicted
Second approach:
def Estimate(kf,x,y):
measured = np.array([[np.float32(coordX)], [np.float32(coordY)]])
estimate=kf.correct(measured)
predicted = kf.predict()
return predicted
Note: the function Estimate is called inside a while loop every time a new pair of coordinates is obtained.
Edit: in these links you can see the results of the first and second approaches, respectively:
First approach
Second approach
I'm trying to mask bad pixels in a dataset taken from a detector. In my attempt to come up with a general way to do this so I can run the same code across different images, I tried a few different methods, but none of them ended up working. I'm pretty new with coding and data analysis in Python, so I could use a hand putting things in terms that the computer will understand.
As an example, consider the matrix
A = np.array([[3,5,50],[30,2,6],[25,1,1]])
What I'm wanting to do is set any element in A that is two standard deviations away from the mean equal to zero. The reason for this is that later in the code, I'm defining a function that only uses the nonzero values for the calculation, since the zeros are part of the mask.
I know this masking technique works, but I tried extending the following code to work with the standard deviation:
mask = np.ones(np.shape(A))
mask.flat[A.flat > 20] = 0
What I tried was:
mask = np.ones(np.shape(A))
for i,j in A:
mask.flat[A[i,j] - 2*np.std(A) < np.mean(A) < A[i,j] + 2*np.std(A)] = 0
Which throws the error:
ValueError: too many values to unpack (expected 2)
If anyone has a better technique to statistically remove bad pixels in an image, I'm all ears. Thanks for the help!
==========
EDIT
After some trial and error, I got to a place that could help clarify my question. The new code is:
for i in A:
for j in i:
mask.flat[ j - 2*np.std(A) < np.mean(A) < j + 2*np.std(A)] = 0
This throws an error saying 'unsupported iterator index'. What I'm wanting to happen is that the for loop iterates across each element in the array, checks if it's less/greater than 2 standard deviations from the mean, and it is, sets it to zero.
Here is an approach that will be sligthly faster on larger images:
import numpy as np
import matplotlib.pyplot as plt
# generate dummy image
a = np.random.randint(1,5, (5,5))
# generate dummy outliers
a[4,4] = 20
a[2,3] = -6
# initialise mask
mask = np.ones_like(a)
# subtract mean and normalise to standard deviation.
# then any pixel in the resulting array that has an absolute value > 2
# is more than two standard deviations away from the mean
cond = (a-np.mean(a))/np.std(a)
# find those pixels and set them to zero.
mask[abs(cond) > 2] = 0
Inspection:
a
array([[ 1, 1, 3, 4, 2],
[ 1, 2, 4, 1, 2],
[ 1, 4, 3, -6, 1],
[ 2, 2, 1, 3, 2],
[ 4, 1, 3, 2, 20]])
np.round(cond, 2)
array([[-0.39, -0.39, 0.11, 0.36, -0.14],
[-0.39, -0.14, 0.36, -0.39, -0.14],
[-0.39, 0.36, 0.11, -2.12, -0.39],
[-0.14, -0.14, -0.39, 0.11, -0.14],
[ 0.36, -0.39, 0.11, -0.14, 4.32]])
mask
array([[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 0, 1],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 0]])
You A is three dimensional so you need to unpack using three variables like below.
A = np.array([[3,5,50],[30,2,6],[25,1,1]])
for i in A:
for j in i:
print(j)
Ive been breaking my head over trying to come up with a recursive way to build the following matrix in python. It is quite a challenge without pointers. Could anyone maybe help me out?
The recursion is the following:
T0 = 1,
Tn+1 = [[Tn, Tn],
[ 0, Tn]]
I have tried many iterations of some recursive function, but I cannot wrap my head around it.
def T(n, arr):
n=int(n)
if n == 0:
return 1
else:
c = 2**(n-1)
Tn = np.zeros((c,c))
Tn[np.triu_indices(n=c)] = self.T(n=n-1, arr=arr)
return Tn
arr = np.zeros((8,8))
T(arr=arr, n=3)
It's not hard to do this, but you need to be careful about the meaning of the zero in the recursion. This isn't really precise for larger values of n:
Tn+1 = [[Tn, Tn],
[ 0, Tn]]
Because that zero can represent a block of zeros for example on the second iteration you have this:
[1, 1, 1, 1],
[0, 1, 0, 1],
[0, 0, 1, 1],
[0, 0, 0, 1]
Those four zeros in the bottom-left are all represented by the one zero in the formula. The block of zeros needs to be the same shape as the blocks around it.
After that it's a matter of making Numpy put thing in the right order and shape for you. numpy.block is really handy for this and makes it pretty simple:
import numpy as np
def makegasket(n):
if n == 0:
return np.array([1], dtype=int)
else:
node = makegasket(n-1)
return np.block([[node, node], [np.zeros(node.shape, dtype=int), node]])
makegasket(3)
Result:
array([[1, 1, 1, 1, 1, 1, 1, 1],
[0, 1, 0, 1, 0, 1, 0, 1],
[0, 0, 1, 1, 0, 0, 1, 1],
[0, 0, 0, 1, 0, 0, 0, 1],
[0, 0, 0, 0, 1, 1, 1, 1],
[0, 0, 0, 0, 0, 1, 0, 1],
[0, 0, 0, 0, 0, 0, 1, 1],
[0, 0, 0, 0, 0, 0, 0, 1]])
If you use larger n you might enjoy matplotlib.pyplot.imshow for display:
from matplotlib.pyplot import imshow
# ....
imshow(makegasket(7))
You don't really need a recursive function to implement this recursion. The idea is to start with the UR corner and build outward. You can even start with the UL corner to avoid some of the book-keeping and flip the matrix along either axis, but this won't be as efficient in the long run.
def build_matrix(n):
size = 2**n
# Depending on the application, even dtype=np.bool might work
matrix = np.zeros((size, size), dtype=np.int)
# This is t[0]
matrix[0, -1] = 1
for i in range(n):
k = 2**i
matrix[:k, -2 * k:-k] = matrix[k:2 * k, -k:] = matrix[:k, -k:]
return matrix
Just for fun, here is a plot of timing results for this implementation vs #Mark Meyer's answer. It shows the slight timing advantage (also memory) of using a looping approach in this case:
Both algorithms run out of memory around n=15 on my machine, which is not too surprising.
I'm working on an image classification problem where I got the train labels as a 1-D numpy array, like [1,2,3,2,2,2,4,4,3,1]. I used
train_y = []
for label in train_label:
if label == 0:
train_y.append([1,0,0,0])
elif label == 1:
train_y.append([0,1,0,0])
elif label == 2:
train_y.append([0,0,1,0])
elif label == 3:
train_y.append([0,0,0,1])
Also I need the len(one_hot_array) = set(train_labels),
but this is not a good method. Please recommend a good method to do so.
It's always a good habit to use numpy for arrays. np.unique() determins the labels you have in train_labels. ix is an array of indices. np.nonzero() gives the indices of train_lables where train_labels == unique_tl[iy].
import numpy as np
train_labels = np.array([2,5,8,2,5,8])
unique_tl = np.unique(train_labels)
NL = len(train_labels) # how many data , 6
nl = len(unique_tl) # how many labels, 3
target = np.zeros((NL,nl),dtype=int)
for iy in range(nl):
ix = np.nonzero(train_labels == unique_tl[iy])
target[ix,iy] = 1
gives
target
array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1],
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
I'll think about a possibility to eliminate the for-loop.
If [2,5,8] is meant as part of [0,1,2,3,4,5,6,7,8], then you can use this answer
make a vector of zeros, and set only one value to 1
target = np.zeros(num_classes)
target[label] = 1
train_y.append(target)
I am new to using the library Sympy. I am need to extract all coefficients of the characteristic polynomial to be used later.
For example, my code is:
import sympy as sp
M = sp.Matrix([[0, 0, 0, 1, 0, 1], [0, 0, 0, 0, 1, 0], [0, 1, 0, 1, 0, -1], [1, 0, -1, 0, 1, 0], [0, 0, 0, 1, 0, 0], [-1, 0, 1, 0, 0, 0]])
lamda = symbols('lamda')
p = M.charpoly(lamda)
print(p)
print(p.coeffs())
which gives output:
PurePoly(lamda**6 + lamda**4 - lamda**2, lamda, domain='ZZ')
[1, 1, -1]
However, I need [1, 0, 1, 0, 1, 0, 0], which includes the zero coefficients of the lamda too the exponents 4, 3, 1, and 0, terms. I would normally use a for loop to iterate over the equation to see which terms are missing so a zero can be inserted into the appropriate spot in the array of coefficients. However, when I attempted to do so, I received an error saying PurePoly type doesn't support indexing. So, I was wondering if anyone knows how to make sympy include the zeros or a way to do it myself? I need will eventually have to incorporate this code into a loop for lots of matrices so I can't manually do it.
Thanks.
When I have questions like this I hope for some sort of intelligent naming of methods for objects and look through the directory of the object:
>>> print([w for w in dir(p) if 'coeff' in w])
['all_coeffs', 'as_coeff_Add', 'as_coeff_Mul', ...]
That all_coeffs is the one you want:
>>> help(p.all_coeffs)
Help on method all_coeffs in module sympy.polys.polytools:
all_coeffs(f) method of sympy.polys.polytools.PurePoly instance
Returns all coefficients from a univariate polynomial ``f``.
>>> p.all_coeffs()
[1,0,1,0,−1,0,0]