Related
I'm training a network to reconstruct coordinates of specific structures in an image. Until now, my loss function contains three 2D vectors (i.e. 6 variables) for the coordinates which are learned via MSE, and three corresponding classifiers learned via SigmoidFocalCrossEntropy, indicating if there are 0, 1, 2 or 3 of these structures present. I thought it might be beneficial to give tensorflow the information that it is neglectable in which order that the vectors are reconstructed as long as the classifier is still correct. Simple example:
loss(tf.constant([[30, 20, 15, 7, 0, 0, 1, 1, 0]], dtype=tf.float32),
tf.constant([[0, 0, 15, 7, 30, 20, 0, 1, 1]], dtype=tf.float32)) == 0
To implement this I used tf.argsort on the magnitude of each vector:
def sort(tensor):
x = tf.unstack(tensor, axis=-1)
squ = []
for i in range(len(x) // 2):
i *= 2
squ.append(x[i] ** 2 + x[i+1] ** 2)
new = tf.stack(squ, axis=-1)
return tf.argsort(new, axis=-1, direction='ASCENDING',
stable=False, name=None)
and consecutively permuted the tensor:
def permute_tensor_structure(tensor, indices):
c = indices + 6
x = indices * 2
y = indices * 2 + 1
v = tf.transpose([x, y])
v = tf.reshape(v, [indices.shape[0], -1])
perm = tf.concat([v, c], axis=-1)
return tf.gather(tensor, perm, batch_dims=1, name=None, axis=-1)
I did the same for my ground truth and got the network up and running.
Minimal example extracted from my code:
import tensorflow as tf
from tensorflow_addons.losses import SigmoidFocalCrossEntropy
def compute_permute_loss(truth, predict):
l2 = tf.keras.losses.MeanSquaredError()
ce = SigmoidFocalCrossEntropy()
indices = sort(predict[:, 0:6])
indices2 = sort(truth[:, 0:6])
predict = permute_tensor_structure(predict, indices)
truth = permute_tensor_structure(truth, indices2)
L2 = l2(predict[:, 0:6], truth[:, 0:6])
BCE = tf.reduce_sum(ce(truth[:, 6:], predict[:, 6:],))
return 3 * L2 + BCE
class Test(tf.test.TestCase):
def test_permutation_loss(self):
tensor1 = tf.constant(
[[30, 20, 15, 7, 0, 0, 1, 1, 0]],
dtype=tf.float32)
tensor2 = permute_tensor_structure(tensor1, tf.constant([[2, 1, 0]]))
loss = compute_permute_loss(tensor1, tensor2)
self.assertEqual(loss, 0,
msg="loss for only permuted tensors is not zero")
tensor3 = tf.constant(
[[29, 22, 15, 7, 0, 0, 1, 1, 0]],
dtype=tf.float32)
loss = compute_permute_loss(tensor3, tensor2)
self.assertAllClose(loss, (1.0 + 4.0) / 2.0,
msg="loss for values is not rmse")
if __name__ == "__main__":
tf.test.main()
# [ RUN ] Test.test_permutation_loss
# [ OK ] Test.test_permutation_loss
However, I'm afraid the permutation in the loss could backfire and impair tensorflows backpropagation. Maybe somebody already faced a similar problem? Or has deeper knowledge on tensorflows graph building and back propagation? I would be grateful for every suggestion or input.
import numpy as np
import sympy as sp
from sympy import *
init_printing()
uVars = list(symbols(', '.join([f'u{n}' for n in range(1, 3 + 1)])))
aVars = list(symbols(', '.join([f'a{n}' for n in range(1, 3 + 1)])))
lambda1, mu = symbols('lambda, mu')
U = np.array([ [0, -uVars[2], uVars[1]], [uVars[2], 0, -uVars[0]], [-uVars[1], uVars[0], 0] ])
a = np.array([ [aVars[0], 0, 0], [0, aVars[1], 0], [0, 0, aVars[2]] ])
I = np.eye(3)
L = a*lambda1 + U
preCharPoly = L - mu*I
preCharPoly_sym = sp.Matrix(preCharPoly)
factor(preCharPoly_sym.det())
The above code outputs the following polynomial:
However, I require the polynomial to be factored with respect to the variables lambda and mu as shown here:
I have been examining the documentation at https://docs.sympy.org/latest/modules/simplify/simplify.html but cannot figure out how to do what is desired. How do I specify factor() or simplify() to perform their tasks with respect to lambda and mu?
I would like to know if there is an efficient method to get sub-arrays from a larger numpy array.
What I have is an application of np.where. I iterate 'manually' over x and y as offsets and apply where with a kernel to each rectangle extracted from the larger array with proper dimensions.
But is there a more direct approach in numpy's collection of methods?
import numpy as np
example = np.arange(20).reshape((5, 4))
# e.g. a cross kernel
a_kernel = np.asarray([[0, 1, 0], [1, 1, 1], [0, 1, 0]])
np.where(a_kernel, example[1:4, 1:4], 0)
# returns
# array([[ 0, 6, 0],
# [ 9, 10, 11],
# [ 0, 14, 0]])
def arrays_from_kernel(a, a_kernel):
width, height = a_kernel.shape
y_max, x_max = a.shape
return [np.where(a_kernel, a[y:(y + height), x:(x + width)], 0)
for y in range(y_max - height + 1)
for x in range(x_max - width + 1)]
sub_arrays = arrays_from_kernel(example, a_kernel)
This returns the arrays I need for further processing.
# [array([[0, 1, 0],
# [4, 5, 6],
# [0, 9, 0]]),
# array([[ 0, 2, 0],
# [ 5, 6, 7],
# [ 0, 10, 0]]),
# ...
# array([[ 0, 9, 0],
# [12, 13, 14],
# [ 0, 17, 0]]),
# array([[ 0, 10, 0],
# [13, 14, 15],
# [ 0, 18, 0]])]
The context: similar to 2D convolution I would like to apply a custom function on each of the subarrays (e.g. product of squared numbers).
At the moment, you're manually advancing a sliding window over the data - stride tricks to the rescue! (And no, I didn't just make that up - there's actually a submodule called stride_tricks in numpy!) Instead of manually building windows into the data, and calling np.where() on them, if you had the windows in an array, you could call np.where() just once. Stride tricks allow you to create such an array without even having to copy the data.
Let me explain. Normal slices in numpy create views into the original data instead of copies. This is done by referring to the original data, but changing the strides used to access the data (ie. how much to jump between two elements or two rows, and so on). Stride tricks allow you to modify those strides more freely than just slicing and reshaping does, so you can eg. iterate over the same data more than once, which is useful here.
Let me demonstrate:
import numpy as np
example = np.arange(20).reshape((5, 4))
a_kernel = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]])
def sliding_window(data, win_shape, **kwargs):
assert data.ndim == len(win_shape)
shape = tuple(dn - wn + 1 for dn, wn in zip(data.shape, win_shape)) + win_shape
strides = data.strides * 2
return np.lib.stride_tricks.as_strided(data, shape=shape, strides=strides, **kwargs)
def arrays_from_kernel(a, a_kernel):
windows = sliding_window(a, a_kernel.shape)
return np.where(a_kernel, windows, 0)
sub_arrays = arrays_from_kernel(example, a_kernel)
The scipy.ndimage module offers a number of filters -- one of which might meet your needs. If none of those filters do what you want, you could use ndimage.generic_filter
to call a custom function on each subarray. ndimage.generic_filter is not as fast as the other ndimage filters, however.
For example,
import numpy as np
example = np.arange(20).reshape((5, 4))
a_kernel = np.asarray([[0, 1, 0], [1, 1, 1], [0, 1, 0]])
# def arrays_from_kernel(a, a_kernel):
# width, height = a_kernel.shape
# y_max, x_max = a.shape
# return [np.where(a_kernel, a[y:(y + height), x:(x + width)], 0)
# for y in range(y_max - height + 1)
# for x in range(x_max - width + 1)]
# sub_arrays = arrays_from_kernel(example, a_kernel)
# for arr in sub_arrays:
# print(arr)
# print('-'*80)
import scipy.ndimage as ndimage
def func(x):
# reject subarrays that extend beyond the border of the `example` array
if not np.isnan(x).any():
y = np.zeros_like(a_kernel, dtype=example.dtype)
np.put(y, np.flatnonzero(a_kernel), x)
print(y)
# Instead or returning 0, you can perform your desired computation on the subarray here.
# Note that you may not need the 2D array y; often, you only need the values in the 1D array x
return 0
result = ndimage.generic_filter(example, func, footprint=a_kernel, mode='constant', cval=np.nan)
For the particular problem of computing the product of squares for each subarray, you
could convert the product into a sum by taking advantage of the fact that A * B = exp(log(A)+log(B)). This would allow you to express the computation as a normal convolution. Now using ndimage.convolve can improve performance a lot. The amount of the improvement depends on the size of example:
import numpy as np
import scipy.ndimage as ndimage
import perfplot
a_kernel = np.asarray([[0, 1, 0], [1, 1, 1], [0, 1, 0]])
def orig(example, a_kernel=a_kernel):
def arrays_from_kernel(a, a_kernel):
width, height = a_kernel.shape
y_max, x_max = a.shape
return [
np.where(a_kernel, a[y : (y + height), x : (x + width)], 1)
for y in range(y_max - height + 1)
for x in range(x_max - width + 1)
]
return [np.prod(x) ** 2 for x in arrays_from_kernel(example, a_kernel)]
def alt(example, a_kernel=a_kernel):
logged = np.log(example)
result = ndimage.convolve(logged, a_kernel, mode="constant", cval=0)[1:-1, 1:-1]
return (np.exp(result) ** 2).ravel()
def make_example(N):
return np.random.random(size=(N, N))
def check(A, B):
return np.allclose(A, B)
perfplot.show(
setup=make_example,
kernels=[orig, alt],
n_range=[2 ** k for k in range(2, 11)],
logx=True,
logy=True,
xlabel="len(example)",
equality_check=check,
)
I am trying to modify a code written by a software developer (Kyle Dickerson) and have written it up like this:
So I have this code:
from __future__ import division
## Kyle Dickerson
## kyle.dickerson#gmail.com
## Jan 15, 2008
##
## Self-organizing map using scipy
## This code is licensed and released under the GNU GPL
## This code uses a square grid rather than hexagonal grid, as scipy allows for fast square grid computation.
## I designed sompy for speed, so attempting to read the code may not be very intuitive.
## If you're trying to learn how SOMs work, I would suggest starting with Paras Chopras SOMPython code:
## http://www.paraschopra.com/sourcecode/SOM/index.php
## It has a more intuitive structure for those unfamiliar with scipy, however it is much slower.
## If you do use this code for something, please let me know, I'd like to know if has been useful to anyone.
from random import *
from math import *
import sys
import scipy
import numpy
class SOM:
def __init__(self, height=4, width=4, FV_size=3, learning_rate=0.005):
self.height = height
self.width = width
self.FV_size = FV_size
self.radius = (height+width)/3
self.learning_rate = learning_rate
self.nodes = scipy.array([[ [random()*255 for
i in range(FV_size)] for x in range(width)] for y in range(height)])
self.nodes = scipy.array([[1,2,3],[4,5,6],[4,5,6],
[4,5,6],[4,5,6], [4,5,6],[4,5,6],[4,5,6],[4,5,6],
[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6]])
print "SOM",self.nodes
def train(self, iterations=1000, train_vector=[[]]):
for t in range(len(train_vector)):
train_vector[t] = scipy.array(train_vector[t])
print "training",train_vector[t],t
time_constant = iterations/log(self.radius)
delta_nodes = scipy.array([[[0 for i in range(self.FV_size)]
for x in range(self.width)] for y in range(self.height)])
for i in range(1, iterations+1):
delta_nodes.fill(0)
radius_decaying=self.radius*exp(-1.0*i/time_constant)
rad_div_val = 2 * radius_decaying * i
learning_rate_decaying=self.learning_rate*exp(-1.0*i/time_constant)
sys.stdout.write("\rTraining Iteration:
" + str(i) + "/" + str(iterations))
for j in range(len(train_vector)):
best = self.best_match(train_vector[j])
for loc in self.find_neighborhood(best, radius_decaying):
influence = exp( (-1.0 * (loc[2]**2)) / rad_div_val)
inf_lrd = influence*learning_rate_decaying
delta_nodes[loc[0],loc[1]] += inf_lrd*
(train_vector[j]- self.nodes[loc[0],loc[1]])
self.nodes += delta_nodes
sys.stdout.write("\n")
# Returns a list of points which live within 'dist' of 'pt'
# Uses the Chessboard distance
# pt is (row, column)
def find_neighborhood(self, pt, dist):
min_y = max(int(pt[0] - dist), 0)
max_y = min(int(pt[0] + dist), self.height)
min_x = max(int(pt[1] - dist), 0)
max_x = min(int(pt[1] + dist), self.width)
neighbors = []
for y in range(min_y, max_y):
for x in range(min_x, max_x):
dist = abs(y-pt[0]) + abs(x-pt[1])
neighbors.append((y,x,dist))
return neighbors
# Returns location of best match, uses Euclidean distance
# target_FV is a scipy array
def best_match(self, target_FV):
loc = scipy.argmin((((self.nodes - target_FV)**2).sum(axis=2))**0.5)
r = 0
while loc > self.width:
loc -= self.width
r += 1
c = loc
return (r, c)
# returns the Euclidean distance between two Feature Vectors
# FV_1, FV_2 are scipy arrays
def FV_distance(self, FV_1, FV_2):
return (sum((FV_1 - FV_2)**2))**0.5
if __name__ == "__main__":
print "Initialization..."
colors = [ [0, 0, 0], [0, 0, 255], [0, 255, 0],
[0, 255, 255], [255, 0, 0], [255, 0, 255],
[255, 255, 0], [255, 255, 255]]
width = 32
height = 32
color_som = SOM(width,height,3,0.05)
print "Training colors..."
color_som.train(1000, colors)
try:
from PIL import Image
print "Saving Image: sompy_test_colors.png..."
img = Image.new("RGB", (width, height))
for r in range(height):
for c in range(width):
img.putpixel((c,r),(int(color_som.nodes[r,c,0]),
int(color_som.nodes[r,c,1]), int(color_som.nodes[r,c,2])))
print "color nodes",color_som.nodes
img = img.resize((width*10, height*10),Image.NEAREST)
img.save("sompy_test_colors.png")
except:
print "Error saving the image, do you have PIL (Python Imaging Library) installed?"
but when I try to go from
self.nodes = scipy.array([[ [random()*255
for i in range(FV_size)] for x in range(width)]
for y in range(height)])
which was in the original code to something like this:
self.nodes = scipy.array([[1,2,3],[4,5,6],[4,5,6],
[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],
[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6]])
I get the error message:
File "sompy5.py", line 112, in <module>
color_som.train(1000, colors)
File "sompy5.py", line 65, in train
best = self.best_match(train_vector[j])
File "sompy5.py", line 92, in best_match
loc = scipy.argmin((((self.nodes - target_FV)**2).sum(axis=2))**0.5)
File "/usr/lib/python2.7/dist-packages/numpy/core/_methods.py",
line 25, in _sum
out=out, keepdims=keepdims)
ValueError: 'axis' entry is out of bounds
Is there something that has to be done to get the vectors to match up?
This part is a 3-D array (3 square brackets to begin the argument):
self.nodes = scipy.array([[ [random()*255 for
i in range(FV_size)] for x in range(width)] for y in range(height)])
This part is a 2-D array:
self.nodes = scipy.array([[1,2,3],[4,5,6],[4,5,6],
[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],
[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6],[4,5,6]])
So you need to turn self.nodes into the appropriate 3-D array.
EDIT: an example of the required syntax:
self.nodes = scipy.array([[ [1,2,3],[4,5,6]] , [[7,8,9],[10,11,12]]])
print(self.nodes)
>>> array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
EDIT 2:
Another option is to build a linear array and then reshape():
myarray = scipy.array([1,2,3,4,5,6,7,8,9,10,11,12])
myarray = myarray.reshape( (2, 2, 3) ) ## 3 numbers for 3 dimensions, but the product must be the same as the number of elements of the original array
print(myarray)
>>> array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
I am quite new to Python, so do excuse me if the following question has a 'duh' answer.
So, I'm trying to solve an ODE using odeint and wish to pass an array. But, the TypeError: can't multiply sequence by non-int of type 'float' keeps cropping up, in the line:
CA0 = (-kd-kn*Cv)*CAi/(1+(CAi/ks))
So, the code is:
from scipy.integrate import odeint
import numpy as np
Ap_data = [2, 7, 91, 1.6, 0.4, 5]
tdata= [0, 1, 4, 5, 4, 20]
Cv_data = [43, 580, 250, 34, 30, 3]
#Define parameters
kn = 1E-5 #change
ks = 1E+5 #change
kd = 0.058
def deriv (CAi,t, Cv):
CA0 = (-kd-kn*Cv)*CAi/(1+(CAi/ks))
return CA0
#Initial conditions
CA_init = 21.6
#Solve the ODE
(CAb_soln) = odeint (derivCAb, CA_init, tdata, (Cv_data,))
print CAb_soln
Some help, please?
Your immediate problem is that your deriv function is trying to multiply the ordinary Python list, Cv_data (passed in as Cv) by float values. If you want to vectorize this operation, use NumPy arrays:
Ap_data = np.array([2, 7, 91, 1.6, 0.4, 5])
tdata= np.array([0, 1, 4, 5, 4, 20])
Cv_data = np.array([43, 580, 250, 34, 30, 3])
to solve this. You now have the problem that odeint fails for the input you give it...
intdy-- t (=r1) illegal
in above message, r1 = 0.4000000000000D+01
t not in interval tcur - hu (= r1) to tcur (=r2)
in above, r1 = 0.4287484688360D+01 r2 = 0.5551311182627D+01
lsoda-- trouble from intdy. itask = i1, tout = r1ls
in above message, i1 = 1
in above message, r1 = 0.4000000000000D+01
Illegal input detected (internal error).
Run with full_output = 1 to get quantitative information.
[[ 21.6 ]
[ 20.37432613]
[ 17.09897165]
[ 16.12866355]
[ 16.12866355]
[ -0.90614016]]
Perhaps you can give more information about what your equation is and how it relates to Cv_data. In particular, your derivative doesn't depend on t, but you have a range of values for this parameter, Cv.
UPDATE: It fails because of your funny time series. odeint works properly if it is monotonic, for example:
from scipy.integrate import odeint
import numpy as np
Ap_data = [2, 7, 91, 1.6, 0.4, 5]
tdata= np.array([0, 1, 4, 5, 10, 20])
Cv_data = np.array([43, 580, 250, 34, 30, 3])
#Define parameters
kn = 1E-5 #change
ks = 1E+5 #change
kd = 0.058
def deriv (CAi,t, Cv):
CA0 = (-kd-kn*Cv)*CAi/(1+(CAi/ks))
return CA0
#Initial conditions
CA_init = 21.6
#Solve the ODE
(CAb_soln) = odeint (deriv, CA_init, tdata, (Cv_data,))
print CAb_soln
The result:
[[ 21.6 ]
[ 20.37432613]
[ 17.09897165]
[ 16.12866355]
[ 12.04306424]
[ 6.71431758]]
Well, as it turns out I cannot post an image yet (being new to stackoverflow). So, the code that I used was-
from scipy.integrate import odeint
import numpy as np
Ap_data = np.array([2, 7, 91, 1.6, 0.4, 5])
tdata= [0, 1, 4, 5, 4, 20]
Cv_data = np.array([43, 580, 250, 34, 30, 3])
#Define parameters
kn = 1E-5 #change
ks = 1E+5 #change
kd = 0.058
def deriv (CAi,t, Cv):
CA0 = (-kd-kn*Cv)*CAi/(1+(CAi/ks))
return CA0
#Initial conditions
CA_init = 21.6
#Solve the ODE
(CAb_soln) = odeint (deriv, CA_init, tdata, (Cv_data,), full_output=True)
print CAb_soln