I want to reconstruct the output shape of my Input layer, in a way that for example if it is (None, 42), I start from 1st neuron and take 1st to 7th, then slide a neuron to right and take 2nd to 8th and so on. in this way output layer would be the shape (None, 36, 7).
I am using keras.layers.Lambda() layer to do that but it is not working properly and dimensions are not what i expect. the code i wrote is as follow:
Inpt_lyr = keras.Input(shape = (42,))
for k in range(42 - 7 + 1):
F = keras.layers.Lambda(lambda x, i, j: x[:,j : j + i])
F.arguments ={ 'i' : 7, 'j' : k}
tmp_rfrm = F(Inpt_lyr)
lst_rfrm.append(tmp_rfrm)
cnctnt_lyr = keras.layers.merge.Concatenate(axis = 0)(lst_rfrm)
tmp_dns = keras.layers.Dense(3 , activation = 'softmax')(cnctnt_lyr)
tmp_model = Model(Inpt_lyr, tmp_dns)
tmp_model.compile(loss = 'categorical_crossentropy', optimizer = 'adam')
tmp_model.summary()
I expect conct_lyr have the shape (None, 36, 7), but it is (None,7)
and it also give the error
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_3:0", shape=(?, 42), dtype=float32) at layer "input_3". The following previous layers were accessed without issue: ['input_4']
Your approach looks sensible to me, but the tensors are currently being concatenated along the batch axis, which is undesirable in this case.
I would suggest expanding dimension 1 while sliding the windows:
for k in range(42 - 7 + 1):
F = keras.layers.Lambda(lambda x, i, j: x[:, None, j: j + i])
# F(Inpt_lyr).shape=(?, 1, 7)
and then concatenate the tensors along the expanded axis:
cnctnt_lyr = keras.layers.merge.Concatenate(axis=1)(lst_rfrm)
# cnctnt_lyr.shape=(?, 36, 7)
Related
I am replicating a pytorch model in keras and ahve problems to see where the extra dimension comes from.
This how my code looks so far:
class Attention(tf.keras.Model):
def __init__(self, input_shape):
super(Attention, self).__init__()
in_features=input_shape[-1]
small_in_features = max(math.floor(in_features/10), 1)
self.d_k = small_in_features
query = tf.keras.models.Sequential()
query.add(tf.keras.layers.Dense(in_features))
query.add(tf.keras.layers.Dense(small_in_features,activation="tanh"))
self.query= query
self.key = tf.keras.layers.Dense(small_in_features)
def call(self, inp):
# inp.shape should be (B,N,C)
q = self.query(inp) # (B,N,C/10)
k = self.key(inp) # B,N,C/10
k = tf.transpose(k)
print(q)
print(k)
x = tf.linalg.matmul(q, k) / math.sqrt(self.d_k) # B,N,N
x = tf.nn.softmax(x) # over rows
x = tf.transpose(x)
x = tf.linalg.matmul(x, inp) # (B, N, C)
return x
But if I want to add it to my Sequential model I get this Error:
ValueError: Dimensions must be equal, but are 1 and 256 for '{{node attention_19/MatMul}} = BatchMatMulV2[T=DT_FLOAT, adj_x=false, adj_y=false](attention_19/sequential_36/Identity, attention_19/transpose)' with input shapes: [?,256,1], [1,256,?].
I have now printed my 'q' and 'k' and it prints out like following:
Tensor("attention_19/sequential_36/Identity:0", shape=(None, 256, 1), dtype=float32)
Tensor("attention_19/transpose:0", shape=(1, 256, None), dtype=float32)
So they are 3 dimensional where one dimension is unfilled.I dont quite understand why it happens.
How can I "remove" the extra dimension or bring this custom layer to work?
Note: The original codes seems to use 3 dimensional Input but I want 2 dimensional input.
TL;DR:
Have input (batch_size,128,60,41,2) and labels (batch_size, 128), each label is either 0, 1, or 2. How should network output be/how to design output?
I am converting audio clips of length 60 seconds each to an array of (128, 60, 41, 2)*. This is my feature data per example.
As for the labels, I have (per example) and array of shape (128,), for each of the 128 things I extract.
So one (feature, label) pair is int the form (feature={128, 60, 41,2}, label={128}).
When I batch the data, the features and labels get appended to; e.g. for a batch of size 10: the features are of shape: (10,128,60,41,2), and labels are of shape (10,128)
My clarified question is: How can I design the network to calculate a loss based on this labels?
The longer version:
The last dense layer should have 3 units, one per class. Now, I have a batch with bs items. Thus, I have labels in the shape (bs,128). How can the network be designed to calculate the loss; the first batch item is of shape (128,60,41,2), and the labels for this first item are in shape (128,). Each label within is either 0, 1, or 2. Now I want to calculate to design the network to have as the last output a shape of (None, 128,3).
None is the batch size, 128 for all the things I extract, and the 3 because I have three classes
Edit: Thanks for the notes, I hopefully clarified the question
*For those further interested:
I use a sliding window over the time axis. For each window, I extract log-scaled spectrograms. Here, 128 is the number of windows, 60 and 41 control the Mel scale, and 2 is for adding a delta dimension.
The code to generate the audio:
def sub_method(fn, label, bands, frames, delta):
def _windows(data, window_size):
start = 0
while start < len(data):
yield int(start), int(start + window_size)
start += (window_size // 2)
window_size = 512 * (frames - 1)
segment_log_specgrams, segment_labels = [], []
sound_clip,sr = librosa.load(fn)
for (start,end) in _windows(sound_clip,window_size):
if(len(sound_clip[start:end]) == window_size):
signal = sound_clip[start:end]
melspec = librosa.feature.melspectrogram(signal,n_mels=bands)
logspec = librosa.amplitude_to_db(melspec)
logspec = logspec.T.flatten()[:, np.newaxis].T
segment_log_specgrams.append(logspec)
segment_labels.append(label)
if delta:
segment_log_specgrams = np.asarray(segment_log_specgrams)
segment_log_specgrams = segment_log_specgrams.reshape(len(segment_log_specgrams),bands,frames,1)
segment_features = np.concatenate((segment_log_specgrams, np.zeros(np.shape(segment_log_specgrams))), axis=3)
for i in range(len(segment_features)):
segment_features[i, :, :, 1] = librosa.feature.delta(segment_features[i, :, :, 0])
else:
segment_features = segment_log_specgrams
if len(segment_features) > 0: # check for empty segments
return 1, segment_features, segment_labels
else:
return 0, 0, 0
Try this:
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, Reshape
from tensorflow.keras import Sequential
model = Sequential([
Reshape((128, -1), input_shape=(128, 60, 41, 2)),
Dense(3)
])
inp = tf.random.uniform([10, 128, 60, 41, 2], dtype=tf.float32)
labels = tf.random.uniform([10, 128], 0, 3, dtype=tf.int32)
pred = model(inp)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(inp, labels)
I want to make a network that uses multiple output.
For example, I want to put an input of list which has the shape of :
[ 8, 128, 128, 3]
Here, 8 is number of images in one set of input, 128 x 128 x 3 is shape of a color image.
And I want my output to be :
[ 8, 128, 128]
Here, 8 is number of images in one set of output, 128 x 128 is shape of a gray image.
So I made my code as following :
f_list = []
for i in range(v):
img_in = M.Input((h, w, 3)) # Input :
feature = spm.encoder(img_in)
f_list.append(feature)
fuse_ave,fuse_max,fuse_min = spm.fusion(f_list)
decode_list = []
for i in range(v):
ffuse = spm.decoder(f_list[i],fuse_ave,fuse_max)
decode_list.append(ffuse)
dl_con = L.concatenate(decode_list,0)
print(dl_con.shape)
epsnet = M.Model(inputs = img_in,outputs = dl_con)
epsnet.compile(optimizer=O.Adam(lr=0.0001,decay = 0.000001), loss='mean_squared_error', metrics=['accuracy'])
epsnet.fit(iml,np.array(gml),batch_size = 5, epochs=50,verbose=1,shuffle=True, validation_split=0.1)
Here, the function decode is as following :
def decoder(input,fuse_a,fuse_M): #input : encoded
infu = L.Concatenate(-1)([input,fuse_a,fuse_M])
f=128
x = L.Conv2DTranspose(filters = f,kernel_size=(5,5),strides=2,padding = 'same')(infu) dding = 'same')(x__)
...
x__ = L.BatchNormalization()(x__)
x__ = L.Activation('relu')(x__)
def sq(x):
x_sq = B.squeeze(x,-1)
return x_sq
xq = L.Lambda(sq)(x__)
return xq
Here I am getting error message :
ValueError: Output tensors to a Model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: Tensor("concatenate_7/concat:0", shape=(?, ?, ?), dtype=float32)
I tried several ways, but I still get same error message. Please give me a breakthrough and thank you very much.
I'm trying to understand CapsNet for classification. while training the network, I'm getting the error Consider casting elements to a supported type in squash function
I have Tried an existing code and printed the output of every layer but unable to understand the reason of error.
Here is code for squash function and CapsNet architecture.
def squash(vectors, axis=-1):
s_squared_norm = K.sum(K.square(vectors), axis, keepdims=True)
scale = s_squared_norm / (1 + s_squared_norm) /K.sqrt(s_squared_norm + K.epsilon())
return scale * vectors
def capsnetwork(self, capsnetwork):
# depending on dataset we define input shape for our network
img = Input(self.shape_high_reso)
print("x",img.shape)
#x (?, 32, 32, 3)
x = Conv2D(filters=256, kernel_size=9, strides=1, padding='valid', name='conv1')(img)
print("x",x.shape)
#x (?, 24, 24, 256)
x = LeakyReLU()(x)
# # original 'Dynamic Routing Between Capsules' paper does not include the batch norm layer after the first conv group
x = BatchNormalization(momentum=0.8)(x)
# filters 256 (n_vectors=8 * channels=32)
x = Conv2D(filters=8 * 32, kernel_size=9, strides=2, padding='valid', name='primarycap_conv2')(x)
print("x",x.shape)
#x (?, 8, 8, 256)
# reshape into the 8D vector for all 32 feature maps combined
# (primary capsule has collections of activations which denote orientation of the digit
# while intensity of the vector which denotes the presence of the digit)
x = Reshape(target_shape=[-1, 8], name='primarycap_reshape')(x)
print("x",x.shape)
#x (?, ?, 8)
# the purpose is to output a number between 0 and 1 for each capsule where the length of the input decides the amount
x = Lambda(self.squash, name='primarycap_squash')(x)
I expected the training of the network should be done, but it outputs an error at line
x = Lambda(self.squash, name='primarycap_squash')(x)
The error is
TypeError: Failed to convert object of type <class 'models.Capsnet.Capsnet'> to Tensor.
Contents: <models.Capsnet.Capsnet object at 0x7fcd234eccc0>.
Consider casting elements to a supported type.
I'm writing a Lambda layer in Keras to compute the multiplication of a tensor and a matrix of constant. But the size went wrong in the output.
In the code, jdes is a tensor with dimension TensorShape([Dimension(None), Dimension(100)]). Further, six_title_embedding is the matrix of constants and it has shape (6, 100).
def cosine_distance(input):
jd = K.l2_normalize(input, axis=-1)
jt_six = K.l2_normalize(six_title_embedding, axis=-1)
return jd * jt_six
distance = Lambda(cosine_distance, output_shape=(None, 6,100))(jdes)
result = Dense(1, activation='sigmoid')(distance)
In the output, I was expecting result to have shape (None, 6, 1) but now it's (6, 1) so the number of batch_size is lost during the computation of the lambda layer. The shape of distance is now TensorShape([Dimension(6), Dimension(100)]). Should this be (None, 6, 100) so that result can have (None, 6, 1)?
You need to create an additional axis for jdes so that when multiplying it with constant tensor, they could be broadcasted to the same shape:
def cosine_distance(input):
jd = K.l2_normalize(input, axis=-1)
jt_six = K.l2_normalize(six_title_embedding, axis=-1)
jd = K.expand_dims(jd, axis=1) # now it would have a shape of (None, 1, 100)
return jd * jt_six # the result would be (None, 6, 100)