I am working on a project where I must use four different images as inputs. I take the inputs run them through a simple model and detect two classes.I am really struggling in the actual setup of the model.
Not sure if I am on the right track. I haven't been able to run the code since I am still unsure of the architecture of the model. The code below is my model how its setup currently. I have all the images already. I took one image and split it into four. Then using the four images, detect one of two classes. If this doesn't make sense or I am taking the wrong direction with this please help.'
# import the necessary packages
from keras.models import Sequential
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Dropout
from keras.layers.core import Dense
from keras.layers import Flatten
from keras.layers import Input
from keras.models import Model
from keras.layers import Dense ,LSTM,concatenate,Input,Flatten
from keras.models import Sequential
from keras.preprocessing import image
from keras.layers import Dense, InputLayer, Conv2D, MaxPool2D, Flatten
import keras
# define four sets of inputs
inputA = Input(shape=(200, 200, 3))
inputB = Input(shape=(200, 200, 3))
inputC = Input(shape=(200, 200, 3))
inputD = Input(shape=(200, 200, 3))
# merge all input images
merged = keras.layers.Concatenate(axis=1)([inputA, inputB, inputC, inputD])
# the first branch operates on the first input through the fourth input
dense1 = keras.layers.Conv2D(16, (2, 2), activation='relu')(merged)
output = keras.layers.Conv2D(16, (2, 2), activation='relu')(dense1)
# apply a FC layer and then a regression prediction on the
# combined outputs
z = keras.layers.Dense(128, activation="relu")(output)
z = keras.layers.Dense(9, activation="softmax")(z)
# then output a single value then our model will accept the inputs of the two branches and
# model = Model(inputs=[tt.output, y.output, t.output, w.output], outputs=z)
model.compile(loss='categorical_crossentropy', optimizer="adam", metrics=['accuracy'])
model.summary()
I think the model is setup correctly to run using four images as inputs, but now I need to setup the classes. My idea with is , to set up a CNN that looks at four different pictures of the same tree. Using those four images to detect the type of tree. I want all four images to be for one tree. Then detect the tree. After detecting the tree moving onto the next four images for a different tree.
Thanks everyone I am very grateful for all your inputs and help.
Related
Assuming there is a model given as an h5 file, i.e., I can not change the code building the model's architecture:
from tensorflow.keras.layers import Input, BatchNormalization
from tensorflow.keras.models import Model
inputs = Input(shape=(4,))
outputs = BatchNormalization()(inputs, training=True)
model = Model(inputs=inputs, outputs=outputs)
model.save('model.h5', include_optimizer=False)
Now I'd like to remove the training=True part, i.e., I want to the BatchNormalization as if it was attached to the model without this flag.
My current attempt looks as follows:
import numpy as np
from tensorflow.keras.models import load_model
model = load_model('model.h5')
for layer in model.layers:
for node in layer.inbound_nodes:
if "training" in node.call_kwargs:
del node.call_kwargs["training"]
model.predict(np.asarray([[1, 2, 3, 4]]))
But the model.predict calls fails with the following error (I'm using TensorFlow 2.5.0):
ValueError: Could not pack sequence. Structure had 1 elements, but flat_sequence had 2 elements. Structure: ((<KerasTensor: shape=(None, 4) dtype=float32 (created by layer 'input_1')>,), {}), flat_sequence: [<tf.Tensor 'model/Cast:0' shape=(None, 4) dtype=float32>, True].
How can this be fixed/worked around?
(When using node.call_kwargs["training"] = False instead of del node.call_kwargs["training"] then model.predict does not crash, but it simply behaves as if nothing was changed, i.e., the modified flag is ignored.)
I found simply saving and re-loading the model again after modifying the call_kwargs helps.
import numpy as np
from tensorflow.keras.models import load_model
model = load_model('model.h5')
# Removing training=True
for layer in model.layers:
for node in layer.inbound_nodes:
if "training" in node.call_kwargs:
del node.call_kwargs["training"]
# The two following lines are the solution.
model.save('model_modified.h5', include_optimizer=False)
model = load_model('model_modified.h5')
model.predict(np.asarray([[1, 2, 3, 4]]))
And all is fine. :)
have you tried
for layer in model.layers:
layer.trainable=False
I am implementing a simple multitask model in Keras. I used the code given in the documentation under the heading of shared layers.
I know that in multitask learning, we share some of the initial layers in our model and the final layers are made individual to the specific tasks as per the link.
I have following two cases in keras API where in the first, I am using keras.layers.concatenate while in the other, I am not using any keras.layers.concatenate.
I am posting the codes as well as the models for each case as follows.
Case-1 code
import keras
from keras.layers import Input, LSTM, Dense
from keras.models import Model
from keras.models import Sequential
from keras.layers import Dense
from keras.utils.vis_utils import plot_model
tweet_a = Input(shape=(280, 256))
tweet_b = Input(shape=(280, 256))
# This layer can take as input a matrix
# and will return a vector of size 64
shared_lstm = LSTM(64)
# When we reuse the same layer instance
# multiple times, the weights of the layer
# are also being reused
# (it is effectively *the same* layer)
encoded_a = shared_lstm(tweet_a)
encoded_b = shared_lstm(tweet_b)
# We can then concatenate the two vectors:
merged_vector = keras.layers.concatenate([encoded_a, encoded_b], axis=-1)
# And add a logistic regression on top
predictions1 = Dense(1, activation='sigmoid')(merged_vector)
predictions2 = Dense(1, activation='sigmoid')(merged_vector)
# We define a trainable model linking the
# tweet inputs to the predictions
model = Model(inputs=[tweet_a, tweet_b], outputs=[predictions1, predictions2])
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
Case-1 Model
Case-2 code
import keras
from keras.layers import Input, LSTM, Dense
from keras.models import Model
from keras.models import Sequential
from keras.layers import Dense
from keras.utils.vis_utils import plot_model
tweet_a = Input(shape=(280, 256))
tweet_b = Input(shape=(280, 256))
# This layer can take as input a matrix
# and will return a vector of size 64
shared_lstm = LSTM(64)
# When we reuse the same layer instance
# multiple times, the weights of the layer
# are also being reused
# (it is effectively *the same* layer)
encoded_a = shared_lstm(tweet_a)
encoded_b = shared_lstm(tweet_b)
# And add a logistic regression on top
predictions1 = Dense(1, activation='sigmoid')(encoded_a )
predictions2 = Dense(1, activation='sigmoid')(encoded_b)
# We define a trainable model linking the
# tweet inputs to the predictions
model = Model(inputs=[tweet_a, tweet_b], outputs=[predictions1, predictions2])
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])
Case-2 Model
In both cases, the LSTMlayer is shared only. In case-1, we have keras.layers.concatenate but in case-2, we don't have any keras.layers.concatenate.
My question is, which one is multitasking, case-1 or case-2? Morover, what is the function of keras.layers.concatenate in case-1?
Both are multi-task models, as this only depends if there are multiple outputs with one task associated to each output.
The difference is that your first model explicitly concatenates features produced by the shared layer, so both output tasks can consider information from both inputs. The second model only has connections from one input directly to one of the outputs, without considering the other input. The only link between models here is that they share the LSTM weights.
I've been following Towards Data Science's tutorial about word2vec and skip-gram models, but I stumbled upon a problem that I cannot solve, despite searching about it for hours and trying a lot of unsuccessful solutions.
https://towardsdatascience.com/understanding-feature-engineering-part-4-deep-learning-methods-for-text-data-96c44370bbfa
The step that it shows you how to build the skip-gram model architecture seems deprecated because of the use of the Merge layer from keras.layers.
I've seem many discussions about it, and the majority of answers was the you need to use the Functional API of Keras to merge layers now. But the problem is, I'm a total beginner in Keras and have no idea how to translate my code from Sequential to Functional, here's the code that the author used (and I copied):
from keras.layers import Merge
from keras.layers.core import Dense, Reshape
from keras.layers.embeddings import Embedding
from keras.models import Sequential
# build skip-gram architecture
word_model = Sequential()
word_model.add(Embedding(vocab_size, embed_size,
embeddings_initializer="glorot_uniform",
input_length=1))
word_model.add(Reshape((embed_size, )))
context_model = Sequential()
context_model.add(Embedding(vocab_size, embed_size,
embeddings_initializer="glorot_uniform",
input_length=1))
context_model.add(Reshape((embed_size,)))
model = Sequential()
model.add(Merge([word_model, context_model], mode="dot"))
model.add(Dense(1, kernel_initializer="glorot_uniform", activation="sigmoid"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
# view model summary
print(model.summary())
# visualize model structure
from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot
SVG(model_to_dot(model, show_shapes=True, show_layer_names=False,
rankdir='TB').create(prog='dot', format='svg'))
And when I run the block, the following error is shown:
ImportError Traceback (most recent call last)
<ipython-input-79-80d604373468> in <module>()
----> 1 from keras.layers import Merge
2 from keras.layers.core import Dense, Reshape
3 from keras.layers.embeddings import Embedding
4 from keras.models import Sequential
5
ImportError: cannot import name 'Merge'
What I'm asking here is some guidance on how to transform this Sequential into a Functional API structure.
This did indeed change. For a dot product, you can now use the dot layer:
from keras.layers import dot
...
dot_product = dot([target, context], axes=1, normalize=False)
...
You have to set the axis parameter according to your data, of course. If you set normalize=True, this gives the cosine proximity. For more information, see the documentation.
To learn about the functional API to Keras, there is a good guide to the functional API in the documentation. It's not difficult to switch if you already understand the sequential API.
Merge seems deprecated so Instead of Merge use Dot directly on embedding (and not with models). Use below code.
from keras.layers import Input
from keras.models import Model
from keras.layers.embeddings import Embedding
from keras.layers.core import Dense, Reshape
from keras.layers import dot
input_target = Input((1,))
input_context = Input((1,))
embedding = Embedding(vocab_size, embed_size, input_length=1, name='embedding')
word_embedding = embedding(input_target)
word_embedding = Reshape((embed_size, 1))(word_embedding)
context_embedding = embedding(input_context)
context_embedding = Reshape((embed_size, 1))(context_embedding)
# now perform the dot product operation
dot_product = dot([word_embedding, context_embedding], axes=1)
dot_product = Reshape((1,))(dot_product)
# add the sigmoid output layer
output = Dense(1, activation='sigmoid')(dot_product)
model = Model(input=[input_target, input_context], output=output)
model.compile(loss='mean_squared_error', optimizer='rmsprop')
# view model summary
print(model.summary())
# visualize model structure
from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot
SVG(model_to_dot(model, show_shapes=True, show_layer_names=False,
rankdir='TB').create(prog='dot', format='svg'))
I build RNN using keras but when I want to change time steps to different size I get an error and I can't get it done
here is my example for dummy data
from numpy import array
import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout
from sklearn.preprocessing import MinMaxScaler
from keras import optimizers
X=array(
[
[#first sample
[0,2],[1,2],[2,2] # three time steps and 2 features
]
,
[# sample 2
[0,2],[1,2],[2,2] # three time steps and 2 features
]
,
[# sample 3
[7,2], [9,2], [4,2] # three time steps and 2 features
]
,
[# sample 4
[2,2], [5,2], [4,2],[7,9] # four steps and 2 features
]
]
)
Y=np.array([1,2,3,4])
model = Sequential()
model.add(LSTM(8, input_shape=(None, 2),return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(32,return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(128,return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(58, activation='softmax'))
optimize=optimizers.RMSprop(lr=0.001, rho=0.9, epsilon=None, decay=0.0)
model.compile(optimize,loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.summary()
model.fit(X,Y,batch_size=1,epochs=50,shuffle=True,verbose=2)
as you can see from the code I have 4 sequence and 2 features in each sequence.
in the last sequence I have 4 time steps instead of 3 and here is the problem if I change it to 3 time steps the code works correctly,
but I want it to work on different time steps how can I achieve that without use padding or masking.
I did read different topics describe different solutions but I can't get it work in the above example
when I try to run above code I got error
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (4, 1)
Your X is not a valid array. A numpy array must be rectangular and not jagged. Keras can only take in valid numpy arrays as inputs. You have two choices:
Feed the samples into your model 1 sample at a time. I.e. use a batch_size of 1, use fit_on_batch or fit_generator rather than just fit. Note that this will remove all vectorization related speed-optimizations and will slow your training down to a crawl if you have a lot of data.
Pad your training set so that they all are of the same time dimension. 0-padding shouldn't really affect the performance of your model. This is the recommended method.
See this thread for more details.
I have a CPU with 20 cores and I am trying to use all the cores to fit a model. I set a tf session with intra_op_parallelism_threads=20 and called model.fit within the same tf session.
The python process utilizes 2000% CPU (as stated by top). However, when comparing the following code with single core configuration (intra_op_parallelism_threads=1) I get the same learning rate.
from keras.layers import Dense, Activation, Dropout
from keras.layers import Input, Conv1D
import numpy as np
from keras.layers.merge import concatenate
from keras.models import Model
import tensorflow as tf
from keras.backend import tensorflow_backend as K
with tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=20)) as sess:
K.set_session(sess)
size=20
batch_size=16
def xor_data_generator():
while True:
data1 = np.random.choice([0, 1], size=(batch_size, size,size))
data2 = np.random.choice([0, 1], size=(batch_size, size,size))
labels = np.bitwise_xor(data1, data2)
yield ([data1, data2], np.array(labels))
a = Input(shape=(size,size))
b = Input(shape=(size,size))
merged = concatenate([a, b])
hidden = Dense(2*size)(merged)
conv1 = Conv1D(filters=size*16, kernel_size=1, activation='relu')(hidden)
hidden = Dropout(0.1)(conv1)
outputs = Dense(size, activation='sigmoid')(hidden)
model = Model(inputs=[a, b], outputs=outputs)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit_generator(xor_data_generator(), steps_per_epoch=batch_size, epochs=10000)
Note that I can't use multi_gpu_model, because I have a system with only 20 CPU cores.
How can I distribute model.fit_generator(xor_data_generator(), steps_per_epoch=batch_size, epochs=10000) on different cores simultaneously?
Have a look at Keras' Sequence object to write your custom generator. It is the underlying object of the ImageDataGenerator to yield image data. The docs contain boilerplate code that you can adapt. If you use it, you can set the use_multiprocessing argument of fit.generator() to True. See also this answer.