I'm playing a little with deep learning and Keras has been my choice due to its simplicity.
I've built a simple multilayer perceptron model for binary classification and fitted it on input data (the same that I'm using for other ML models and which are working ok).
The Following picture displays the Model summary:
The first dense layer was defined as such:
model.add(Dense(18, input_dim=len(X_encoded.columns), activation = "relu", kernel_initializer="uniform"))
When I attempt to predict over a loop like so:
for vals in X_encoded.values:
print("Survives?", model.predict([vals], batch_size=1))
I get the following error:
ValueError: Error when checking input: expected dense_90_input to have shape (35,) but got array with shape (1,)
These are my variable sizes:
print("Shape of vals:", vals.shape, "Number of Columns and First Layer Dimension:", len(X_encoded.columns))
Result:
Shape of vals: (35,) Number of Columns and First Layer Dimension: 35
As you can see, these match in size which is the expected input.
What is going on? When I pass the entire dataframe "predict" it works correctly, but not when I pass a single value...
You need an array, not a list. You only use a list for multiple input tensors.
model.predict(np.array([vals]), batch_size=1)
But why not:
model.predict(X_encoded.values, batch_size=1)
Related
I am trying to implement a learning-to-rank model using a pre-trained BERT available on tensorflow hub. I am using a variation of ListNet loss function, which requires each training instance to be a list of several ranked documents in relation to a query. I need the model to be able to accept data in a shape (batch_size, list_size, sentence_length), where the model loops over the 'list_size' axis in each training instance, returns the ranks and passes them to the loss function. In a simple model that only consists of dense layers, this is easily done by augmenting the dimensions of the input layer. For example:
from tensorflow.keras.layers import Dense, Input
from tensorflow.keras import Model
input = Input([6,10])
x = Dense(20,activation='relu')(input)
output = Dense(1, activation='sigmoid')(x)
model = Model(inputs=input, outputs=output)
...now the model will perform 6 forward passes over vectors of length 10 before calculating the loss and updating gradients.
I am trying to do the same with the BERT model and its preprocessing layer:
import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_text as text
bert_preprocess_model = hub.KerasLayer('https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-4_H-512_A-8/1')
bert_model = hub.KerasLayer('https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3')
text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text')
processed_input = bert_preprocess_model(text_input)
output = bert_model(processed_input)
model = tf.keras.Model(text_input, output)
But when I try to change the shape of 'text_input' to, say, (6), or meddle with it in any way really, it always results in the same type of error:
ValueError: Could not find matching function to call loaded from the SavedModel. Got:
Positional arguments (3 total):
* Tensor("inputs:0", shape=(None, 6), dtype=string)
* False
* None
Keyword arguments: {}
Expected these arguments to match one of the following 4 option(s):
Option 1:
Positional arguments (3 total):
* TensorSpec(shape=(None,), dtype=tf.string, name='sentences')
* False
* None
Keyword arguments: {}
....
As per https://www.tensorflow.org/hub/api_docs/python/hub/KerasLayer, it seems like you can configure the input shape of hub.KerasLayer via tf.keras.layers.InputSpec. In my case, I guess it would be something like this:
bert_preprocess_model.input_spec = tf.keras.layers.InputSpec(ndim=2)
bert_model.input_spec = tf.keras.layers.InputSpec(ndim=2)
When I run the above code, the attributes indeed get changed, but when trying to build the model, the same exact error appears.
Is there any way to easily resolve this without the necessity to create a custom training loop?
Suppose you have a batch of B examples, each with exactly N text strings, which makes a 2-dimensional Tensor of shape [B, N]. With tf.reshape(), you can turn that into a 1-dimensional tensor of shape [B*N], send it through BERT (which preserves the order of inputs) and then reshape it back to [B,N]. (There's also tf.keras.layers.Reshape, but that hides the batch dimension from you.)
If it's not exactly N text strings each time, you'll have to do some bookkeeping on the side (e.g., store inputs in a tf.RaggedTensor, run BERT on its .values, and construct a new RaggedTensor with the same .row_splits from the result.)
I have a model defined in Keras like so:
model.add(Dense(40, input_shape=(4,), activation='relu'))
#more layers
For some reason, I cannot fit the data, despite trying many different shapes. I have a states variable like so:
[array([[0.00076 , 0.001208 , 0.00127778, 0.22222222]]), array([[0.00074 , 0.00186 , 0.00116667, 0.22222222]])]
Each entry has 4 inputs. However, when I try to run:
model.fit(states, Y, epochs=1)
I keep getting this error:
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 100 arrays:
How do I reshape this array to work with my setup?
I am using Keras for building Conv Net for the first time. My layers are as follows:
layers = [
Conv2D(8,kernel_size=(4,4),padding='same',input_shape=( 200, 180,3),kernel_initializer="glorot_normal",data_format="channels_first"),
Activation("relu"),
MaxPooling2D(pool_size=(8,8),padding='same',data_format='channels_first'),
Conv2D(16,(2,2),padding='same',kernel_initializer="glorot_normal"),
Activation("relu"),
MaxPooling2D(pool_size=(4,4),padding='same',data_format='channels_first'),
Conv2D(4,(3,3),padding='same',kernel_initializer="glorot_normal"),
Activation("relu"),
MaxPooling2D(pool_size=(2,2),padding='same',data_format='channels_first'),
Flatten(),
Dense(2,input_shape=(48,)),
Softmax(axis=-1)
]
#Edit, here is the part for compiling the model and fitting it
model = Sequential(layers)
model.compile(optimizer="adam",loss="sparse_categorical_crossentropy"
metrics=["accuracy"])
trainHistory = model.fit(x=X_train,y=Y_train,batch_size=3,epochs=1000)
My labels array is of shape (,2). But when I try to use fit on the model, it gives me the error that softmax_1 expected to have shape (1,). But I have clearly mentioned units of Dense as 2 and softmax returns output of the same dimension as the input.
So where did the 1, came from? I tried to use dummy label array of 1 dimension and it runs. So what am I doing wrong? How do I use 2 dimensional array that I have?
The problem is that you are using sparse_categorical_crossentropy as the loss function. This loss function is used when the given labels (i.e. Y_train) are encoded as integers (i.e. 0, 1, 2, ...). However, If the labels are one-hot encoded, which seems to be the case in your code, you need to use categorical_crossentropy as the loss function instead.
I'm new to Keras and am trying to test out a model I've just trained.
I'm using Tensorflow backend and Python 3.
However, the shape my input has and the shape Keras says it has in an error are completely different. Here's my code:
testnote = np.zeros((3,))
testnote[0] = 70
testnote[1] = 70
print(testnote.shape)
pred = model.predict(testnote)
print(pred)
My consistent output is "(3,)" for the shape of testnote and then an error for my predict line: "ValueError: Error when checking input: expected dense_1_input to have shape (3,) but got array with shape (1,)"
How is it that Keras reads testnote as having shape (1,) when I've just confirmed that the shape is (3,)? Is it using some sort of different standard for what "shape" means? I've tried reshaping and adding brackets and a bunch of other things, but I don't really know what the problem is.
For additional context, the model takes in an array with 3 scalar input (representing pitch, velocity, and instrument class) and outputs an array with 1025 scalar outputs. I am carefully not using the word "dimension" since I think this is where I'm getting confused, and technically both are only 1 dimension. I'm sure there are many problems with my model which I will have to fix following this. However, I'd like to just get this prediction function working so I can understand what my output looks like.
Thanks in advance for any help.
A Keras Model implicitly expects that your data (passed as a np array) has a dimension for the batch size. Currently, your model is interpreting testnote as being 3 examples of shape 1. Try adding the batch dimension to 'testnote' as follows:
testnote = testnote.reshape(1,-1)
This will reshape testnote to shape (1, 3), so that you explicitly define the batch size to be 1.
I'm trying to feed sentences in which each world has word2vec representation.
How can I do it in tensorflow seq2seq models?
Suppose the variable
enc_inp = [tf.placeholder(tf.int32, shape=(None,10), name="inp%i" % t)
for t in range(seq_length)]
Which has dimensions [num_of_observations or batch_size x word_vec_representation x sentense_lenght].
when I pass it to embedding_rnn_seq2seq
decode_outputs, decode_state = seq2seq.embedding_rnn_seq2seq(
enc_inp, dec_inp, stacked_lstm,
seq_length, seq_length, embedding_dim)
error occurs
ValueError: Linear is expecting 2D arguments: [[None, 10, 50], [None, 50]]
Also there is a more complex problem
How can i pas as input a vector, not a scalar to first cell of my RNN?
By now it looks like (when we are about any sequence)
get first value of sequence (scalar)
compute First layer RNN First layer embedding cell output
compute First layer RNN Second layer embedding cell output
etc
But this is needed:
Get first value of sequence (vector)
compute First layer RNN First layer cell output (as ordinary computing simple perceptron when Input is a vector)
compute First layer RNN Second layer embedding cell output (as ordinary computing simple perceptron when Input is a vector)
The main point is that:
seq2seq make inside themself word embedding.
Here is reddit question and answer
Also, if smbd wants to use pretrained Word2Vec there are ways to do it,
see:
stackoverflow 1
stackoverflow 2
So this can be used no only for word embedding