I am working on a project in TensorFlow that performs operations on already-trained machine learning models. Following the tutorial TFLearn Quickstart, I built a deep neural network that predicts survival from the Titanic Dataset. I would like to use the TFLearn model in the same way that I would use a TensorFlow model.
The TFLearn docs homepage says
Full transparency over Tensorflow. All functions are built over tensors and can be used independently of TFLearn
This makes me think that I would be able to pass tensors as inputs, etc. to the the TFLearn model.
# Build neural network
net = tflearn.input_data(shape=[None, 6])
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net)
# Define model
model = tflearn.DNN(net)
# Start training (apply gradient descent algorithm)
model.fit(data, labels, n_epoch = 10, batch_size = 16, show_metric = False)
test = preprocess([[3, 'Jack Dawson', 'male', 19, 0, 0, 'N/A', 5.0000]], to_ignore)
# Make into a tensor
testTF = [tf.constant(i) for i in test]
# Pass the tensor into the predictor
print(model.predict([testTF]))
At present, when I pass a tensor into the model I am greeted with ValueError: setting an array element with a sequence.
Specifically, how can you pass tensors into a TFLearn model?
Generally, what limits are placed on how I can use tensors on a TFLearn model?
I don't know if you're still looking for an answer to your problem, but I think the issue is on your very last line:
print(model.predict([testTF]))
Try this instead:
print(model.predict(testTF))
I think that you nested a list inside another list. This isn't a TFlearn issue per se. Hope that helps.
Related
I have a CNN model trained using EfficientNetB6.
My task is to extract the features of this trained model by removing the last dense layer and then using those weights to train a boosting model.
i did this using Pytorch earlier and was able to extract the weights from the layers i was interested and predicted on my validation set and then boosted.
I am doing this now in tensorflow but currently stuck.
Below is my model structure and I have tried using the code on the website but did not had any luck.
I want to remove the last dense layer and predict on the validation set using the remaining layers.
I tried using :
layer_name = 'efficientnet-b6'
intermediate_layer_model = tf.keras.Model(inputs = model.input, outputs = model.get_layer(layer_name).output)
but i get an error "
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 760, 760, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []"
Any way to resolve this?
Sorry my bad.
I simply added a GlobalAveragePooling2D layer after the efficientnet layer and i am able to extract the weights and continue :)
just for reference:
def build_model(dim=CFG['net_size'], ef=0):
inp = tf.keras.layers.Input(shape=(dim,dim,3))
base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)
x = base(inp)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(1,activation='sigmoid')(x)
model = tf.keras.Model(inputs=inp,outputs=x)
opt = tf.keras.optimizers.Adam(learning_rate=0.001)
loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05)
model.compile(optimizer=CFG['optimizer'],loss=loss,metrics=[tf.keras.metrics.AUC(name='auc')])
print(model.summary())
return model
I'm kind of a newbie to tensorflow and building neural networks.
I'm trying to make a neural network with the tf.keras api that will take a single input, and give 3 outputs. Here is my code:
import os
import tensorflow as tf
from tensorflow import keras
import numpy as np
train_times = np.array([[1],[2],[3],[4],[5],[6],[7],[8]])
train_sensors = np.array([[0.1,0.15,0.2],[0.25,0.3,0.35],[0.4,0.45,0.5],[0.55,0.6,0.65],[0.7,0.75,0.8],[0.85,0.9,0.95],[0.05,0.33,0.56],[0.8,0.35,0.9]])
test_times = np.array([[1],[2],[3],[4],[5],[6],[7],[8]])
test_sensors = np.array([[0.1,0.15,0.2],[0.25,0.3,0.35],[0.4,0.45,0.5],[0.55,0.6,0.65],[0.7,0.75,0.8],[0.85,0.9,0.95],[0.05,0.33,0.56],[0.8,0.35,0.9]])
print(train_sensors[0].shape)
def create_model():
model = tf.keras.models.Sequential([
keras.layers.Dense(5, activation=tf.nn.relu, input_shape=(1,), name="Input"),
keras.layers.Dense(10,activation=tf.nn.relu, name="Middle"),
keras.layers.Dropout(0.2),
keras.layers.Dense(3, activation=tf.nn.softmax, name="Out")
])
model.compile(optimizer=tf.keras.optimizers.Adam(), loss=tf.keras.losses.sparse_categorical_crossentropy,
metrics=['accuracy'])
return model
model = create_model()
model.summary()
checkpoint_path = "sensor_predict.ckpt"
checkpoint_dir = os.path.dirname(checkpoint_path)
cp_callback = tf.keras.callbacks.ModelCheckpoint(checkpoint_path,save_weights_only=True,verbose=1)
model.fit(x=train_times, y=train_sensors,epochs = 10,validation_data = (test_sensors, test_times), callbacks = [cp_callback])
I have specified that the last layer should have three outputs, but I get this error every time I run it:
ValueError: Error when checking target: expected Out to have shape (1,) but got array with shape (3,)
I can't figure out why it seems to think I want a single output from the network.
NOTE: The dataset I am using is not what I will actually use. I'm just trying to get a functional network, and then I'll generate the data later.
Your loss function (tf.keras.losses.sparse_categorical_crossentropy) is expecting the training vector to be one hot encoded. Change it to tf.keras.losses.mse, for example, and I think it will work.
See tensorflow docs for the definition.
I am trying to do transfer learning in Keras + Tensorflow on a selected subset of Places-205 dataset, containing only 27 categories. I am using InceptionV3, DenseNet121 and ResNet50, pre-trained on ImageNet, and add a couple of extra layers to adapt to my classes. If the model is ResNet, I add Flatten + Dense for classfication, and if it is DenseNet or Inceptionv3, I add Global Avg Pool + Dense (relu) + Dense (classification).
This is the code snippet:
x = base_model.output
if FLAGS.model in 'resnet50':
x = Flatten(name="flatten")(x)
else:
x = GlobalAveragePooling2D()(x)
# Let's add a fully-connected layer
x = Dense(1024, activation = 'relu')(x)
# And a logistic layer
predictions = Dense(classes, activation = 'softmax')(x)
For DenseNet and Inceptionv3 the training is ok, and the validation accuracy hits 70%, but for ResNet the validation accuracy stays fixed at 0.0369/0.037 (which is 1/27, my number of classes). It seems like it always predicts one class, but it's weird because its training progresses ok and the unspecific model code is exactly the same as for DenseNet and InceptionV3, which do work as expected.
Do you have any idea why it happens?
Thanks a lot!
I had a similar issue as you #Ciprian Andrei Focsaneanu, and what I have found to have worked was to make the previous layers (before the fully connected layers) trainable, as the filters/features of the ResNet50 were not suitable for my application.
Strangely enough, I also trained the VGG16 models, which was initially on the same images (imagenet) but its filters worked for my application, but I digress.
Here's the link to a page that inspired me to do this: https://datascience.stackexchange.com/questions/16840/multi-class-neural-net-always-predicting-1-class-after-optimization
Hope this helps!
I am new to machine learning and Tensorflow and want to do a simple 2-dimensional classification with data, that cannot be linear separated.
On the left side, you can see the training data for the model.
The right side shows, what the trained model predicts.
As of now I am overfitting my model, so every possible input is fed to the model.
My expected result would be a very high accurancy as the model already 'knows' each answer.
Unfortunately the Deep Neural Network I am using is only able to separate by a linear divider, which doesn't fit my data.
This is how I train my Model:
def testDNN(data):
"""
* data is a list of tuples (x, y, b),
* where (x, y) is the input vector and b is the expected output
"""
# Build neural network
net = tflearn.input_data(shape=[None, 2])
net = tflearn.fully_connected(net, 100)
net = tflearn.fully_connected(net, 100)
net = tflearn.fully_connected(net, 100)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net)
# Define model
model = tflearn.DNN(net)
# check if we already have a trained model
# Start training (apply gradient descent algorithm)
model.fit(
[(x,y) for (x,y,b) in data],
[([1, 0] if b else [0, 1]) for (x,y,b) in data],
n_epoch=2, show_metric=True)
return lambda x,y: model.predict([[x, y]])[0][0]
Most of it is taken from the examples of tflearn, so I do not exactly understand, what every line does.
You need an activation function in your network for non-linearity. An activation function is the way for a neural network to fit non-linear function. Tflearn by default uses a linear activation, you could change this to 'sigmoid' and see if the results improve.
I started using a very basic Deep Belief Network in Node.js but it wasn't fast enough. Essentially it was using a X and Y where each is an array of arrays; X is the data to train and Y is the result.
So I would feed it something like var x=[[1,2,3], [1,3,2]] etc. etc. and y=[[1,0], [1,0]]. Then I would give some data such as [2,3,1] and it would predict the y.
I'm lost on how to do this in tfslearn. I can learn on my own but I've hit a point where I'm not sure what to even Google.
I can get the examples working if it's just a single array.
Every time I try using an array of arrays I get:
cannot feed value of shape
I was setting the input shape incorrectly for my data set. This helped a lot: http://tflearn.org/tutorials/quickstart.html
# Data loading and preprocessing
# Building deep neural network
net = tflearn.input_data(shape=[None, 4])
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 1, activation='softmax')
net = tflearn.regression(net)
# Training
model = tflearn.DNN(net)
model.fit(X, Y, n_epoch=10, batch_size=16, show_metric=True)