I am trying to make a movie recommendation model using keras:
import pandas as pd
from sklearn.model_selection import train_test_split
import keras
from keras.layers import Input, Embedding, Dot, Flatten
rating = pd.read_csv("./ratings.csv",usecols=[0,1,2])
users = len(rating.userId.unique())
movies = len(rating.movieId.unique())
embed_size = 3
train, test = train_test_split(rating, test_size=0.2)
movie_input = Input(shape=[1], name="movie_in")
movie_embed = Embedding(movies, embed_size, name="movie_embed")(movie_input)
movie_vector = Flatten(name="flatten_movies")(movie_embed)
user_input = Input(shape=[1], name="user_in")
user_embed = Embedding(users, embed_size, name="user_embed")(user_input)
user_vector = Flatten(name="flatten_users")(user_embed)
prod = Dot(axes=-1, name="dot-product")([movie_vector, user_vector])
model = keras.Model(inputs=[user_input, movie_input], outputs=prod)
model.compile(optimizer='adam', loss='mse')
model.fit(x=[train.userId, train.movieId], y=train.rating,epochs=10,
verbose=0)
When I try to train the model I am getting the following error:
tensorflow.python.framework.errors_impl.InvalidArgumentError:
indices[15,0]= 7438 is not in [0, 5000)
[[{{node movie_embed/embedding_lookup}} = GatherV2[Taxis=DT_INT32,
Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:#training/Adam/Assign_2"],
_device="/job:localhost/replica:0/task:0/device:CPU:0"]
(movie_embed/embeddings/read, movie_embed/Cast,
training/Adam/gradients/movie_embed/embedding_lookup_grad/concat/axis)]]
But most of the online tutorials use the same code, it works properly for them.
Your movie_embed embedding layer (basically a lookup table) has 5000 rows, so it expects integers between 0 and 5000 as input. You are giving it 7438 as input, which causes the error. There are probably 5000 unique values in rating.movieId, but apparently also values outside of the interval [0, 5000). You'll need to map your train.userId integers onto this interval to make it work.
Related
I'm trying to follow the tutorial given here.
This tutorial trains a Keras model using a genetic algorithm, with the PyGAD package. I'm interested in the binary classification case. My input matrix is of dimension 10000x20. Hence, I've created the following model using Keras:
input_layer = tensorflow.keras.layers.Input(20)
dense_layer1 = tensorflow.keras.layers.Dense(500, activation="relu")(input_layer)
dense_layer2 = tensorflow.keras.layers.Dense(500, activation="relu")(dense_layer1)
output_layer = tensorflow.keras.layers.Dense(1, activation="softmax")(dense_layer2)
model = tensorflow.keras.Model(inputs=input_layer, outputs=output_layer)
keras_ga = pygad.kerasga.KerasGA(model=model,
num_solutions=10)
However, when I go to run the algorithm, using ga_instance.run(), I get the error:
ValueError: Shapes (10000,) and (10000, 1) are incompatible
I can't figure out why I'm getting this error? I want my Keras model to have 2 hidden layers, each with 500 hidden nodes and 1 output node.
I think the problem is related to how each output is represented in the array. if you have a single output for 10000 instances, then this is an example of preparing the data that works with PyGAD. Its shape is (1000, 1).
numpy.random.uniform(0, 1, (1000, 1))
Here is a code that works but for a simple network architecture because, based on the fitness function you used, the fitness sometimes is NaN.
As I do not have the same data you used, I generated the input/output data randomly.
import tensorflow.keras
import pygad.kerasga
import numpy
import pygad
def fitness_func(solution, sol_idx):
global data_inputs, data_outputs, keras_ga, model
model_weights_matrix = pygad.kerasga.model_weights_as_matrix(model=model,
weights_vector=solution)
model.set_weights(weights=model_weights_matrix)
predictions = model.predict(data_inputs)
cce = tensorflow.keras.losses.CategoricalCrossentropy()
solution_fitness = 1.0 / (cce(data_outputs, predictions).numpy() + 0.00000001)
# print("solution_fitness", cce(data_outputs, predictions).numpy(), solution_fitness)
return solution_fitness
def callback_generation(ga_instance):
print("Generation = {generation}".format(generation=ga_instance.generations_completed))
print("Fitness = {fitness}".format(fitness=ga_instance.best_solution(ga_instance.last_generation_fitness)[1]))
data_inputs = numpy.random.uniform(0, 1, (1000, 20))
data_outputs = numpy.random.uniform(0, 1, (1000, 1))
# create model
from tensorflow.keras.layers import Dense, Dropout
l1_rate=1e-6
l2_rate = 1e-6
input_layer = tensorflow.keras.layers.InputLayer(20)
dense_layer1 = tensorflow.keras.layers.Dense(10, activation="relu",kernel_regularizer=tensorflow.keras.regularizers.l1_l2(l1=l1_rate, l2=l2_rate))
output_layer = tensorflow.keras.layers.Dense(1, activation="sigmoid")
model = tensorflow.keras.Sequential()
model.add(input_layer)
model.add(dense_layer1)
model.add(Dropout(0.2))
model.add(output_layer)
keras_ga = pygad.kerasga.KerasGA(model=model,
num_solutions=10)
# Run pygad
num_generations = 30
num_parents_mating = 5
initial_population = keras_ga.population_weights
ga_instance = pygad.GA(num_generations=num_generations,
num_parents_mating=num_parents_mating,
initial_population=initial_population,
fitness_func=fitness_func,
on_generation=callback_generation)
ga_instance.run()
Thanks for using PyGAD!
I'm trying to put a collection of images through a neural network, but I can't figure out how to get a large collection of images to go into a tensorflow model, as trying to convert the collection into a numpy array causes a memory error.
I should note that I am very new to tensorflow.
import numpy as np
from skimage.io import imread_collection
from tensorflow import keras
from tensorflow.keras import layers
def gen(arr):return(i.reshape(400*600*3) for i in arr) # Only used in Attempt2.
labelFile=open("lables_text_file.txt","r")
labels=labelFile.read()
labelFile.close()
labels=getTrain(labels)#Converts to a tuple containing the lables in order.
data = imread_collection("path_to_images/*.jpg", conserve_memory=True)
train=data[:-len(data)//4]
trainLabels=labels[:-len(data)//4]
test=data[-len(data)//4:]
testLabels=labels[-len(data)//4:]
#train = train.reshape(-1, 400*600*3) # Attempt1
#test = test.reshape(-1, 400*600*3) # Attempt1
#train = gen(train) # Attempt2
#test = gen(test) # Attempt2
trainLabels = keras.utils.to_categorical(trainLabels, 23)
testLabels = keras.utils.to_categorical(testLabels, 23)
model=keras.Sequential([keras.Input(shape=(400*600*3,)),
layers.Dense(600, name='hidden1', activation='relu'),
layers.Dense(400, name='hidden2', activation='relu'),
layers.Dense(46, name='hidden3', activation='relu'),
layers.Dense(23, activation="softmax")])
optimizer = keras.optimizers.Adam(learning_rate=0.0015)
model.compile(loss=keras.losses.CategoricalCrossentropy(), optimizer=optimizer, metrics=[keras.metrics.CategoricalAccuracy()])
model.fit(train,trainLabels,batch_size=128,epochs=8,validation_data=(test,testLabels), shuffle=True)
When I run the code as is, this is the result:
ValueError: Failed to find data adapter that can handle input: <class 'skimage.io.collection.ImageCollection'>, <class 'numpy.ndarray'>
When I try to use Attempt1, this is the result:
AttributeError: 'ImageCollection' object has no attribute 'reshape'
When I try to use Attempt2, this is the result:
ValueError: `y` argument is not supported when using python generator as input.
How can I put the data into `model.fit, such that it will successfully train the neural network?
I think I may have solved the problems.
Working code:
import numpy as np
from skimage.io import imread_collection
from tensorflow import keras
from tensorflow.keras import layers
def gen(arr,labels):return((arr[i].reshape(-1,400*600*3),labels[i].reshape(-1,23)) for i in range(len(arr)))
labelFile=open("lables_text_file.txt","r")
labels=labelFile.read()
labelFile.close()
labels=getTrain(labels)#Converts to a tuple containing the lables in order.
data = imread_collection("path_to_images/*.jpg", conserve_memory=True)
train=data[:-len(data)//4]
trainLabels=labels[:-len(data)//4]
test=data[-len(data)//4:]
testLabels=labels[-len(data)//4:]
#train = train.reshape(-1, 400*600*3) # Attempt1
#test = test.reshape(-1, 400*600*3) # Attempt1
trainLabels = keras.utils.to_categorical(trainLabels, 23)
testLabels = keras.utils.to_categorical(testLabels, 23)
train = gen(train,trainLabels) # Attempt2
test = gen(test,testLabels) # Attempt2
model=keras.Sequential([keras.Input(shape=(400*600*3,)),
layers.Dense(600, name='hidden1', activation='relu'),
layers.Dense(400, name='hidden2', activation='relu'),
layers.Dense(46, name='hidden3', activation='relu'),
layers.Dense(23, activation="softmax")])
optimizer = keras.optimizers.Adam(learning_rate=0.0015)
model.compile(loss=keras.losses.CategoricalCrossentropy(), optimizer=optimizer, metrics=[keras.metrics.CategoricalAccuracy()])
model.fit(train,None,batch_size=128,epochs=8,validation_data=(test,testLabels), shuffle=True)
The solution was to pass in a generator that returns two-tuples containing the input and label (instead of passing the labels in directly), but there were other problems that I may include in this answer if I get the time.
I know there are several questions about this here, but I haven't found one which fits exactly my problem.
I'm trying to fit an LSTM with data from Pandas DataFrames but getting confused about the format I have to provide them.
I created a small code snipped which shall show you what I try to do:
import pandas as pd, tensorflow as tf, random
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
targets = pd.DataFrame(index=pd.date_range(start='2019-01-01', periods=300, freq='D'))
targets['A'] = [random.random() for _ in range(len(targets))]
targets['B'] = [random.random() for _ in range(len(targets))]
features = pd.DataFrame(index=targets.index)
for i in range(len(features)) :
features[str(i)] = [random.random() for _ in range(len(features))]
model = Sequential()
model.add(LSTM(units=targets.shape[1], input_shape=features.shape))
model.compile(optimizer='adam', loss='mae')
model.fit(features, targets, batch_size=10, epochs=10)
this results to:
ValueError: Input 0 of layer sequential is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [10, 300]
which I expect relates to the dimensions of the features DataFrame provided. I guess that once fixed this the next error would mention the targets DataFrame.
As far as I understand, 'units' parameter of my first layer defines the output dimensionality of this model. The inputs have to have a 3D shape, but I don't know how to create them out of the 2D world of the Data Frames.
I hope you can help me understanding the reshape mechanism in Python and how to use them in combination with Pandas DataFrames. (I'm quite new to Python and came from R)
Thankls in advance
Lets looks at the few popular ways in LSTMs are used.
Many to Many
Example: You have a sentence (composed of words in sequence). Give these sequence of words you would like to predict the Parts of speech (POS) of each word.
So you have n words and you feed each word per timestep to the LSTM. Each LSTM timestep (also called LSTM unwrapping) will produce and output. The word is represented by a a set of features normally word embeddings. So the input to LSTM is of size bath_size X time_steps X features
Keras code:
inputs = keras.Input(shape=(10,3))
lstm = keras.layers.LSTM(8, input_shape = (10, 3), return_sequences = True)(inputs)
outputs = keras.layers.TimeDistributed(keras.layers.Dense(5, activation='softmax'))(lstm)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss='categorical_crossentropy', optimizer='adam')
X = np.random.randn(4,10,3)
y = np.random.randint(0,2, size=(4,10,5))
model.fit(X, y, epochs=2)
print (model.predict(X).shape)
Many to One
Example: Again you have a sentence (composed of words in sequence). Give these sequence of words you would like to predict sentiment of the sentence if it is positive or negative.
Keras code
inputs = keras.Input(shape=(10,3))
lstm = keras.layers.LSTM(8, input_shape = (10, 3), return_sequences = False)(inputs)
outputs =keras.layers.Dense(5, activation='softmax')(lstm)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss='categorical_crossentropy', optimizer='adam')
X = np.random.randn(4,10,3)
y = np.random.randint(0,2, size=(4,5))
model.fit(X, y, epochs=2)
print (model.predict(X).shape)
Many to multi-headed
Example: You have a sentence (composed of words in sequence). Give these sequence of words you would like to predict sentiment of the sentence as well the author of the sentence.
This is multi-headed model where one head will predict the sentiment and another head will predict the author. Both the heads share the same LSTM backbone.
Keras code
inputs = keras.Input(shape=(10,3))
lstm = keras.layers.LSTM(8, input_shape = (10, 3), return_sequences = False)(inputs)
output_A = keras.layers.Dense(5, activation='softmax')(lstm)
output_B = keras.layers.Dense(5, activation='softmax')(lstm)
model = keras.Model(inputs=inputs, outputs=[output_A, output_B])
model.compile(loss='categorical_crossentropy', optimizer='adam')
X = np.random.randn(4,10,3)
y_A = np.random.randint(0,2, size=(4,5))
y_B = np.random.randint(0,2, size=(4,5))
model.fit(X, [y_A, y_B], epochs=2)
y_hat_A, y_hat_B = model.predict(X)
print (y_hat_A.shape, y_hat_B.shape)
What you are looking for is Many to Multi head model where your predictions for A will be made by one head and another head will make predictions for B
The input data for the LSTM has to be 3D.
If you print the shapes of your DataFrames you get:
targets : (300, 2)
features : (300, 300)
The input data has to be reshaped into (samples, time steps, features). This means that targets and features must have the same shape.
You need to set a number of time steps for your problem, in other words, how many samples will be used to make a prediction.
For example, if you have 300 days and 2 features the time step can be 3. So that three days will be used to make one prediction (you can choose this arbitrarily). Here is the code for reshaping your data (with a few more changes):
import pandas as pd
import numpy as np
import tensorflow as tf
import random
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
data = pd.DataFrame(index=pd.date_range(start='2019-01-01', periods=300, freq='D'))
data['A'] = [random.random() for _ in range(len(data))]
data['B'] = [random.random() for _ in range(len(data))]
# Choose the time_step size.
time_steps = 3
# Use numpy for the 3D array as it is easier to handle.
data = np.array(data)
def make_x_y(ts, data):
"""
Parameters
ts : int
data : numpy array
This function creates two arrays, x and y.
x is the input data and y is the target data.
"""
x, y = [], []
offset = 0
for i in data:
if offset < len(data)-ts:
x.append(data[offset:ts+offset])
y.append(data[ts+offset])
offset += 1
return np.array(x), np.array(y)
x, y = make_x_y(time_steps, data)
print(x.shape, y.shape)
nodes = 100 # This is the width of the network.
out_size = 2 # Number of outputs produced by the network. Same size as features.
model = Sequential()
model.add(LSTM(units=nodes, input_shape=(x.shape[1], x.shape[2])))
model.add(Dense(out_size)) # For the output a Dense (fully connected) layer is used.
model.compile(optimizer='adam', loss='mae')
model.fit(x, y, batch_size=10, epochs=10)
Well, just to finalize this issue I would like to provide one solution I have meanwhile worked on. The class TimeseriesGenerator in tf.keras.... enabled me quite easy to provide the data in the right shape to an LSTM model
from keras.preprocessing.sequence import TimeseriesGenerator
import numpy as np
window_size = 7
batch_size = 8
sampling_rate = 1
train_gen = TimeseriesGenerator(X_train.values, y_train.values,
length=window_size, sampling_rate=sampling_rate,
batch_size=batch_size)
valid_gen = TimeseriesGenerator(X_valid.values, y_valid.values,
length=window_size, sampling_rate=sampling_rate,
batch_size=batch_size)
test_gen = TimeseriesGenerator(X_test.values, y_test.values,
length=window_size, sampling_rate=sampling_rate,
batch_size=batch_size)
There are many other ways on implementing generators e.g. using the more_itertools which provides the function windowed, or making use of tensorflow.Dataset and its function window.
For me the TimeseriesGenerator was sufficient to feed the tests I did.
In case you would like to see an example modeling the DAX based on some stocks I'm sharing a notebook on Github.
I try creating a neural network, having two inputs of a particular size (here four) each and one output of the same size size (so also four). Unfortunately, I always get this error when running my code:
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not
the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays:
[array([[[-1.07920336, 1.16782929, 1.40131554, -0.30052492],
[-0.50067655, 0.54517916, -0.87033621, -0.22922157]],
[[-0.53766128, -0.03527806, -0.14637072, 2.32319071],
[ 0...
I think, the problem lies in the fact, that once I pass the data for training, the input shape is either incorrect or I have a datatype issue. Hence, there is an extra list bracket around the array.
I'm using Tensorflow 1.9.0 (due to project restrictions). I already checked the search function and tried solutions provided here. Following is an example code for reproducting the error of mine:
import numpy as np
import tensorflow as tf
from tensorflow import keras
import keras.backend as K
from tensorflow.keras import layers, models
def main():
ip1 = keras.layers.Input(shape=(4,))
ip2 = keras.layers.Input(shape=(4,))
dense = layers.Dense(3, activation='sigmoid', input_dim=4) # Passing the value in a weighted manner
merge_layer = layers.Concatenate()([ip1, ip2]) # Concatenating the outputs of the first network
y = layers.Dense(6, activation='sigmoid')(merge_layer) # Three fully connected layers
y = layers.Dense(4, activation='sigmoid')(y)
model = keras.Model(inputs=[ip1, ip2], outputs=y)
model.compile(optimizer='adam',
loss='mean_squared_error')
model.summary()
# dataset shape: 800 samples, 2 inputs for sequential model, 4 input size
X_train = np.random.randn(800, 2, 4)
y_train = np.random.randn(800, 4)
X_test = np.random.randn(200, 2, 4)
y_test = np.random.randn(200, 4)
history = model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=1000, batch_size=32)
if __name__ == '__main__':
main()
When there is multiple inputs keras expects list of multiple arrays. The size of the list corresponds to number of inputs you have for the model.
So basically you need to pass a list of 2 array each with shape (X,4)
X_train1 = np.random.randn(800, 4)
X_train2=np.random.randn(800,4)
y_train = np.random.randn(800, 4)
X_test1 = np.random.randn(200, 4)
X_test2 = np.random.randn(200, 4)
y_test = np.random.randn(200, 4)
history = model.fit([X_train1,X_train2], y_train, validation_data=([X_test1,X_test2], y_test), epochs=1000, batch_size=32)
I am trying to create a tensorflow keras deep learning model to predict whether a number is odd vs even. However, I cannot figure out how to actually make it work. It's current accuracy ranges between 40-60%, making me assume its just guessing. How can I improve this model? I also have no idea what loss function to use. This is my first model so I am very much a noob. I have also tried changing the layer sizes and that doesn't seem to yield much either. I have also tried varying the size of my sample data (random numbers with a label list of whether they are odd or even). I used as little as 50 to as high as the full 10000. Any help is much appreciated!
Code:
import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt
#The next few lines just import and preprocess the data
dataFile = open('oddEvenData.txt','r')
dataLabels = open('oddEvenLabels.txt','r')
data = dataFile.read()
labels = dataLabels.read()
train_data = data.split(',')
train_labels = labels.split(',')
train_data.remove('')
train_labels.remove('')
train_data = [int(x) for x in train_data]
train_labels = [int(x) for x in train_labels]
dataFile.close()
dataLabels.close()
#Creates and pre process's test data set
test_data = [[123],[56],[273],[834],[778]]
test_labels = [[0],[1],[0],[1],[1]]
test_data = np.array(test_data)
test_labels = np.array(test_labels)
test_data = test_data/10000
#Builds framework
model = keras.Sequential([
keras.layers.Dense(32, activation = 'relu',input_shape = (1,)),
keras.layers.Dense(128,activation = 'relu'),
keras.layers.Dense(64,activation = 'relu'),
keras.layers.Dense(2,activation = 'softmax')
])
model.compile(optimizer = 'adam',loss = 'sparse_categorical_crossentropy',metrics = ['accuracy'])
model.fit(train_data,train_labels,epochs = 10)
test_loss,test_acc = model.evaluate(test_data,test_labels)
print('Tested Acc:'+str(test_acc))
testNum = [int(input('Input test num: '))]
prediction = model.predict(np.array(testNum))
print(prediction)