In the zoo data, there are several types of each sample.
For making the binary classification, type 0,1,2,3 gets label 0 and type 4,5,6 gets label 1.
It's too hard for me to make split the types into two labels. So, I adjusted the data just adding the labels like below screen capture.
and I built the model like below.
import tensorflow as to
import numpy as np
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense, Dropout
zoo = np.loadtxt('~\\data-04-zoo.csv', delimiter=',', dtype=np.float32)
features = zoo[:, 0:-1]
labels = zoo[:, [-1]]
X_train, X_test, Y_train, Y_test = train_test_split(features, labels, test_size=0.25, shuffle=True, random_state=77)
print('X_train shape:', X_train.shape)
print('X_test shape:', X_test.shape)
print('y_train shape:', Y_train.shape)
print('y_test shape:', Y_test.shape)
model = Sequential()
model.add(Dense(40, input_dim=17, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(40, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='Adam', metrics=['accuracy'])
model.fit(X_train, Y_train, validation_split=0.25, shuffle=True, epochs=20, batch_size=2)
score = model.evaluate(X_test, Y_test, batch_size=2)
print(score)
My professor said that model can't be taught.
What's the problem?? I don't know how to adjust the codes.
The Result
Related
I have a classified network for the MNIST dataset (csv) with 10 labels which are numbers (0,1,2,3,4,5,6,7,8,9) and after training the network, I run the predict_classes for test_data. I want to know for each of the data in test_set what is the score of every label(0,1,2,3,4,5,6,7,8,9) in y_pred.
for example if predict_classes say that for first data the label is "7" what is the score of 7 and what is the scores of other labels such (0,1,2,3,4,5,6,8,9)
How can I write its code?
from keras import models
import numpy as np
from keras import layers
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, LSTM, BatchNormalization
mnist = tf.keras.datasets.mnist
#Load dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
#normalizing data
x_train = x_train / 255.0
x_test = x_test / 255.0
# bulid model
model = Sequential()
model.add(LSTM(15, input_shape=(x_train.shape[1:]), return_sequences=True, activation="tanh", unroll=True))
model.add(LSTM(15, dropout=0.2, activation="tanh", unroll=True))
#model.add(LSTM(1, activation='tanh'))
#model.add(LSTM(2, activation='tanh'))
model.add(Dense(5, activation='tanh' ))
model.add(Dense(10, activation='sigmoid'))
model.summary()
opt = tf.keras.optimizers.Adam(lr=1e-3, decay=1e-5)
model.compile(loss='sparse_categorical_crossentropy', optimizer=opt,
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=2, validation_data=(x_test, y_test))
y_pred = model.predict_classes(x_test)
print(y_pred)
Instead of using model.predict_classes(), you can use model.predict() (https://www.tensorflow.org/api_docs/python/tf/keras/Model#predict).
This returns an array with the probability for all of the possible classes.
I am trying to build a Keras model for a classification model and I get and error while I am trying to fit the data.
ValueError: Shapes (None, 99) and (None, 2) are incompatible
Code:
import warnings
warnings.filterwarnings(action = 'ignore')
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, classification_report
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical
df = pd.read_csv('train.csv')
del df['ST_CASE']
df
target_column = ['MVISOBSC']
predictors = list(set(list(df.columns))-set(target_column))
df[predictors] = df[predictors]/df[predictors].max()
X = df[predictors].values
y = df[target_column].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=42)
print(X_train.shape); print(X_test.shape)
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
model = Sequential()
model.add(Dense(500, activation='relu', input_dim=6))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
try:
model.fit(X_train, y_train, epochs = 20)
except Exception as e:
print(e)
Shape values:
X_train = (1282, 6)
X_test = (550, 6)
y_train = (1282)
y_test = (550)
I have also tried reshaping the X_train and X_test, but it does not have any effect on the error.
The no. of units in the last Dense layer must match the dimensionality of the outputs.
# Reshape the labels
y_train = np.expand_dims( y_train , axis=1 )
y_test = np.expand_dims( y_test , axis=1 )
model = Sequential()
model.add(Dense(500, activation='relu', input_dim=6))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(1, activation='softmax'))
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
I use python to multi-class text classification , my data set contains 25000 Arabic tweets divided into 10 classes[sport, politics,....]
When I use
training = pd.read_csv('E:\cluster data\One_File_nonnormalizenew2norm.txt', sep="*")
training.dropna(inplace=True)
training.columns = ["text", "class1"]
training['class1'] = training.class1.astype('category').cat.codes
training.dropna(inplace=True)
# create our training data from the tweets
text = training['text']
y = (training['class1'])
from sklearn.model_selection import train_test_split
sentences_train, sentences_test, y_train, y_test = train_test_split(text, y, test_size=0.25, random_state=1000)
from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
vectorizer.fit(sentences_train)
X_train = vectorizer.transform(sentences_train)
X_test = vectorizer.transform(sentences_test)
X_train
from sklearn.linear_model import LogisticRegression
classifier = LogisticRegression()
classifier.fit(X_train, y_train)
score = classifier.score(X_test, y_test)
print("Accuracy:", score)
Accuracy: 0.9525099601593625
When I use keras:
model = Sequential()
max_words=5000
model.add(Dense(512, input_shape=(input_dim,), activation='softmax'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='softmax'))
model.add(Dropout(0.5))
model.add(Dense(1,activation='softmax'))
model.add(Dense(10))
model.summary()
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.fit(X_train, y_train, batch_size=150, epochs=5, verbose=1, validation_split=0.3,shuffle=True)
predicted = model.predict(X_test)
predicted = np.argmax(predicted, axis=1)
accuracy_score(y_test, predicted)
0.28127490039840636
where the mistake???
update
I change the code to:
model = Sequential()
max_words=5000
model.add(Dense(512, input_shape=(input_dim,)))
model.add(Dropout(0.5))
model.add(Dense(256))
model.add(Dropout(0.5))
#model.add(Dense(1,activation='sigmoid'))####
model.add(Dense(10))
model.summary()
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.fit(X_train, y_train,batch_size=150,epochs=10,verbose=1,validation_split=0.3,shuffle=True)
predicted = model.predict(X_test)
predicted = np.argmax(predicted, axis=1)
accuracy_score(y_test, predicted)
0.7201593625498008
still bad accuracy!!!
Some ideas.
Remove all softmax activations (as #Matias said).
Remove the model.add(Dense(1,activation='softmax')), it's probably destroying your results.
Do more than 5 epochs.
You are not using the same tweets for validation in the two approaches.
You should probably give the accuracy on both the training and the testing datasets to be sure what is going on.
I am trying text classification using the bag of word model. Everything works fine till I use the test set for testing and evaluation of accuracy but how we can check the class of a single statement.
I have a data frame with 2 classes labels and body.
cout_vect = CountVectorizer()
final_count = cout_vect.fit_transform(df['body'].values.astype('U'))
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
X_train, X_test, y_train, y_test = train_test_split(final_count, df['label'], test_size = .3, random_state=25)
model = Sequential()
model.add(Dense(264, input_dim=X_train.shape[1], activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(16, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
y_train = np_utils.to_categorical(y_train, num_classes=3)
y_test = np_utils.to_categorical(y_test, num_classes=3)
model.fit(X_train, y_train, epochs=50, batch_size=32)
model.evaluate(x=X_test, y=y_test, batch_size=None, verbose=1, sample_weight=None)
Now I want to predict this statement using my model. How to do this
I tried converting my statement to vector using the count vectorizer but according to the bag of word approach, it is just an 8 dimension vector.
x = "Your account balance has been deducted for 4300"
model.predict(x, batch_size=None, verbose=0, steps=None)
You need to do this:
# First transform the sentence to bag-of-words according to the already learnt vocabulary
x = cout_vect.transform([x])
# Then send the feature vector to the predict
print(model.predict(x, batch_size=None, verbose=0, steps=None))
You have not shown how you "I tried converting my statement to vector using the count vectorizer but according to the bag of word approach, it is just an 8 dimension vector.", but I'm guessing you did this:
cout_vect.fit_transform([x])
If you call fit() (or fit_transform()), the vectorizer will forget all the previous training and only remember the current vocab, hence you only got a feature vector of size 8, whereas your previous vector was of higher size.
I am currently working on handwritten digit recognition of regional languages. Currently, I am focusing on Oriya. I test the MNIST dataset through the CNN model and I am trying to apply the model on my Oriya dataset. Model is performing poorly. It is giving the wrong predictions. I have a dataset of 4971 samples.
How to improve the accuracy?
Here's my code:
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.optimizers import SGD,RMSprop,adam
from keras.utils import np_utils
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
import os
import theano
from PIL import Image
from numpy import *
# SKLEARN
from sklearn.utils import shuffle
from sklearn.cross_validation import train_test_split
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# input image dimensions
img_rows, img_cols = 28, 28
# number of channels
img_channels = 1
path2 = '/home/saumya/Desktop/Oriya/p' #path of folder of images
imlist = os.listdir(path2)
im1 = array(Image.open('/home/saumya/Desktop/Oriya/p' + '/'+ imlist[0])) # open one image to get size
m,n = im1.shape[0:2] # get the size of the images
imnbr = len(imlist) # get the number of images
# create matrix to store all flattened images
immatrix = array([array(Image.open('/home/saumya/Desktop/Oriya/p' + '/'+ im2)).flatten()
for im2 in imlist],'f')
label=np.ones((num_samples,),dtype = int)
label[1:503]=0
label[503:1000]=1
label[1000:1497]=2
label[1497:1995]=3
label[1995:2493]=4
label[2493:2983]=5
label[2983:3483]=6
label[3483:3981]=7
label[3981:4479]=8
label[4479:4972]=9
print(label[1000])
data,Label = shuffle(immatrix,label, random_state=2)
train_data = [data,Label]
img=immatrix[2496].reshape(img_rows,img_cols)
plt.imshow(img)
plt.show()
(X, y) = (train_data[0],train_data[1])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)
X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28).astype('float32')
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28).astype('float32')
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]
def baseline_model():
# create model
model = Sequential()
model.add(Conv2D(32, (3,3), input_shape=(1, 28, 28), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
#model.add(Conv2D(64, (5, 5), input_shape=(1, 10, 10), activation='relu'))
#model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes, activation='softmax', name = 'first_dense_layer'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
# build the model
model = baseline_model()
# Fit the model
hist=model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=30, batch_size=100, verbose=2)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("CNN Error: %.2f%%" % (100-scores[1]*100))
score = model.evaluate(X_test, y_test, verbose=0)
print('Test Loss:', score[0])
print('Test accuracy:', score[1])
test_image = X_test[0:1]
print (test_image.shape)
print(model.predict(test_image))
print(model.predict_classes(test_image))
print(y_test[0:1])
# define the larger model
def larger_model():
# create model
model = Sequential()
model.add(Conv2D(30, (5, 5), input_shape=(1, 28, 28), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(15, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(50, activation='relu', name='first_dense_layer'))
model.add(Dense(num_classes, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
# build the model
model = larger_model()
# Fit the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Large CNN Error: %.2f%%" % (100-scores[1]*100))
I am trying to resize my model using opencv, it is generating the following error:
/root/mc-x64-2.7/conda-bld/opencv-3_1482254119970/work/opencv-3.1.0/modules/imgproc/src/imgwarp.cpp:3229: error: (-215) ssize.area() > 0 in function resize
How to improve the accuracy?
A bit hard to give a detailed answer from what you posted, and without seeing some data sample, but still will attempt a stab at this. What I can see that may help improve you accuracy is:
Get more data. In Deep Learning one usually works with big amount of data, and models almost always improve when adding more data. If you can't obtain new data you may try to generate more samples with the ones you got, by adding noise or similar modifications.
I see you currently have 30 and 10 epochs on the training of your model. I suggest you increase the number of epochs, so your model has more time to converge. This also most of the times improves performance up to a point.
I also see that your batch size is 100 and 200 on your models. You can try reducing the batch size of your training process, so your training performs gradient update more times on each epoch (remember that you can even use batch_size=1 to upgrade your model for each sample, instead of batches).
Alternatively, you can try iteratively increasing the complexity and layers of your architecture and compare your performances. It is best to start with a simple model, train and test, and then add layers and nodes until you are satisfied with the results. I also see you have tried a hybrid convolutional and non-convolutional approach; you can well try starting with just one of the approaches before increasing the complexity of your architecture.