I am in full and bad surprise. Same program everything is perfect. I just slept and opened the Google colab today to run the program. This is my first ever deep learning program. It ran perfectly yesterday. But when I run today, it is giving a weird error. Need help. Why it is giving such error? How to solve it?
Google colab screenshot:
Code:
#Step3: test_img_path: Location of the image we want the model to predict
test_img = image.load_img(test_img_path,target_size=(224,224))
#Step4: Deep learning models expect a batch of images represented by array
# At this stage we will have a processed image of size 224x224x3.
# Convert it to a batch of images denoted by nx224x224x3 where n denotes total images
# In this case, n=1
test_img_array = image.img_to_array(test_img)
# Convert the array to a batch
test_img_batch = np.expand_dims(test_img_array,axis=0)
#Step5: At the data level, an original image data is stored in the in terms of the pixels.
# Now, normalizing the image
nor_testimg = preprocess_input(test_img_batch)
#Step6: Import the model and input our test image
# Model here means, it is already trained by someone else and I don't have to do it again
# Moreover, they made their hardwork or trained model freely available to every on on the keras, we just download it
model = tf.keras.applications.resnet50.ResNet50()
#Step7: Lets see how and what the model would predict
predict_testimg = model.predict(nor_testimg)
# Decode the predictions
print(decode_predictions(predict_testimg,top=3)[0])
In the above code, tf.keras.applications.resnet50.ResNet50() is the one causing the problem when I run it today. The same program ran successfully yesterday. Now, if I remove end brackets tf.keras.applications.resnet50.ResNet50, it runs perfectly but raised an error in the next line of the code.
The issue is not with you and it lies in Keras as its trying to decode a string with utf 8. If I can get some more part of it might be able to help then
Related
I have 31.216 labeled images for object detection. I used LabelIMG program to label images and it's in Pascal-VOC format. I want to create a tflite model for my Kotlin project. However, I have serious problems.
First, If I use my local environment, I tried to install tflite-model-maker library in PyCharm using pip install tflite-model-maker. It downloaded ~30GB and Python still says unresolved reference. Then I tried to add the library from here but it also didn't work. I couldn't achieve importing the library.
On the second way, I used Google Colab. Following this tutorial from Tensorflow. I mounted my Google Drive in Colab and edited all codes for my dataset path. I ran this line model.export(export_dir='.', tflite_filename='AslModel.tflite') lastly and it create model file in the colab directory. I continued to run next line model.evaluate_tflite('AslModel.tflite', val_data) and it gave 16 hours ETA and after 14 hours Google Colab runtime gave an error and all runtime has been reset. Now, I have a tflite and I tested it. Since there is no evaluation step, it makes bad predictions. I started all over again but Google Colab gave an error again. I guess ~7 hours training + ~16 hours of evaluation is impossible with Google Colab because there is 24h limit. Thus, my question is how can I run the evaluation step only?
model is defining in this line and it takes 7 hours model = object_detector.create(train_data, model_spec=spec, batch_size=4, train_whole_model=True, epochs=20, validation_data=val_data). Instead of this line, I want to initialize my tflite file to a model like model = LoadModel(PATH_OF_MY_TFLITE). I couldn't find any load method so I'm stuck there.
To sum up, objective is training the Pascal-VOC formatted dataset. I couldn't import the libraries for Python and with Google Colab I have raw tflite model but it needs evaluation and I can't run previous steps due to time limit. Lastly, I bought Colab Pro but I spent all my compute unit. I don't even know what is the purpose of compute unit. I'm waiting for suggestions. Thank you.
I am following this tensorflow tutorial notebook to classify images of flowers:
https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/images/classification.ipynb#scrollTo=U-e-XzMeyH2O
Everything seems OK until the final cell in the notebook where the trained model is used to predict the class of a new image. I am getting identical predictions for all inputs.
I tried adding:
print(predictions)
print(score)
Then predicting on the sample image (of a sunflower):
sunflower_url = "https://storage.googleapis.com/download.tensorflow.org/example_images/592px-Red_sunflower.jpg"
outputs:
[[-2.1131027 -1.3355725 0.29224062 3.8924832 1.3749899 ]]
tf.Tensor([0.00220911 0.00480723 0.02448191 0.896212 0.07228985], shape=(5,), dtype=float32)
This image most likely belongs to sunflowers with a 89.62 percent confidence.
But if I just change the input to a picture of a rose, like:
sunflower_url = "https://images.photowall.com/products/64377/rose-flower.jpg"
outputs:
[[-2.1131027 -1.3355725 0.29224062 3.8924832 1.3749899 ]]
tf.Tensor([0.00220911 0.00480723 0.02448191 0.896212 0.07228985], shape=(5,), dtype=float32)
This image most likely belongs to sunflowers with a 89.62 percent confidence.
I have seen that there can be many model related issues (scaling / overfitting etc) which can cause identical outputs, however it seems strange that a tutorial example would fail in this way. So I suspect there is something more obvious that I am missing.
The issue here was related to the line
sunflower_path = tf.keras.utils.get_file('Red_sunflower', origin=sunflower_url)
When downloading a new image, it was NOT overwriting the stored image. So the model was making a prediction against the same input every time.
I manually defined the save location and made sure to get the input from that, then it worked.
Note that you do not need to rescale the image, this is handled within the predict() function from keras as already defined in the model.
I'm making an mobile app that uses TensorFlow Lite to perform text classification on tweets. I've done this successfully with the TensorFlow sample model but this model is trained on IMDB movie reviews and I want to have a custom model on device that is trained on tweets to increase accuracy. I have training and test sets for this domain and am trying to create a custom model following this tutorial https://www.tensorflow.org/lite/tutorials/model_maker_text_classification
I'm running into a pythion KeyError though and cant figure out why. here is a screen shot
You can see on the right a sample of my csv, I have a label and Sentence column, using the TextClassifierDataLoader. I don't understand why this key error is happening, I'm interpreting this as it can't find a column named "Sentence" but cleary it's there.
Any ideas?
Seems to me that is_training is not an available argument in the class, so just take out that line/argument if you want it to compile (what it was supposed to do in the first place, I haven't looked into).
https://github.com/tensorflow/examples/blob/1dc6978e2141e7a5efebcf6971b3afa9cb055679/tensorflow_examples/lite/model_maker/core/data_util/text_dataloader.py#L90
Issue was the column " Sentence" had an empty space at the beginning
I faced a strange challenge trying to train neural network using code from github, it is huggingface conversational model.
What happens: even i use my own dataset for training result remains the same like with original dataset. My hypothesis that it is a somehow cache problem - old dataset continuously get loaded from cached and replace my.
Them when i launch actual interactive session with neural network it works, but without my data, even if i pass model checkpoint.
Why i think of cache: in this repo author use automatic downloading and caching neural network model in /home/joo/.cache/torch/pytorch_transformers/ if no parameter specified in terminal.
I have created an issue on Github. BUT i am not sure is that a problem specific for this repo only, or it is a common problem with retraining neural networks i faced first time.
https://github.com/huggingface/transfer-learning-conv-ai/issues/36
Some copypaste from issue:
I am still curious, was not able to pass my dataset:
I added to original 200mb json my personality
trained once more with --dataset_path ./my.json
invoke interact.py with new checkpoint and path python ./interact.py --model_checkpoint
./runs/Oct08_18-22-53_joo-tf_openai-gpt/ --dataset_path ./my.json
and it reports Gathered 18878 personalities (but not 18879, with my own).
I changed the code in interact.py to choose my first perosnality this way
was: personality = random.choice(personalities)
become: personality = personalities[0]
and this first personality is not mine.
Solved: it is a specific issue to this repo, just hardcoded dataset path.
But still why it doesn't load first time - no answer
I have a few thousand pictures I want to train a model with tflearn. But somehow I have some problems to prepare the images to the right data format. I tried tflearn.data_utils.image_preloader() but I'm getting a ValueError. So I tried to write my own image_loader.py file, but with so many pictures my RAM is running full.
Does anyone know a good tutorial, example or anything to write a CNN with my own image set, along with details on how to preprocess the data for tflearn?
The Tensorflow tutorial is a good place to start. Preprocessing is covered in this section.