I am trying to use the Open Images dataset to train a binary CNN model (Orange vs. Not Orange).
I use the OID v4 toolkit to download images of few classes both in train and test.
Now I'm stuck with how to conert the multiclass shape in each directory to a binary.
I believe I need some tool to change the subfolders (=classes) name.
I've succeeded using os and shutil packages to manipulate the directories as requested.
TXS.
Related
I need to train tensorflow deeplab model with my shoes dataset. Then i will use this model in order to remove background of image shoe. How could i train it ? Could you explain step by step ? You have any example for this situation ?
tensorflow/deeplab
You will need read some parts of Deeplab code
Download repo
Now you need to put your data in tfrecord in proper format
Use some of scripts in https://github.com/tensorflow/models/tree/master/research/deeplab/datasets to download and generate example datasets
Prepare analogous script for your shoes dataset
Add information about data to Deeplab source file https://github.com/tensorflow/models/blob/master/research/deeplab/datasets/data_generator.py add info in analogous format like example datasets
Check flags for architecture https://github.com/tensorflow/models/blob/master/research/deeplab/common.py
Check specific flags and then train, export, count statistics or visualize using train.py, vis.py, export_model.py, eval.py in folder https://github.com/tensorflow/models/tree/master/research/deeplab
I want to fine tune existing OpenCV DNN face detector to a face images database that I own. I have opencv_face_detector.pbtxt and opencv_face_detector_uint8.pb tensorflow files provided by OpenCV. I wonder if based on this files is there any way to fit the model to my data? So far, I haven't also managed to find any tensorflow training script for this model in OpenCV git repository and I only know, that given model is and SSD with resnet-10 as a backbone. I am also not sure, reading the information on the internet, if I can resume training from .pb file. Are you aware of availability of any scripts defining the model, that could be used for training? Would pbtxt and pb files be enough to continue training on new data?
Also, I noticed that there is a git containing caffe version of this model https://github.com/weiliu89/caffe/tree/ssd. Although I never worked with caffe before, would it be possible/easier to use existing weight (caffe .pg and .pbtxt files are also available in OpenCV's github) and fit the model to my dataset?
I don't see a way to do this in opencv, but I think you'd be able to load the model into tensorflow and use model.fit() to retrain.
The usual advice about transfer learning applies. You'd probably want to freeze most of the early layers and only retrain the last one or two. A slow learning rate would be advised as well.
Directory structure:
Data
-Cats
--<images>.jpg
-Dogs
--<images>.jpg
I'm training a (n-ary) classification model. I want to create an input_fn for serving these images for training.
image dimensions are (200, 200, 3). I have a (keras) generator for them, if they can be used somehow.
I've been looking for a while but haven't found an easy way to do this. I thought this should be a standard use-case? e.g. Keras provides flow_from_directory to serve keras models. I need to use a tf.estimator for AWS Sagemaker so I'm stuck with it.
By using the tf dataset Module you can feed your data directly into your estimator. You basically have 3 ways to integrate this into your api:
1. convert your images into tfrecords and use tfrecorddataset
2 use the tf dataset from generator function to use generators
3 try introducing these decoder functions into your inputpipeline
I need to train from scratch a CNN over a COCO dataset with a specific configuration: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/embedded_ssd_mobilenet_v1_coco.config
Thus, I installed TF Object Detection API and I downloaded the COCO dataset. However the dataset is in .h5 extension.
Is it possible to run the training with this kind of file or do I need to convert it in images in someway? If that is possible, what would the command be?
PS: I was not able to find a pre-trained model with that config, this is why I need to train a cnn from scratch.
My suggestion would be to convert the .hdf5 file to a .tfrecord file, you can find examples of how to do this here.
I wanted to train the inception model like shown in the tensorflow github-tutorial.
Except i wanted to use a selfmade Dataset of TFRecord files.
bazel build inception/imagenet_train
bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=32 --train_dir=/tmp/imagenet_train --data_dir=/tmp/imagenet_data
I changed the data directory to the folder with my own TFRecord files.
Now i´am wondering whether i´am realy training from scratch, or if this is the same thing like the "retraining the last layer -Tutorial"
Yes you are training from scratch: see the code