I am currently working on a project for identification of mood/emotions of a person .
As the first step we are working on a image recognition , detection and tracking python code.
I went through the various different approach towards this problem and found out.
1)Haar cascade method (fast but no scope for recognition and reading expressions ).
2)Neural networks(Great at image recognition ie for details such as smile/ anger..... ).
I am confused with neural networks , ie the approach.
We can first use haar cascade to detect the faces with ease(really fast) then use either canny edge detection or Cropping to crop out the part of the face.
After that is done i have no clue on how to proceed .
This is my idea of it.
continue using haar cascade method to detect the features of the face like eyes, nose ,cheek ,lips....
then find out the distance between them to find out ratios which we can further use to form a neural network .
Different internal layers would be used to detect different features.
We can use differential method to optimize the cost by altering the weights of the synapses .
How good is the approach and is there a better way to do it .
Like say we can use canny edge to detect the edges and then make a new matrix just out of the edges and then use this to train the data.
I dont know , i am really confused.
Anyways thanks in advance for all answers
Image processing libraries such as scikit-image or OpenCV are a good place to start. For example, here's an example of canny edge detection in OpenCV.
Regarding neural networks, as lejlot pointed out, you've got to ask yourself how much you want to build from scratch.
An example for building your own neural network based on some parameters (which you'd have to define for your facial features), I suggest you read through A Neural Network in 11 lines of Python which illustrates some of problems you might face (especially part 2 where it's about image processing too).
What you seem to need are Convolutional Neurl Networks (check this http://cs231n.github.io/convolutional-networks/ to learn about them).
Convolutional Neural Networks (CNN for short) are a kind of neural nets that learn to extract visual features from an image and how to relate those features to recognize what's on the image, so you don't need to detect all the features, just give a CNN a bunch of labeled face pictures and it will learn to identificate the mood of the eprson.
What you can do is to detect the face in every picture (openCV is good enough at detecting faces) and then crop and align each face so all the faces have the same size. Then feed the CNN with all the faces, and it will gradually learn to recognize the emotions of a person.
Related
As an example I have two pictures with a particular type of clothing of a certain brand.
I can download a lot of different images of this same piece, and color, of clothing
I want to create a model which can recognize the item based on a picture.
I tried to do it using this example:
https://www.tensorflow.org/tutorials/keras/classification.
This can recognize the type of clothing (eg shirt or shoe or trousers, etc) But not a specific item and color.
My goal is to have a model that can tell me that the person on my first picture is wearing the item of my second picture.
As mentioned I can upload a few variations of this same item to train my model, if that would be the best approach.
I also tried to use https://pillow.readthedocs.io
This can do something with color recognition but does not solve my initial goal.
i don't think that CNN can help you in your problemes, take a look at the SIFT Technique see this for more détails.it is used for image matching and i think it's better in your cas. if your not looking to get in to much detailes the opencv is a python (and c++ i think) library that has image matching function that are easy to use more détails .
As mentionned by #nadji mansouri, I would use SIFT technique as it suits your need. But I want just to correct something, CNN is also a thing in this case. This being said, I wouldn't tackle the problem as a classification problem, but rather using Distance Metric Learning, i.e, training a model to generate embeddings that are similar in the space when the inputs are similar, and distant otherwise. But to do this you need a large representative dataset.
In short, I suggest starting with SIFT, using OpenCV, or open source implementations on GitHub, playing around with the parameters and see what fits your case best, and then see if it's really necessary to switch to a neural network, and in this case tackling the problem as a metric learning task, maybe with something like siamese networks.
Some definitions:
Metric learning is an approach based directly on a distance metric that aims to establish similarity or dissimilarity between data (images in your case). Deep Metric Learning on the other hand uses Neural Networks to automatically learn discriminative features from the data and then compute the metric. source.
The Scale-Invariant Feature Transform (SIFT) is a method used in computer vision to detect and describe local features in images. The algorithm is invariant to image scale and rotation, and robust to changes in illumination and affine distortion. SIFT features are represented by local image gradients, which are calculated at various scales and orientations, and are used to identify keypoints in an image. These keypoints and their associated descriptor vectors can then be used for tasks such as image matching, object recognition, and structure from motion. source, with modification.
I am trying to face recognise by python Face-recognition library
I have tried below code for below image
Code :
import face_recognition
image = face_recognition.load_image_file("img/bill.jpeg")
property(image)
face_locations = face_recognition.face_locations(image)
print(len(face_locations))
For below image I am getting output for total face : 6
Image :
But When I am trying by a cartoon image
I am getting output : 0
How can I recognise cartoon face by face-recognition?
Sorry to say but if the face recognition is good it should not recognize cartoon faces, it's designed to recognize human faces and therefore should only tell you how many human faces it is on the image, otherwise it's a bad designed algorithm. If you want a machine-learning algorithm to recognize cartoon faces you would have to train it your self for that specific test.
I did a quick search on google and the first things I found was an article named "Cartoon Face Recognition: A Benchmark Dataset" at https://arxiv.org/pdf/1907.13394.pdf . Maybe you can find an already existing machine-learning algorithm that have been trained to recognize cartoon faces.
Hope this helped and I hope you find what you're looking for.
--------------------------------EDIT--------------------------------
I found these two git repositories, could be worth looking into more
https://github.com/srvCodes/Cartoon-Face-Detection-and-Recognition
https://github.com/hako/dissertation
The last link is a link for emotions of cartoon character.
Short answer: You need to train a new model to detect cartoon characters.
Long explanation:
Because cartoon characters have different facial features from normal
human faces. For cartoons, the face edges are smooth, perfectly round
eyes, smooth-shaped mouth, and cartoonish face structure.
The pre-trained model that you are using doesn't know to identify these structures, it hasn't seen such images during training.
A model detects a face using a lot of filters, these filters could detect lines and shapes in an image. If all these filters combine and give a high output, then there is a face in that location.
So, you either have to look for models that are trained on cartoons, or label
and train a model by your self.
I wish to know whether I can use an Inception or ResNet model to identify faces. I want to know whether transfer learning and training is even considerable for my task.
I just want to be able to identify faces but I am also curious whether I can retrain/optimize a pre-trained model for my task.
Or have I been reading of things wrong; do I need to get a pre-trained model that was designed for faces?
I have tried poking around with Inception and VGG16 but I have not trained them for faces. I am working on it but I want to know whether this is even viable or simply a waste of time. If I use transfer learning with FaceNet I think I'll be better off.
Transfer learning for facial detection would be a great way to go ahead. Also, yes transfer learning with facenet is a great idea.
Also, for transfer learning to work it is not necessary that the model had to be initially pre-trained with only faces like using facenet. A model pre-trained with imagenet would also be pretty darn good! This is a very hot topic, so do not try to reinvent the wheel. There are many repositories that have already done this using transfer learning from imagenet dataset and using resnet50 with astonishingly good results.
Here is a link to one such repository:
https://github.com/loheden/face_recognition_with_siamese_network
Also note that siamese networks is a technique that is especially good in the facial recognition use case. The concept of siamese is really simple: take two images and compare the features of these two images. If the similarity in features are above a set threshold, then the two images are the same (the two faces are the same) else not the same (face not recognized).
Here is a research paper on siamese networks for facial recognition.
Also, here is a two-part tutorial on how to implement the siamese network for facial recognition using transfer learning:
http://www.loheden.com/2018/07/face-recognition-with-siamese-network.html
http://www.loheden.com/2018/07/face-recognition-with-siamese-network_29.html
The above tutorial's code is in the first Github link I shared at the beginning of this answer.
I am using machine learning to recognize chess pieces. HoG is the feature used in my code to identify the different set of pieces, however, the results are not that accurate because of how HoG adjusts the image. Are there any other better ways to identify images, specifically chess pieces?
fd, hog_image = hog(img, orientations=16, pixels_per_cell=(8, 8),cells_per_block=(2, 2), visualize=True)
The main idea is to use convolutional neural networks if you have a classification problem. For the object detection (localization + classification) You can use TensorFlow object detection API as well as YOLO models.
Here you can read more
https://towardsdatascience.com/how-to-train-your-own-object-detector-with-tensorflows-object-detector-api-bec72ecfe1d9
https://github.com/tensorflow/models
https://medium.com/#jonathan_hui/real-time-object-detection-with-yolo-yolov2-28b1b93e2088
http://cs231n.github.io/convolutional-networks/
I am doing a project for recognizing expressions.my plan is to use LBP and SVM.For Implementing I choose Opencv and Python.I know that there is a inbuilt function createLBPHFaceRecognizer in opencv for LBP calculations.Which training method is used in createLBPHFaceRecognizer.train, is it SVM or K nearest or anything else?
I worked on a similar problem and if I was you, I would first detect face landmarks (best is DLib: http://dlib.net/webcam_face_pose_ex.cpp.html but in C++) then classify face points to get expression. They made something great in JavaScript (face landmarks, expressions classification - https://github.com/auduno/clmtrackr) but it is not pretty robust in detecting face landmarks.