I am working on a project where I am trying to cluster different small image sets (each set has around 100 images) without knowing how many classes can each set contain (it can go from 2 to 4). I have tried using a convolutional autoencoder for feature extraction and dimensionality reduction and then ran HDBSCAN on the reduced images, but in vain.
Does anyone have a suggestion for which feature extraction/clustering algorithm I can go with?
Related
I'm trying to solve a regression problem using a Python Keras CNN (Tensorflow as the backbone), where I try to predict a single y-value based on an 8-dimensional satellite image (23x45 pixels) that I have fetched from Google Earth Engine using their Python API. I currently have 280 images that I augment to get 2500 images using flipping and random noise. The data is normalized & standardized and I have removed outliers and images with only zeros.
I've tested numerous CNN-architecture, for example, this:
(Convolution2D(4,4,3),MaxPooling2D(2,2),dense(50),dropout(0.4),dense(30),dropout(0.4),dense(1)
This results in a weird behaviour where the predicted value is in mainly two distinct groups or clusters (where each group has very little variance). The true value has a much higher variance. See image below.
I have chosen not to publish any code snippets as my question is more of a general nature. What might lead to such clustered predictions? Are there any good common tricks to improve the results?
I've tried to solve the problem using a normal neural network and regression tools from SciKit-Learn, by flattening the images to one long array (length 23x45x8 = 8280). That doesn't result in clustering, although the accuracy is still quite low. I assume that is due to insufficient or inappropriate data.
Plotted Truth (x) vs Prediction (y) which shows that the prediction is heavily clustered
your model is quite simple, it cannot even properly extract feature, so i guess it is under fit. and your dropout is 40% in 2 layers, which quite high for such small network. you also have linear activation, it seems that way.
and yes number of sample can also have effect on group prediction, mostly class with majority of samples is chosen.
i have also noticed some of your truth values are greater than 1 and less than 0. you have to normalize properly and use proper activation function.
I have a folder with hundres/thousands of images, some of them look alike. I would like to create clusters separating those images (those which look alike in the same cluster).
I can't determine the number of clusters that will be needed, it depends on the images.
Does anyone have an idea on how to do this using Python, OpenCV and which algorithm to use?
I've made some research and found that AffinityPropagation or DBSCAN can be useful for me but I don't know where to start (how to encode my images, what should I pass to those algorithms etc...)
Unfortunately it is not that simple with images, since naively clustering would result in clusters of images with the same colors, not the same "content". You can use a neural network as a feature extractor for the images, I see two options:
Use a pre-trained network and get the features from an intermediate layer
Train an autoencoder on your dataset, and use the latent features
Option 1 is cheaper since you can easily find pre-trained models, option 2 is much more computationally expensive but should work better, especially if there is no pre-trained model on your domain.
This tutorial (randomly found on the internet) seems to be a good introduction to method 2.
I have a database of images that contains identity cards, bills and passports.
I want to classify these images into different groups (i.e identity cards, bills and passports).
As I read about that, one of the ways to do this task is clustering (since it is going to be unsupervised).
The idea for me is like this: the clustering will be based on the similarity between images (i.e images that have similar features will be grouped together).
I know also that this process can be done by using k-means.
So the problem for me is about features and using images with K-means.
If anyone has done this before, or has a clue about it, please would you recommend some links to start with or suggest any features that can be helpful.
Most simple way to get good results will be to break down the problem into two parts :
Getting the features from the images: Using the raw pixels as features will give you poor results. Pass the images through a pre trained CNN(you can get several of those online). Then use the last CNN layer(just before the fully connected) as the image features.
Clustering of features : Having got the rich features for each image, you can do clustering on these(like K-means).
I would recommend implementing(using already implemented) 1, 2 in Keras and Sklearn respectively.
Label a few examples, and use classification.
Clustering is as likely to give you the clusters "images with a blueish tint", "grayscale scans" and "warm color temperature". That is a quote reasonable way to cluster such images.
Furthermore, k-means is very sensitive to outliers. And you probably have some in there.
Since you want your clusters correspond to certain human concepts, classification is what you need to use.
I have implemented Unsupervised Clustering based on Image Similarity using Agglomerative Hierarchical Clustering.
My use case had images of People, so I had extracted the Face Embedding (aka Feature) Vector from each image. I have used dlib for face embedding and so each feature vector was 128d.
In general, the feature vector of each image can be extracted. A pre-trained VGG or CNN network, with its final classification layer removed; can be used for feature extraction.
A dictionary with KEY as the IMAGE_FILENAME and VALUE as the FEATURE_VECTOR can be created for all the images in the folder. This will make the co-relation between the filename and it’s feature vector easier.
Then create a single feature vector say X, which comprises of individual feature vectors of each image in the folder/group which needs to be clustered.
In my use case, X had the dimension as : NUMBER OF IMAGE IN THE FOLDER, 128 (i.e SIZE OF EACH FEATURE VECTOR). For instance, Shape of X : 50,128
This feature vector can then be used to fit an Agglomerative Hierarchical Cluster. One needs to fine tune the distance threshold parameter empirically.
Finally, we can write a code to identify which IMAGE_FILENAME belongs to which cluster.
In my case, there were about 50 images per folder so this was a manageable solution. This approach was able to group image of a single person into a single clusters. For example, 15 images of PERSON1 belongs to CLUSTER 0, 10 images of PERSON2 belongs to CLUSTER 2 and so on…
Acquisition
I have images that contains defaults areas...Original image's size is arround 3072x16000, that is huge ! Lenght can randomly change, depending on product length.
The image is from a profilometer and look like it :
The speed of the convoyer is fixed. Image length depend on product's.
We can't do "online" processing because acquisition and image processing are from different suppliers.
The first supplier give individual image of products, product lenght is not fixed.
Defaults
Defaults are quite small (less than 256x256px), then i cropped it, learn a CNN how to recognize it from a conform area (both 256x256x1 px).
The aim is to focus the network training on ROIs because i don't have a huge database of images.
I need very high resolution to get small defaults. The classification verdict is on small 256x256px subimage.
I'll have arround 20 classes of defaults and 4 classes of conforms areas (depending on where i am in the image).
I use greylevel image to identify defaults.
I can classify my 256x256px small image between "Good"/"Bad" classes.
If one area is identified as "Bad", the product is "suspicious" and segregated...
CNN
I used TensorFlow and retrained a mobilenet network, that work well on 256x256 images, as if training was long.
Now i face other issue. Input images size are in reality arround 3072x16000 pixel.
Is there a recommanded way to use my pretrain CNN on theses huge images ?
How should i cut it and pass it to my CNN?
Many Thanks !
![sample training input]http://www.cs.toronto.edu/~vmnih/data/mass_roads/train/sat/10078660_15.tiff
![sample training output]
http://www.cs.toronto.edu/~vmnih/data/mass_roads/train/map/10078660_15.tif
I am a beginner to CNN , and have worked with the MNIST dataset in which we input 28x28x3 images and output a 10x1 vector containing probabilities of the 10 classes(0,1,2,3---,9).
How do we extract only the road pixels from the input image and display them, as is represented by the output image?
This problems is a binary segmentation problem. In a sense you learn a mapping from satellite images and predict for each pixel, iff this pixel is part of the road. A simple algorithm to do this would be to check if the pixel color is part of some range.
A CNN naturally will learn you a more complicated function based on the local neighborhood of said pixel. One repo to get you started should be this one: https://github.com/jocicmarko/ultrasound-nerve-segmentation. Therein they use a similar algorithm to segment ultrasound images using CNNs. You just have to use 3 input channels instead of 1 and everything else should be quite similar.