Reusing models from grabcut in OpenCV

Reusing models from grabcut in OpenCV - python

I used the interactive grabcut.py from the OpenCV samples to segment an image and saved the foreground and background models. Then I used these models to segment more images of the same kind, as I don't want to retrain the model each time.
After running the grabcut algorithm, the mask is all zeros (all background) and therefore it doesn't segment anything.
from matplotlib import pyplot as plt
import numpy as np
import cv2
img = cv2.imread('usimg1.jpg')
mask = np.zeros(img.shape[:2], np.uint8)
bgdModel = np.load('bgdmodel.npy')
fgdModel = np.load('fgdmodel.npy')
cv2.grabCut(img, mask, None, bgdModel, fgdModel, 5, cv2.GC_EVAL)
mask = np.where((mask==2) | (mask==0), 0, 1).astype('uint8')
img = img * mask[:, :, np.newaxis]
plt.imshow(img)
plt.show()
I tried to initialize the algorithm with a mask or a rectangle but this produces an error because the models are not empty (which is what I actually want).
How do I have to pass the pre-trained models to the algorithm, such that they are not retrained from scratch each time I'm segmenting an image?
EDIT
After rayryeng's comment I implemented following code:
cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 0, cv2.GC_INIT_WITH_RECT)
cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 2, cv2.GC_EVAL)
It seems to work but the first call now changes my model. In the source code it calls learnGMMs without checking whether a pretrained model is provided.

You have the correct line of thinking where you use cv2.GC_EVAL so that you only need to perform the segmentation without having to compute the models again.
Unfortunately even when you use this flag, this is a limitation with the OpenCV source itself. If you look at the actual C++ implementation when you encounter the GC_EVAL condition, it does this towards the end of the cv::grabcut method. Note that the Python cv2.grabCut method is a wrapper for cv::grabcut:
if( mode == GC_EVAL )
checkMask( img, mask );
const double gamma = 50;
const double lambda = 9*gamma;
const double beta = calcBeta( img );
Mat leftW, upleftW, upW, uprightW;
calcNWeights( img, leftW, upleftW, upW, uprightW, beta, gamma );
for( int i = 0; i < iterCount; i++ )
{
GCGraph<double> graph;
assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );
learnGMMs( img, mask, compIdxs, bgdGMM, fgdGMM );
constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
estimateSegmentation( graph, mask );
}
You'll see that GC_EVAL is only encountered once in the code and that's to check the validity of the inputs. The culprit is the learnGMMs function. Even though you specified the trained models, these get reset because the call to learnGMMs ignores the GC_EVAL flag, so this gets called regardless of whatever flag you specify as the input.
Inspired by this post: OpenCV - GrabCut with custom foreground/background models, what you can do is you'll have to modify the OpenCV source yourself and inside the loop you can place an if statement to check for the GC_EVAL flag prior to calling learnGMMs:
if( mode == GC_EVAL )
checkMask( img, mask );
const double gamma = 50;
const double lambda = 9*gamma;
const double beta = calcBeta( img );
Mat leftW, upleftW, upW, uprightW;
calcNWeights( img, leftW, upleftW, upW, uprightW, beta, gamma );
for( int i = 0; i < iterCount; i++ )
{
GCGraph<double> graph;
assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );
if (mode != GC_EVAL) // New
learnGMMs( img, mask, compIdxs, bgdGMM, fgdGMM );
constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
estimateSegmentation( graph, mask );
}
This should be able to use the pre-trained models without having to learn them all over again at each iteration. Once you make the change, you'll have to recompile the source again and that should hopefully be able to use your pre-trained models without clearing them when you use the cv2.GC_EVAL flag.
For the future, I have opened up a issue on the official repo for OpenCV. Hopefully they'll fix this when they have the time: https://github.com/opencv/opencv/issues/9191

Related

Azure Kinect - c++ green screen example in Python - How to create an openCV mask from a depth image?

I am trying to translate the green screen sample (https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/examples/green_screen/main.cpp) from c++ to python, however, there is one part I cannot figure out.
This is the C++ code from the sample:
k4a::image main_color_image = captures[0].get_color_image();
k4a::image main_depth_image = captures[0].get_depth_image();
// let's green screen out things that are far away.
// first: let's get the main depth image into the color camera space
k4a::image main_depth_in_main_color = create_depth_image_like(main_color_image);
main_depth_to_main_color.depth_image_to_color_camera(main_depth_image, &main_depth_in_main_color);
cv::Mat cv_main_depth_in_main_color = depth_to_opencv(main_depth_in_main_color);
cv::Mat cv_main_color_image = color_to_opencv(main_color_image);
// single-camera case
cv::Mat within_threshold_range = (cv_main_depth_in_main_color != 0) &
(cv_main_depth_in_main_color < depth_threshold);
// show the close details
cv_main_color_image.copyTo(output_image, within_threshold_range);
// hide the rest with the background image
background_image.copyTo(output_image, ~within_threshold_range);
cv::imshow("Green Screen", output_image);
In Python, using pyk4a, I have translated it to:
capture = k4a.get_capture()
if(np.any(capture.depth) and np.any(capture.color)):
color = capture.color
depth_in_color_camera_space = capture.transformed_depth
depth_in_color_camera_space = cv2.normalize(depth_in_color_camera_space, None, 0, 255, cv2.NORM_MINMAX, dtype=cv2.CV_8U)
However, now I am stuck on creating the mask. I have tried multiple things with cv copyTo , threshold and bitwise_and, but none worked for me.
To help understand the C++ and python code better:
create_depth_image_like():
This is a method provided by the sample that creates a k4a::image. This is not needed for python because it just provides an array.
static k4a::image create_depth_image_like(const k4a::image &im)
{
return k4a::image::create(K4A_IMAGE_FORMAT_DEPTH16,
im.get_width_pixels(),
im.get_height_pixels(),
im.get_width_pixels() * static_cast<int>(sizeof(uint16_t)));
}
depth_image_to_color_camera():
This method is provided by the Azure Kinect SDK and is available in python as capture.transformed_depth. It transforms the depth image to be in the same image space as the color image or vice versa.
depth_to_opencv():
I am not sure if I need to use this method. Also, I don't really know what it does to be honest.
static cv::Mat depth_to_opencv(const k4a::image &im)
{
return cv::Mat(im.get_height_pixels(),
im.get_width_pixels(),
CV_16U,
(void *)im.get_buffer(),
static_cast<size_t>(im.get_stride_bytes()));
}
color_to_opencv():
Same goes for this one. I think I don't need them because python returns an array already, however, I am not sure.
static cv::Mat color_to_opencv(const k4a::image &im)
{
cv::Mat cv_image_with_alpha(im.get_height_pixels(), im.get_width_pixels(), CV_8UC4, (void *)im.get_buffer());
cv::Mat cv_image_no_alpha;
cv::cvtColor(cv_image_with_alpha, cv_image_no_alpha, cv::COLOR_BGRA2BGR);
return cv_image_no_alpha;
}
So, what's left for me to translate to python is the mask (defined in the C++ code as within_threshold_range. However, this is the part I just can't figure out. Any help would be greatly appreciated!

Opencv: Jetmap or colormap to grayscale, reverse applyColorMap()

To convert to colormap, I do
import cv2
im = cv2.imread('test.jpg', cv2.IMREAD_GRAYSCALE)
im_color = cv2.applyColorMap(im, cv2.COLORMAP_JET)
cv2.imwrite('colormap.jpg', im_color)
Then,
cv2.imread('colormap.jpg')
# ??? What should I do here?
Obviously, reading it in grayscale (with , 0) wouldn't magically give us the grayscale, so how do I do it?

You could create an inverse of the colormap, i.e., a lookup table from the colormap values to the associated gray values. If using a lookup table, exact values of the original colormap are needed. In that case, the false color images will most likely need to be saved in a lossless format to avoid colors being changed. There's probably a faster way to do map over the numpy array. If exact values cannot be preserved, then a nearest neighbor lookup in the inverse map would be needed.
import cv2
import numpy as np
# load a color image as grayscale, convert it to false color, and save false color version
im_gray = cv2.imread('test.jpg', cv2.IMREAD_GRAYSCALE)
cv2.imwrite('gray_image_original.png', im_gray)
im_color = cv2.applyColorMap(im_gray, cv2.COLORMAP_JET)
cv2.imwrite('colormap.png', im_color) # save in lossless format to avoid colors changing
# create an inverse from the colormap to gray values
gray_values = np.arange(256, dtype=np.uint8)
color_values = map(tuple, cv2.applyColorMap(gray_values, cv2.COLORMAP_JET).reshape(256, 3))
color_to_gray_map = dict(zip(color_values, gray_values))
# load false color and reserve space for grayscale image
false_color_image = cv2.imread('colormap.png')
# apply the inverse map to the false color image to reconstruct the grayscale image
gray_image = np.apply_along_axis(lambda bgr: color_to_gray_map[tuple(bgr)], 2, false_color_image)
# save reconstructed grayscale image
cv2.imwrite('gray_image_reconstructed.png', gray_image)
# compare reconstructed and original gray images for differences
print('Number of pixels different:', np.sum(np.abs(im_gray - gray_image) > 0))

The other answer works if you have exact color values.
If your colors have been compressed lossily (JPEG), you need a different approach.
Here's an approach using FLANN. It finds the nearest color and tells you the difference too, so you can handle implausible values.
complete notebook: https://gist.github.com/crackwitz/ccd54145bec1297ccdd4a0c8f4971deb
Highlights:
norm = cv.NORM_L2
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)
fm = cv.FlannBasedMatcher(index_params, search_params)
# JET, BGR order, excluding special palette values (>= 256)
fm.add(255 * np.float32([jet._lut[:256, (2,1,0)]])) # jet
fm.train()
# look up all pixels
query = im.reshape((-1, 3)).astype(np.float32)
matches = fm.match(query)
# statistics: `result` is palette indices ("grayscale image")
output = np.uint16([m.trainIdx for m in matches]).reshape(height, width)
result = np.where(output < 256, output, 0).astype(np.uint8)
dist = np.uint8([m.distance for m in matches]).reshape(height, width)
Source of colormapped picture: Separating Object Contours OpenCV

Have been facing a similar problem while working with a served JPEG compressed image. Since I am on c++, resorting to matplotlib is not an option.
The alternative is to fetch one of the lookup tables (lut) corresponding to the desired colormap, e.g. "jet", available in the imgproc/src/colormap.cpp source file. Unfortunately, what could be easily retrieved using cv::colormap::Jet(n) (where 'n' would even allow to interpolate more points) is not accessible through OpenCV's API.
That said, here is my solution based on #Christoph Rackwitz's answer:
// GNU Octave colormap "jet" as in cv::colormap::Jet()._lut
const float r[] = { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.00588235294117645f,0.02156862745098032f,0.03725490196078418f,0.05294117647058827f,0.06862745098039214f,0.084313725490196f,0.1000000000000001f,0.115686274509804f,0.1313725490196078f,0.1470588235294117f,0.1627450980392156f,0.1784313725490196f,0.1941176470588235f,0.2098039215686274f,0.2254901960784315f,0.2411764705882353f,0.2568627450980392f,0.2725490196078431f,0.2882352941176469f,0.303921568627451f,0.3196078431372549f,0.3352941176470587f,0.3509803921568628f,0.3666666666666667f,0.3823529411764706f,0.3980392156862744f,0.4137254901960783f,0.4294117647058824f,0.4450980392156862f,0.4607843137254901f,0.4764705882352942f,0.4921568627450981f,0.5078431372549019f,0.5235294117647058f,0.5392156862745097f,0.5549019607843135f,0.5705882352941174f,0.5862745098039217f,0.6019607843137256f,0.6176470588235294f,0.6333333333333333f,0.6490196078431372f,0.664705882352941f,0.6803921568627449f,0.6960784313725492f,0.7117647058823531f,0.7274509803921569f,0.7431372549019608f,0.7588235294117647f,0.7745098039215685f,0.7901960784313724f,0.8058823529411763f,0.8215686274509801f,0.8372549019607844f,0.8529411764705883f,0.8686274509803922f,0.884313725490196f,0.8999999999999999f,0.9156862745098038f,0.9313725490196076f,0.947058823529412f,0.9627450980392158f,0.9784313725490197f,0.9941176470588236f,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.9862745098039216f,0.9705882352941178f,0.9549019607843139f,0.93921568627451f,0.9235294117647062f,0.9078431372549018f,0.892156862745098f,0.8764705882352941f,0.8607843137254902f,0.8450980392156864f,0.8294117647058825f,0.8137254901960786f,0.7980392156862743f,0.7823529411764705f,0.7666666666666666f,0.7509803921568627f,0.7352941176470589f,0.719607843137255f,0.7039215686274511f,0.6882352941176473f,0.6725490196078434f,0.6568627450980391f,0.6411764705882352f,0.6254901960784314f,0.6098039215686275f,0.5941176470588236f,0.5784313725490198f,0.5627450980392159f,0.5470588235294116f,0.5313725490196077f,0.5156862745098039f,0.5f };
const float g[] = { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.001960784313725483f,0.01764705882352935f,0.03333333333333333f,0.0490196078431373f,0.06470588235294117f,0.08039215686274503f,0.09607843137254901f,0.111764705882353f,0.1274509803921569f,0.1431372549019607f,0.1588235294117647f,0.1745098039215687f,0.1901960784313725f,0.2058823529411764f,0.2215686274509804f,0.2372549019607844f,0.2529411764705882f,0.2686274509803921f,0.2843137254901961f,0.3f,0.3156862745098039f,0.3313725490196078f,0.3470588235294118f,0.3627450980392157f,0.3784313725490196f,0.3941176470588235f,0.4098039215686274f,0.4254901960784314f,0.4411764705882353f,0.4568627450980391f,0.4725490196078431f,0.4882352941176471f,0.503921568627451f,0.5196078431372548f,0.5352941176470587f,0.5509803921568628f,0.5666666666666667f,0.5823529411764705f,0.5980392156862746f,0.6137254901960785f,0.6294117647058823f,0.6450980392156862f,0.6607843137254901f,0.6764705882352942f,0.692156862745098f,0.7078431372549019f,0.723529411764706f,0.7392156862745098f,0.7549019607843137f,0.7705882352941176f,0.7862745098039214f,0.8019607843137255f,0.8176470588235294f,0.8333333333333333f,0.8490196078431373f,0.8647058823529412f,0.8803921568627451f,0.8960784313725489f,0.9117647058823528f,0.9274509803921569f,0.9431372549019608f,0.9588235294117646f,0.9745098039215687f,0.9901960784313726f,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.9901960784313726f,0.9745098039215687f,0.9588235294117649f,0.943137254901961f,0.9274509803921571f,0.9117647058823528f,0.8960784313725489f,0.8803921568627451f,0.8647058823529412f,0.8490196078431373f,0.8333333333333335f,0.8176470588235296f,0.8019607843137253f,0.7862745098039214f,0.7705882352941176f,0.7549019607843137f,0.7392156862745098f,0.723529411764706f,0.7078431372549021f,0.6921568627450982f,0.6764705882352944f,0.6607843137254901f,0.6450980392156862f,0.6294117647058823f,0.6137254901960785f,0.5980392156862746f,0.5823529411764707f,0.5666666666666669f,0.5509803921568626f,0.5352941176470587f,0.5196078431372548f,0.503921568627451f,0.4882352941176471f,0.4725490196078432f,0.4568627450980394f,0.4411764705882355f,0.4254901960784316f,0.4098039215686273f,0.3941176470588235f,0.3784313725490196f,0.3627450980392157f,0.3470588235294119f,0.331372549019608f,0.3156862745098041f,0.2999999999999998f,0.284313725490196f,0.2686274509803921f,0.2529411764705882f,0.2372549019607844f,0.2215686274509805f,0.2058823529411766f,0.1901960784313728f,0.1745098039215689f,0.1588235294117646f,0.1431372549019607f,0.1274509803921569f,0.111764705882353f,0.09607843137254912f,0.08039215686274526f,0.06470588235294139f,0.04901960784313708f,0.03333333333333321f,0.01764705882352935f,0.001960784313725483f,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 };
const float b[] = { 0.5f,0.5156862745098039f,0.5313725490196078f,0.5470588235294118f,0.5627450980392157f,0.5784313725490196f,0.5941176470588235f,0.6098039215686275f,0.6254901960784314f,0.6411764705882352f,0.6568627450980392f,0.6725490196078432f,0.6882352941176471f,0.7039215686274509f,0.7196078431372549f,0.7352941176470589f,0.7509803921568627f,0.7666666666666666f,0.7823529411764706f,0.7980392156862746f,0.8137254901960784f,0.8294117647058823f,0.8450980392156863f,0.8607843137254902f,0.8764705882352941f,0.892156862745098f,0.907843137254902f,0.9235294117647059f,0.9392156862745098f,0.9549019607843137f,0.9705882352941176f,0.9862745098039216f,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.9941176470588236f,0.9784313725490197f,0.9627450980392158f,0.9470588235294117f,0.9313725490196079f,0.915686274509804f,0.8999999999999999f,0.884313725490196f,0.8686274509803922f,0.8529411764705883f,0.8372549019607844f,0.8215686274509804f,0.8058823529411765f,0.7901960784313726f,0.7745098039215685f,0.7588235294117647f,0.7431372549019608f,0.7274509803921569f,0.7117647058823531f,0.696078431372549f,0.6803921568627451f,0.6647058823529413f,0.6490196078431372f,0.6333333333333333f,0.6176470588235294f,0.6019607843137256f,0.5862745098039217f,0.5705882352941176f,0.5549019607843138f,0.5392156862745099f,0.5235294117647058f,0.5078431372549019f,0.4921568627450981f,0.4764705882352942f,0.4607843137254903f,0.4450980392156865f,0.4294117647058826f,0.4137254901960783f,0.3980392156862744f,0.3823529411764706f,0.3666666666666667f,0.3509803921568628f,0.335294117647059f,0.3196078431372551f,0.3039215686274508f,0.2882352941176469f,0.2725490196078431f,0.2568627450980392f,0.2411764705882353f,0.2254901960784315f,0.2098039215686276f,0.1941176470588237f,0.1784313725490199f,0.1627450980392156f,0.1470588235294117f,0.1313725490196078f,0.115686274509804f,0.1000000000000001f,0.08431372549019622f,0.06862745098039236f,0.05294117647058805f,0.03725490196078418f,0.02156862745098032f,0.00588235294117645f,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 };
// Declare the lut (please note it has to be a 3xN Mat of CV_32F)
int N(sizeof(r)/sizeof(float));
cv::Mat lut(cv::Mat(cv::Size(3,N), CV_32F, cv::Scalar(0.0)));
for (int i(0); i < N; ++i) {
lut.at<float>(i, 0, 0) = 255.0 * b[i];
lut.at<float>(i, 1, 0) = 255.0 * g[i];
lut.at<float>(i, 2, 0) = 255.0 * r[i];
}
// Initialize the FlannBasedMatcher
auto index_params = new cv::flann::KDTreeIndexParams(5);
auto search_params = new cv::flann::SearchParams(50);
cv::FlannBasedMatcher matcher(index_params, search_params);
matcher.add(lut);
matcher.train();
// Convert the image pixels to perform the query (3xH*W Mat of CV_32F)
int QLEN(im.rows*im.cols);
cv::Mat query(cv::Mat(cv::Size(3, QLEN), CV_32F, cv::Scalar(0.0)));
int i(0);
for (int y(0); y < im.rows; ++y) {
for (int x(0); x < im.cols; ++x) {
query.at<float>(i, 0) = float(im.at<cv::Vec3b>(y, x)[0]);
query.at<float>(i, 1) = float(im.at<cv::Vec3b>(y, x)[1]);
query.at<float>(i, 2) = float(im.at<cv::Vec3b>(y, x)[2]);
++i;
}
}
// Lookup all image pixels
std::vector<cv::DMatch> matches;
matcher.match(query, matches);
// Reconstruct the greyscale image
cv::Mat im_grey(cv::Mat(cv::Size(1, QLEN), CV_32F, cv::Scalar(0.0)));
for (int i(0); i < QLEN; ++i) {
im_grey.at<float>(i, 0) = matches[i].trainIdx / 255.0;
}
im_grey = im_grey.reshape(0, {im.rows,im.cols});

Above is brilliant answer from Christoph Rackwitz! But this is a bit confusing due Python Notebook specifics. Here is a full code for conversion.
from matplotlib import colormaps # colormaps['jet'], colormaps['turbo']
from matplotlib.colors import LinearSegmentedColormap
from matplotlib._cm import _jet_data
def convert_jet_to_grey(img):
(height, width) = img.shape[:2]
cm = LinearSegmentedColormap("jet", _jet_data, N=2 ** 8)
# cm = colormaps['turbo'] swap with jet if you use turbo colormap instead
cm._init() # Must be called first. cm._lut data field created here
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)
fm = cv2.FlannBasedMatcher(index_params, search_params)
# JET, BGR order, excluding special palette values (>= 256)
fm.add(255 * np.float32([cm._lut[:256, (2, 1, 0)]])) # jet
fm.train()
# look up all pixels
query = img.reshape((-1, 3)).astype(np.float32)
matches = fm.match(query)
# statistics: `result` is palette indices ("grayscale image")
output = np.uint16([m.trainIdx for m in matches]).reshape(height, width)
result = np.where(output < 256, output, 0).astype(np.uint8)
# dist = np.uint8([m.distance for m in matches]).reshape(height, width)
return result # , dist uncomment if you wish accuracy image

To find the number of circles in an image using OpenCV

I have an image as below :
Can anyone tell me how to detect the number of circles in it.I'm using Hough circle transform to achieve this and this is my code:
# import the necessary packages
import numpy as np
import sys
import cv2
# load the image, clone it for output, and then convert it to grayscale
image = cv2.imread(str(sys.argv[1]))
output = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# detect circles in the image
circles = cv2.HoughCircles(gray, cv2.cv.CV_HOUGH_GRADIENT, 1.2, 5)
no_of_circles = 0
# ensure at least some circles were found
if circles is not None:
# convert the (x, y) coordinates and radius of the circles to integers
circles = np.round(circles[0, :]).astype("int")
no_of_circles = len(circles)
# loop over the (x, y) coordinates and radius of the circles
for (x, y, r) in circles:
# draw the circle in the output image, then draw a rectangle
# corresponding to the center of the circle
cv2.circle(output, (x, y), r, (0, 255, 0), 4)
cv2.rectangle(output, (x - 5, y - 5), (x + 5, y + 5), (0, 128, 255), -1)
# show the output image
cv2.imshow("output", np.hstack([image, output]))
print 'no of circles',no_of_circles
I'm getting wrong answers for this code.Can anyone tell me where I went wrong?

i tried a tricky way to detect all circles.
i found HoughCircles parameters manually
HoughCircles( src_gray, circles, HOUGH_GRADIENT, 1, 50, 40, 46, 0, 0 );
the tricky part is
flip( src, flipped, 1 );
hconcat( src,flipped, flipped );
hconcat( flipped, src, src );
flip( src, flipped, 0 );
vconcat( src,flipped, flipped );
vconcat( flipped, src, src );
flip( src, src, -1 );
will create a model like below before detection.
the result is like this
the c++ code can be easily converted to python
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
Mat src, src_gray, flipped, display;
if (argc < 2)
{
std::cerr<<"No input image specified\n";
return -1;
}
// Read the image
src = imread( argv[1], 1 );
if( src.empty() )
{
std::cerr<<"Invalid input image\n";
return -1;
}
flip( src, flipped, 1 );
hconcat( src,flipped, flipped );
hconcat( flipped, src, src );
flip( src, flipped, 0 );
vconcat( src,flipped, flipped );
vconcat( flipped, src, src );
flip( src, src, -1 );
// Convert it to gray
cvtColor( src, src_gray, COLOR_BGR2GRAY );
// Reduce the noise so we avoid false circle detection
GaussianBlur( src_gray, src_gray, Size(9, 9), 2, 2 );
// will hold the results of the detection
std::vector<Vec3f> circles;
// runs the actual detection
HoughCircles( src_gray, circles, HOUGH_GRADIENT, 1, 50, 40, 46, 0, 0 );
// clone the colour, input image for displaying purposes
display = src.clone();
Rect rect_src(display.cols / 3, display.rows / 3, display.cols / 3, display.rows / 3 );
rectangle( display, rect_src, Scalar(255,0,0) );
for( size_t i = 0; i < circles.size(); i++ )
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
Rect r = Rect( center.x-radius, center.y-radius, radius * 2, radius * 2 );
Rect intersection_rect = r & rect_src;
if( intersection_rect.width * intersection_rect.height > r.width * r.height / 3 )
{
// circle center
circle( display, center, 3, Scalar(0,255,0), -1, 8, 0 );
// circle outline
circle( display, center, radius, Scalar(0,0,255), 3, 8, 0 );
}
}
// shows the results
imshow( "results", display(rect_src));
// get user key
waitKey();
return 0;
}

This SO post describes detection of semi-circles, and may be a good start for you:
Detect semi-circle in opencv
If you get stuck in OpenCV, try coding up the solution yourself. Writing a Hough circle finder parameterized for your particular application is relatively straightforward. If you write application-specific Hough algorithms a few times, you should be able to write a reasonable solution in less time than it takes to sort through a bunch of google results, decipher someone else's code, and so on.
You definitely don't need Canny edge detection for an image like this, but it won't hurt.
Other libraries (esp. commercial ones) will allow you to set more parameters for Hough circle finding. I would've expected some overload of the HoughCircle function to allow a struct of search parameters to be passed in, including the minimum percentage of circle completeness (arc length) allowed.
Although it's good to learn both RANSAC and Hough techniques--and, over time, more exotic techniques--I wouldn't necessarily recommend using RANSAC when you have circles defined so nicely and crisply. Without offering specific evidence, I'll just claim that fiddling with RANSAC parameters may be less intuitive than fiddling with Hough parameters.

HoughCircles needs some parameter tuning to work properly.
It could be that in your case the default values of Param1 and Param2 (set to 100) are not good.

You can fine tune your detection with HoughCircle, by computing the ultimate eroded. It will give you the number of circles in your image.

If there are only circles and background on the input you can count the number of connected components and ignore the component associated with background. This will be the simplest and most robust solution

Finding Grayscale or Color images with Opencv python

I would like to automate and filter out grayscale & color images with the help of openCV python. I've tried to run histogram on color & grayscale images, find the result below
Tried Code:
import cv2
import numpy as np
import sys
img = cv2.imread(sys.argv[1])
h = np.zeros((300,256,3))
bins = np.arange(256).reshape(256,1)
color = [ (255,0,0),(0,255,0),(0,0,255) ]
for ch, col in enumerate(color):
hist_item = cv2.calcHist([img],[ch],None,[256],[0,256])
cv2.normalize(hist_item,hist_item,0,255,cv2.NORM_MINMAX)
hist=np.int32(np.around(hist_item))
pts = np.column_stack((bins,hist))
cv2.polylines(h,[pts],False,col)
h=np.flipud(h)
cv2.imshow('colorhist',h)
cv2.waitKey(0)
Can I automate the same without creating the histogram chart for each file?

Expanding on the channel comparisons above, using numpy array slicing and assuming image is RGB or HSV like colorspace:
def isbw(img):
#img is a numpy.ndarray, loaded using cv2.imread
if len(img.shape) > 2:
looks_like_rgbbw = not False in ((img[:,:,0:1] == img[:,:,1:2]) == (img[:,:,1:2] == img[:,:,2:3]))
looks_like_hsvbw = not (True in (img[:,:,0:1] > 0) or True in (img[:,:,1:2] > 0))
return looks_like_rgbbw or looks_like_hsvbw
else:
return True
Easy to expand to check other colorspace conditions.
Not extensively tested for 'edge/outlier' cases (e.g. other possible formats). Will fail for a red channel only (BGR) image as this will look like a black and white HSV image, so trusting to the cv2 cvtColor conversion to BGR format may be better depending on image subjects. Other 'edge' cases may exist.

here is a sample c++ code to determine if the image is color or grayscale. i think you can easily convert it to python.
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include "iostream"
using namespace cv;
bool isGrayImage( Mat img ) // returns true if the given 3 channel image is B = G = R
{
Mat dst;
Mat bgr[3];
split( img, bgr );
absdiff( bgr[0], bgr[1], dst );
if(countNonZero( dst ))
return false;
absdiff( bgr[0], bgr[2], dst );
return !countNonZero( dst );
}
int main(int argc, char** argv)
{
static const char* str[] = {" is a COLOR image"," is a GRAY image"};
char* filename = argc >= 2 ? argv[1] : (char*)"fruits.jpg";
Mat src = imread(filename);
if(src.data)
{
std::cout << filename << str[isGrayImage( src )] << std::endl;
imshow(filename, src );
waitKey();
}
return 0;
}

You can load image as color using CV_LOAD_IMAGE_COLOR (1) flag (imread documentation):
img = cv2.imread(sys.argv[1], 1)
and then check if image has the same pixel values for red, green, blue channels for every pixel.
for x in range(0, width):
for y in range(0, height):
if img[x, y, 0] == img[x, y, 1] == img[x, y, 2]:
# needs to be true for all pixels
else:
# not grayscale
You can also try to use channels method when loading image using CV_LOAD_IMAGE_ANYDEPTH flag.

Zero keypoints detection using ORB descriptor

I am trying to calculate ORB (Oriented FAST and Rotated BRIEF) features for a database of images. The nexr task is to use a Bag Of Words approach in order to calculate the final features of images. My problem is that in some cases I get 0 keypoints from images of the database (either in ORB or in BRISK implementation). My code is from here.
img = cv2.imread('D:/_DATABASES/clothes_second/striped_141.descr',0)
orb = cv2.ORB()
kp = orb.detect(img,None)
kp, des = orb.compute(img, kp)
img2 = cv2.drawKeypoints(img,kp,color=(0,255,0), flags=0)
plt.imshow(img2),plt.show()
What could be done here, at least orb find one keypoint? How is it possible to use dense sampling for those cases?

You can use a dense feature detector, like the one implemented in C++: http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_feature_detectors.html#densefeaturedetector
The thing is, I'm not sure if that has been ported to python yet. But, since the algorithm is not so hard, you could implement it yourself. Here is the implementation in C++:
void DenseFeatureDetector::detectImpl( const Mat& image, vector<KeyPoint>& keypoints, const Mat& mask ) const
{
float curScale = static_cast<float>(initFeatureScale);
int curStep = initXyStep;
int curBound = initImgBound;
for( int curLevel = 0; curLevel < featureScaleLevels; curLevel++ )
{
for( int x = curBound; x < image.cols - curBound; x += curStep )
{
for( int y = curBound; y < image.rows - curBound; y += curStep )
{
keypoints.push_back( KeyPoint(static_cast<float>(x), static_cast<float>(y), curScale) );
}
}
curScale = static_cast<float>(curScale * featureScaleMul);
if( varyXyStepWithScale ) curStep = static_cast<int>( curStep * featureScaleMul + 0.5f );
if( varyImgBoundWithScale ) curBound = static_cast<int>( curBound * featureScaleMul + 0.5f );
}
KeyPointsFilter::runByPixelsMask( keypoints, mask );
}
However, as you will notice, this implementation does not deal with the angle of the keypoints. That can be a problem if your images have rotation.

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

Reusing models from grabcut in OpenCV - python

Related

Azure Kinect - c++ green screen example in Python - How to create an openCV mask from a depth image?

Opencv: Jetmap or colormap to grayscale, reverse applyColorMap()

To find the number of circles in an image using OpenCV

Finding Grayscale or Color images with Opencv python

Zero keypoints detection using ORB descriptor

Categories

Resources