I've been trying to blend two images. The current approach I'm taking is, I obtain the coordinates of the overlapping region of the two images, and only for the overlapping regions, I blend with a hardcoded alpha of 0.5, before adding it. SO basically I'm just taking half the value of each pixel from overlapping regions of both the images, and adding them. That doesn't give me a perfect blend because the alpha value is hardcoded to 0.5. Here's the result of blending of 3 images:
As you can see, the transition from one image to another is still visible. How do I obtain the perfect alpha value that would eliminate this visible transition? Or is there no such thing, and I'm taking a wrong approach?
Here's how I'm currently doing the blending:
for i in range(3):
base_img_warp[overlap_coords[0], overlap_coords[1], i] = base_img_warp[overlap_coords[0], overlap_coords[1],i]*0.5
next_img_warp[overlap_coords[0], overlap_coords[1], i] = next_img_warp[overlap_coords[0], overlap_coords[1],i]*0.5
final_img = cv2.add(base_img_warp, next_img_warp)
If anyone would like to give it a shot, here are two warped images, and the mask of their overlapping region: http://imgur.com/a/9pOsQ
Here is the way I would do it in general:
int main(int argc, char* argv[])
{
cv::Mat input1 = cv::imread("C:/StackOverflow/Input/pano1.jpg");
cv::Mat input2 = cv::imread("C:/StackOverflow/Input/pano2.jpg");
// compute the vignetting masks. This is much easier before warping, but I will try...
// it can be precomputed, if the size and position of your ROI in the image doesnt change and can be precomputed and aligned, if you can determine the ROI for every image
// the compression artifacts make it a little bit worse here, I try to extract all the non-black regions in the images.
cv::Mat mask1;
cv::inRange(input1, cv::Vec3b(10, 10, 10), cv::Vec3b(255, 255, 255), mask1);
cv::Mat mask2;
cv::inRange(input2, cv::Vec3b(10, 10, 10), cv::Vec3b(255, 255, 255), mask2);
// now compute the distance from the ROI border:
cv::Mat dt1;
cv::distanceTransform(mask1, dt1, CV_DIST_L1, 3);
cv::Mat dt2;
cv::distanceTransform(mask2, dt2, CV_DIST_L1, 3);
// now you can use the distance values for blending directly. If the distance value is smaller this means that the value is worse (your vignetting becomes worse at the image border)
cv::Mat mosaic = cv::Mat(input1.size(), input1.type(), cv::Scalar(0, 0, 0));
for (int j = 0; j < mosaic.rows; ++j)
for (int i = 0; i < mosaic.cols; ++i)
{
float a = dt1.at<float>(j, i);
float b = dt2.at<float>(j, i);
float alpha = a / (a + b); // distances are not between 0 and 1 but this value is. The "better" a is, compared to b, the higher is alpha.
// actual blending: alpha*A + beta*B
mosaic.at<cv::Vec3b>(j, i) = alpha*input1.at<cv::Vec3b>(j, i) + (1 - alpha)* input2.at<cv::Vec3b>(j, i);
}
cv::imshow("mosaic", mosaic);
cv::waitKey(0);
return 0;
}
Basically you compute the distance from your ROI border to the center of your objects and compute the alpha from both blending mask values. So if one image has a high distance from the border and other one a low distance from border, you prefer the pixel that is closer to the image center. It would be better to normalize those values for cases where the warped images aren't of similar size.
But even better and more efficient is to precompute the blending masks and warp them. Best would be to know the vignetting of your optical system and choose and identical blending mask (typically lower values of the border).
From the previous code you'll get these results:
ROI masks:
Blending masks (just as an impression, must be float matrices instead):
image mosaic:
There are 2 obvious problems with your images:
Border area has distorted lighting conditions
That is most likely caused by the optics used to acquire images. So to remedy that you should use only inside part of the images (cut off few pixels from border.
So when cut off 20 pixels from the border and blending to common illumination I got this:
As you can see the ugly border seam is away now only the illumination problems persists (see bullet #2).
Images are taken at different lighting conditions
Here the subsurface scattering effects hits in making the images "not-compatible". You should normalize them to some uniform illumination or post process the blended result line by line and when coherent bump detected multiply the rest of line so the bump will be diminished.
So the rest of the line should be multiplied by constant i0/i1. These kind if bumps can occur only on the edges between overlap values so you can either scan for them or use those positions directly ... To recognize valid bump it should have neighbors nearby in previous and next lines along the whole image height.
You can do this also in y axis direction in the same way ...
Related
I'm writing for Android with OpenCV. I'm segmenting an image similar to below using marker-controlled watershed, without the user manually marking the image. I'm planning to use the regional maxima as markers.
minMaxLoc() would give me the value, but how can I restrict it to the blobs which is what I'm interested in? Can I utilize the results from findContours() or cvBlob blobs to restrict the ROI and apply maxima to each blob?
First of all: the function minMaxLoc finds only the global minimum and global maximum for a given input, so it is mostly useless for determining regional minima and/or regional maxima. But your idea is right, extracting markers based on regional minima/maxima for performing a Watershed Transform based on markers is totally fine. Let me try to clarify what is the Watershed Transform and how you should correctly use the implementation present in OpenCV.
Some decent amount of papers that deal with watershed describe it similarly to what follows (I might miss some detail, if you are unsure: ask). Consider the surface of some region you know, it contains valleys and peaks (among other details that are irrelevant for us here). Suppose below this surface all you have is water, colored water. Now, make holes in each valley of your surface and then the water starts to fill all the area. At some point, differently colored waters will meet, and when this happen, you construct a dam such that they don't touch each other. In the end you have a collection of dams, which is the watershed separating all the different colored water.
Now, if you make too many holes in that surface, you end up with too many regions: over-segmentation. If you make too few you get an under-segmentation. So, virtually any paper that suggests using watershed actually presents techniques to avoid these problems for the application the paper is dealing with.
I wrote all this (which is possibly too naïve for anyone that knows what the Watershed Transform is) because it reflects directly on how you should use watershed implementations (which the current accepted answer is doing in a completely wrong manner). Let us start on the OpenCV example now, using the Python bindings.
The image presented in the question is composed of many objects that are mostly too close and in some instances overlapping. The usefulness of watershed here is to separate correctly these objects, not to group them into a single component. So you need at least one marker for each object and good markers for the background. As an example, first binarize the input image by Otsu and perform a morphological opening for removing small objects. The result of this step is shown below in the left image. Now with the binary image consider applying the distance transform to it, result at right.
With the distance transform result, we can consider some threshold such that we consider only the regions most distant to the background (left image below). Doing this, we can obtain a marker for each object by labeling the different regions after the earlier threshold. Now, we can also consider the border of a dilated version of the left image above to compose our marker. The complete marker is shown below at right (some markers are too dark to be seen, but each white region in the left image is represented at the right image).
This marker we have here makes a lot of sense. Each colored water == one marker will start to fill the region, and the watershed transformation will construct dams to impede that the different "colors" merge. If we do the transform, we get the image at left. Considering only the dams by composing them with the original image, we get the result at right.
import sys
import cv2
import numpy
from scipy.ndimage import label
def segment_on_dt(a, img):
border = cv2.dilate(img, None, iterations=5)
border = border - cv2.erode(border, None)
dt = cv2.distanceTransform(img, 2, 3)
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)
_, dt = cv2.threshold(dt, 180, 255, cv2.THRESH_BINARY)
lbl, ncc = label(dt)
lbl = lbl * (255 / (ncc + 1))
# Completing the markers now.
lbl[border == 255] = 255
lbl = lbl.astype(numpy.int32)
cv2.watershed(a, lbl)
lbl[lbl == -1] = 0
lbl = lbl.astype(numpy.uint8)
return 255 - lbl
img = cv2.imread(sys.argv[1])
# Pre-processing.
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, img_bin = cv2.threshold(img_gray, 0, 255,
cv2.THRESH_OTSU)
img_bin = cv2.morphologyEx(img_bin, cv2.MORPH_OPEN,
numpy.ones((3, 3), dtype=int))
result = segment_on_dt(img, img_bin)
cv2.imwrite(sys.argv[2], result)
result[result != 255] = 0
result = cv2.dilate(result, None)
img[result == 255] = (0, 0, 255)
cv2.imwrite(sys.argv[3], img)
I would like to explain a simple code on how to use watershed here. I am using OpenCV-Python, but i hope you won't have any difficulty to understand.
In this code, I will be using watershed as a tool for foreground-background extraction. (This example is the python counterpart of the C++ code in OpenCV cookbook). This is a simple case to understand watershed. Apart from that, you can use watershed to count the number of objects in this image. That will be a slightly advanced version of this code.
1 - First we load our image, convert it to grayscale, and threshold it with a suitable value. I took Otsu's binarization, so it would find the best threshold value.
import cv2
import numpy as np
img = cv2.imread('sofwatershed.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
Below is the result I got:
( even that result is good, because great contrast between foreground and background images)
2 - Now we have to create the marker. Marker is the image with same size as that of original image which is 32SC1 (32 bit signed single channel).
Now there will be some regions in the original image where you are simply sure, that part belong to foreground. Mark such region with 255 in marker image. Now the region where you are sure to be the background are marked with 128. The region you are not sure are marked with 0. That is we are going to do next.
A - Foreground region:- We have already got a threshold image where pills are white color. We erode them a little, so that we are sure remaining region belongs to foreground.
fg = cv2.erode(thresh,None,iterations = 2)
fg :
B - Background region :- Here we dilate the thresholded image so that background region is reduced. But we are sure remaining black region is 100% background. We set it to 128.
bgt = cv2.dilate(thresh,None,iterations = 3)
ret,bg = cv2.threshold(bgt,1,128,1)
Now we get bg as follows :
C - Now we add both fg and bg :
marker = cv2.add(fg,bg)
Below is what we get :
Now we can clearly understand from above image, that white region is 100% foreground, gray region is 100% background, and black region we are not sure.
Then we convert it into 32SC1 :
marker32 = np.int32(marker)
3 - Finally we apply watershed and convert result back into uint8 image:
cv2.watershed(img,marker32)
m = cv2.convertScaleAbs(marker32)
m :
4 - We threshold it properly to get the mask and perform bitwise_and with the input image:
ret,thresh = cv2.threshold(m,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
res = cv2.bitwise_and(img,img,mask = thresh)
res :
Hope it helps!!!
ARK
Foreword
I'm chiming in mostly because I found both the watershed tutorial in the OpenCV documentation (and C++ example) as well as mmgp's answer above to be quite confusing. I revisited a watershed approach multiple times to ultimately give up out of frustration. I finally realized I needed to at least give this approach a try and see it in action. This is what I've come up with after sorting out all of the tutorials I've come across.
Aside from being a computer vision novice, most of my trouble probably had to do with my requirement to use the OpenCVSharp library rather than Python. C# doesn't have baked-in high-power array operators like those found in NumPy (though I realize this has been ported via IronPython), so I struggled quite a bit in both understanding and implementing these operations in C#. Also, for the record, I really despise the nuances of, and inconsistencies in most of these function calls. OpenCVSharp is one of the most fragile libraries I've ever worked with. But hey, it's a port, so what was I expecting? Best of all, though -- it's free.
Without further ado, let's talk about my OpenCVSharp implementation of the watershed, and hopefully clarify some of the stickier points of watershed implementation in general.
Application
First of all, make sure watershed is what you want and understand its use. I am using stained cell plates, like this one:
It took me a good while to figure out I couldn't just make one watershed call to differentiate every cell in the field. On the contrary, I first had to isolate a portion of the field, then call watershed on that small portion. I isolated my region of interest (ROI) via a number of filters, which I will explain briefly here:
Start with source image (left, cropped for demonstration purposes)
Isolate the red channel (left middle)
Apply adaptive threshold (right middle)
Find contours then eliminate those with small areas (right)
Once we have cleaned the contours resulting from the above thresholding operations, it is time to find candidates for watershed. In my case, I simply iterated through all contours greater than a certain area.
Code
Say we've isolated this contour from the above field as our ROI:
Let's take a look at how we'll code up a watershed.
We'll start with a blank mat and draw only the contour defining our ROI:
var isolatedContour = new Mat(source.Size(), MatType.CV_8UC1, new Scalar(0, 0, 0));
Cv2.DrawContours(isolatedContour, new List<List<Point>> { contour }, -1, new Scalar(255, 255, 255), -1);
In order for the watershed call to work, it will need a couple of "hints" about the ROI. If you're a complete beginner like me, I recommend checking out the CMM watershed page for a quick primer. Suffice to say we're going to create hints about the ROI on the left by creating the shape on the right:
To create the white part (or "background") of this "hint" shape, we'll just Dilate the isolated shape like so:
var kernel = Cv2.GetStructuringElement(MorphShapes.Ellipse, new Size(2, 2));
var background = new Mat();
Cv2.Dilate(isolatedContour, background, kernel, iterations: 8);
To create the black part in the middle (or "foreground"), we'll use a distance transform followed by threshold, which takes us from the shape on the left to the shape on the right:
This takes a few steps, and you may need to play around with the lower bound of your threshold to get results that work for you:
var foreground = new Mat(source.Size(), MatType.CV_8UC1);
Cv2.DistanceTransform(isolatedContour, foreground, DistanceTypes.L2, DistanceMaskSize.Mask5);
Cv2.Normalize(foreground, foreground, 0, 1, NormTypes.MinMax); //Remember to normalize!
foreground.ConvertTo(foreground, MatType.CV_8UC1, 255, 0);
Cv2.Threshold(foreground, foreground, 150, 255, ThresholdTypes.Binary);
Then we'll subtract these two mats to get the final result of our "hint" shape:
var unknown = new Mat(); //this variable is also named "border" in some examples
Cv2.Subtract(background, foreground, unknown);
Again, if we Cv2.ImShow unknown, it would look like this:
Nice! This was easy for me to wrap my head around. The next part, however, got me quite puzzled. Let's look at turning our "hint" into something the Watershed function can use. For this we need to use ConnectedComponents, which is basically a big matrix of pixels grouped by the virtue of their index. For example, if we had a mat with the letters "HI", ConnectedComponents might return this matrix:
0 0 0 0 0 0 0 0 0
0 1 0 1 0 2 2 2 0
0 1 0 1 0 0 2 0 0
0 1 1 1 0 0 2 0 0
0 1 0 1 0 0 2 0 0
0 1 0 1 0 2 2 2 0
0 0 0 0 0 0 0 0 0
So, 0 is the background, 1 is the letter "H", and 2 is the letter "I". (If you get to this point and want to visualize your matrix, I recommend checking out this instructive answer.) Now, here's how we'll utilize ConnectedComponents to create the markers (or labels) for watershed:
var labels = new Mat(); //also called "markers" in some examples
Cv2.ConnectedComponents(foreground, labels);
labels = labels + 1;
//this is a much more verbose port of numpy's: labels[unknown==255] = 0
for (int x = 0; x < labels.Width; x++)
{
for (int y = 0; y < labels.Height; y++)
{
//You may be able to just send "int" in rather than "char" here:
var labelPixel = (int)labels.At<char>(y, x); //note: x and y are inexplicably
var borderPixel = (int)unknown.At<char>(y, x); //and infuriatingly reversed
if (borderPixel == 255)
labels.Set(y, x, 0);
}
}
Note that the Watershed function requires the border area to be marked by 0. So, we've set any border pixels to 0 in the label/marker array.
At this point, we should be all set to call Watershed. However, in my particular application, it is useful just to visualize a small portion of the entire source image during this call. This may be optional for you, but I first just mask off a small bit of the source by dilating it:
var mask = new Mat();
Cv2.Dilate(isolatedContour, mask, new Mat(), iterations: 20);
var sourceCrop = new Mat(source.Size(), source.Type(), new Scalar(0, 0, 0));
source.CopyTo(sourceCrop, mask);
And then make the magic call:
Cv2.Watershed(sourceCrop, labels);
Results
The above Watershed call will modify labels in place. You'll have to go back to remembering about the matrix resulting from ConnectedComponents. The difference here is, if watershed found any dams between watersheds, they will be marked as "-1" in that matrix. Like the ConnectedComponents result, different watersheds will be marked in a similar fashion of incrementing numbers. For my purposes, I wanted to store these into separate contours, so I created this loop to split them up:
var watershedContours = new List<Tuple<int, List<Point>>>();
for (int x = 0; x < labels.Width; x++)
{
for (int y = 0; y < labels.Height; y++)
{
var labelPixel = labels.At<Int32>(y, x); //note: x, y switched
var connected = watershedContours.Where(t => t.Item1 == labelPixel).FirstOrDefault();
if (connected == null)
{
connected = new Tuple<int, List<Point>>(labelPixel, new List<Point>());
watershedContours.Add(connected);
}
connected.Item2.Add(new Point(x, y));
if (labelPixel == -1)
sourceCrop.Set(y, x, new Vec3b(0, 255, 255));
}
}
Then, I wanted to print these contours with random colors, so I created the following mat:
var watershed = new Mat(source.Size(), MatType.CV_8UC3, new Scalar(0, 0, 0));
foreach (var component in watershedContours)
{
if (component.Item2.Count < (labels.Width * labels.Height) / 4 && component.Item1 >= 0)
{
var color = GetRandomColor();
foreach (var point in component.Item2)
watershed.Set(point.Y, point.X, color);
}
}
Which yields the following when shown:
If we draw on the source image the dams that were marked by a -1 earlier, we get this:
Edits:
I forgot to note: make sure you're cleaning up your mats after you're done with them. They WILL stay in memory and OpenCVSharp may present with some unintelligible error message. I should really be using using above, but mat.Release() is an option as well.
Also, mmgp's answer above includes this line: dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8), which is a histogram stretching step applied to the results of the distance transform. I omitted this step for a number of reasons (mostly because I didn't think the histograms I saw were too narrow to begin with), but your mileage may vary.
I have two images:
I want to measure how straight/smooth the text borders are rendered.
First image is rendered perfectly straight, so it deserves a quality measure 1. On the other hand, the second image is rendered with a lot of variant curves (rough in a way) that is why it deserves a quality measure less than 1. How will I measure it using image processing or any Python function or any function written in other languages?
Clarification :
There are font styles that are rendered originally with straight strokes but there are also font styles that are rendered smoothly just like the cursive font styles. What I'm really after is to differentiate the text border surface roughness of the characters by giving it a quality measure.
I want to measure how straight/smooth the text borders are rendered in an image.
Inversely, it can also be said that I want to measure how rough the text borders are rendered in an image.
I don't know any python function, but I would:
1) Use potrace to trace the edges and convert them to bezier curves. Here's a vizualisation:
2) Then let's zoom to the top part of the P for example:
You draw lines perpendicular to the curve for a finite length (let's say 100 pixels). You plot the color intensity (you can convert to HSI or HSV and use one of those channels, or just convert to grayscale and take the pixel value directly) over that line:
3) Then you calculate the standard deviation of the derivative. Small standard deviation means sharp edges, large standard deviation means blurry edges. For a perfect edge, the standard deviation would be zero.
4) For every edge were you drew a perpendicular line, you now have a "smoothness" value. You can then average all the smoothness values per edge, per letter, per word or per image, as you see fit. Also, the more perpendicular lines you draw, the more accurate your smoothness value, but the more computationally intensive.
I would try something simple like creating a 'roughness' metric using a few functions from the opencv library, since it's easy to work with in Python (and C++, as well as other wrappers).
For example (without actual source, since I'm typing on my phone):
Preprocess to create binary images (many standard ways).
Use cv2.findContours to get outlines of the letters.
Use cv2.arcLength on each contour as denominators.
Use cv2.approxPolyDP to simplify each contour.
Use cv2.arcLength on each simplified contour as numerators.
Calculate ratios of simplified over full arc lengths.
In step 5, ratios closer to 1.0 require less simplification, so they're presumably less rough. Ratios closer to 0.0 require a lot of simplification, and are therefore probably very rough. Of course, you'll have to tweak the contour finding code to get appropriate outlines to work with, and you'll need to manage numerical precision to keep the math calculations meaningful, but hopefully the idea is clear enough.
OpenCV also has the useful functions cv2.convexHull and cv2.convexityDefects that you might find interesting in related work. However, they didn't seem appropriate for the letters here, since internal features on letters like M for example would be more challenging to address.
Speaking of rough things, I admit this algorithmic outline is incredibly rough! However, I hope it gives you a useful idea to try that seems straightforward to implement quickly to start getting quantitative feedback.
One idea might be simply to get the average of the number of vertices per character in Python/OpenCV using cv2.CHAIN_APPROX_SIMPLE.
Since you have the same characters and you want to know how straight they are, the CHAIN_APPROX_SIMPLE measures only horizontal and vertical corner vertices. For your first image, there should be much fewer vertices than for your second image.
CHAIN_APPROX_SIMPLE compresses horizontal, vertical, and diagonal
segments and leaves only their end points. For example, an up-right
rectangular contour is encoded with 4 points.
import cv2
import numpy as np
# read image
img = cv2.imread('lemper1.png')
#img = cv2.imread('lemper2.png')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
# invert
thresh = 255 - thresh
# get contours and compute average number of vertices per character (contour)
result = img.copy()
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
num_contours = 0
sum = 0
for cntr in contours:
cv2.drawContours(result, [cntr], 0, (0,0,255), 1)
num_vertices = len(cntr)
sum = sum + num_vertices
num_contours = num_contours + 1
smoothness = (sum / num_contours)
print(smoothness)
# save resulting images
cv2.imwrite('lemper1_contours.png',result)
#cv2.imwrite('lemper2_contours.png',result)
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("contours", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
First image average number of vertices: 49.666666666666664
Second image average number of vertices: 179.14285714285714
So smaller number of vertices means straighter characters.
Preface
There's a few nice ideas presented here based around the properties of the character contours as polylines. Whilst there is some inherent flaws in this approach due to them being a function of resolution and scale, I would like to offer one further interruption of the same. My algorithm is still susceptible but it may offer a different perspective.
Theory
The method I propose is to compare common characters by the number of inflections in their contours. In this context, what I mean by inflection, is a sign change between the cross products of successive polyline segments as vector. For example; consider a polyline contour of a circle, starting at the mid y coordinate and the x+ most coordinate. If we were to trace the polyline contour CW (clockwise) around the perimeter, each line segment would be incrementally a CW transform of the prior. If at any time a segment turned "away" or "outwards", this transform would be CCW (counter-clockwise) and the cross product will invert. A "rough" circle will therefore have inflections, a "perfect" or "smooth" circle will have none.
Algorithm
The algorithm follows the steps below using the Emgu.CV. C# code below that:
The images are loaded and converted to binary by means of thresholding
The binary images then undergo contour detection and these contours are sort by their bounding box, left to right, so and their indices match the occurrence order of the character they contour.
Each contour is then re-pointed to an equal number of segments in order to normalize for scale and resolution differences between images/characters.
Each contour is "walked" and its number of inflections counted.
// [Some basic extensions are omitted for clarity]
// Load the images
Image<Rgb, byte> baseLineImage = new Image<Rgb, byte>("BaseLine.png");
Image<Rgb, byte> testCaseImage = new Image<Rgb, byte>("TestCase.png");
// Convert them to Gray Scale
Image<Gray, byte> baseLineGray = baseLineImage.Convert<Gray, byte>();
Image<Gray, byte> testCaseGray = testCaseImage.Convert<Gray, byte>();
// Threshold the images to binary
Image<Gray, byte> baseLineBinary = baseLineGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
Image<Gray, byte> testCaseBinary = testCaseGray.ThresholdBinaryInv(new Gray(100), new Gray(255));
// Some dilation required on the test image so that the characters are continuous
testCaseBinary = testCaseBinary.Dilate(3);
// Extract the the contours from the images to isolate the character profiles
// and sort them left to right so as the indicies match the character order
VectorOfVectorOfPoint baseLineContours = new VectorOfVectorOfPoint();
Mat baseHierarchy = new Mat();
CvInvoke.FindContours(
baseLineBinary,
baseLineContours,
baseHierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple);
var baseLineContoursList = baseLineContours.ToList();
baseLineContoursList.Sort(new ContourComparer());
VectorOfVectorOfPoint testCaseContours = new VectorOfVectorOfPoint();
Mat testHierarchy = new Mat();
CvInvoke.FindContours(
testCaseBinary,
testCaseContours,
testHierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple);
var testCaseContoursList = testCaseContours.ToList();
testCaseContoursList.Sort(new ContourComparer());
var baseLineRepointedContours = RepointContours(baseLineContoursList, 50);
var testCaseRepointedContours = RepointContours(testCaseContoursList, 50);
var baseLineInflectionCounts = GetContourInflections(baseLineRepointedContours);
var testCaseInflectionCounts = GetContourInflections(testCaseRepointedContours);
Inflection Detection/Counting
static List<List<Point>> GetContourInflections(List<VectorOfPoint> contours)
{
// A resultant list to return the inflection points
List<List<Point>> result = new List<List<Point>>();
// Calculate the forward to reverse cross product at each vertex
List<double> crossProducts;
// Points used to store 2D Vectors as X,Y (I,J)
Point priorVector, forwardVector;
foreach (VectorOfPoint contour in contours)
{
crossProducts = new List<double>();
for (int p = 0; p < contour.Size; p++)
{
// Determine the vector to the prior to this vertex
priorVector = p == 0 ?
priorVector = new Point()
{
X = contour[p].X - contour[contour.Size - 1].X,
Y = contour[p].Y - contour[contour.Size - 1].Y
} :
priorVector = new Point()
{
X = contour[p].X - contour[p - 1].X,
Y = contour[p].Y - contour[p - 1].Y
};
// Determine the vector to the next vector
// If this is the lst vertex, loop back to vertex 0
forwardVector = p == contour.Size - 1 ?
new Point()
{
X = contour[0].X - contour[p].X,
Y = contour[0].Y - contour[p].Y,
} :
new Point()
{
X = contour[p + 1].X - contour[p].X,
Y = contour[p + 1].Y - contour[p].Y,
};
// Calculate the cross product of the prior and forward vectors
crossProducts.Add(forwardVector.X * priorVector.Y - forwardVector.Y * priorVector.X);
}
// Given the calculated cross products, detect the inflection points
List<Point> inflectionPoints = new List<Point>();
for (int p = 1; p < contour.Size; p++)
{
// If there is a sign change between this and the prior cross product, an inflection,
// or change from CW to CCW bearing increments has occurred. To and from zero products
// are ignored
if ((crossProducts[p] > 0 && crossProducts[p-1] < 0) ||
(crossProducts[p] < 0 && crossProducts[p-1] > 0))
{
inflectionPoints.Add(contour[p]);
}
}
result.Add(inflectionPoints);
}
return result;
}
Output
L: Baseline Inflections:0 Testcase Inflections:22
E: Baseline Inflections:1 Testcase Inflections:16
M: Baseline Inflections:4 Testcase Inflections:15
P: Baseline Inflections:11 Testcase Inflections:17
E: Baseline Inflections:1 Testcase Inflections:10
R: Baseline Inflections:9 Testcase Inflections:16
Contours (Blue) and Inflections (Red)
I want to detect small squares circled in red on the image. But the problem is that they are on another white line. I want to know how to separate those squares from the white line and detect them.
I have used OpenCV Python to write the code. What I have done until now is that I cropped the image so that I get access only to the circular part of the image. Then I cropped the image to get the required part that is the white line. Then I used erosion so that the white line will vanish and the squares remain in the image. Then used Hough circles to detect the squares. This does work for some images but it cannot be generalized. Please help me in finding a generalized code for this. Let me know the logic and also the python code.
Also could anyone help me detect that aruco marker on the image. Its getting rejected. I dont know why.
Image is in this link. Detect small squares on an image
here's C++ code with distanceTransform.
Since I nearly only used openCV functions, you can probably easily convert it to Python code.
I removed the white stripe at the top of the image manually, hope this isn't a problem.
int main()
{
cv::Mat input = cv::imread("C:/StackOverflow/Input/SQUARES.png", cv::IMREAD_GRAYSCALE);
cv::Mat thres = input > 0; // make binary mas
cv::Mat dst;
cv::distanceTransform(thres, dst, CV_DIST_L2, 3);
double min, max;
cv::Point minPt, maxPt;
cv::minMaxLoc(dst, &min, &max, 0, 0);
double distThres = max*0.65; // a real clustering would be better. This assumes that the white circle thickness is a bout 50% of the square size, so 65% should be ok...
cv::Mat squaresMask = dst >= distThres;
cv::imwrite("C:/StackOverflow/Input/SQUARES_mask.png", squaresMask);
std::vector<std::vector<cv::Point> > contours;
cv::findContours(squaresMask, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_NONE);
cv::Mat output;
cv::cvtColor(input, output, cv::COLOR_GRAY2BGR);
for (int i = 0; i < contours.size(); ++i)
{
cv::Point2f center;
float radius;
cv::minEnclosingCircle(contours[i], center, radius);
cv::circle(output, center, 5, cv::Scalar(255, 0, 255), -1);
//cv::circle(output, center, radius, cv::Scalar(255, 0, 255), 1);
}
cv::imwrite("C:/StackOverflow/Input/SQUARES_output.png", output);
cv::imshow("output", output);
cv::waitKey(0);
}
this is the input:
this it the squaresMask after distance transform
and this is the result
I'm working to detect cells within microscope images like the one below. There are often spurious contours that get drawn due to imperfections on the microscope slides, like the one below the legend in the figure below.
I'm currently using this solution to clean these up. Here's the basic idea.
# Create image of background
blank = np.zeros(image.shape[0:2])
background_image = cv2.drawContours(blank.copy(), background_contour, 0, 1, -1)
for i, c in enumerate(contours):
# Create image of contour
contour_image = cv2.drawContours(blank.copy(), contours, i, 1, -1)
# Create image of focal contour + background
total_image = np.where(background_image+contour_image>0, 1, 0)
# Check if contour is outside postive space
if total_image.sum() > background_image.sum():
continue
This works as expected; if the total_image area is greater than the area of the background_image then c must be outside the region of interest. But drawing all of these contours is incredibly slow and checking thousands of contours takes hours. Is there a more efficient way to check if contours overlap that doesn't require drawing the contours?
I assume the goal is to exclude the external contour from further analysis? If so, the easiest is to use the red background contour as a mask. Then use the masked image to detect the blue cells.
# Create image of background
blank = np.zeros(image.shape[0:2], dtype=np.uint8)
background_image = cv2.drawContours(blank.copy(), background_contour, 0, (255), -1)
# mask input image (leaves only the area inside the red background contour)
res = cv2.bitwise_and(image,image,mask=background_image )
#[detect blue cells]
assuming you are trying to find points on the different contours that are overlaping
consider contour as
vector<vector<Point> > contours;
..... //obtain you contrours.
vector<Point> non_repeating_points;
for(int i=0;i<contours.size();i++)
{
for(int j=0;j<contours[i].size();j++)
{
Point this_point= countour[i][j];
for(int k=0;k<non_repeating_points.size();k++)
{//check this list for previous record
if(non_repeating_points[k] == this_point)
{
std::cout<< "found repeat points at "<< std::endl;
std::cout<< this_point << std::endl;
break;
}
}
//if not seen before just add it in the list
non_repeating_points.push_back(this_point);
}
}
I just wrote it without compile. but I think you can understand the idea.
the information you provide is not enough.
In case you mean to find the nearest connected boundary. And there is no overlapping.
you can declare a local cluster near the point non_repeating_points[k]. Call it surround_non_repeating_points[k];
you can control the distance that can be considered as intercept and push all of them in this surround_non_repeating_points[k];
Then just check in a loop for
if(surround_non_repeating_points[k] == this_point)
I'm trying to implement binary image filter (to get monochrome binary image) using python & PyQT5, and, to retrieve new pixel colors I use the following method:
def _new_pixel_colors(self, x, y):
color = QColor(self.pixmap.pixel(x, y))
result = qRgb(0, 0, 0) if all(c < 127 for c in color.getRgb()[:3]) else qRgb(255, 255, 255)
return result
Could It be a correct sample of binary filter for RGB image? I mean, is that a sufficient condition to check whether the pixel is brighter or darker then (127,127,127) Gray color?
And please, do not provide any solutions with opencv, pillow, etc. I'm only asking about the algorithm itself.
I would at least compare against intensity i=R+G+B ...
For ROI like masks you can use any thresholding techniques (adaptive thresholding is the best) but if your resulting image is not a ROI mask and should resemble the visual features of the original image then the best conversion I know of is to use Dithering.
The Idea behind BW dithering is to convert gray scales into BW patterns preserwing the shading. The result is often noisy but preserves much much more visual details. Here simple naive C++ dithering (sorry not a Python coder):
picture pic0,pic1;
// pic0 - source img
// pic1 - output img
int x,y,i;
color c;
// resize output to source image size clear with black
pic1=pic0; pic1.clear(0);
// dithering
i=0;
for (y=0;y<pic0.ys;y++)
for (x=0;x<pic0.xs;x++)
{
// get source pixel color (AARRGGBB)
c=pic0.p[y][x];
// add to leftovers
i+=WORD(c.db[picture::_r]); // _r,_g,_b are just constants 0,1,2
i+=WORD(c.db[picture::_g]);
i+=WORD(c.db[picture::_b]);
// threshold white intensity is 255+255+255=765
if (i>=384){ i-=765; c.dd=0x00FFFFFF; } else c.dd=0;
// copy to destination image
pic1.p[y][x]=c;
}
So its the same as in the link above but using just black and white. i is the accumulated intensity to be placed on the image. xs,ys is the resolution and c.db[] is color channel access.
If I apply this on colored image like this:
The result looks like this:
As you can see all the details where preserved but a noisy patterns emerge ... For printing purposes was sometimes the resolution of the image multiplied to enhance the quality. If you change the naive 2 nested for loops with a better pattern (like 16x16 squares etc) then the noise will be conserved near its source limiting artifacts. There are also approaches that use pseudo random patterns (put the leftover i near its source pixel in random location) that is even better ...
But for a BW dithering even naive approach is enough as the artifacts are just one pixel in size. For colored dithering the artifacts could create unwanted horizontal line patterns of several pixels in size (depends on used palette mis match the worse palette the bigger artifacts...)
PS just for comparison to other answer threshold outputs this is the same image dithered:
Image thresholding is the class of algorithms you're looking for - a binary threshold would set pixels to 0 or 1, yes.
Depending on the desired output, consider converting your image first to other color spaces, in particular HSL, with the luminance channel. Using (127, 127, 127) as a threshold does not uniformly take brightness into account because each channel of RGB is the saturation of R, G, or B; consider this image:
from PIL import Image
import colorsys
def threshold_pixel(r, g, b):
h, l, s = colorsys.rgb_to_hls(r / 255., g / 255., b / 255.)
return 1 if l > .36 else 0
# return 1 if r > 127 and g > 127 and b > 127 else 0
def hlsify(img):
pixels = img.load()
width, height = img.size
# Create a new blank monochrome image.
output_img = Image.new('1', (width, height), 0)
output_pixels = output_img.load()
for i in range(width):
for j in range(height):
output_pixels[i, j] = threshold_pixel(*pixels[i, j])
return output_img
binarified_img = hlsify(Image.open('./sample_img.jpg'))
binarified_img.show()
binarified_img.save('./out.jpg')
There is lots of discussion on other StackExchange sites on this topic, e.g.
Binarize image data
How do you binarize a colored image?
how can I get good binary image using Otsu method for this image?