Develop Reference

Create a probability map of image with Kernel Density Estimation

I'm working with mammograms that have calcifications (which are brighter spots than the surrounding tissue).
This is one of the images I have:
original image
I got this image by creating a heat map:
heat map of original image
But the heat map considers the overall brightness of the image. To create a probability map that considers the local luminosity I have thought about making a Kernel Density Estimation with a Gaussian filter, but I am having problems with the implementation.
My aim is to get a result similar to scikit-learn's "Kernel Density Estimate of Species Distributions" example. This is the code I tried to use:
import numpy as np
import cv2 as cv
from sklearn.neighbors import KernelDensity
img = cv.imread("mammo.tif", 0)
kde = KernelDensity(kernel="gaussian")
kde.fit(img)
sco = kde.score_samples(img)
The original image has a size of 4084x3328. I would like to get a probability map of the same size, but what I get in the variable sco is a vector of 4084x1 with all negative values.

In kde.fit(img) every row of img is treated as a single observation with 3328 features. Thus you are fitting a kernel density estimate where the kernel is a multivariate Gaussian distribution with 3328 variables. Then kde.score_samples(img) again computes the score for each row of img which results in 4084 values. Moreover, these values are logarithms of probabilities - hence they are negative.

How to use cv2.findHomography() with Squeezed Images

I am using the python bindings for opencv. I am using keypoint detection and description (ie SURF, SIFT,...) to find a template image contained within a target image, but there is a catch: the template can be "squeezed" in the target image, so that the aspect ratio is different than the target image.
This does not work with findHomography(), since it assumes a simple perspective transform, which cannot have this sort of stretching.
Are there any ways to do this? I have thought about incrementally stretching the target image different amounts to change the aspect ratio, and using findHomography at each iteration, but as far as I can tell there is no way of comparing the quality of a fit (since I'm using RANSAC to find the best fit), so I can't tell at which squeeze level it fits best.
Perhaps counting the number of points that matched correctly from the RANSAC by looking at the length of the returned mask? This seems sorta gross.

This does not work with findHomography(), since it assumes a simple perspective transform, which cannot have this sort of stretching.
This is not true; even an affine warp includes stretching the aspect ratios and even shear distortion, and homographies expand this by even non-uniform distortions. For example, the affine transformation given by the matrix
2 0 0
0 1 0
will stretch an image horizontally by a factor of two, as seen with this short program:
import cv2
import numpy as np
img = cv2.imread('lena.png')
affine_warp = np.array([[2, 0, 0], [0, 1, 0]], dtype=np.float32)
dsize = (img.shape[1]*2, img.shape[0])
warped_img = cv2.warpAffine(img, affine_warp, dsize)
cv2.imshow("2x Horizontal Stretching", warped_img)
cv2.waitKey(0)
Producing the output:
So that is not your issue. Homographies allow even stronger warping. Are you running RANSAC yourself or letting the findHomography() function decide your points via RANSAC? Please post your expected output and your current code, possibly in a new question that reflects the problems you're facing.

How to remove noise from a histogram equalized image?

I have an image which I'm equalizing and then using clahe histogram on, like so:
self.equ = cv2.equalizeHist(self.result_array)
clahe = cv2.createCLAHE(clipLimit=100.0, tileGridSize=(8,8))
self.cl1 = clahe.apply(self.equ)
This is the result I get:
I want to get rid of all the black dots which is noise. Ultimately, I'm trying to extract out the blood vessels, which are black in the image shown above, in trying to do so, the noise makes the extraction inaccurate.

A large part of my thesis was on reducing the noise in images, and there was a technique I used which reduced noise in images while preserving the sharp edges of information in the image. I quote myself here:
An effective technique for removing noise from fringe patterns is to filter the image
using sine-cosine filtering [reference]. A low-pass filter is convolved with the two images that
result from taking the sine and cosine of the fringe pattern image, which are then
divided to obtain the tangent, restoring the phase pattern but with reduced noise. The
advantage of this technique is that the process can be repeated multiple times to
reduce noise while maintaining the sharp details of the phase transitions.
And here is the code I used:
import numpy as np
from scipy import ndimage
def scfilter(image, iterations, kernel):
"""
Sine‐cosine filter.
kernel can be tuple or single value.
Returns filtered image.
"""
for n in range(iterations):
image = np.arctan2(
ndimage.filters.uniform_filter(np.sin(image), size=kernel),
ndimage.filters.uniform_filter(np.cos(image), size=kernel))
return image
There, image was a numpy array representing the image, linearly rescaled to put black at 0 and white at 2 * pi, and kernel is the size in image pixels of the uniform filter applied to the data. It shouldn't take too many iterations to see a positive result, maybe in the region of 5 to 20.
Hope that helps :)

microscopy image segmentation: bacteria segmentation with python

I am trying to segment some microscopy bright-field images showing some E. coli bacteria.
The picture I am working with resembles this one (even if this one is obtained with phase contrast):
my problem is that after running my segmentation function (OtsuMask below) I cannot distinguish dividing bacteria (you can try my code below on the sample image). This means that I get one single labeled region for a couple of bacteria which are joined by their end, instead of two different labeled images.
The boundary between two dividing bacteria is too narrow to be highlighted by the morphological operations I perform on the thresholded image, but I guess there must be a way to achieve my goal.
Any ideas/suggestions?
import scipy as sp
import numpy as np
from scipy import optimize
import mahotas as mht
from scipy import ndimage
import pylab as plt
def OtsuMask(img,dilation_size=2,erosion_size=1,remove_size=500):
img_thres=np.asarray(img)
s=np.shape(img)
p0=np.array([0,0,0])
p0[0]=(img[0,0]-img[0,-1])/512.
p0[1]=(img[1,0]-img[1,-1])/512.
p0[2]=img.mean()
[x,y]=np.meshgrid(np.arange(s[1]),np.arange(s[0]))
p=fitplane(img,p0)
img=img-myplane(p,x,y)
m=img.min()
img=img-m
img=abs(img)
img=img.astype(uint16)
"""perform thresholding with Otsu"""
T = mht.thresholding.otsu(img,2)
print T
img_thres=img
img_thres[img<T*0.9]=0
img_thres[img>T*0.9]=1
img_thres=-img_thres+1
"""morphological operations"""
diskD=createDisk(dilation_size)
diskE=createDisk(erosion_size)
img_thres=ndimage.morphology.binary_dilation(img_thres,diskD)
labeled_im,N=mht.label(img_thres)
label_sizes=mht.labeled.labeled_size(labeled_im)
labeled_im=mht.labeled.remove_regions(labeled_im,np.where(label_sizes<remove_size))
figure();
imshow(labeled_im)
return labeled_im
def myplane(p,x,y):
return p[0]*x+p[1]*y+p[2]
def res(p,data,x,y):
a=(data-myplane(p,x,y));
return array(np.sum(np.abs(a**2)))
def fitplane(data,p0):
s=shape(data);
[x,y]=meshgrid(arange(s[1]),arange(s[0]));
print shape(x), shape(y)
p=optimize.fmin(res,p0,args=(data,x,y));
print p
return p
def createDisk( size ):
x, y = np.meshgrid( np.arange( -size, size ), np.arange( -size, size ) )
diskMask = ( ( x + .5 )**2 + ( y + .5 )**2 < size**2)
return diskMask
THE FIRST PART OF THE CODE IN OtsuMask CONSIST OF A PLANE FITTING AND SUBTRACTION.

A similar approach to the one described in this related stackoverflow answer can be used here.
It goes basically like this:
threshold your image, as you have done
apply a distance transform on the thresholded image
threshold the distance transform, so that only a small 'seed' part of each bacterium remains
label these seeds, giving each one a different shade of gray
(also add a labeled seed for the background)
execute the watershed algorithm with these seeds and the distance transformed image, to get the separatd contours of your bacteria
Check out the linked answer for some pictures that will make this much clearer.

A few thoughts:
Otsu may not be a good choice, as you may even use a fixed threshold (your bacteria are black).
Thresholding the image with any method will remove a lot of useful information.
I do not have a complete recipe for you, but even this very simple thing seems to give a lot of interesting information:
import matplotlib.pyplot as plt
import cv2
# cv2 is only used to read the image into an array, use only green channel
bact = cv.imread("/tmp/bacteria.png")[:,:,1]
# draw a contour image with fixed threshold 50
fig = plt.figure()
ax = fig.add_subplot(111)
ax.contourf(bact, levels=[0, 50], colors='k')
This gives:
This suggests that if you use contour-tracing techniques with fixed contours, you will receive quite nice-looking starting points for dilation and erosion. So, two differences in thresholding:
Contouring uses much more of the grayscale information than simple black/white thresholding.
The fixed threshold seems to work well with these images, and if illumination correction is needed, Otsu is not the best choice.

One day skimage Watershed segmentation was more useful for me, than any OpenCV samples. It uses some code borrowed from Cellprofiler project (python-based tool for sophisticated cell image analysis). Hint: use Euclidean distance transform from opencv, it's faster than scipy implementation. Also peak_local_max function has distance parameter, which useful for precise single cells distinguishing. I think this function is more robust in finding cell peaks than rude threshold (because intensity of cells may vary).
You can find scipy watershed implementation, but it has weird behavior.

Which kind of interpolation best for resizing image?

I have a numpy array that I wish to resize using opencv.
Its values range from 0 to 255. If I opt to use cv2.INTER_CUBIC, I may get values outside this range. This is undesirable, since the resized array is supposed to still represent an image.
One solution is to clip the results to [0, 255]. Another is to use a different interpolation method.
It is my understanding that using INTER_AREA is valid for down-sampling an image, but works similar to nearest neighbor for upsampling it, rendering it less than optimal for my purpose.
Should I use INTER_CUBIC (and clip), INTER_AREA, or INTER_LINEAR?
an example for values outside of range using INTER_CUBIC:
a = np.array( [ 0, 10, 20, 0, 5, 2, 255, 0, 255 ] ).reshape( ( 3, 3 ) )
[[ 0 10 20]
[ 0 5 2]
[255 0 255]]
b = cv2.resize( a.astype('float'), ( 4, 4 ), interpolation = cv2.INTER_CUBIC )
[[ 0. 5.42489886 15.43670964 21.29199219]
[ -28.01513672 -2.46422291 1.62949324 -19.30908203]
[ 91.88964844 25.07939219 24.75106835 91.19140625]
[ 273.30322266 68.20603609 68.13853455 273.15966797]]
Edit: As berak pointed out, converting the type to float (from int64) allows for values outside the original range. the cv2.resize() function does not work with the default 'int64' type. However, converting to 'uint8' will automatically saturate the values to [0..255].
Also, as pointed out by SaulloCastro, another related answer demonstrated scipy's interpolation, and that there the defualt method is the cubic interpolation (with saturation).

If you are enlarging the image, you should prefer to use INTER_LINEAR or INTER_CUBIC interpolation.
If you are shrinking the image, you should prefer to use INTER_AREA interpolation.
Cubic interpolation is computationally more complex, and hence slower than linear interpolation. However, the quality of the resulting image will be higher.

To overcome such problem you should find out the new size of the given image where the interpolation can be made. And copy interpolated sampled image on the target image like:
# create target image and copy sample image into it
(wt, ht) = imgSize # target image size
(h, w) = img.shape # given image size
fx = w / wt
fy = h / ht
f = max(fx, fy)
newSize = (max(min(wt, int(w / f)), 1),
max(min(ht, int(h / f)), 1)) # scale according to f (result at least 1 and at most wt or ht)
img = cv2.resize(img, newSize, interpolation=cv2.INTER_CUBIC) #INTER_CUBIC interpolation
target = np.ones([ht, wt]) * 255 # shape=(64,800)
target[0:newSize[1], 0:newSize[0]] = img
Some of the possible interpolation in openCV are:
INTER_NEAREST – a nearest-neighbor interpolation
INTER_LINEAR – a bilinear interpolation (used by default)
INTER_AREA – resampling using pixel area relation. It may be a preferred method for image decimation, as it gives moire’-free results. But when the image is zoomed, it is similar to the INTER_NEAREST method.
INTER_CUBIC – a bicubic interpolation over 4×4 pixel neighborhood
INTER_LANCZOS4 – a Lanczos interpolation over 8×8 pixel neighborhood
See here for results in each interpolation.

I think you should start with INTER_LINEAR which is the default option for resize() function. It combines sufficiently good visual results with sufficiently good time performance (although it is not as fast as INTER_NEAREST). And it won't create those out-of-range values.

My this answer is based on testing. And in the end it supports the answer of #shivam. I tested on these interpolation method, for both combination, shrinking and enlarging. And after enlarging I calculated psnr with orignal image.
[cv2.INTER_AREA,
cv2.INTER_BITS,
cv2.INTER_BITS2,
cv2.INTER_CUBIC,
cv2.INTER_LANCZOS4,
cv2.INTER_LINEAR,
cv2.INTER_LINEAR_EXACT,
cv2.INTER_NEAREST]
shirking=0.25
enlarge=4
Tested this on 165 images of different shapes. For results I picked maximum and 2nd maximum psnr and calculated the count.
For the Maximum the count for interpolation is show in image below.
From this test the maximum psnr is given by the combination of AREA and LANCZOS4 which gave max psnr for 141/204 images.
I also wanted to include the 2nd maximum. So here are the results of 2nd Maximum only.
Here the AREA and CUBIC gave the 2nd best result. 19/204 has the highest psnr and 158/347 have the 2nd highes psnr using AREA + CUBIC.
This results were vague so I opened the files for which CUBIC gave the highes psnr. Turns out the images with lot of texture/abstraction gave highest psnr using CUBIC.
So I did furthure tests only for AREA+CUBIC and AREA+LANCZOS4. I came to the conclusion that if you are shrinking image <10 times then go for the LANCZOS4. It will give you better results for less than 10times zoom and if the image is large than it's better than CUBIC.
As for my program I was shiriking image 4 times so for me the AREA+LANCZOS4 works better.
Scripts and images: https://github.com/crackaf/triple-recovery/tree/main/tests