Develop Reference

Python Image segementation and extraction

I am trying to figure out how to extract multiple objects from a greyscale image with mineral grains. I need to segment the different grains and then extract each of them to save as a separate image file.
While doing research, I have found that everyone uses skimage. I'm worried that some grains will not be extracted (mineralgrains).
Has anyone had to work on a similar problem?

I'm not entirely sure what the mineral grains are in this image, but segmenting and labeling connected components is pretty straight forward using skimage algorithms.
The simplest approach that I know of is accomplished in two major steps:
Generate a binary image. The most straight forward methods for this are using thresholding based methods such as those elaborated on here: https://scikit-image.org/docs/dev/auto_examples/segmentation/plot_thresholding.html
Separating connected objects. This is often accomplished through use of a watershed algorithm as elaborated on here: https://scikit-image.org/docs/stable/auto_examples/segmentation/plot_watershed.html
Assuming that the mineral grains are the bigger objects in the example image, a first pass approach may look something like the following
import numpy as np
from scipy import ndimage as ndi
from skimage import io
from skimage.filters import gaussian, threshold_li
from skimage.morphology import remove_small_holes, remove_small_objects
from skimage.segmentation import watershed
from skimage.feature import peak_local_max
from skimage.measure import label
# read in the image
img = io.imread('fovAK.png', as_gray=True)
gauss_sigma = 1 # standard deviation for Gaussian kernel for removing a bit of noise from image
obj_min_size = 5000 # minimum # of pixels for labeled objects
min_hole_size = 5000 # The maximum area, in pixels, of a contiguous hole that will be filled
watershed_footprin = 100 # size of local region within which to search for peaks for watershed
#########################
#make a binary image using threshold
#########################
img_blurred = gaussian(img, gauss_sigma) # remove a bit of noise from image
img_thresh = threshold_li(img_blurred) # use histogram based method to determine intensity threshold
img_binary = img_blurred > img_thresh # make a binary image
# make a border around the image so that holes at edge are not filled
ydims, xdims = img.shape
img_binary[0, :] = 0
img_binary[ydims-1, :] = 0
img_binary[:, 0] = 0
img_binary[:, xdims-1] = 0
img_rsh = remove_small_holes(img_binary, min_hole_size) # removes small holdes in binary image
img_rso = remove_small_objects(img_rsh, obj_min_size) # removes small objects in binary image
#############################
#use watershed algorithm to seperate connected objects
####################################
distance = ndi.distance_transform_edt(img_rso) # distance transform of image
coords = peak_local_max(distance, labels=img_rso, footprint=np.ones((ws_footprint, ws_footprint))) # local pead distances
mask = np.zeros(distance.shape, dtype=bool) # mask for watershed
mask[tuple(coords.T)] = True
markers, _ = ndi.label(mask)
labels = watershed(-distance, markers, mask=img_rso)
This is absolutely a first pass on your example image and is by no means perfect. You may need to use a different thresholding algorithm or tweak other parameters but hopefully this gets you moving in the right direction.
From there you can just iterate over the labeled objects and save individual images.
There are definitely more sophisticated approaches to these kinds of problems ranging from edge detectors and bandpass filters all the out to using ML based methods but hopefully this gets you moving in the right direction.

Merge image-segments depending on length of the watershed-line in-between using Python, Numpy and Scikit-Image/OpenCV

I am working on a watershedding-based segmentation algorithm to segment fluorescence images such as this one:
As result I obtain a Numpy array with labels for each segment. These are separated by a watershed lines, if the corresponding regions in the fluorescence image have a sufficiently large intensity-drop-off between them. For very large intensity-drop-offs they are completely separated through simple thresholding. The result for the image above is this:
My algorithm performs well for the vast majority of cases. However, it sometimes it has a slight tendency to oversegment. Such as in this case from the image above:
Since these cases will be difficult to improve by working further on the intensity-based segmentation itself (and I run the risk of breaking other things), I want to instead selectively merge adjacent segments based on the length of the watershed-line between them and the averaged maximum width of the two segments above and below.
I know what I have to do on a pixel-for-pixel basis:
Find pixels that have two different label-values in their direct neighborhood. Store these pixels separately for each segment-pair (with corresponding segment-labels).
Calculate the number of these pixels for each pair of adjacent segments to obtain the length of the watershed-line.
Calculate the maximum width (horizontally for simplicity) of the adjacent segments.
Merge the adjacent segments, if the watershed-line is longer than a given threshold-fraction (user-defined) of the averaged width of the two segments. I could do this by converting the labels to a binary mask, filling the watershed line using the stored pixels where applicable, and relabelling the binary mask.
Since in Python iterating over individual pixels is generally slow, I am unsure how to write performant code for this. Therefore I am looking for suggestions on how to implement this with Numpy and Skimage (OpenCV is also an option).

You didn't provide how you got your initial segments. Despite this, I think improving the watershed lines could solve your problem and this can be done in the watershed hierarchy framework, with the Higra package.
I specify an initial ordering of the watershed by the image complement and recompute its watershed lines with another attribute (volume).
The intensity drop and area that you describe are the volume attribute, and you can control the segmentation by its threshold in the hierarchy.
Here it is a working example:
import cv2
import numpy as np
import higra as hg
from skimage.morphology import remove_small_objects, label
import matplotlib.pyplot as plt
def main():
img_path = "fig.png"
img = cv2.imread(img_path)
img = img[:,:,0].copy()
img = img.max() - img
size = img.shape[:2]
graph = hg.get_4_adjacency_graph(size)
edge_weights = hg.weight_graph(graph, img, hg.WeightFunction.mean)
tree, altitudes = hg.quasi_flat_zone_hierarchy(graph, edge_weights)
attr = hg.attribute_volume(tree, altitudes)
saliency = hg.saliency(tree, attr)
# Take a look at this :)
# grid = hg.graph_4_adjacency_2_khalimsky(graph, saliency)
# plt.imshow(grid)
# plt.show()
attr_thold = np.mean(saliency) / 4 # arbitrary
area_thold = 500 # arbitrary
segments = hg.labelisation_horizontal_cut_from_threshold(tree, attr, attr_thold)
segments = label(remove_small_objects(segments, area_thold))
plt.imshow(segments)
plt.show()
if __name__ == "__main__":
main()
Here it is the result.

microscopy image segmentation: bacteria segmentation with python

I am trying to segment some microscopy bright-field images showing some E. coli bacteria.
The picture I am working with resembles this one (even if this one is obtained with phase contrast):
my problem is that after running my segmentation function (OtsuMask below) I cannot distinguish dividing bacteria (you can try my code below on the sample image). This means that I get one single labeled region for a couple of bacteria which are joined by their end, instead of two different labeled images.
The boundary between two dividing bacteria is too narrow to be highlighted by the morphological operations I perform on the thresholded image, but I guess there must be a way to achieve my goal.
Any ideas/suggestions?
import scipy as sp
import numpy as np
from scipy import optimize
import mahotas as mht
from scipy import ndimage
import pylab as plt
def OtsuMask(img,dilation_size=2,erosion_size=1,remove_size=500):
img_thres=np.asarray(img)
s=np.shape(img)
p0=np.array([0,0,0])
p0[0]=(img[0,0]-img[0,-1])/512.
p0[1]=(img[1,0]-img[1,-1])/512.
p0[2]=img.mean()
[x,y]=np.meshgrid(np.arange(s[1]),np.arange(s[0]))
p=fitplane(img,p0)
img=img-myplane(p,x,y)
m=img.min()
img=img-m
img=abs(img)
img=img.astype(uint16)
"""perform thresholding with Otsu"""
T = mht.thresholding.otsu(img,2)
print T
img_thres=img
img_thres[img<T*0.9]=0
img_thres[img>T*0.9]=1
img_thres=-img_thres+1
"""morphological operations"""
diskD=createDisk(dilation_size)
diskE=createDisk(erosion_size)
img_thres=ndimage.morphology.binary_dilation(img_thres,diskD)
labeled_im,N=mht.label(img_thres)
label_sizes=mht.labeled.labeled_size(labeled_im)
labeled_im=mht.labeled.remove_regions(labeled_im,np.where(label_sizes<remove_size))
figure();
imshow(labeled_im)
return labeled_im
def myplane(p,x,y):
return p[0]*x+p[1]*y+p[2]
def res(p,data,x,y):
a=(data-myplane(p,x,y));
return array(np.sum(np.abs(a**2)))
def fitplane(data,p0):
s=shape(data);
[x,y]=meshgrid(arange(s[1]),arange(s[0]));
print shape(x), shape(y)
p=optimize.fmin(res,p0,args=(data,x,y));
print p
return p
def createDisk( size ):
x, y = np.meshgrid( np.arange( -size, size ), np.arange( -size, size ) )
diskMask = ( ( x + .5 )**2 + ( y + .5 )**2 < size**2)
return diskMask
THE FIRST PART OF THE CODE IN OtsuMask CONSIST OF A PLANE FITTING AND SUBTRACTION.

A similar approach to the one described in this related stackoverflow answer can be used here.
It goes basically like this:
threshold your image, as you have done
apply a distance transform on the thresholded image
threshold the distance transform, so that only a small 'seed' part of each bacterium remains
label these seeds, giving each one a different shade of gray
(also add a labeled seed for the background)
execute the watershed algorithm with these seeds and the distance transformed image, to get the separatd contours of your bacteria
Check out the linked answer for some pictures that will make this much clearer.

A few thoughts:
Otsu may not be a good choice, as you may even use a fixed threshold (your bacteria are black).
Thresholding the image with any method will remove a lot of useful information.
I do not have a complete recipe for you, but even this very simple thing seems to give a lot of interesting information:
import matplotlib.pyplot as plt
import cv2
# cv2 is only used to read the image into an array, use only green channel
bact = cv.imread("/tmp/bacteria.png")[:,:,1]
# draw a contour image with fixed threshold 50
fig = plt.figure()
ax = fig.add_subplot(111)
ax.contourf(bact, levels=[0, 50], colors='k')
This gives:
This suggests that if you use contour-tracing techniques with fixed contours, you will receive quite nice-looking starting points for dilation and erosion. So, two differences in thresholding:
Contouring uses much more of the grayscale information than simple black/white thresholding.
The fixed threshold seems to work well with these images, and if illumination correction is needed, Otsu is not the best choice.

One day skimage Watershed segmentation was more useful for me, than any OpenCV samples. It uses some code borrowed from Cellprofiler project (python-based tool for sophisticated cell image analysis). Hint: use Euclidean distance transform from opencv, it's faster than scipy implementation. Also peak_local_max function has distance parameter, which useful for precise single cells distinguishing. I think this function is more robust in finding cell peaks than rude threshold (because intensity of cells may vary).
You can find scipy watershed implementation, but it has weird behavior.

Fit curve to segmented image

In my current data analysis I have some segmented Images like for example below.
My Problem is that I would like to fit a polynom or spline (s.th. one-dimensional) to
a certain area (red) in the segmented image. ( the result would be the black line).
Usually i would use something like orthogonal distance regression, the problem is that this
needs some kind of fit function which I don't have in this case.
So what would be the best approach to do this with python/numpy?
Is there maybe some standard algorithm for this kind of problem?
UPDATE:
it seems my drawing skills are probably not the best, the red area in the picture could also have some random noise and does not have to be completely connected (there could be small gaps due to noise).
UPDATE2:
The overall target would be to have a parametrized curve p(t) which returns the position i.e. p(t) => (x, y) for t in [0,1]. where t=0 start of black line, t= 1 end of black line.

I used scipy.ndimage and this gist as a template. This gets you almost there, you'll have to find a reasonable way to parameterize the curve from the mostly skeletonized image.
from scipy.misc import imread
import scipy.ndimage as ndimage
# Load the image
raw = imread("bG2W9mM.png")
# Convert the image to greyscale, using the red channel
grey = raw[:,:,0]
# Simple thresholding of the image
threshold = grey>200
radius = 10
distance_img = ndimage.distance_transform_edt(threshold)
morph_laplace_img = ndimage.morphological_laplace(distance_img,
(radius, radius))
skeleton = morph_laplace_img < morph_laplace_img.min()/2
import matplotlib.cm as cm
from pylab import *
subplot(221); imshow(raw)
subplot(222); imshow(grey, cmap=cm.Greys_r)
subplot(223); imshow(threshold, cmap=cm.Greys_r)
subplot(224); imshow(skeleton, cmap=cm.Greys_r)
show()
You may find other answers that reference skeletonization useful, an example of that is here:
Problems during Skeletonization image for extracting contours

Counting particles using image processing in python

Is there any good algorithm for detecting particles on a changing background intensity?
For example, if I have the following image:
Is there a way to count the small white particles, even with the clearly different background that appears towards the lower left?
To be a little more clear, I would like to label the image and count the particles with an algorithm that finds these particles to be significant:
I have tried many things with the PIL, cv , scipy , numpy , etc. modules.
I got some hints from this very similar SO question, and it appears at first glance that you could take a simple threshold like so:
im = mahotas.imread('particles.jpg')
T = mahotas.thresholding.otsu(im)
labeled, nr_objects = ndimage.label(im>T)
print nr_objects
pylab.imshow(labeled)
but because of the changing background you get this:
I have also tried other ideas, such as a technique I found for measuring paws, which I implemented in this way:
import numpy as np
import scipy
import pylab
import pymorph
import mahotas
from scipy import ndimage
import cv
def detect_peaks(image):
"""
Takes an image and detect the peaks usingthe local maximum filter.
Returns a boolean mask of the peaks (i.e. 1 when
the pixel's value is the neighborhood maximum, 0 otherwise)
"""
# define an 8-connected neighborhood
neighborhood = ndimage.morphology.generate_binary_structure(2,2)
#apply the local maximum filter; all pixel of maximal value
#in their neighborhood are set to 1
local_max = ndimage.filters.maximum_filter(image, footprint=neighborhood)==image
#local_max is a mask that contains the peaks we are
#looking for, but also the background.
#In order to isolate the peaks we must remove the background from the mask.
#we create the mask of the background
background = (image==0)
#a little technicality: we must erode the background in order to
#successfully subtract it form local_max, otherwise a line will
#appear along the background border (artifact of the local maximum filter)
eroded_background = ndimage.morphology.binary_erosion(background, structure=neighborhood, border_value=1)
#we obtain the final mask, containing only peaks,
#by removing the background from the local_max mask
detected_peaks = local_max - eroded_background
return detected_peaks
im = mahotas.imread('particles.jpg')
imf = ndimage.gaussian_filter(im, 3)
#rmax = pymorph.regmax(imf)
detected_peaks = detect_peaks(imf)
pylab.imshow(pymorph.overlay(im, detected_peaks))
pylab.show()
but this gives no luck either, showing this result:
Using the regional max function, I get images which almost appear to be giving correct particle identification, but there are either too many, or too few particles in the wrong spots depending on my gaussian filtering (images have gaussian filter of 2,3, & 4):
Also, it would need to work on images similar to this as well:
This is the same type of image above, just at a much higher density of particles.
EDIT: Solved solution: I was able to get a decent working solution to this problem using the following code:
import cv2
import pylab
from scipy import ndimage
im = cv2.imread('particles.jpg')
pylab.figure(0)
pylab.imshow(im)
gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (5,5), 0)
maxValue = 255
adaptiveMethod = cv2.ADAPTIVE_THRESH_GAUSSIAN_C#cv2.ADAPTIVE_THRESH_MEAN_C #cv2.ADAPTIVE_THRESH_GAUSSIAN_C
thresholdType = cv2.THRESH_BINARY#cv2.THRESH_BINARY #cv2.THRESH_BINARY_INV
blockSize = 5 #odd number like 3,5,7,9,11
C = -3 # constant to be subtracted
im_thresholded = cv2.adaptiveThreshold(gray, maxValue, adaptiveMethod, thresholdType, blockSize, C)
labelarray, particle_count = ndimage.measurements.label(im_thresholded)
print particle_count
pylab.figure(1)
pylab.imshow(im_thresholded)
pylab.show()
This will show the images like this:
(which is the given image)
and
(which is the counted particles)
and calculate the particle count as 60.

I had solved the "variable brightness in background" by using a tuned difference threshold with a technique called Adaptive Contrast. It works by performing a linear combination (a difference, in the case) of a grayscale image with a blurred version of itself, then applying a threshold to it.
Convolve the image with a suitable statistical operator.
Subtract the original from the convolved image, correcting intensity scale/gamma if necessary.
Threshold the difference image with a constant.
(original paper)
I did this very successfully with scipy.ndimage, in the floating-point domain (way better results than integer image processing), like this:
original_grayscale = numpy.asarray(some_PIL_image.convert('L'), dtype=float)
blurred_grayscale = scipy.ndimage.filters.gaussian_filter(original_grayscale, blur_parameter)
difference_image = original_grayscale - (multiplier * blurred_grayscale);
image_to_be_labeled = ((difference_image > threshold) * 255).astype('uint8') # not sure if it is necessary
labelarray, particle_count = scipy.ndimage.measurements.label(image_to_be_labeled)
Hope this helps!!

I cannot really give a definite answer, but here are a few pointers:
The function mahotas.morph.regmax might be better than the maximum filter as it removes pseudo-maxima. Perhaps combine this with a global threshold, with a local threshold (such as the mean over a window) or both.
If you have several images and the same uneven background, then maybe you can compute an average background and normalize against that, or use empty images as your estimate of background. This would be the case if you have a microscope, and like every microscope I've seen, the illumination is uneven.
Something like:
average = average_of_many(images)
# smooth it
average = mahotas.gaussian_filter(average,24)
Now you preprocess your images, like:
preproc = image/average
or something like that.