I am trying to write a script to calculate the angle between two bones given an x-ray.
A sample x-ray would look like the following:
I am trying to calculate the midline of each bone, essentially a line following the midpoints of the two sides of a bone, and then compare the angle between the two midlines.
I have tried using OpenCV to get the outline of the bones, but it does not seem accurate enough and gets lots of extra data. I am stuck on how to move next and how I would calculate the midline. I am quite new to image processing but have experience with Python.
Getting edges using OpenCV results:
Code for OpenCV:
import cv2
# Load the image
img = cv2.imread("xray-3.jpg")
# Find the contours
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(img,60,200)
im2, contours, hierarchy = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
hierarchy = hierarchy[0] # get the actual inner list of hierarchy descriptions
# For each contour, find the bounding rectangle and draw it
cv2.drawContours(img, contours, -1, (0,255,0), 3)
# Finally show the image
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
If it's not cheating, i'd recommend cropping the image to not in include as much of the labels and scales as possible without removing any areas of interest.
That being said, I think your method of getting the contours will be usable if you do some preprocessing to the image. One algorithm that might do the trick is a Difference of Gaussians (DoG) filter which will bring out the edges a little more. I modified slightly this code which will compute the DoG filter using a few different sigma and k values.
from skimage import io, feature, color, filters, img_as_float
from matplotlib import pyplot as plt
raw_img = io.imread('xray-3.jpg')
original_image = img_as_float(raw_img)
img = color.rgb2gray(original_image)
k = 1.6
plt.subplot(2,3,1)
plt.imshow(original_image)
plt.title('Original Image')
for idx,sigma in enumerate([4.0, 8.0, 16.0, 32.0]):
s1 = filters.gaussian(img, k*sigma)
s2 = filters.gaussian(img, sigma)
# multiply by sigma to get scale invariance
dog = s1 - s2
plt.subplot(2,3,idx+2)
print("min: {} max: {}".format(dog.min(), dog.max())
plt.imshow(dog, cmap='RdBu')
plt.title('DoG with sigma=' + str(sigma) + ', k=' + str(k))
ax = plt.subplot(2, 3, 6)
blobs_dog = [(x[0], x[1], x[2]) for x in feature.blob_dog(img, min_sigma=4, max_sigma=32, threshold=0.5, overlap=1.0)]
# skimage has a bug in my version where only maxima were returned by the above
blobs_dog += [(x[0], x[1], x[2]) for x in feature.blob_dog(-img, min_sigma=4, max_sigma=32, threshold=0.5, overlap=1.0)]
#remove duplicates
blobs_dog = set(blobs_dog)
img_blobs = color.gray2rgb(img)
for blob in blobs_dog:
y, x, r = blob
c = plt.Circle((x, y), r, color='red', linewidth=2, fill=False)
ax.add_patch(c)
plt.imshow(img_blobs)
plt.title('Detected DoG Maxima')
plt.show()
At first glance, it appears that sigma=8.0, k=1.6 might be your best bet as this seems to best exaggerate the edges of the lower leg while getting rid of the noise across it. Particularly over that of the subjects left (image right) leg. Give your edge detection another go and play around with k and sigma and let me know what you get :)
If the results look good you should be able to get a center point between the edges detected for either leg in each row of the image. Then just find the line of best fit for the mid points for either leg and you should be good to go. You will also need to isolate one leg from another, so again, if it's not cheating, maybe crop the image down the middle into two images.
Related
I have an image like so (my apologies for anyone who finds this to be too much):
And would like to get an image like this (with the borders filled in as a segmentation should be):
As you can see, the segmentation should be defined by the "physical" borders present in the image, perhaps taking into account shadows, edges, etc.
I have tried using the Canny edge filter, but this seems to show me edges that are not desirable (even changing the parameters) and I'm not sure how to go forward in that direction.
My closest attempt has been using K-means clustering, but there seems to be two downsides to using this:
Completely unrelated portions of the image are labeled as the same cluster just because their RGB values are similar.
Because the algorithm depends on the average color values in a cluster, more lit parts of an image are labeled different clusters than darker ones even though I need them to be the same.
Here is the image I get using K-means:
And here is the code I used to get it:
import cv2
import numpy as np
original = cv2.imread('liver_annotation_yiHXgxp.png')
alpha = 3
beta = 0
contrast = cv2.convertScaleAbs(original, alpha=alpha, beta=beta)
kernel = np.ones((5,5),np.float32)/25
blur = cv2.filter2D(contrast,-1,kernel)
image = cv2.cvtColor(contrast, cv2.COLOR_BGR2RGB)
pixel_values = image.reshape((-1, 3))
pixel_values = np.float32(pixel_values)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.2)
k = 5
_, labels, (centers) = cv2.kmeans(pixel_values, k, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)
centers = np.uint8(centers)
labels = labels.flatten()
segmented_image = centers[labels.flatten()]
segmented_image = segmented_image.reshape(image.shape)
# Results
cv2.imshow('original', original)
cv2.imshow('contrast', contrast)
cv2.imshow('blur', blur)
cv2.imshow('adjusted', segmented_image)
cv2.waitKey()
I am new to Python and I have a question of how to get the boundaries of a figure that plotted by matplotlib imshow. For example, the following figure is the one that plotted by imshow and I want to get the coordinate of the boundaries and plot the boundaries over the same image. I hope you can help me.
Here is the code that I wrote for the first two lines.
df = pd.read_csv('data.txt', sep='\t')
plt.imshow(df,origin='lower')
The stackoverflow does not allow me upload the raw data. But please download it from my google drive folder:
https://drive.google.com/file/d/1ZPxtAz7vjsdFjeRmfop2cgpfcuGlO5PI/view?usp=sharing
Here is you original array:
One option is to compute a threshold on your array and then calculate the difference between the binary dilation and the threshold:
from scipy.ndimage.morphology import binary_dilation
array_thresh = np.array(array<0.8, dtype='int')
plt.imshow(binary_dilation(array_thresh)-array_thresh)
To perform the overlay, you need to mask the second array with numpy.ma.masked_where:
a_cnt = (binary_dilation(a_thresh)-a_thresh)
a_cnt_masked = np.ma.masked_where(a_cnt==0, a_cnt)
plt.imshow(array)
plt.imshow(a_cnt_masked, cmap='hsv', interpolation='none')
You can also use a specialized library, such as opencv:
import cv2
im = cv2.threshold(array, 1, 255, cv2.THRESH_BINARY)[1]
cnt = cv2.findContours(im.astype('uint8'), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]
im2 = sum([cv2.drawContours(np.zeros(array.shape), [c], 0, (255,0,0), 1) for c in cnt])
plt.imshow(im2)
You might need to play around with the different parameters of opencv.
I have contour plots created in Matplotlib, that I need to analyze further to see if they are closed curves, and then look at area, convexity, solidity, etc. for cellular structures. In Matplotlib, they are of type LineCollection and Path.
In OpenCV, I cannot pass a float array to cv2.contourArea or similar functions. On the other hand, converting to uint8 coordinates loses important data like nesting structure. In this case, I need to get to the inner nested convex contours.
Are there any options to find information like area, convex hull, bounding rectangle in Python?
I could enlarge the image, but I'm worried it might skew the picture unpredictably.
For example: Attached image with floating point and integer coordinates.
I assume, you have full control over the Matplotlib part. So, let's try to get an image from there, which can you easily use for further image processing with OpenCV.
We start with some common contour plot as shown in your question:
You can set the levels parameter to get a single contour level. That's helpful to work on several levels individually. In the following, I will focus on levels=[1.75] (the most inner green ellipse). Later, you can simply loop through all desired levels, and perform your analyses.
For our custom contour plot, we will set a fixed x, y domain, for example [-3, 3] x [-2, 2], using xlim and ylim. So, we have known dimensions for the actual canvas. We get rid of the axes using axis('off'), and the margins around the canvas using tight_layout(pad=0). What's left is the plain canvas in full size (figure size was adjusted to (10, 5), and colors are automatically adjusted to the number of levels):
Now, we save the canvas to some NumPy array, cf. this Q&A. From there, we can perform any OpenCV operation. For finding the combined area of this level contours, we might threshold the grayscaled image, find all contours, and calculate their areas using cv2.contourArea. We sum those areas, and get the whole area in pixels. From the known canvas dimensions, we know the whole canvas area in "units", and from the image dimensions, we know the whole canvas area in pixels. So, we just need to divide the whole contour area (in pixels) by the whole canvas area (in pixels), and multiply with the whole canvas area (in "units").
That'd be the whole code:
import cv2
import matplotlib.pyplot as plt
import numpy as np
# Generate some data for some contour plot
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-2.0, 2.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = np.exp(-(X + 1.5)**2 - Y**2)
Z2 = np.exp(-(X - 1.5)**2 - Y**2)
Z = (Z1 + Z2) * 2
# Custom contour plot
x_min, x_max = -3, 3
y_min, y_max = -2, 2
fig = plt.figure(2, figsize=(10, 5)) # Set large figure size
plt.contour(X, Y, Z, levels=[1.75]) # Set single levels if needed
plt.xlim([x_min, x_max]) # Explicitly set x limits
plt.ylim([y_min, y_max]) # Explicitly set y limits
plt.axis('off') # No axes shown at all
plt.tight_layout(pad=0) # No margins at all
# Get figure's canvas as NumPy array, cf. https://stackoverflow.com/a/7821917/11089932
fig.canvas.draw()
img = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
img = img.reshape(fig.canvas.get_width_height()[::-1] + (3,))
# Grayscale, and threshold image
mask = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
mask = cv2.threshold(mask, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY_INV)[1]
# Find contours, calculate areas (pixels), sum to get whole area (pixels) for certain level
cnts = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
area = np.sum(np.array([cv2.contourArea(cnt) for cnt in cnts]))
# Whole area (coordinates) from canvas area (pixels), and x_min, x_max, etc.
area = area / np.prod(mask.shape[:2]) * (x_max - x_min) * (y_max - y_min)
print('Area:', area)
The output area seems reasonable:
Area: 0.861408
Now, you're open to do any image processing with OpenCV you like. Always remember to convert any results in pixels to some result in "units".
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.9.1
PyCharm: 2021.1.1
Matplotlib: 3.4.1
NumPy: 1.20.2
OpenCV: 4.5.1
I am having a image here. The region within the yellow lines is my region of interest, as shown in this image here, which is also one of my objective. Here's my planning / steps:
Denoise, color filtering, masking and Canny edging (DONE)
Coordinates of the edges (DONE)
Select coordinates of certain vertices, for example
Draw polygon with those vertices' coordinates
Here's the code:
import cv2
import numpy as np
from matplotlib import pyplot as plt
frame = cv2.imread('realtest.jpg')
denoisedFrame = cv2.fastNlMeansDenoisingColored(frame, None, 10, 10, 7, 21)
HSVframe = cv2.cvtColor(denoisedFrame, cv2.COLOR_BGR2HSV)
lower_yellowColor = np.array([15,105,105])
upper_yellowColor = np.array([25,255,255])
whiteMask = cv2.inRange(HSVframe, lower_yellowColor, upper_yellowColor)
maskedFrame = cv2.bitwise_and(denoisedFrame, denoisedFrame, mask=whiteMask)
grayFrame = cv2.cvtColor(maskedFrame, cv2.COLOR_BGR2GRAY)
gaussBlurFrame = cv2.GaussianBlur(grayFrame, (5,5), 0)
edgedFrame = cv2.Canny(grayFrame, 100, 200)
#Coordinates of each white pixels that make up the edges
ans = []
for y in range(0, edgedFrame.shape[0]):
for x in range(0, edgedFrame.shape[1]):
if edgedFrame[y, x] != 0:
ans = ans + [[x, y]]
ans = np.array(ans)
#print(ans.shape)
#print(ans[0:100, :])
cv2.imshow("edged", edgedFrame)
cv2.waitKey(0)
cv2.destroyAllWindows()
As you can see, I have successfully done step number (2) in getting the coordinates of each white pixels that make the edges. Whereas for the next step, step number (3), I am stuck. I have tried the coding here, but getting error that says 'ValueError: too many values to unpack (expected 2)'.
Please help teaching me in finding good vertices for constructing a polygon that is as close to the yellow lines as possible.
I have split the answer into two parts
Part 1: Finding the good vertices to construct a polygon
The required vertices around an image containing edges can be done using OpenCV's inbuilt cv2.findContours() function. It returns the image with contours, vertices of the contours and the hierarchy of the contours.
One can find vertices of contours in two ways:
cv2.CHAIN_APPROX_NONE plots ALL the coordinates(boundary points) on each contour
cv2.CHAIN_APPROX_SIMPLE plots ONLY the most necessary coordinates on each contour. It doesn't store all the points. Only the most required coordinates that best represent the contours are stored.
In your case option 2 can be opted. After finding the edges you can do the following:
image, contours, hier = cv2.findContours(edgedFrame, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
contours contains the vertices of every contour in the image edgedFrame
Part 2: Constructing the polygon
Opencv has an in-built function for this as well cv2.convexHull() After finding those points you can draw them using cv2.drawContours().
for cnt in contours:
hull = cv2.convexHull(cnt)
cv2.drawContours(frame, [hull], -1, (0, 255, 0), 2)
cv2.imshow("Polygon", frame)
You can obtain a better approximation of the desired edge by doing some more pre-processing while creating the mask
The image below shows an aerial photo of a house block (re-oriented with the longest side vertical), and the same image subjected to Adaptive Thresholding and Difference of Gaussians.
Images: Base; Adaptive Thresholding; Difference of Gaussians
The roof-print of the house is obvious (to the human eye) on the AdThresh image: it's a matter of connecting some obvious dots. In the sample image, finding the blue-bounded box below -
Image with desired rectangle marked in blue
I've had a crack at implementing HoughLinesP() and findContours(), but get nothing sensible (probably because there's some nuance that I'm missing). The python script-chunk that fails to find anything remotely like the blue box, is as follows:
import cv2
import numpy as np
from matplotlib import pyplot as plt
# read in full (RGBA) image - to get alpha layer to use as mask
img = cv2.imread('rotated_12.png', cv2.IMREAD_UNCHANGED)
grey = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Otsu's thresholding after Gaussian filtering
blur_base = cv2.GaussianBlur(grey,(9,9),0)
blur_diff = cv2.GaussianBlur(grey,(15,15),0)
_,thresh1 = cv2.threshold(grey,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
thresh = cv2.adaptiveThreshold(grey,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,11,2)
DoG_01 = blur_base - blur_diff
edges_blur = cv2.Canny(blur_base,70,210)
# Find Contours
(ed, cnts,h) = cv2.findContours(grey, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = sorted(cnts, key = cv2.contourArea, reverse = True)[:4]
for c in cnts:
approx = cv2.approxPolyDP(c, 0.1*cv2.arcLength(c, True), True)
cv2.drawContours(grey, [approx], -1, (0, 255, 0), 1)
# Hough Lines
minLineLength = 30
maxLineGap = 5
lines = cv2.HoughLinesP(edges_blur,1,np.pi/180,20,minLineLength,maxLineGap)
print "lines found:", len(lines)
for line in lines:
cv2.line(grey,(line[0][0], line[0][1]),(line[0][2],line[0][3]),(255,0,0),2)
# plot all the images
images = [img, thresh, DoG_01]
titles = ['Base','AdThresh','DoG01']
for i in xrange(len(images)):
plt.subplot(1,len(images),i+1),plt.imshow(images[i],'gray')
plt.title(titles[i]), plt.xticks([]), plt.yticks([])
plt.savefig('a_edgedetect_12.png')
cv2.destroyAllWindows()
I am trying to set things up without excessive parameterisation. I'm wary of 'tailoring' an algorithm for just this one image since this process will be run on hundreds of thousands of images (with roofs/rooves of different colours which may be less distinguishable from background). That said, I would love to see a solution that 'hit' the blue-box target - that way I could at the very least work out what I've done wrong.
If anyone has a quick-and-dirty way to do this sort of thing, it would be awesome to get a Python code snippet to work with.
The 'base' image ->
Base Image
You should apply the following:
1. Contrast Limited Adaptive Histogram Equalization-CLAHE and convert to gray-scale.
2. Gaussian Blur & Morphological transforms (dialation, erosion, etc) as mentioned by #bad_keypoints. This will help you get rid of the background noise. This is the most tricky step as the results will depend on the order in which you apply (first Gaussian Blur and then Morphological transforms or vice versa) and the window sizes you choose for this purpose.
3. Apply Adaptive thresholding
4. Apply Canny's Edge detection
5. Find contour having four corner points
As said earlier you need to tweak with input parameters of these functions and also need to validate these parameters with other images. As it might be possible that it will work for this case but not for other cases. Based on trial and error you need to fix the parameter values.