I'm trying to filter out images that do not contain any or much visible structure from those that have a visible object in them so I can feed them into an self-supervised neural network.
I want to keep images like this, and I want to remove images like this:
I'm converting chemical imaging data to numpy arrays containing the signal intensity as float data, then use matplotlib to generate these images. To try to filter out the blank images, I first smoothed the images by setting each pixel value to the mean of its surrounding pixel to minimize noise. Then I found the standard deviation (σ) and mean (μ) and tried to filter out the bad images based on a σ, σ/μ, or σ^2/μ threshold, the latter of which somewhat worked. But if I set a threshold each image must exceed, such as σ^2/μ = 500, to apply to all datasets, it would remove far too many images from some or remove few to none from others.
Here's an example of me smoothing out the image and comparing σ^2/μ.
np.load(example.npy)
smoothed_image = np.empty(imgs.shape[1:])
for i, image in enumerate(imgs):
for x in range(imgs.shape[1]):
for y in range(imgs.shape[2]):
# Select pixels to average
subset = image[np.clip(x-3, 0, None):np.clip(x+4, None, image.shape[0]-1),
np.clip(y-3, 0, None):np.clip(y+4, None, image.shape[1]-1)]
subset_ave = np.mean(subset)
smoothed_image[x,y] = subset_ave
smoothed_image[x,y]
# Show stddev^2/mean and related image
print(f'stddev^2/mean = {smoothed_image.std()**2/smoothed_image.mean()})
plt.imshow(image)
plt.show()
plt.close()
I need to filter this data in an unsupervised fashion, so checking and changing the threshold for each dataset isn't an option. In addition, this process adds a significant amount of time to my data processing due to the ordering of my workflow. I tried to find other options online, but I don't think I know what to search to find information about this specific issue.
Here is some example data. Selecting any index on axis 0 (ex. images[8]) will give you a single image array.
Any suggestions on what methods I could use to filter images like this, preferably without very time consuming computation?
Thanks in advance!
My first thought is to use an aggressive threshold to suppress nearly all the noise, then simply take the sum of the image and set a threshold that way, kind of like:
image_thresh = image - 100 # where image is a numpy array and 100 would surely suppress noise, but not features
image_thresh[image_thresh<0] = 0
image_sum = np.sum( image_thresh )
Another way is to use OpenCV and look for ellipses above a certain size. You could reference such a page as this one to get started on that.
Related
I have image that contains many no data pixels. The image is 2d numpy array and the no-data values are "None". Whenever I try to apply on it filters, seems like the none values are taken into account into the kernel and makes my pixels dissapear.
For example, I have this image:
I have tried to apply on it the lee filter with this function (taken from Speckle ( Lee Filter) in Python):
from scipy.ndimage.filters import uniform_filter
from scipy.ndimage.measurements import variance
def lee_filter(img, size):
img_mean = uniform_filter(img, (size, size))
img_sqr_mean = uniform_filter(img**2, (size, size))
img_variance = img_sqr_mean - img_mean**2
overall_variance = variance(img)
img_weights = img_variance / (img_variance + overall_variance)
img_output = img_mean + img_weights * (img - img_mean)
return img_output
but the results looks like this:
with the warnning:
UserWarning: Warning: converting a masked element to nan. dv =
np.float64(self.norm.vmax) - np.float64(self.norm.vmin)
I have also tried to use the library findpeaks.
from findpeaks import findpeaks
import findpeaks
#lee enhanced filter
image_lee_enhanced = findpeaks.lee_enhanced_filter(img, win_size=3, cu=0.25)
but I get the same blank image.
When I used median filter on the same image with ndimage is worked no problem.
My question is how can I run those filters on the image without letting the None values interrupt the results?
edit: I prefer not to set no value pixels to 0 because the pixel range value is between -50-1 (is an index values). In addition i'm afraid that if I change it to any other value e.g 9999) it will also influence the filter (am I wrong?)
Edit 2:
I have read Cris Luengo answer and I have tried to apply something similar with the scipy.ndimage median filter as I have realized that the result is disorted as well.
This is the original image:
I have tried masking the Null values:
idx = np.ma.masked_where(img,img!=None)[:,1]
median_filter_img = ndimage.median_filter(img[idx].reshape(491, 473), size=10)
zeros = np.zeros([img.shape[0],img.shape[1]])
zeros[idx] = median_filter_img
The results looks like this (color is darker to see the problem in the edges):
As it can bee seen, seems like the edges values are inflluences by the None values.
I have done this also with img!=0 but got the same problem.
(just to add: the pixels vlues are between 1 to -35)
If you want to apply a linear smoothing filter, then you can use the Normalized Convolution.
The basic recipe is:
Create a mask image that is 1 for the pixels with data, and 0 for the pixels without data.
Set the pixels without data to any number, for example 0. NaN is not valid because it spreads in the computations.
Apply the linear smoothing filter to the image multiplied by the mask.
Apply the linear smoothing filter to the mask.
Divide the two results.
Basically, we normalize the result of the linear smoothing filter (convolution) by the number of pixels with data within the filter window.
In regions where the smoothed mask is 0 (far away from data), we will divide 0 by 0, so special care needs to be taken there.
Note that normalized convolution can be used also for uncertain data, where the mask image gets values in between 0 and 1 indicating the confidence we have in each pixel. Pixels thought to be noisy can be set to a value closer to 0 than the other pixels, for example.
The recipe above is only valid for linear smoothing filters. Normalized convolution can be done with other linear filters, for example derivative filters, but the resulting recipe is different. See for example here the equation for Normalized Convolution to compute the derivative.
For non-linear filters, other approaches are necessary. Non-linear smoothing filters, for example, will often avoid affecting edges, and so will work quite well in images with missing data, if the missing pixels are set to 0, or some value far outside of the data range. The concept of keeping a mask image that indicates which pixels have data and which don't is always a good idea.
Seems like a simple solution is to set the non values to zero. I don't know how you would get around this, because most image processing kernels require some value to for you to apply.
a[numpy.argwhere(a==None)] = 0
I have a set of consecutive frames from a video. For example like this
({F1, F20}, {F34, F40}, {F89, F200}). So here from F1 to F20 represents one segment or a sequence of frames. My question is how can I know which segment has the most variability as most of the frames are quite similar in a segment. In this case, I want the second segment to be the output.
The most obvious variant is to calc pixelwise absotule difference. If you have high-resolution videos you can sacrifice accuracy for performance by downscaling frames, the overall picture should not change much because of this.
For each frame perform:
curr_frame = # ... however you get it
curr_frame_downscaled = cv2.resize(curr_frame, (400, 400), cv2.INTER_AREA)
diff_index = np.sum(np.abs(curr_frame_downscaled - prev_frame_downscaled))
prev_frame_downscaled = curr_frame_downscaled
After this you can operate diffs as you wish e.g. find diff sum for each segment or average diff etc. You can also test different color spaces but keep in mind they can have different value ranges among channels, so some of them require normalization.
I don't know anything about your segments though. Video where each frame swaps white and black screens will be considered as extremely different in spite of there are only two frame states. In this way try to sort frames with np.diff() before.
This is how I tried:
(1) use PIL.Image to open the original(say 100*100) and target(say 20*20) image and convert them into np.array;
(2) start from every pixel in the original one as a starting position, crop a 20*20 area and compare every pixel RGB with the target.
(3) If the total difference is under certain given level, then stop and output the specific starting pixel position in the original one.
The problem is, step(3) costs over 10s which is much too long, even step(2) costs over 0.04s and I hope to optimize my program. In both steps I used For to iterate array, is there a more efficient way?
To compare two signals (or images) at different displacements one can use cross-correlation.
If you have the scipy package you can use 2D cross-correlation to measure how similar the two images are when you slide one image over the other.
This example is copied from the correlate2d function:
from scipy import signal
from scipy import misc
lena = misc.lena() - misc.lena().mean()
template = np.copy(lena[235:295, 310:370]) # right eye
template -= template.mean()
lena = lena + np.random.randn(*lena.shape) * 50 # add noise
corr = signal.correlate2d(lena, template, boundary='symm', mode='same')
y, x = np.unravel_index(np.argmax(corr), corr.shape) # find the match
If you don't want to use a toolbox you could implement the cross-correlation yourself.
EDIT: I have found a solution :D thanks for the help.
I've created an image processing algorithm which extracts this image from the data. It's complex, so I won't go into detail, but this image is essentially a giant numpy array (it's visualizing angular dependence of pixel intensity of an object).
I want to write a program which automatically determines when the curves switch direction. I have the data and I also have this image, but it turns out doing something meaningful with either has been tricky. Thresholding fails because there are bands of different background color. Sobel operators and Hough Transforms also do not work well for this same reason.
This is really easy for humans to see when this switch happens, but not so easy to tell a computer. Any tips? Thanks!
Edit: Thanks all, I'm now fitting lines to this image after convolution with general gaussian and skeletonization of the result. Any pointers on doing this would be appreciated :)
You can take a weighted dot product of successive columns to get a one-dimensional signal that is much easier to work with. You might be able to extract the patterns using this signal:
import numpy as np
A = np.loadtxt("img.txt")
N = A.shape[0]
L = np.logspace(1,2,N)
X = []
for c0,c1 in zip(A.T, A.T[1:]):
x = c0.dot(c1*L) / (np.linalg.norm(c0)*np.linalg.norm(c1))
X.append(x)
X = np.array(X)
import pylab as plt
plt.matshow(A,alpha=.5)
plt.plot(X*3-X.mean(),'k',lw=2)
plt.axis('tight')
plt.show()
This is absolutely not a complete answer to the question, but a useful observation that is too long for a comment. I'll delete if a better answer comes along.
With the help of Mark McCurry, I was able to get a good result.
Step 1: Load original image. Remove background by subtracting median of each vertical column from itself.
no_background=[]
for i in range(num_frames):
no_background.append(orig[:,i]-np.median(orig,1))
no_background=np.array(no_background).T
Step 2: Change negative values to 0.
clipped_background = no_background.clip(min=0)
Step 3: Extract a 1D signal. Take weighted sum of the vertical columns, which relates the max intensity in a column to its position.
def exp_func(x):
return np.dot(np.arange(len(x)), np.power(x, 10))/(np.sum(np.power(x, 10)))
weighted_sum = np.apply_along_axis(exp_func,0, clipped_background)
Step 4: Take the derivative of 1D signal.
conv = np.convolve([-1.,1],weighted_sum, mode='same')
pl.plot(conv)
Step 5: Determine when the derivative changes sign.
signs=np.sign(conv)
pl.plot(signs)
pl.ylim(-1.2,1.2)
Step 6: Apply median filter to above signal.
filtered_signs=median_filter(signs, 5) #pick window size based on result. second arg and odd number.
pl.plot(filtered_signs)
pl.ylim(-1.2,1.2)
Step 7: Find the indices (frame locations) of when the sign switches. Plot result.
def sign_switch(oneDarray):
inds=[]
for ind in range(len(oneDarray)-1):
if (oneDarray[ind]<0 and oneDarray[ind+1]>0) or (oneDarray[ind]>0 and oneDarray[ind+1]<0):
inds.append(ind)
return np.array(inds)
switched_frames = sign_switch(filtered_signs)
For detecting tip positions or turning points, you might try using a corner detector on the original image (not the skeletonized one). As a corner detector the structure tensor could be applicable. The structure tensor is also useful for calculating the local orientation in an image.
I have an image from an electron micrograph depicting dense and rare layers in a biological system, as shown below.
The layers in question are in the middle of the image, starting just to near the label "re" and tapering up to the left. I would like to:
1) count the total number of dark/dense and light/rare layers
2) measure the width of each layer, given that the black scale bar in the bottom right is 1 micron long
I've been trying to do this in Python. If I crop the image beforehand so as to only contain parts of a few layers, such the 3 dark and 3 light layers shown here:
I am able to count the number of layers using the code:
import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage
from PIL import Image
tap = Image.open("VDtap.png").convert('L')
tap_a = np.array(tap)
tap_g = ndimage.gaussian_filter(tap_a, 1)
tap_norm = (tap_g - tap_g.min())/(float(tap_g.max()) - tap_g.min())
tap_norm[tap_norm < 0.5] = 0
tap_norm[tap_norm >= 0.5] = 1
result = 255 - (tap_norm * 255).astype(np.uint8)
tap_labeled, count = ndimage.label(result)
plt.imshow(tap_labeled)
plt.show()
However, I'm not sure how to incorporate the scale bar and measure the widths of these layers that I have counted. Even worse, when analyzing the entire image so as to include the scale bar I am having trouble even distinguishing the layers from everything else that is going on in the image.
I would really appreciate any insight in tackling this problem. Thanks in advance.
EDIT 1:
I've made a bit of progress on this problem so far. If I crop the image beforehand so as to contain just a bit of the layers, I've been able to use the following code to get at the thicknesses of each layer.
import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage
from PIL import Image
from skimage.measure import regionprops
tap = Image.open("VDtap.png").convert('L')
tap_a = np.array(tap)
tap_g = ndimage.gaussian_filter(tap_a, 1)
tap_norm = (tap_g - tap_g.min())/(float(tap_g.max()) - tap_g.min())
tap_norm[tap_norm < 0.5] = 0
tap_norm[tap_norm >= 0.5] = 1
result = 255 - (tap_norm * 255).astype(np.uint8)
tap_labeled, count = ndimage.label(result)
props = regionprops(tap_labeled)
ds = np.array([])
for i in xrange(len(props)):
if i==0:
ds = np.append(ds, props[i].bbox[1] - 0)
else:
ds = np.append(ds, props[i].bbox[1] - props[i-1].bbox[3])
ds = np.append(ds, props[i].bbox[3] - props[i].bbox[1])
Essentially, I discovered the Python module skimage, which can take a labeled image array and return the four coordinates of a boundary box for each labeled object; the 1 and [3] positions give the x coordinates of the boundary box, so their difference yields the extent of each layer in the x-dimension. Also, the first part of the for loop (the if-else condition) is used to get the light/rare layers that precede each dark/dense layer, since only the dark layers get labeled by ndimage.label.
Unfortunately this is still not ideal. Firstly, I would like to not have to crop the image beforehand, as I intend to repeat this procedure for many such images. I've considered that perhaps the (rough) periodicity of the layers could be highlighted using some sort of filter, but I'm not sure if such a filter exists? Secondly, the code above really only gives me the relative width of each layer - I still haven't figured out a way to incorporate the scale bar so as to get the actual widths.
I don't want to be a party-pooper, but I think your problem is harder than you first thought. I can't post a working code snippet because there are so many parts of your post that require in depth attention. I have worked in several bio/med labs and this work is usual done with a human to tag specific image points and a computer to calculate distances. That being said, one should probably try to automate =D.
To you, the problem is a simple, yet tedious job, of getting out a ruler and making a few hundred measurements. Perfect for a computer right? Well yes and no. The computer has no idea how to identify any of the bands in the picture and has to be told exactly what its looking for, and that will be tricky.
Identifying the scale bar
What do you know about the scale bars in all your images. Are they always the same number of vertical and horizontal pictures, are they always solid black? Are there always just one bar (what about the solid line for the letter r)? My suggestion is to try a wavelet transform. Imagine the 2d analog to the function
(probably helps to draw this function)
f(x) =
0 if |x| > 1,
1 if |x| <1 && |x| > 0.5
-1 if |x| < 0.5
Then when our wavelet f(x, y) is convolved over the image, the output image will have high values only when it finds the black scale bar. Also the length that I set to 1 can also be tuned for wavelets and that will help you find the scale bar too.
Finding the ridges
I'd solve the above problem first because it seems easier and sets you up for this one. I'd construct another wavelet for this one but just as a preprocessing step. For this wavelet I'd try a 2d 0-sum box function again, but this try to match three (or more) boxes next to each other. Also in addition to the height and width parameters for the box, we need a spacing and tilt angle parameter. You probably don't have to get very close to the actual value, just close enough that the rest of the image blackens out.
Measuring the ridges
There are lots and lots of ways to do this, but let's use our previous step for simplicity. Take your 3 box wavelet answer and it should be centered at the middle ridge and report a box "width" that is the average width of those three ridges it has captured. Probably close enough considering how slowly the widths are changing!
Good hunting!