Develop Reference

affine transformation using nearest neighbor in python

I want to make an affine transformation and afterwards use nearest neighbor interpolation while keeping the same dimensions for input and output images. I use for example the scaling transformation T= [[2,0,0],[0,2,0],[0,0,1]]. Any idea how can I fill the black pixels with nearest neighbor ? I tryied giving them the min value of neighbors' intensities. For ex. if a pixel has neighbors [55,22,44,11,22,55,23,231], I give it the value of min intensity: 11. But the result is not anything clear..
import numpy as np
from matplotlib import pyplot as plt
#Importing the original image and init the output image
img = plt.imread('/home/left/Desktop/computerVision/SET1/brain0030slice150_101x101.png',0)
outImg = np.zeros_like(img)
# Dimensions of the input image and output image (the same dimensions)
(width , height) = (img.shape[0], img.shape[1])
# Initialize the transformation matrix
T = np.array([[2,0,0], [0,2,0], [0,0,1]])
# Make an array with input image (x,y) coordinations and add [0 0 ... 1] row
coords = np.indices((width, height), 'uint8').reshape(2, -1)
coords = np.vstack((coords, np.zeros(coords.shape[1], 'uint8')))
output = T # coords
# Arrays of x and y coordinations of the output image within the image dimensions
x_array, y_array = output[0] ,output[1]
indices = np.where((x_array >= 0) & (x_array < width) & (y_array >= 0) & (y_array < height))
# Final coordinations of the output image
fx, fy = x_array[indices], y_array[indices]
# Final output image after the affine transformation
outImg[fx, fy] = img[fx, fy]
The input image is:
The output image after scaling is:

well you could simply use the opencv resize function
import cv2
new_image = cv2.resize(image, new_dim, interpolation=cv.INTER_AREA)
it'll do the resize and fill in the empty pixels in one go
more on cv2.resize

If you need to do it manually, then you could simply detect dark pixels in resized image and change their value to mean of 4 neighbour pixels (for example - it depends on your required alghoritm)
See: nereast neighbour, bilinear, bicubic, etc.

determining the average colour of a given circular sample of an image?

What I am trying to achieve is similar to photoshop/gimp's eyedropper tool: take a round sample of a given area in an image and return the average colour of that circular sample.
The simplest method I have found is to take a 'regular' square sample, mask it as a circle, then reduce it to 1 pixel, but this is very CPU-demanding (especially when repeated millions of times).
A more mathematically complex method is to take a square area and average only the pixels that fall within a circular area within that sample, but determining what pixel is or isn't within that circle, repeated, is CPU-demanding as well.
Is there a more succinct, less-CPU-demanding means to achieve this?

Here's a little example of skimage.draw.circle() which doesn't actually draw a circle but gives you the coordinates of points within a circle which you can use to index Numpy arrays with.
#!/usr/bin/env python3
import numpy as np
from skimage.io import imsave
from skimage.draw import circle
# Make rectangular canvas of mid-grey
w, h = 200, 100
img = np.full((h, w), 128, dtype=np.uint8)
# Get coordinates of points within a central circle
Ycoords, Xcoords = circle(h//2, w//2, 45)
# Make all points in circle=200, i.e. fill circle with 200
img[Ycoords, Xcoords] = 200
# Get mean of points in circle
print(img[Ycoords, Xcoords].mean()) # prints 200.0
# DEBUG: Save image for checking
imsave('result.png',img)

I'm sure that there's a more succinct way to go about it, but:
import math
import numpy as np
import imageio as ioimg # as scipy's i/o function is now depreciated
from skimage.draw import circle
import matplotlib.pyplot as plt
# base sample dimensions (rest below calculated on this).
# Must be an odd number.
wh = 49
# tmp - this placement will be programmed later
dp = 500
#load work image (from same work directory)
img = ioimg.imread('830.jpg')
# convert to numpy array (droppying the alpha while we're at it)
np_img = np.array(img)[:,:,:3]
# take sample of resulting array
sample = np_img[dp:wh+dp, dp:wh+dp]
#==============
# set up numpy circle mask
## this mask will be multiplied against each RGB layer in extracted sample area
# set up basic square array
sample_mask = np.zeros((wh, wh), dtype=np.uint8)
# set up circle centre coords and radius values
xy, r = math.floor(wh/2), math.ceil(wh/2)
# use these values to populate circle area with ones
rr, cc = circle(xy, xy, r)
sample_mask[rr, cc] = 1
# add axis to make array multiplication possible (do I have to do this)
sample_mask = sample_mask[:, :, np.newaxis]
result = sample * sample_mask
# count number of nonzero values (this will be our median divisor)
nz = np.count_nonzero(sample_mask)
sample_color = []
for c in range(result.shape[2]):
sample_color.append(int(round(np.sum(result[:,:,c])/nz)))
print(sample_color) # will return array like [225, 205, 170]
plt.imshow(result, interpolation='nearest')
plt.show()
Perhaps asking this question here wasn't necessary (it has been a while since I've python-ed, and was hoping that some new library had been developed for this since), but I hope this can be a reference for others who have the same goal.
This operation will be performed for every pixel in the image (sometimes millions of times) for thousands of images (scanned pages), so therein are my performance issue worries, but thanks to numpy, this code is pretty quick.

Unable to increase the region of interest of an image

I am trying to increase the region of interest of an image using the below algorithm.
First, the set of pixels of the exterior border of the ROI is de termined, i.e., pixels that are outside the ROI and are neighbors (using four-neighborhood) to pixels inside it. Then, each pixel value of this set is replaced with the mean value of its neighbors (this time using eight-neighborhood) inside the ROI. Finally, the ROI is expanded by inclusion of this altered set of pixels. This process is repeated and can be seen as artificially increasing the ROI.
The pseudocode is below -
while there are border pixels:
border_pixels = []
# find the border pixels
for each pixel p=(i, j) in image
if p is not in ROI and ((i+1, j) in ROI or (i-1, j) in ROI or (i, j+1) in ROI or (i, j-1) in ROI) or (i-1,j-1) in ROI or (i+1,j+1) in ROI):
add p to border_pixels
# calculate the averages
for each pixel p in border_pixels:
color_sum = 0
count = 0
for each pixel n in 8-neighborhood of p:
if n in ROI:
color_sum += color(n)
count += 1
color(p) = color_sum / count
# update the ROI
for each pixel p=(i, j) in border_pixels:
set p to be in ROI
Below is my code
img = io.imread(path_dir)
newimg = np.zeros((584, 565,3))
mask = img == 0
while(1):
border_pixels = []
for i in range(img.shape[0]):
for j in range(img.shape[1]):
for k in range(0,3):
if(i+1<=583 and j+1<=564 and i-1>=0 and j-1>=0):
if ((mask[i][j][k]) and ((mask[i+1][j][k]== False) or (mask[i-1][j][k]==False) or (mask[i][j+1][k]==False) or (mask[i][j-1][k]==False) or (mask[i-1][j-1][k] == False) or(mask[i+1][j+1][k]==False))):
border_pixels.append([i,j,k])
if len(border_pixels) == 0:
break
for (each_i,each_j,each_k) in border_pixels:
color_sum = 0
count = 0
eight_neighbourhood = [[each_i-1,each_j],[each_i+1,each_j],[each_i,each_j-1],[each_i,each_j+1],[each_i-1,each_j-1],[each_i-1,each_j+1],[each_i+1,each_j-1],[each_i+1,each_j+1]]
for pix_i,pix_j in eight_neighbourhood:
if (mask[pix_i][pix_j][each_k] == False):
color_sum+=img[pix_i,pix_j,each_k]
count+=1
print(color_sum//count)
img[each_i][each_j][each_k]=(color_sum//count)
for (i,j,k) in border_pixels:
mask[i,j,k] = False
border_pixels.remove([i,j,k])
io.imsave("tryout6.png",img)
But it is not doing any change in the image.I am getting the same image as before
so I tried plotting the border pixel on a black image of the same dimension for the first iteration and I am getting the below result-
I really don't have any idea where I am doing wrong here.

Here's a solution that I think works as you have requested (although I agree with #Peter Boone that it will take a while). My implementation has a triple loop, but maybe someone else can make it faster!
First, read in the image. With my method, the pixel values are floats between 0 and 1 (rather than integers between 0 and 255).
import urllib
import matplotlib.pyplot as plt
import numpy as np
from skimage.morphology import binary_dilation, binary_erosion, disk
from skimage.color import rgb2gray
from skimage.filters import threshold_otsu
# create a file-like object from the url
f = urllib.request.urlopen("https://i.stack.imgur.com/JXxJM.png")
# read the image file in a numpy array
# note that all pixel values are between 0 and 1 in this image
a = plt.imread(f)
Second, add some padding around the edges, and threshold the image. I used Otsu's method, but #Peter Boone's answer works well, too.
# add black padding around image 100 px wide
a = np.pad(a, ((100,100), (100,100), (0,0)), mode = "constant")
# convert to greyscale and perform Otsu's thresholding
grayscale = rgb2gray(a)
global_thresh = threshold_otsu(grayscale)
binary_global1 = grayscale > global_thresh
# define number of pixels to expand the image
num_px_to_expand = 50
The image, binary_global1 is a mask that looks like this:
Since the image is three channels (RGB), I process the channels separately. I noticed that I needed to erode the image by ~5 px because the outside of the image has some unusual colors and patterns.
# process each channel (RGB) separately
for channel in range(a.shape[2]):
# select a single channel
one_channel = a[:, :, channel]
# reset binary_global for the each channel
binary_global = binary_global1.copy()
# erode by 5 px to get rid of unusual edges from original image
binary_global = binary_erosion(binary_global, disk(5))
# turn everything less than the threshold to 0
one_channel = one_channel * binary_global
# update pixels one at a time
for jj in range(num_px_to_expand):
# get 1 px ring of to update
px_to_update = np.logical_xor(binary_dilation(binary_global, disk(1)),
binary_global)
# update those pixels with the average of their neighborhood
x, y = np.where(px_to_update == 1)
for x, y in zip(x,y):
# make 3 x 3 px slices
slices = np.s_[(x-1):(x+2), (y-1):(y+2)]
# update a single pixel
one_channel[x, y] = (np.sum(one_channel[slices]*
binary_global[slices]) /
np.sum(binary_global[slices]))
# update original image
a[:,:, channel] = one_channel
# increase binary_global by 1 px dilation
binary_global = binary_dilation(binary_global, disk(1))
When I plot the output, I get something like this:
# plot image
plt.figure(figsize=[10,10])
plt.imshow(a)

This is an interesting idea. You're going to want to use masks and some form of mean ranks to accomplish this. Going pixel by pixel will take you a while, instead you want to use different convolution filters.
If you do something like this:
image = io.imread("roi.jpg")
mask = image[:,:,0] < 30
just_inside = binary_dilation(mask) ^ mask
image[~just_inside] = [0,0,0]
you will have a mask representing just the pixels inside of the ROI. I also set the pixels not in that area to 0,0,0.
Then you can get the pixels just outside of the roi:
just_outside = binary_erosion(mask) ^ mask
Then get the mean bilateral of each channel:
mean_blue = mean_bilateral(image[:,:,0], selem=square(3), s0=1, s1=255)
#etc...
This isn't exactly correct, but I think it should put you in the right direction. I would check out image.sc if you have more general questions about image processing. Let me know if you need more help as this was more general direction than working code.

How to update numpy array based based on coordinates

I have an image that is given as below, which I am trying to convert into a binary image.
To convert this image into a binary image, I did:
image[image > 0] = 255
This creates a binary image with colored region having only white pixels. But I also want to convert the pixels that are above the colored region to value 255. How could I do this? I do not only want to convert the colored pixels to white but also the area above it. Is there I could do this? The area denoted by arrows will remain black (i.e area after the colored region)
UPDATE
Also, how could I approach if the edges are as shown below:

If I understood correctly your problem a more elaborated approach should be taken to get the desired result.
First of all the approach to use a simple threshold creates a noisy approach.
I used a modified image of your sample:
If you apply your thresholding then a result like this might come up:
A finer thresholding can be useful here:
image3 = cv2.inRange(image, np.array([10, 10, 10]), np.array([255, 255, 255]))
which creates a binary image as a result (resembles your desired output except for the upper strip):
TO get rid of the strip I would (it's just an approach not the perfect one though) use something to find the corner created by the white region and then use it to draw all the region above it with black:
ind = np.where(image3 == 255)
max_x = np.max(ind[1])
max_y = ind[0][np.argmax(ind[1])]
image3[:max_y, :max_x] = 255
And the result would be like this:

By all mean this is not the perfect answer. But it might be something helpful.
I recreated the image as follows:
Then I read it and followed your path with making it binary first (with a small modification to reduce noises):
import numpy as np
from matplotlib import pyplot as plt
img = plt.imread("sample.jpg")
img2 = img.copy()
img2[img2.sum(-1) > 30] = 255
img2[img2.sum(-1) <= 30] = 0
Here is the result after this modification:
OPTION 1
This might not be what you asked but it is similar to one of the solutions discussed in the comments, and I think it is partly correct:
i, j = np.where(img2.sum(-1) > 0) # find all white coordinates
i, j = (i[j.argmax()], j[j.argmax()]) # the corner white point into the black
img2[:i, :j] = 255 # paint whine all the left-above rectangle from this point
Here is the final result:
This is an imperfect but pretty simple pure numpy solution.
OPTION 2
In this solution, we need some simple calculus and linear algebra. Take two points in 2D space and draw a line between them. So, what is the function of the borderline?
point2 = (i, j) # same i and j from OPTION1 (coordinates of the top-right corner)
point1 = (img2.shape[0], img2[-1].sum(-1).argmin()) # the bottom-right white corner.
a = (point2[1] - point1[1]) / (point2[0] - point1[0])
c = point1[1] - a * point1[0]
f = lambda x: int(a * x + c)
Now, paint all areas to the left of the line:
for i in range(img2.shape[0]):
img2[:i, :f(i)+1] = 255
Here is the result:

Convert red line in image to numpy list

I want to measure the height difference to ground of the detected object in this pipe. The lower red line shall be the marker for minimum height. I thought I might first convert the lower red line into a numpy list but how would I do that? The red circles are drawn with cv2.circle() function.
Edit:
Thanks to ZdaR I am closer to solving my problem. This is his solution rewritten to work with python3:
import cv2
import numpy as np
def get_center(arr):
return sum(arr)/len(arr)
def get_cluster_centers(arr, tolerance):
clusters = [[arr[0]]]
for ele in arr[1:]:
if abs(clusters[-1][0] - ele) < tolerance:
clusters[-1].append(ele)
else:
clusters.append([ele])
clusters_centers = map(get_center, clusters)
return clusters_centers
img = cv2.imread("/home/artur/Desktop/0.png")
# Segment the red color
mask = cv2.inRange(img, np.array([0, 0, 255]), np.array([0, 0, 255]))
for i in mask:
print(i)
# Let's iterate the middle column and get the distance between the two red lines.
half_width = int(mask.shape[1]/2)
middle_column = mask[:, half_width]
idx = np.where(middle_column == 255)
# Since the width of line is not 1 px so we need to cluster the pixels to get a single center value.
centers = list(get_cluster_centers(idx[0], 5))
if len(centers) == 2:
print("Distance between lines:", centers[1] - centers[0], "px")
It measures the pixel distance between upper and lower red line with the help of the middle column of the image. How would I iterate over all columns to determine the minimum distance between those two line or better, between detected object and lower red line? Do I get it right that this solution only takes the middle column into account?

You can first segment the red color from the input image to get a binary mask then assuming that your red line is centered to the input image, we take the center column of that image and iterate over the column to find the red dot locations then simply find the distance in pixels as:
import cv2
import numpy as np
def get_center(arr):
return sum(arr)/len(arr)
def get_cluster_centers(arr, tolerance):
clusters = [[arr[0]]]
for ele in arr[1:]:
if abs(clusters[-1][0] - ele) < tolerance:
clusters[-1].append(ele)
else:
clusters.append([ele])
clusters_centers = map(get_center, clusters)
return clusters_centers
img = cv2.imread("/home/anmol/Downloads/HK3WM.png")
# Segment the red color
mask = cv2.inRange(img, np.array([0, 0, 255]), np.array([0, 0, 255]))
# Let's iterate the middle column and get the distance between the two red lines.
half_width = mask.shape[1]/2
middle_column = mask[:, half_width]
idx = np.where(middle_column == 255)
# Since the width of line is not 1 px so we need to cluster the pixels to get a single center value.
centers = get_cluster_centers(idx[0], 5)
if len(centers) == 2:
print "Distance between lines:", centers[1] - centers[0], "px"
PS: I am kind on in hurry if this does not explain something feel free to ask in comments.