Develop Reference

Why I am not able to extract the object using the masked image?

Here are two images in the drive, one is masked and another one is real image:
https://drive.google.com/drive/folders/1hv3NLQeIHT5Iicgnt74S5qFRG5aJyMLw?usp=sharing
I want to get the masked object out of the real image, not in the white color but in the actual color that it does have.
Here is the code that I hav written:
import cv2
import numpy as np
img = cv2.imread('im010.jpg')
mask = cv2.imread('im010.png')
img_foreground = np.array((mask/255)*img)
cv2.imshow('', img_foreground)
cv2.waitKey()
I have converted all masked elements into one, and multiplied it with the actual image. Since the rest of the pixels that are not masked are black, there it will be zero. If any number multiply with it, it remains zero. Multiplication with one value element ends up the same value of the real image. But I have ploted it, it is showing the same masked img?
Can anyone show me the solution?

Here are 4 different ways to do that in Python/OpenCV/Numpy.
import cv2
import numpy as np
# read input
img = cv2.imread('im010.jpg')
# Method 1 -- Bitwise_And
mask = cv2.imread('im010.png',0)
result1 = cv2.bitwise_and(img, img, mask=mask)
cv2.imwrite('im010_masked1.jpg', result1)
cv2.imshow("result1", result1)
cv2.waitKey(0)
cv2.destroyAllWindows()
# Method 2 -- Python Multiplication
mask = cv2.imread('im010.png')
result2 = (img * mask.astype(np.float64)/255).clip(0,255).astype(np.uint8)
cv2.imwrite('im010_masked2.jpg', result2)
cv2.imshow("result2", result2)
cv2.waitKey(0)
cv2.destroyAllWindows()
# Method 3 -- Numpy Blackening
mask = cv2.imread('im010.png',0)
result3 = img.copy()
result3[mask==0] = (0,0,0)
cv2.imwrite('im010_masked3.jpg', result3)
cv2.imshow("result3", result3)
cv2.waitKey(0)
cv2.destroyAllWindows()
# Method 4 -- Numpy Where
mask = cv2.imread('im010.png')
result4 = np.where(mask==255, img, mask)
cv2.imwrite('im010_masked4.jpg', result4)
cv2.imshow("result4", result4)
cv2.waitKey(0)
cv2.destroyAllWindows()
They all produce the following result:

your code just needs an additional /255.
import cv2
import numpy as np
img = cv2.imread('im010.jpg')
mask = cv2.imread('im010.png')
img_foreground = np.array((mask/255)*(img/255))
cv2.imshow('', img_foreground)
cv2.waitKey()
that's because, when you look at the values, they're floats, right? and when they're floats, they must be in the range of 0.0 to 1.0, because that's what imshow expects.
when you give it values scaled differently, it still maps 1.0 to white. any larger value (like 2, 3...) is also white, even if they would be shown as very dark in a range of 0..255, which goes for uint8 type input.

What is wrong with my focus stacking algorithm?

You can see in the final focus stacked image that the whole image is in focus. However, pieces of the image are missing and I have no clue why. The basic steps of my algorithm are:
Access images. Convert images to grayscale, blur the gray images a bit, then find the Laplacian of these images. I store all Laplaced images in a list.
Cycle through pixels in a blank image using for loops. Every iteration creates a list containing the pixel intensities of the gray, blurred, Laplaced images at that pixel value. Find the max pixel intensity. Then look at the BGR value of the ORIGINAL image where the max pixel intensity came from. Set the BGR value of the blank pixel equal to that of the max-intensity pixel.
Here is my code:
images = glob2.glob("Pics\\step*") # Accesses images in the Pics folder
laps = [] # A list to contain Laplacians of images in Pics
i=0
for image in images:
img = cv.imread(image) # Reads image in Pics
images[i] = img # Bc line 6 only creates a list of image NAMES (ie strings), not actual images, this replaces string w image
img = cv.cvtColor(img, cv.COLOR_BGR2GRAY) # Converts image to grayscale
gauss = cv.GaussianBlur(img, (3,3), 0) # Blurs grayed image a bit
lap = cv.Laplacian(gauss, cv.CV_64F) # Converts blurred, gray image to Laplacian
lap = np.uint8(np.absolute(lap)) # Converts to Laplacian
laps.append(lap) # Adds Laplacian to laps
i += 1
sample = laps[0] # Arbitrarily accesses the first image in laps so that we can get dimensions for line 22
fs = np.zeros((sample.shape[0], sample.shape[1], 3), dtype='uint8') # Creates a blank image with the dimensions of sample
for x in range(sample.shape[0]): # The for loops go through every x and y value
for y in range(sample.shape[1]):
intensities = [laps[0][x,y], laps[1][x,y], laps[2][x,y], laps[3][x,y], laps[4][x,y], laps[5][x,y]] # List of intensities of laplacian images
color = images[intensities.index(max(intensities))][x,y] # Finds BGR value of the x,y pixel in the ORIGINAL image corresponding to the highest intensity
fs[x, y] = color # Sets pixel of blank fs image to the color of the pixel with the strongest intensity
cv.imshow('FS', fs)
Here is what the code produces:
Broken Focus Stacked Image

I was inspired by your code and made this simple script, which seems to work fine. (I do not need to align images.) Using mask to select pixels in focus may be faster, but I haven't tried to compare both versions. I would appreciate any advice on how to improve it.
from pathlib import Path
from imageio import imread, imwrite
import numpy as np
import matplotlib.pyplot as plt
from skimage.color import rgb2hsv, rgb2gray
from skimage import img_as_float, img_as_ubyte
from scipy.ndimage.filters import gaussian_filter
from skimage.filters.rank import gradient
from skimage.morphology import disk
im_dir = Path("test")
sigma = 3
print("_____ load images _____")
fps = [f for f in im_dir.glob("*.jpg")]
print([f.name for f in fps])
images_rgb = [imread(f) for f in fps]
images_rgb_cube = np.array(images_rgb)
print("images_rgb_cube", images_rgb_cube.shape, images_rgb_cube.dtype)
print("_____ images to grey _____")
#images_grey = [rgb2hsv(im)[:,:,2] for im in ims] # slow
images_grey = [rgb2gray(im) for im in images_rgb] # faster
print("_____ get gradients _____")
selection_element = disk(sigma) # matrix of n pixels with a disk shape
grads = [gradient(im, selection_element) for im in images_grey]
grads = np.array(grads)
print("grads", grads.shape, grads.dtype)
print("_____ get mask _____")
mask_grey = grads.max(axis=0, keepdims=1) == grads # https://stackoverflow.com/questions/47678252/mask-from-max-values-in-numpy-array-specific-axis
mask_rgb = np.repeat(mask_grey[:, :, :, np.newaxis], 3, axis=3)
print("mask_rgb", mask_rgb.shape, mask_rgb.dtype)
print("_____ apply mask _____")
image_sharp = images_rgb_cube * mask_rgb
image_sharp = image_sharp.max(axis=0)
print("image_sharp", image_sharp.shape, image_sharp.dtype)
print("_____ save image _____")
imwrite(im_dir / "stacked.jpeg", image_sharp)
plt.imshow(image_sharp)
plt.show()
print("_____ save masks _____")
print("mask_grey", mask_grey.shape, mask_grey.dtype)
for i in range(mask_grey.shape[0]):
mask = mask_grey[i]
fp = im_dir / "{}_mask.jpeg".format(fps[i].stem)
imwrite(fp, img_as_ubyte(mask))
print("saved", fp, mask.shape, mask.dtype)

Smooth the edges of binary images (Face) using Python and Open CV

I am looking for a perfect way to smooth edges of binary images. The problem is the binary image appears to be a staircase like borders which is very unpleasing for my further masking process.
I am attaching a raw binary image that is to be converted into smooth edges and I am also providing the expected outcome. I am also looking for a solution that would work even if we increase the dimensions of the image.
Problem Image Expected Outcome

To preserve the sharpness of a binary image, I would recommend applying something like a median filter. Here is an example of this:
from PIL import Image, ImageFilter
image = Image.open('input_image.png')
image = image.filter(ImageFilter.ModeFilter(size=13))
image.save('output_image.png')
which gives us the following results:
Figure 1. Left: The original input image. Right: The output image with a median filter of size 13.
Increasing the size of the filter would increase the degree of smoothing, but of course this comes as a trade-off because you also lose high-frequency information such as the sharp corner on the bottom-left of this sample image. Unfortunately, high-frequency features are similar in nature to the staircase-like borders.

You can do that in Python/OpenCV with the help of Skimage by blurring the binary image. Then apply a one-sided clip.
Input:
import cv2
import numpy as np
import skimage.exposure
# load image
img = cv2.imread('bw_image.png')
# blur threshold image
blur = cv2.GaussianBlur(img, (0,0), sigmaX=3, sigmaY=3, borderType = cv2.BORDER_DEFAULT)
# stretch so that 255 -> 255 and 127.5 -> 0
# C = A*X+B
# 255 = A*255+B
# 0 = A*127.5+B
# Thus A=2 and B=-127.5
#aa = a*2.0-255.0 does not work correctly, so use skimage
result = skimage.exposure.rescale_intensity(blur, in_range=(127.5,255), out_range=(0,255))
# save output
cv2.imwrite('bw_image_antialiased.png', result)
# Display various images to see the steps
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
You will have to adjust the amount of blur for the degree of aliasing in the image.

How to add synthetic noise in an image with specified error probability

I want to create synthetic noise within an image. How will I degrade the black and white image with errors, with an independent probability of error at each point. How will I do that in Python (e.g. Error probability = 0.0011)?

Here's a vectorized approach using OpenCV + skimage.util.random_noise. You can experiment with noise modes such as localvar, pepper, s&p, and speckle to obtain the desired result. You can set the proportion of noise with the amount parameter. Here's an example using s&p with amount=0.011:
import cv2
import numpy as np
from skimage.util import random_noise
# Load the image
image = cv2.imread('1.png', 0)
# Add salt-and-pepper noise to the image
noise = random_noise(image, mode='s&p', amount=0.011)
# The above function returns a floating-point image in the range [0, 1]
# so need to change it to 'uint8' with range [0,255]
noise = np.array(255 * noise, dtype=np.uint8)
cv2.imshow('noise',noise)
cv2.imwrite('noise.png',noise)
cv2.waitKey()

Here's an example program simply replacing the "degraded" pixels with black, using the Pillow library
from PIL import Image
import random
img = Image.open('text.png')
pixels = img.load()
for x in range(img.size[0]):
for y in range(img.size[1]):
if random.random() < 0.011:
pixels[x,y] = 0 # only 1 number given since the image is grayscale
img.save('text_degraded.png')
I've increased the probability to 0.011 to make it more noticeable, here's the output

Low Pass Filter for blurring an image

I'm trying to blur an image using fft by passing a low pass filter that I created but the output yields to be an image full of gray noise. I'm just trying to follow the basics here but it seems like there is something wrong with my implementation:
from scipy import fftpack
import numpy as np
import imageio
from PIL import Image, ImageDraw
image1 = imageio.imread('image.jpg',as_gray=True)
#convert image to numpy array
image1_np=np.array(image)
#fft of image
fft1 = fftpack.fftshift(fftpack.fft2(image1_np))
#Create a low pass filter image
x,y = image1_np.shape[0],image1_np.shape[1]
#size of circle
e_x,e_y=50,50
#create a box
bbox=((x/2)-(e_x/2),(y/2)-(e_y/2),(x/2)+(e_x/2),(y/2)+(e_y/2))
low_pass=Image.new("L",(image1_np.shape[0],image1_np.shape[1]),color=0)
draw1=ImageDraw.Draw(low_pass)
draw1.ellipse(bbox, fill=255)
low_pass_np=np.array(low_pass)
low_pass_fft=fftpack.fftshift(fftpack.fft2(low_pass))
#multiply both the images
filtered=np.multiply(fft1,low_pass_fft)
#inverse fft
ifft2 = abs(fftpack.ifft2(fftpack.ifftshift(filtered)))
#save the image
imageio.imsave('fft-then-ifft.png', ifft2.astype(np .uint8))

As mentioned in comments by Cris Luengo, there are a few things that need to be corrected:
The provided elliptical shape for the low-pass filter makes sense in the frequency-domain, so you shouldn't be computing its FFT.
The filter magnitude of 255 scales the results by the same amount. As you store such large values, the uint8 type wraps around to keep only the 8 least significant bits, resulting in something that looks like noise. This can be fixed by simply changing the value of the filter:
draw1.ellipse(bbox, fill=1)
After readjusting the scaling, there computed filtered may still get slightly out of the desired 0-255 range in some areas of the image. This creates wrap-around spots (black areas in regions surrounded by white pixels, white areas in regions surrounded by black pixels, or even gradient bands where the image goes from white to black to white). To avoid this is common to clip the values to the 0-255 range with the following:
ifft2 = np.real(fftpack.ifft2(fftpack.ifftshift(filtered)))
ifft2 = np.maximum(0, np.minimum(ifft2, 255))
After making these corrections, you should have the following code:
from scipy import fftpack
import numpy as np
import imageio
from PIL import Image, ImageDraw
image1 = imageio.imread('image.jpg',as_gray=True)
#convert image to numpy array
image1_np=np.array(image1)
#fft of image
fft1 = fftpack.fftshift(fftpack.fft2(image1_np))
#Create a low pass filter image
x,y = image1_np.shape[0],image1_np.shape[1]
#size of circle
e_x,e_y=50,50
#create a box
bbox=((x/2)-(e_x/2),(y/2)-(e_y/2),(x/2)+(e_x/2),(y/2)+(e_y/2))
low_pass=Image.new("L",(image1_np.shape[0],image1_np.shape[1]),color=0)
draw1=ImageDraw.Draw(low_pass)
draw1.ellipse(bbox, fill=1)
low_pass_np=np.array(low_pass)
#multiply both the images
filtered=np.multiply(fft1,low_pass_np)
#inverse fft
ifft2 = np.real(fftpack.ifft2(fftpack.ifftshift(filtered)))
ifft2 = np.maximum(0, np.minimum(ifft2, 255))
#save the image
imageio.imsave('fft-then-ifft.png', ifft2.astype(np .uint8))
And the following filtered image: