Related
I want to emulate the blur of a cheap camera lens (like Holga).
Blur is very weak close to the photo center.
And it's getting more decisive close to corners.
I wrote the code and it works in general.
Input image:
Result image:
.
But I feel that it could be done better and faster.
I've found a similar question but it still has no answer.
How to improve an algorithm speed and avoid iteration over pixels?
UPDATE:
It's not the same as standard Gaussian or 2D filter blur with constant kernel size.
import cv2
import numpy as np
import requests
from tqdm import tqdm
import warnings
warnings.filterwarnings("ignore")
def blur(img=None, blur_radius=None, test=False):
# test image loading
if img is None:
test=True
print('test mode ON')
print('loading image...')
url = r'http://www.lenna.org/lena_std.tif'
resp = requests.get(url, stream=True).raw
img = np.asarray(bytearray(resp.read()), dtype="uint8")
img = cv2.imdecode(img, cv2.IMREAD_COLOR)
cv2.imwrite('img_input.png', img)
print('image loaded')
# channels splitting
img_lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
l, a, b = cv2.split(img_lab)
if test:
cv2.imwrite('l_channel.png', l)
print('l channel saved')
# make blur map
height, width = l.shape[:2]
center = np.array([height/2, width/2])
diag = ((height / 2) ** 2 + (width / 2) ** 2) ** 0.5
blur_map = np.linalg.norm(
np.indices(img.shape[:2]) - center[:,None,None] + 0.5,
axis = 0
)
if blur_radius is None:
blur_radius = int(max(height, width) * 0.03)
blur_map = blur_map / diag
blur_map = blur_map * blur_radius
if test:
blur_map_norm = cv2.normalize(blur_map, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_32F)
cv2.imwrite('blur_map.png', blur_map_norm)
print('blur map saved')
# very inefficient blur algorithm!!!
l_blur = np.copy(l)
for x in tqdm(range(width)):
for y in range(height):
kernel_size = int(blur_map[y, x])
if kernel_size == 0:
l_blur[y, x] = l[y, x]
continue
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (kernel_size, kernel_size))
cut = l[
max(0, y - kernel_size):min(height, y + kernel_size),
max(0, x - kernel_size):min(width, x + kernel_size)
]
if cut.shape == kernel.shape:
cut = (cut * kernel).mean()
else:
cut = cut.mean()
l_blur[y, x] = cut
if test: cv2.imwrite('l_blur.png', l_blur); print('l_blur saved')
if test: print('done')
return l_blur
blur()
The only way to implement a filter where the kernel is different for every pixel is to create the kernel for each pixel and apply it in a loop, like OP's code does. The Fourier transform does not apply to this case. Python is a very slow language, the same algorithm implemented in a compiled language would be much faster. Unless there is some predefined structure in how the kernel is created at each pixel, there is no way to reduce the complexity of the algorithm.
For example, the uniform filter with a square kernel (commonly called the "box" filter) can be computed based on the integral image, using only 4 additions per pixel. This implementation should be able to choose a different kernel size at each pixel without any additional cost.
DIPlib has an implementation of an adaptive Gaussian filter [disclaimer: I'm an author of DIPlib, but I did not implement this functionality]. Here is the documentation.
This filter applies a Gaussian filter, but the Gaussian kernel is scaled and rotated differently at every pixel.
Lens blur is not a Gaussian, but it's not easy to see the difference by eye in most cases; the difference matters only if there is a very small dot with high contrast.
OP's case would be implemented as follows:
import diplib as dip
img = dip.ImageRead('examples/trui.ics')
blur_map = dip.CreateRadiusSquareCoordinate(img.Sizes())
blur_map /= dip.Maximum(blur_map)
img_blur = dip.AdaptiveGauss(img, [0, blur_map], sigmas=[5])
(the blur_map here is defined differently, I chose a quadratic function of the distance to the center, because I think it looks really nice; use dip.CreateRadiusCoordinate() to reproduce OP's map).
I've chosen a maximum blur of 5 (this is the sigma, in pixels, of the Gaussian, not the footprint of the kernel), and blur_map here scales this sigma with a factor between 0 in the middle and 1 at the corners of the image.
Another interesting effect would be as follows, with increasing blur tangential to each circle centered in the middle of the image, with very little blur radially:
angle_map = dip.CreatePhiCoordinate(img.Sizes())
img_blur = dip.AdaptiveGauss(img, [angle_map, blur_map], sigmas=[8,1])
Here is one way to apply (uniform, non-varying) lens defocus blur in Python/OpenCV by transforming both the image and filter to the Fourier (frequency) domain.
Read the input
Take dft of input to transform to Fourier domain
Draw a white filled circle on a black background the size of the input as a mask (filter kernel). This is the defocus kernel in the spatial domain, i.e. a circular rect function.
Blur the circle slightly to anti-alias the edge
Roll the mask so that the center is at the origin (top left corner) and normalize so that the sum of values = 1
Take dft of mask to transform to Fourier domain where its amplitude profile is a jinx function.
Multiply the two dft images to apply the blur
Take the idft of the product to transform back to spatial domain
Get the magnitude of the real and imaginary components of the product, clip and convert to uint8 as the result
Save the result
Input:
import numpy as np
import cv2
# read input and convert to grayscale
img = cv2.imread('lena_512_gray.png', cv2.IMREAD_GRAYSCALE)
# do dft saving as complex output
dft_img = np.fft.fft2(img, axes=(0,1))
# create circle mask
radius = 32
mask = np.zeros_like(img)
cy = mask.shape[0] // 2
cx = mask.shape[1] // 2
cv2.circle(mask, (cx,cy), radius, 255, -1)[0]
# blur the mask slightly to antialias
mask = cv2.GaussianBlur(mask, (3,3), 0)
# roll the mask so that center is at origin and normalize to sum=1
mask_roll = np.roll(mask, (256,256), axis=(0,1))
mask_norm = mask_roll / mask_roll.sum()
# take dft of mask
dft_mask_norm = np.fft.fft2(mask_norm, axes=(0,1))
# apply dft_mask to dft_img
dft_shift_product = np.multiply(dft_img, dft_mask_norm)
# do idft saving as complex output
img_filtered = np.fft.ifft2(dft_shift_product, axes=(0,1))
# combine complex real and imaginary components to form (the magnitude for) the original image again
img_filtered = np.abs(img_filtered).clip(0,255).astype(np.uint8)
cv2.imshow("ORIGINAL", img)
cv2.imshow("MASK", mask)
cv2.imshow("FILTERED DFT/IFT ROUND TRIP", img_filtered)
cv2.waitKey(0)
cv2.destroyAllWindows()
# write result to disk
cv2.imwrite("lena_512_gray_mask.png", mask)
cv2.imwrite("lena_dft_numpy_lowpass_filtered_rad32.jpg", img_filtered)
Mask - Filter Kernel In Spatial Domain:
Result for Circle Radius=4:
Result for Circle Radius=8:
Result for Circle Radius=16:
Result for Circle Radius=32
:
ADDITION
Using OpenCV for the dft, etc rather than Numpy, the above becomes:
import numpy as np
import cv2
# read input and convert to grayscale
img = cv2.imread('lena_512_gray.png', cv2.IMREAD_GRAYSCALE)
# do dft saving as complex output
dft_img = cv2.dft(np.float32(img), flags = cv2.DFT_COMPLEX_OUTPUT)
# create circle mask
radius = 32
mask = np.zeros_like(img)
cy = mask.shape[0] // 2
cx = mask.shape[1] // 2
cv2.circle(mask, (cx,cy), radius, 255, -1)[0]
# blur the mask slightly to antialias
mask = cv2.GaussianBlur(mask, (3,3), 0)
# roll the mask so that center is at origin and normalize to sum=1
mask_roll = np.roll(mask, (256,256), axis=(0,1))
mask_norm = mask_roll / mask_roll.sum()
# take dft of mask
dft_mask_norm = cv2.dft(np.float32(mask_norm), flags = cv2.DFT_COMPLEX_OUTPUT)
# apply dft_mask to dft_img
dft_product = cv2.mulSpectrums(dft_img, dft_mask_norm, 0)
# do idft saving as complex output, then clip and convert to uint8
img_filtered = cv2.idft(dft_product, flags=cv2.DFT_SCALE+cv2.DFT_REAL_OUTPUT)
img_filtered = img_filtered.clip(0,255).astype(np.uint8)
cv2.imshow("ORIGINAL", img)
cv2.imshow("MASK", mask)
cv2.imshow("FILTERED DFT/IFT ROUND TRIP", img_filtered)
cv2.waitKey(0)
cv2.destroyAllWindows()
# write result to disk
cv2.imwrite("lena_512_gray_mask.png", mask)
cv2.imwrite("lena_dft_opencv_defocus_rad32.jpg", img_filtered)
I want to demonstrate the Gaussian Kernel used in openCV. cv2.GaussianBlurr(img, kernel_size, sigma) for explanation purposes.
I know how to demonstrate the image which results after applying the blur, and that is not my objective here.
My objective is to demonstrate the kernel automatically for any used sigma, and any used kernel size!
I have seen a code(mentioned down) but I prefer to use something more related to instruction used in OpenCV, rather than just a general mathematical dependent approach.
The expected output kernel is something like this:
import cv2
import numpy as np
# Read Image
img_path = 'image.jpg'
img = cv2.imread(img_path)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# Gaussian Blurr
Kernel = np.ones((15,15))
sigma = 2
Blurred_Image = cv2.GaussianBlur(img, (Kernel.shape[0], Kernel.shape[1]), sigma)
Gaussian Kernel Manual Code:
def dnorm(x, mu, sd):
return 1 / (np.sqrt(2 * np.pi) * sd) * np.e ** (-np.power((x - mu) / sd, 2) / 2)
def gaussian_kernel(size, sigma=1, verbose=False):
kernel_1D = np.linspace(-(size // 2), size // 2, size)
for i in range(size):
kernel_1D[i] = dnorm(kernel_1D[i], 0, sigma)
kernel_2D = np.outer(kernel_1D.T, kernel_1D.T)
kernel_2D *= 1.0 / kernel_2D.max()
if verbose:
plt.imshow(kernel_2D, interpolation='none',cmap='gray')
plt.title("Image")
plt.show()
return kernel_2D
Here is one way in Python/OpenCV.
- Read the input
- Create a delta image (one white pixel in the center of a black background)
- Blur the image
- List item
- Resize the image to enlarge it
- Stretch the image to full dynamic range
- Save the result
import cv2
import numpy as np
import skimage.exposure as exposure
# create delta image
dims = 30
dims2 = 30 // 2
delta = np.zeros((dims,dims,3), dtype=np.float32)
delta[dims2:dims2+1, dims2:dims2+1] = (255,255,255)
# blur image
blur = cv2.GaussianBlur(delta, (0,0), sigmaX=5, sigmaY=5)
# resize 16x
dims4x = dims * 16
resized = cv2.resize(blur, (dims4x,dims4x), interpolation = cv2.INTER_AREA)
# stretch to full dynamic range
result = exposure.rescale_intensity(resized, in_range='image', out_range=(0,255)).astype(np.uint8)
# save image
cv2.imwrite('delta.png',delta)
cv2.imwrite('gaussian_blur_view.png',result)
# show the images
cv2.imshow("delta", delta)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Delta image:
Result:
I'm trying to add a random noise from uniform distribution between min pixel
value and 0.1 times the maximum pixel value to each pixel for each channel of original image.
Here's my code so far:
[in]:
import cv2
import numpy as np
import matplotlib.pyplot as plt
# Read image with cv2
image = cv2.imread('example_image.jpg' , 1)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Display image
imshow(image_rgb)
# R,G,B channel separation
R, G, B = cv2.split(image_rgb)
# Creating Noise
noise_R = np.random.uniform(R.min(),R.max()*0.1, R.size)
noise_R.shape = (256,256)
noise_G = np.random.uniform(B.min(),B.max()*0.1, G.size)
noise_G.shape = (256,256)
noise_B = np.random.uniform(G.min(), G.max()*0.1, B.size)
noise_B.shape = (256,256)
# Adding noise to each channel separately
R = R + noise_R
G = G + noise_G
B = B + noise_B
rgb_noise = R + G + B
noisy_image = image + rgb_noise
[out]:
ValueError: operands could not be broadcast together with shapes (256,256,3) (256,256)
I'm getting an ValueError that the array shapes for rgb_noise and image are not equal. I've tried changing the shape of rgb_noise to that of image's but the I get a size error. How to fix it ? Is there any better method ?
Your solution is a bit verbose, and could be made more compact.
However, the reason why you do not get white-ish noise is that you compute your red channel differently from the other two.
Changing this:
noise_R = np.random.uniform(R_min,R_max*0.3, image_G.size)
to this:
noise_R = np.random.uniform(R_min,R_max*0.1, image_R.size)
You can be simplistic and add the noise by only the numpy array.
import numpy
import matplotlib.pyplot as plt
import cv2
Look, plotting the image will only work good with jupyter notebooks.
Do cv2.imshow() for other IDEs.
1) Have your Image
img = cv2.imread('path').astype(np.uint0)
2) Make a random noise
r, g, b = img.shape
noise = np.random.randint(0,255,r*g*b).reshape(r,g,b)
3) Blend them
image_with_noise = cv2.addWeighted(img,0.5,noise,0.5,0)
You can adjust the value of alpha and beta values.
There you have a noisy image!
I received this opencv error 1820: error: (-215:Assertion failed) when developing some Trilinear Interpolation Code
I'm trying to figure out what it means.
I'm also currently wondering if my coding methodology is correct for Trilinear.
Also, is there a way to get the flow algorithm to work in colour
instead of grayscale? I tried in colour, but received a similar error.
Three Frames for download here
Frame 1
Frame 2
Frame 3
Code:
import numpy as np
import cv2
import scipy as sp
Image1_Fp = ".\\3325_71.png" to
Image2_Fp = ".\\3325_72.png" to
Image3_Fp = ".\\3325_73.png" to
Write_Image_Location_Tri_Sampled = ".\\3325_syn_Tri.png"
def imread(filename):
im = cv2.imread(filename)
return im
if __name__ == '__main__':
im1 = imread(Image1_Fp)
im2 = imread(Image2_Fp)
im3 = imread(Image3_Fp)
im3gray = cv2.cvtColor(im3, cv2.COLOR_BGR2GRAY)
im1gray = cv2.cvtColor(im1, cv2.COLOR_BGR2GRAY)
flow = cv2.calcOpticalFlowFarneback(im1gray, im3gray, flow=1, pyr_scale=0.5, levels=2, winsize=10, iterations=5, poly_n=7, poly_sigma=1.2, flags=0)
# Trilinear Interpolation using meshgrid and remap
half_flow = 0.5 * flow # reduce the flow lines all by half.
h, w = flow.shape[:2]
grid_x, grid_y = np.meshgrid(h, w) # Identifies the coordinates of the grid with x, and y value,
coor_x_1 = grid_x + half_flow[:, :, 0] # So the flow is a displacement, and you need to add the pixel location to it to get where that displacement came from on the grid
coor_y_1 = grid_y + half_flow[:, :, 1]
# Finds the interpolated intensity from the pixels closest to the flow line from RGB Image 1
output_1 = cv2.remap(im1, coor_x_1, coor_y_1, cv2.INTER_CUBIC, borderMode =cv2.BORDER_REPLICATE)
coor_x_2 = grid_x - half_flow[:, :, 0]
coor_y_2 = grid_y - half_flow[:, :, 1]
# Finds the interpolated intensity from the pixels closest to the flow line from RGB Image 2
output_2 = cv2.remap(im2, coor_x_2, coor_y_2, cv2.INTER_CUBIC, borderMode =cv2.BORDER_REPLICATE)
# combined half of each
Combined_Output = (output_1 / 2) + (output_2 / 2) # Applies Trilinear part for a synthesized batch
cv2.imwrite(Write_Image_Location_Tri_Sampled, Combined_Output)
I am expecting to see a synthesized image that looks similar to the one in-between the two images, but I receive this error.
Exception has occurred: error
OpenCV(4.1.0) C:\projects\opencv-python\opencv\modules\imgproc\src\imgwarp.cpp:1820: error: (-215:Assertion failed) ((map1.type() == CV_32FC2 || map1.type() == CV_16SC2) && map2.empty()) || (map1.type() == CV_32FC1 && map2.type() == CV_32FC1) in function 'cv::remap'
File "..\OpenCVOpticalFLowAndTrilinearInterp.py", line 90, in <module>
output_1 = cv2.remap(im1, coor_x_1, coor_y_1, cv2.INTER_CUBIC, borderMode =cv2.BORDER_REPLICATE)
Forgive me if I am unable to explain well because I am not native speaker.
I am working on blurring the part of image according to the white part of segmentation map. For example here is my segmentation image ( bmp image ).
.
Now what I want is to blur the part of original image where the pixels are white in the segmentation map. I just wrote the following code to so.
mask = mask >= 0.5
mask = np.reshape(mask, (512, 512))
mh, mw = 512, 512
mask_n = np.ones((mh, mw, 3))
mask_n[:,:,0] *= mask
mask_n[:,:,1] *= mask
mask_n[:,:,2] *= mask
# discard padded area
ih, iw, _ = image_n.shape
delta_h = mh - ih
delta_w = mw - iw
top = delta_h // 2
bottom = mh - (delta_h - top)
left = delta_w // 2
right = mw - (delta_w - left)
mask_n = mask_n[top:bottom, left:right, :]
# addWeighted
image_n = image_n *1 + cv2.blur(mask_n * 0.8, (800, 800))
Please help me, Thanks.
You can do it in the following steps:
Load original image and mask image.
Blur the whole original image and save it in a different variable.
Use np.where() method to select the pixels from the mask where you want blurred values and then replace it.
See the sample code below:
import cv2
import numpy as np
img = cv2.imread("./image.png")
blurred_img = cv2.GaussianBlur(img, (21, 21), 0)
mask = cv2.imread("./mask.png")
output = np.where(mask==np.array([255, 255, 255]), blurred_img, img)
cv2.imwrite("./output.png", output)
Here's an alternative to the solution proposed by #Chris Henri. It relies on scipy.ndimage.filters.gaussian_filter and NumPy's boolean indexing:
from skimage import io
import numpy as np
from scipy.ndimage.filters import gaussian_filter
import matplotlib.pyplot as plt
mask = io.imread('https://i.stack.imgur.com/qJiKf.png')
img = np.random.random(size=mask.shape[:2])
idx = mask.min(axis=-1) == 255
blurred = gaussian_filter(img, sigma=3)
blurred[~idx] = 0
fig, axs = plt.subplots(1, 3, figsize=(12, 4))
for ax, im in zip(axs, [img, mask, blurred]):
ax.imshow(im, cmap='gray')
ax.set_axis_off()
plt.show(fig)
Here is yet another alternative to do so, useful though when you have a 2D segmentation array indicating the segmented object class of pixel (mutually exclusive) for every index (i,j), and a 3D image on which you want to apply the blur.
def gaussian_blur(image: np.ndarray,
segmentation: np.ndarray,
classes_of_interest: list,
gaussian_variance: float = 10) -> np.ndarray:
'''
Function that applies a gaussian filter to the image,
specifically to the pixels contained in the possible segmented classes.
Returns an image (np.ndarray) where the gaussian blur intensity is
regulated by the parameter gaussian_variance.
'''
#Apply masking to select only the indices where the specific class is present
mask = np.isin(segmentation, classes_of_interest)
#Creating a 3D mask for all the channels and place it at channel axis
mask_3d = np.stack([mask,mask,mask], axis=2)
#Mask the image according to the 3D mask
img_masked = np.where(mask_3d, img, 0).astype(np.int8)
#Define gaussian blur noisy function
def noisy(image):
row,col,ch= image.shape
mean = 0
var = gaussian_variance
sigma = np.sqrt(var)
gauss = np.random.normal(mean,sigma,(row,col,ch))
gauss = gauss.reshape(row,col,ch)
#Sums up gaussian noise to img
noisy = image + gauss
return noisy.astype(np.uint8)
#Blurs the masked segmentation
img_masked_noisy = noisy(img_masked)
#Puts the blurred part back in the original image as substitution
img[mask_3d] = img_masked_noisy[mask_3d]
return img
And here is a toy example:
import numpy as np
possible_classes = [1,2,3]
#Setting up a toy example with a small image,
#shape (N, N, 3)
img = np.floor(np.random.random(size=(8,8,3)) * 256).astype(np.uint8)
#Setting up a fake segmentation with 3 mutually exclusive possible classes,
#shape (N, N)
segmentation = np.random.choice(possible_classes, size=(8,8))
new_img_blurred = gaussian_blur(img,
segmentation= segmentation,
classes_of_interest= possible_classes[:2])