Develop Reference

Combining foreground png image with jpg background

I've bellow function:
def alphaMerge(small_foreground, background, top, left):
result = background.copy()
fg_b, fg_g, fg_r, fg_a = cv.split(small_foreground)
print(fg_b, fg_g, fg_r, fg_a)
fg_a = fg_a / 255.0
label_rgb = cv.merge([fg_b * fg_a, fg_g * fg_a, fg_r * fg_a])
height, width = small_foreground.shape[0], small_foreground.shape[1]
part_of_bg = result[top:top + height, left:left + width, :]
bg_b, bg_g, bg_r = cv.split(part_of_bg)
part_of_bg = cv.merge([bg_b * (1 - fg_a), bg_g * (1 - fg_a), bg_r * (1 - fg_a)])
cv.add(label_rgb, part_of_bg, part_of_bg)
result[top:top + height, left:left + width, :] = part_of_bg
return result
if __name__ == '__main__':
folder_dir = r"C:\photo_datasets\products_small"
logo = cv.imread(r"C:\Users\PiotrSnella\photo_datasets\discount.png", cv.IMREAD_UNCHANGED)
for images in os.listdir(folder_dir):
input_path = os.path.join(folder_dir, images)
image_size = os.stat(input_path).st_size
if image_size < 8388608:
img = cv.imread(input_path, cv.IMREAD_UNCHANGED)
height, width, channels = img.shape
if height > 500 and width > 500:
result = alphaMerge(logo, img, 0, 0)
cv.imwrite(r'C:\photo_datasets\products_small_output_cv\{}.png'.format(images), result)
I want to combine two pictures, one with the logo which I would like to apply on full dataset from folder products_small. I'm getting a error part_of_bg = cv.merge([bg_b * (1 - fg_a), bg_g * (1 - fg_a), bg_r * (1 - fg_a)]) ValueError: operands could not be broadcast together with shapes (720,540) (766,827)
I tried other combining options and still get the error about problem with shapes, the photo could be a problem or something with the code?
Thank you for your help guys :)

Here is one way to do that in Python/OpenCV. I will place a 20% resized logo onto the pants image at coordinates 660,660 on the right side pocket.
Read the background image (pants)
Read the foreground image (logo) unchanged to preserve the alpha channel
Resize the foreground (logo) to 20%
Create a transparent image the size of the background image
Insert the resized foreground (logo) into the transparent image at the desired location
Extract the alpha channel from the inserted, resized foreground image
Extract the base BGR channels from the inserted, resized foreground image
Blend the background image and the base BGR image using the alpha channel as a mask using np.where(). Note all images must be the same dimensions and 3 channels
Save the result
Background Image:
Foreground Image:
import cv2
import numpy as np
# read background image
bimg = cv2.imread('pants.jpg')
hh, ww = bimg.shape[:2]
# read foreground image
fimg = cv2.imread('flashsale.png', cv2.IMREAD_UNCHANGED)
# resize foreground image
fimg_small = cv2.resize(fimg, (0,0), fx=0.2, fy=0.2)
ht, wd = fimg_small.shape[:2]
# create transparent image
fimg_new = np.full((hh,ww,4), (0,0,0,0), dtype=np.uint8)
# insert resized image into transparent image at desired coordinates
fimg_new[660:660+ht, 660:660+wd] = fimg_small
# extract alpha channel from foreground image as mask and make 3 channels
alpha = fimg_new[:,:,3]
alpha = cv2.merge([alpha,alpha,alpha])
# extract bgr channels from foreground image
base = fimg_new[:,:,0:3]
# blend the two images using the alpha channel as controlling mask
result = np.where(alpha==(0,0,0), bimg, base)
# save result
cv2.imwrite("pants_flashsale.png", result)
# show result
cv2.imshow("RESULT", result)
cv2.waitKey(0)
Result:

This just requires some multiplication and subtraction.
Your overlay has an actual alpha channel, not just a boolean mask. You should use it. It makes edges look better than just a hard boolean mask.
I see one issue with your overlay: it doesn't have any "shadow" to give the white text contrast against a potentially white background.
When you resize RGBA data, it's not trivial. You'd better export the graphic from your vector graphics program in the desired resolution in the first place. Resizing after the fact requires operations to make sure partially transparent pixels (neither 100% opaque nor 100% transparent) are calculated properly so undefined "background" from the fully transparent areas of the overlay image is not mixed into those partially transparent pixels.
base = cv.imread("U3hRd.jpg")
overlay = cv.imread("OBxGQ.png", cv.IMREAD_UNCHANGED)
(bheight, bwidth) = base.shape[:2]
(oheight, owidth) = overlay.shape[:2]
print("base:", bheight, bwidth)
print("overlay:", oheight, owidth)
# place overlay in center
#ox = (bwidth - owidth) // 2
#oy = (bheight - oheight) // 2
# place overlay in top left
ox = 0
oy = 0
overlay_color = overlay[:,:,:3]
overlay_alpha = overlay[:,:,3] * np.float32(1/255)
# "unsqueeze" (insert 1-sized color dimension) so numpy broadcasting works
overlay_alpha = np.expand_dims(overlay_alpha, axis=2)
composite = base.copy()
base_roi = base[oy:oy+oheight, ox:ox+owidth]
composite_roi = composite[oy:oy+oheight, ox:ox+owidth]
composite_roi[:,:] = overlay_color * overlay_alpha + base_roi * (1 - overlay_alpha)

This is what you wanted on top left corner. Noticed, the logo on white foreground doesn't work on background on pant.jpg.
Just 17 lines of codes compared to
import cv2
import numpy as np
img1 = cv2.imread('pant.jpg')
overlay_img1 = np.ones(img1.shape,np.uint8)*255
img2 = cv2.imread('logo3.png')
rows,cols,channels = img2.shape
overlay_img1[0:rows, 0:cols ] = img2
img2gray = cv2.cvtColor(overlay_img1,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray,220,255,cv2.THRESH_BINARY_INV)
mask_inv = cv2.bitwise_not(mask)
temp1 = cv2.bitwise_and(img1,img1,mask = mask_inv)
temp2 = cv2.bitwise_and(overlay_img1,overlay_img1, mask = mask)
result = cv2.add(temp1,temp2)
cv2.imshow("Result",result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Logo resize(320x296):

OpenCV: Highlights Adjustment Without Affecting Shadows

I originally attempted to use the script located here to darken the highlights of a captured 8mm film frame without affecting the shadows.
I never call the script with parameters that returned an acceptable improvement in the image. I have since attempted to try this code:
import numpy as np
import cv2
def ResizeWithAspectRatio(image, width=None, height=None, inter=cv2.INTER_AREA):
dim = None
(h, w) = image.shape[:2]
if width is None and height is None:
return image
if width is None:
r = height / float(h)
dim = (int(w * r), height)
else:
r = width / float(w)
dim = (width, int(h * r))
return cv2.resize(image, dim, interpolation=inter)
def gammaCorrection(src, gamma):
invGamma = 1 / gamma
table = [((i / 255) ** invGamma) * 255 for i in range(256)]
table = np.array(table, np.uint8)
return cv2.LUT(src, table)
image = cv2.imread(r'X:\temp2\Test00001-02.jpg')
resize = ResizeWithAspectRatio(image, width=1280) # Resize by width OR
# resize = ResizeWithAspectRatio(image, height=1280) # Resize by height
cv2.imshow('Image', resize)
#gammaImg = gammaCorrection(resize, 0.3)
#cv2.imshow('Gamma corrected image', gammaImg)
# Convert to HSV and take V channel
V = cv2.cvtColor(resize,cv2.COLOR_BGR2HSV)[...,2]
# Threshold V channel at 100 to make alpha channel (A)
_, A = cv2.threshold(V,150,255,cv2.THRESH_BINARY)
# Threshold V channel at 100 to make alpha channel (A)
_, A2 = cv2.threshold(V,0,255,cv2.THRESH_BINARY)
# Stack A channel onto RGB channels
result = np.dstack((resize,A))
# Stack A channel onto RGB channels
fullchannel = np.dstack((resize,A2))
gammaImg = gammaCorrection(result, 0.3)
final_im = cv2.addWeighted(fullchannel, 1, gammaImg, 0.5, 0)
# Save result
cv2.imwrite(r'X:\temp2\original_im.png',resize)
cv2.imwrite(r'X:\temp2\final_im.png',final_im)
cv2.imwrite(r'X:\temp2\gamma_adjusted.png',gammaImg)
cv2.imwrite(r'X:\temp2\fullchannel.png',fullchannel)
The following images show the output of this script.
Original Image:
Gamma Adjusted Image:
Full Channel Image:
Final Image:
The final image actually makes the highlights in the image even brighter, so there is something I am doing wrong when I merge the gamma adjusted image with the full channel image.
So, any thoughts on getting just the highlights above a set threshold to darken and not affect the darker regions of the image?

Extract specific member of k-mean cluster of an image

I have an image (front facing man) with 4 different colors (background, hair, skin-tone, and cloth). I used k-mean with k=4, and image is segmented. Now what I want to do is extract out the hair out of the image.
I used canny edge detection to detect edge, which helped to detect the point in hair area(Pointed out by red dot). Now, I want to extract hair area, as the member of k-mean pointed out by the red dot. Is it possible?
Or is there any other way to extract out hair area from image of a person?
Code done till now is:
import cv2
import numpy as np
image1 = cv2.imread('Test1.jpg')
#Resizing Image for fixed width
def image_resize(image1, width = None, height = None, inter =
cv2.INTER_AREA):
# initialize the dimensions of the image to be resized and
# grab the image size
dim = None
(h, w) = image1.shape[:2]
# if both the width and height are None, then return the
# original image
if width is None and height is None:
return image1
# check to see if the width is None
if width is None:
# calculate the ratio of the height and construct the
# dimensions
r = height / float(h)
dim = (int(w * r), height)
# otherwise, the height is None
else:
# calculate the ratio of the width and construct the
# dimensions
r = width / float(w)
dim = (width, int(h * r))
# resize the image
resized = cv2.resize(image1, dim, interpolation = inter)
# return the resized image
return resized
img1 = image_resize(image1, width = 500)
cv2.imshow("Resized", img1)
cv2.waitKey(0)
#Detecting Edge of image
canny = cv2.Canny(img1, 100, 150)
cv2.imshow("Edge", canny)
cv2.waitKey(0)
coords = np.nonzero(canny)
topmost_y = np.min(coords[0])
#Blurring effect
img2 = cv2.medianBlur(img1, 5)
cv2.imshow("Blurred", img2)
cv2.waitKey(0)
#K-mean approach
Z = img2.reshape((-1,3))
Z = np.float32(Z)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
K=4
ret, label1, center1 = cv2.kmeans(Z, K, None,
criteria, 10,
cv2.KMEANS_RANDOM_CENTERS)
center1 = np.uint8(center1)
res1 = center1[label1.flatten()]
output1 = res1.reshape((img2.shape))
cv2.circle(output1, (250, topmost_y + 20), 5, (0,0,255), -1)
cv2.imshow("k = 4", output1)
cv2.waitKey(0)
cv2.destroyAllWindows()
Images:
,
,
,
,

Given the code you already have you can get the xy coordinates of the cluster to which the hair belongs with just a few extra lines. You can also create an image that shows only the hair's cluster:
# find the index of the cluster of the hair
mask = label1.reshape(output1.shape[:-1])
khair = mask[(topmost_y + 20, 250)]
# get a mask that's True at all of the indices of hair's group
hairmask = mask==khair
# get the hair's cluster's xy coordinates
xyhair = hairmask.nonzero()
# plot an image with only the hair's cluster on a white background
cv2.imwrite("khair.jpg", np.where(hairmask[..., None], img1, [255,255,255]))
Here's what the hair's cluster looks like:
Once you have the hair's cluster, you can then find the blob that represents "just the hair". Here's how you'd do that:
import scipy.ndimage as snd
# label all connected blobs in hairmask
bloblab = snd.label(hairmask, structure=np.ones((3,3)))[0]
# create a mask for only the hair
haironlymask = bloblab == bloblab[topmost_y + 20, 250]
# get an image with just the hair and then crop it
justhair = np.where(haironlymask[..., None], img1, [255,255,255])
nz = haironlymask.nonzero()
justhair = justhair[nz[0].min():nz[0].max(), nz[1].min():nz[1].max()]
# save the image of just the hair on a white background
cv2.imwrite("justhair.jpg", justhair)
and here's the image of your hair by itself:

Now that you have one point in this hair region, propagate this point to all the other points.
The pseudo code would be:
set = red point
while set of hair doesn't change:
add all points (i-1, j) (i+1, j) (i, j-1) (i, j+1) to the set
intersect the set with the mask of brown points
At the end, you will have a mask with the hair.
You can do that easily in numpy by starting with a Boolean image with just one True element at the red dot and then use |= and &= operators. I suspect OpenCV also has this kind of morphological dilation operator.

Classifying non-MNIST image after learning MNIST

My machine learning algorithm has already learned the 70000 images in the MNIST database. I want to test it on an image not included in the MNIST dataset. However, my predict function cannot read the array representation of my test image.
How do I test my algorithm on an external image?
Why is my code failing?
PS I'm using python3
Error Received:
Traceback (most recent call last):
File "hello_world2.py", line 28, in <module>
print(sgd_clf.predict(arr))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/base.py", line 336, in predict
scores = self.decision_function(X)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/base.py", line 317, in decision_function
% (X.shape[1], n_features))
ValueError: X has 15 features per sample; expecting 784
Code:
# Common Imports
import numpy as np
from sklearn.datasets import fetch_mldata
from sklearn.linear_model import SGDClassifier
from PIL import Image
from resizeimage import resizeimage
# loading and learning MNIST data
mnist = fetch_mldata('MNIST original')
x, y = mnist["data"], mnist["target"]
sgd_clf = SGDClassifier(random_state=42)
sgd_clf.fit(x, y)
# loading and converting to array a non-MNIST image of a "5", which is in the same folder
img = Image.open("5.png")
arr = np.array(img)
# trying to predict that the image is a "5"
img = Image.open("5.png")
img = img.convert('L') #makes it greyscale
img = resizeimage.resize_thumbnail(img, [28,28])
arr = np.array(img)
print(sgd_clf.predict(arr)) # ERROR... why????????? How do you fix it?????

It's not simply a matter of resizing, the image needs the digit centered and white on black etc. I've been working on a function to this job. This is the current version that uses opencv, although it could do with further improvement, including using PIL instead of opencv, but it should give an idea of the steps required.
def open_as_mnist(image_path):
"""
Assume this is a color or grey scale image of a digit which has not so far been preprocessed
Black and White
Resize to 20 x 20 (digit in center ideally)
Sharpen
Add white border to make it 28 x 28
Convert to white on black
"""
# open as greyscale
image = cv2.imread(image_path, 0)
# crop to contour with largest area
cropped = do_cropping(image)
# resizing the image to 20 x 20
resized20 = cv2.resize(cropped, (20, 20), interpolation=cv2.INTER_CUBIC)
cv2.imwrite('1_resized.jpg', resized20)
# gaussian filtering
blurred = cv2.GaussianBlur(resized20, (3, 3), 0)
# white digit on black background
ret, thresh = cv2.threshold(blurred, 127, 255, cv2.THRESH_BINARY_INV)
padded = to20by20(thresh)
resized28 = padded_image(padded, 28)
# normalize the image values to fit in the range [0,1]
norm_image = np.asarray(resized28, dtype=np.float32) / 255.
# cv2.imshow('image', norm_image)
# cv2.waitKey(0)
# # Flatten the image to a 1-D vector and return
flat = norm_image.reshape(1, 28 * 28)
# return flat
# normalize pixels to 0 and 1. 0 is pure white, 1 is pure black.
tva = [(255 - x) * 1.0 / 255.0 for x in flat]
return tva
def padded_image(image, tosize):
"""
This method adds padding to the image and makes it to a tosize x tosize array,
without losing the aspect ratio.
Assumes desired image is square
:param image: the input image as numpy array
:param tosize: the final dimensions
"""
# image dimensions
image_height, image_width = image.shape
# if not already square then pad to square
if image_height != image_width:
# Add padding
# The aim is to make an image of different width and height to a sqaure image
# For that first the biggest attribute among width and height are determined.
max_index = np.argmax([image_height, image_width])
# if height is the biggest one, then add padding to width until width becomes
# equal to height
if max_index == 0:
#order of padding is: top, bottom, left, right
left = int((image_height - image_width) / 2)
right = image_height - image_width - left
padded_img = cv2.copyMakeBorder(image, 0, 0,
left,
right,
cv2.BORDER_CONSTANT)
# else if width is the biggest one, then add padding to height until height becomes
# equal to width
else:
top = int((image_width - image_height) / 2)
bottom = image_width - image_height - top
padded_img = cv2.copyMakeBorder(image, top, bottom, 0, 0, cv2.BORDER_CONSTANT)
else:
padded_img = image
# now that it's a square, add any additional padding required
image_height, image_width = padded_img.shape
padding = tosize - image_height
# need to handle where padding is not divisiable by 2
left = top = int(padding/2)
right = bottom = padding - left
resized = cv2.copyMakeBorder(image, top, bottom, left, right, cv2.BORDER_CONSTANT)
return resized

If you want to read a picture then resize it, please try
In [1]: import PIL.Image as Image
In [2]: img = Image.open('2.jpg', mode='r')
In [3]: img.mode
Out[3]: 'RGB'
In [4]: img.size
Out[4]: (2880, 1800)
In [5]: img_new = img.resize([4000, 4000], Image.ANTIALIAS)
In [6]: img_new2 = img.resize([32, 32], Image.ANTIALIAS)
Docs are here
This is the 2.jpg, sorry, it is not a digit.
This picture is from the Internet, sorry, I forget the source.
If you encounter the mode is 'RGBA', I recommend you transfer it to 'RGB' mode,
newimg = Image.new('RGB', img.size)
newimg.paste(img, mask=img.split()[3])
return newimg

Please try this:
img = Image.open("5.png")
img = img.resize((28,28))
img = img.convert('L') #makes it greyscale

python opencv panorama blacklines

I am working on panorama with Python OpenCV. Can someone show me how to get rid of the black lines in my final images? I am thinking of maybe I should first check for the color I.e. 0,0,0 before copying it to the atlas image, but I am not quite sure how to do that.
def warpTwoImages(img1, img2, H):
'''warp img2 to img1 with homograph H'''
h1,w1 = img1.shape[:2]
h2,w2 = img2.shape[:2]
pts1 = np.float32([[0,0],[0,h1],[w1,h1],[w1,0]]).reshape(-1,1,2)
pts2 = np.float32([[0,0],[0,h2],[w2,h2],[w2,0]]).reshape(-1,1,2)
pts2_ = cv2.perspectiveTransform(pts2, H)
pts = np.concatenate((pts1, pts2_), axis=0)
[xmin, ymin] = np.int32(pts.min(axis=0).ravel() - 0.5)
[xmax, ymax] = np.int32(pts.max(axis=0).ravel() + 0.5)
t = [-xmin,-ymin]
Ht = np.array([[1,0,t[0]],[0,1,t[1]],[0,0,1]]) # translate
result = cv2.warpPerspective(img2, Ht.dot(H), (xmax-xmin, ymax-ymin))
result[t[1]:h1+t[1],t[0]:w1+t[0]] = img1
return result

This answer depends on warpPrespicteve function to work with RGBA.
You can try to use the alpha channel of each image.
Before wrapping convert each image to RGBA (See the code below) were the alpha channel will be 0 for the black lines and for all other pixels it will be 255.
import cv2
import numpy as np
# Read img
img = cv2.imread('i.jpg')
# Create mask from all the black lines
mask = np.zeros((img.shape[0],img.shape[1]),np.uint8)
cv2.inRange(img,(0,0,0),(1,1,1),mask)
mask[mask==0]=1
mask[mask==255]=0
mask = mask*255
b_channel, g_channel, r_channel = cv2.split(img)
# Create a new image with 4 channels the forth channel Aplha will give the opacity for each pixel
newImage = cv2.merge((b_channel, g_channel, r_channel, mask))