I have the following problem:
I want to extract only the color of a blue pen from scanned images that also contain grayscale and black printed areas on a white page background.
I'm okay with disregarding any kind of grayscale (not colored) pixel values and only keeping the blue parts, there won't be any dominant color other than blue on the images.
It sounds like a simple task, but the problem is that through the scanning process, the entire image contains colored pixels, including blue ones, even the grayscale or black parts, so I'm not sure how to go about isolating those parts and keeping only the blue ones, here is a closeup to show what I mean:
Here is what an image would look like for reference:
I would like the output to be a new image, containing only the parts drawn / written in blue pen, in this case the drawing of the hedgehog / eye.
So I've tried to isolate an HSV range for blue-ish colors in the image using this code:
img = cv.imread("./data/scan_611a720bcd70bafe7beb502d.jpg")
img_hsv = cv.cvtColor(img, cv.COLOR_BGR2HSV)
# accepted color range for blue pen
lower_blue = np.array([90, 35, 140])
upper_blue = np.array([150, 255, 255])
# preparing the mask to overlay
mask = cv.inRange(img_hsv, lower_blue, upper_blue)
inverted_mask = cv.bitwise_not(mask)
mask_blur = cv.GaussianBlur(inverted_mask, (5, 5), 0)
ret, mask_thresh = cv.threshold(mask_blur, 0, 255, cv.THRESH_BINARY + cv.THRESH_OTSU)
# The black region in the mask has the value of 0,
# so when multiplied with original image removes all non-blue regions
result = cv.bitwise_and(img, img, mask=mask)
cv.imshow("Result", mask_thresh)
k = cv.waitKey(0)
However the result is this:
Many parts of the picture that are drawn in black such as the cloud image are not removed since as mentioned, they contain blue / colored pixels due to the scanning process.
Is there any method that would allow for a clean isolation of those blue parts of the image even with those artifacts present?
The solution would need to work for any kind of image like this, the one given is just an example, but as mentioned the only color present would be the blue pen apart from the grey/black areas.
Maybe try the opposite- search for black parts first and then do some erosion around this black mask and remove all around it before you are searching for the blue. The "main" color in the cloud is still black so you can play around this.
You should realign the color planes of your scan. Then you're at least rid of those color fringes. I'd recommend scanning a sheet of graph paper to calibrate.
This is done using OpenCV's findTransformECC.
Complete examples can be found here:
https://docs.opencv.org/master/dd/d93/samples_2cpp_2image_alignment_8cpp-example.html
https://learnopencv.com/image-alignment-ecc-in-opencv-c-python/
And here's specific code to align the color planes of the picture given in the question:
https://gist.github.com/crackwitz/b8867b46f320eae17f4b2684416c79ea
(all it does is split the color planes, call findTransformECC and warpPerspective, merge the color planes)
I want to take one image, and overlay it as its outline only without background/filling. I have one image that is an outline in PNG format, that has had its background, as well as the contents within the outline removed, so that when opened, all is transparent except the outline, similar to this image:
However, when I open the image and try to overlay it in OpenCV, the background and area within the outline shows as all-white, showing the full rectangle of the image's dimensions and obscuring the background image.
However, what I want to do is the following, where only the outline is overlayed on the background image, like so:
Bonus points if you can help me with changing the color of the outline as well.
I don't want to deal with any blending with alphas, as I need the background to appear in full, and want the outline very clear.
In this special case, your image has some alpha channel you can use. Using Boolean array indexing, you can access all values 255 in the alpha channel. What's left to do, is setting up some region of interest (ROI) in the "background" image w.r.t. some position, and in that ROI, you again use Boolean array indexing to set all pixels to some color, i.e. red.
Here's some code:
import cv2
# Open overlay image, and its dimensions
overlay_img = cv2.imread('1W7HZ.png', cv2.IMREAD_UNCHANGED)
h, w = overlay_img.shape[:2]
# In this special case, take the alpha channel of the overlay image, and
# check for value 255; idx is a Boolean array
idx = overlay_img[:, :, 3] == 255
# Open image to work on
img = cv2.imread('path/to/your/image.jpg')
# Position for overlay image
top, left = (50, 50)
# Access region of interest with overlay image's dimensions at position
# img[top:top+h, left:left+w] and there, use Boolean array indexing
# to set the color to red (for example)
img[top:top+h, left:left+w, :][idx] = (0, 0, 255)
# Save image
cv2.imwrite('output.png', img)
That's the output for some random "background" image:
For the general case, i.e. without a proper alpha channel, you could threshold the overlay image to set up a proper mask for the Boolean array indexing.
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.8.5
OpenCV: 4.5.1
----------------------------------------
I am trying to use the Pillow (python-imaging-library) Python library in order to create an outline/stroke/border (with any color and width chosen) around my .png image. You can see here the original image and my wanted result (create by a phone app):
https://i.stack.imgur.com/4x4qh.png
You can download the png file of the original image here: https://pixabay.com/illustrations/brain-character-organ-smart-eyes-1773885/
I have done it in the medium size(1280x1138) but maybe it is better to do it with the smallest size (640x569).
I tried to solve the problem with two methods.
METHOD ONE
The first method is to create a fully blacked image of the brain.png image, enlarge it, and paste the original colored brain image on top of it. Here is my code:
brain_black = Image.open("brain.png") #load brain image
width = brain_black.width #in order not to type a lot
height = brain_black.height #in order not to type a lot
rectangle = Image.new("RGBA", (width, height), "black") #creating a black rectangle in the size of the brain image
brain_black.paste(rectangle, mask=brain_black) #pasting on the brain image the black rectangle, and masking it with the brain picture
#now brain_black is the brain.png image, but all its pixels are black. Let's continue:
brain_black = brain_black.resize((width+180, height+180)) #resizing the brain_black by some factor
brain_regular = Image.open("brain.png") #load the brain image in order to paste later on
brain_black.paste(brain_regular,(90,90), mask=brain_regular) #paste the regular (colored) brain on top of the enlarged black brain (in x=90, y=90, the middle of the black brain)
brain_black.save("brain_method_resize.png") #saving the image
This method doesn't work, as you can see in the image link above. It might have worked for simple geometric shapes, but not for a complicated shape like this.
METHOD TWO
The second method is to load the brain image pixels data into a 2-dimensional array, and loop over all of the pixels. Check the color of every pixel, and in every pixel which is not transparent (means A(or Alpha) is not 0 in the rgbA form) to draw a black pixel in the pixel above, below, right, left, main diagonal down, main diagonal up, secondary diagonal (/) down and secondary diagonal (/) up. Then to draw a pixel in the second pixel above, the second pixel below and etc. this was done with a "for loop" where the number of repetitions is the wanted stroke width (in this example is 30). Here is my code:
brain=Image.open("brain.png") #load brain image
background=Image.new("RGBA", (brain.size[0]+400, brain.size[1]+400), (0, 0, 0, 0)) #crate a background transparent image to create the stroke in it
background.paste(brain, (200,200), brain) #paste the brain image in the middle of the background
pixelsBrain = brain.load() #load the pixels array of brain
pixelsBack=background.load() #load the pixels array of background
for i in range(brain.size[0]):
for j in range(brain.size[1]):
r, c = i+200, j+200 #height and width offset
if(pixelsBrain[i,j][3]!=0): #checking if the opacity is not 0, if the alpha is not 0.
for k in range(30): #the loop
pixelsBack[r, c + k] = (0, 0, 0, 255)
pixelsBack[r, c - k] = (0, 0, 0, 255)
pixelsBack[r + k, c] = (0, 0, 0, 255)
pixelsBack[r - k, c] = (0, 0, 0, 255)
pixelsBack[r + k, c + k] = (0, 0, 0, 255)
pixelsBack[r - k, c - k] = (0, 0, 0, 255)
pixelsBack[r + k, c - k] =(0, 0, 0, 255)
pixelsBack[r - k, c + k] = (0, 0, 0, 255)
background.paste(brain, (200,200), brain) #pasting the colored brain onto the background, because the loop "destroyed" the picture.
background.save("brain_method_loop.png")
This method did work, but it is very time-consuming (takes about 30 seconds just for one picture and 30 pixels stroke). I want to do it for many pictures so this method is not good for me.
Is there an easier and better way to reach my wanted result using Python Pillow library. How can I do it?
And also, how can I fasten my loop code (I understood something about Numpy and OpenCV, which is better for this purpose?)
I know that if a phone app could do it in a matter of milliseconds, also python can, but I didn't find any way to do it.
Thank you.
I tried some solution similar with photoshop stroke effect using OpenCV (It is not perfect and I still finding better solution)
This algorithm is based on euclidean distance transform. I also tried dilation algorithm with ellipse kernel structure, it is bit different with photoshop, and there are some information that distance transform is the way that photoshop using.
def stroke(origin_image, threshold, stroke_size, colors):
img = np.array(origin_image)
h, w, _ = img.shape
padding = stroke_size + 50
alpha = img[:,:,3]
rgb_img = img[:,:,0:3]
bigger_img = cv2.copyMakeBorder(rgb_img, padding, padding, padding, padding,
cv2.BORDER_CONSTANT, value=(0, 0, 0, 0))
alpha = cv2.copyMakeBorder(alpha, padding, padding, padding, padding, cv2.BORDER_CONSTANT, value=0)
bigger_img = cv2.merge((bigger_img, alpha))
h, w, _ = bigger_img.shape
_, alpha_without_shadow = cv2.threshold(alpha, threshold, 255, cv2.THRESH_BINARY) # threshold=0 in photoshop
alpha_without_shadow = 255 - alpha_without_shadow
dist = cv2.distanceTransform(alpha_without_shadow, cv2.DIST_L2, cv2.DIST_MASK_3) # dist l1 : L1 , dist l2 : l2
stroked = change_matrix(dist, stroke_size)
stroke_alpha = (stroked * 255).astype(np.uint8)
stroke_b = np.full((h, w), colors[0][2], np.uint8)
stroke_g = np.full((h, w), colors[0][1], np.uint8)
stroke_r = np.full((h, w), colors[0][0], np.uint8)
stroke = cv2.merge((stroke_b, stroke_g, stroke_r, stroke_alpha))
stroke = cv2pil(stroke)
bigger_img = cv2pil(bigger_img)
result = Image.alpha_composite(stroke, bigger_img)
return result
def change_matrix(input_mat, stroke_size):
stroke_size = stroke_size - 1
mat = np.ones(input_mat.shape)
check_size = stroke_size + 1.0
mat[input_mat > check_size] = 0
border = (input_mat > stroke_size) & (input_mat <= check_size)
mat[border] = 1.0 - (input_mat[border] - stroke_size)
return mat
def cv2pil(cv_img):
cv_img = cv2.cvtColor(cv_img, cv2.COLOR_BGRA2RGBA)
pil_img = Image.fromarray(cv_img.astype("uint8"))
return pil_img
output = stroke(test_image, threshold=0, stroke_size=10, colors=((0,0,0),))
I can't do a fully tested Python solution for you at the moment as I have other commitments, but I can certainly show you how to do it in a few milliseconds and give you some pointers.
I just used ImageMagick at the command line. It runs on Linux and macOS (use brew install imagemagick) and Windows. So, I extract the alpha/transparency channel and discard all the colour info. Then use a morphological "edge out" operation to generate a fat line around the edges of the shape in the alpha channel. I then invert the white edges so they become black and make all the white pixels transparent. Then overlay on top of the original image.
Here's the full command:
magick baby.png \( +clone -alpha extract -morphology edgeout octagon:9 -threshold 10% -negate -transparent white \) -flatten result.png
So that basically opens the image, messes about with a cloned copy of the alpha layer inside the parentheses and then flattens the black outline that results back onto the original image and saves it. Let's do the steps one at a time:
Extract the alpha layer as alpha.png:
magick baby.png -alpha extract alpha.png
Now fatten the edges, invert and make everything not black become transparent and save as overlay.png:
magick alpha.png -morphology edgeout octagon:9 -threshold 10% -negate -transparent white overlay.png
Here's the final result, change the octagon:9 to octagon:19 for fatter lines:
So, with PIL... you need to open the image and convert to RGBA, then split the channels. You don't need to touch the RGB channels just the A channel.
im = Image.open('baby.png').convert('RGBA')
R, G, B, A = im.split()
Some morphology needed here - see here.
Merge the original RGB channels with the new A channel and save:
result = Image.merge((R,G,B,modifiedA))
result.save('result.png')
Note that there are Python bindings to ImageMagick called wand and you may find it easier to translate my command-line stuff using that... wand. Also, scikit-image has an easy-to-use morphology suite too.
I've written this function which is based on morphological dilation and lets you set the stroke size and color. But it's EXTREMELY slow and it seems to not work great with small elements.
If anyone can help me speed it up it would be extremely helpful.
def addStroke(image,strokeSize=1,color=(0,0,0)):
#Create a disc kernel
kernel=[]
kernelSize=math.ceil(strokeSize)*2+1 #Should always be odd
kernelRadius=strokeSize+0.5
kernelCenter=kernelSize/2-1
pixelRadius=1/math.sqrt(math.pi)
for x in range(kernelSize):
kernel.append([])
for y in range(kernelSize):
distanceToCenter=math.sqrt((kernelCenter-x+0.5)**2+(kernelCenter-y+0.5)**2)
if(distanceToCenter<=kernelRadius-pixelRadius):
value=1 #This pixel is fully inside the circle
elif(distanceToCenter<=kernelRadius):
value=min(1,(kernelRadius-distanceToCenter+pixelRadius)/(pixelRadius*2)) #Mostly inside
elif(distanceToCenter<=kernelRadius+pixelRadius):
value=min(1,(pixelRadius-(distanceToCenter-kernelRadius))/(pixelRadius*2)) #Mostly outside
else:
value=0 #This pixel is fully outside the circle
kernel[x].append(value)
kernelExtent=int(len(kernel)/2)
imageWidth,imageHeight=image.size
outline=image.copy()
outline.paste((0,0,0,0),[0,0,imageWidth,imageHeight])
imagePixels=image.load()
outlinePixels=outline.load()
#Morphological grayscale dilation
for x in range(imageWidth):
for y in range(imageHeight):
highestValue=0
for kx in range(-kernelExtent,kernelExtent+1):
for ky in range(-kernelExtent,kernelExtent+1):
kernelValue=kernel[kx+kernelExtent][ky+kernelExtent]
if(x+kx>=0 and y+ky>=0 and x+kx<imageWidth and y+ky<imageHeight and kernelValue>0):
highestValue=max(highestValue,min(255,int(round(imagePixels[x+kx,y+ky][3]*kernelValue))))
outlinePixels[x,y]=(color[0],color[1],color[2],highestValue)
outline.paste(image,(0,0),image)
return outline
Very simple and primitive solution: use PIL.ImageFilter.FIND_EDGES to find edge of drawing, it is about 1px thick, and draw a circle in every point of the edge. It is quite fast and require few libs, but has a disadvantage of no smoothing.
from PIL import Image, ImageFilter, ImageDraw
from pathlib import Path
def mystroke(filename: Path, size: int, color: str = 'black'):
outf = filename.parent/'mystroke'
if not outf.exists():
outf.mkdir()
img = Image.open(filename)
X, Y = img.size
edge = img.filter(ImageFilter.FIND_EDGES).load()
stroke = Image.new(img.mode, img.size, (0,0,0,0))
draw = ImageDraw.Draw(stroke)
for x in range(X):
for y in range(Y):
if edge[x,y][3] > 0:
draw.ellipse((x-size,y-size,x+size,y+size),fill=color)
stroke.paste(img, (0, 0), img )
# stroke.show()
stroke.save(outf/filename.name)
if __name__ == '__main__':
folder = Path.cwd()/'images'
for img in folder.iterdir():
if img.is_file(): mystroke(img, 10)
Solution using PIL
I was facing the same need: outlining a PNG image.
Here is the input image:
Input image
I see that some solution have been found, but in case some of you want another alternative, here is mine:
Basically, my solution workflow is as follow:
Read and fill the non-alpha chanel of the PNG image with the border
color
Resize the unicolor image to make it bigger
Merge the original image to the bigger unicolor image
Here you go! You have an outlined PNG image with the width and color of your choice.
Here is the code implementing the workflow:
from PIL import Image
# Set the border and color
borderSize = 20
color = (255, 0, 0)
imgPath = "<YOUR_IMAGE_PATH>"
# Open original image and extract the alpha channel
im = Image.open(imgPath)
alpha = im.getchannel('A')
# Create red image the same size and copy alpha channel across
background = Image.new('RGBA', im.size, color=color)
background.putalpha(alpha)
# Make the background bigger
background=background.resize((background.size[0]+borderSize, background.size[1]+borderSize))
# Merge the targeted image (foreground) with the background
foreground = Image.open(imgPath)
background.paste(foreground, (int(borderSize/2), int(borderSize/2)), foreground.convert("RGBA"))
imageWithBorder = background
imageWithBorder.show()
And here is the outputimage:
Output image
Hope it helps!
I found a way to do this using the ImageFilter module, it is much faster than any custom implementation that I've seen here and doesn't rely on resizing which doesn't work for convex hulls
from PIL import Image, ImageFilter
stroke_radius = 5
img = Image.open("img.png") # RGBA image
stroke_image = Image.new("RGBA", img.size, (255, 255, 255, 255))
img_alpha = img.getchannel(3).point(lambda x: 255 if x>0 else 0)
stroke_alpha = img_alpha.filter(ImageFilter.MaxFilter(stroke_radius))
# optionally, smooth the result
stroke_alpha = stroke_alpha.filter(ImageFilter.SMOOTH)
stroke_image.putalpha(stroke_alpha)
output = Image.alpha_composite(stroke_image, img)
output.save("output.png")
I am trying to combine some parts of the image together while still maintaining some parts unchanged.
This is first image
This is the code to get the first image, the parameter for the input are img which is original image but already colorized with green while jawline,eyebrows,etc are (x,y) coordinates to cut those parts from the image
def getmask(img,jawline,eyebrows,eyes,mouth):
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
imArray = np.asarray(img)
# create mask
polygon = jawline.flatten().tolist()
maskIm = Image.new('L', (imArray.shape[1], imArray.shape[0]), 0)
ImageDraw.Draw(maskIm).polygon(polygon, outline=1, fill='white')
#ImageDraw.Draw(maskIm).polygon(polygon, outline=(1))
# draw eyes
righteyes=eyes[0:6].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(righteyes, outline=1, fill='black')
lefteyes=eyes[6:].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(lefteyes, outline=1, fill='black')
# draw eyebrows
rightbrows=eyebrows[0:6].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(rightbrows, outline=2, fill='black')
leftbrows=eyebrows[6:].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(leftbrows, outline=2, fill='black')
# draw mouth
mouth=mouth.flatten().tolist()
ImageDraw.Draw(maskIm).polygon(mouth, outline=1, fill='black')
mask = np.array(maskIm)
mask = np.multiply(img,mask)+ np.multiply((1-mask),np.ones((L,P,3)))
return mask
This is the second image which will fill the white blank inside the first image
I used this code to cut the parts which is very similar to the code on first image.
def getface(img,eyebrows,eyes,mouth):
im=img.copy()
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
imArray = np.asarray(img)
# create mask
maskIm = Image.new('L', (imArray.shape[1], imArray.shape[0]), 0)
righteyes=eyes[0:6].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(righteyes, outline=1,fill='white')
lefteyes=eyes[6:].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(lefteyes, outline=1,fill='white')
# draw eyebrows
rightbrows=eyebrows[0:6].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(rightbrows, outline=2, fill='white')
leftbrows=eyebrows[6:].flatten().tolist()
ImageDraw.Draw(maskIm).polygon(leftbrows, outline=2, fill='white')
# draw mouth
mouth=mouth.flatten().tolist()
ImageDraw.Draw(maskIm).polygon(mouth, outline=1, fill='white')
cutted_part = np.array(maskIm)
cutted_part = cv2.bitwise_or(im,im,mask=mask)
return cutted_part
So far I have tried to combine those two images by first inversing the second image so that the black background become white and then multiply the first and second image. But the result isn't satisfactory.
As you can see, there are some white space between the combined area and I notice that some part from second image become smaller or missing which I suspect create those white space when combined (Please don't mind the slightly different color on the result). Maybe someone can share how to resolve this problem or has better ways to combine 2 images together?
If you provide your results as actual pictures instead of cropped screenshots we can reproduce your problem, so far i would recommend:
Invert the background of your cutout (black to white) and then simply combine both pictures either by adding them (They need to have the same dimensions, which i presume is the case.) or overlaying them by using opencv's addWeighted function to adjust opacity.
I created the following image (image 3) using a threshold mask (image 2) on image 1. I am trying to convert all the pixels outside of the central image of image 3 (of lungs) to one colour (for example black) using opencv. Basically so that I am left with just the image of the lungs against a uniform background (or even transparent). My problem has been the similarity of the very outer pixels to those inside the lungs on image 3. Is this possible to do using opencv?
Simply floodFill() the mask from the boundaries of the image with black. See the flood fill step in my answer here to see it used in another scenario.
Similarly, you can use floodFill() to find which pixels connect to the edges of the image, which means you can use it to put back the holes in the lungs from thresholding. See my answer here for a different example of this hole-filling process.
I copy and pasted the code straight from the above answers, only modifying the variable names:
import cv2
import numpy as np
img = cv2.imread('img.jpg', 0)
mask = cv2.imread('mask.png', 0)
# flood fill to remove mask at borders of the image
h, w = img.shape[:2]
for row in range(h):
if mask[row, 0] == 255:
cv2.floodFill(mask, None, (0, row), 0)
if mask[row, w-1] == 255:
cv2.floodFill(mask, None, (w-1, row), 0)
for col in range(w):
if mask[0, col] == 255:
cv2.floodFill(mask, None, (col, 0), 0)
if mask[h-1, col] == 255:
cv2.floodFill(mask, None, (col, h-1), 0)
# flood fill background to find inner holes
holes = mask.copy()
cv2.floodFill(holes, None, (0, 0), 255)
# invert holes mask, bitwise or with mask to fill in holes
holes = cv2.bitwise_not(holes)
mask = cv2.bitwise_or(mask, holes)
# display masked image
masked_img = cv2.bitwise_and(img, img, mask=mask)
masked_img_with_alpha = cv2.merge([img, img, img, mask])
cv2.imwrite('masked.png', masked_img)
cv2.imwrite('masked_transparent.png', masked_img_with_alpha)
Edit: As an aside, "transparency" is basically a mask: the values tell you how opaque each pixel is. If the pixel is 0, its totally transparent, if it's 255 (for uint8) then it's completely opaque, if it's in-between then it's partially transparent. So the exact same mask used here at the end could be stacked onto the image to create the fourth alpha channel (you can use cv2.merge or numpy to stack) where it will make every 0 pixel in the mask totally transparent; simply save the image as a png for the transparency. The above code creates an image with alpha transparency as well as an image with a black background.
Here the background looks white because it is transparent, but if you save the image to your system you'll see it actually is transparent. FYI OpenCV actually ignores the alpha channel during imshow() so you'll only see the transparency on saving the image.
Edit: One last note...here your thresholding has removed some bits of the lungs. I've added back in the holes from thresholding that occur inside the lungs but this misses some chunks along the boundary that were removed. If you do contour detection on the mask, you can actually smooth those out a bit as well if it's important. Check out the "Contour Approximation" section on OpenCV's contour features tutorial. Basically it will try to smooth the contour but stick within some certain epsilon distance from the actual contour. This might be useful and is easy to implement, so I figured I'd throw it as a suggestion at the end here.