Develop Reference

How to automatically detect a specific feature from one image and map it to another mask image? Then how to smoothen only the corners of the image?

Using the dlib library I was able to mask the mouth feature from one image (masked).
masked
Similarly, I have another cropped image of the mouth that does not have the mask (colorlip).
colorlip
I had scaled and replaced the images (replaced) and using np.where as shown in the code below.
replaced
#Get the values of the lip and the target mask
lip = pred_toblackscreen[bbox_lip[0]:bbox_lip[1], bbox_lip[2]:bbox_lip[3],:]
target = roi[bbox_mask[0]:bbox_mask[1], bbox_mask[2]:bbox_mask[3],:]
cv2.namedWindow('masked', cv2.WINDOW_NORMAL)
cv2.imshow('masked', target)
#Resize the lip to be the same scale/shape as the mask
lip_h, lip_w, _ = lip.shape
target_h, target_w, _ = target.shape
fy = target_h / lip_h
fx = target_w / lip_w
scaled_lip = cv2.resize(lip,(0,0),fx=fx,fy=fy)
cv2.namedWindow('colorlip', cv2.WINDOW_NORMAL)
cv2.imshow('colorlip', scaled_lip)
update = np.where(target==[0,0,0],scaled_lip,target)
cv2.namedWindow('replaced', cv2.WINDOW_NORMAL)
cv2.imshow('replaced', update)
But the feature shape (lip) in 'colorlip' does not match the 'masked' image. So, there is a misalignment and the edges of the mask look sharp as if the image has been overlayed. How to solve this problem? And how to make the final replaced image look more subtle and normal?

**Update #2: OpenCV Image Inpainting to smooth jagged borders.
OpenCV python inpainting should help with rough borders. Using the mouth landmark model, mouth segmentation mask from DL model or anything that was used the border location can be found. From that draw border with a small chosen width around the mouth contour in a new image and use it as a mask for inpainting. The mask I provided need to be inverted to work.
In input masks one of the mask is wider, one has shadow and last one is narrow. The six output images are generated with radius value of 5 and 20 for all three masks.
Code
import numpy as np
# import cv2 as cv2
# import cv2
import cv2.cv2 as cv2
img = cv2.imread('images/lip_img.png')
#mask = cv2.imread('images/lip_img_border_mask.png',0)
mask = cv2.imread('images/lip_img_border_mask2.png',0)
#mask = cv2.imread('images/lip_img_border_mask3.png',0)
mask = np.invert(mask)
# Choose appropriate method and radius.
radius = 20
dst = cv2.inpaint(img, mask, radius, cv2.INPAINT_TELEA)
# dst = cv2.inpaint(img, mask, radius, cv2.INPAINT_NS)
cv2.imwrite('images/inpainted_lip.jpg', dst)
cv2.imshow('dst',dst)
cv2.waitKey(0)
cv2.destroyAllWindows()
Input Image and Masks
Output Images
**Update #1: Added Deep Image harmonization based blending methods.
Try OpenCV seamless cloning for subtle replacement and getting rid of sharp edges. Also deep learning based image inpainting on sharp corners or combining it with seamless clone may provide better results.
Deep learning based Image Harmonization can be another approach to blend together two images such that the cropped part matches the style of background image. Even in this case the pixel intensity will change to match the background but blending will be smoother. Links are added to bottom of the post.
Example
This code example is based on learnopencv seamless cloning example,
# import cv2
from cv2 import cv2
import numpy as np
src = cv2.imread("images/src_img.jpg")
dst = cv2.imread("images/dest_img.jpg")
src_mask = cv2.imread("images/src_img_rough_mask.jpg")
src_mask = np.invert(src_mask)
cv2.namedWindow('src_mask', cv2.WINDOW_NORMAL)
cv2.imshow('src_mask', src_mask)
cv2.waitKey(0)
# Where to place image.
center = (500,500)
# Clone seamlessly.
output = cv2.seamlessClone(src, dst, src_mask, center, cv2.NORMAL_CLONE)
# Write result
cv2.imwrite("images/opencv-seamless-cloning-example.jpg", output)
cv2.namedWindow('output', cv2.WINDOW_NORMAL)
cv2.imshow('output', output)
cv2.waitKey(0)
Source Image
Rough Mask Image
Destination Image
Final Image
Reference
https://docs.opencv.org/4.5.4/df/da0/group__photo__clone.html
https://learnopencv.com/seamless-cloning-using-opencv-python-cpp/
https://learnopencv.com/face-swap-using-opencv-c-python/
https://github.com/JiahuiYu/generative_inpainting
https://docs.opencv.org/4.x/df/d3d/tutorial_py_inpainting.html
Deep Image Harmonization
https://github.com/bcmi/Image-Harmonization-Dataset-iHarmony4
https://github.com/wasidennis/DeepHarmonization
https://github.com/saic-vul/image_harmonization
https://github.com/wuhuikai/GP-GAN
https://github.com/junleen/RainNet
https://github.com/bcmi/BargainNet-Image-Harmonization
https://github.com/vinthony/s2am

Fill image up to a size with Python

I am writing a handwriting recognition app and my inputs have to be of a certain size (128x128). When I detect a letter it looks like this:
That image for instance has a size of 40x53. I want to make it 128x128, but simply resizing it lowers the quality especially for smaller images. I want to somehow fill the rest up to 128x128 with the 40x53 in the middle. The background color should also stay relatively the same. I am using Python's opencv but I am new to it. How can I do this, and is it even possible?

Here you can get what you have asked using outputImage. Basically I have added a border using copyMakeBorder method. You can refer this for more details. You have to set the color value as you want in the value parameter. For now it is white [255,255,255].
But I would rather suggest you to resize the original image, seems like it is the better option than what you have asked. Get the image resized you can use resized in the following code. For your convenience I have added both methods in this code.
import cv2
import numpy as np
inputImage = cv2.imread('input.jpg', 1)
outputImage = cv2.copyMakeBorder(inputImage,37,38,44,44,cv2.BORDER_CONSTANT,value=[255,255,255])
resized = cv2.resize(inputImage, (128,128), interpolation = cv2.INTER_AREA)
cv2.imwrite('output.jpg', outputImage)
cv2.imwrite('resized.jpg', resized)

I believe you want to scale your image.
This code might help:
import cv2
img = cv2.imread('name_of_image', cv2.IMREAD_UNCHANGED)
# Get original size of image
print('Original Dimensions: ',img.shape)
# Percentage of the original size
scale_percent = 220
width = int(img.shape[1] * scale_percent / 100)
height = int(img.shape[0] * scale_percent / 100)
dim = (width, height)
# Resize/Scale the image
resized = cv2.resize(img, dim, interpolation = cv2.INTER_AREA)
# The new size of the image
print('Resized Dimensions: ',resized.shape)
cv2.imshow("Resized image", resized)
cv2.waitKey(0)
cv2.destroyAllWindows()

OpenCV return None whan imread is called with GDAL support

I have a very simple program in python with OpenCV and GDAL. In this program i read GeoTiff image with the following line
image = cv2.imread(sys.argv[1], cv2.IMREAD_LOAD_GDAL | cv2.IMREAD_COLOR)
The problem is for a specific image imread return None. I am using images from: https://www.sensefly.com/drones/example-datasets.html
Image in Assessing crops with RGB imagery (eBee SQ) > Map (orthomosaic) works well. Its size is: 19428, 19784 with 4 bands.
Image in Urban mapping (eBee Plus/senseFly S.O.D.A.) > Map (orthomosaic) doesn't work. Its size is: 26747, 25388 and 4 bands.
Any help to figure out what is the problem?
Edit: I tried the solution suggested by #en_lorithai and it works, the problem is then I need to do some image processing with OpenCV and the image loaded by GDAL has several issues
GDAL load images as RGB instead of BGR (used by default in OpenCV)
The image shape expected by OpenCV is (width, height, channels) and GDAL return an image with (channels, width, height) shape
The image returned by GDAL is flipped in Y-axe and rotate clockwise by 90 degree.
The image loaded by OpenCV is (resized to 700x700):
The image loaded by GDAL (after change shape, of course) is (resized to 700x700)
Finally, If I try to convert this image from BGR to RGB with
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
I get (resized to 700x700)
I can convert from GDAL format to OpenCV format with the following code
image = ds.ReadAsArray() #Load image with GDAL
tmp = image.copy()
image[0] = tmp[2,:,:] # swap read channel and blue channel
image[2] = tmp[0,:,:]
image = np.swapaxes(image,2,0) # convert from (height, width, channels) to (channels, height, width)
image = cv2.flip(image,0) # flip in Y-axis
image = cv2.transpose(image) # Rotate by 90 degress (clockwise)
image = cv2.flip(image,1)
The problem is I think that this is a very slow process and I want to know if there is a automatic convert-process.

You can try and open the image in gdal instead
from osgeo import gdal
g_image = gdal.Open('161104_hq_transparent_mosaic_group1.tif')
a_image = g_image.ReadAsArray()
can't test as i don't have enough available memory to open that image.
Edit: equivalent operation on another image
from osgeo import gdal
import matplotlib.pyplot as plt
g_image = gdal.Open('Water-scenes-014.jpg') # 3 channel rgb image
a_image = g_image.ReadAsArray()
s_image = np.dstack((a_image[0],a_image[1],a_image[2]))
plt.imshow(s_image) # show image in matplotlib (no need for color swap)
s_image = cv2.cvtColor(s_image,cv2.COLOR_RGB2BGR) # colorswap for cv
cv2.imshow('name',s_image)
Another method of getting individual bands from gdal
g_image = gdal.Open('image_name.PNG')
band1 = g_image.GetRasterBand(1).ReadAsArray()
You can then do a numpy dstack of each of the bands.

Drawing semi-transparent polygons in PIL

How do you draw semi-transparent polygons using the Python Imaging Library?

Can you draw the polygon on a separate RGBA image then use the Image.paste(image, box, mask) method?
Edit: This works.
from PIL import Image
from PIL import ImageDraw
back = Image.new('RGBA', (512,512), (255,0,0,0))
poly = Image.new('RGBA', (512,512))
pdraw = ImageDraw.Draw(poly)
pdraw.polygon([(128,128),(384,384),(128,384),(384,128)],
fill=(255,255,255,127),outline=(255,255,255,255))
back.paste(poly,mask=poly)
back.show()
http://effbot.org/imagingbook/image.htm#image-paste-method

I think #Nick T's answer is good, but you need to be careful when using his code as written with a very large background image, especially in the case that you may be annotating several polygons on said image. This is something I do when processing huge satellite images with some object detection code and annotating the detections using a transparent rectangle. To make the code efficient no matter the size of the background image, I make the following suggestion.
I would modify the solution to specify that the polygon image that you will paste be only as large as required to hold the polygon, not the same size as the back image. The coordinates of the polygon are specified with respect to the local bounding box, not the global image coordinates. Then you paste the polygon image at the offset in the larger background image.
import Image
import ImageDraw
img_size = (512,512)
poly_size = (256,256)
poly_offset = (128,128) #location in larger image
back = Image.new('RGBA', img_size, (255,0,0,0) )
poly = Image.new('RGBA', poly_size )
pdraw = ImageDraw.Draw(poly)
pdraw.polygon([ (0,0), (256,256), (0,256), (256,0)],
fill=(255,255,255,127), outline=(255,255,255,255))
back.paste(poly, poly_offset, mask=poly)
back.show()

Using the Image.paste(image, box, mask) method will convert the alpha channel in the pasted area of the background image into the corresponding transparency value of the polygon image.
The Image.alpha_composite(im1,im2) method utilizes the alpha channel of the "pasted" image, and will not turn the background transparent. However, this method again needs two equally sized images.

PIL Image.resize() not resizing the picture

I have some strange problem with PIL not resizing the image.
from PIL import Image
img = Image.open('foo.jpg')
width, height = img.size
ratio = floor(height / width)
newheight = ratio * 150
img.resize((150, newheight), Image.ANTIALIAS)
img.save('mugshotv2.jpg', format='JPEG')
This code runs without any errors and produces me image named mugshotv2.jpg in correct folder, but it does not resize it. It does something to it, because the size of the picture drops from 120 kb to 20 kb, but the dimensions remain the same.
Perhaps you can also suggest way to crop images into squares with less code. I kinda thought that Image.thumbnail does it, but what it did was that it scaled my image to 150 px by its width, leaving height 100px.

resize() returns a resized copy of an image. It doesn't modify the original. The correct way to use it is:
from PIL import Image
#...
img = img.resize((150, newheight), Image.ANTIALIAS)
source
I think what you are looking for is the ImageOps.fit function. From PIL docs:
ImageOps.fit(image, size, method, bleed, centering) => image
Returns a sized and cropped version of
the image, cropped to the requested
aspect ratio and size. The size
argument is the requested output size
in pixels, given as a (width, height)
tuple.

[Update]
ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.image.resize((100,100),Image.ANTIALIAS)

Today you should use something like this:
from PIL import Image
img = Image.open(r"C:\test.png")
img.show()
img_resized = img.resize((100, 100), Image.Resampling.LANCZOS)
img_resized.show()