When photographing a sheet of paper (e.g. with phone camera), I get the following result (left image) (jpg download here). The desired result (processed manually with an image editing software) is on the right:
I would like to process the original image with openCV to get a better brightness/contrast automatically (so that the background is more white).
Assumption: the image has an A4 portrait format (we don't need to perspective-warp it in this topic here), and the sheet of paper is white with possibly text/images in black or colors.
What I've tried so far:
Various adaptive thresholding methods such as Gaussian, OTSU (see OpenCV doc Image Thresholding). It usually works well with OTSU:
ret, gray = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY)
but it only works for grayscale images and not directly for color images. Moreover, the output is binary (white or black), which I don't want: I prefer to keep a color non-binary image as output
Histogram equalization
applied on Y (after RGB => YUV transform)
or applied on V (after RGB => HSV transform),
as suggested by this answer (Histogram equalization not working on color image - OpenCV) or this one (OpenCV Python equalizeHist colored image):
img3 = cv2.imread(f)
img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2YUV)
img_transf[:,:,0] = cv2.equalizeHist(img_transf[:,:,0])
img4 = cv2.cvtColor(img_transf, cv2.COLOR_YUV2BGR)
cv2.imwrite('test.jpg', img4)
or with HSV:
img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2HSV)
img_transf[:,:,2] = cv2.equalizeHist(img_transf[:,:,2])
img4 = cv2.cvtColor(img_transf, cv2.COLOR_HSV2BGR)
Unfortunately, the result is quite bad since it creates awful micro contrasts locally (?):
I also tried YCbCr instead, and it was similar.
I also tried CLAHE (Contrast Limited Adaptive Histogram Equalization) with various tileGridSize from 1 to 1000:
img3 = cv2.imread(f)
img_transf = cv2.cvtColor(img3, cv2.COLOR_BGR2HSV)
clahe = cv2.createCLAHE(tileGridSize=(100,100))
img_transf[:,:,2] = clahe.apply(img_transf[:,:,2])
img4 = cv2.cvtColor(img_transf, cv2.COLOR_HSV2BGR)
cv2.imwrite('test.jpg', img4)
but the result was equally awful too.
Doing this CLAHE method with LAB color space, as suggested in the question How to apply CLAHE on RGB color images:
import cv2, numpy as np
bgr = cv2.imread('_example.jpg')
lab = cv2.cvtColor(bgr, cv2.COLOR_BGR2LAB)
lab_planes = cv2.split(lab)
clahe = cv2.createCLAHE(clipLimit=2.0,tileGridSize=(100,100))
lab_planes[0] = clahe.apply(lab_planes[0])
lab = cv2.merge(lab_planes)
bgr = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)
cv2.imwrite('_example111.jpg', bgr)
gave bad result too. Output image:
Do an adaptive thresholding or histogram equalization separately on each channel (R, G, B) is not an option since it would mess with the color balance, as explained here.
"Contrast strechting" method from scikit-image's tutorial on Histogram Equalization:
the image is rescaled to include all intensities that fall within the 2nd and 98th percentiles
is a little bit better, but still far from the desired result (see image on top of this question).
TL;DR: how to get an automatic brightness/contrast optimization of a color photo of a sheet of paper with OpenCV/Python? What kind of thresholding/histogram equalization/other technique could be used?
Contrast and brightness can be adjusted using alpha (α) and beta (β), respectively. These variables are often called the gain and bias parameters. The expression can be written as
OpenCV already implements this as cv2.convertScaleAbs() so we can just use this function with user defined alpha and beta values.
import cv2
image = cv2.imread('1.jpg')
alpha = 1.95 # Contrast control (1.0-3.0)
beta = 0 # Brightness control (0-100)
manual_result = cv2.convertScaleAbs(image, alpha=alpha, beta=beta)
cv2.imshow('original', image)
cv2.imshow('manual_result', manual_result)
cv2.waitKey()
But the question was
How to get an automatic brightness/contrast optimization of a color photo?
Essentially the question is how to automatically calculate alpha and beta. To do this, we can look at the histogram of the image. Automatic brightness and contrast optimization calculates alpha and beta so that the output range is [0...255]. We calculate the cumulative distribution to determine where color frequency is less than some threshold value (say 1%) and cut the right and left sides of the histogram. This gives us our minimum and maximum ranges. Here's a visualization of the histogram before (blue) and after clipping (orange). Notice how the more "interesting" sections of the image are more pronounced after clipping.
To calculate alpha, we take the minimum and maximum grayscale range after clipping and divide it from our desired output range of 255
α = 255 / (maximum_gray - minimum_gray)
To calculate beta, we plug it into the formula where g(i, j)=0 and f(i, j)=minimum_gray
g(i,j) = α * f(i,j) + β
which after solving results in this
β = -minimum_gray * α
For your image we get this
Alpha: 3.75
Beta: -311.25
You may have to adjust the clipping threshold value to refine results. Here's some example results using a 1% threshold with other images: Before -> After
Automated brightness and contrast code
import cv2
import numpy as np
from matplotlib import pyplot as plt
# Automatic brightness and contrast optimization with optional histogram clipping
def automatic_brightness_and_contrast(image, clip_hist_percent=1):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Calculate grayscale histogram
hist = cv2.calcHist([gray],[0],None,[256],[0,256])
hist_size = len(hist)
# Calculate cumulative distribution from the histogram
accumulator = []
accumulator.append(float(hist[0]))
for index in range(1, hist_size):
accumulator.append(accumulator[index -1] + float(hist[index]))
# Locate points to clip
maximum = accumulator[-1]
clip_hist_percent *= (maximum/100.0)
clip_hist_percent /= 2.0
# Locate left cut
minimum_gray = 0
while accumulator[minimum_gray] < clip_hist_percent:
minimum_gray += 1
# Locate right cut
maximum_gray = hist_size -1
while accumulator[maximum_gray] >= (maximum - clip_hist_percent):
maximum_gray -= 1
# Calculate alpha and beta values
alpha = 255 / (maximum_gray - minimum_gray)
beta = -minimum_gray * alpha
'''
# Calculate new histogram with desired range and show histogram
new_hist = cv2.calcHist([gray],[0],None,[256],[minimum_gray,maximum_gray])
plt.plot(hist)
plt.plot(new_hist)
plt.xlim([0,256])
plt.show()
'''
auto_result = cv2.convertScaleAbs(image, alpha=alpha, beta=beta)
return (auto_result, alpha, beta)
image = cv2.imread('1.jpg')
auto_result, alpha, beta = automatic_brightness_and_contrast(image)
print('alpha', alpha)
print('beta', beta)
cv2.imshow('auto_result', auto_result)
cv2.waitKey()
Result image with this code:
Results with other images using a 1% threshold
An alternative version is to add gain and bias to an image using saturation arithmetic instead of using OpenCV's cv2.convertScaleAbs(). The built-in method does not take an absolute value, which would lead to nonsensical results (e.g., a pixel at 44 with alpha = 3 and beta = -210 becomes 78 with OpenCV, when in fact it should become 0).
import cv2
import numpy as np
# from matplotlib import pyplot as plt
def convertScale(img, alpha, beta):
"""Add bias and gain to an image with saturation arithmetics. Unlike
cv2.convertScaleAbs, it does not take an absolute value, which would lead to
nonsensical results (e.g., a pixel at 44 with alpha = 3 and beta = -210
becomes 78 with OpenCV, when in fact it should become 0).
"""
new_img = img * alpha + beta
new_img[new_img < 0] = 0
new_img[new_img > 255] = 255
return new_img.astype(np.uint8)
# Automatic brightness and contrast optimization with optional histogram clipping
def automatic_brightness_and_contrast(image, clip_hist_percent=25):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Calculate grayscale histogram
hist = cv2.calcHist([gray],[0],None,[256],[0,256])
hist_size = len(hist)
# Calculate cumulative distribution from the histogram
accumulator = []
accumulator.append(float(hist[0]))
for index in range(1, hist_size):
accumulator.append(accumulator[index -1] + float(hist[index]))
# Locate points to clip
maximum = accumulator[-1]
clip_hist_percent *= (maximum/100.0)
clip_hist_percent /= 2.0
# Locate left cut
minimum_gray = 0
while accumulator[minimum_gray] < clip_hist_percent:
minimum_gray += 1
# Locate right cut
maximum_gray = hist_size -1
while accumulator[maximum_gray] >= (maximum - clip_hist_percent):
maximum_gray -= 1
# Calculate alpha and beta values
alpha = 255 / (maximum_gray - minimum_gray)
beta = -minimum_gray * alpha
'''
# Calculate new histogram with desired range and show histogram
new_hist = cv2.calcHist([gray],[0],None,[256],[minimum_gray,maximum_gray])
plt.plot(hist)
plt.plot(new_hist)
plt.xlim([0,256])
plt.show()
'''
auto_result = convertScale(image, alpha=alpha, beta=beta)
return (auto_result, alpha, beta)
image = cv2.imread('1.jpg')
auto_result, alpha, beta = automatic_brightness_and_contrast(image)
print('alpha', alpha)
print('beta', beta)
cv2.imshow('auto_result', auto_result)
cv2.imwrite('auto_result.png', auto_result)
cv2.imshow('image', image)
cv2.waitKey()
Robust Locally-Adaptive Soft Binarization! That's what I call it.
I've done similar stuffs before, for a bit different purpose, so this may not perfectly fit for your needs, but hope it helps (also I wrote this code at night for personal use so it's ugly). In a sense, this code was intended to solve a more general case compared to yours, where we can have a lot of structured noise on the background (see demo below).
What this code does? Given a photo of a sheet of paper, it will whiten it so that it can be perfectly printable. See example images below.
Teaser: that's how your pages will look like after this algorithm (before and after). Notice that even the color marker annotations are gone, so I don't know if this will fit your use case but the code might be useful:
To get a perfectly clean results, you might need to toy around with filtering parameters a bit, but as you can see, even with default parameters it works quite well.
Step 0: Cut the images to fit closely to the page
Let's asume you somehow did this step (it seems like that in the examples you provided). If you need a manual annotate-and-rewarp tool, just pm me! ^^ The results of this step is below (the examples I use here are arguably harder than the one you provided, whilst it may not exactly match your case):
From this we can immediately see the following problems:
Lightening condition is not even. This means all simple binarization methods won't work. I tried a lot of solutions available in OpenCV, as well as their combinations, none of them worked!
A lot of background noise. In my case, I needed to remove the grid of the paper, and also the ink from the other side of the paper that is visible through the thin sheet.
Step 1: Gamma correction
The reasoning of this step is to balance out the contrast of the whole image (since your image can be slightly overexposed/underexposed depending to the lighting condition).
This may seem at first as an unnecessary step, but the importance of it cannot be underestimated: in a sense, it normalizes the images to the similar distributions of exposures, so that you can choose meaningful hyper-parameters later (e.g. the DELTA parameter in next section, the noise filtering parameters, parameters for morphological stuffs, etc.)
# Somehow I found the value of `gamma=1.2` to be the best in my case
def adjust_gamma(image, gamma=1.2):
# build a lookup table mapping the pixel values [0, 255] to
# their adjusted gamma values
invGamma = 1.0 / gamma
table = np.array([((i / 255.0) ** invGamma) * 255
for i in np.arange(0, 256)]).astype("uint8")
# apply gamma correction using the lookup table
return cv2.LUT(image, table)
Here are results of gamma adjusting:
You can see that it is a bit more... "balanced" now. Without this step, all parameters that you will pick by hand in later steps will become less robust!
Step 2: Adaptive Binarization to Detect the Text Blobs
In this step, we will adaptively binarize out the text blobs. I will add more comments later, but the idea basically is following:
We divide the image into blocks of size BLOCK_SIZE. The trick is to choose its size large enough so that you still get a large chunk of text and background (i.e. larger than any symbols that you have), but small enough to not suffer from any lightening condition variations (i.e. "large, but still local").
Inside each block, we do locally-adaptive binarization: we look at the median value and hypothesize that it is the background (because we chose the BLOCK_SIZE large enough to have the majority of it to be background). Then, we further define DELTA — basically just a threshold of "how far away from median we will still consider it as background?".
So, the function process_image gets the job done. Moreover, you can modify the preprocess and postprocess functions to fit your need (however, as you can see from the example above, the algorithm is pretty robust, i.e. it works quite well out-of-the-box without modifying too much the parameters).
The code of this part assumes the foreground to be darker than the background (i.e. ink on paper). But you can easily change that by tweaking the preprocess function: instead of 255 - image, return just image.
# These are probably the only important parameters in the
# whole pipeline (steps 0 through 3).
BLOCK_SIZE = 40
DELTA = 25
# Do the necessary noise cleaning and other stuffs.
# I just do a simple blurring here but you can optionally
# add more stuffs.
def preprocess(image):
image = cv2.medianBlur(image, 3)
return 255 - image
# Again, this step is fully optional and you can even keep
# the body empty. I just did some opening. The algorithm is
# pretty robust, so this stuff won't affect much.
def postprocess(image):
kernel = np.ones((3,3), np.uint8)
image = cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel)
return image
# Just a helper function that generates box coordinates
def get_block_index(image_shape, yx, block_size):
y = np.arange(max(0, yx[0]-block_size), min(image_shape[0], yx[0]+block_size))
x = np.arange(max(0, yx[1]-block_size), min(image_shape[1], yx[1]+block_size))
return np.meshgrid(y, x)
# Here is where the trick begins. We perform binarization from the
# median value locally (the img_in is actually a slice of the image).
# Here, following assumptions are held:
# 1. The majority of pixels in the slice is background
# 2. The median value of the intensity histogram probably
# belongs to the background. We allow a soft margin DELTA
# to account for any irregularities.
# 3. We need to keep everything other than the background.
#
# We also do simple morphological operations here. It was just
# something that I empirically found to be "useful", but I assume
# this is pretty robust across different datasets.
def adaptive_median_threshold(img_in):
med = np.median(img_in)
img_out = np.zeros_like(img_in)
img_out[img_in - med < DELTA] = 255
kernel = np.ones((3,3),np.uint8)
img_out = 255 - cv2.dilate(255 - img_out,kernel,iterations = 2)
return img_out
# This function just divides the image into local regions (blocks),
# and perform the `adaptive_mean_threshold(...)` function to each
# of the regions.
def block_image_process(image, block_size):
out_image = np.zeros_like(image)
for row in range(0, image.shape[0], block_size):
for col in range(0, image.shape[1], block_size):
idx = (row, col)
block_idx = get_block_index(image.shape, idx, block_size)
out_image[block_idx] = adaptive_median_threshold(image[block_idx])
return out_image
# This function invokes the whole pipeline of Step 2.
def process_image(img):
image_in = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
image_in = preprocess(image_in)
image_out = block_image_process(image_in, BLOCK_SIZE)
image_out = postprocess(image_out)
return image_out
The results are nice blobs like this, closely following the ink trace:
Step 3: The "Soft" Part of Binarization
Having the blobs that covers the symbols and a little bit more, we can finally do the whitening procedure.
If we look more closely at the photos of sheets of papers with text (especially those that have hand writings), the transformation from "background" (white paper) to "foreground" (the dark color ink) is not sharp, but very gradual. Other binarization-based answers in this section proposes a simple thresholding (even if they are locally-adaptive, it is still a threshold), which works okay for printed text, but will produce not-so-pretty results with hand writings.
So, the motivation of this section is that we want to preserve that effect of gradual transmission from black to white, just as natural photos of sheets of papers with natural ink. The final purpose for that is to make it printable.
The main idea is simple: the more the pixel value (after thresholding above) differs from the local min value, the more likely it is belonging to the background. We can express this using a family of Sigmoid functions, re-scaled to the range of local block (so that this function is adaptively scaled thorough the image).
# This is the function used for composing
def sigmoid(x, orig, rad):
k = np.exp((x - orig) * 5 / rad)
return k / (k + 1.)
# Here, we combine the local blocks. A bit lengthy, so please
# follow the local comments.
def combine_block(img_in, mask):
# First, we pre-fill the masked region of img_out to white
# (i.e. background). The mask is retrieved from previous section.
img_out = np.zeros_like(img_in)
img_out[mask == 255] = 255
fimg_in = img_in.astype(np.float32)
# Then, we store the foreground (letters written with ink)
# in the `idx` array. If there are none (i.e. just background),
# we move on to the next block.
idx = np.where(mask == 0)
if idx[0].shape[0] == 0:
img_out[idx] = img_in[idx]
return img_out
# We find the intensity range of our pixels in this local part
# and clip the image block to that range, locally.
lo = fimg_in[idx].min()
hi = fimg_in[idx].max()
v = fimg_in[idx] - lo
r = hi - lo
# Now we use good old OTSU binarization to get a rough estimation
# of foreground and background regions.
img_in_idx = img_in[idx]
ret3,th3 = cv2.threshold(img_in[idx],0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Then we normalize the stuffs and apply sigmoid to gradually
# combine the stuffs.
bound_value = np.min(img_in_idx[th3[:, 0] == 255])
bound_value = (bound_value - lo) / (r + 1e-5)
f = (v / (r + 1e-5))
f = sigmoid(f, bound_value + 0.05, 0.2)
# Finally, we re-normalize the result to the range [0..255]
img_out[idx] = (255. * f).astype(np.uint8)
return img_out
# We do the combination routine on local blocks, so that the scaling
# parameters of Sigmoid function can be adjusted to local setting
def combine_block_image_process(image, mask, block_size):
out_image = np.zeros_like(image)
for row in range(0, image.shape[0], block_size):
for col in range(0, image.shape[1], block_size):
idx = (row, col)
block_idx = get_block_index(image.shape, idx, block_size)
out_image[block_idx] = combine_block(
image[block_idx], mask[block_idx])
return out_image
# Postprocessing (should be robust even without it, but I recommend
# you to play around a bit and find what works best for your data.
# I just left it blank.
def combine_postprocess(image):
return image
# The main function of this section. Executes the whole pipeline.
def combine_process(img, mask):
image_in = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
image_out = combine_block_image_process(image_in, mask, 20)
image_out = combine_postprocess(image_out)
return image_out
Some stuffs are commented since they are optional. The combine_process function takes the mask from the previous step, and executes the whole composition pipeline. You can try to toy with them for your specific data (images). The results are neat:
Probably I will add more comments and explanations to the code in this answer. Will upload the whole thing (together with cropping and warping code) on Github.
This method should work well for your application. First you find a threshold value that separates the distribution modes well in the intensity histogram then rescale the intensity using that value.
from skimage.filters import threshold_yen
from skimage.exposure import rescale_intensity
from skimage.io import imread, imsave
img = imread('mY7ep.jpg')
yen_threshold = threshold_yen(img)
bright = rescale_intensity(img, (0, yen_threshold), (0, 255))
imsave('out.jpg', bright)
I'm here using Yen's method, can learn more about this method on this page.
I think the way to do that is 1) Extract the chroma (saturation) channel from HCL colorspace. (HCL works better than HSL or HSV). Only colors should have non-zero saturation, so bright, and gray shades will be dark. 2) Threshold that result using otsu thresholding to use as a mask. 3) Convert your input to grayscale and apply local area (i.e., adaptive) thresholding. 4) put the mask into the alpha channel of the original and then composite the local area thresholded result with the original, so that it keeps the colored area from the original and everywhere else uses the local area thresholded result.
Sorry, I do not know OpeCV that well, but here are the steps using ImageMagick.
Note that channels are numbered starting with 0. (H=0 or red, C=1 or green, L=2 or blue)
Input:
magick image.jpg -colorspace HCL -channel 1 -separate +channel tmp1.png
magick tmp1.png -auto-threshold otsu tmp2.png
magick image.jpg -colorspace gray -negate -lat 20x20+10% -negate tmp3.png
magick tmp3.png \( image.jpg tmp2.png -alpha off -compose copy_opacity -composite \) -compose over -composite result.png
ADDITION:
Here is Python Wand code, which produces the same output result. It needs Imagemagick 7 and Wand 0.5.5.
#!/bin/python3.7
from wand.image import Image
from wand.display import display
from wand.version import QUANTUM_RANGE
with Image(filename='text.jpg') as img:
with img.clone() as copied:
with img.clone() as hcl:
hcl.transform_colorspace('hcl')
with hcl.channel_images['green'] as mask:
mask.auto_threshold(method='otsu')
copied.composite(mask, left=0, top=0, operator='copy_alpha')
img.transform_colorspace('gray')
img.negate()
img.adaptive_threshold(width=20, height=20, offset=0.1*QUANTUM_RANGE)
img.negate()
img.composite(copied, left=0, top=0, operator='over')
img.save(filename='text_process.jpg')
First we separate text and color markings. This can be done in a color space with a color saturation channel. I used instead a very simple method inspired by this paper: the ration of min(R,G,B)/ max(R,G,B) will be near 1 for (light) gray areas and << 1 for colored areas. For dark gray areas we get anything between 0 and 1, but this doesn't matter: either these areas go to the color mask and are then added as is or they are not included in the mask and are contributed to the output from the binarized text. For black we use the fact that 0/0 becomes 0 when converted to uint8.
The grayscale image text gets locally thresholded to produce a black and white image. You can pick your favorite technique from this comparison or that survey. I chose the NICK technique that copes well with low contrast and is rather robust, i.e. the choice of the parameter k between about -0.3 and -0.1 works well for a very wide range of conditions which is good for automatic processing. For the sample document provided the chosen technique doesn't play a big role as it is relatively uniformly illuminated, but in order to cope with non-uniformly illuminated images it should be a local thresholding technique.
In the final step, the color areas are added back to the binarized text image.
So this solution is very similar to #fmw42's solution (all credit for the idea to him) with the exception of the different color detection and binarization methods.
image = cv2.imread('mY7ep.jpg')
# make mask and inverted mask for colored areas
b,g,r = cv2.split(cv2.blur(image,(5,5)))
np.seterr(divide='ignore', invalid='ignore') # 0/0 --> 0
m = (np.fmin(np.fmin(b, g), r) / np.fmax(np.fmax(b, g), r)) * 255
_,mask_inv = cv2.threshold(np.uint8(m), 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
mask = cv2.bitwise_not(mask_inv)
# local thresholding of grayscale image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
text = cv2.ximgproc.niBlackThreshold(gray, 255, cv2.THRESH_BINARY, 41, -0.1, binarizationMethod=cv2.ximgproc.BINARIZATION_NICK)
# create background (text) and foreground (color markings)
bg = cv2.bitwise_and(text, text, mask = mask_inv)
fg = cv2.bitwise_and(image, image, mask = mask)
out = cv2.add(cv2.cvtColor(bg, cv2.COLOR_GRAY2BGR), fg)
If you don't need the color markings, you can simply binarize the grayscale image:
image = cv2.imread('mY7ep.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
text = cv2.ximgproc.niBlackThreshold(gray, 255, cv2.THRESH_BINARY, at_bs, -0.3, binarizationMethod=cv2.ximgproc.BINARIZATION_NICK)
This is a C# transpilation(performed via https://github.com/uxmal/pytocs) for nathancy's answer for Emgu.CV wrapper library:
/// <summary>
/// <see>https://stackoverflow.com/questions/56905592/automatic-contrast-and-brightness-adjustment-of-a-color-photo-of-a-sheet-of-pape/75455163#75455163</see>
/// </summary>
public static (Mat autoResult, int alpha, int beta) AutomaticBrightnessAndContrast(Mat image, double clipHistPercent = 1)
{
var gray = new Mat();
CvInvoke.CvtColor(image, gray, ColorConversion.Bgr2Gray);
// Calculate grayscale histogram
var hist = new Mat();
var grayVector = new VectorOfMat(gray);
CvInvoke.CalcHist(grayVector, new[] {0}, null, hist, new[] {256}, new[] {0f, 256}, false);
var histSize = hist.Rows;
// Calculate cumulative distribution from the histogram
var accumulator = new List<float> {hist.Get<float>(0, 0)};
foreach (var index in Enumerable.Range(1, histSize - 1))
accumulator.Add(accumulator[index - 1] + hist.Get<float>(index, 0));
// Locate points to clip
var maximum = accumulator[255];
clipHistPercent *= maximum / 100.0;
clipHistPercent /= 2.0;
// Locate left cut
var minimumGray = 0;
while (accumulator[minimumGray] < clipHistPercent)
minimumGray += 1;
// Locate right cut
var maximumGray = histSize - 1;
while (accumulator[maximumGray] >= maximum - clipHistPercent)
maximumGray -= 1;
// Calculate alpha and beta values
var alpha = 255 / (maximumGray - minimumGray);
var beta = -minimumGray * alpha;
var autoResult = new Mat();
CvInvoke.ConvertScaleAbs(image, autoResult, alpha, beta);
return (autoResult, alpha, beta);
}
public static class MatExtension
{
/// <summary>
/// <see>https://stackoverflow.com/questions/32255440/how-can-i-get-and-set-pixel-values-of-an-emgucv-mat-image/69537504#69537504</see>
/// </summary>
public static unsafe T Get<T>(this Mat mat, int row, int col) =>
new ReadOnlySpan<T>(mat.DataPointer.ToPointer(), mat.Rows * mat.Cols * mat.ElementSize)
[(row * mat.Cols) + col];
}
If you are using OpenCvSharp, just modify all invokes to OpenCV with updated parameters like Rotate an image without cropping in OpenCV in C++
Also note that OpenCvSharp already has Mat.Set<> method that functions same as mat.at<> in the original OpenCV, so we don't have to copy these methods from How can I get and set pixel values of an EmguCV Mat image?
My code currently consists of loading the image, which is successful and I don't believe has any connection to the problem.
Then I go on to transform the color image into a np.array named rgb
# convert image into array
rgb = np.array(img)
red = rgb[:,:,0]
green = rgb[:,:,1]
blue = rgb[:,:,2]
To double check my understanding of this array, in case that may be the root of the issue, it is an array such that rgb[x-coordinate, y-coordinate, color band] which holds the value between 0-255 of either red, green or blue.
Then, my idea was to make a nested for loop to traverse all pixels of my image (620px,400px) and sort them based on the ratio of green to blue and red in an attempt to single out the greener pixels and set all others to black or 0.
for i in range(xsize):
for j in range(ysize):
color = rgb[i,j] <-- Index error occurs here
if(color[0] > 128):
if(color[1] < 128):
if(color[2] > 128):
rgb[i,j] = [0,0,0]
The error I am receiving when trying to run this is as follows:
IndexError: index 400 is out of bounds for axis 0 with size 400
I thought it may have something to do with the bounds I was giving i and j so I tried only sorting through a small inner portion of the image but still got the same error. At this point I am lost as to what is even the root of the error let alone even the solution.
In direct answer to your question, the y axis is given first in numpy arrays, followed by the x axis, so interchange your indices.
Less directly, you will find that for loops are very slow in Python and you are generally better off using numpy vectorised operations instead. Also, you will often find it easier to find shades of green in HSV colourspace.
Let's start with an HSL colour wheel:
and assume you want to make all the greens into black. So, from that Wikipedia page, the Hue corresponding to Green is 120 degrees, which means you could do this:
#!/usr/local/bin/python3
import numpy as np
from PIL import Image
# Open image and make RGB and HSV versions
RGBim = Image.open("image.png").convert('RGB')
HSVim = RGBim.convert('HSV')
# Make numpy versions
RGBna = np.array(RGBim)
HSVna = np.array(HSVim)
# Extract Hue
H = HSVna[:,:,0]
# Find all green pixels, i.e. where 100 < Hue < 140
lo,hi = 100,140
# Rescale to 0-255, rather than 0-360 because we are using uint8
lo = int((lo * 255) / 360)
hi = int((hi * 255) / 360)
green = np.where((H>lo) & (H<hi))
# Make all green pixels black in original image
RGBna[green] = [0,0,0]
count = green[0].size
print("Pixels matched: {}".format(count))
Image.fromarray(RGBna).save('result.png')
Which gives:
Here is a slightly improved version that retains the alpha/transparency, and matches red pixels for extra fun:
#!/usr/local/bin/python3
import numpy as np
from PIL import Image
# Open image and make RGB and HSV versions
im = Image.open("image.png")
# Save Alpha if present, then remove
if 'A' in im.getbands():
savedAlpha = im.getchannel('A')
im = im.convert('RGB')
# Make HSV version
HSVim = im.convert('HSV')
# Make numpy versions
RGBna = np.array(im)
HSVna = np.array(HSVim)
# Extract Hue
H = HSVna[:,:,0]
# Find all red pixels, i.e. where 340 < Hue < 20
lo,hi = 340,20
# Rescale to 0-255, rather than 0-360 because we are using uint8
lo = int((lo * 255) / 360)
hi = int((hi * 255) / 360)
red = np.where((H>lo) | (H<hi))
# Make all red pixels black in original image
RGBna[red] = [0,0,0]
count = red[0].size
print("Pixels matched: {}".format(count))
result=Image.fromarray(RGBna)
# Replace Alpha if originally present
if savedAlpha is not None:
result.putalpha(savedAlpha)
result.save('result.png')
Keywords: Image processing, PIL, Pillow, Hue Saturation Value, HSV, HSL, color ranges, colour ranges, range, prime.
To extract the color, we have this function
# define range of blue color in HSV
lower_blue = np.array([110,50,50])
upper_blue = np.array([130,255,255])
# Threshold the HSV image to get only blue colors
mask = cv2.inRange(hsv, lower_blue, upper_blue)
How do we actually visualize the range(lower_blue,upper_blue) I define on hsv space?
Also How do I actually plot a hsv color,but it is not working...?
I have this code:
upper = np.array([60, 255, 255])
upper = cv2.cvtColor(upper, cv2.COLOR_HSV2BGR)
upper = totuple(upper/-255)
print(upper)
plt.imshow([[upper]])
What are HSV colors
HSV, like HSL (or in OpenCV, HLS), is one of the cylindrical colorspaces.
The name is somewhat descriptive of how their values are referenced.
The hue is represented as degrees from 0 to 360 (in OpenCV, to fit into the an 8-bit unsigned integer format, they degrees are divided by two to get a number from 0 to 179; so 110 in OpenCV is 220 degrees). If you were to take a "range" of hue values, it's like cutting a slice from a cake. You're just taking some angle chunk of the cake.
The saturation channel is how far from the center you are---the radius you're at. The center has absolutely no saturation---only gray colors from black to white. If you took a range of these values, it is akin to shaving off the outside of the cylinder, or cutting out a circle from the center. For example, if the range is 0 to 255, then the range 0 to 127 would be a cylinder only extending to half the radius; the range 127 to 255 would be cutting an inner cylinder with half the radius out.
The value channel is a slightly confusing name; it's not exactly darkness-to-brightness because the highest value represents the direct color, while the lowest value is black. This is the height of the cylinder. Not too hard to imagine cutting a slice of the cylinder vertically.
Ranges of HSV values
The function cv2.inRange(image, lower_bound, upper_bound) finds all values of the image between lower_bound and upper_bound. For instance, if your image was a 3x3 image (just for simple demonstration purposes) with 3-channels, it might look something like this:
# h channel # s channel # v channel
100 150 250 150 150 100 50 75 225
50 100 125 75 25 50 255 100 50
0 255 125 100 200 250 50 75 100
If we wanted to select hues between 100 and 200, then our lower_b should be [100, 0, 0] and upper_b should be [200, 255, 255]. That way our mask would only take into account values in the hue channel, and not be affected by the saturation and value. That's why HSV is so popular---you can select colors by hue regardless of their brightness or darkness, so a dark red and bright red can be selected just by specifying the min and max of the hue channel.
But say we only wanted to select bright white colors. Take a look back at the cylinder model---we see that white is given at the top-center of the cylinder, so where s values are low, and v values are high, and the color angle doesn't matter. So the lower_b would look something like [0, 0, 200] and upper_b would look something like [255, 50, 255]. That means all H values will be included and won't affect our mask. But then only S values between 0 and 50 would be included (towards the center of the cylinder) and only V values from 200 to 255 will be included (towards the top of the cylinder).
Visualizing a range of colors from HSV
One way to visualize all the colors in a range is to create gradients going the length of both directions for each of two channels, and then animate over the changing third channel.
For instance, you could create a gradient of values from left to right for the range of S values, from top to bottom for the range of V values, and then loop over each H value. This whole program could look something like this:
import numpy as np
import cv2
lower_b = np.array([110,50,50])
upper_b = np.array([130,255,255])
s_gradient = np.ones((500,1), dtype=np.uint8)*np.linspace(lower_b[1], upper_b[1], 500, dtype=np.uint8)
v_gradient = np.rot90(np.ones((500,1), dtype=np.uint8)*np.linspace(lower_b[1], upper_b[1], 500, dtype=np.uint8))
h_array = np.arange(lower_b[0], upper_b[0]+1)
for hue in h_array:
h = hue*np.ones((500,500), dtype=np.uint8)
hsv_color = cv2.merge((h, s_gradient, v_gradient))
rgb_color = cv2.cvtColor(hsv_color, cv2.COLOR_HSV2BGR)
cv2.imshow('', rgb_color)
cv2.waitKey(250)
cv2.destroyAllWindows()
Now this gif shows a new H value every frame. And from left to right we have the min to max S values, and from top to bottom we have the min to max V values. Every single one of the colors showing up in this animation will be selected from your image to be part of your mask.
Make your own inRange() function
To fully understand the OpenCV function, the easiest way is just to make your own function to complete the task. It's not difficult at all, and not very much code.
The idea behind the function is simple: find where the values of each channel fall between min and max, and then & all the channels together.
def inRange(img, lower_b, upper_b):
ch1, ch2, ch3 = cv2.split(img)
ch1m = (lower_b[0] <= ch1) & (ch1 <= upper_b[0])
ch2m = (lower_b[1] <= ch2) & (ch2 <= upper_b[1])
ch3m = (lower_b[2] <= ch3) & (ch3 <= upper_b[2])
mask = ch1m & ch2m & ch3m
return mask.astype(np.uint8)*255
You can read the OpenCV docs to see that this is indeed the formula used. And we can verify it too.
lower_b = np.array([200,200,200])
upper_b = np.array([255,255,255])
mask = cv2.inRange(img, lower_b, upper_b) # OpenCV function
mask2 = inRange(img, lower_b, upper_b) # above defined function
print((mask==mask2).all()) # checks that the masks agree on all values
# True
How to find the right colors
It can be a little tricky to find the correct values to use for a particular image. There is an easy way to experiment, though. You can create trackbars in OpenCV and use them to control the min and max for each channel and have the Python program update your mask every time you change the values. I made a program for this which you can grab on GitHub here. Here's an animated .gif of it being used, to demonstrate: