What i'm trying to do:
Take an image
Apply a gaussian filter to the image
Equalise the image based on OpenCv equaliseHist function
From the equalised image take the thresholded image:
if in the thresholded image pixel is equal to black then in the actual image decrease the brightness of that pixel, if its white in the threshold image increase the brightness in the actual image
Below is currently what code i have so far:
img = cv2.imread('1.bmp')
img = cv2.GaussianBlur(img,(5,5),0)
""" Take the image and slipt it into the HSV colour spectrem """
h,s,v= cv2.split(cv2.cvtColor(img, cv2.COLOR_BGR2HSV))
""" Equalise the histogram for the V value """
eq_V = cv2.equalizeHist(v)
""" Merge all the values back into one image """
eq_image = cv2.cvtColor(cv2.merge([h, s, eq_V]), cv2.COLOR_HSV2BGR)
ret1,th1 = cv2.threshold(eq_image,127,255,cv2.THRESH_BINARY)
[rows, columns, channels] = img.shape
blacks = np.zeros((rows,columns,channels))
whites = np.zeros((rows,columns,channels))
for row in range(rows):
for column in range(columns):
if th1[row,column].all() == 0:
""" Equalise the v value """
blacks[row,column] = v[row,column]
else:
""" Equalise the v value """
whites[row,column] = v[row,column]
addTogether = cv2.add(blacks,whites)
cv2.imshow("frame", addTogether.astype(np.uint8))
cv2.waitKey(0)
cv2.destroyAllWindows()
My problem
The part that i am struggling with is step 4. mentioned above. I am currently building an image based on whats black and whats white and then adding those images together to get back to my orginal image state.
What i seem to be unable to do/figure out is how to adjust the v value of the hsv for the image and then how to merge the whole image back together with the new v values alsongside the hue and saturation values.
Image from a latest attempt
Related
I am writing some code to display a camera input to a 32*32 LED array.
My code to get the image and display it looks like this:
def start_cam(x,y):
# Start the webcam
webcam = cv2.VideoCapture(0)
# Set frame rate to 45 frames per second
frame_rate = 45
# Loop 45 times per second
while True:
# Capture a frame from the webcam
ret, frame = webcam.read()
# Resize the frame to 16x16
frame = cv2.resize(frame, (x, y))
frame = sp_noise(frame,0.85)
# Get input orientation
orientation = 0
# Rotate the frame by 90 degrees based on user input
if orientation == 1:
frame = np.rot90(frame)
elif orientation == 2:
frame = np.rot90(frame, 2)
elif orientation == 3:
frame = np.rot90(frame, 3)
# Initialize empty list to store RGB values
rgb_list = []
# Loop through each pixel in the frame
for i in range(x):
for j in range(y):
# Get RGB values of each pixel
r, g, b = frame[i, j]
# Append RGB values to list
rgb_list = rgb_list + [b, g, r]
# Print the list of RGB values
#print(rgb_list)
rgb_out = []
for i in rgb_list:
rgb_out.append(gamma[i]//2)
rgb_out = sp_noise(rgb_out,0.2)
temp_send(rgb_out, x,y)
I have a function already made called sp.noise that adds salt and pepper static to the image based on a value between 0-1. I would like to make a second image processing function that would have the image go from being fully colored at a value of 0, to fully gray at a value of 1.
How could I go about making a smooth gray-scale function for my RGB value NP array?
I wrote a function that simply computes both gray and color, and averages them weighing them based on the input value, but that is incredibly inefficient. And reduces my FPS to unusable levels.
To "make an image fully gray" is to desaturate an image; so removing the colors while retaining the hue and brightness of the pixels. You can:
First, convert your RGB image into HSL space. This will convert your (red, green, blue) pixel triplets into (hue, saturation, lightness) triplets, where "how much color a pixel has" is contained within the single value saturation.
For OpenCV you can use something like output = cv2.cvtColor(img, cv2.COLOR_RGB2HSV) or with cv2.COLOR_BGR2HSV depending on your input color
Simple example from GeeksforGeeks
OpenCV example
Then you can write a simple desaturation function. For example, a function to multiply the saturation of each pixel by your value of range [0,1]. This will make the image "fully gray" at 0, and "fully color" at 1.
(Optional) You can then convert the image back to RGB if necessary with the same function, but different flag: output = cv2.cvtColor(img, cv2.COLOR_HSV2RGB)
The aim is to take a coloured image, and change any pixels within a certain luminosity range to black. For example, if luminosity is the average of a pixel's RGB values, any pixel with a value under 50 is changed to black.
I’ve attempted to begin using PIL and converting to grayscale, but having trouble trying to find a solution that can identify luminosity value and use that info to manipulate a pixel map.
There are many ways to do this, but the simplest and probably fastest is with Numpy, which you should get accustomed to using with image processing in Python:
from PIL import Image
import numpy as np
# Load image and ensure RGB, not palette image
im = Image.open('start.png').convert('RGB')
# Make into Numpy array
na = np.array(im)
# Make all pixels of "na" where the mean of the R,G,B channels is less than 50 into black (0)
na[np.mean(na, axis=-1)<50] = 0
# Convert back to PIL Image to save or display
result = Image.fromarray(na)
result.show()
That turns this:
Into this:
Another slightly different way would be to convert the image to a more conventional greyscale, rather than averaging for the luminosity:
# Load image and ensure RGB
im = Image.open('start.png').convert('RGB')
# Calculate greyscale version
grey = im.convert('L')
# Point process over pixels to make mask of darker ones
mask = grey.point(lambda p: 255 if p<50 else 0)
# Paste black (i.e. 0) into image where mask indicates it is dark
im.paste(0, mask=mask)
Notice that the blue channel is given considerably less significance in the ITU-R 601-2 luma transform that PIL uses (see the lower 114 weighting for Blue versus 299 for Red and 587 for Green) in the formula:
L = R * 299/1000 + G * 587/1000 + B * 114/1000
so the blue shades are considered darker and become black.
Another way would be to make a greyscale and a mask as above. but then choose the darker pixel at each location when comparing the original and the mask:
from PIL import Image, ImageChops
im = Image.open('start.png').convert('RGB')
grey = im.convert('L')
mask = grey.point(lambda p: 0 if p<50 else 255)
res = ImageChops.darker(im, mask.convert('RGB'))
That gives the same result as above.
Another way, pure PIL and probably closest to what you actually asked, would be to derive a luminosity value by averaging the channels:
# Load image and ensure RGB
im = Image.open('start.png').convert('RGB')
# Calculate greyscale version by averaging R,G and B
grey = im.convert('L', matrix=(0.333, 0.333, 0.333, 0))
# Point process over pixels to make mask of darker ones
mask = grey.point(lambda p: 255 if p<50 else 0)
# Paste black (i.e. 0) into image where mask indicates it is dark
im.paste(0, mask=mask)
Another approach could be to split the image into its constituent RGB channels, evaluate a mathematical function over the channels and mask with the result:
from PIL import Image, ImageMath
# Load image and ensure RGB
im = Image.open('start.png').convert('RGB')
# Split into RGB channels
(R, G, B) = im.split()
# Evaluate mathematical function over channels
dark = ImageMath.eval('(((R+G+B)/3) <= 50) * 255', R=R, G=G, B=B)
# Paste black (i.e. 0) into image where mask indicates it is dark
im.paste(0, mask=dark)
I created a function that returns a list with True if the pixel has a luminosity of less than a parameter, and False if it doesn't. It includes an RGB or RGBA option (True or False)
def get_avg_lum(pic,avg=50,RGBA=False):
num=3
numd=4
if RGBA==False:
num=2
numd=3
li=[[[0]for y in range(0,pic.size[1])] for x in range(0,pic.size[0])]
for x in range(0,pic.size[0]):
for y in range(0,pic.size[1]):
if sum(pic.getpixel((x,y))[:num])/numd<avg:
li[x][y]=True
else:
li[x][y]=False
return(li)
a=get_avg_lum(im)
The pixels match in the list, so (0,10) on the image is [0][10] in the list.
Hopefully this helps. My module is for standard PIL objects.
I have a screenshot received from an iPhone, both dark and light mode.
I need to use OCR to extract the URL but am unable to do so with the underlining that appears.
What would be the best way to remove the horizontal lines from the message? Except the phone number, it doesn't matter if other parts of the screenshot are distorted.
I've tried approaches as described in
Removing Horizontal Lines in image (OpenCV, Python, Matplotlib)
https://docs.opencv.org/3.2.0/d1/dee/tutorial_moprh_lines_detection.html
https://legacy.imagemagick.org/discourse-server/viewtopic.php?t=22338
And none seem to work well, at all.
Here's a possible solution for your problem. I'm using mock screenshots, since, like I suggested, it is better to use lossless images to get a better result. The main idea here is to extract the color of the text box and to fill the rest of the image with that color, then threshold the image. By doing this, we will reduce the intensity variation and obtain a better thresholded image - since the image histogram will contain fewer intensity values. These are the steps:
Crop the image to a ROI (Region Of Interest)
Get the colors in that ROI via K-Means
Get the color of the text box
Flood-fill the ROI with the color of the text box
Apply Otsu's thresholding to get a binary image
Get OCR of the image
Suppose this is our test images, one uses a a "light" theme while the other uses a "dark" theme:
I'll be using pyocr as OCR engine. Let's use image one, the code would be this:
# imports:
from PIL import Image
import numpy as np
import cv2
import pyocr
import pyocr.builders
tools = pyocr.get_available_tools()
# The tools are returned in the recommended order of usage
tool = tools[0]
langs = tool.get_available_languages()
lang = langs[0]
# image path
path = "D://opencvImages//"
fileName = "mockText.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Convert RGB to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Set the ROI location:
roiX = 0
roiY = 235
roiWidth = 750
roiHeight = 1080
# Crop the ROI:
smsROI = grayscaleImage[roiY:roiHeight, roiX:roiWidth]
The first bit crops the ROI - everything that is of interest, leaving out the "header" and the "footer" of the image, where's there's info that we really don't need. This is the current ROI:
Wouldn't be nice to (approximately) get all the colors used in the image? Fortunately that's what Color Quantization gives us - a reduced pallet of the average colors present in an image, provided the number of the colors we are looking for. Let's apply K-Means and use 3 clusters to group this colors.
In our test images, most of the pixels are background - so, the largest cluster of pixels will belong to the background. The text represents the smallest cluster of pixels. That leaves the remaining cluster our target - the color of the text box. Let's apply K-Means, then. We need to format the data before, though, because K-Means needs float re-arranged arrays:
# Reshape the data to width x height, number of channels:
kmeansData = smsROI.reshape((-1,1))
# convert the data to np.float32
kmeansData = np.float32(kmeansData)
# define criteria, number of clusters(K) and apply kmeans():
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 5, 1.0)
# Define number of clusters (3 colors):
K = 3
# Run K-means:
_, _, center = cv2.kmeans(kmeansData, K, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)
# Convert the centers to uint8:
center = np.uint8(center)
# Sort centers from small to largest:
center = sorted(center, reverse=False)
# Get text color and min color:
textBoxColor = int(center[1][0])
minColor = min(center)[0]
print("Minimum Color is: "+str(minColor))
print("Text Box Color is: "+str(textBoxColor))
The info of interest is in center. That's where our colors are. After sorting this list and getting the minimum color value (that I'll use later to distinguish between a light and a dark theme) we can print the values. For the first test image, these values are:
Minimum Color is: 23
Text Box Color is: 225
Alright, so far so good. We have the color of the text box. Let's use that and flood-fill the entire ROI at position (x=0, y=0):
# Apply flood-fill at seed point (0,0):
cv2.floodFill(smsROI, mask=None, seedPoint=(0, 0), newVal=textBoxColor)
The result is this:
Very nice. Let's apply Otsu's thresholding on this bad boy:
# Threshold via Otsu:
_, binaryImage = cv2.threshold(smsROI, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
Now, here comes the minColor part. If you are processing a dark theme screenshot and threshold it you will get white text on black background. If you were to process a light theme screenshot you would get black text on white background. We will always produce the same no matter the input: white text and black background. Let's check the min color, if this equals 0 (black) you just received a dark theme screenshot and you don't need to invert the image. Otherwise, invert the image:
# Process "Dark Theme / Light Theme":
if minColor != 0:
# Invert image if is not already inverted:
binaryImage = 255 - binaryImage
cv2.imshow("binaryImage", binaryImage)
cv2.waitKey(0)
For our first test image, the result is:
Notice the little bits of small noise. Let's apply an area filter (function defined at the end of the post) to get rid of pixels below a certain area threshold:
# Run a minimum area filter:
minArea = 10
binaryImage = areaFilter(minArea, binaryImage)
This is the filtered image:
Very nice. Lastly, I write this image and use pyocr to get the text as a string:
cv2.imwrite(path + "ocrText.png", binaryImage)
txt = tool.image_to_string(
Image.open(path + "ocrText.png"),
lang=lang,
builder=pyocr.builders.TextBuilder()
)
print("Image text is: "+txt)
Which results in:
Image text is: 301248 is your Amazon
verification code
If you test the second image you get the same exact result. This is the definition and implementation of the areaFilter function:
def areaFilter(minArea, inputImage):
# Perform an area filter on the binary blobs:
componentsNumber, labeledImage, componentStats, componentCentroids = \
cv2.connectedComponentsWithStats(inputImage, connectivity=4)
# Get the indices/labels of the remaining components based on the area stat
# (skip the background component at index 0)
remainingComponentLabels = [i for i in range(1, componentsNumber) if componentStats[i][4] >= minArea]
# Filter the labeled pixels based on the remaining labels,
# assign pixel intensity to 255 (uint8) for the remaining pixels
filteredImage = np.where(np.isin(labeledImage, remainingComponentLabels) == True, 255, 0).astype('uint8')
return filteredImage
I'm using Template Matching to detect for smaller images in a large image.
After detecting it , i would grab the center point(x y) of main picture of the detected image.
Could anyone advice how I could grab the shade/color of that particular center point?
I understand the Template Matching ignores color , based this example, is there anyway to grab the color intensity of the particular pixel? of that center point
# Python program to illustrate
# template matching
import cv2
import numpy as np
import time
import sys
# Read the main image
img_rgb = cv2.imread('test.png')
# Convert it to grayscale
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
# Read the template
template = cv2.imread('template.png',0)
# Store width and heigth of template in w and h
w, h = template.shape[::-1]
# Perform match operations.
res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
# Specify a threshold
threshold = 0.90
# Store the coordinates of matched area in a numpy array
loc = np.where( res >= threshold)
xyMiddle = ""
for pt in zip(*loc[::-1]):
xyMiddle = str(pt[0] + w/2) +"," +str(pt[1] + h/5)
if(xyMiddle != ""):
print(xyMiddle)
The grayscale image has just a single channel and the colour image has 3 or 4 channels (BGR or BGRA).
Once you have the pixel coordinates, the pixel value in the grayscale image will be an intensity value or you can get the BGR values from that pixel in the original image. That is, img_gray[y][x] will return an intensity value in the range 0-255, and img_rgb[y][x] will return a list of [B, G, R (, A)] values, each of which will have intensity values in the range 0-255.
Thus the value returned when you call e.g. img_gray[10][50] or print(img_gray[10][50]) is the pixel value at x=50, y=10. Similarly the value returned when you call e.g. img_rgb[10][50] is the pixel value at x=50, y=10, but calling it in this way will return the list of pixel values for that location e.g. [93 238 27] for RGB or [93 238 27 255] for RGBA. To get just the B, G or R value you would call img_rgb[10][50][chan] where for chan, B=0, G=1, R=2.
I am analysing images from accelerated ageing tests. I would like to display this cumulative sum of electrical discharges that happened on the surface on a diagram of what that setup of electrodes and total sample was like for reference.
Total size of the picture is equal to the size of the sample. It is a rectangle 5cm by 120cm. This is the cumulative image.
In the cumulative plot the aspects are less rectangular as all activity happens between electrodes so when previously summing frames I rejected areas that were not between electrodes in order to improve performance.
This image was generated with matplotlib with jet colouring. As such it was originally a data matrix. In order to add it to the reference picture I have saved this image as png and reloaded it. This operation changed the aspect ratio slightly but I resized it correctly later.
I have used this function to add to a white rectangle with the dimensions same as the raw footage. Initially I added scaled both top and bottom electrodes to the white background image.
def overlay(background_img, img_to_overlay_t, x, y, negpos, overlay_size=None):
bg_img = background_img.copy()
if overlay_size is not None:
img_to_overlay_t = cv.resize(img_to_overlay_t.copy(), overlay_size)
# Extract the alpha mask of the RGBA image, convert to RGB
b,g,r,a = cv.split(img_to_overlay_t)
overlay_color = cv.merge((b,g,r))
# Apply some simple filtering to remove edge noise
mask = cv.medianBlur(a,5)
#mask = np.zeros(source_img.shape[:2], np.uint8)
h, w, _ = overlay_color.shape
roi = bg_img[y:y+h, x:x+w]
## works sort of but with removed details
#mask = mask.astype(np.int8)
# Black-out the area behind the logo in our original ROI
img1_bg = cv.bitwise_and(roi.copy(),roi.copy(),mask = cv.bitwise_not(mask))
# Mask out the logo from the logo image.
img2_fg = cv.bitwise_and(overlay_color,overlay_color,mask = mask)
img2_fg = img2_fg.astype(np.float32)
img1_bg = img1_bg.astype(np.float32)
if negpos == False:
bg_img[y:y+h, x:x+w] = cv.subtract(img1_bg, img2_fg)
else:
bg_img[y:y+h, x:x+w] = cv.add(img1_bg, img2_fg)
return bg_img
It yields this diagram.
Everything worked fine up until this point. Problem was trying to add cumulative in between the electrodes. My issue comes with being unable to slot that cumulative plot into the reference layout image of the electrodes and total sample size. I get
OpenCV(3.4.1) Error: Assertion failed ((mtype == 0 || mtype == 1) && _mask.sameSize(*psrc1))
at
img1_bg = cv.bitwise_and(roi.copy(),roi.copy(),mask = cv.bitwise_not(mask))
I can fix that by casting mask to uint8 or int8
mask = mask.astype(np.uint8)
This has it's own issues as the blue discharge part ends up lossing all the features and basically consists of 3 different colours only.
This the outcome I'd like to have displayed/saved at the end. I've no idea how to get to it.