I am trying to build a squat detector in openCV for python for a University project and have been testing a variety of different methods, such as people object detection, barbell tracking and motion detection.
One method I am currently testing at the moment is using openCV's cv2.createBackgroundSubtractorMOG2() function to create a foreground mask of each video frame. Using this mask, I then find the contours which I can use to then draw a bounding box around the (hopefully) detected object (person) in the foreground.
However, looking at some select frames, I can see that sometimes when there looks like a clear object (person) in the foreground mask, the resulting bounding box is either missing or not accurate at all. Please see below for my code and two frame outputs:
cap = cv2.VideoCapture('/content/drive/MyDrive/openCV/GW_BS2.mp4')
minArea = 5000 # Min size in pixels for a region of an image to be considered actual motion
frameNum = 0
backSub = cv2.createBackgroundSubtractorMOG2(150, 16, False)
while True:
# Get current frame
frame = cap.read()[1]
# If frame is None, end the loop as video end has been reached
if frame is None:
break
# Resize the frame, convert to gray and blur ready for back sub
frame = imutils.resize(frame, width=500) # Resize as no point processing large images
fgMask = backSub.apply(frame)
# Find contours to find outlines of white regions of threshold
cnts, _ = cv2.findContours(fgMask.copy(), cv2.RETR_TREE,
cv2.CHAIN_APPROX_NONE)
for cnt in cnts:
(x, y, w, h) = cv2.boundingRect(cnt)
if cv2.contourArea(cnt) < minArea:
continue
cv2.drawContours(frame, cnt, -1, (255, 0, 0), 3)
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.putText(frame, str(cv2.contourArea(cnt)), (x+5, y+15),
cv2.FONT_HERSHEY_SIMPLEX, 0.75, (0,0,0))
# Add frame number to top left of frame
cv2.rectangle(frame, (10, 2), (100, 20), (255, 255, 255), -1)
cv2.putText(frame, str(frameNum), (15, 15),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,0,0))
if frameNum%10 == 0:
# Show every 10 frames
cv2_imshow(frame)
cv2_imshow(fgMask)
frameNum += 1
(I am using Google colab, so please excuse the weird cv2_imshow() function.)
See below for a couple of foreground masks and their respective output frame:
As you can see, there is seemingly quite a good foreground mask here, but looking at the top output frame, there is no contours, nor bounding box. Whereas the bottom example, there is a slightly noisier foreground mask with some erroneous boxes and contours.
Any help on how to get the box to cover the whole person will be greatly appreciated!
Many thanks,
Harry
Related
I have an image I need to draw a bounding box around and I'm trying to use the code at bottom of this post.
The issue I have is that I have tried to blur the blue box shape to remove its details e.g.
cv2.blur(img,(20,20))
but the blurred image doesnt seem to have a good enough edge to produce a bounding box.
I have found that if I use my code below with a plain blue square with sharp edges of the same size as the image below, it works just fine. It seems the detail within the blue area stops a boundary box being drawn.
I was hoping someone could tell me how to get this to work please.
import cv2
# Load the image - container completely blurred out so no hinges,locking bars , writing etc are visible, just a blank blue square
img = cv2.imread('blue_object.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
edged = cv2.Canny(img, 120,890)
# Apply adaptive threshold
thresh = cv2.adaptiveThreshold(edged, 255, 1, 1, 11, 2)
thresh_color = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
# apply some dilation and erosion to join the gaps - change iteration to detect more or less area's
thresh = cv2.dilate(thresh,None,iterations = 50)
thresh = cv2.erode(thresh,None,iterations = 50)
# Find the contours
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# For each contour, find the bounding rectangle and draw it
for cnt in contours:
area = cv2.contourArea(cnt)
if area > 50000:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The image below is the yellow container after lower HSV mask range of 8,0,0 and upper range of 78,255,255 . Trees are above and to top right of the container, so hard to separate the tress from the container to put a proper bounding box around it.Happy to move to chat if that helps.
You're converting to gray, throwing all that valuable color information away. You're also Canny-ing, which is generally a bad idea. Beginners don't have the judgment to apply Canny sensibly. Best stay away from it.
This can be solved with the usual approach:
transform colorspace into something useful, say HSV
inRange on well-saturated blue
some morphology operations to clean up debris
bounding box
That is assuming you are looking for a blue container, or any well-saturated color really.
im = cv.imread("i4nPk.jpg")
hsv = cv.cvtColor(im, cv.COLOR_BGR2HSV)
lower = (90, 84, 0)
upper = (180, 255, 255)
mask1 = cv.inRange(hsv, lower, upper)
mask2 = cv.erode(mask1, kernel=None, iterations=2)
(x, y, w, h) = cv.boundingRect(mask2) # yes that works on masks too
canvas = cv.cvtColor(mask2, cv.COLOR_GRAY2BGR)
cv.rectangle(canvas, (x,y), (x+w, y+h), color=(0,0,255), thickness=3)
I need some help with a project. My intention is to crop videos of sonographies with OpenCV and python in order to process them further. The features I am looking for are:
Loop through all available videos in a folder
find the contours and crop
export each video with one fixed size and resolution
Now i am a bit stuck on the contour finding and cropping part. I would like OpenCV to automatically recognize a bounding box around the shape of the sonography, knowing that all videos have the particular conus shape. Besides, it would be great if the non-relevant clutter could be removed. Can you help me? Attached you can find one original frame of the videos and the desired result.
import cv2
import numpy as np
cap = cv2.VideoCapture('video.mjpg')
# (x, y, w, h) = cv2.boundingRect(c)
# cv2.rectangle(frame, (x,y), (x+w, y+h), (0, 255, 0), 20)
# roi = frame[y:y+h, x:x+w]
while True:
ret, frame = cap.read()
# (height, width) = frame.shape[:2]
sky = frame[0:100, 0:200]
cv2.imshow('Video', sky)
if cv2.waitKey(1) == 27:
exit(0)
For the first frame of video; you can use this to detect the bounding-box of the image and then you can crop it or whatever you want :)
import sys
import cv2
import numpy as np
# Load our image
dir = sys.path[0]
org = cv2.imread(dir+'/im.png')
im=org.copy()
H,W=im.shape[:2]
# Convert image to grayscale
im=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
# remove noise
im=cv2.GaussianBlur(im,(21,21),21)
im=cv2.erode(im,np.ones((5,5)))
# remove horizantal line
im=cv2.GaussianBlur(im,(5,0),21)
blr=im.copy()
# make binary image
im=cv2.threshold(im,5,255,cv2.THRESH_BINARY)[1]
# draw black border around image to better detect blobs:
cv2.rectangle(im,(0,0),(W,H),0,thickness=W//25)
bw=im.copy()
# Invert the black and white colors
im=~im
# Find contours and sort them by width
cnts, _ = cv2.findContours(im, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
cnts.sort(key=lambda x: cv2.boundingRect(x)[2],reverse=True)
# Change the type and channels of image copies
im=cv2.cvtColor(im,cv2.COLOR_GRAY2BGR)
bw=cv2.cvtColor(bw,cv2.COLOR_GRAY2BGR)
blr=cv2.cvtColor(blr,cv2.COLOR_GRAY2BGR)
# Find the second biggest blob
x, y, w, h = cv2.boundingRect(cnts[1])
cv2.rectangle(org, (x, y), (x+w, y+h), (128, 0, 255), 10)
cv2.rectangle(im, (x, y), (x+w, y+h), (128, 255, 0), 10)
print(x,y,w,h)
# Save final result
top=np.hstack((blr,bw))
btm=np.hstack((im,org))
cv2.imwrite(dir+'/img_.png',np.vstack((top,btm)))
Bounding-Box area:
133 25 736 635
Cut and save the final image:
org = cv2.imread(dir+'/im.png')
cv2.imwrite(dir+'/img_.png',org[y:y+h,x:x+w])
I tried the code provided bellow to segment each digit in this image and put a contour around it then crop it out but it's giving me bad results, I'm not sure what I need to change or work on.
The best idea I can think of right now is filtering the 4 largest contours in the image except the image contour itself.
The code I'm working with:
import sys
import numpy as np
import cv2
im = cv2.imread('marks/mark28.png')
im3 = im.copy()
gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.adaptiveThreshold(blur, 255, 1, 1, 11, 2)
################# Now finding Contours ###################
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
samples = np.empty((0, 100))
responses = []
keys = [i for i in range(48, 58)]
for cnt in contours:
if cv2.contourArea(cnt) > 50:
[x, y, w, h] = cv2.boundingRect(cnt)
if h > 28:
cv2.rectangle(im, (x, y), (x + w, y + h), (0, 0, 255), 2)
roi = thresh[y:y + h, x:x + w]
roismall = cv2.resize(roi, (10, 10))
cv2.imshow('norm', im)
key = cv2.waitKey(0)
if key == 27: # (escape to quit)
sys.exit()
elif key in keys:
responses.append(int(chr(key)))
sample = roismall.reshape((1, 100))
samples = np.append(samples, sample, 0)
responses = np.array(responses, np.float32)
responses = responses.reshape((responses.size, 1))
print
"training complete"
np.savetxt('generalsamples.data', samples)
np.savetxt('generalresponses.data', responses)
I need to change the if condition on height probably but more importantly I need if conditions to get the 4 largest contours on the image. Sadly, I haven't managed to find what I'm supposed to be filtering.
This is the kind of results I get, I'm trying to escape getting those inner contours on the digit "zero"
Unprocessed images as requested: example 1 example 2
All I need is an idea on what I should filter for, don't write code please. Thank you community.
You almost have it. You have multiple bounding rectangles on each digit because you are retrieving every contour (external and internal). You are using cv2.findContours in RETR_LIST mode, which retrieves all the contours, but doesn't create any parent-child relationship. The parent-child relationship is what discriminates between inner (child) and outter (parent) contours, OpenCV calls this "Contour Hierarchy". Check out the docs for an overview of all hierarchy modes. Of particular interest is RETR_EXTERNAL mode. This mode fetches only external contours - so you don't get multiple contours and (by extension) multiple bounding boxes for each digit!
Also, it seems that your images have a red border. This will introduce noise while thresholding the image, and this border might be recognized as the top-level outer contour - thus, every other contour (the children of this parent contour) will not be fetched in RETR_EXTERNAL mode. Fortunately, the border position seems constant and we can eliminate it with a simple flood-fill, which pretty much fills a blob of a target color with a substitute color.
Let's check out the reworked code:
# Imports:
import cv2
import numpy as np
# Set image path
path = "D://opencvImages//"
fileName = "rhWM3.png"
# Read Input image
inputImage = cv2.imread(path+fileName)
# Deep copy for results:
inputImageCopy = inputImage.copy()
# Convert BGR to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
The first step is to get the binary image with all the target blobs/contours. This is the result so far:
Notice the border is white. We have to delete this, a simple flood-filling at position (x=0,y=0) with black color will suffice:
# Flood-fill border, seed at (0,0) and use black (0) color:
cv2.floodFill(binaryImage, None, (0, 0), 0)
This is the filled image, no more border!
Now we can retrieve the external, outermost contours in RETR_EXTERNAL mode:
# Get each bounding box
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
Notice you also get each contour's hierarchy as second return value. This is useful if you want to check out if the current contour is a parent or a child. Alright, let's loop through the contours and get their bounding boxes. If you want to ignore contours below a minimum area threshold, you can also implement an area filter:
# Look for the outer bounding boxes (no children):
for _, c in enumerate(contours):
# Get the bounding rectangle of the current contour:
boundRect = cv2.boundingRect(c)
# Get the bounding rectangle data:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Estimate the bounding rect area:
rectArea = rectWidth * rectHeight
# Set a min area threshold
minArea = 10
# Filter blobs by area:
if rectArea > minArea:
# Draw bounding box:
color = (0, 255, 0)
cv2.rectangle(inputImageCopy, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 2)
cv2.imshow("Bounding Boxes", inputImageCopy)
# Crop bounding box:
currentCrop = inputImage[rectY:rectY+rectHeight,rectX:rectX+rectWidth]
cv2.imshow("Current Crop", currentCrop)
cv2.waitKey(0)
The last three lines of the above snippet crop and show the current digit. This is the result of detected bounding boxes for both of your images (the bounding boxes are colored in green, the red border is part of the input images):
I want to ignore all rectangles or contours that are overlapping or inside a big rectangle, I found many solutions but no one work in my case.
import numpy as np
import cv2
import imutils
cap = cv2.VideoCapture('rtsp://admin:admin#192.168.1.72')
#read the first frame from camera for our background
_,first_frame = cap.read()
#We’ll also convert the image to grayscale since color has no bearing on our motion detection
first_gray = cv2.cvtColor(first_frame, cv2.COLOR_BGR2GRAY)
#Due to tiny variations in the digital camera sensors, no two frames will be 100% same, to account for this and apply Gaussian smoothing
first_gray = cv2.GaussianBlur(first_gray, (21, 21), 0)
open('/tmp/test.txt', 'w').close()
while(1):
_, frame = cap.read()
#We’ll also convert the image to grayscale since color has no bearing on our motion detection
gray_frame = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
#Due to tiny variations in the digital camera sensors, no two frames will be 100% same, to account for this and apply Gaussian smoothing
blurFrame = cv2.GaussianBlur(gray_frame, (21, 21), 0)
#Computing the difference between two frames is a simple subtraction
diff = cv2.absdiff(first_gray, blurFrame)
_,thresh = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)
# dilate the thresholded image to fill in holes
thresh = cv2.dilate(thresh, None, iterations=2)
#find contours on thresholded image
contours,_ = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
pixelList = \[\]
for contour in contours:
if( cv2.contourArea(contour) > 100):
(x, y, w, h) = cv2.boundingRect(contour)
pixelList.append(list((x, y, w, h)))
cv2.rectangle(frame,(x,y),(x+w,y+h),(0,255,0),2)
if len(pixelList) !=0:
with open("/tmp/test.txt", "a") as myfile:
myfile.write(str(pixelList)+'\n')
orgFrame = cv2.resize(frame, (600, 600))
diffFrame = cv2.resize(diff, (300, 300))
cv2.imshow('diffFrameBlur',diff)
cv2.imshow('frameBlur',frame)
k = cv2.waitKey(1) & 0xff
if k == 27:
break
cap.release()
cv2.destroyAllWindows()
Please look at the image attached in here you will find that lots of contours detected inside a big contour, I really want to eliminate these all contours(small) that is inside a big contour or even you can say rectangle that, I draw after calculating area.
Compare the top left point and bottom right point of each rectangle, contained in another rectangle, then eliminate them.
Use this function below to check if a point is inside the rectangle.
def rectContains(rect,pt):
in = rect[0] < pt[0] < rect[0]+rect[2] and rect[1] < pt[1] < rect[1]+rect[3]
return in
Call this function only for top left and bottom right for each rectangle, and if its contained inside another rectangle, eliminate them.
If you are intending to make it faster, the reduce the number of comparisons.
For all the contours detected, sort them in the order of size,
cntsSorted = sorted(cnts, key=lambda x: cv2.contourArea(x))
From the sorted contours and start from smallest, and compare it with rectangle which are largest. basically first element with the last element and so on
I am working on a project which ask me to detect text area in an image. This is the result I achieved until now using the code below.
Original Image
Result
The code is the following:
import cv2
import numpy as np
# read and scale down image
img = cv2.pyrDown(cv2.imread('C:\\Users\\Work\\Desktop\\test.png', cv2.IMREAD_UNCHANGED))
# threshold image
ret, threshed_img = cv2.threshold(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY),
127, 255, cv2.THRESH_BINARY)
# find contours and get the external one
image, contours, hier = cv2.findContours(threshed_img, cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
# with each contour, draw boundingRect in green
# a minAreaRect in red and
# a minEnclosingCircle in blue
for c in contours:
# get the bounding rect
x, y, w, h = cv2.boundingRect(c)
# draw a green rectangle to visualize the bounding rect
cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), thickness=1, lineType=8, shift=0)
# get the min area rect
#rect = cv2.minAreaRect(c)
#box = cv2.boxPoints(rect)
# convert all coordinates floating point values to int
#box = np.int0(box)
# draw a red 'nghien' rectangle
#cv2.drawContours(img, [box], 0, (0, 0, 255))
# finally, get the min enclosing circle
#(x, y), radius = cv2.minEnclosingCircle(c)
# convert all values to int
#center = (int(x), int(y))
#radius = int(radius)
# and draw the circle in blue
#img = cv2.circle(img, center, radius, (255, 0, 0), 2)
print(len(contours))
cv2.drawContours(img, contours, -1, (255, 255, 0), 1)
cv2.namedWindow('contours', 0)
cv2.imshow('contours', img)
while(cv2.waitKey()!=ord('q')):
continue
cv2.destroyAllWindows()
As you can see, this can do more than I need. Look for commented parts if you need more.
By the way, what I need is to bound every text area in a single rectangle not (near) every char which the script is finding. Filter the single number or letter and to round everything in a single box.
For example, the first sequence in a box, the second in another one and so on.
I searched a bit and I found something about "filter rectangle area". I don't know if it is useful for my purpose.
Tooked a look also at some of the first result on Google but most of them don't work very well. I guess the code need to be tweaked a bit but I am a newbie in OpenCV world.
Solved using the following code.
import cv2
# Load the image
img = cv2.imread('image.png')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# smooth the image to avoid noises
gray = cv2.medianBlur(gray,5)
# Apply adaptive threshold
thresh = cv2.adaptiveThreshold(gray,255,1,1,11,2)
thresh_color = cv2.cvtColor(thresh,cv2.COLOR_GRAY2BGR)
# apply some dilation and erosion to join the gaps - change iteration to detect more or less area's
thresh = cv2.dilate(thresh,None,iterations = 15)
thresh = cv2.erode(thresh,None,iterations = 15)
# Find the contours
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# For each contour, find the bounding rectangle and draw it
for cnt in contours:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv2.rectangle(thresh_color,(x,y),(x+w,y+h),(0,255,0),2)
# Finally show the image
cv2.imshow('img',img)
cv2.imshow('res',thresh_color)
cv2.waitKey(0)
cv2.destroyAllWindows()
Parameters that need to be modified to obtain the result below is numbers of iterations in erode and dilate functions.
Lower values will create more bounding rectangles around (nearly) every digit/character.
Result