I am trying to do some simple drawings. I wanted to use opencv (cv2) because on a second project I have to display a small animation (rectangle, size depending on a variable; updated every X seconds). However, I do not have experience with image processing libraries and opencv.
I am running into a lot of problems, one of which is that I do not know how to display/close images. The image I am creating is a simple fixation cross, black; on a light gray background:
import numpy as np
import cv2
screen_width = 1024
screen_height = 768
img = np.zeros((screen_height, screen_width, 3), np.uint8) # Black image
img = img + 210 # light gray
screen_center = (screen_width//2, screen_height//2)
rect_width = int(0.2*screen_width)
rect_height = int(0.02*screen_height)
xP1 = screen_center[0] - rect_width//2
yP1 = screen_center[1] + rect_height//2
xP2 = screen_center[0] + rect_width//2
yP2 = screen_center[1] - rect_height//2
cv2.rectangle(img, (xP1, yP1), (xP2, yP2), (0, 0, 0), -1)
xP1 = screen_center[0] - rect_height//2
yP1 = screen_center[1] + rect_width//2
xP2 = screen_center[0] + rect_height//2
yP2 = screen_center[1] - rect_width//2
cv2.rectangle(img, (xP1, yP1), (xP2, yP2), (0, 0, 0), -1)
N.B: If there is a better way to create it, I am also interested :)
My goal is for this first project to do have the following code structure:
img = load_saved_img() # The created fixation cross
display_image()
add_text_to_image('texte to add')
# do stuff
# for several minutes
while something:
do_this()
remove_text_from_image() # Alternatively, go back to the initial image/change the image
# do stuff
# for several minutes
while something:
do_this()
close_image()
I know I can add text with cv2.putText() and that I can this way create a second image with the text. What I do not know is how can I manage the displaying of the different images; especially in a light-weight fashion while "doing stuff" on the background. Most people seems to use cv2.waitKey() which is not suited since I do not want to have any user input and since it seems to be something similar to a time.sleep() during which the program is basically paused.
Any tips welcome, even on other libraries and implementation :)
As proposed by #Miki, the combination of .imshow() and .waitKey(1) is working.
cv2.imshow(window, img)
cv2.waitKey(1)
However, those can not be used with time.sleep() to pause the program. Sometimes, the display will not be updated. For instance, on a 3 second countdown:
import time
import cv2
window = 'Name of the window'
def countdown(window, images):
"""
images = [image3, image2, image1]
"""
for img in images:
cv2.imshow(window, img)
cv2.waitKey(1)
time.sleep(1)
Sometimes one of the displays will be skipped. Instead, changing the parameter of cv2.waitKey() to 1000 (timer needed) and removing the use of the time module works best, if no keyboard input is expected during this time.
Related
So guys I'll explain quickly.
I have a fixed camera and I took a photo thus obtaining the "background".
Then my friend stood in front of the camera and I took another photo.
I want to get an image where there is only my friend in the foreground and where it is necessary to delete the background.
I have tried many methods (absdiff(), tensorflow + bodypix and more) but the only method that is giving me good results is using SubtractorKNN
import numpy as np
import cv2
import sys
backgroundSubtractor = cv2.createBackgroundSubtractorKNN(detectShadows=True)
# apply the algorithm for background images using learning rate > 0
for i in range(1, 16):
bgImageFile = "background.jpg"
print ("Opening background", bgImageFile)
bg = cv2.imread(bgImageFile)
backgroundSubtractor.apply(bg, learningRate=0.9)
# apply the algorithm for detection image using learning rate 0
stillFrame = cv2.imread("background-with-friend.jpg")
fgmask = backgroundSubtractor.apply(stillFrame, learningRate=0.9)
kernel = np.ones((3,3),np.uint8)
morphology_img = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN,kernel,iterations=1)
nuovo = morphology_img
#'nuovo.jpg' is nuovo
ok= cv2.imread('nuovo.jpg')
giona = cv2.cvtColor(ok, cv2.COLOR_BGR2GRAY)
ret,range = cv2.threshold(giona,250,255,cv2.THRESH_BINARY)
cv2.imshow("nuovo kernel", cv2.resize(nuovo, (0, 0), fx=0.5, fy=0.5))
cv2.imshow("range", cv2.resize(range, (0, 0), fx=0.5, fy=0.5))
THE QUESTION IS:
there is a way to reconstruct the outline (e.g. left leg, face, arms), fill inside (then eliminate the black spots) and eliminate the white dots in the background (I have already used cv2.morphologyEx but use a kernel bigger would have further ruined the outline of the person).
Is possible ?
If I can get the figure of the person then I can remove the background from the original image.
EDIT
I used cv2.createBackgroundSubtractorKNN, then cv2.morphologyExand finally cv2.threshold(img,250,255,cv2.THRESH_BINARY)to delete shadows
I am trying to remove a fixed background from an image with a single free-falling object. The image has a single free falling object and it has a white background with a circular patch in the middle.
Below is my code for the above task. I have used OpenCV BackgroundSubtractorKNN and BackgroundSubtractorMOG2 algorithm to achieve this task. The left images should be given as input and the code should produce the right images as output.
import numpy as np
import cv2
import sys
import os
#backgroundSubtractor = cv2.createBackgroundSubtractorMOG2()
backgroundSubtractor = cv2.createBackgroundSubtractorKNN()
# apply the algorithm for background images using learning rate > 0
for i in range(1, 16):
bgImageFile = "background/BG.png"
print("Opening background", bgImageFile)
bg = cv2.imread(bgImageFile)
backgroundSubtractor.apply(bg, learningRate=0.5)
# apply the algorithm for detection image using learning rate 0
dirc = os.getcwd()
filepath = os.path.join(dirc,'data')
if not os.path.exists('saved_bgRemoved'):
os.makedirs('saved_bgRemoved')
for file in os.listdir(filepath):
stillFrame = cv2.imread(os.path.join(filepath,file))
fgmask = backgroundSubtractor.apply(stillFrame, learningRate=0)
bgImg = cv2.bitwise_and(stillFrame,stillFrame,mask=fgmask)
# show both images
cv2.imshow("original", stillFrame)
cv2.imshow("mask", fgmask)
cv2.imshow("Cut Image", bgImg)
cv2.waitKey()
cv2.destroyAllWindows()
cv2.imwrite(os.path.join('saved_bgRemoved',file), bgImg)
My code works very well with the above dataset, but it fails to work with the image data below:
It also doesn't work if the object is colored in greyish texture. I think it works well when the pixel distribution of the object is uniform and different from the background (i.e. the circular patch).
Is there any other best way to achieve this task, so that it can subtract the background even from the hollow area of the object, without subtracting parts of the object?
use below code, I think it now works
import cv2, os
def remove_bg(bg_path,im_path):
bg = cv2.imread(bg_path)
im = cv2.imread(im_path)
row,col,_ = im.shape
for i in range(0,row):
for j in range(0,col):
if ( bg[i][j][0] == im[i][j][0] and bg[i][j][1] == im[i][j][1] and bg[i][j][2] == im[i][j][2] ):
im[i][j] = [0,0,0] #it will replace background with black color, you can change it for example to [255,0,0] to replace it with red
return(im)
directory,_=os.path.split(__file__)
bg_path = directory + "\\background.png"
im_path = directory + "\\data6.png"
result = remove_bg(bg_path,im_path)
cv2.imshow("result", result)
cv2.waitKey()
cv2.imwrite(directory + "\\Result.png", result)
First time asking a question on SO.
I am trying to find a fast way to read the screen live (60fps+). Screenshot to numpy is a fast method, but does not match that speed. There is a brilliant answer in this question for pixels: Most efficient/quickest way to parse pixel data with Python?
I tried changing GetPixel to this long form for BMP, but that reduces it to 5fps:
t1 = time.time()
count = 0
width = win32api.GetSystemMetrics(win32con.SM_CXVIRTUALSCREEN)
height = win32api.GetSystemMetrics(win32con.SM_CYVIRTUALSCREEN)
left = win32api.GetSystemMetrics(win32con.SM_XVIRTUALSCREEN)
top = win32api.GetSystemMetrics(win32con.SM_YVIRTUALSCREEN)
while count < 1000:
hwin = win32gui.GetDesktopWindow()
hwindc = win32gui.GetWindowDC(hwin)
srcdc = win32ui.CreateDCFromHandle(hwindc)
memdc = srcdc.CreateCompatibleDC()
bmp = win32ui.CreateBitmap()
bmp.CreateCompatibleBitmap(srcdc, width, height)
memdc.SelectObject(bmp)
memdc.BitBlt((0, 0), (width, height), srcdc, (left, top), win32con.SRCCOPY)
bmpinfo = bmp.GetInfo()
bmpInt = bmp.GetBitmapBits(False)
count +=1
t2 = time.time()
tf = t2-t1
it_per_sec = int(count/tf)
print (str(it_per_sec) + " iterations per second")
I watched a youtube video of a guy working on C# where he said GetPixel opens and closes memory and that's why doing a GetPixel on each individual pixel has a lot of overhead. He suggested to lock the entire data field and only then do getpixel. I don't know how to do that, so any help will be appreciated. (EDIT: this link might refer to that Unsafe Image Processing in Python like LockBits in C# )
There is also another method which gets a memory address of the bitmap, but I don't know what to do with it. The logic there is that I should be able to read memory from that point into any numpy array, but I have not been able to do that.
Any other option to read the screen fast will also be appreciated.
There must be a way, the GPU knows what pixels to draw at each location, that means there must be a memory bank somehere or a data stream we can tap into.
P.S. why a highspeed requirement? I am working on work automation tools that have a lot of overhead already and I am hoping to optimize screen data stream to help that part of the project.
The code below uses MSS, which if modified to show no output can reach 44fps for 1080p. https://python-mss.readthedocs.io/examples.html#opencv-numpy
import time
import cv2
import mss
import numpy
with mss.mss() as sct:
# Part of the screen to capture
monitor = {'top': 40, 'left': 0, 'width': 800, 'height': 640}
while 'Screen capturing':
last_time = time.time()
# Get raw pixels from the screen, save it to a Numpy array
img = numpy.array(sct.grab(monitor))
# Display the picture
#cv2.imshow('OpenCV/Numpy normal', img)
# Display the picture in grayscale
# cv2.imshow('OpenCV/Numpy grayscale',
# cv2.cvtColor(img, cv2.COLOR_BGRA2GRAY))
print('fps: {0}'.format(1 / (time.time()-last_time)))
# Press "q" to quit
if cv2.waitKey(25) & 0xFF == ord('q'):
cv2.destroyAllWindows()
break
Still not perfect though as it is not 60fps+ and using a raw repackaged buffer from the GPU would be a better solution if possible.
Good morning,
I'm currently trying to study real-time liquid surface deformations by sending a laser sheet on the surface and gathering its reflection. What I obtain is typically a bright curve at each timestep, and I wish to analyze its coordinates.
I thus brought myself to write a Python script, which is displayed right below (The analysis part is retaken from laser curved line detection using opencv and python, as it represents nearly exactly what I'm trying to do, except that I'm working with a video flow) :
import cv2
from PIL import Image
import cv2.cv as cv
import numpy as np
import time
myfile = open("hauteur.txt","w")
#Import camera flow
class Target:
def __init__(self):
self.capture = cv.CaptureFromCAM(0)
cv.namedWindow("Target", 1)
cv.SetCaptureProperty(self.capture,cv.CV_CAP_PROP_FRAME_WIDTH, 150)
cv.SetCaptureProperty(self.capture,cv.CV_CAP_PROP_FRAME_HEIGHT, 980)
cv.SetCaptureProperty(self.capture,cv.CV_CAP_PROP_FPS, 60 )
def run(self):
frame = cv.QueryFrame(self.capture)
frame_size = cv.GetSize(frame)
color_image_cv = cv.CreateImage(cv.GetSize(frame), 8, 3)
color_image = np.array(color_image_cv)
grey_image = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_8U, 1)
first = True
t = time.clock()
# Frame analysis
while True:
ret, bw = cv2.threshold(color_image, 0, 255, cv2.THRESH_BINARY)
contours, hierarchy = cv2.findContours(bw, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
curves = np.zeros((img.shape[0], img.shape[1], 3), np.uint8)
for i in range(len(contours)):
for col in range(draw.shape[1]):
M = cv2.moments(draw[:, col])
if M['m00'] != 0:
x = col
y = int (M['m01']/M['m00'])
curves[y, x, :] = (0, 0, 255)
res = {'X' : x, 'Y' : y, 't' : t}
print res
myfile.write('{X}\t{Y}\t{t}'.format(**res))
myfile.write("\n")
cv2.ShowImage("Target", color_image)
# Listen for ESC key
c = cv2.WaitKey(7) % 0x100
if c == 27:
break
if __name__=="__main__":
t = Target()
t.run()
However, the use of cv and cv2 functions within the same code seems to bring a nice mess and I get the error
src data type = 17 is not supported
from line
ret, bw = cv2.threshold(color_image, 0, 255, cv2.THRESH_BINARY)
I understand this arises from the way cv and cv2 functions create and store images, but any conversion process I try doesn't seem to work, and I didn't find equivalent cv2 functions to insert in my video flow importing part (but, as you may understand, I'm clearly not a programming pro and I may have skipped what I'd need in the documentation). Is there then a way to conciliate these cv and cv2 functions, or get a equivalent camera flow with cv2 functions ?
Bonus question : How fast can an script like this run (considering that I'd eventually need this to run at 300-400 fps, I'm not even sure this is actually feasible) ?
Thanks for your attention
ok, cv2 video code:
def __init__(self):
self.capture = cv2.VideoCapture(0)
cv2.namedWindow("Target", 1)
self.capture.set(cv2.CAP_PROP_FRAME_WIDTH, 150)
self.capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 980)
self.capture.set(cv2.CAP_PROP_FPS, 60 )
def run(self):
ok, frame = self.capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY);
...
Bonus question : ofc, it can only run as fast, as the capture delivers. 300fps seems absurd, 30fps, more likely.
I have many skeletonized images like this:
How can i detect a cycle, a loop in the skeleton?
Are there "special" functions that do this or should I implement it as a graph?
In case there is only the graph option, can the python graph library NetworkX can help me?
You can exploit the topology of the skeleton. A cycle will have no holes, so we can use scipy.ndimage to find any holes and compare. This isn't the fastest method, but it's extremely easy to code.
import scipy.misc, scipy.ndimage
# Read the image
img = scipy.misc.imread("Skel.png")
# Retain only the skeleton
img[img!=255] = 0
img = img.astype(bool)
# Fill the holes
img2 = scipy.ndimage.binary_fill_holes(img)
# Compare the two, an image without cycles will have no holes
print "Cycles in image: ", ~(img == img2).all()
# As a test break the cycles
img3 = img.copy()
img3[0:200, 0:200] = 0
img4 = scipy.ndimage.binary_fill_holes(img3)
# Compare the two, an image without cycles will have no holes
print "Cycles in image: ", ~(img3 == img4).all()
I've used your "B" picture as an example. The first two images are the original and the filled version which detects a cycle. In the second version, I've broken the cycle and nothing gets filled, thus the two images are the same.
First, let's build an image of the letter B with PIL:
import Image, ImageDraw, ImageFont
image = Image.new("RGBA", (600,150), (255,255,255))
draw = ImageDraw.Draw(image)
fontsize = 150
font = ImageFont.truetype("/usr/share/fonts/truetype/liberation/LiberationMono-Regular.ttf", fontsize)
txt = 'B'
draw.text((30, 5), txt, (0,0,0), font=font)
img = image.resize((188,45), Image.ANTIALIAS)
print type(img)
plt.imshow(img)
you may find a better way to do that, particularly with path to the fonts. Ii would be better to load an image instead of generating it. Anyway, we have now something to work on:
Now, the real part:
import mahotas as mh
img = np.array(img)
im = img[:,0:50,0]
im = im < 128
skel = mh.thin(im)
noholes = mh.morph.close_holes(skel)
plt.subplot(311)
plt.imshow(im)
plt.subplot(312)
plt.imshow(skel)
plt.subplot(313)
cskel = np.logical_not(skel)
choles = np.logical_not(noholes)
holes = np.logical_and(cskel,noholes)
lab, n = mh.label(holes)
print 'B has %s holes'% str(n)
plt.imshow(lab)
And we have in the console (ipython):
B has 2 holes
Converting your skeleton image to a graph representation is not trivial, and I don't know of any tools to do that for you.
One way to do it in the bitmap would be to use a flood fill, like the paint bucket in photoshop. If you start a flood fill of the image, the entire background will get filled if there are no cycles. If the fill doesn't get the entire image then you've found a cycle. Robustly finding all the cycles could require filling multiple times.
This is likely to be very slow to execute, but probably much faster to code than a technique where you trace the skeleton into graph data structure.