I am writing an opencv program where I track position of an object by use of a usb camera. To make sure I get as high frame rate as possible with the camera I am using I made a threaded process that read the images from the camera. The image processing is done in another loop which also writes the object position to the file.
Now I want a way to avoid processing the same frame multiple times. So I thought I could compare the image just processed with that available from the the video stream thread.
First I tried to use if frame1 == frame2, but got error message that "the truth value of an array with more than one element is ambiguous. Use a.any() or a.all()."
After some googling I found cv2.compare and the flag CMP_EQ. Made a sample code, and made it work in some way. However, my question is. How could this be done in an easier or better way?
import cv2
cv2.namedWindow('image1', cv2.WINDOW_NORMAL)
cv2.namedWindow('image2', cv2.WINDOW_NORMAL)
frame1 = cv2.imread("sample1.png")
frame2 = frame1
frame3 = cv2.imread("sample2.png")
compare1 = cv2.compare(frame1,frame2,0)
compare2 = cv2.compare(frame1,frame3,0)
cv2.imshow('image1', compare1)
cv2.imshow('image2', compare2)
if compare1.all():
print "equal"
else:
print "not equal"
if compare2.all():
print "equal"
else:
print "not equal"
cv2.waitKey(0)
cv2.destroyAllWindows()
open("image1.jpg","rb").read() == open("image2.jpg","rb").read()
should tell you if they are exactly the same ...
I was doing something close to what you are doing; I was trying to get the difference. I used the subtract function. It may help you.
UPDATE:
import cv2
import numpy as np
a = cv2.imread("sample1.png")
b = cv2.imread("sample2.png")
difference = cv2.subtract(a, b)
result = not np.any(difference)
if result is True:
print "Pictures are the same"
else:
cv2.imwrite("ed.jpg", difference )
print "Pictures are different, the difference is stored as ed.jpg"
How about giving your Images an index?
Pseudocode:
class Frame
{
cvImage img;
uint idx;
}
Than simply check if the current index is greater than the last one you processed.
It is simple and definitely faster than any image processing based approach.
You can compare the size of two image files as the first level of check for reduced computational complexity. With compression, it is highly unlikely for two different image files to have the same size to the accuracy of the number of bytes. With equal file size, you can then compare the image contents.
You should try something like this.
import cv2
import numpy as np
original = cv2.imread("rpi1.jpg")
duplicate = cv2.imread("rpi2.jpg")
if original.shape == duplicate.shape:
print("The images have same size and channels")
difference = cv2.subtract(original, duplicate)
b, g, r = cv2.split(difference)
if cv2.countNonZero(b) == 0 and cv2.countNonZero(g) == 0 and
cv2.countNonZero(r) == 0:
print("The images are completely Equal")
else:
print("images are different")
Related
I am trying to process several folders that contain many rasters; in each folder, there are rasters with different dates on the same area. In order to save some time, I want to use the multiprocessing (or multithreading?) module to work in parallel.
Basically, my script does this: for one pixel, it makes some calculation on the first pixel and loads it to a numpy array if the number is higher than the previous one that was in the numpy array; then it continues with another pixel. The result should be several numpy arrays (one per folder).
It works fine without multiprocessing; when I try to multiprocess it, it gets very slow and doesn't take advantage of all 10 cores:
Here is my code:
import os, sys, math, time, datetime
import numpy as np
from numpy import *
from osgeo import gdal,gdal_array,osr
from itertools import islice
from multiprocessing import Pool, Process
import multiprocessing
#prints full size numpy array instead of extract
np.set_printoptions(threshold=sys.maxsize)
#define tresholds for dNBR, NBR and NDVI difference (ratio NDVIafter/NDVIbefore)
dNBRthreshold=0.15
RdNBRthreshold=0.4
NDVIdiffThreshold=0.1
def proc (path):
#print information to a log file
log = open(path+"\\myprog.log", "a")
sys.stdout = log
#create a list of all files in the current directory
ListImages=[]
for file in os.listdir(path):
if file.endswith(".tif"):
ListImages.append(os.path.join(path, file))
#sort the list aphabetically
ListImages.sort()
print ("Image list: ", ListImages)
#create empty numpy array the same size as the first image and with number of bands defined by user
firstImage=gdal.Open(ListImages[0])
band0 = firstImage.GetRasterBand(1)
arrayOfFirstImage = band0.ReadAsArray()
listEmpty=[]
#create numpy array with same size as first image but dimension defined by user in "range"
for x in range(30):
name="emptyArray_" + str(x)
#create raster with same size as first image
name=np.full_like(arrayOfFirstImage, np.nan, dtype=np.double)
listEmpty.append(name)
arrayStack=np.stack(listEmpty)
num_dim, num_rows,num_cols = arrayStack.shape
listRows = list(range(num_rows))
#creates loop over all pixels in raster
for row in range(num_rows):
print("row number: ", row)
for col in range(num_cols):
#reset counter for band as script is working with a new pixel; cntrForBand is used to change arrayStack bands that will be written on
cntrForBand=0
print("col number: ", col)
#loop for all images in list ListImages to get image 1
#user ITER to be able to jump 7 o 22 loops
iterListImages = iter(ListImages)
for image in iterListImages:
#get number of image in the List of Images
indexImage1 = ListImages.index(image)
#get its full path
img1Path=os.path.abspath(image)
print ("path image 1: " + img1Path)
print ("index Image 1: ",indexImage1)
#open geotiff with gdal
img = gdal.Open(image)
#get first band data of image 1: NDVI value
band1Image1=img.GetRasterBand(1)
#get second band data of image 1: NBR value
band2Image1 = img.GetRasterBand(2)
## compute statistics of band 1
if band1Image1.GetMinimum() is None or band1Image1.GetMaximum()is None:
band1Image1.ComputeStatistics(0)
print("Statistics computed.")
## compute statistics of band 2
if band2Image1.GetMinimum() is None or band2Image1.GetMaximum()is None:
band2Image1.ComputeStatistics(0)
print("Statistics computed.")
#converts gdal array (raster or band) into a numpy array:
band1Image1asArray = band1Image1.ReadAsArray()
#print ("NDVI array= ",band1Image1asArray)
band2Image1asArray = band2Image1.ReadAsArray()
#Get NDVI value of pixel of interest
itemNDVIimage1=band1Image1asArray[row][col]
print("itemNDVIimage1: ",itemNDVIimage1)
#Get NBR value of pixel of interest
itemImage1=band2Image1asArray[row][col]
print("itemImage1: ",itemImage1)
#if pixel has no value, don´t do anything
if itemImage1== band2Image1.GetNoDataValue() or itemImage1==-32768:
print("row number: ", row)
print("col number: ", col)
print ("image 1 pixel with no data value; initiating with another image")
#if pixel has a value, proceed
else:
#reset switch to False (switch is used to skip images
switch1=False
#list of numbers for image 2: from index of image + 1 to index of image 1 + 8
listImg2=[indexImage1+1,indexImage1+2,indexImage1+3,indexImage1+4,indexImage1+5,indexImage1+6,indexImage1+7,indexImage1+8]
for indexImg2 in listImg2:
print("length list image: ", len(ListImages))
print ("Current indexImg2: ", indexImg2)
print("row number: ", row)
print("col number: ", col)
#if number of image 2 is above number of images in list, stop (all images have been processed)
if indexImg2>=len(ListImages):
break
#if not, proceed
else:
#open next image in the list (next date)
image2=gdal.Open(ListImages[indexImg2])
img2Path=os.path.abspath(ListImages[indexImg2])
print ("path image 2: " + img2Path)
#get image 2 NDVI value for this pixel
band1Image2 = image2.GetRasterBand(1)
band1Image2AsArray = band1Image2.ReadAsArray()
itemNDVIimage2=band1Image2AsArray[row][col]
print("item image 2, Band 1 (NDVI): ", itemNDVIimage2)
#get image 2 NBR value for this pixel
band2Image2 = image2.GetRasterBand(2)
band2Image2AsArray = band2Image2.ReadAsArray()
#print ("Image 2, Band 2:",band2Image2AsArray)
itemImage2=band2Image2AsArray[row][col]
print("item image 2: ", itemImage2)
#if image 2 has no value for NBR band, stop and continue with next image 2
if itemImage2== band2Image2.GetNoDataValue() or itemImage2==-32768:
print ("image 2 pixel with no data value; initiating with another image")
else:
#calculate dNBR, NBR and NDVI difference between the two images
dNBR=itemImage1-itemImage2
RdNBR=dNBR/(math.sqrt(abs(itemImage1)))
NDVIdiff=1-itemNDVIimage2/itemNDVIimage1
print ("dNBR: ",dNBR)
print ("RdNBR: ", RdNBR)
print ("NDVI difference: ", NDVIdiff)
#if dNBR equals exactly 0, it means that image 1 and image 2 were the same; stop and continue with next image
if dNBR==0:
print("same image for image 1 and image2; initiating with another image for image 2")
#if dNBR, NBR or NDVI difference values are under thresholds, stop and continue with next image
elif dNBR<dNBRthreshold or RdNBR<RdNBRthreshold or NDVIdiff<NDVIdiffThreshold :
print("dNBR or RdNBR or NDVIdiff under threshold; continue with next image for image 2")
else:
#open empty image and set new dNBR and RdNBR and date values in first, second and third band respectively. in ArrayStack, first number is number of band (first is zero) then row then column.
#if dNBR or RdNBR values is above value already saved in the array or if current value is empty (nan), overwrite it; else, don't overwrite it
print ("current dNBR value for this cell in arrayStack: ",arrayStack[cntrForBand][row][col])
if (dNBR>arrayStack[cntrForBand][row][col] and RdNBR>arrayStack[cntrForBand+1][row][col]) or (math.isnan(arrayStack[cntrForBand][row][col])):
#keep dNBR, RdNBR and date value in first, second and third of the three bands (hence cntrForBand for dNBR, cntrForBand+1 for RdNBR and cntrForBand+2 for Date)
arrayStack[cntrForBand][row][col]= dNBR
arrayStack[cntrForBand+1][row][col]= RdNBR
#arrayStack[0,0,0]=dNBR
#date value put in second band
date=int(img2Path[-15:-8])
arrayStack[cntrForBand+2][row][col]= date
print ("arrayStack updated: ",arrayStack)
#turn switch on to skip 22 images (forest and therefore fire won't come back soon...)
switch1= True
else:
#print(arrayStack)
print ("dNBR value lower than value already in arrayStack; not changing value")
#if one value of dNBR and RdNBR is above threshold during loops with image 1 and 2, then skip 6 monts and continue with image 1 + 22
#else, continue with image 1 + 7
if switch1==True:
next(islice(iterListImages, 44, 44), None) # consume 22
print("a value has been found for this set of 8 images; continuing with image 1 + 44")
#cntr for band increments with 3 so that next round three other bands of arrayStack get the dNBR, NBR and Date values
cntrForBand=cntrForBand+3
print ("cntrForBand=",cntrForBand)
else:
#if no high value found, go to image+7 in list
next(islice(iterListImages, 7, 7), None)
print("No value found for this set of 8 images; continuing with next image (+1)")
print ("done!!!!")
print (arrayStack)
np.save(path+"\\FINAL.csv", arrayStack)
print("file FINAL.csv saved")
if __name__ == '__main__':
listFolders= [ f.path for f in os.scandir("C:\\incendios\\Temp3") if f.is_dir() ]
print (listFolders, type(listFolders))
cpuCount = os.cpu_count()
print ("number of core: ",cpuCount)
p = Pool(10)
print(p.map(proc,listFolders))
If a run a very simple code that uses NumPy, it works perfectly fine and uses 100% CPU and all 10 cores:
import multiprocessing
import time
import numpy as np
start = time.time()
print("hello")
array=np.random.rand(3000,3000)
def worker():
"""worker function"""
mult=np.dot(array, array)
print (mult)
return mult
if __name__ == '__main__':
jobs = []
for i in range(50):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()
end = time.time()
print(end - start)
I know that NumPy can cause some issues with multiprocessing, but this doesn't seem to be the issue I have here.
So I guess there is something wrong with my code that makes it difficult to process with multiple cores. Is there anything I can do to improve it?
PS: I'm using Windows 10 64 bits and python 3.5.0 and the script works fine without multiprocessing...
EDIT:
to answer Mark Stechell´s question: Actually I have 10 folders; each folder has around 900 rasters that cover one area per folder, with one raster every 8 days from 2000 to 2020. These rasters are satellite images that I have already processed; the first band is a Vegetation Index (called NDVI) and the second one is a Burn Area Index (NBR, a basic index used to identify forestry fires); in this script, I use those data to calculate other indexes (dNBR and RdNBR; the last one is a relative index, it means that I compare NBR indexes of two different dates to detect a significant change). If those indexes are high enough (threshold are defined at the beginning of the script) which means that a forestry fire was detected, I keep the NDVI and RdNBR values in a numpy array with the date. But I only do this comparison with 8 following dates; if no significant value has been found, the script goes on with another image in the list and its 7 following images (chronologically); If a significant value has been found, the script jumps 22 images in the list because another forestry fire won´t happen again in this area before a long time..
Following mkrieger1´s advice, I am trying to simplify this as much as a can to see where the problem is. I am also going to try to use Pool in the very simple code I mentioned to see if this works
So, following mkrieger1´s advice (many thanks, now I know...), I´ve tried to run my script line by line to see where the problem was. It is clearly related to the GDAL library. the getNoDataValue(), getMinimum() and getMaximum() functions are the problem here for multiprocessing. I have change the code with functions related to other library (if itemImage1==getNoDataValue () has been changed with if math.isnan(x) for instance).
Now it is working perfectly...
I hope it will help others with the same issue.
Many thanks!
How do I know if my webcam is currently working? I tried to check " stream.read () " because it returns "None" when the camera is not active. But when the camera is active " stream.read () " returns an array and I get an error "The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()". How do I fix this?
my code:
import cv2
import time
from tkinter import *
stream = cv2.VideoCapture(0)
time.sleep(10)
while True:
r, f = stream.read ()
a=f
print(a)
if a==None:
print("No active")
else:
print("Active")
cv2.imshow('IP Camera stream',f)
# f = imutils.resize(f, width=400)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()
read() returns both an array and a boolean to indicate whether it was successful in reading a frame, the r in your code. Use that value instead:
if r == False:
print("No frame read")
else:
print("Succes")
docs
That checks if a frame was read. However, a frame might not be read even when the camera is active. The best way to check if the camera is active is to check :
open = stream.isOpened()
if open:
print('Camera active')
docs
f is numpy's array and to check it is not None you have to check
if f is None:
instead of
if f == None:
When f has array then you compare array == None and it tries to compare None with every element in array and it is ambiguous - so it ask to use any() or all()
I am running some code called bcf.py it is very long and convoluted, but in short, it extracts 300 feature points from each image from a group of folders. So potentially there could be hundreds of images.
When I go to run the script, everything works except I have to keep pressing the return button to extract all the feature points and repeat this button pressing for every image, which is frustrating.
Why does it do this, and how would I fix it? The goal is to press the wait key once and extract the features.
Thank you
I do not know what this is called to be able to search for an answer.
def _svm_classify_test(self):
clf = self._load_classifier()
label_to_cls = self._load_label_to_class_mapping()
testing_data = []
labels = []
for (cls, idx) in self.data.keys():
testing_data.append(self.data[(cls, idx)]['spp_descriptor'])
labels.append(hash(cls))
predictions = clf.predict(testing_data)
correct = 0
for (i, label) in enumerate(labels):
if predictions[i] == label:
correct += 1
else:
print("Mistook %s for %s" % (label_to_cls[label], label_to_cls[predictions[i]]))
print("Correct: %s out of %s (Accuracy: %.2f%%)" % (correct, len(predictions), 100. * correct / len(predictions)))
def show(self, image):
cv2.imshow('image', image)
_ = cv2.waitKey()
The goal is to press the wait key once and automatically runs through all the images and extract the features.
The function cv.waitKey([, delay]) as it is explained in the documentation, it may take a value which you can consider as timeout. That means, you can pass a 10 and it will block the function for 10 milliseconds for a keyboard input.
For your case, I do not see where in the code you use your function show so I can't know exactly how you should do it to have that behaviour, but as a pseudocode for you to get an idea, it will be something like:
filenames = [] #lets assume your filenames are here
for f in filenames:
img = cv2.imread(f)
cv2.imshow("image", img)
cv2.waitKey(10)
If you want a pause at the beginning you can do an imshow outside the loop and a waitkey with 0 after it. Also, you can play with the amount of time, like 5000 to display it for 5 seconds before continuing.
But if it takes too long to process you may consider putting the imshow part in a thread, since the window maybe unresponsive after the waitKey while it waits for the feature extraction process to finish. Also, it may be good to add something like 'q' to quit the app or something.... These are just some suggestions :)
I currently am working on a project where I need to parse a video and pass it through several models. The videos come in at 60fps. I do not need to run every frame through the models. I am running into some issue when trying to skip the unneeded frames. I have tried two methods which are both fairly slow.
Method 1 code snippet:
The issue here is that I am still reading every frame of the video. Only every 4th frames is run through my model.
cap = cv2.VideoCapture(self.video)
while cap.isOpened():
success, frame = cap.read()
if count % 4 !=0:
count += 1
continue
if success:
''Run frame through models''
else:
break
Method 2 code snippet:
This method is slower. I am trying to avoid reading unnecessary frames in this case.
cap = cv2.VideoCapture(video)
count=0
while True:
if count % 4 != 0:
cap.set(cv2.CV_CAP_PROP_POS_FRAMES, count*15)
count+=1
success, frame = cap.read()
Any advice on how to achieve this in the most efficient way would be greatly appreciated.
Getting and setting frames by changing CV_CAP_PROP_POS_FRAMES is not accurate (and slow) due to how video compression works: by using keyframes.
It might help to not use the read() function at all. Instead use grab() and only retrieve() the needed frames. From the documentation: The (read) methods/functions combine VideoCapture::grab() and VideoCapture::retrieve() in one call.
grab() gets the frame data, and retrieve() decodes it (the computationally heavy part). What you might want to do is only grab frames you want to skip but not retrieve them.
Depending on your system and opencv build, you could also have ffmpeg decode video using hardware acceleration.
As I get it you are trying to process every fourth frame. You are using the condition:
if count % 4 != 0
which is triggered 3 out of 4 frames instead (you are processing frames 1, 2, 3, 5, 6, 7 etc)! Use the opposite:
if count % 4 == 0
Also although code snippets the two method does not seem to process the same frames. Although in both cases your counter seems to increase by 1 in each frame you actually point to the 15xframe of that counter in the second case (cap.set(cv2.CV_CAP_PROP_POS_FRAMES, count*15).
Some comments on your code also (maybe I misunderstood something):
Case 1:
while cap.isOpened():
success, frame = cap.read()
if count % 4 !=0:
count += 1
continue
here you seem to count only some frames (3 out of 4 as mentioned) since frames which are multiples of 4 are skipped: condition count % 4 !=0 is not met in that case and your counter is not updated although you read a frame. So, you have an inaccurate counter here. It's not shown how and where do you process you frames to judge that part though.
Case 2:
while True:
if count % 4 != 0:
cap.set(cv2.CV_CAP_PROP_POS_FRAMES, count*15)
count+=1
success, frame = cap.read()
here you read frames only if condition is met so in this code snippet you don't actually read any frame since frame 0 does not trigger the condition! If you update the counter outside the if scope it's not clear here. But if you do you should also read a frame there. Anyway more code should be revealed to tell.
As a general advice you should update the counter everytime you read a frame.
Instead putting a threshold on number of frames, which would make opencv process ALL the frames (which you rightly pointed out, slows the video processing), it's better to use CAP_PROP_POS_MSEC link and offload that processing to cv2. By using this option you can configure cv2 to sample 1 frame every nth Millisecond. So setting subsample_rate=1000 in vidcap.set(cv2.CAP_PROP_POS_MSEC, (frame_count * subsample_rate)) would sample 1 frame every 1 second (as 1000 milliseconds equals 1 second). Hopefully this improves your video processing speed.
def extractImagesFromVideo(path_in, subsample_rate, path_out, saveImage, resize=(), debug=False):
vidcap = cv2.VideoCapture(path_in)
if not vidcap.isOpened():
raise IOError
if debug:
length = int(vidcap.get(cv2.cv2.CAP_PROP_FRAME_COUNT))
width = int(vidcap.get(cv2.cv2.CAP_PROP_FRAME_WIDTH))
height = int(vidcap.get(cv2.cv2.CAP_PROP_FRAME_HEIGHT))
fps = vidcap.get(cv2.cv2.CAP_PROP_FPS)
print 'Length: %.2f | Width: %.2f | Height: %.2f | Fps: %.2f' % (length, width, height, fps)
success, image = vidcap.read() #extract first frame.
frame_count = 0
while success:
vidcap.set(cv2.CAP_PROP_POS_MSEC, (frame_count*subsample_rate))
success, image = vidcap.read()
if saveImage and np.any(image):
cv2.imwrite(os.path.join(path_out, "%s_%d.png" % (frame_count)), image)
frame_count = frame_count + 1
return frame_count
There seems to be something strange going on in this loop. I tried to debug it, and I have no explanation for what is going on.
This code is from another Stack Overflow post, and I made some edits to it.
modelImages = ['/home/lie/Desktop/dylan.jpg','/home/lie/Desktop/melissa.jpg']
for modelImage in modelImages:
**print modelImage**
template=cv2.imread(modelImage)
templateg = cv2.cvtColor(template,cv2.COLOR_BGR2GRAY)
keys = surf.detect(templateg)
keys,desc = surfDescriptorExtractor.compute(templateg, keys)
count = 0
for h,des in enumerate(desc):
des = np.array(des,np.float32).reshape((1,64))
retval, results, neigh_resp, dists = knn.find_nearest(des,1)
res,dist = int(results[0][0]),dists[0][0]
if dist<0.1: # draw matched keypoints in red color
count=count + 1
print "space"**
The important parts have asterisks. This is a portion of the code that was suppose to identify similarities among images of faces. What the code does is not important. The strange part is that this loop is executing 1 time for an array of size two.
The output is:
/home/lie/Desktop/dylan.jpg
/home/lie/Desktop/melissa.jpg
space
Notice that both strings in modelImages are printed before space. By the way this is part of a function that is called from a loop. This seems to be more of a python issue than an opencv issue. It almost seems like there is a hidden continue statment
Thanks