Hope you guys are doing well.
I have a question about opencv and extracting the same point from number of images.
Say, we have a dataset of around 100 images. (maybe more but for this purpose it will suffice).
Image will look something like:
As you can see in the image, there is an area marked in RED. I have marked that using Paint for this purpose. It indicates the highest point of the heap of soil. All the 100 images we have look more or less the same (without that crane in the background. But it can be removed using some opencv techniques, so that is not an issue). But this heap of soil can be either on the left hand side or the right hand side. According to its position, the coordinate of the highest point in the heap will change.
So, my question is, how to find this position given that the heap can be either on left or right side?
Note that this position can be relative to some object (for example in this image, midpoint of the crane) or if the images are of different size than we can resize the images to have same dimensions and take the coordinates of the point w.r.t the image itself.
How do we find out the highest point of the heap though? Should we manually go through each image, label that point and make a dataset with the images and bounding boxes? Or is there another decent soulution to this?
Also, if the soil heap is labelled manually (Like shading the required area i.e. heap of an image) using Paint or some other software, would that help too? But I cannot think of anything to do after this.
Thank You.
So I am sure this can be done in a better way than my answer, but here goes:
"Also, if the soil heap is labelled manually (Like shading the required area i.e. heap of an image) using Paint or some other software, would that help too? But I cannot think of anything to do after this."
In regards to that particular statement if you mark out the region of interest in a distinctive color and shape, like you have done in the above example. You can use opencv to detect that particular region of interest and its coordinates within the image.
I think the best solution is deep learning because detective always has different backgrounds. You can use Faster rcnn, or if you want speed, you can make nice detectives with a good training using Yolo algorithm. You can find Github repo easily. The mathematics of work is described in these links.
Faster RCNN https://arxiv.org/abs/1506.01497
Yolo https://pjreddie.com/darknet/yolo/
basically you can resize image. Keep aspect ratio!
def image_resize(image, width = None, height = None, inter = cv2.INTER_AREA):
dim = None
(h, w) = image.shape[:2]
if width is None and height is None:
return image
if width is None:
# calculate the ratio of the height and construct the
# dimensions
r = height / float(h)
dim = (int(w * r), height)
else:
# calculate the ratio of the width and construct the
# dimensions
r = width / float(w)
dim = (width, int(h * r))
# resize the image
resized = cv2.resize(image, dim, interpolation = inter)
# return the resized image
return resized
image = image_resize(image, height = ..What you want..)
Related
A chem student asked me for help with plotting image segmenetation:
A stationary camera takes a picture of the experimental setup every second over a period of a few minutes, so like 300 images yield.
The relevant parts in the setup are two adjacent layers of differently-colored foams observed from the side, a 2-color sandwich shrinking from both sides, basically, except one of the foams evaporates a bit faster.
I'd like to segment each of the images in the way that would let me plot both foam regions' "width" against time.
Here is a "diagram" :)
I want to go from here --> To here
Ideally, given a few hundred of such shots, in which only the widths change, I get an array of scalars back that I can plot. (Going to look like a harmonic series on either side of the x-axis)
I have a bit of python and matlab experience, but have never used OpenCV or Image Processing toolbox in matlab, or actually never dealt with any computer vision in general. Could you guys throw like a roadmap of what packages/functions to use or steps one should take and i'll take it from there?
I'm not sure how to address these things:
-selecting at which slice along the length of the slice the algorithm measures the width(i.e. if the foams are a bit uneven), although this can be ignored.
-which library to use to segment regions of the image based on their color, (some k-means shenanigans probably), and selectively store the spatial parameters of the resulting segments?
-how to iterate that above over a number of files.
Thank you kindly in advance!
Assume your Intensity will be different after converting into gray scale ( if not, just convert to other color space like HSV or LAB, then just use one of the components)
img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
First, Threshold your grayscaled input into a few bands
ret,thresh1 = cv.threshold(img,128,255,cv.THRESH_BINARY)
ret,thresh2 = cv.threshold(img,27,255,cv.THRESH_BINARY_INV)
ret,thresh3 = cv.threshold(img,77,255,cv.THRESH_TRUNC)
ret,thresh4 = cv.threshold(img,97,255,cv.THRESH_TOZERO)
ret,thresh5 = cv.threshold(img,227,255,cv.THRESH_TOZERO_INV)
The value should be tested out by your actual data. Here Im just give a example
Clean up the segmented image using median filter with a radius larger than 9. I do expect some noise. You can also use ROI here to help remove part of noise. But personally I`m lazy, I just wrote program to handle all cases and angle
threshholed_images_aftersmoothing = cv2.medianBlur(threshholed_images,9)
Each band will be corresponding to one color (layer). Now you should have N segmented image from one source. where N is the number of layers you wish to track
Second use opencv function bounding rect to find location and width/height of each Layer AKA each threshholed_images_aftersmoothing. Eg. boundingrect on each sub-segmented images.
C++: Rect boundingRect(InputArray points)
Python: cv2.boundingRect(points) → retval¶
Last, the rect have x,y, height and width property. You can use a simple sorting order to sort from top to bottom layer based on rect attribute x. Run though all vieo to obtain the x(layer id) , height vs time graph.
Rect API
Public Attributes
_Tp **height** // this is what you are looking for
_Tp width
_Tp **x** // this tells you the position of the band
_Tp y
By plot the corresponding heights (|AB| or |CD|) over time, you can obtain the graph you needed.
The more correct way is to use Kalman filter to track the position and height graph as I would expect some sort of bubble will occur and will interfere with the height of the layers.
To be honest, i didnt expect a chem student to be good at this. Haha good luck
Anything wrong you can find me here or Email me if i`m not watching stackoverflow
You can select a region of interest straight down the middle of the foams, a few pixels wide. If you stack these regions for each image it will show the shrink over time.
If for example you use 3 pixel width for the roi, the result of 300 images will be a 900 pixel wide image, where the left is the start of the experiment and the right is the end. The following image can help you understand:
Though I have not fully tested it, this code should work. Note that there must only be images in the folder you reference.
import cv2
import numpy as np
import os
# path to folder that holds the images
path = '.'
# dimensions of roi
x = 0
y = 0
w = 3
h = 100
# store references to all images
all_images = os.listdir(path)
# sort images
all_images.sort()
# create empty result array
result = np.empty([h,0,3],dtype=np.uint8)
for image in all_images:
# load image
img = cv2.imread(path+'/'+image)
# get the region of interest
roi = img[y:y+h,x:x+w]
# add the roi to previous results
result = np.hstack((result,roi))
# optinal: save result as image
# cv2.imwrite('result.png',result)
# display result - can also plot with matplotlib
cv2.imshow('Result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Update after question edit:
If the foams have different colors, your can use easily separate them by color by converting the image you hsv and using inrange (example). This creates a mask (=2D array with values from 0-255, one for each pixel) that you can use to calculate average height and extract the parameters and area of the image.
You can find a script to help you find the HSV colors for separation on this GitHub
This may be called "Region of Interest" I'm not exactly sure. But, what I'd like to do is rather easy to explain.
I have a photo that I need to align to a grid.
https://snag.gy/YaAWdg.jpg
For starters; the little text that says "here" must be 151px from the top of the screen.
Next; from "here" to position 8 of the chin must be 631px
Finally; a straight line down the middle of the picture at line 28 on the nose must be done.
If i'm not making sense please tell me to elaborate.
I have the following ideas (this is pseudo code)
It is simply to loop until the requirements are met with a resize function, a lot like brute forcing but thats all i can think of..
i.e..
while (top.x,y = 151,0)
img.top-=1 ## this puts the image one pixel above until reaching the desired positioning
while (top.x,y & eight.x,y != 631)
resize += 1 # resize by 1 pixel until the height is reached
## center the nose
image.position = nose.
Consider switching the order of your operations to prevent the need to iterate. A little bit of math and a change of perspective should do the trick:
1.) Resize the image such that the distance from "here" to the chin is 631px.
2.) Use a region of interest to crop your image so that "here" is 151px from the top of the screen.
3.) Draw your line.
EDIT:
The affine transform in OpenCV would work to morph your image into the proper fill, assuming that you have all the proper constraints defined.
If all you need to do is a simple scale... First calculate the distance between points, using something like this.
Point2f a(10,10);
Point2f b(100,100);
float euclideanDist(Point& p, Point& q) {
Point diff = p - q;
return cv::sqrt(diff.x*diff.x + diff.y*diff.y);
}
Then create a scale factor to resize your image
float scaleFactor = euclideanDist(a,b) / 631;
cv::resize(input, output, cv::Size(), scaleFactor, scaleFactor, cv::INTER_LINEAR);
Using both instances of scaleFactor will create a uniform scaling in X&Y. Using two different scale factors will scale X and Y independently.
Take a look to OpenCV's tutorial, for images with faces you can use Haar Cascades to simplify the work. (https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_objdetect/py_face_detection/py_face_detection.html#face-detection)
Otherwise, look at ROI (Region of Interest) to extract an area and apply your algorithm on it (resize or crop)
I would like to use Python (and the PIL library) to resize a picture (eventually with width > height or height > width) in a fixed size, let's say (144px * 144px).
I've already found topics about cropping the picture, but I would like to :
Find the biggest side of the picture
Do a thumbnail of the picture with biggest side = 144px and the other side = a ratio
Center the smallest side
Fill the borders of the smallest side with a transparent background so that the final picture is (144px * 144px)
Any ideas on how to do that? For the two first points I would use :
image = PIL.Image.open(pic)
image.thumbnail((144,144), PIL.Image.BILINEAR)
But then I'm blocked. Thanks!
I have a code:
def compare_frames(frame1, frame2):
# cropping ranges of two images
frame1, frame2 = similize(frame1, frame2)
sc = 0
h = numpy.zeros((300,256,3))
frame1= cv2.cvtColor(frame1,cv2.COLOR_BGR2HSV)
frame2= cv2.cvtColor(frame2,cv2.COLOR_BGR2HSV)
bins = numpy.arange(256).reshape(256,1)
color = [ (255,0,0),(0,255,0),(0,0,255) ]
for ch, col in enumerate(color):
hist_item1 = cv2.calcHist([frame1],[ch],None,[256],[0,255])
hist_item2 = cv2.calcHist([frame2],[ch],None,[256],[0,255])
cv2.normalize(hist_item1,hist_item1,0,255,cv2.NORM_MINMAX)
cv2.normalize(hist_item2,hist_item2,0,255,cv2.NORM_MINMAX)
sc = sc + (cv2.compareHist(hist_item1, hist_item2, cv2.cv.CV_COMP_CORREL)/len(color))
return sc
It works, but if image have color noise (more darken/lighten tint) it's not working and give similarity equals is 0.5. (need 0.8)
An image 2 is more darken than image 1.
Can you suggest me FAST comparison algorythm ignore light, blur, noise on images or modify that?
Note:
i have template matching algorythm too:
But it works slowly than i need although similarity is 0.95.
def match_frames(frame1, frame2):
# cropping ranges of two images
frame1, frame2 = similize(frame1, frame2)
result = cv2.matchTemplate(frame1,frame2,cv2.TM_CCOEFF_NORMED)
return numpy.amax(result)
Thanks
Your question is one of the classic ones in computer vision and image processing. Many doctoral theses have been written and scores of papers in conferences and journals.
In short direct pixel comparisons will not work in this case. A transformation of some kind is needed to take you to a different feature space. You could do something simple or complex depending on the requirements you have in mind. You could compute edges or corners. One suggestion already mentioned is the FAST corner detection. This would be a good choice as would SIFT etc... There are many others you could use but it will depend on how much the two images can vary and in what ways.
For example, if there is only going to be global color changes, tint, etc the approach would be different than if the images could be rotated or the object position changing in size (i.e. camera zoom).
Strictly speaking for the case you mention features such as FAST, SIFT, or even edges would work reasonably well. Check http://en.wikipedia.org/wiki/Feature_detection_%28computer_vision%29 for more information
Image patch descriptors (SIFT, SURF...) are usually monochromatic and expect black-and-white images. Thus, for any approach (point matching, frame matching...) I would advise you to change the color space to Lab or YUV first and then work on the luminance plane.
FAST is a (fast) corner detection algorithm. A corner is obviously insensitive to noise and contrast, but may be affected by blur (bad position, bad corner response for example). FAST does not include a descriptor part however, so your matching should then rely on geometric proximity. If you need a descriptor part, then you need to switch to one of the many other keypoint descriptors (SIFT, SURF, FAST + BRIEF/BRISK/ORB/FREAK...).
I have som PNG image links that I want to download, "convert to thumbnails" and save to PDF using Python and Cairo.
Now, I have a working code, but I don't know how to control image size on paper. Is there a way to resize a PyCairo Surface to the dimensions I want (which happens to be smaller than the original)? I want the original pixels to be "shrinked" to a higher resolution (on paper).
Also, I tried Image.rescale() function from PIL, but it gives me back a 20x20 pixel output (out of a 200x200 pixel original image, which is not the banner example on the code). What I want is a 200x200 pixel image plotted inside a 20x20 mm square on paper (instead of a 200x200 mm square as I am getting now)
My current code is:
#!/usr/bin/python
import cairo, urllib, StringIO, Image # could I do it without Image module?
paper_width = 210
paper_height = 297
margin = 20
point_to_milimeter = 72/25.4
pdfname = "out.pdf"
pdf = cairo.PDFSurface(pdfname , paper_width*point_to_milimeter, paper_height*point_to_milimeter)
cr = cairo.Context(pdf)
cr.scale(point_to_milimeter, point_to_milimeter)
f=urllib.urlopen("http://cairographics.org/cairo-banner.png")
i=StringIO.StringIO(f.read())
im=Image.open(i)
# are these StringIO operations really necessary?
imagebuffer = StringIO.StringIO()
im.save(imagebuffer, format="PNG")
imagebuffer.seek(0)
imagesurface = cairo.ImageSurface.create_from_png(imagebuffer)
### EDIT: best answer from Jeremy, and an alternate answer from mine:
best_answer = True # put false to use my own alternate answer
if best_answer:
cr.save()
cr.scale(0.5, 0.5)
cr.set_source_surface(imagesurface, margin, margin)
cr.paint()
cr.restore()
else:
cr.set_source_surface(imagesurface, margin, margin)
pattern = cr.get_source()
scalematrix = cairo.Matrix() # this can also be used to shear, rotate, etc.
scalematrix.scale(2,2) # matrix numbers seem to be the opposite - the greater the number, the smaller the source
scalematrix.translate(-margin,-margin) # this is necessary, don't ask me why - negative values!!
pattern.set_matrix(scalematrix)
cr.paint()
pdf.show_page()
Note that the beautiful Cairo banner does not even fit the page...
The ideal result would be that I could control the width and height of this image in user space units (milimeters, in this case), to create a nice header image, for example.
Thanks for reading and for any help or comment!!
Try scaling the context when you draw the image.
E.g.
cr.save() # push a new context onto the stack
cr.scale(0.5, 0.5) # scale the context by (x, y)
cr.set_source_surface(imagesurface, margin, margin)
cr.paint()
cr.restore() # pop the context
See: http://cairographics.org/documentation/pycairo/2/reference/context.html for more details.
This is not answering the question, I just wanted to share heltonbiker's current code edited to run with Python 3.2:
import cairo, urllib.request, io
from PIL import Image
paper_width = 210
paper_height = 297
margin = 20
point_to_millimeter = 72/25.4
pdfname = "out.pdf"
pdf = cairo.PDFSurface( pdfname,
paper_width*point_to_millimeter,
paper_height*point_to_millimeter
)
cr = cairo.Context(pdf)
cr.scale(point_to_millimeter, point_to_millimeter)
# load image
f = urllib.request.urlopen("http://cairographics.org/cairo-banner.png")
i = io.BytesIO(f.read())
im = Image.open(i)
imagebuffer = io.BytesIO()
im.save(imagebuffer, format="PNG")
imagebuffer.seek(0)
imagesurface = cairo.ImageSurface.create_from_png(imagebuffer)
cr.save()
cr.scale(0.5, 0.5)
cr.set_source_surface(imagesurface, margin, margin)
cr.paint()
cr.restore()
pdf.show_page()
Jeremy Flores solved my problem very well by scaling the target surface before setting the imagesurface as source. Even though, perhaps some day you actually NEED to resize a Surface (or transform it in any way), so I will briefly describe the rationale used in my alternate answer (already included in the question), deduced after thoroughly reading the docs:
Set your surface as the context's source - it implicitly creates a cairo.Pattern!!
Use Context.get_source() to get the pattern back;
Create a cairo.Matrix;
Apply this matrix (with all its transforms) to the pattern;
Paint!
The only problem seems to be the transformations working always around the origin, so that scaling and rotation must be preceeded and followed by complementary translations to the origin (bleargh).