Lucas Kanade: How to calculate distance between tracked points - python

I'm using lucas-kanade opencv implementation to track objects between frames. I want to be able to do the following two things:
Calculate the distance moved by each point between frames
Track bounding boxes for each object across frames
I have obtained the features to track using cv2.goodFeaturesToTrack(). I also add the bounding boxes of objects to the features to be tracked. Right now I am using the following to calculate distance between the points
np.sqrt(np.square(new_pts - old_pts).sum(axis=1).sum(axis=1)). I am not quite sure if this is the correct way to do this because the indices of the points might be different in the new_pts.
Is the assumption that every index in old_pts corresponds to the same feature in new_pts array correct?
Secondly, is there a way to track bounding boxes across frames using lucas kanade?

In new_pts points have the same index. But they can be not founded - see to the status array: if status[i] == 1 then new_pts[i] contains a new coordinates of the old_pts[i].
For the more robustness it can to search direct flow (goodFeaturesToTrack(frame1) -> LK flow), backward flow (goodFeaturesToTrack(frame2) -> LK flow) and leave the points whose coordinates are equal in both directions.

Related

Measure distance between meshes

For my project, I need to measure the distance between two STL files. I wrote a script that allows reading the files, positioning them in relation to each other in the desired position. Now, in the next step I need to check the distance from one object to the other. Is there a function or script available on a library that allows me to carry out this process? Because then I’m going to want to define metrics like interpenetration area, maximum negative distance etc etc so I need to check first the distance between those objects and see if there is like mesh intersection and mesure that distance. I put the url for the combination of the 2 objects that I want to mesure the distance:
https://imgur.com/wgNaalh
Pyvista offers a really easy way of calculating just that:
import pyvista as pv
import numpy as np
mesh_1 = pv.read(**path to mesh 1**)
mesh_2 = pv.read(**path to mesh 2**)
closest_cells, closest_points = mesh_2.find_closest_cell(mesh_1.points, return_closest_point=True)
d_exact = np.linalg.norm(mesh_1 .points - closest_points, axis=1)
print(f'mean distance is: {np.mean(d_exact)}')
For more methods and examples, have a look at:
https://docs.pyvista.org/examples/01-filter/distance-between-surfaces.html#using-pyvista-filter
To calculate the distance between two meshes, first one needs to check whether these meshes intersect. If not, then the resulting distance can be computed as the distance between two closest points, one from each mesh (as on the picture below).
If the meshes do intersect, then it is necessary to find the part of each mesh, which is inside the other mesh, then find two most distant points, one from each inner part. The distance between these points will be the maximum deepness of the meshes interpenetration. It can be returned with negative sign to distinguish it from the distance between separated meshes.
In Python, one can use MeshLib library and findSignedDistance function from it as follows:
import meshlib.mrmeshpy as mr
mesh1 = mr.loadMesh("Cube.stl")
mesh2 = mr.loadMesh("Torus.stl"))
z = mr.findSignedDistance(mesh1, mesh2)
print(z.signedDist) // 0.3624192774295807

An algorithm that efficiently computes the distance of one labeled pixel to its nearest differently labeled pixel

I apologize for my lengthy title name. I have two questions, where the second question is based on the first one.
(1). Suppose I have a matrix, whose entries are either 0 or 1. Now, I pick an arbitrary 0 entry. Is there an efficient algorithm that searches the nearest entry with label 1 or calculates the distance between the chosen 0 entry and its nearest entry with label 1?
(2). Suppose now the distribution of entries 1 has a geometric property. To make this statement clearer, imagine this matrix as an image. In this image, there are multiple continuous lines (not necessarily straight). These lines form several boundaries that partition the image into smaller pieces. Assume the boundaries are labeled 1, whereas all the pixels in the partitioned area are labeled 0. Now, similar to (1), I pick a random pixel labeled as 0, and I hope to find out the coordinate of the nearest pixel labeled as 1 or the distance between them.
A hint/direction for part (1) is enough for me. If typing up an answer takes too much time, it is okay just to tell me the name of the algorithm, and I will look it up.
p.s.: If I post this question in an incorrect section, please let me know. I will re-post it to an appropriate section. Thank you!
I think that if you have a matrix, you can run a BFS version where the matrix A will be your graph G and the vertex v will be the arbitrary pixel you chose.
There is an edge between any two adjacent cells in the matrix.

how to optimize performances of geometry operations

I am looking for an approach to optimize performances of geometry operations. My goal is to count how many points (205,779) within a series of polygons (21,562). Using python and R are preferable as well as GIS software, like ArcGIS, QGIS.
Here are solutions I have searched and written.
using ArcGIS: one of examples is in http://support.esri.com/cn/knowledgebase/techarticles/detail/30779 -> although I did not try it, it always take a large amount of time in spatial join, based on my previous experiences.
using GDAL, OGR: Here is an example: http://geoexamples.blogspot.tw/2012/06/density-maps-using-gdalogr-python.html -> It takes 5 to 9 seconds for every polygon.
using Shapely prepared geometry operations with a loop: Here is my example, and it takes 2.7 to 3.0 seconds for every polygon. (Note that points is Point objects in a list)
prep_poly=[]
for i in polygons:
mycount=[]
for j in points:
if prep(i).contains(j):
mycount.append(1) #count how many points within polygons
prep_poly.append(sum(mycount)) #sum once for every polygon
mycount=[]
using Shapely prepared geometry operations with a filter: Here is my example, and it takes about 3.3 to 3.9 seconds for every polygon.(Note that points is a MultiPoint object)
prep_poly=[]
for i in polygons:
prep_poly.append(len(filter(prep(i).contains, point1)))
Though prepared geometry operations did improve the performances, it is still time-consuming to process lots of polygons. Any suggestion? Thanks!
Rather than looking through every pixel on the screen for every rectangle, you can do the following (Python code):
first_pixel = any pixel in the polygon
px_list = [] # array with pixels left to check
px_list.append(first_pixel) # add pixel to list of pixels to process
count = 0
while len(array) > 0: # pixels left in pixel list
curr_pixel = array[0]
for pixel in get_adjacent_pixels(curr_pixel): # find adjacent pixels
# ie (vertical, horizontal, diagonal)
if pixel in shape:
px_list.append(pixel) # add pixel to list
px_list.remove(curr_pixel)
count += 1
Essentially, the same way that path finding works. Check this wiki article for a visual representation of the above algorithm:
http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm#Algorithm
If you have no easy way to find starting points you could loop through all of the points once, checking for each point whether it is contained by a shape, and then storing that point together with the shape in a separate list and deleting it from the original shapes-for-which-we-have-no-point-yet list.

Comparing two Multidimensional Numpy arrays

I'm working on a image analysis code for detecting motion in subsequent image snapshots. To do this, I decided to use the optical flow function in opencv that finds good points to track in an initial image and predict the points in a subsequent image.
# feature extraction of points to track
pt = cv2.goodFeaturesToTrack(img1,**features_param)
# convert points to floating-point
p0 =np.float32(pt).reshape(-1,1,2)
# get predicted points using lucas-kanade optical flow
p1,st,err =cv2.calcOpticalFlowPyrLK(img1, img2,p0,
None,**lk_params)
In order to find points that were predicted correctly the optical flow function is run in reverse (second image first). Then an absolute difference is calculated between initial points (tracked points) and the backward predicted (p0r), if the value is below one then that points was predicted correctly if not it is a "bad" point.
# forward-backward error detection
p0r,st,err =cv2.calcOpticalFlowPyrLK(img2,img1,p1,
None,**lk_params)
# get correctly predicted points via absolute difference
d = abs(p0-p0r).reshape(-1, 2).max(-1)
good = d < 1
Go through the predicted points p1 and find values that fit the "good" condition.
# cycle through all current and new keypoints and only keep
# those that satisfy the "good" condition above
# Initialize a list to hold new keypoints
new_keypoints = list()
# get good points
for (x, y), good_flag,ind in zip(p1.reshape(-1, 2), good,enumerate(good)):
if not good_flag:
continue
new_keypoints.append((x,y))
I need to check which original points in p0 ended up predicted in new_keypoints.
After a torturing my brain I managed to get a solution to my problem. I created a for loop that goes through each point in the numpy array (p0) and predicts the point, if meets the "good" criterion it is appended to a list otherwise it is omitted. I then proceed to calculate the euclidean distance between the point and its newly predicted position. Here's the code:
Solution
# feature extraction of points to track
pt = cv2.goodFeaturesToTrack(img1,**features_param)
p0 =np.float32(pt).reshape(-1,1,2)
pr,st,err =cv2.calcOpticalFlowPyrLK(img1, img2,p0,
None,**lk_params)
# append correctly predicted points
dist = list()
for loop in p0:
p1,st,err =cv2.calcOpticalFlowPyrLK(img1, img2,loop,
None,**lk_params)
p0r,st,err =cv2.calcOpticalFlowPyrLK(img2,img1,p1,
None,**lk_params)
# calculate euclidean distance of predicted points
if abs(loop-p0r).reshape(-1, 2).max(-1) < 1:
dst = distance.euclidean(loop,p0r)
dist.append(dst)

Measuring rectangles at odd angles with a low resolution input matrix (Linear regression classification?)

I'm trying to solve the following problem:
Given an input of, say,
0000000000000000
0011111111110000
0011111111110000
0011111111110000
0000000000000000
0000000111111110
0000000111111110
0000000000000000
I need to find the width and height of all rectangles in the field. The input is actually a single column at a time (think like a scanner moves from left to right) and is continuous for the duration of the program (that is, the scanning column doesn't move, but the rectangles move over it).
In this example, I can 'wait for a rectangle to begin' (that is, watch for zeros changing to 1s) and then watch it end (ones back to zeros) and measure the piece in 'grid units'. This will work fine for the simple case outlined above, but will fail is the rectangle is tilted at an angle, for example:
0000000000000000
0000011000000000
0000111100000000
0001111111000000
0000111111100000
0000011111110000
0000000111100000
0000000011000000
I had originally thought that the following question would apply:
Dynamic programming - Largest square block
but now i'm not so sure.
I have little to no experience with regression or regression testing, but I think that I could represent this as an input of 8 variables.....
Well to be honest i'm not sure how I would do this at all. The sizes that this part of the code extracts need to be fitted against rectangles of known sizes (ie, from a database).
I initially thought I could feed the known data as training exercises and store the positive test results, but I'm really not sure where to go from here.
Thanks for any advice you might have.
Collect the transition points (from a 1 to a 0 or vice-versa) as you're scanning, then figure the length and width either directly from there, or from the convex hull of each object.
If rectangles can overlap, then you'll have bigger issues.
I'd take following steps:
get all columns together in a matrix (this is needed for proper filtering)
now apply a filter (need to google for it a bit) to sharpen edges and corners
create some structure to hold data for next steps (this can have many different solutions, choose your favorite and/or optimal)
scan vertically (column by column) and for each segment of consequent 'ones' found in a column (segment means you have found it's start end end y coordinates) do:
check that this segment overlaps some segment in the previous column
if it does not, consider this a new rect. Create a rect object and assign it's handle to the segment. for the new rect, update it's metrics (this operation takes just the segment's coordinates - x, ymin, ymax, and will be discussed later)
if it does, assume this is the same rect, take the rect's handle, assign this handle to the current segment then get the rect by it's handle and update it's metrics
That's pretty it. After this you will have a pool of rect objects each having four coordinates of its corners. Do some primitive math to approximate rect's width and height.
So where is the magic? Well, it all happens in the update rect metrics routine.
For each rect we have 13 metrics:
min X => ymin1, ymax1
max X => ymin2, ymax2
min Y => xmin1, xmax1
max Y => xmin2, xmax2
average vertical segment length
First of all we have to determine if this rect is properly aligned within our scan grid. To do this we compare values average vertical segment length and max Y - min Y. If they are the same (i'd choose a threshold around 97%, and then tune it for the best results), then we assume the following coordinates for our rect:
(min X, max Y)
(min X, min Y)
(max X, max Y)
(max X, min Y).
In other case out rect is rotated and in this case we take it's coordinates as follows:
(min X, (ymin1+ymax1)/2)
((xmin1+xmax1)/2, min Y)
(max X, (ymin2+ymax2)/2)
((xmin2+xmax2)/2, max Y)
I posed this question to a friend, and he suggested:
When seeing a 1 for the first time, store it as a new shape. Flood fill it to the right, and add those points to the same shape.
Any input pixel that is'nt in a shape now is a new shape. Do the same flood fill.
On the next input column, flood again from the original shape points. Add new pixels to the corresponding shape
If any flood fill does not add any new pixels for two consecutive columns, you have a completed shape. Move on, and try to determine it's dimensions
This then leaves us with getting the dimensions for a shape we isolated (like in example 2).
For this, we thought up:
If the number of leftmost pixels in the shape is below the average number of pixels per column, then the peice is probably rotated. Thus, find the corners by getting the outermost pixels. Use distance formula between all of them. Largest = hypotenuse, others = width or height.
Otherwise, this peice is probably perfectly aligned, so the corners are probably just the topleft most pixel, bottom right most pixel, etc
What do you all think?

Categories

Resources