I have dataset of 1000 points on a plane. I represented all the possible pairs of points in P and calculated the distances of all possible pairs.
What I have to do is: For a given n, calculate all the n-th nearest points for all points p in P.
What I did before:
P_pairs = [((33, 9), (34, 13)), ((33, 9), (62, 119)), ((33, 9), (33, 7)), ((33, 9), (48, 123)), ...]
listofdistances = [{'((33, 9), (34, 13))': 4.123105625617661}, {'((33, 9), (62, 119))': 113.75851616472501}, {'((33, 9), (33, 7))': 2.0}, ...]
In this context, I am stuck in sorting listofdistances such that for every point, there are the minimum n distances as values left.
Maybe I have to calculate the n-th nearest points directly, instead of calculating all the distances of the points. But I don't exactly know how.
P = [(33, 9), (34, 13), (62, 119), (33, 7), (48, 123)]
P = np.array(P)
x, y = P[:,0], P[:,1]
# Create a distance table of point (row) vs point (column)
dist = np.sqrt((x - x[:,None])**2 + (y - y[:,None])**2)
# The diagonals are 0, as the distance of a point to itself is 0,
# but we want that to have a large value so it comes last in sorting
np.fill_diagonal(dist, np.inf)
# Get the sorted index for each row
idx = dist.argsort(axis=1)
Now if you want the nth nearest neighbours, with n = 3, you get that with idx = idx[:,:3]. And for the first point you can now do
P[0] # the point itself
P[idx[0]] # its nearest neighbours
dist[0,idx[0]] # their distances
Creating a list of all possible pairs and then a list of single key dictionaries with distances as values does create a sorting headache. I would instead vectorize this job and use numpy.
import numpy as np
P = np.array([(33, 9), (34, 13), (62, 119), ...])
# Finds the n closest points to p in P
def n_closest_points(p, P, n)
p_vector = np.tile(p, (len(P), 1))
dists = np.linalg.norm(P-p_vector, axis=1)
sorted_dists = np.sort(dists)
# Exclude the 0th element as the distance from p to itself is 0
return sorted_dists[1:n+1]
Related
Disclaimer: This question is regarding OpenPose but the key here is actually to figure how to use the output (coordinates stored in the JSON) and not how to use OpenPose, so please consider reading it to the end.
I have a video of a person from the side on a bike (profile of him sitting so we see the right side). I use the OpenPose to extract the coordinates of the skeleton. The OpenPose provides the coordinates in a JSON file looking like (see docs for explanation):
{
"version": 1.3,
"people": [
{
"person_id": [
-1
],
"pose_keypoints_2d": [
594.071,
214.017,
0.917187,
523.639,
216.025,
0.797579,
519.661,
212.063,
0.856948,
539.251,
294.394,
0.873084,
619.546,
304.215,
0.897219,
531.424,
221.854,
0.694434,
550.986,
310.036,
0.787151,
625.477,
339.436,
0.845077,
423.656,
319.878,
0.660646,
404.111,
321.807,
0.650697,
484.434,
437.41,
0.85125,
404.13,
556.854,
0.791542,
443.261,
319.801,
0.601241,
541.241,
370.793,
0.921286,
502.02,
494.141,
0.799306,
592.138,
198.429,
0.943879,
0,
0,
0,
562.742,
182.698,
0.914112,
0,
0,
0,
537.25,
504.024,
0.530087,
535.323,
500.073,
0.526998,
486.351,
500.042,
0.615485,
449.168,
594.093,
0.700363,
431.482,
594.156,
0.693443,
386.46,
560.803,
0.803862
],
"face_keypoints_2d": [],
"hand_left_keypoints_2d": [],
"hand_right_keypoints_2d": [],
"pose_keypoints_3d": [],
"face_keypoints_3d": [],
"hand_left_keypoints_3d": [],
"hand_right_keypoints_3d": []
}
]
}
From what I understand, each JSON is a frame of the video.
My goal is to detect the angles of specific coordinates like right knee, right arm, etc. For example:
openpose_angles = [(9, 10, 11, "right_knee"),
(2, 3, 4, "right_arm")]
This is based on the following OpenPose skeleton dummy:
What I did is to calculate the angle between three coordinates (using Python):
temp_df = json.load(open(os.path.join(jsons_dir, file)))
listPoints = list(zip(*[iter(temp_df['people'][person_number]['pose_keypoints_2d'])] * 3))
count = 0
lmList2 = {}
for x,y,c in listPoints:
lmList2[count]=(x,y,c)
count+=1
p1=angle_cords[0]
p2=angle_cords[1]
p3=angle_cords[2]
x1, y1 ,c1= lmList2[p1]
x2, y2, c2 = lmList2[p2]
x3, y3, c3 = lmList2[p3]
# Calculate the angle
angle = math.degrees(math.atan2(y3 - y2, x3 - x2) -
math.atan2(y1 - y2, x1 - x2))
if angle < 0:
angle += 360
This method I saw on some blog (which I forgot where), but was related to OpenCV instead of OpenPose (not sure if makes the difference), but see angles that do not make sense. We showed it to our teach and he suggested us to use vectors to calculate the angles, instead of using math.atan2. But we got confued on how to implment this.
To summarize, here is the question - What will be the best way to calculate the angles? How to calculate them using vectors?
Your teacher is right. I suspect the problem is that 3 points can make up 3 different angles depending on the order. Just consider the angles in a triangle. Also you seem to ignore the 3rd coordinate.
Reconstruct the Skeleton
In your picture you indicate that the edges/bones of the skeleton are
edges = {(0, 1), (0, 15), (0, 16), (1, 2), (1, 5), (1, 8), (2, 3), (3, 4), (5, 6), (6, 7), (8, 9), (8, 12), (9, 10), (10, 11), (11, 22), (11, 24), (12, 13), (13, 14), (14, 19), (14, 21), (15, 17), (16, 18), (19, 20), (22, 23)}
I get the points from your json file with
np.array(pose['people'][0]['pose_keypoints_2d']).reshape(-1,3)
Now I plot that ignoring the 3rd component to get an idea what I am working with. Notice that this does not change the proportions much since the 3rd component is really small compared to the others.
One definitely recognizes an upside down man. I notice that there seems to be some kind of artifact but I suspect this is just an error in recognition and would be better in an other frame.
Calculate the Angle
Recall that the dot product divided by the product of the norm gives the cosine of the angle. See the wikipedia article on dot product. I'll include the relevant picture from that article. So now I can get the angle of two joined edges like this.
def get_angle(edge1, edge2):
assert tuple(sorted(edge1)) in edges
assert tuple(sorted(edge2)) in edges
edge1 = set(edge1)
edge2 = set(edge2)
mid_point = edge1.intersection(edge2).pop()
a = (edge1-edge2).pop()
b = (edge2-edge1).pop()
v1 = points[mid_point]-points[a]
v2 = points[mid_point]-points[b]
angle = (math.degrees(np.arccos(np.dot(v1,v2)
/(np.linalg.norm(v1)*np.linalg.norm(v2)))))
return angle
For example if you wanted the elbow angles you could do
get_angle((3, 4), (2, 3))
get_angle((5, 6), (6, 7))
giving you
110.35748420197164
124.04586139643376
Which to me makes sense when looking at my picture of the skeleton. It's a bit more than a right angle.
What if I had to calculate the angle between two vectors that do not share one point?
In that case you have to be more careful because in that case the vectors orientation matters. Firstly here is the code
def get_oriented_angle(edge1, edge2):
assert tuple(sorted(edge1)) in edges
assert tuple(sorted(edge2)) in edges
v1 = points[edge1[0]]-points[edge1[1]]
v2 = points[edge2[0]]-points[edge2[1]]
angle = (math.degrees(np.arccos(np.dot(v1,v2)
/(np.linalg.norm(v1)*np.linalg.norm(v2)))))
return angle
As you can see the code is much easier because I don't order the points for you. But it is dangerous since there are two angles between two vectors (if you don't consider their orientation). Make sure both vectors point in the direction of the points you're considering the angle at (both in the opposite direction works too).
Here is the same example as above
get_oriented_angle((3, 4), (2, 3)) -> 69.64251579802836
As you can see this does not agree with get_angle((3, 4), (2, 3))! If you want the same result you have to put the 3 first (or last) in both cases.
If you do
get_oriented_angle((3, 4), (3, 2)) -> 110.35748420197164
It is the same angle as above.
I have 9 points (longitudes, latitudes in degrees) on the surface of Earth follows.
XY = [(100, 10), (100, 11), (100, 13), (101, 10), (101, 11), (101, 13), (103, 10), (103, 11), (103, 13)]
print (len(XY))
# 9
I wanted to extract those points which are at least 3 degrees far from each other.
I tried it as follows.
results = []
for point in XY:
x1,y1 = point
for result in results:
x2,y2 = result
distance = math.hypot(x2 - x1, y2 - y1)
if distance >= 3:
results.append(point)
print (results)
But output is empty.
edit 2
from sklearn.metrics.pairwise import haversine_distances
from math import radians
results = []
for point in XY:
x1,y1 = [radians(_) for _ in point]
for result in results:
distance = haversine_distances((x1,y1), (x2,y2))
print (distance)
if distance >= 3:
results.append(point)
print (results)
Still the result is empty
edit 3
results = []
for point in XY:
x1,y1 = point
for point in XY:
x2,y2 = point
distance = math.hypot(x2 - x1, y2 - y1)
print (distance)
if distance >= 3:
results.append(point)
print (results)
print (len(results))
# 32 # unexpected len
Important: You've said you want to "Extract those points which are at least 3 degrees far from each other" but then you've used the Euclidean distance with math.hypot(). As mentioned by #martineau, this should use the Haversine angular distance.
Since your points are "(longitudes, latitudes in degrees)", they first need to be converted to radians. And the pairs should be flipped so that latitude comes first, as required by the haversine_distances() function. That can be done with:
XY_r = [(math.radians(lat), math.radians(lon)) for lon, lat in XY]
Here's the kicker - none of the combnation-making or looping is necesssary. If haversine_distances() is passed in a list of points, it will calculate the distances between all of them and provide a result as an array of arrays. These can then be converted back to degrees and checked; or convert 3 degrees to radians and then check against h-dists.
import math
import numpy as np
from sklearn.metrics.pairwise import haversine_distances
XY = [(100, 10), (100, 11), (100, 13), (101, 10), (101, 11), (101, 13), (103, 10), (103, 11), (103, 13)]
# convert to radians and flip so that latitude is first
XY_r = [(math.radians(lat), math.radians(lon)) for lon, lat in XY]
distances = haversine_distances(XY_r) # distances array-of-arrays in RADIANS
dist_criteria = distances >= math.radians(3) # at least 3 degrees (in radians) away
results = [point for point, result in zip(XY, dist_criteria) if np.any(result)]
print(results)
print(len(results))
print('<3 away from all:', set(XY) - set(results))
Output:
[(100, 10), (100, 11), (100, 13), (101, 10), (101, 13), (103, 10), (103, 11), (103, 13)]
8
<3 away from all: {(101, 11)}
Wrt the previous edit and your original code:
Your first two attempts are giving empty results because of this:
results = []
for point in XY:
...
for result in results:
results is initialised as an empty list. So the for result in results loop will directly exit. Nothing inside the loop executes.
The 3rd attempt is getting you 32 results because of repetitions. You've got:
for point in XY:
...
for point in XY:
so some points you get will be the same point.
To avoid duplicates in the loops:
Add a check for it and go to the next iteration:
if (x1, y1) == (x2, y2):
continue
Btw, you're mangling the point variable because it's reused in both loops. It doesn't cause a problem but makes your code harder to debug. Either make them point1 and point2, or even better, instead of for point in XY: x1, y1 = point, you can directly do for x1, y1 in XY - that's called tuple unpacking.
for x1, y1 in XY:
for x2, y2 in XY:
if (x1, y1) == (x2, y2):
continue
...
You also need to change result to be a set instead of a list so that the same point is not re-added to the results when it's more than 3 away from another point. Sets don't allow duplicates, that way points don't get repeated in results.
Use itertools.combinations() to get unique pairs of points without repetitions. This allows you to skip the duplicate check (unless XY actually has duplicate points) and brings the previous block down to one for-loop:
import itertools
import math
results = set() # unique results
for (x1, y1), (x2, y2) in itertools.combinations(XY, r=2):
distance = math.hypot(x2 - x1, y2 - y1) # WRONG! see above
if distance >= 3:
# add both points
results.update({(x1, y1), (x2, y2)})
print(results)
print(len(results))
print('<3 away from all:', set(XY) - results)
The (wrong) output:
{(103, 11), (100, 13), (101, 13), (100, 10), (103, 10), (101, 10), (103, 13), (100, 11)}
8
<3 away from all: {(101, 11)}
(The result is the same but merely by coincidence of the input data.)
I have a face-vertex mesh like the following image has.
Face-Vertex Mesh
I have 'face_list' and 'vertex_list' data sets, but not sure how to efficiently calculate a list of edges.
Compute the normal vectors of the faces. If the normal vectors of 2 adjacent faces point in different directions, the two vertices shared by the face form an edge.
The normal vectors of a face can be computed with the Cross product. For a face with the vertices A, B, C, the unit normal vector is:
N = normalized(cross(B-A, C-A))
The normal vectors of the faces can be compared with the Dot product. 2 normal vectors N1 and N2 are equal directed if:
equally_directed = abs(dot(N1, N2)) == 1
Use a library for the vector arithmetic. For example OpenGL Mathematics (GLM) library for Python or NumPy.
Minimal example:
import glm, math
vertices = [(-1,-1,-1), ( 1,-1,-1), ( 1, 1,-1), (-1, 1,-1), (-1,-1, 1), ( 1,-1, 1), ( 1, 1, 1), (-1, 1, 1)]
faces = [(0,1,2), (0,2,3), (5,4,7), (5,7,6), (4,0,3), (4,3,7), (1,5,6), (1,6,2), (4,5,1), (4,1,0), (3,2,6), (3,6,7)]
v = [glm.vec3(vertex) for vertex in vertices]
nv_list = [glm.normalize(glm.cross(v[i1]-v[i0], v[i2]-v[i0])) for i0,i1,i2 in faces]
edge_threshold = 0.99
edges = []
for i, f1 in enumerate(faces):
for j, f2 in enumerate(faces[i+1:]):
edge_candidates = [(f1[0], f1[1]), (f1[1], f1[2]), (f1[2], f1[0])]
for ei0, ei1 in edge_candidates:
if ei0 in f2 and ei1 in f2:
cos_nv = math.fabs(glm.dot(nv_list[i], nv_list[j+i+1]))
if abs(cos_nv) < edge_threshold:
edges.append((ei0, ei1))
print(len(edges))
print(edges)
Output:
12
[(1, 2), (0, 1), (3, 0), (2, 3), (4, 7), (5, 4), (6, 5), (7, 6), (4, 0), (3, 7), (1, 5), (6, 2)]
I have a list of tuples having coordinates in (x,y) format. I want to sort/arrange it in counter-clockwise direction. ex:
[(0,1),(3,1),(-1,0),(2,2)]
The arranged list should be:
[(3,1),(2,2),(0,1),(-1,0)]
Note: The list can have 'n' of tuples and (0,0) can be a part of list.
You could use the 2-argument arctangent to compute the angle from (1, 0) and use that to sort:
>>> vec = [(0,1),(3,1),(-1,0),(2,2)]
>>> sorted(vec, key=lambda p: math.atan2(p[1], p[0])) # atan2(y, x)
[(3, 1), (2, 2), (0, 1), (-1, 0)]
(Image courtesy of Wikipedia.)
I'm currently trying to implement the Fourth Nelson rule from:
https://en.wikipedia.org/wiki/Nelson_rules
I.e. given a list of numbers of length N, I want to know if there exists a consecutive sequence of numbers that are alternating in direction of length n. 'Alternating' means consecutive numbers go up, then down, then up, etc.
My data is in (t,x) tuples. 't' stands for the time axis, always increasing. 'x' is the value associated with the time and the series we are concerned with. For example:
data = [(0, 2.5), (1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
Here, the alternating x value sequence is for everything but the first tuple. See the below graph:
The alternating sequence is highlighted in red. The rule looks for 14 points in a row, but I want to generalize this to n-points in a row. (n < N) I can't just output True or False, I want to output the tuple of points that satisfy the condition. In other words, the output would be:
outliers = [(1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
I've tried a few things, none of which resulted in the desired output. These included things like np.diff() and np.sign(). I have a feeling itertools() can do this, but I can't quite get there.
Any input is greatly appreciated.
Here's a first cut at your algorithm in straight Python:
data = [(0, 2.5), (1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
n = 5
t0, x0 = data.pop(0)
outliers = []
up = bool(x0 > 0)
for t, x in data:
if (x < x0 and up) or (x > x0 and not up):
if not outliers:
outliers = [(t0,x0)]
outliers.append((t,x))
up = not up
else:
if len(outliers) >= n:
print 'outliers =', outliers
outliers = []
t0,x0 = t,x
if len(outliers) >= n:
print 'outliers =', outliers