Disclaimer: This question is regarding OpenPose but the key here is actually to figure how to use the output (coordinates stored in the JSON) and not how to use OpenPose, so please consider reading it to the end.
I have a video of a person from the side on a bike (profile of him sitting so we see the right side). I use the OpenPose to extract the coordinates of the skeleton. The OpenPose provides the coordinates in a JSON file looking like (see docs for explanation):
{
"version": 1.3,
"people": [
{
"person_id": [
-1
],
"pose_keypoints_2d": [
594.071,
214.017,
0.917187,
523.639,
216.025,
0.797579,
519.661,
212.063,
0.856948,
539.251,
294.394,
0.873084,
619.546,
304.215,
0.897219,
531.424,
221.854,
0.694434,
550.986,
310.036,
0.787151,
625.477,
339.436,
0.845077,
423.656,
319.878,
0.660646,
404.111,
321.807,
0.650697,
484.434,
437.41,
0.85125,
404.13,
556.854,
0.791542,
443.261,
319.801,
0.601241,
541.241,
370.793,
0.921286,
502.02,
494.141,
0.799306,
592.138,
198.429,
0.943879,
0,
0,
0,
562.742,
182.698,
0.914112,
0,
0,
0,
537.25,
504.024,
0.530087,
535.323,
500.073,
0.526998,
486.351,
500.042,
0.615485,
449.168,
594.093,
0.700363,
431.482,
594.156,
0.693443,
386.46,
560.803,
0.803862
],
"face_keypoints_2d": [],
"hand_left_keypoints_2d": [],
"hand_right_keypoints_2d": [],
"pose_keypoints_3d": [],
"face_keypoints_3d": [],
"hand_left_keypoints_3d": [],
"hand_right_keypoints_3d": []
}
]
}
From what I understand, each JSON is a frame of the video.
My goal is to detect the angles of specific coordinates like right knee, right arm, etc. For example:
openpose_angles = [(9, 10, 11, "right_knee"),
(2, 3, 4, "right_arm")]
This is based on the following OpenPose skeleton dummy:
What I did is to calculate the angle between three coordinates (using Python):
temp_df = json.load(open(os.path.join(jsons_dir, file)))
listPoints = list(zip(*[iter(temp_df['people'][person_number]['pose_keypoints_2d'])] * 3))
count = 0
lmList2 = {}
for x,y,c in listPoints:
lmList2[count]=(x,y,c)
count+=1
p1=angle_cords[0]
p2=angle_cords[1]
p3=angle_cords[2]
x1, y1 ,c1= lmList2[p1]
x2, y2, c2 = lmList2[p2]
x3, y3, c3 = lmList2[p3]
# Calculate the angle
angle = math.degrees(math.atan2(y3 - y2, x3 - x2) -
math.atan2(y1 - y2, x1 - x2))
if angle < 0:
angle += 360
This method I saw on some blog (which I forgot where), but was related to OpenCV instead of OpenPose (not sure if makes the difference), but see angles that do not make sense. We showed it to our teach and he suggested us to use vectors to calculate the angles, instead of using math.atan2. But we got confued on how to implment this.
To summarize, here is the question - What will be the best way to calculate the angles? How to calculate them using vectors?
Your teacher is right. I suspect the problem is that 3 points can make up 3 different angles depending on the order. Just consider the angles in a triangle. Also you seem to ignore the 3rd coordinate.
Reconstruct the Skeleton
In your picture you indicate that the edges/bones of the skeleton are
edges = {(0, 1), (0, 15), (0, 16), (1, 2), (1, 5), (1, 8), (2, 3), (3, 4), (5, 6), (6, 7), (8, 9), (8, 12), (9, 10), (10, 11), (11, 22), (11, 24), (12, 13), (13, 14), (14, 19), (14, 21), (15, 17), (16, 18), (19, 20), (22, 23)}
I get the points from your json file with
np.array(pose['people'][0]['pose_keypoints_2d']).reshape(-1,3)
Now I plot that ignoring the 3rd component to get an idea what I am working with. Notice that this does not change the proportions much since the 3rd component is really small compared to the others.
One definitely recognizes an upside down man. I notice that there seems to be some kind of artifact but I suspect this is just an error in recognition and would be better in an other frame.
Calculate the Angle
Recall that the dot product divided by the product of the norm gives the cosine of the angle. See the wikipedia article on dot product. I'll include the relevant picture from that article. So now I can get the angle of two joined edges like this.
def get_angle(edge1, edge2):
assert tuple(sorted(edge1)) in edges
assert tuple(sorted(edge2)) in edges
edge1 = set(edge1)
edge2 = set(edge2)
mid_point = edge1.intersection(edge2).pop()
a = (edge1-edge2).pop()
b = (edge2-edge1).pop()
v1 = points[mid_point]-points[a]
v2 = points[mid_point]-points[b]
angle = (math.degrees(np.arccos(np.dot(v1,v2)
/(np.linalg.norm(v1)*np.linalg.norm(v2)))))
return angle
For example if you wanted the elbow angles you could do
get_angle((3, 4), (2, 3))
get_angle((5, 6), (6, 7))
giving you
110.35748420197164
124.04586139643376
Which to me makes sense when looking at my picture of the skeleton. It's a bit more than a right angle.
What if I had to calculate the angle between two vectors that do not share one point?
In that case you have to be more careful because in that case the vectors orientation matters. Firstly here is the code
def get_oriented_angle(edge1, edge2):
assert tuple(sorted(edge1)) in edges
assert tuple(sorted(edge2)) in edges
v1 = points[edge1[0]]-points[edge1[1]]
v2 = points[edge2[0]]-points[edge2[1]]
angle = (math.degrees(np.arccos(np.dot(v1,v2)
/(np.linalg.norm(v1)*np.linalg.norm(v2)))))
return angle
As you can see the code is much easier because I don't order the points for you. But it is dangerous since there are two angles between two vectors (if you don't consider their orientation). Make sure both vectors point in the direction of the points you're considering the angle at (both in the opposite direction works too).
Here is the same example as above
get_oriented_angle((3, 4), (2, 3)) -> 69.64251579802836
As you can see this does not agree with get_angle((3, 4), (2, 3))! If you want the same result you have to put the 3 first (or last) in both cases.
If you do
get_oriented_angle((3, 4), (3, 2)) -> 110.35748420197164
It is the same angle as above.
Related
I've build a graph with networkx, that looks like this: Graph
I want to fill every singel cell with a specified color. The Graph was drawn by nx.draw_networkx_edges() (returns a LineCollection). I found a similar question here (Fill area between lines), but the solution in the comments, doesn't worked for me.
I've also used plt.fill_between with a simpler graph and manually set the values:
plt.fill_between([1, 2], [2, 2], color='yellow')
plt.fill_between([1.75, 2, 3], [1.25, 2, 2], color='purple')
plt.fill_between([0, 1, 1.25], [2, 2, 1.25], color='red')
plt.fill_between([0.75, 1.25, 1.75, 2.25], [0.75, 1.25, 1.25, 0.75], color='blue')
plt.fill_between([2, 2.25, 3], [0, 0.75, 1], color='pink')
plt.fill_between([0, 0.75, 1], [1, 0.75, 0], color='green')
And it turns out pretty good (result), but the problem with that is, that the filling depends on the order when the cells get filled and that would make the algorithm for it way to complicated, I guess.
Does anyone knows a better and simpler solution?
Edit:
I tried to convert the Graph into a Voronoi-Diagram to try the solution of #JohanC, but the runtime is pretty long and the solution for larger graphs isn't exact. For the calculation of the centroids I used this Center of Polygon
def find_Centroid(v):
sum_A = 0
sum_x = 0
sum_y = 0
for i in range(len(v)):
next = i+1 if i != len(v)-1 else 0
sum_A += v[i][0]*v[next][1] - v[next][0]*v[i][1]
sum_x += (v[i][0] + v[next][0]) * (v[i][0]*v[next][1] - v[next][0]*v[i][1])
sum_y += (v[i][1] + v[next][1]) * (v[i][0]*v[next][1] - v[next][0]*v[i][1])
A = 1/2 * sum_A
Cx = 1/(6*A) * sum_x
Cy = 1/(6*A) * sum_y
return Cx, Cy
# Get all cells of Graph (I think that takes most of the time)
cycle = nx.minimum_cycle_basis(SVG)
centroids = list()
# calculate all centroids of the cells
for c in cycle:
subG = SVG.subgraph(c)
sortedCycle = sortGraphNodes(subG)
centroid = find_Centroid(sortedCycle)
SVG.add_node((centroid[0], centroid[1]))
centroids.append(centroid)
vor = Voronoi(centroids)
voronoi_plot_2d(vor)
plt.show()
Result small graph
Result large graph
Using the first code block from the question that shows filling the simpler graph, I constructed an example network.
The edges are listed below:
edges = [((1, 2), (2, 2)),
((1, 2), (0, 2)),
((1, 2), (1.25, 1.25)),
((2, 2), (1.75, 1.25)),
((2, 2), (3, 2)),
((1.75, 1.25), (1.25, 1.25)),
((1.75, 1.25), (2.25, 0.75)),
((3, 2), (3, 1)),
((0, 2), (0, 1)),
((1.25, 1.25), (0.75, 0.75)),
((0.75, 0.75), (0, 1)),
((0.75, 0.75), (1, 0)),
((2.25, 0.75), (2, 0)),
((2.25, 0.75), (3, 1)),
((2, 0), (1, 0)),
((2, 0), (3, 0)),
((3, 1), (3, 0)),
((0, 1), (0, 0)),
((1, 0), (0, 0))]
With this network, we do not need to use any Voronoi
diagram (although very pleasing to they eye) to fill
the cells of the network.
The basis for the solution is to use the minimum cycle
basis iterator for the network, and then correct each
cycle for following actual edges in the network (see
documentation for minimum cycle basis
"nodes are not necessarily returned in a order by
which they appear in the cycle").
The solution becomes the following, assuming edges
from above:
import matplotlib.pyplot as plt
import networkx as nx
G = nx.Graph()
for edge in edges:
G.add_edge(edge[0], edge[1])
pos = {x: x for x in G.nodes}
options = {
"node_size": 10,
"node_color": "lime",
"edgecolors": "black",
"linewidths": 1,
"width": 1,
"with_labels": False,
}
nx.draw_networkx(G, pos, **options)
# Fill all cells of graph
for cycle in nx.minimum_cycle_basis(G):
full_cycle = cycle.copy()
cycle_path = [full_cycle.pop(0)]
while len(cycle_path) < len(cycle):
for nb in G.neighbors(cycle_path[-1]):
if nb in full_cycle:
idx = full_cycle.index(nb)
cycle_path.append(full_cycle.pop(idx))
break
plt.fill(*zip(*cycle_path))
plt.show()
The resulting graph looks like this:
This algorithm scales better than the Voronoi / centroid
approach listed in the edit to the question, but suffers
from the same inefficiencies for large networks (O(m^2n),
according to the reference in the
documentation for minimum cycle basis).
I have a face-vertex mesh like the following image has.
Face-Vertex Mesh
I have 'face_list' and 'vertex_list' data sets, but not sure how to efficiently calculate a list of edges.
Compute the normal vectors of the faces. If the normal vectors of 2 adjacent faces point in different directions, the two vertices shared by the face form an edge.
The normal vectors of a face can be computed with the Cross product. For a face with the vertices A, B, C, the unit normal vector is:
N = normalized(cross(B-A, C-A))
The normal vectors of the faces can be compared with the Dot product. 2 normal vectors N1 and N2 are equal directed if:
equally_directed = abs(dot(N1, N2)) == 1
Use a library for the vector arithmetic. For example OpenGL Mathematics (GLM) library for Python or NumPy.
Minimal example:
import glm, math
vertices = [(-1,-1,-1), ( 1,-1,-1), ( 1, 1,-1), (-1, 1,-1), (-1,-1, 1), ( 1,-1, 1), ( 1, 1, 1), (-1, 1, 1)]
faces = [(0,1,2), (0,2,3), (5,4,7), (5,7,6), (4,0,3), (4,3,7), (1,5,6), (1,6,2), (4,5,1), (4,1,0), (3,2,6), (3,6,7)]
v = [glm.vec3(vertex) for vertex in vertices]
nv_list = [glm.normalize(glm.cross(v[i1]-v[i0], v[i2]-v[i0])) for i0,i1,i2 in faces]
edge_threshold = 0.99
edges = []
for i, f1 in enumerate(faces):
for j, f2 in enumerate(faces[i+1:]):
edge_candidates = [(f1[0], f1[1]), (f1[1], f1[2]), (f1[2], f1[0])]
for ei0, ei1 in edge_candidates:
if ei0 in f2 and ei1 in f2:
cos_nv = math.fabs(glm.dot(nv_list[i], nv_list[j+i+1]))
if abs(cos_nv) < edge_threshold:
edges.append((ei0, ei1))
print(len(edges))
print(edges)
Output:
12
[(1, 2), (0, 1), (3, 0), (2, 3), (4, 7), (5, 4), (6, 5), (7, 6), (4, 0), (3, 7), (1, 5), (6, 2)]
I'm using Shapely's polygons for human-generated data. Multiple humans were asked to draw polygons around certain features in images. For each image, we thus have n MultiPolygon's, where n equals the number of participants per image.
I can plot each of these Multipolygon's.
fig, ax = plt.subplots()
for ii, multi_poly in enumerate(multi_polys):
for poly in multi_poly.geoms:
x,y = poly.exterior.xy
plt.plot(x,y, c=colors[ii])
We can see that at some locations, the Multipolygon's overlap, while at others there is no overlap.
I wish to get the overlap, or intersections, of these polygons.
This should be trivial, as I can do something like:
intersection = multi_a.intersection(multi_b) \
.intersection(multi_c) \
.intersection(multi_d) \
.inters...
I can plot this intersection on the previous plot and we see:
This looks pretty good. However, this method only returns the regions where all Multipoloygon's overlap. Is there a way to get the intersection where 75% of the polygons overlap? Or where 50% overlap?
A code example: The following dummy data gives this figure:
P1 = Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])
P2 = Polygon([(2.5, 2), (3, 2), (3, 3), (2.5, 3)])
multi_a = MultiPolygon([P1, P2])
P1 = Polygon([(-1, -1), (-1, 2),(2, 2), (2, -1)])
P2 = Polygon([(3, 3), (4, 2), (4, 4), (3, 4)])
multi_b = MultiPolygon([P1,P2])
P1 = Polygon([(-2, 4), (2.2, 4),(2.2, -2), (-2, -2)])
P2 = Polygon([(-1.5, 3), (-1.1, 3), (-1.5, -1), (-1.1, -1)])
multi_c = MultiPolygon([P1,P2])
P1 = Polygon([(2.5, -1), (3.2, -1),(3.2, -2), (-2, -2)])
P2 = Polygon([(3, 0), (4, 0), (3, 1), (4, 1)])
multi_d = MultiPolygon([P1,P2])
On these four multipolygons, the intersection method would returns no intersection as there is not a single spot that all four multipolygons occupy. However, the blue square, accentuated with yellow marker, is occupied by the blue, orange and green polygon. Thus 75% of the multipolygons overlap at this location.
Is there a way (preferably using Shapely) to get the locations where 75% of polygons overlap?
The accepted answer appeared to break down in certain conditions.
I found the problem: the boundaries of the contain shape sometimes overlap the inter polygons. I can replace if geom.contains(polygon): with if geom.contains(polygon.buffer(-1)): and it works exactly as intended.
One approach is to split all geometries to get a flat list of non-intersecting regions on the XY-plane, and then see how many of the original geometries contain each region. Any region that is contained by at least some threshold number of the original geometries can be added to the result. This is easier to explain using a combination of code and illustrations.
To start with, we'll need to fix one issue. The example you put together has a couple invalid geometries, which will cause Shapely to throw an error when attempting to query spatial relationships (e.g., calling contains or intersects). You can check that using the is_valid property, and get a more detailed information by calling explain_validity:
from shapely.geometry import Polygon
from shapely.validation import explain_validity
P2 = Polygon([(-1.5, 3), (-1.1, 3), (-1.5, -1), (-1.1, -1)])
>>> P2.is_valid
False
>>> explain_validity(P2)
'Self-intersection[-1.3 1]'
Basically, it's not happy about shapes like these being expressed as a Polygon when they should be multi-polygons:
So to make your example valid, some of your multi-polygons will have 3 rather than 2 constituent polygons:
P1 = Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])
P2 = Polygon([(2.5, 2), (3, 2), (3, 3), (2.5, 3)])
multi_a = MultiPolygon([P1, P2])
P1 = Polygon([(-1, -1), (-1, 2),(2, 2), (2, -1)])
P2 = Polygon([(3, 3), (4, 2), (4, 4), (3, 4)])
multi_b = MultiPolygon([P1,P2])
P1 = Polygon([(-2, 4), (2.2, 4),(2.2, -2), (-2, -2)])
P2 = Polygon([(-1.5, 3), (-1.1, 3), (-1.3, 1)])
P3 = Polygon([(-1.5, -1), (-1.3, 1), (-1.1, -1)])
multi_c = MultiPolygon([P1,P2,P3])
P1 = Polygon([(2.5, -1), (3.2, -1),(3.2, -2), (-2, -2)])
P2 = Polygon([(3, 0), (4, 0), (3.5, 0.5)])
P3 = Polygon([(3.5, 0.5), (3, 1), (4, 1)])
multi_d = MultiPolygon([P1,P2,P3])
Hopefully your real source data has valid geometries (or you have some way of converting them to become valid geometries - which incidentally is an upcoming feature in Shapely 1.8, implemented via make_valid, but it is not yet released), since otherwise the method described below will not work.
With that out of the way, the first step is to get a flat list of non-intersecting regions from your list of shapes. To do this, we start with the original list of intersecting shapes (note darker shading where multiple shapes overlap):
Convert them to lines using linemerge (in combination with unaryunion):
Then polygonize the result:
It might not be clear from the picture, but the idea is none of these geometries intersect - some of these polygons have holes in them (in cases where one shape previously contained another). So this represents the "flat list of non-intersecting regions on the XY-plane" that I was referring to in the beginning.
The code for the process so far looks like this:
from shapely.geometry import Polygon, MultiPolygon
from shapely.ops import linemerge, unary_union, polygonize
# Original shape definitions here (snipped)...
shapes = [multi_a, multi_b, multi_c, multi_d]
lines = unary_union(linemerge([geom.exterior for shape in shapes for geom in shape.geoms]))
polygons = list(polygonize(lines))
Next, we check each resulting region in the polygons list, and check how many shapes from the original list it intersected with. If it's above the threshold (defined here as 0.75 * len(shapes), then we add it to the result:
threshold = 0.75 * len(shapes)
def overlaps(polygon, shape):
for geom in shape.geoms:
if geom.contains(polygon):
return True
return False
result = []
for polygon in polygons:
containing_shapes = []
for shape in shapes:
if overlaps(polygon, shape):
containing_shapes.append(shape)
if len(containing_shapes) >= threshold:
result.append(polygon)
If you're dealing with a large dataset, checking intersections in a nested loop like that can be quite slow (O(N^2)), so you can probably speed it up using an STRtree:
from shapely.strtree import STRtree
# (Previous code here to get the flattened list of polygons...)
tree = STRtree([geom for shape in shapes for geom in shape.geoms])
result = []
for polygon in polygons:
matches = [o.wkt for o in tree.query(polygon) if o.contains(polygon)]
if len(matches) >= threshold:
result.append(polygon)
I’m working with tetris pieces.
The pieces are defined with coordinates, where each piece has an origin block (0,0)
So an L piece could be defined as [(0,0), (0,1), (0,2), (1,2)] as well as [(0,-1), (0,0), (0,1), (1,1)] depending on where you place the origin block.
I want to check whether a set of coordinates A e.g. [(50,50), (50,51), (50,52), (51,52)] matches the shape of a given tetris piece B.
I’m currently using numpy to take away one of the A values from every value in A to reach relative coordinates, then compare with B. The ordering of A will always been in increasing order, but is not guarenteed to match the ordering of B. B is stored in a list with other tetris pieces, and throughout the program, it's origin block will remain the same. This method below seems inefficient and doesn’t account for rotations / reflections of B.
def isAinB(A,B): # A and B are numpy arrays
for i in range(len(A)):
matchCoords = A - A[i]
setM = set([tuple(x) for x in matchCoords])
setB = set([tuple(x) for x in B])
if setM == setB: # Sets are used here because the ordering of M and B are not guarenteed to match
return True
return False
Is there an efficient method / function to implement this? (Accounting for rotations and reflections aswell if possible)
This is one way to approach it. The idea is to first build all the set of variations of a piece in some canonical coordinates (you can do this once per piece kind and reuse it), then put the given piece in the same canonical coordinates and compare.
# Rotates a piece by 90 degrees
def rotate_coords(coords):
return [(y, -x) for x, y in coords]
# Returns a canonical coordinates representation of a piece as a frozen set
def canonical_coords(coords):
x_min = min(x for x, _ in coords)
y_min = min(y for _, y in coords)
return frozenset((y - y_min, x - x_min) for x, y in coords)
# Makes all possible variations of a piece (optionally including reflections)
# as a set of canonical representations
def make_piece_variations(piece, reflections=True):
variations = {canonical_coords(piece)}
for i in range(3):
piece = rotate_coords(piece)
variations.add(canonical_coords(piece))
if reflections:
piece_reflected = [(y, x) for x, y in piece]
variations.update(make_piece_variations(piece_reflected, False))
return variations
# Checks if a given piece is in a set of variations
def matches_piece(piece, variations):
return canonical_coords(piece) in variations
These are some tests:
# L-shaped piece
l_piece = [(0, 0), (0, 1), (0, 2), (1, 2)]
l_piece_variations = make_piece_variations(l_piece, reflections=True)
# Same orientation
print(matches_piece([(50, 50), (50, 51), (50, 52), (51, 52)], l_piece_variations))
# True
# Rotated
print(matches_piece([(50, 50), (51, 50), (52, 50), (52, 49)], l_piece_variations))
# True
# Reflected and rotated
print(matches_piece([(50, 50), (49, 50), (48, 50), (48, 49)], l_piece_variations))
# True
# Rotated and different order of coordinates
print(matches_piece([(50, 48), (50, 50), (49, 48), (50, 49)], l_piece_variations))
# True
# Different piece
print(matches_piece([(50, 50), (50, 51), (50, 52), (50, 53)], l_piece_variations))
# False
This is not a particularly smart algorithm, but it works with minimal constraints.
EDIT: Since in your case you say that the first block and the relative order will always be the same, you can redefine the canonical coordinates as follows to make it just a bit more optimal (although the performance difference will probably be negligible and its use will be more restricted):
def canonical_coords(coords):
return tuple((y - coords[0][0], x - coords[0][1]) for x, y in coords[1:])
The first coordinate will always be (0, 0), so you can skip that and use it as reference point for the rest, and instead of a frozenset you can use a tuple for the sequence of coordinates.
I'm currently trying to implement the Fourth Nelson rule from:
https://en.wikipedia.org/wiki/Nelson_rules
I.e. given a list of numbers of length N, I want to know if there exists a consecutive sequence of numbers that are alternating in direction of length n. 'Alternating' means consecutive numbers go up, then down, then up, etc.
My data is in (t,x) tuples. 't' stands for the time axis, always increasing. 'x' is the value associated with the time and the series we are concerned with. For example:
data = [(0, 2.5), (1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
Here, the alternating x value sequence is for everything but the first tuple. See the below graph:
The alternating sequence is highlighted in red. The rule looks for 14 points in a row, but I want to generalize this to n-points in a row. (n < N) I can't just output True or False, I want to output the tuple of points that satisfy the condition. In other words, the output would be:
outliers = [(1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
I've tried a few things, none of which resulted in the desired output. These included things like np.diff() and np.sign(). I have a feeling itertools() can do this, but I can't quite get there.
Any input is greatly appreciated.
Here's a first cut at your algorithm in straight Python:
data = [(0, 2.5), (1, 2.1), (2, 1.7), (3, 2.0), (4, 0.3), (5, 0.8), (6, -1.2), (7, -0.5)]
n = 5
t0, x0 = data.pop(0)
outliers = []
up = bool(x0 > 0)
for t, x in data:
if (x < x0 and up) or (x > x0 and not up):
if not outliers:
outliers = [(t0,x0)]
outliers.append((t,x))
up = not up
else:
if len(outliers) >= n:
print 'outliers =', outliers
outliers = []
t0,x0 = t,x
if len(outliers) >= n:
print 'outliers =', outliers