I have some 3D motion capture data for training a deep learning model, but the loss is too high so I need to apply normalization to adjust the input data.
The list below is the steps I am planning to do for the normalization:
Set the skeleton's left sole to be the origin point (0, 0, 0)
All joint points (x, y, z) minus the left sole point(xls, yls, zls)
Set the skeleton's height as 1
Divide all the joint point y-axis values by the y-value of the head point
Calculate the angle between the waist-line and x-axis line
Calculate the angle between the two lines, the waist-line (The line connected by the Left-waist point and Right-waist point), and the line parallel to the X-axis extending from the midpoint of the waist-line.
Every joint point multiply with the rotation matrix
Multiply the rotation matrix with the y-axis(height axis) as the rotation axis (right-handed coordinate system: the value on the X-axis is greater on the right than on the left)
The rotation matrix is obtained as follows:
[[cos_theta, 0, neg_sin_theta],
[0, 1, 0],
[sin_theta, 0, cos_theta]]
From a mathematical perspective, I drew an image as shown below to depict the normalization by multiplying with a rotation matrix, hoping the skeleton can rotate with the waist and let the waist-line always be parallel with the x-axis :
Now I want to normalize the 3D skeleton coordinates by multiplying the rotation matrix:
(I tested 3 different rotation matrixes. The above mentioned rotation matrix produced very high distortion, so I tested with the matrix below and got a more reasonable rotation, which I don't know why...)
rotate_matrix =[[cos_theta, neg_sin_theta, 0],
[sin_theta, cos_theta, 0],
[0, 0, 1]]
Got the result of normalization:
The normalization using this rotation matrix still produces weird result, but at least it seems to rotate with the height-axis as desired.
The original skeleton movement:
Hope someone give some advice of this normalization, or provide other ideas to deal with 3D skeleton normalization.
Thank you in advance for your input.
Related
I have a series of radar point clouds and I have used shapfile to segment out several of these areas, which are all rectangular in shape from the z-axis. I would like to know if there is a way to rotate them so that one edge is parallel to the x or y axis. My idea is to create obb enclosing boxes with abb enclosing boxes and then compare the two and rotate them cyclically. Thanks!
aabb = cloud.get_axis_aligned_bounding_box()
aabb.color = (1, 0, 0)
obb = cloud.get_oriented_bounding_box()
obb.color = (0, 1, 0)
You should be able to call cloud.rotate() directly to which you pass a 3x3 rotation matrix.
To get that 3x3 rotation matrix you have multiple options:
axis-angle: open3d.geometry.get_rotation_matrix_from_axis_angle
quaternion: open3d.geometry.get_rotation_matrix_from_quaternion
euler : open3d.geometry.get_rotation_matrix_from_xyz (and variants (from_xzy, from_yxz, from_yzx, from_zxy, from_zyx))
(angles are in radians, not degrees)
e.g.
cloud.rotate(o3d.geometry.get_rotation_matrix_from_axis_angle([np.radians(90), 0, 0]))
(this should apply a 90 degrees rotation on the x axis: adjust as needed)
I have the coordinates of 6 points in an image
(170.01954650878906, 216.98866271972656)
(201.3812255859375, 109.42137145996094)
(115.70114135742188, 210.4272918701172)
(45.42426300048828, 97.89037322998047)
(167.0367889404297, 208.9329833984375)
(70.13690185546875, 140.90538024902344)
I have a point as center [89.2458, 121.0896]. I am trying to re-calculate the position of points in python using 4 rotation degree (from 0,90,-90,180) and 6 scaling factor (0.5,0.75,1,1.10,1.25,1.35,1.5).
My question is how can I rotate and scale the abovementioned points relative to the center point and get the new coordinates of those 6 points?
Your help is really appreciated.
Mathematics
A mathematical approach would be to represent this data as vectors from the center to the image-points, translate these vectors to the origin, apply the transformation and relocate them around the center point. Let's look at how this works in detail.
Representation as vectors
We can show these vectors in a grid, this will produce following image
This image provides a nice way to look at these points, so we can see our actions happening in a visual way. The center point is marked with a dot at the beginning of all the arrows, and the end of each arrow is the location of one of the points supplied in the question.
A vector can be seen as a list of the values of the coordinates of the point so
my_vector = [point[0], point[1]]
could be a representation for a vector in python, it just holds the coordinates of a point, so the format in the question could be used as is! Notice that I will use the position 0 for the x-coordinate and 1 for the y-coordinate throughout my answer.
I have only added this representation as a visual aid, we can look at any set of two points as being a vector, no calculation is needed, this is only a different way of looking at those points.
Translation to origin
The first calculations happen here. We need to translate all these vectors to the origin. We can very easily do this by subtracting the location of the center point from all the other points, for example (can be done in a simple loop):
point_origin_x = point[0] - center_point[0] # Xvalue point - Xvalue center
point_origin_y = point[1] - center_point[1] # Yvalue point - Yvalue center
The resulting points can now be rotated around the origin and scaled with respect to the origin. The new points (as vectors) look like this:
In this image, I deliberately left the scale untouched, so that it is clear that these are exactly the same vectors (arrows), in size and orientation, only shifted to be around (0, 0).
Why the origin
So why translate these points to the origin? Well, rotations and scaling actions are easy to do (mathematically) around the origin and not as easy around other points.
Also, from now on, I will only include the 1st, 2nd and 4th point in these images to save some space.
Scaling around the origin
A scaling operation is very easy around the origin. Just multiply the coordinates of the point with the factor of the scaling:
scaled_point_x = point[0] * scaling_factor
scaled_point_y = point[1] * scaling_factor
In a visual way, that looks like this (scaling all by 1.5):
Where the blue arrows are the original vectors and the red ones are the scaled vectors.
Rotating
Now for rotating. This is a little bit harder, because a rotation is most generally described by a matrix multiplication with this vector.
The matrix to multiply with is the following
(from wikipedia: Rotation Matrix)
So if V is the vector than we need to perform V_r = R(t) * V to get the rotated vector V_r. This rotation will always be counterclockwise! In order to rotate clockwise, we simply need to use R(-t).
Because only multiples of 90° are needed in the question, the matrix becomes a almost trivial. For a rotation of 90° counterclockwise, the matrix is:
Which is basically in code:
rotated_point_x = -point[1] # new x is negative of old y
rotated_point_y = point[0] # new y is old x
Again, this can be nicely shown in a visual way:
Where I have matched the colors of the vectors.
A rotation 90° clockwise will than be
rotated_counter_point_x = point[1] # x is old y
rotated_counter_point_y = -point[0] # y is negative of old x
A rotation of 180° will just be taking the negative coordinates or, you could just scale by a factor of -1, which is essentially the same.
As last point of these operations, might I add that you can scale and/or rotated as much as you want in a sequence to get the desired result.
Translating back to the center point
After the scaling actions and/or rotations the only thing left is te retranslate the vectors to the center point.
retranslated_point_x = new_point[0] + center_point_x
retranslated_point_y = new_point[1] + center_point_y
And all is done.
Just a recap
So to recap this long post:
Subtract the coordinates of the center point from the coordinates of the image-point
Scale by a factor with a simply multiplication of the coordinates
Use the idea of the matrix multiplication to think about the rotation (you can easily find these things on Google or Wikipedia).
Add the coordinates of the center point to the new coordinates of the image-point
I realize now that I could have just given this recap, but now there is at least some visual aid and a slight mathematical background in this post, which is also nice. I really believe that such problems should be looked at from a mathematical angle, the mathematical description can help a lot.
(forgive my terminology - it has been a long time since I took an advanced math class)
Let's say I have n "planes" each "perpendicular" to a single axis in m-dimensional space. No two planes are perpendicular to the same axis. I believe I can safely assume that there will be some intersection between all n planes.
I want to project point a onto the intersection and get the position vector for the result.
For example:
I have a single plane whose normal vector is (0.75, 0, 0) and a point a at position (0.25, 0, 1). I want to get the position vector of point a projected onto the plane.
Another example:
I have two planes represented by normal vectors (0.5, 0, 0) and (0, 1, 0). I have a point a at position (0.1, 0.1, 0.1). I want to get the position vector of the point projected onto the result of the intersection between my two planes (a line)
Your "planes" in m-dimensional space are (m-1)-dimensional objects. They are usually referred to as hyperplanes — a generalization of planes, 2-dimensional objects in 3-dimensional space. To define a hyperplane you need not only a normal vector but also a point (think of lines in two-dimensional space: all parallel lines share the same direction, and in order to isolate one you need to specify a point).
I suspect you mean all of your hyperplanes to pass through the origin (in which case indeed there is a point in the intersection — the origin itself), and I interpret your "being perpendicular to a single axis" as saying that the normal vectors all point along some coordinate axis (in other words, they have a single nonzero component). In that case, all you have to do to find the projection of an arbitrary point (vector, really) onto the intersection is set to zero the components of the point (again, vector, really) along the normal vectors of your hyperplanes.
Let me go through your examples:
The (hyper)plane in 3-dimensional space with normal vector (0.75, 0, 0) is the yz-plane: the projection of an arbitrary point (x, y, z) is (0, y, z) — the hyperplane has a normal vector along the first coordinate, so set to zero the first component of the point (for the last time: vector, really). In particular, (0.25, 0, 1) projects to (0, 0, 1).
The planes perpendicular to (0.5, 0, 0) and (0, 1, 0) are the yz- and xz-planes. Their intersection is the z-axis. The projection of the point (0.1, 0.1, 0.1) is (0, 0, 0.1).
The projection can be computed by solving an overdetermined system in the sense of least squares, with lstsq. The columns of the matrix of the system is formed by normal vectors, used as columns (hence, transpose on the second line below).
coeff are coefficients to be attached to these normal vectors; this linear combination of normal vectors is subtracted from the given point to obtain the projection
import numpy as np
normals = np.transpose(np.array([[0.5, 0, 0], [0, 1, 0]])) # normals
point = np.array([0.1, 0.1, 0.1]) # point
coeff = np.linalg.lstsq(normals, point)[0]
proj = point - np.dot(normals, coeff)
print(proj)
Output: [0, 0, 0.1].
I am doing my best to replicate the algorithm described here in this paper for making an inpainting algorithm. The idea is to get the contour or edge points of the part of the image that needs to be inpainted. In order to find the most linear point in the region, the orthogonal normal vector is found. On page 6, a short description of the implementation is given.
In our implementation the contour
δΩ of the target region is modelled as a dense list of image point
locations. Given a point p ∈ δΩ, the normal direction np
is computed as follows: i) the positions of the
“control” points of δΩ are filtered via a bi-dimensional Gaussian
kernel and, ii) np is estimated as the unit vector orthogonal to
the line through the preceding and the successive points in the
list.
So it appears that I need to put all these points in a gaussian filter. How do I set up a bi-dimensional Gaussian filter when we have a single dimension or a list of points?
Lets say our contour is a box shape at points, then I create a 1 dimensional list of points: [1,1],[1,2],[1,3],[2,1],[2,3],[3,1],[3,2],[3,3]. Do I need to simply make a new 2d matrix table and put the points in and leave the middle point at [2,2] empty, then run a Gaussian filter on it? This doesn't seem very dense though.
I am trying to run this through python libraries.
a dense list of image points
is simply a line.
You are basically applying a gaussian filter to a black and white image where the line is black and background is white, from what I understand. I think by doing that, they approximate the curve model fitting.
Convolve all of the points in the 2D region surrounding the point and then overwrite the point with the result.
This will make any curve on the edge of the target region less sharp, lowering the noise in the calculation of the normal, which would be the vector orthogonal to the two points that surround the current one.
I have a cube of particles which I've projected onto a 2D grid, Projecting the particles onto the grid by a clouds in cells and weighting them by a scalar.
I would then like the gradient of the scalar at every grid point. In 2D I am doing this using np.gradient and I get two arrays with the gradient in the x and y directions:
gradx, grady = np.gradient(grid)
Does anyone have any idea how I can generalize this to 3 Dimensions? The Clouds in Cells in 3D is fine but I am then left with a grid with the shape (700, 700, 700).
As far as I can see np.gradient can't deal with this?
Thanks,
Daniel
The Numpy documentation indicates that gradient works for any dimensions:
numpy.gradient(f, *varargs)
Return the gradient of an N-dimensional array.
The gradient is computed using central differences in the interior and
first differences at the boundaries. The returned gradient hence has
the same shape as the input array.
Parameters :
f: array_like. An N-dimensional array containing samples
of a scalar function.
*varargs: 0, 1, or N scalars specifying the sample distances in each direction, that is: dx, dy, dz, ... The default distance is 1.
Returns :
g: ndarray. N arrays of
the same shape as f giving the derivative of f with respect to each
dimension.
Seems like you should be able to extend your 2-dimensional code to 3D like you would expect.