I have a triangle mesh and look for a way to get programmatically for a given (x,z) 2D point all y coordinates which are represented by the mesh (x,y1,z),(x,y2,z) ..., preferable in python. I have the mesh stored in one of the common file formats (.stl , .obj ...)
The problem behind this question is that i convert a 2D face image into a 3D mesh of the face (using the marvelous https://github.com/sicxu/Deep3DFaceRecon_pytorch project) and then want to map the depth information of the 3D model back to the 2d image (to build something fancy in blender ...)
I finally found a solution for the problem which is both slow and inelegant but does the job for me for now.
I use section_multiplane function of the trimesh python library for that. Basically i use this function to intersect the mesh with 2d planes parallel to the y,z-plane and calculate the depth values by analyzing the resultant 2D Path. The library is very fast in calculating the intersections but the part i wrote - extracting the depth information from the 2D Paths - is painfully slow right now. (which doesnt matter in my particular application)
The code for that I have now is deeply interwoven in my particular application so it doesn't make sense to share it but if someone is interested in this approach there is a very helpful example included in the trimesh library which covers the crucial points: section_multiplane example
I am sure there are much more elegant solutions for this problem available but I wanted to share this approach in case somebody struggles finding a better approach too ...
Heatmap displaying the extracted depth information:
depth info:
Related
I have a point cloud and meshes (vertices=points of the point cloud).
I want to project the point cloud with a certain virtual camera.
Here, since the point cloud is sparse, the rendered result includes the points which should be occluded by foreground objects.
To resolve this issue, I want to use mesh information to identify which points should be occluded.
Is there any smart way to do this in python?
Kind advice will be greatly appreciated.
After hours of searching, I conclude that I have to re-implement a novel rendering pipeline to achieve my goal.
So, instead of this, I use a mesh-based renderer to render a depth map.
And then I simply project the points of the point cloud with a projection matrix.
Here, I use the depth map to check whether the point fits with the depth or not.
If the projected point is the one that should be occluded, then the depth of the point would be larger than the depth map value at the corresponding pixel.
So, such points should be ignored while rendering.
I know that this is a less elegant and inefficient trick but anyway it works very well :)
I currently have 2 paths generated by a motion planning algorithm in Python. I interpolated into two 3D Polylines with equidistant points. Next I need to find whether these two paths of different lengths intersect (OR even ideally the point(s) at which they get near a certain distance of each other, if any). This is to eventually simulate a collision avoidance algorithm.
After much searching I have found multiple libraries and algorithms that can return points/line intersections of two Polylines, however they are all in 2D. I tried using the Bentley-Ottmann algorithm/python library, the Shapely library, the Geometry3D library, and other 2D solutions found on SO, but none have worked in a 3D environment. Next I tried using a 2D geometry library to find the intersections in the x-y, x-z, and y-z planes individually as shown in the graphs below showing a simple case, but I'm not sure where to go from there or whether that's even the right approach. The Polylines would usually contain 300+ points.
Any help pointing me towards a 3D algorithm or simple adjustments to make to the 2D libraries work in 3D would be really appreciated!
3D graph view, x-y view, z-y view.
I'm a wee bit stuck.
I have a 3D point cloud (an array of (n,3) vertices), in which I am trying to generate a 3D triangular mesh from. So far I have had no luck.
The format my data comes in:
(x,y) values in regularly spaced (z) intervals. Think of the data as closed loop planar contours stored slice by slice in the z direction.
The vertices in my data must be absolute positions for the mesh triangles (i.e. I don't want them to be smoothed out such that the volume begins to change shape, but linear interpolation between the layers is fine).
Illustration:
Z=2. : ..x-------x... <- Contour 2
Z=1.5: ...\......|... <- Join the two contours into a mesh.
Z=1. : .....x----x... <- Contour 1
Repeat for n slices, end up with an enclosed 3D triangular mesh.
Things I have tried:
Using Open3D:
The rolling ball (pivot) method can only get 75% of the mesh completed and leaves large areas incomplete (despite a range of ball sizes). It has particular problems at the top and bottom slices where there tends to be large gaps in the middle (i.e. a flat face).
The Poisson reconstruction method smooths out the volume too much and I no longer have an accurate representation of the volume. This occurs at all depths from 3-12.
CGAL:
I cannot get this to work for the life of me. SWIG is not very good, the implementation of CGAL using SWIG is also not very good.
There are two PyBind implementations of CGAL however they have not incorporated the 3D triangulation libraries from CGAL.
Explored other modules like PyMesh, TriMesh, TetGen, Scikit-Geometry, Shapely etc. etc. I may have missed the answer somewhere along the line.
Given that my data is a list of closed-loop planar contours, it seems as though there must be some simple solution to just "joining" adjacent slice contours into one big 3d mesh. Kind of like you would in blender.
There are non-python solutions (like MeshLab) that may well solve these problems, but I require a python solution. Does anyone have any ideas? I've had a bit of a look into VTK and ITK but haven't found exactly what I'm looking for as of yet.
I'm also starting to consider that maybe I can interpolate intermediate contours between slices, and fill the contours on the top and bottom with vertices to make the data a bit more "pivot ball" method friendly.
Thank you in advance for any help, it is appreciated.
If there is a good way of doing this that isn't coded yet, I promise to code it and make it available for people in my situation :)
Actually there are two ways of having meshlab functionality in python:
The first is MeshLabXML (https://github.com/3DLIRIOUS/MeshLabXML ) a third party, is a Python scripting interface to meshlab scripting interface
the second is PyMeshLab (https://github.com/cnr-isti-vclab/PyMeshLab ) an ongoing effort done by the MeshLab authors, (currently in alpha stage) to have a direct Python bindings to all the meshlab filters
There is a very neat paper titled "Technical Note: an algorithm and software for conversion of radiotherapy contour‐sequence data to ready‐to‐print 3D structures" in the Journal of Medical Physics that describes this problem quite nicely. No python packages are required, however it is more easily implemented with numpy. No need for any 3D packages.
A useful excerpt is provided:
...
The number of slices (2D contours) constituting the specified structure is determined.
The number of points in each slice is determined.
Cartesian coordinates of each of the points in each slice are extracted and stored within dedicated data structures...
Numbers of points in each slice (curve) are re‐arranged in such a way, that the starting points (points with indices 0) are the closest points between the subsequent slices. Renumeration starts at point 0, slice 0 (slice with the lowest z coordinate).
Orientation (i.e., the direction determined by the increasing indices of points with relation to the interior/exterior of the curve) of each curve is determined. If differences between slices are found, numbering of points in non‐matching curves (and thus, orientation) is reversed.
The lateral surface of the considered structure is discretized. Points at the neighboring layers are arranged into threes, constituting triangular facets for the STL file. For each triangle the closest points with the subsequent indices from each layer are connected.
Lower and upper base surfaces of the considered structure are discretized. The program iterates over every subsequent three points on the curve and checks if they belong to a convex part of the edge. If yes, they are connected into a facet, and the middle point is removed from further iterations.
So basically it's a problem of aligning datasets in each slice to the nearest value of each slice. Then aligning the orientation of each contour. Then joining the points between two layers based on distance.
The paper also provides code to do this (for a DICOM file), however I re-wrote it myself and it works a charm.
I hope this helps others! Make sure you credit the author's in any work you do that uses this.
A recent feature of pymadcad can do things like this, not sure through if it fits your exact expectation in term of "pivot ball" or such things, checkout the doc for blending
Starting from a list of outlines, it can generate blended surfaces to join them:
For your purpose, I guess the best is one of:
blendpair(line1, line2)
junction(*lines)
I am using OpenCV and Python.
Let say I have this sequence video of the car. And I have tracked some 'interesting points' of the car with the cv2.goodFeaturesToTrack and cv2.calcOpticalFlowPyrLK. Now, given the traced points, I want to estimate a very rough shape (maybe a 3D box) of the car and its distance from the camera. It doesn't need to be that accurate.
On top of that, I want it to be keep updating in real time. The closest youtube video I can find that can give a view of what I am trying to achieve is this. I have found a new Structure from Motion module in OpenCV, but it is more on building a 3D model from a collection of points.
The question is, what is the best way of achieving this and what kind of library I can use (especially in order to construct the 3D space)?
And it is also OK if somehow I need to use C++ for this (although I am still not good in it yet).
Thanks.
As far as I am aware there is no inbuilt polygon functionality for Python. I want to create a 3D map and figured that polygons would be the best way to go about it.
Not wanting to reinvent the wheel I did some googling and found that there's a lot of Python stuff out there, but I couldn't find what I wanted. Thus before I reinvent the wheel (or invent it a whole), does anybody know of a Polygon system for Python?
Note that it does need to be 3D (I found quite a few 2D ones). Note also that I am not interested in the displaying of them but in storing them and the datastructure within Python.
Thanks
One of the most complete geography/mapping systems available for Python that I know about is GeoDjango. This works on top of the Django, an MVC framework. With it comes a large collection of polygon, line and distance calculation tools that can even take into account the curvature of the earth's surface if need be.
With that said, the quickest way I can think of to produce a 3D map is using a height map. Create a two dimensional list of tuples containing (x, y, z) coordinates. Each tuple represents an evenly spaced point on a grid, mapped out by the dimensions of the array. This creates a simple plane along the X and Z axes; the ground plane. The polygons that make up the plane are quads, a polygon with four sides.
Next, to produce the three dimensional height, simply give each point a Y value. This will create peaks and valleys in your ground plane.
How you render this will be up to you, and converting your grid of points into a polygon format that something like OpenGL can understand may take some work, but have a look at Visual Python, its the simplest 3D library I've seen for Python.
I think you mean Polyhedron, not Polygon .. and you might wanna look at vpython
CGAL is a C++ geometry library which, amongst other things, models Polyhedra (3D flat-surfaced shapes)
It has Python bindings available. The documentation for the Polygon3 class is here:
http://cgal-python.gforge.inria.fr/Manual/CGAL.Polyhedron.html#Polyhedron_3