I have a simple mesh in the shape of the a human face, that I need to animate in terms of various facial expressions given the coordinate positions of the different facial features (like inner outer corners of eyes, lips, eyebrows etc) for different frames (>500) as text files.
I'm fairly new to blender and python but from a little research it was obvious that the answer lies in using shape keys, but I don't know how to go about it.
Are you asking for a way to do this, but programatically?
Here's a tutorial on scripting them. I have't scripted with Blender in a long time, but the documentation is out there (somewhere).
If you must, just run the help() function on modules and functions. They are somewhat documented internally.
I think this is what you need. Check the source code and you will get with the solution. The key is to use shape keys.
https://www.stickmanventures.com/labs/demo/webgl-threejs-morph-target/#
Related
First of all, sorry if this is rather basic but it is certainly not my field of expertise.
So, I'm working with protein surface and I have this cavity:
Protein cavity
It is part of a larger, watertight, triangular mesh (.ply format) that represents a protein surface.
What I want to do, is find whether this particular "sub-mesh" is found in other proteins. However, I'm not looking for a perfect fit, rather similar "sub-meshes" since the only place I will find this exact shape is in the original protein.
I've been reading the docs for the Python modules trimesh and open3d. Trimesh does have a comparison module, but it doesn't seem to have the functionality I'm looking for. Also, open3d has a "compute point cloud distance" function that is recommended to compare the difference between two point cloud or meshes.
However, since what I'm actually trying to find is similarity, I would need a way to fit my cavity's "sub-mesh" onto the surface of the protein I'm analyzing, and then "score" how different or deformed the fitted submesh is. Another way would be to rotate and translate my sub-mesh to match the most vertices and faces on the protein surface and score that I guess.
Just a heads-up, I'm a biotechnologist, self-taught in Python and with extremely limited experience in anything 3D. At this point, anything helps, be it a paper, Python module or whatever knowledge you have that you think might be useful.
Thank you very much for any help you can provide with this!
So lets say I have a objects like a face. I take multiple pictures of the face all at different angles and from far and close. I have a sort of idea of how to make a 3d model out of these pictures but don't know how to accomplish them. My idea goes likes this.
First make code that gets the image object and gets rid of all background "noise".
Second find what part of the 3d model the picture is about and place a tag on the image for where it should fit.
Third collect and overlap all the images together to create a 3d object.
Anyone have any idea how to accomplish any of these steps or any ideas how to create a 3d model out of a series of images? I use python 3.10.4.
It seems that you are asking if there are some Python modules that would help to implement a complete photogrammetry process.
Please note that, even in the existing (and commercial) photogrammetry solutions, the process is not always fully-automated, sometimes it require some manual tweaking & point cloud selection.
Anyway, to the best of my knowledge, what you asked requires to implement the following steps:
detecting common features between the different photographs
infer the position in space of the camera that took each photograph
generate a point cloud of the photographs based on their relative position in space and the common features
convert the point cloud in a 3D mesh.
Possibly, all of these steps can be implemented in Python but I'm not aware that such a "off-the-shelf" module does exist.
There's this commercial solution called: Metashape from Agisoft, it has a python module you can use, but beware that it has its pitfalls (it threw segmentation fault for me at the end of processing which makes things... icky) and the support kind of ignores bigger problems and you can expect that they would ignore your ticket. Still, does the job quite well.
I am thinking about creating a database system for images where they are stored with compact signatures and then matched against a "query image" that could be a resized, cropped, brightened, rotated or a flipped version of the stored one. Note that I am not talking about image similarity algorithms but rather strictly about duplicate detection. This would make things a lot simpler. The system wouldn't care if two images have an elephant on them, it would only be important to detect if the two images are in fact the same image.
Histogram comparisons simply won't work for cropped query images. The only viable way to go I see is shape/edge detection. Images would first be somehow discretized, every pixel being converted to an 8-level grayscale for example. The discretized image will contain vast regions in the same colour which would help indicate shapes. These shapes then could be described with coefficients and their relative position could be remembered. Compact signatures would be produced out of that. This process will be carried out over each image being stored and over each query image when a comparison has to be performed. Does that sound like an efficient and realisable algorithm? To illustrate this idea:
removed dead ImageShack link
I know this is an immature research area, I have read Wikipedia on the subject and I would ask you to propose your ideas about such an algorithm.
SURF should do its job.
http://en.wikipedia.org/wiki/SURF
It is fast an robust, it is invariant on rotations and scaling and also on blure and contrast/lightning (but not so strongly).
There is example of automatic panorama stitching.
Check article on SIFT first
http://en.wikipedia.org/wiki/Scale-invariant_feature_transform
If you want to do a feature detection driven model, you could perhaps take the singular value decomposition of the images (you'd probably have to do a SVD for each color) and use the first few columns of the U and V matrices along with the corresponding singular values to judge how similar the images are.
Very similar to the SVD method is one called principle component analysis which I think will be easier to use to compare between images. The PCA method is pretty close to just taking the SVD and getting rid of the singular values by factoring them into the U and V matrices. If you follow the PCA path, you might also want to look into correspondence analysis. By the way, the PCA method was a common method used in the Netflix Prize for extracting features.
How about converting this python codes to C back?
Check out tineye.com They have a good system that's always improving. I'm sure you can find research papers from them on the subject.
The article you might be referring to on Wikipedia on feature detection.
If you are running on Intel/AMD processor, you could use the Intel Integrated Performance Primitives to get access to a library of image processing functions. Or beyond that, there is the OpenCV project, again another library of image processing functions for you. The advantage of a using library is that you can try various algorithms, already implemented, to see what will work for your situation.
I'm producing an ugv prototype. The goal is to perform the desired actions to the targets set within the maze. When I surf the Internet, the mere right to navigate in the labyrinth is usually made with a distance sensor. I want to consult more ideas than the question.
I want to navigate the labyrinth by analyzing the image from the 3d stereo camera. Is there a resource or successful method you can suggest for this? As a secondary problem, the car must start in front of the entrance of the labyrinth, see the entrance and go in, and then leave the labyrinth after it completes operations in the labyrinth.
I would be glad if you suggest a source for this problem. :)
The problem description is a bit vague, but i'll try to highlight some general ideas.
An useful assumption is that labyrinth is a 2D environment which you want to explore. You need to know, at every moment, which part of the map has been explored, which part of the map still needs exploring, and which part of the map is accessible in any way (in other words, where are the walls).
An easy initial data structure to help with this is a simple matrix, where each cell represents a square in the real world. Each cell can be then labelled according to its state, starting in an unexplored state. Then you start moving, and exploring. Based on the distances reported by the camera, you can estimate the state of each cell. The exploration can be guided by something such as A* or Q-learning.
Now, a rather subtle issue is that you will have to deal with uncertainty and noise. Sometimes you can ignore it, sometimes you don't. The finer the resolution you need, the bigger is the issue. A probabilistic framework is most likely the best solution.
There is an entire field of research of the so-called SLAM algorithms. SLAM stands for simultaneous localization and mapping. They build a map using some sort of input from various types of cameras or sensors, and they build a map. While building the map, they also solve the localization problem within the map. The algorithms are usually designed for 3d environments, and are more demanding than the simpler solution indicated above, but you can find ready to use implementations. For exploration, something like Q-learning still have to be used.
I have load an obj file to render my opengl model using pyopengl and pygame. The 3D model show successfully.
Below is the 3D model i render with obj file, Now i cut my model into ten pieces through y axis , my question is how to get the sectional drawing in each piece?
I'm really very new to openGL, Is there any way can do that?
There are two ways to do this and both use clipping to "slice" the object.
In older versions of OpenGL you can use user clip planes to "isolate" the slices you desire. You probably want to rotate the object before you clip it, but it's unclear from your question. You will need to call glClipPlane() and you will need to enable it using glEnable with the argument GL_CLIP_PLANE0, GL_CLIP_PLANE1, ...
If you don't understand what a plane equation is you will have to read up on that.
In theory you should check to see how many user clip planes exist on your GPU by calling glGetIntegerv with argument GL_MAX_CLIP_PLANES but all GPUs support at least 6.
Since user clip planes are deprecated in modern Core OpenGL you will need to use a shader to get the same effect. See gl_ClipDistance[]
Searching around on Google should get you plenty of examples for either of these.
Sorry not to provide source code but I don't like to post code unless I am 100% sure it works and I don't have the time right now to check it. However I am 100% sure you can easily find some great examples on the internet.
Finally, if you can't make it work with clip planes and some hacks to make the cross sections visible then this may indeed be complicated because creating closed cross sections from an existing model is a hard problem.
You would need to split the object, and then rotate the pieces so that they are seen from the side. (Or move the camera. The two ideas are equivalent. But if you're coding this from scratch, you don't really have the abstraction of a 'camera'.) At that point, you can just render all the slices.
This is complicated to do in raw OpenGL and python, essentially because objects in OpenGL are not solid. I would highly recommend that you slice the object into pieces ahead of time in a modeling program. If you need to drive those operations with scripting, perhaps look into Blender's python scripting system.
Now, to explain why:
When you slice a real-life orange, you expect to get cross sections. You expect to be able to see the flesh of the fruit inside, with all those triangular pieces.
There is nothing inside a standard polygonal 3D model.
Additionally, as the rind of a real orange has thickness, it is possible to view the rind from the side. In contrast, one face of a 3D model is infinitely thin, so when you view it from the side, you will see nothing at all. So if you were to render the slices of this simple model, from the side, each render would be completely blank.
(Well, the bits at the end will have 'caps', like the ends of a loaf a bread, but the middle sections will be totally invisible.)
Without a programming library that has a conception of what a cut is, this will get very complicated, very fast. Simply making the cuts is not enough. You must seal up the holes created by slicing into the original shape, if you want to see the cross-sections. However, filling up the cross sections has to be done intelligently, otherwise you'll wind up with all sorts of weird shading artifacts (fyi: this is caused by n-gons, if you want to go discover more about those issues).
To return to the original statement:
Modeling programs are designed to address problems such as these, so I would suggest you leverage their power if possible. Or at least, you can examine how Blender implements this functionality, as it is open source.
In Blender, you could make these cuts with the knife tool*, and then fill up the holes with the 'make face' command (just hit F). Very simple, even for those who are not great at art. I encourage you to learn a little bit about 3D modeling before doing too much 3D programming. It personally helped me a lot.
*(The loop cut tool may do the job as well, but it's hard to tell without understanding the topology of your model. You probably don't want to get into understanding topology right now, so just use the knife)