I hope you don't mind if I ask for a bit of advice regarding modelling robotic systems. I've recently become rather interested in using inverse kinematics (IK) to control a 5 dof robotic manipulator. I have a solid foundation in IK but what I'm having trouble with is a way to visualize how the manipulator moves with respect to joint angles.
I've looked into using 3D toolkits (such as Blender, Panda3D, vPython) to create a 3d model of the arm, but I'm not sure if I should be looking something with physics support. I'm also not sure how well I can model motion with these packages. Anyone have any suggestions? What I'm NOT looking for is a full blown robotic simulator like Microsoft's Robotic Studio, I'd like to start with the basics and learn how everything works first, ie code the IK in Python, then visualize the motion in 3D. I'm very familiar with Python, so something that interfaces with Python would be preferable.
Thanks!
Well the great thing about Blender is that its API is actually in python!
In addition, it supports inverse kinematics (IK) quite well in addition to many other modeling tools.
Blender Cookie is a great resource.
Here is a tutorial on making IK rigs in Blender.
Blenders python api is documented quite extensively, and it even has an interactive python shell built right in so that you can see the effects of your script as you go along.
The physics engine that blender uses is the popular bullet physics engine, which has been used in many commercial games as well as a few feature films (2012 among them).
This isn't really a hard problem, is it? Presumably you're working out the math on your own; so if your robotic arm is visualized as, say, a few rectangular solids then all you need is something that will render these at the x,y,z coordinates and with the orientation vector you supply, updating when need be. OpenGL should do just fine for this, you could probably do it in <50 lines.
Python-Ogre is a wrapper around the very mature C++ engine OGRE 3D. I have used it for a pretty significant project and I would give the Python wrapper a mixed but mostly positive review. It's very capable and does a good job of wrapping all C++ functionality in a mostly Pythonic interface. You can get the physics support through one of the many OGRE extension modules which provide those capabilities.
The biggest negative is that the documentation for the Python interface is poor, and you will have to rely on the C++ library's documentation to make sense of some things. To give you a sense of what's possible, I was able to implement a scripting language around OGRE using Python and PyV8 (the language was Javascript), as I prototyped a C++ project.
The resulting prototype was very mature and robust, and I was able to wrap the internals of OGRE with high-level Javascript. With the scripting language I was able to script animation, materials, rendering and physics.
Related
I was wondering if I could build an augmented reality system in Python using OpenCV and SLAM. If so, do you have any tutorials or documentation that you could recommend? I've been scratching my head for awhile now trying to find resources to start with, any help would be greatly appreciated!
If I were to be a bit more specific, it would be on how would I be able to integrate SLAM and AR together. SLAM acting as a form of mapping so that the AR would know where to place objects in.
if I want to be honest, python is not strong enough to bring you "RealTime Monocular SLAM System". So, firstly you should consider writing your SLAM system in C++ that is highly recommended for RealTime Systems!
Secondly, You Can see some openSource SLAM systems here (Stella-VSLAM, ORBSLAM3, PTAM). But Consider that for developing a SLAM system, you should gain knowledge in wide range of computer science related topics! and the main reason of why ARCore and ARKit are working great is their efficient SLAM system. you can also read this resource for more info on SLAM systems. If you have more questions, please don't hesitate to ask!
Where to Start
In order to gain some knowledge on SLAM and Computer Vision I would recommend watching Cyrill Stachniss' SLAM course and reading the papers ORB-SLAM, ORB-SLAM2, ORB-SLAM3, and DSO. For Computer Vision I recommend reading R. Szeliski book.
Which Language to Use
I wrote my thesis on SLAM and AR systems, and the outcome is the following: State-of-the-art SLAM systems which achieve the best accuracy are still using machine learning techniques: SURF, ORB descriptors, Bag of Words (BoW) etc. All of the systems (ORB-SLAM3, DM-VIO, DSO) are written in C++.
I'm always using C++ for programming SLAM, and only sometimes I use Python to write scripts for example to fix the recovered trajectory.
SLAM + AR
There's no much resources on this subject, although the idea is simple. SLAM system has to give you the camera location, usually as the 4x4 transformation matrix, where the first 3x3 matrix is the rotation matrix, and the last 3x1 column is the translation part. Example of the transformation matrix.
Having the camera location, you can use the projective geometry to project the AR objects on the camera frame. ORB-SLAM2 has a nice AR demo to study; basically they display a 2D image, and put the 3D rendered image on top of that.
They use Pangolin, so you need to know how to use OpenGL, Pangolin. I recommend studying Pangolin by its' examples, as it mostly documented through them.
I am developing a 3D Graphing Calculator software in Unity3D. I am struggling to find good numerical/symbolic math libraries written in C/C#. Also, I want to make the calculator programmable (like the TI-84 Calculators everyone uses in school).
I found Python to be a very good language for the backbone of this project. SymPy and NumPy are really good math libraries. And Python would be really good for users to program the calculator. (Here is already a calculator, the Numworks, that supports Python programs. The team ported Python to the hardware.)
If I am developing for the PC, I could just use Python with openGL, or use IronPython in Unity.
However, I'm developing for the Nintendo 3DS. It may sound like a strange hardware choice, but it makes good sense. The 3DS has a stereoscopic display perfect for showing 3D functions. It has a joy stick to operate the camera. It has a touch screen to handle input. Imagine showing a height map of a 3D function on the lower display. You can use it to trace the function. You can draw a region D on it, and have the 3DS integrate over the region. You can take directional derivatives with your stylus. The possibility is endless.
The problem, of course, is that Unity doesn't support Python, and IronPython only supports Windows/Mac/Linux/Android.
Do you have any suggestions for me?
For example, any possible way to use python in unity for the 3DS? Any good math library in C/C#? Can I implement a good enough Python interpreter in C# to handle user's program (doesn't have to be complaint)?
You could try the Math.Net library for C#, it should have the functionality you need, or you could try IronPython, here is an article that might help you get started.
http://shrigsoc.blogspot.com/2016/07/ironpython-and-unity.html
I am currently in a project where a lot of 3D printing designs need to be done. They are all parameterized, so I'd like to write a python code to generate those design files (in .STL format) for me. I was wondering that, is there a python package that can do this? Because currently I am all doing those by hand using SolidWorks.
Thanks!
Yes there is... It's called FreeCAD.
The assembly module is already in the devel version (as of 06/15/2014) and will be of production quality really soon for real assemblies!
http://freecadweb.org/
Yes, more than one.
In my humble experience, I tried many Open Source tools for parametric CAD modeling using Python (FreeCAD, Rhino-Grasshopper, Blender, Salome).
All of them are valid options and the best one is represented by your ability to either model or code.
I recently favour SALOME (www.salome-platform.org) because of the straight forward "dump study" option, the continue development and the good API documentation.
Particularly I did some 3d prints using the exportSTL command once I had a solid worthy of printing and it was ok.
Nevertheless, if you intend to work on surfaces rather than solids, I don't think you will find anything worthy Open Source (Rhino has a little price to pay).
There is also a new one ! called pymadcad
It's a library meant to do complete CAD stuff only with python scripts.
At contrary to FreeCAD, Pymadcad is natively dealing with triangular meshes so it makes it very easy to import/export .stl files.
There is a growing amount of surface generation functions (extrusion, revolution, tube, screw, smooth surface, ...). And there is also all the stuff to generate and deal with 3D primtives such as Lines, Arc, ...
Here is a brief look at the features
I will soon be starting a final year Engineering project, consisting of the real-time tracking of objects moving on a 2D-surface. The objects will be registered by my algorithm using feature extraction.
I am trying to do some research to decide whether I should use MATLAB or use Python Numpy (Numerical Python). Some of the factors I am taking into account:
1.) Experience
I have reasonable experience in both, but perhaps more experience in image processing using Numpy. However, I have always found MATLAB to be very intuitive and easy to pick up.
2.) Real-Time abilities
It is very important that my choice be able to support the real-time acquisition of video data from an external camera. I found this link for MATLAB showing how to do it. I am sure that the same would be possible for Python, perhaps using the OpenCV library?
3.) Performance
I have heard, although never used, that MATLAB can easily split independent calculations across multiple cores. I should think that this would be very useful, and I am not sure whether the same is equally simple for Numpy?
4.) Price
I know that there is a cost associated with MATLAB, but I will be working at a university and thus will have access to full MATLAB without any cost to myself, so price is not a factor.
I would greatly appreciate any input from anyone who has done something similar, and what your experience was.
Thanks!
Python (with NumPy, SciPy and MatPlotLib) is the new Matlab. So I strongly recommend Python over Matlab.
I made the change over a year ago and I am very happy with the results.
Here it is a short pro/con list for Python and Matlab
Python pros:
Object Oriented
Easy to write large and "real" programs
Open Source (so it's completely free to use)
Fast (most of the heavy computation algorithms have a python wrapper to connect with C libraries e.g. NumPy, SciPy, SciKits, libSVM, libLINEAR)
Comfortable environment, highly configurable (iPython, python module for VIM, ...)
Fast growing community of Python users. Tons of documentation and people willing to help
Python cons:
Could be a pain to install (especially some modules in OS X)
Plot manipulation is not as nice/easy as in Matlab, especially 3D plots or animations
It's still a script language, so only use it for (fast) prototyping
Python is not designed for multicore programming
Matlab pros:
Very easy to install
Powerful Toolboxes (e.g. SignalProcessing, Systems Biology)
Unified documentation, and personalized support as long as you buy the licence
Easy to have plot animations and interactive graphics (that I find really useful for running experiments)
Matlab cons:
Not free (and expensive)
Based on Java + X11, which looks extremely ugly (ok, I accept I'm completely biased here)
Difficult to write large and extensible programs
A lot of Matlab users are switching to Python :)
I would recommend python.
I switched from MATLAB -> python about 1/2 way through my phd, and do not regret it. At the most simplistic, python is a much nicer language, has real objects, etc.
If you expect to be doing any parts of your code in c/c++ I would definitely recommend python. The mex interface works, but if your build gets complicated/big it starts to be a pain and I never sorted out how to effectively debug it. I also had great difficulty with mex+allocating large blocks interacting with matlab's memory management (my inability to fix that issue is what drove me to switch).
As a side note/self promotion, I have Crocker-Grier in c++ (with swig wrappers) and pure python.
If you're experienced with both languages it's not really a decision criterion.
Matlab has problems coping with real time settings especially since most computer vision algorithms are very costly. This is the advantage of using a tried and tested library such as OpenCV where many of the algorithms you'll be using are efficiently implemented. Matlab offers the possibility of compiling code into Mex-files but that is a lot of work.
Matlab has parallel for loops parfor which makes multicore processing easy (or at least easier). But the question is if that will suffice to get real-time speeds.
No comment.
The main advantage of Matlab is that you'll obtain a running program very quickly due to its good documentation. But I found that code reusability is bad with Matlab unless you put a heavy emphasis on it.
I think the final decision has to be if you have to/can run your algorithm real-time which I doubt in Matlab, but that depends on what methods you're planning to use.
Others have made a lot of great comments (I've opined on this topic before in another answer https://stackoverflow.com/a/5065585/392949) , but I just wanted to point out that Python has a number of really excellent tools for parallel computing/splitting up work across multiple cores. Here's a short and by no means comprehensive list:
IPython Parallel toolkit: http://ipython.org/ipython-doc/dev/parallel/index.html
mpi4py: https://code.google.com/p/mpi4py
The multiprocessing module in the standard library: http://docs.python.org/library/multiprocessing.html
pyzmq: http://zeromq.github.com/pyzmq/ (what the IPython parallel toolkit is based on)
parallel python (pp): http://www.parallelpython.com/
Cython's wrapping of openmp: http://docs.cython.org/src/userguide/parallelism.html
You will also probably find cython to be much to be a vastly superior tool compared to what Matlab has to offer if you ever need to interface external C-libraries or write C-extensions, and it has excellent numpy support built right in.
There is a list with a bunch of other options here:
http://wiki.python.org/moin/ParallelProcessing
I would like to know similar, concrete simulations, as the simulation about watering a field here.
What is your favorite library/internet page for such simulations in Python?
I know little Simpy, Numpy and Pygame. I would like to get examples about them.
If you are looking for some game physics (collisions, deformations, gravity, etc.) which looks real and is reasonably fast consider re-using some physics engine libraries.
As a first reference, you may want to look into pymunk, a Python wrapper of Chipmunk 2D physics library. You can find a list of various Open Source physics engines (2D and 3D) in Wikipedia.
If you are looking for physically correct simulations, no matter what language you want to use, it will be much slower (almost never real-time), and you need to use some numerical analysis software (and probably to write something yourself). Exact answer depends on the problem you want to solve. It is a fairly complicated field (of math).
For example, if you need to do simulations in continuum mechanics or electromagnetism, you probably need Finite Difference, Finite Volume or Finite Element methods. For Python, there are some ready-to-use libraries, for example: FiPy (FVM), GetFem++ (FEM), FEniCS/DOLFIN (FEM), and some other.
Here is some simple astronomy related python. And here is a hardcore code from the same guy.
And Eagleclaw solves and plots various hyperbolic equations using some python. However, most of the code is written in Fortran to do the computations and python to plot the results. If you are studying physics though you may have to get used to this kind of Fortran wrapped code. It is a reality. But this isn't really what your looking for I guess. The good thing it that it is documented in a literate programming style so it should be understandable.
Maybe PyODE?
I've heard of PyBox2D, which is a port of the really nice Box2D. To quote the site:
Box2D is a feature rich 2d rigid body physics engine, written in C++ by Erin Catto. It has been used in many games, including Crayon Physics Deluxe, winner of the 2008 Independent Game Festival Grand Prize.