I thought about tackling a new project in which I use Tensorflow object detection API to detect Euro pallets (eg. pic).
My ultimate goal is to know how far I am away from the pallet and which relative position I have to it. So I thought about first detecting the euro pallet in an RGB feed from a kinect camera and then using its 3D feature to get the distance to the pallet.
But how do I go about the relative position of the pallet? I could create different classes, for example one is "Front view laying pallet" another one Side view laying pallet etc. but I think for that to be accurate I'd need quite a few pictures for each class for it to be valid? Like 200 for each class?
Since my guess is that there are no such labeled datasets yet thats quite a pain to create by myself.
Another way I could think of, is if I label my pallets with segmentation instead of bounding boxes, maybe there is another way to find out my relative position to the pallet? I never did semantic segmentation labeling myself but can anyone name any good programs which I could use?
I'm hoping someone can help point me in the right direction. Any help would be appreciated.
Some ideas: assuming detection and segmentation with classifier(s) works, one could then try feature detection like edges / lines to obtain clues about its orientation (bounding box).
Of course this will be tricky for simple feature detection because of very different surfaces (wood, dirt), backgrounds and lighting.
Also, "markerless tracking" (a topic in augmented reality) and "bin picking" (actually applied in the automation industry) may be keywords for similar problems, although you are probably not starting with an unordered pile of pallets.
Related
I am new to python and opencv. I am analysing images of clouds, and I need to remove the buildings, so that the subsequent analysis will have less noise. I tried using Canny edge detection and then fill in the contours, but did not get too far. I also tried thresholding by pixel colours, but cannot reliably exclude just the buildings and not other parts of the image containing the clouds.
Is there a way I can efficiently and accurately remove the buildings and keep all of the clouds/sky? Thanks for the tips in advance.
You could use a computer vision model that finds the buildings. There may be some open source ones out there. The only one I can think of at the moment is this semantic segmentation model. There should be details on how to implement it, but there could definitely be others out there.
https://github.com/CSAILVision/semantic-segmentation-pytorch
I think one of the classes is buildings and you could theoretically run the model and get the dimensions of the building and take it out.
I intend to make a 3D model based on multi view stereo images ( basically 2D plane images of the same object from different angles and orientation) inside Blender from scratch.However, I am new to Blender.
I wanted to know if there are any tutorials of how to project a single pixel or point in the space of Blender's 3D environment using python. If not tutorial, any documentation. I am still learning about this whole 3D construction thing and pretty new to this, so I am not sure maybe these points are displayed using a 3 dimensional matrix/array ?
Basically I want to implement 3D construction based on a paper written by some researchers. Mostly every such project is in C++. I want to do it in Python in Blender, and if I am capable enough, make these libraries open source.
Suggest me any pre-requisite if you think that shall help me. I have just started my 3rd year of BSc Computer Science course, and very new to the world of Computer Graphics.
(My skillset is C, Java and Python.)
I would be very glad and appreciate any help.
Thank You
[Link to websitehttps://vision.in.tum.de/research/image-based_3d_reconstruction/multiviewreconstruction[][1]]
image2
Yes, it can very likely be done in Blender, and in Python at least for small geometries / low resolution.
A valid approach for the kind of scenarios you seem to want to play with is based on the idea of "space carving" or "silhouette projection". A good description in is an old paper by Kutulakos and Seitz, which was based in part on earlier work by Szelisky.
Given a good estimation of the silhouettes, these methods can correctly reconstruct all convex portions of the object's surface, and the subset of concavities that are resolved in the photo hull. The remaining concavities are "patched" over and need to be reconstructed using a different method (e.g. stereo, or structured light). For the surfaces that can be reconstructed, space carving is generally more robust than stereo (since it is insensitive to the color and surface texture of the object), and can work on surfaces where structured light struggles (e.g. surfaces with specularities, or very dark objects with low reflectance for a laser stripe)
The basic idea is to use the silhouettes of the projection of the object in cameras around it to "remove" mass from an initial volume (e.g. a box) encompassing the object, a bit like a sculptor carving a statue by removing material from a block of marble.
Computationally, you can do it representing the volume of space of interest using an octree, initialized with a minimal level of subdivision, and then progressively refined. The refinement consists of projecting the vertices of the octree leaves in the cameras, and identifying which leaves are completely outside or partially inside the silhouettes. The former are pruned, while the latter are split, and the process continues until no more leaves can be split or a maximul level of subdivision is reached. The hull of the octree is then extracted as a "watertight" mesh using standard methods.
Apart from the above paper, a way more detailed description can be found on an old patent by Geometrix - it sold a scanner based on the above ideas around year 2000. Here is what it looked like:
fig:Shoe in the red circle is to be detected
I am trying to create a python script using cv2 that can recognize the shoe of the baller and determine whether the shoe is beyond, on or before the white line(refer to the image).
I have no idea about any kind of approach to use, what kind of algorithms might be helpful. Need some guideline, please help!
(Image is attached)
I realize this would work better as a comment because it isn't a full answer, but I don't have enough rep yet to leave comments, haha.
You may be interested in OpenCV's Canny Edge detection algorithm:
http://docs.opencv.org/trunk/da/d22/tutorial_py_canny.html
This will allow you to find shapes within your image.
Also, you can find similarly colored blobs using SimpleBlobDetector:
https://www.learnopencv.com/blob-detection-using-opencv-python-c/
This should make it fairly easy to detect the white line.
In order to detect a more complex object like the shoe, you'll probably have to make something like a object detection cascade file and use a CascadeClassifier to find it:
http://docs.opencv.org/2.4/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html#cascade-classifier
http://johnallen.github.io/opencv-object-detection-tutorial/
Basically, you take a bunch of pictures to "teach" what the object looks like, and output that info to a file that a CascadeClassifier can use to detect objects in input images. It may be hard to distinguish between different brands of shoe though, if you need it to be that specific. Also, you may need to adjust the input images (saturation, brightness, etc) before trying to detect objects in order to get good results.
I have an image captured by android camera. Is it possible to calculate depth of object in the image ? Image contains object and background only. Any suggestion, explanation or links that you think can help me will be appreciated.
OpenCV is the library you need.
I did some depth identification of water levels in pure white background a few days ago. Generally, if you want to identify the depth, you can convert the question to identify the edge of the changing colors. In this case, you can convert the colorful pictures to grey and identify the changing of while-black-grey interface. OpenCV is capable of doing the job at high speed.
Hope it helps. Let me know if you need further help.
Edits:
If you want to find the actual depths, you need to project the coordinate system of your pictures to the real world, or vice versa. To do it, you have to know a fix location as your reference and the relationship between pixels and real distances.
What I did is find the fixed location and set it as zero. Afterwards, I measured a length of an object in the picture, and also calculated the pixel amount of the object. Therefore I obtained the relationship between pixels and real distances.
Note that these procedures may involve errors in the identification. I did it very carefully and the error was acceptable in my case.
With only one image, accurate depth estimation is near impossible. However, there are various methods of estimating depth under certain assumptions or the availability of the camera calibration matrix. As mentioned by #WenlongLiu, OpenCV is a very good place to start with.
My question is rather about feasibility of a task.
Note that I have read the solution of this question, however you can guess I am not dealing with rectangles and cameras here.
Situation:
I need to save lot of pictures in a folder all of them obeying to these rules:
In each picture, there is ONLY one object.
The object can be anything (car, horse, human hand ...)
The size and the format of the picture belong to certain set.
The background of the object is ALWAYS white.
The color of the object itself can be anything else (including, why not, areas of white pixels)
Goal:
I want to detect if the object of each image is CENTERED.
Development environment:
Python
OpenCV
Do you think this is feasible ?
I hope my question is not too broad. I just ask if this can be done automatically without human intervention on the pictures. I have thousands of them. The program will save in a separate folder pictures in which the object is not centered.
EDIT:
Following the comments and answer above: for me, a centered object is the one if I draw a square or rectangle around it, the edges of the square/rectangle must be equivalently distant from let and right sides of the image, whereas the top and the bottom of the object must be equivalently distant from the top and bottom of the picture.
Yep this is very feasible. However, depending on the type of objects the images contain, they are different ways to accomplish this. Assuming the objects in the images all have a uniform color you can easily perform a color detection algorithm, find the centre point of the object in terms of pixels and find it's position using the image resolution as the reference.
As the background is always white as specified, this is probably your best method as you can just extract all the non white (Or different shade of white) objects within the image.
if you do decide to go with this approach, i should be able to point you to some relevant code
Although writing in c++, more information on this can be found in the link below.
http://opencv-srf.blogspot.co.uk/2010/09/object-detection-using-color-seperation.html
the link is based on object detection in a video but as a video is just a series images the same concept can be used on images