I'm looking for OpenCV or other Python function that displays a NumPy array as an image like this:
Referenced from this tutorial.
What function creates this kind of grey-scale image with pixel values display?
Is there a color image equivalent?
MATLAB has a function called showPixelValues().
The best way to do this is to search "heat map" or "confusion matrix" rather than image, then there are two good options:
Using matplotlib only, with imshow() and text() as building blocks the solution is actually not that hard, and here are some examples.
Using seaborn which is a data visualisation package and the solution is essentially a one-liner using seaborn.heatmap() as shown in these examples.
My problem was really tunnel vision, coming at it from an image processing mindset, and not thinking about what other communities have a need to display the same thing even if they call it by a different name.
is there a python version for https://www.mathworks.com/help/images/sliding-neighborhood-operations.html, or a way to do it efficiently?
I want to be able to fit a custom sized window say [2x2] over an image/array, and apply a custom function to the values on that window. For example, like the matlab page, it could be setting the max of the surrounding numbers, though i know a convolution can do that. Specifically, I want to apply https://python-colormath.readthedocs.io/en/latest/delta_e.html the delta2000 function to each pixel and its neighbors and see the results.
I have not been able to find anything.
For discrete convolution operations which are the most common sliding neighborhood methods (to apply filters on images like gaussian, blurring, e.t.c), OpenCV is a good library (and for any image processing task).
You should check also Numpy, which is used to operate with arrays (for instance, 2D or 3D images) and works fine with OpenCV
Here is a cropped example (about 11x9 pixels) of the kind of images (which ultimately are actually all of size 28x28, but stored in memory flattened as a 784-components array) I will be trying to apply the algorithm on:
Basically, I want to be able to recognize when this shape appears (red lines are used to put emphasis on the separation of the pixels, while the surrounding black border is used to better outline the image against the white background of StackOverflow):
The orientation of it doesn't matter: it must be detected in any of its possible representations (rotations and symmetries) along the horizontal and vertical axis (so, for example, a 45° rotation shouldn't be considered, nor a diagonal symmetry: only consider 90°, 180°, and 270° rotations, for example).
There are two solutions to be found on that image that I first presented, though only one needs to be found (ignore the gray blurr surrounding the white region):
Take this other sample (which also demonstrates that the white figures inside the images aren't always fully surrounded by black pixels):
The function should return True because the shape is present:
Now, there is obviously a simple solution to this:
Use a variable such as pattern = [[1,0,0,0],[1,1,1,1]], produce its variations, and then slide all of the variations along the image until an exact match is found at which point the whole thing just stops and returns True.
This would, however, in the worst case scenario, take up to 8*(28-2)*(28-4)*(2*4) which is approximately 40000 operations for a single image, which seem a bit overkill (if I did my quick calculations right).
I'm guessing one way of making this naive approach better would be to first of all scan the image until I find the very first white pixel, and then start looking for the pattern 4 rows and 4 columns earlier than that point, but even that doesn't seem good enough.
Any ideas? Maybe this kind of function has already been implemented in some library? I'm looking for an implementation or an algorithm that beats my naive approach.
As a side note, while kind of a hack, I'm guessing this is the kind of problem that can be offloaded to the GPU but I do not have much experience with that. While it wouldn't be what I'm looking for primarily, if you provide an answer, feel free to add a GPU-related note.
EDIT:
I ended up making an implementation of the accepted answer. You can see my code in this Gist.
If you have too many operations, think how to do less of them.
For this problem I'd use image integrals.
If you convolve a summing kernel over the image (this is a very fast operation in fft domain with just conv2,imfilter), you know that only locations where the integral is equal to 5 (in your case) are possible pattern matching places. Checking those (even for your 4 rotations) should be computationally very fast. There can not be more than 50 locations in your example image that fit this pattern.
My python is not too fluent, but this is the proof of concept for your first image in MATLAB, I am sure that translating this code should not be a problem.
% get the same image you have (imgur upscaled it and made it RGB)
I=rgb2gray(imread('https://i.stack.imgur.com/l3u4A.png'));
I=imresize(I,[9 11]);
I=double(I>50);
% Integral filter definition (with your desired size)
h=ones(3,4);
% horizontal and vertical filter (because your filter is not square)
Ifiltv=imfilter(I,h);
Ifilth=imfilter(I,h');
% find the locations where integral is exactly the value you want
[xh,yh]=find(Ifilth==5);
[xv,yv]=find(Ifiltv==5);
% this is just plotting, for completeness
figure()
imshow(I,[]);
hold on
plot(yh,xh,'r.');
plot(yv,xv,'r.');
This results in 14 locations to check. My standard computer takes 230ns on average on computing both image integrals, which I would call fast.
Also GPU computing is not a hack :D. Its the way to go with a big bunch of problems because of the enormous computing power they have. E.g. convolutions in GPUs are incredibly fast.
The operation you are implementing is an operator in Mathematical Morphology called hit and miss.
It can be implemented very efficiently as a composition of two erosions. If the shape you’re detecting can be decomposed into a few simple geometrical shapes (especially rectangles are quick to compute) then the operator can be even more efficient.
You’ll find very efficient erosions in most image processing libraries, for example try OpenCV. OpenCV also has a hit and miss operator, here is a tutorial for how to use it.
As an example for what output to expect, I generated a simple test image (left), applied a hit and miss operator with a template that matches at exactly one place in the image (middle), and again with a template that does not match anywhere (right):
I did this in MATLAB, not Python, because I have it open and it's easiest for me to use. This is the code:
se = [1,1,1,1 % Defines the template
0,0,0,1];
img = [0,0,0,0,0,0 % Defines the test image
0,1,1,1,1,0
0,0,0,0,1,0
0,0,0,0,0,0
0,0,0,0,0,0
0,0,0,0,0,0];
img = dip_image(img,'bin');
res1 = hitmiss(img,se);
res2 = hitmiss(img,rot90(se,2));
% Quick-and-dirty display
h = dipshow([img,res1,res2]);
diptruesize(h,'tight',3000)
hold on
plot([5.5,5.5],[-0.5,5.5],'r-')
plot([11.5,11.5],[-0.5,5.5],'r-')
The code above uses the hit and miss operator as I implemented in DIPimage. This same implementation is available in DIPlib's Python bindings as dip.HitAndMiss() (install with pip install diplib):
import diplib as dip
# ...
res = dip.HitAndMiss(img, se)
I have thousands of polygons given their 4 corner coordinates (quadrilaterals) and would like to convert them to a raster representation as a numpy 2d array.
A lot of gridding algorithms exist like the popular scanline fill in graphics. (see http://www.cs.rit.edu/~icss571/filling/how_to.html or http://cs.uvm.edu/~rsnapp/teaching/cs274/lectures/scanlinefill.pdf )
Octave implements this in the poly2mask function (e.g. http://octave.sourceforge.net/image/function/poly2mask.html).
Is there a similar function also in Numpy?
I still don't get how this algorithms works in detail and, thus, I would be very grateful if you can give me some hints on how to implement it in Python/Numpy efficiently.
Or would it be better to code it in CPython (which I am not familiar with either) for speed reasons?
There are a few different functions for this in the scipy ecosystem (in no order):
1) The most widely-available option is to use matplotlib's points_inside_poly. However, it's very suboptimal for filling a regular grid (i.e. it's an explicit point in polygon test, rather than a "scanline" approach).
2) mahotas implements a fill_polygon function that's quite efficient: http://mahotas.readthedocs.org/en/latest/polygon.html#drawing
3) skimage (scikits-image) implements a draw.polygon function that should be at least as efficient, if not more so: http://scikit-image.org/docs/dev/api/skimage.draw.html#skimage.draw.polygon
4) Finally, you can also use PIL for this and convert the image to a numpy array. Have a look at the ImageDraw module: http://effbot.org/imagingbook/imagedraw.htm
Overally, I'd reccommend installing skimage and using it. It's a very useful library. However, if you can't install scikits image for some reason, the other options should help.
OpenCVproject also has polygon fill function: cv2.fillPoly
does anybody have an idea how to normalize the
scipy.ndimage.filters.correlate
function to get :
XCM = 1/N(xc(a-mu_a,b-mu_b)/(sig_a*sig_b))
What is N for the correlation? It usually is the # of datapoints / pixels for images.
Which value shall I choose for scipy.ndimage.filters.correlate?
My images differ in size. I guess the scipy correlate function pads the small image into zeros?
The size of the final matrix
N = XCM.sizeX() * XCM.sizeY()
?
Thanks,
El
It looks to me like you're trying to compute the normalized cross-correlation of two images (I suspect you're probably trying to do template matching?). This answer assumes that the normalized cross-correlation is what you want.
When you compute the normalized cross-correlation between your two images, you're doing the equivalent of subtracting the mean and dividing by the standard deviation to both your template and reference image in the region where they overlap.
Here, N would be equal to the number of pixels in your template, which is the same as the number of pixels in the local region of overlap between the template and the reference image as you slide the template over the reference.
You should read the Wikipedia article on cross-correlation, and in particular this bit for the definition of normalized cross-correlation and some explanation for what each of the terms mean.
This article by Lewis (1995) has a more in-depth explanation, and also describes some neat tricks for efficiently computing the normalized cross-correlation.
I also wrote my own Python functions for template matching including normalized cross-correlation based on Lewis and some snippets of MATLAB. You can find the source here.
Let me know if you have more questions and I'll have a go at explaining.
Normalized Cross-Correlation (NCC) is also included in scikit-image as skimage.feature.match_template. See this template matching example.
You can also do the same with OpenCV with the matchTemplate method. There are many good bindings from Python to OpenCV, but it's a bit overkill if you only need template matching. I'd go with scikit-image.