I attach a zip archive with all the files needed to illustrate and reproduce the problem.
(I don't have permissions to upload images yet...)
I have an image (test2.png in the zip archive ) with curved lines.
I try to warp it so the lines are straight.
I thought of using scikit-image transform, and in particular transform.PolynomialTransform because the transformation involves high order distortions.
So first I measure the precise position of each line at regular intervals in x to define the input interest points (in the file source_test2.csv).
Then I compute the corresponding desired positions, located along a straight line (in the file destination_test2.csv).
The figure correspondence.png shows how it looks like.
Next, I simply call transform.PolynomialTransform() using a polynomial of order 3.
It finds a solution, but when I apply it using transform.warp(), the result is crazy, as illustrated in the file Crazy_Warped.png
Anybody can tell what I am doing wrong?
I tried polynomial of order 2 without luck...
I managed to get a good transformation for a sub-image (the first 400 columns only).
Is transform.PolynomialTransform() completely unstable in a case like mine?
Here is the entire code:
import numpy as np
import matplotlib.pyplot as plt
import asciitable
import matplotlib.pylab as pylab
from skimage import io, transform
# read image
orig=io.imread("test2.png",as_grey=True)
# read tables with reference points and their desired transformed positions
source=asciitable.read("source_test2.csv")
destination=asciitable.read("destination_test2.csv")
# format as numpy.arrays as required by scikit-image
# (need to add 1 because I started to count positions from 0...)
source=np.column_stack((source["x"]+1,source["y"]+1))
destination=np.column_stack((destination["x"]+1,destination["y"]+1))
# Plot
plt.imshow(orig, cmap='gray', interpolation='nearest')
plt.plot(source[:,0],source[:,1],'+r')
plt.plot(destination[:,0],destination[:,1],'+b')
plt.xlim(0,orig.shape[1])
plt.ylim(0,orig.shape[0])
# Compute the transformation
t = transform.PolynomialTransform()
t.estimate(destination,source,3)
# Warping the image
img_warped = transform.warp(orig, t, order=2, mode='constant',cval=float('nan'))
# Show the result
plt.imshow(img_warped, cmap='gray', interpolation='nearest')
plt.plot(source[:,0],source[:,1],'+r')
plt.plot(destination[:,0],destination[:,1],'+b')
plt.xlim(0,img_warped.shape[1])
plt.ylim(0,img_warped.shape[0])
# Save as a file
io.imsave("warped.png",img_warped)
Thanks in advance!
There are a couple of things wrong here, mainly they have to do with coordinate conventions. For example, if we examine the code where you plot the original image, and then put the clicked point on top of it:
plt.imshow(orig, cmap='gray', interpolation='nearest')
plt.plot(source[:,0],source[:,1],'+r')
plt.xlim(0,orig.shape[1])
plt.ylim(0,orig.shape[0])
(I've taken out the destination points to make it cleaner) then we get the following image:
As you can see, the y-axis is flipped, if we invert the y-axis with:
source[:,1] = orig.shape[0] - source[:,1]
before plotting, then we get the following:
So that is the first problem (don't forget to invert the destination points as well), the second has to do with the transform itself:
t.estimate(destination,source,3)
From the documentation we see that the call takes the source points first, then the destination points. So the order of those arguments should be flipped.
Lastly, the clicked points are of the form (x,y), but the image is stored as (y,x), so we have to transpose the image before applying the transform and then transpose back again:
img_warped = transform.warp(orig.transpose(), t, order=2, mode='constant',cval=float('nan'))
img_warped = img_warped.transpose()
When you make these changes, you get the following warped image:
These lines aren't perfectly flat but it makes much more sense.
Thank you very much for the detailed answer! I cannot believe I did not see the axis inversion problem... Thanks for catching it!
But I am afraid your final solution does not solve my problem... The image you get is still crazy. It should be continuous, no have such big holes and weird distortions... (see final solution below)
I found I could get a reasonable solution using RANSAC:
from skimage.measure import ransac
t, inliers = ransac((destination,source), transform.PolynomialTransform, min_samples=20,residual_threshold=1.0, max_trials=1000)
outliers = inliers == False
I then get the following result
Note that I think I was right using (destination,source) in that order! I think it has to do with the fact that transform.warp requires the inverse_map as input for the transformation object, not the forward map. But maybe I am wrong? The good result I am getting suggest it's correct.
I guess that Polynomial transforms are too unstable, and using RANSAC allows to get a reasonable solution.
My problem was then to find a way to change the polynomial order in the RANSAC call...
transform.PolynomialTransform() does not take any parameters, and uses by default a 2nd order polynomial, but from the result I can see I would need a 3rd or 4th order polynomial.
So I opened a new question, and got a solution from Stefan van der Walt. Follow the link to see how to do it.
Thanks again for your help!
Related
I am trying to rotate a picture with 11 channels using scipy.ndimage.rotate.
It is stored as an array in the variable im_arr and I use the following line to rotate it:
im_arr = ndimage.rotate(im_arr,-angle)
Yet when I execute the script I get the following error message:
ValueError: negative dimensions are not allowed
I can't quite understand what this means or what this is supposed to tell me. Nor do I know how to solve it. I had the same error message beforehand when I used ndimage.zoom to resize it. The issue there was from my understanding that by using a positive factor for the zoom the picture would have had negative indices for some reason. This was simply solved by putting a - in front of the second zoom factor.
dim = (im_new_x, -im_new_y,1)
im_arr = ndimage.zoom(im_arr,dim)
In this case this is not possible though. No matter what I do here I can't think of a way to bypass this issue.
I am experimenting with some computer vision techniques, specifically feature detection. I am trying to identify features by conducting auto-correlation between an image and a feature-kernel.
However, the resulting correlation-matrix doesn't make sense to me... Can anyone help me understand how to interpret or visualize this matrix, so that it's apparent where the feature is located?
Feature Kernel:
Original Image:
Code:
import cv2
import pprint
import numpy
import scipy.ndimage
from matplotlib import pyplot as plt
import skimage.feature
# load the image
img = cv2.imread('./lenna.jpg')[:,:,0]
f_kernel = cv2.imread('./lenna_feature.jpg')[:,:,0]
def matched_filter(img, f_kernel, detect_thres):
result = scipy.ndimage.correlate(img, f_kernel)
print("Feature Match Template")
plt.imshow(skimage.feature.match_template(img, f_kernel))
plt.show()
return result
plt.imshow(matched_filter(img,f_kernel,1))
print("Correlation Matrix")
plt.show()
Result:
So, in the first result-image, there's an obvious maximum-point at (150,200). I am interpreting this as the most likely location of the feature.
However, in the second result-image, correlation matrix result, there is no obvious pattern. I was expecting that there would be some, obvious high-correlation point.
Help?
skimage.feature.match_template computes the normalized cross correlation. That is, for each location over the image, the image patch and the template are normalized (subtract mean and divide by standard deviation) and then multiplied together and averaged. This computes the correlation coefficient of the image patch and the template. The correlation coefficient is a value between 1 and -1. A correlation coefficient of 1 indicates that the image patch is a linear modification of the template (i.e. constant1 * template + constant2).
scipy.ndimage.correlate computes the correlation (same as convolution, but without mirroring the kernel). That is, here we do not first normalize the image patch. Places where the image has higher values will automatically also have a higher correlation, even if not at all similar to the template.
I can't reproduce your result. For me, the second image has a shape that looks a bit like Lena.
Anyway, you don't want to use the correlation, you want to use the correlation coefficient to do template matching. Pure correlation is not normalized, so it's rather an averaging filter than a template matching.
Edit: Added the corellation image
I am trying to select an area of an image to do some analysis on that specific area of the image.
However, when I searched online, I am only able to find guides on how to select a rectangular area. I need to select an area that is drawn using my mouse. An example of such area is included bellow.
Would anyone be able to recommend to me some key words or libraries to search to help me with this or links to guides that do something similar?
Also, I am not sure if it is necessary information but the analysis I am trying to do on the region of interest is to Find a ratio of amount of white to black pixels in that specific area.
I produced a simple working example based on this answer. I also tried using scipy.ndimage.morphology.fill_binary_holes, but could not get it to work. Note that the provided function takes a little longer since it is assuming that the input image is grayscale and not binarized.
I specifically avoided the usage of OpenCV, since I find the setup to be a bit tedious, but I think it should also provide an equivalent (see here).
Additionally, my "binarization" is kind of hacky, but you can probably figure out how to parse your image into a valid format yourself (and it might be easier if you produce the result within a program). In any case, I would suggest making sure that you have a proper image format, since jpeg's compression might violate your connectivity, and cause issues in certain cases.
import scipy as sp
import numpy as np
import scipy.ndimage
import matplotlib.pyplot as plt
def flood_fill(test_array,h_max=255):
input_array = np.copy(test_array)
el = sp.ndimage.generate_binary_structure(2,2).astype(np.int)
inside_mask = sp.ndimage.binary_erosion(~np.isnan(input_array), structure=el)
output_array = np.copy(input_array)
output_array[inside_mask]=h_max
output_old_array = np.copy(input_array)
output_old_array.fill(0)
el = sp.ndimage.generate_binary_structure(2,1).astype(np.int)
while not np.array_equal(output_old_array, output_array):
output_old_array = np.copy(output_array)
output_array = np.maximum(input_array,sp.ndimage.grey_erosion(output_array, size=(3,3), footprint=el))
return output_array
x = plt.imread("test.jpg")
# "convert" to grayscale and invert
binary = 255-x[:,:,0]
filled = flood_fill(binary)
plt.imshow(filled)
This produces the following result:
Basically, i have a corpus of ~10,000 STL files, and i need to turn them all into 32x32x32 arrays of 1's and 0's (voxels)
I already have this script that turns STL files into voxels; https://github.com/rcpedersen/stl-to-voxel , but sometimes even though i specify that i need a 32x32x32 array, it will give me some huge array, and also along with being buggy, it takes FOREVER (processed ~600 files in 48 hours...)
Would it be easier to attempt to fix this script, or to write my own? It doesnt seem like voxelizing an STL would be a hard task, but I don't know any of the methods out there for this; if there are any strategies/tips, anything would be greatly appreciated.
Sorry to be a bummer, but voxelisation is actually quite a hard task. And not something Python is suitable to do quickly. Even for the simple slice/crossing test I would think a c++ implementation will beat python 1:100. I recommend libigl. Or do it on the GPU for realtime :) Look for conservative rasterization. But that is for "good" meshes that are non intersecting and closed. Otherwise it becomes a lot harder. Look for "generalized winding numbers" - also in igl.
Basicly voxelizing facet surface means separation inside form outside. It can be done in different ways: easiest way is to find signed distance from each voxel but it requeres input mesh to be closed, other way is to find winding number. You can find implemetation of both in MeshLib. Also there is python module that can help you:
pip install --upgrade pip
pip install meshlib
from meshlib import mrmeshpy as mm
# load mesh
mesh = mm.loadMesh(mm.Path("path_to_file.stl"))
mtvParams = mm.MeshToVolumeParams()
# signed will have negative values inside mesh and positive outside, but requires closed mesh
mtvParams.type = mm.MeshToVolumeParamsType.Signed
# voxels with presice distance - 3 inside, 3 - outside
mtvParams.surfaceOffset = 3
# find correct voxel size to have 32x32x32 volume
meshBox = mesh.computeBoundingBox()
boxSize = meshBox.max-meshBox.min
mtvParams.voxelSize = boxSize / 27.0
voxels = mm.meshToVolume(mesh,mtvParams)
# save voxels as tiff slices
vsParams = mm.VoxelsSaveSavingSettings()
vsParams.path = "save_voxels_dir"
vsParams.slicePlane = mm.SlicePlane.XY
mm.saveAllSlicesToImage(voxels,vsParams)
Is it possible for matplotlib only update the newest point to the figure instead of re-draw the whole figure?
For example: this may be the fastest way for dynamic plotting
initiate:
fig1 = Figure(figsize = (8.0,8.0),dpi = 100)
axes1 = fig1.add_subplot(111)
line1, = axes1.plot([],[],animated = True)
when new data is coming:
line1.set_data(new_xarray,new_yarray)
axes1.draw_artist(line1)
fig1.canvas.update()
fig1.canvas.flush_events()
But this will re-draw the whole figure! I'm think whether this is possible:
when new data is coming:
axes1.draw_only_last_point(new_x,new_y)
update_the_canvas()
It will only add this new point(new_x,new_y) to the axes instead of re-draw every point.
And if you know which graphic library for python can do that, please answer or comment, thank you so much!!!!!
Really appreciate your help!
Is only redrawing the entire figure the problem, i.e. it is ok to redraw the line itself as long as the figure is unchanged? Is the data known beforehand?
If the answer to those questions are NO, and YES, then it might be worth looking into the animate-class for matplotlib. One example where the data is known beforehand, but the points are plotted one by one is this example. In the example, the figure is redrawn if the newest point is outside of the current x-lim. If you know the range of your data you can avoid it by setting the limits beforehand.
You might also want to look into this answer, the animate example list or the animate documentation.
this is my (so far) little experience.
I started some month ago with Python(2.x) and openCV (2.4.13) as graphic library.I found in may first project that openCV for python works with numpy structure as much as matplotlib and (with slight difference) they can work together.
I had to update some pixel after some condition. I first did my elaboration from images with opencv obtaining a numpy 2D array, like a matrix.
The trick is: opencv mainly thinks about input as images, in terms of X as width first, then Y as height. The numpy structure wants rows and columns wich in fact is Y before X.
With this in mind I updated pixel by pixel the image-matrix A and plot it again with a colormap
import matplotlib as plt
import cv2
A = cv2.imread('your_image.png',0) # 0 means grayscale
# now you loaded an image in a numpy array A
for every new x,y pixel
A[y,x] = new pixel intensity value
plot = plt.imshow(A, 'CMRmap')
plt.show()
If you want images again, consider use this
import matplotlib.image as mpimg
#previous code
mpimg.imsave("newA.png", A)
If you want to work with colors remember that images in colour are X by Y by 3 numpy array but matplotlib has RGB as the right order of channels, openCv works with BGR order. So
C = cv2.imread('colour_reference.png',1) # 1 means BGR
A[y,x,0] = newRedvalue = C[y,x][2]
A[y,x,1] = newGreenvalue = C[y,x][1]
A[y,x,2] = newBluevalue = C[y,x][0]
I hope this will help you in some way