Develop Reference

Coordinate transformation using affine method

I am trying to convert coordinates from one system to another using the affine transfomation method on python. the code seems to be running but the output is looking strange, here is my code:
import json
import math
import numpy as np
labels = open('labels3.txt')
read_ready = labels.read()
JSONfile = json.loads(read_ready)
#input_system should be the ground plane(georefrenced) images
#notes on this set of coordinates, assuming that the points arent rotated, may introduce the rotation in # the very end
input_sys = np.array([[0,0,0],[0,23133,0],[35093,23133,0],[35093,0,0]])
#output system should be the slant range coords, these arent too special should be pretty straightforward
output_sys = np.array([[0,0,0],[0,16384,0],[16384,16384,0],[16384,0,0]])
#now find which point should be transformed
p = np.array([7954,9338,0])
#finding the transformation
l = len(input_sys)
entry = lambda r,d: np.linalg.det(np.delete(np.vstack([r,input_sys.T,np.ones(l)]),d,axis = 0))
M = np.array([[(-1)**i * entry(R,i) for R in output_sys.T] for i in range(l+1)])
A,t = np.hsplit(M[1:].T/(-M[0])[:,None], [l-1])
t = np.transpose(t)[0]
#output transform
print("Affine transformation matrix:\n", A)
print("Affine transformation translation vector:\n", t)
#print("Testing:")
#for p,P in zip(np.array(input_sys), np.array(output_sys)):
#image_p = np.dot(A,p)+t
#result ="[OK]" if np.allclose(image_p,P) else"[ERROR]"
#print(p, " mapped to: ", image_p, " ; expected: ", P, result)
#print('Calculation:')
P = np.dot(A,p)+t
print(p,'mapped to:', P)
the output for the matrix M is a zero matrix which is not correct , can someone help me find a way to get a nonzero matrix out for M?

Python OpenCV Get Angle Direction of fitLine

Using Python OpenCv, How does one find the Angle of fitLine through an element?
I am trying to use debugger and locate, since I do not see it here in documentation ,
rows,cols = img.shape[:2]
[vx,vy,x,y] = cv2.fitLine(cnt, cv2.DIST_L2,0,0.01,0.01)
lefty = int((-x*vy/vx) + y)
righty = int(((cols-x)*vy/vx)+y)
img = cv2.line(img,(cols-1,righty),(0,lefty),(0,255,0),2)
Fitline References:
Fitline Documentation
Open CV Tutorial
Not sure which items to acquire for slope in debugger

As you already have a unit vector that defines the direction of your line [vx, vy] from,
[vx,vy,x,y] = cv2.fitLine(cnt, cv2.DIST_L2,0,0.01,0.01)
Then if you are looking for the angle between your fitted line & the x axis you can use the dot product to get the angle,
import numpy as np
x_axis = np.array([1, 0]) # unit vector in the same direction as the x axis
your_line = np.array([vx, vy]) # unit vector in the same direction as your line
dot_product = np.dot(x_axis, your_line)
angle_2_x = np.arccos(dot_product)

Crop a pattern-based area of multiple images in python

I have a big number of screenshots that need to be cropped. All the images look similar - there is a rectangular window with blue border, containing some graphical elements inside. This window is contained inside another one but I need to crop only the inner window. Across all images the dimensions of the inner window are different and so is the content. The content in most cases includes elements with rectangular form and sometimes - blue border, the same border as the inner window. I am mentioning this because I am thinking of the following flow:
A script that goes through all images in the target directory. For each of them:
Find the area to be cropped (inner window)
Crop the area
Save the file
How can this be done? Python is not compulsory, can be any other too also.

It's not straightforward but this is a possible recipe:
import matplotlib.pyplot as plt
import numpy as np
def synthimage():
w,h = 300,200
im = np.random.randint(0,255,(w,h,3))/255
xa = np.random.randint(50,w-60)
xb = xa + np.random.randint(50,90)
ya = np.random.randint(50,h-60)
yb = ya + np.random.randint(20,50)
im[xa:xb,ya] = np.array([1,0,0])
im[xa:xb,yb] = np.array([1,0,0])
im[xa,ya:yb] = np.array([1,0,0])
im[xb,ya:yb] = np.array([1,0,0])
return im
def getRectPoints(im):
x,y = [],[]
for i in range(im.shape[0]):
for j in range(im.shape[1]):
if (im[i,j]-np.array([1,0,0])).sum()==0:
x.append(i)
y.append(j)
return np.array(x),np.array(y)
def denoise(x,y):
nx,ny = [],[]
for i in range(x.shape[0]):
d = np.sqrt((x[i]-x)**2+(y[i]-y)**2)
m = d<2
if len(m.nonzero()[0])>2:
nx.append(x[i])
ny.append(y[i])
return np.array(nx),np.array(ny)
im = synthimage()
plt.imshow(np.swapaxes(im,0,1),origin='lower',interpolation='nearest')
plt.show()
x,y = getRectPoints(im)
plt.scatter(x,y,c='red')
plt.xlim(0,300)
plt.ylim(0,200)
plt.show()
nx,ny = denoise(x,y)
plt.scatter(nx,ny,c='red')
plt.xlim(0,300)
plt.ylim(0,200)
plt.show()
#Assuming rectangle has no rotation (otherwise check Scipy ConveHull)
xmi = nx.min()
xma = nx.max()
ymi = ny.min()
yma = ny.max()
new = np.ones(im.shape)
new[xmi:xma,ymi:yma] = im[xmi:xma,ymi:yma]
plt.imshow(np.swapaxes(new,0,1),origin='lower',interpolation='nearest')
plt.show()
, the name of the functions should be self-explaining. Synthetic data was generated for the purpose of this exercise. The results are (in order):
Obviously each one of this steps can be changed depending on the requirements but this would be a functional solution for the majority of case-studies.

How to remove rings from convolve healpix map?

I'm applying convolution techniques to convolve 2 datasets, a healpix map with nside = 256 and a primary beam of shape (256, 256) in order to measure the total intensity from the convolved healpix map. My problem is that after convolving my map with the primary beam i get rings in my convolved map. I've tried normalizing it with either lanczos or Gaussian kernel to take care of the rings but all these approaches have failed.
In my code below, i used the query function in scipy to search for the nearest pixels in my healpix map within a given radius and take the sum of the product of the corresponding pixels in the primary beam using map coordinate. The final image i get has rings in it. Please can anyone help me solve this problem? Thanks in advance.
def query_npix(nside, npix, radius):
print 'searching for nearest pixels:......'
t1, t2 = hp.pix2ang(nside, np.arange(npix))
tree = spatial.cKDTree(zip(t1, t2))
dist, ipix_indx = tree.query(zip(t1, t2), k = 150, distance_upper_bound = radius)
r1, r2 = hp.pix2ang(nside, ipix_indx)
ra = r1.T - t1
dec = r2.T - t2
print 'Done searching'
return np.array(dist), np.array(ipix_indx), np.array(ra.T), np.array(dec.T)
def fullSky_convolve(healpix_map, primary_beam_fits, ipix_indx, dist, radius, r1, r2):
measured_map = []
hdulist = openFitsFile(primary_beam_fits)
beam_data = hdulist[0].data
header = hdulist[0].header
nside = hp.get_nside(healpix_map[0, ...])
npix = hp.get_map_size(healpix_map[0, ...]) # total number of pixels in the map must be 12 * nside^2
crpix1, crval1, cdelt1 = [ header.get(x) for x in "CRPIX1", "CRVAL1", "CDELT1" ]
crpix2, crval2, cdelt2 = [ header.get(x) for x in "CRPIX2", "CRVAL2", "CDELT2" ]
# beam centres in pixel coordinates
xc = crpix1-1 + (np.rad2deg(r1.ravel()) - crval1)/(256*cdelt1)
yc = crpix2-1 + (np.rad2deg(r2.ravel()) - crval2)/(256*cdelt2)
#xc = (np.rad2deg(r1.ravel()) )/cdelt1
for j in xrange(4):
print 'started Stokes: %d' %j
for iter in xrange(0 + j, 16, 4):
outpt = np.zeros(shape = npix, dtype=np.float64)
#by = outpt.copy()
# mask beam
bm_data = beam_data[iter]
#masked_beam= beam_data[iter]
shape = bm_data.shape
rad = np.linspace(-shape[0]/2,shape[-1]/2,shape[0])
rad2d = np.sqrt(rad[np.newaxis,:]**2+rad[:,np.newaxis]**2)
mask = rad2d <= radius/abs(cdelt2)
masked_beam = bm_data*mask
s1 = ndimage.map_coordinates(masked_beam, [xc, yc], mode = 'constant')
bm_map = s1.reshape(dist.shape[0], dist.shape[-1])
for itr in xrange(npix):
g_xy = (1.0/(np.sqrt(2*np.pi)*np.std(dist[itr])))*np.exp(-(dist[itr])**2/(2*np.var(dist[itr])))
#weighted_healpix_map = np.convolve(healpix_map[j, ...][ipix_indx[itr]], g_xy/g_xy.sum(), mode='same')
weighted_healpix_map = ndimage.filters.convolve(healpix_map[j, ...][ipix_indx[itr]], g_xy/g_xy.sum(), mode='reflect')
#outpt[itr] = np.sum(weighted_healpix_map*(bm_map[itr]/bm_map[itr].sum()))
outpt[itr] = np.sum(weighted_healpix_map*(bm_map[itr]))
#print 'itr', itr
alpha = file('pap%d.save'%iter, 'wb')
#h_map = ndimage.filters.gaussian_filter(outpt, sigma = 3.)
cPickle.dump(outpt, alpha, protocol = cPickle.HIGHEST_PROTOCOL)
alpha.close()
print 'Just dumped stripp%d.save:-------'%iter
print 'Loading dumped files:-------'
loaded_objects = []
for itr4 in xrange(16):
alpha = file('stripp%d.save'%itr4, 'rb')
loaded_objects.append(cPickle.load(alpha))
alpha.close()
measured_map.append(copy.deepcopy(loaded_objects))
return measured_map

Remember that HEALPix maps can be in either "Ring" or "Nested" format. It sounds like you may need to add the keyword nest=True to your healpy functions like hp.pix2ang. If your input maps are in nested format, this keyword is needed.
For example:
I recently tried using the healpy.smoothing() function, and found my resulting image to have rings (perhaps like you described), upon viewing the output map with healpix.mollview(). The rings disappeared and the image was presented as I expected, after running mollview with the nested=True keyword. Check what ordering schemes your input files use
Reference:
http://healpy.readthedocs.org/en/latest/tutorial.html#creating-and-manipulating-maps
Healpix supports two different ordering schemes, RING or NESTED. By
default, healpy maps are in RING ordering. In order to work with
NESTED ordering, all map related functions support the nest keyword,
for example: hp.mollview(m, nest=True, title="Mollview image NESTED")

3d skeleton from segmentation

I want to create a skeleton based on an existing segmentation, similar to what is done here (from sk-image):
However I want to do this on 3D data. Is there code for that somewhere out there? Preferably in python but any language helps.
I am aware of this great site, however I think they don't offer any code.
I am planning on using that on volumes of about 500x500x500 Pixels, so it should scale well...

I am developing this tools in this link below. The function getSkeletonize3D in program of the name convOptimize.py lets you thin your 3D data. It took about 30 minutes to give the result for the 512 cube I have. Let me know if you have any problems. https://github.com/3Scan/3scan-skeleton. The paper I used for implementing is in the comments in the code below
Basically how this 3D skeletonization algorithm works is, in each pass it has 12 subiterations in which it removes boundaries in specific directions iteratively, until you get a skeleton in the center.
The main python code that is needed for skeletonizing your data is as below. As it requires imports from different other porgrams rotationalOperators which has an import from another file called Thin3dtemplates. I recommend you downlaod rotationalOperators, Thin3dtemplates, convoptimize python scripting files and also download lookuparray.npy which is a file that is used as a lookup table in an numpy array format pre-calculated for validating a voxel for marking to be deleted or not. You need python > 3 version, scipy, numpy and pyeda modules installed to run these codes.
import numpy as np
import time
from scipy import ndimage
from scipy.ndimage.filters import convolve
"""
the following subiteration functions are how each image is rotated to the next direction for removing
boundary voxels in the order described in the reference paper
us, ne, wd,..
"""
from rotationalOperators import firstSubiteration, secondSubiteration, thirdSubiteration, fourthSubiteration, fifthSubiteration, sixthSubiteration, seventhSubiteration, eighthSubiteration, ninthSubiteration, tenthSubiteration, eleventhSubiteration, twelvethSubiteration
"""
reference paper
http://web.inf.u-szeged.hu/ipcg/publications/papers/PalagyiKuba_GMIP1999.pdf
input should be a binary image/ already segmented
"""
"""
array that has calculated the validity of the 14 templates beforehand and stored each index which is
decimal number of the binary string of 26 values (sqrt(3) connectivity) that are around a single voxel
"""
lookUpTablearray = np.load('lookupTablearray.npy')
def _convolveImage(arr, flippedKernel):
arr = np.ascontiguousarray(arr, dtype=np.uint64)
result = convolve(arr, flippedKernel, mode='constant', cval=0)
result[arr == 0] = 0
return result
"""
each of the 12 iterations corresponds to each of the following
directions - us, ne, wd, es, uw, nd, sw, un, ed, nw, ue, sd
imported from template expressions
evaluated in advance using pyeda
https://pyeda.readthedocs.org/en/latest/expr.html
"""
sElement = ndimage.generate_binary_structure(3, 1)
def _getBouondariesOfimage(image):
"""
function to find boundaries/border/edges of the array/image
"""
erode_im = ndimage.morphology.binary_erosion(image, sElement)
boundaryIm = image - erode_im
return boundaryIm
"""
each of the 12 iterations corresponds to each of the following
directions - us, ne, wd, es, uw, nd, sw, un, ed, nw, ue, sd
imported from template expressions
evaluated in advance using pyeda
https://pyeda.readthedocs.org/en/latest/expr.html
"""
directionList = [firstSubiteration, secondSubiteration, thirdSubiteration, fourthSubiteration,
fifthSubiteration, sixthSubiteration, seventhSubiteration, eighthSubiteration,
ninthSubiteration, tenthSubiteration, eleventhSubiteration, twelvethSubiteration]
def _skeletonPass(image):
"""
each pass consists of 12 serial subiterations and finding the
boundaries of the padded image/array
"""
boundaryIm = _getBouondariesOfimage(image)
numPixelsremovedList = [] * 12
boundaryIndices = list(set(map(tuple, list(np.transpose(np.nonzero(boundaryIm))))))
for i in range(0, 12):
convImage = _convolveImage(image, directionList[i])
totalPixels, image = _applySubiter(image, boundaryIndices, convImage)
print("number of pixels removed in the {} direction is {}". format(i, totalPixels))
numPixelsremovedList.append(totalPixels)
numPixelsremoved = sum(numPixelsremovedList)
return numPixelsremoved, image
def _applySubiter(image, boundaryIndices, convImage):
"""
each subiteration paralleley reduces the border voxels in 12 directions
going through each voxel and marking if it can be deleted or not in a
different image named temp_del and finally multiply it with the original
image to delete the voxels so marked
"""
temp_del = np.zeros_like(image)
# boundaryIndicesCopy = copy.deepcopy(boundaryIndices)
lenB = len(boundaryIndices)
for k in range(0, lenB):
temp_del[boundaryIndices[k]] = lookUpTablearray[convImage[boundaryIndices[k]]]
numpixel_removed = np.einsum('ijk->', image * temp_del, dtype=int)
image[temp_del == 1] = 0
return numpixel_removed, image
def getSkeletonize3D(image):
"""
function to skeletonize a 3D binary image with object in brighter contrast than background.
In other words, 1 = object, 0 = background
"""
assert np.max(image) in [0, 1]
zOrig, yOrig, xOrig = np.shape(image)
padImage = np.lib.pad(image, 1, 'constant', constant_values=0)
start_skeleton = time.time()
pass_no = 0
numpixel_removed = 0
while pass_no == 0 or numpixel_removed > 0:
numpixel_removed, padImage = _skeletonPass(padImage)
print("number of pixels removed in pass {} is {}".format(pass_no, numpixel_removed))
pass_no += 1
print("done %i number of pixels in %f seconds" % (np.sum(image), time.time() - start_skeleton))
return padImage[1: zOrig + 1, 1: yOrig + 1, 1: xOrig + 1]
if __name__ == '__main__':
sample = np.ones((5, 5, 5), dtype=np.uint8)
resultSkel = getSkeletonize3D(sample)
# gives a single voxel at the center
print("resultSkel", resultSkel)