I am doing a License/Number plate recognition project and I'm on the stage of completion but there is a small problem, I have successfully recognized the characters, consider the below example:
This is an input image, I got the prediction as 2791 2g rj14
As you can, the ocr did a great job but the arrangement is destroyed (DESTROYING the whole purpose). Sometimes it does outputs in the correct sequence but sometimes it does not, so when it does not output in the correct sequence I'm trying to develop an algorithm which will take the predicted num_plate string as input and rearrange it on the basis of my country (India).
Below are some images which tell us about the format of Indian Number/License Plate.
Also, I have collected all the states but for right now, I just want to do for only the 3 states which are: Delhi (DL), Haryana (HR), UttarPradesh (UP). More info : https://en.wikipedia.org/wiki/List_of_Regional_Transport_Office_districts_in_India
total_states_list = [
'AN','AP','AR','AS','BR','CG','CH','DD','DL','DN','GA','GJ','HR','HP','JH','JK','KA','KL',
'LD','MH','ML','MN','MP','MZ','NL','OD','PB','PY','RJ','SK','TN','TR','TS','UK','UP','WB'
]
district_codes = {
'DL': ['1','2','3','4','5','6','7','8','9','10','11','12','13'],
'HR': [01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,
71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99
]
}
So, I have been trying but cannot come up with an algorithm which rearranges the sequence in the required sequence if it is not. Any help would be really appreciated.
Details about OCR
Using keras-ocr, I'm getting the following output for the input image:
[
('hrlz', array([[ 68.343796, 42.088367],
[196.68803 , 26.907867],
[203.00832 , 80.343094],
[ 74.66408 , 95.5236 ]], dtype=float32)),
('c1044', array([[ 50.215836, 113.09602 ],
[217.72466 , 92.58473 ],
[224.3968 , 147.07387 ],
[ 56.887985, 167.58516 ]], dtype=float32))
]
source: https://keras-ocr.readthedocs.io/en/latest/examples/using_pretrained_models.html
Inside the keras_ocr.tools.drawAnnotations they are I think getting the predictions boxes. So I located this file and found the implementation of drawAnnotations function and here it is:
def drawAnnotations(image, predictions, ax=None):
if ax is None:
_, ax = plt.subplots()
ax.imshow(drawBoxes(image=image, boxes=predictions, boxes_format='predictions'))
predictions = sorted(predictions, key=lambda p: p[1][:, 1].min())
left = []
right = []
for word, box in predictions:
if box[:, 0].min() < image.shape[1] / 2:
left.append((word, box))
else:
right.append((word, box))
ax.set_yticks([])
ax.set_xticks([])
for side, group in zip(['left', 'right'], [left, right]):
for index, (text, box) in enumerate(group):
y = 1 - (index / len(group))
xy = box[0] / np.array([image.shape[1], image.shape[0]])
xy[1] = 1 - xy[1]
ax.annotate(s=text,
xy=xy,
xytext=(-0.05 if side == 'left' else 1.05, y),
xycoords='axes fraction',
arrowprops={
'arrowstyle': '->',
'color': 'r'
},
color='r',
fontsize=14,
horizontalalignment='right' if side == 'left' else 'left')
return ax
How should I go about and get the (x,y,w,h) and then somehow sort/print according to y/x of number_plate bbox?
EDIT - 2
I managed to get the bounding box of characters as you can see in the image below:
using the function cv2.polylines(box), where box are the same coordinates where I have pasted the output earlier. Now how can I print them in a sequence like, left to right... using the y/x as suggested by people in the comments.
If you can get the coordinates of each identified text box, then:
Rotate the coordinates so the boxes are parallel with the X-axis
Scale the Y-coordinates so they can be rounded to integers, so that boxes that are side-by-side will get the same integer Y-coordinate (like a line number)
Sort the data by Y, then X coordinate
Extract the texts in that order
Here is an example of such sequence:
data = [
('hrlz', [[ 68.343796, 42.088367],
[196.68803 , 26.907867],
[203.00832 , 80.343094],
[ 74.66408 , 95.5236 ]]),
('c1044',[[ 50.215836, 113.09602 ],
[217.72466 , 92.58473 ],
[224.3968 , 147.07387 ],
[ 56.887985, 167.58516 ]])
]
# rotate data to align with X-axis
a, b = data[0][1][:2]
dist = ((b[1] - a[1]) ** 2 + (b[0] - a[0]) ** 2) ** 0.5
sin = (b[1] - a[1]) / dist
cos = (b[0] - a[0]) / dist
data = [
(text, [(x * cos + y * sin, y * cos - x * sin) for x, y in box]) for text, box in data
]
# scale Y coordinate to integers
a, b = data[0][1][1:3]
height = b[1] - a[1]
data = [
(round(box[0][1] / height), box[0][0], text)
for text, box in data
]
# sort by Y, then X
data.sort()
# Get text in the right order
print("".join(text for _, _, text in data))
This assumes that the points of the boxes are given in the following clockwise order:
top-left, top-right, bottom-right, bottom-left
Related
I have two sets of three 3d points of the same rigid body that's shape is a (non-perfect) right triangle.
The sets are the initial and final states. The points in each of the sets are corresponding. Both have its first/front points at the origin. The initial state represents when the figures orientation is at (0,0,0) in terms of pitch, roll, yaw. I am trying to find the orientation of the final state in the same format or similar with a different order. I dont want R matrix or quaternions.
I decieded to implement this method of finding the R matrix from two sets of three points. Specifically its the bottom part that starts with "More information, easier computation". I replaced the P's and Q's with I's and F's. In this method he assumes that the "distances between the Pi's and Qi's are the same". I believe I can assert this by doing:
assert round(math.sqrt(sum(a**2 for a in initial_state[1])), 3) == round(math.sqrt(sum(a**2 for a in final_state[1])), 3)
assert round(math.sqrt(sum(a**2 for a in initial_state[2])), 3) == round(math.sqrt(sum(a**2 for a in final_state[2])), 3)
After finding the R matrix I convert it to angles. I'm not sure whats wrong with the implementation but its giving some wacky angles. It may be due to the rotationMatrixToEulerAngles() method as well.
import matplotlib.pyplot as plt
import numpy as np
import math
def rotationMatrixToEulerAngles(R):
sy = math.sqrt(R[0,0] * R[0,0] + R[1,0] * R[1,0])
singular = sy < 1e-6
if not singular :
x = math.atan2(R[2,1] , R[2,2])
y = math.atan2(-R[2,0], sy)
z = math.atan2(R[1,0], R[0,0])
else :
x = math.atan2(-R[1,2], R[1,1])
y = math.atan2(-R[2,0], sy)
z = 0
result = np.array([math.degrees(x)*-1, math.degrees(y)*-1, math.degrees(z)*-1])
# make sure not to return -0
for i, _ in enumerate(result):
if result[i] == -0.0:
result[i] = 0
return result
fig = plt.figure(figsize=(7,7))
ax = fig.add_subplot(111, projection='3d')
initial_state = np.array(
[[ 0.0, 0.0, 0.0 ],
[-15.79080794, 0.08799, 0.0 ],
[ 0.0, -12.49253637, 0.0 ]])
final_state = np.array(
[[ 0.0, 0.0, 0.0 ],
[-15.76038, 0.39484, 0.901 ],
[ -0.28012, -12.47225, -0.6542 ]])
# ------ Implementation of Strategy ------
i4 = np.add(np.cross(np.subtract(initial_state[1], initial_state[0]), np.subtract(initial_state[2], initial_state[0])), initial_state[0])
f4 = np.add(np.cross(np.subtract(final_state[1], final_state[0]), np.subtract(final_state[2], final_state[0])), final_state[0])
I = np.array([np.subtract(initial_state[1], initial_state[0]),
np.subtract(initial_state[2], initial_state[0]),
np.subtract(i4, initial_state[0])])
F = np.array([np.subtract(final_state[1], final_state[0]),
np.subtract(final_state[2], final_state[0]),
np.subtract(f4, final_state[0])])
r_matrix = F # np.linalg.inv(I)
angles = rotationMatrixToEulerAngles(r_matrix)
print(angles[0])
print(angles[1])
print(angles[2])
# plot inital vs final state
for o in initial_state:
ax.scatter(o[0], o[1], o[2])
for o in final_state:
ax.scatter(o[0], o[1], o[2])
plt.xlabel("X")
plt.ylabel("Y")
plt.legend(["1", "2", "3", "a", "b", "c"])
ax.set_ylim3d(-18, 2)
ax.set_xlim3d(-18, 2)
ax.set_zlim3d(-18, 2)
plt.show()
Output angles:
-40.135994612369494
-34.858079104181336
-1.0182498153168882
Theres no way its turned 40 and 35 degrees.
I'm trying to test each red point for inclusion with the blue circle. However, the path for my circle has some strange values which is what I believe is causing the inclusion test to not work as intended.
The axis list in the code below represent the max & min for the longitude and latitude respectively. Given that the circle is plotted at the right location I expect its path to have vertices within that range which is not the case.
Where am I going wrong?
from matplotlib.patches import Ellipse
import matplotlib.path as mpltPath
axis = [4.7469287189121001, 5.0340994897259534, 52.282706941081258, 52.432452803031282]
unitX = (axis[1]-axis[0])/10
unitY = (axis[3]-axis[2])/10
fig, ax = plt.subplots(figsize=(8, 6))
for i, s in enumerate(housing_prices_shapes['2015']):
ax.plot(s[:,0], s[:,1], linewidth=0.5, c='0.5')
circle = Ellipse(housing_prices_shapes['2015'][0][0], width=unitX, height=unitY, edgecolor='b', facecolor='None')
ax.add_patch(circle)
listings_coordinates = airbnb_prices['2015'][["longitude", "latitude"]]
path_temp = circle.get_path()
transform = circle.get_transform()
new_path = transform.transform_path(path_temp)
path = mpltPath.Path(new_path.vertices)
flag = path.contains_points(listings_coordinates)
ax.scatter(listings_coordinates['longitude'].values, listings_coordinates['latitude'].values, c='r', s=0.5)
Each value used to create the circle prints as follow:
print(housing_prices_shapes['2015'][0][0], unitX, unitY)
[ 4.94147517 52.3670552 ] 0.028717077081385333 0.01497458619500236
The path variable which I expect to be in the same range as the longitude and latitude print as this, which is way off:
print(new_path.vertices)
array([[ 374.41773395, 221.41011283],
[ 380.33706714, 221.41011283],
[ 386.01475666, 223.12842659],
[ 390.2003573 , 226.18661544],
[ 394.38595794, 229.24480429],
[ 396.73773395, 233.39318067],
[ 396.73773395, 237.71811283],
[ 396.73773395, 242.04304498],
[ 394.38595794, 246.19142136],
[ 390.2003573 , 249.24961022],
[ 386.01475666, 252.30779907],
[ 380.33706714, 254.02611283],
[ 374.41773395, 254.02611283],
[ 368.49840076, 254.02611283],
[ 362.82071123, 252.30779907],
[ 358.63511059, 249.24961022],
[ 354.44950995, 246.19142136],
[ 352.09773395, 242.04304498],
[ 352.09773395, 237.71811283],
[ 352.09773395, 233.39318067],
[ 354.44950995, 229.24480429],
[ 358.63511059, 226.18661544],
[ 362.82071123, 223.12842659],
[ 368.49840076, 221.41011283],
[ 374.41773395, 221.41011283],
[ 374.41773395, 221.41011283]])
And of course no points are flagged as True:
print(any(flag))
False
As ImportanceOfBeingErnest noted in a comment, you shouldn't transform your ellipse path. Well, using the untransformed path wouldn't directly be useful either; you could probably make use of circle.get_verts().
But let me cut through your Gordian Knot: why not explicitly test for falling inside your ellipse? The equation of an ellipse with center (x0,y0) and semi-axes of length a and b is
(x-x0)^2/a^2 + (y-y0)^2/b^2 = 1
and it's really simple to see that the inside of the ellipse is then defined by the inequality
(x-x0)^2/a^2 + (y-y0)^2/b^2 < 1
(it's easy to see this for a circle, and you can think of an ellipse as a circle that went through a linear transform along one of its axes).
So use logical indexing to find which points are inside your ellipse! The only thing you need to watch out for is that the parameters passed to Ellipse are 2*a and 2*b:
points = airbnb_prices['2015'][['longitude', 'latitude']] # shape (N,2)
center = housing_prices_shapes['2015'][0][0] # shape (2,) broadcasts to (N,2)
a = unitX / 2 # scalar
b = unitY / 2 # scalar
# make use of broadcasting while we're at it
flag = ((points-center)**2 / np.array([a,b])**2).sum(axis=1) < 1
Now flag is a shape-(N,) logical array, i.e. the same shape and size as expected from your original call to contains_points.
I have the following problem. I have a numpy array of coordinates (entry 0 to 2) and want to define all the coordinates of small boxes between pairs of my coordiante list instead of creating a huge box around the minimum and maximum of all my coordinates in the list. The boxes should have a range of 5 around the coordinate pairs for example.
My list for example looks like:
[[ 24.313 294.679 1.5 1. 0. ]
[ 25.51 295.263 1.5 2. 0. ]
[ 26.743 294.526 1.5 3. 0. ]
...,
[ 30.362 307.242 10.779 95. 0. ]
[ 29.662 307.502 10.38 96. 0. ]
[ 29.947 308.99 11.147 97. 0. ]]
My first idea is to calculate the minumum and maximum of each pair and use itertools.product to create the coordinates for the small boxes. So i want to have a box around 24.313 294.679 1.5 and 25.51 295.263 1.5, next a box aorund 25.51 295.263 1.5 and 26.743 294.526 1.5 and so on. For better understanding, i want the coordinates like here, but in 3D of course:
And not like here:
Is there any easy numpy, scipy approach to do this?
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
# create some data; in 2D so we can plot stuff
x = np.linspace(0, 2*np.pi, 10)
y = np.sin(x)
data = np.c_[x,y]
# --------------------------------------------------
# core bit: get boxes
# bboxes = np.array([data[:-1], np.diff(data, axis=0)]).transpose([1,2,0]) # shorter but with negative widths, etc
data_pairs = np.array([data[:-1], data[1:]])
minima = data_pairs.min(axis=0)
maxima = data_pairs.max(axis=0)
widths = maxima-minima
bboxes = np.array([minima, widths]).transpose(1,2,0)
# --------------------------------------------------
# plot
fig, ax = plt.subplots(1,1)
ax.plot(data[:,0], data[:,1], 'ko')
for bbox in bboxes:
patch = Rectangle(xy=bbox[:,0], width=bbox[0,1], height=bbox[1,1], linewidth=0., alpha=0.5)
ax.add_artist(patch)
plt.show()
with pads:
# padded boxes:
pad = 0.1
N, D = data.shape
correction = pad*np.ones((N-1,D))
padded = bboxes.copy()
padded[:,:,0] -= correction
padded[:,:,1] += 2*correction
fig, ax = plt.subplots(1,1)
ax.plot(data[:,0], data[:,1], 'ko')
for bbox in padded:
patch = Rectangle(xy=bbox[:,0], width=bbox[0,1], height=bbox[1,1], linewidth=0., alpha=0.5, facecolor='red')
ax.add_artist(patch)
ax.set_xlim(0-pad, 2*np.pi+pad)
ax.set_ylim(-1-pad, 1+pad)
plt.show()
If I
define an intrinsic camera matrix A and poses [rvec, ...], [tvec, ...],
use them as parameters in cv2.projectPoints to generate the the images that would be generated by a camera when it views a grid of circles,
Detect the features (cv2.findCirclesGrid) in the resulting images
Use cv2.calibrateCamera on the feature detections to recover the camera parameters
Shouldn't I recover the original intrinsic and extrinsic parameters?
The full code at the bottom of this question does this process, but does not
recover the original camera parameters:
Kept 4 full captures out of 4 images
calibration error 133.796093439
Simulation matrix:
[[ 5.00000000e+03 0.00000000e+00 3.20000000e+02]
[ 0.00000000e+00 5.00000000e+03 2.40000000e+02]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
Estimated matrix:
[[ 1.0331118 0. 317.58445168]
[ 0. 387.49075886 317.98450481]
[ 0. 0. 1. ]]
I.e. the mean error is huge, and the estimated camera matrix does not look like
the simulation camera matrix orginally used to generate the test images.
I'd expect that this sort of closed-loop simulation should result in a very good estimate of the intrinsic camera matrix. What am I doing wrong that this approach for validating cameraCalibration doesn't seem to work?
Edits in response to AldurDisciple comment
1) Added new function in code below direct_generation_of_points that skips
the image generation functions and uses cv2.projectPoints directly to
compute the circle locations that are passed into cv2.calibrateCamera.
This works correctly.
But this is confusing: the estimated circle locations (derived from my simulated
images) are typically within about a 10'th of a pixel from the exact ones, the main
difference is that the points are in a different order:
# compare the y-component's
In [245]: S.dots[0][:,0,1]
Out[245]:
array([ 146.33618164, 146.30953979, 146.36413574, 146.26707458,
146.17976379, 146.30110168, 146.17236328, 146.35955811,
146.33454895, 146.36776733, 146.2612915 , 146.21359253,
146.23895264, 146.27839661, 146.27764893, 177.51347351,
177.57495117, 177.53858948, 177.48587036, 177.63012695,
177.48597717, 177.51727295, 177.5202179 , 177.52545166,
177.57287598, 177.51008606, 177.51296997, 177.53715515,
177.53053284, 177.58164978, 208.69573975, 208.7252655 ,
208.69616699, 208.73510742, 208.63375854, 208.66760254,
208.71517944, 208.74360657, 208.62438965, 208.59814453,
208.67456055, 208.72662354, 208.70921326, 208.63339233,
208.70820618, 239.8401947 , 240.06373596, 239.87176514,
240.04118347, 239.97781372, 239.97572327, 240.04475403,
239.95411682, 239.80995178, 239.94726562, 240.01327515,
239.82675171, 239.99989319, 239.90107727, 240.07745361,
271.31692505, 271.28417969, 271.28216553, 271.33111572,
271.33279419, 271.33584595, 271.30758667, 271.21173096,
271.28588867, 271.3387146 , 271.33770752, 271.2104187 ,
271.38504028, 271.25054932, 271.29376221, 302.52420044,
302.47903442, 302.41482544, 302.39868164, 302.47793579,
302.49789429, 302.45016479, 302.48071289, 302.50463867,
302.51422119, 302.46307373, 302.42077637, 302.60791016,
302.48162842, 302.46142578, 333.70709229, 333.75698853,
333.64157104, 333.64926147, 333.6647644 , 333.69546509,
333.73342896, 333.76846313, 333.57540894, 333.76605225,
333.74307251, 333.60968018, 333.7739563 , 333.70132446,
333.62057495], dtype=float32)
In [246]: S.exact_dots[0][:,0,1]
Out[246]:
array([ 146.25, 177.5 , 208.75, 240. , 271.25, 302.5 , 333.75,
146.25, 177.5 , 208.75, 240. , 271.25, 302.5 , 333.75,
<< snipped 10 identical rows >>
146.25, 177.5 , 208.75, 240. , 271.25, 302.5 , 333.75,
146.25, 177.5 , 208.75, 240. , 271.25, 302.5 , 333.75,
146.25, 177.5 , 208.75, 240. , 271.25, 302.5 , 333.75], dtype=float32)
Here's the working version of what I'm trying to do:
import scipy
import cv2
import itertools
def direct_generation_of_points():
''' Skip the part where we actually generate the image,
just use cv2.projectPoints to generate the exact locations
of the grid centers.
** This seems to work correctly **
'''
S=Setup()
t=tvec(0.0,0.0,1.6) # keep the camera 1.6 meters away from target, looking at the origin
rvecs=[ rvec(0.0,0.0,0.0), rvec(0.0, scipy.pi/6,0.0), rvec(scipy.pi/8,0.0,0.0), rvec(0.0,0.0,0.5) ]
S.poses=[ (r,t) for r in rvecs ]
S.images='No images: just directly generate the extracted circle locations'
S.dots=S.make_locations_direct()
calib_flags=cv2.CALIB_ZERO_TANGENT_DIST|cv2.CALIB_SAME_FOCAL_LENGTH
calib_flags=calib_flags|cv2.CALIB_FIX_K3|cv2.CALIB_FIX_K4
calib_flags=calib_flags|cv2.CALIB_FIX_K5|cv2.CALIB_FIX_K6
S.calib_results=cv2.calibrateCamera( [S.grid,]*len(S.dots), S.dots, S.img_size, cameraMatrix=S.A, flags=calib_flags)
print "calibration error ", S.calib_results[0]
print "Simulation matrix: \n", S.A
print "Estimated matrix: \n", S.calib_results[1]
return S
def basic_test():
''' Uses a camera setup to
(1) generate an image of a grid of circles
(2) detects those circles
(3) generate an estimated camera model from the circle detections
** This does not work correctly **
'''
S=Setup()
t=tvec(0.0,0.0,1.6) # keep the camera 1.6 meters away from target, looking at the origin
rvecs=[ rvec(0.0,0.0,0.0), rvec(0.0, scipy.pi/6,0.0), rvec(scipy.pi/8,0.0,0.0), rvec(0.0,0.0,0.5) ]
S.poses=[ (r,t) for r in rvecs ]
S.images=S.make_images()
S.dots=extract_dots( S.images, S.grid_size[::-1] )
S.exact_dots=S.make_locations_direct()
calib_flags=cv2.CALIB_ZERO_TANGENT_DIST|cv2.CALIB_SAME_FOCAL_LENGTH
calib_flags=calib_flags|cv2.CALIB_FIX_K3|cv2.CALIB_FIX_K4|cv2.CALIB_FIX_K5
calib_flags=calib_flags|cv2.CALIB_FIX_K6
S.calib_results=cv2.calibrateCamera( [S.grid,]*len(S.dots), S.dots, S.img_size, cameraMatrix=S.A, flags=calib_flags)
print "calibration error ", S.calib_results[0]
print "Simulation matrix: \n", S.A
print "Estimated matrix: \n", S.calib_results[1]
return S
class Setup(object):
''' Class to simulate a camera, produces images '''
def __init__(self):
self.img_size=(480,640)
self.A=scipy.array( [ [5.0e3, 0.0, self.img_size[1]/2],
[ 0.0, 5.0e3, self.img_size[0]/2],
[ 0.0, 0.0, 1.0 ] ],
dtype=scipy.float32 )
# Nx, Ny, spacing, dot-size
self.grid_spec=( 15, 7, 0.01, 0.001 )
self.grid=square_grid_xy( self.grid_spec[0], self.grid_spec[1], self.grid_spec[2])
# a pose is a pair: rvec, tvec
self.poses=[ ( rvec(0.0, scipy.pi/6, 0.0), tvec( 0.0,0.0,1.6) ),
]
#property
def grid_size(self):
return self.grid_spec[:2]
def make_images(self):
return [make_dots_image(self.img_size, self.A, rvec, tvec, self.grid, self.grid_spec[-1] ) for (rvec,tvec) in self.poses]
def make_locations_direct(self):
return [cv2.projectPoints( self.grid, pose[0], pose[1], self.A, None)[0] for pose in self.poses]
def square_grid_xy( nx, ny, dx ):
''' Returns a square grid in the xy plane, useful
for defining test grids for camera calibration
'''
xvals=scipy.arange(nx)*dx
yvals=scipy.arange(ny)*dx
xvals=xvals-scipy.mean(xvals)
yvals=yvals-scipy.mean(yvals)
res=scipy.zeros( [3, nx*ny], dtype=scipy.float32 )
for (i,(x,y)) in enumerate( itertools.product(xvals, yvals)):
res[:,i]=scipy.array( [x,y,0.0] )
return res.transpose()
# single pixel dots were not detected?
#def make_single_pixel_dots( img_size, A, rvec, tvec, grid, dist_k=None):
# rgb=scipy.ones( img_size+(3,), dtype=scipy.uint8 )*0xff
# (dot_locs, jac)=cv2.projectPoints( grid, rvec, tvec, A, dist_k)
# for p in dot_locs:
# (c,r)=(int(p[0][0]+0.5), int(p[0][1]+0.5))
# if 0<=c<img_size[1] and 0<=r<img_size[0]:
# rgb[r,c,:]=0
# return rgb
def make_dots_image( img_size, A, rvec, tvec, grid, dotsize, dist_k=None):
''' Make the image of the dots, uses cv2.projectPoints to construct the image'''
# make white image
max_intensity=0xffffffff
intensity=scipy.ones( img_size, dtype=scipy.uint32)*max_intensity
# Monte-Carlo approach to draw the dots
for dot in grid:
deltas=2*dotsize*( scipy.rand(1024, 3 )-0.5) # no. of samples must be small relative to bit-depth of intensity array
deltas[:,2]=0
indicator=scipy.where( scipy.sum( deltas*deltas, 1)<dotsize*dotsize, 1, 0.0)
print "inside fraction: ", sum(indicator)/len(indicator)
(pts,jac)=cv2.projectPoints( dot+deltas, rvec, tvec, A, dist_k )
pts=( p for (ind,p) in zip(indicator, pts) if ind )
for p in pts:
(c,r)=( int(p[0][0]+0.5), int( p[0][1]+0.5 ) )
if r>=0 and c>=0 and c<img_size[1] and r<img_size[0]:
intensity[r,c]=intensity[r,c]-6
else:
print "col, row ", (c,r), " point rejected"
# rescale so that image goes from 0x0 to max intensity
min_intensity=min(intensity.flat)
# normalize the intensity
intensity=0xff*( (intensity-min_intensity)/float(max_intensity-min_intensity) )
pixel_img=scipy.ones( intensity.shape+(3,), dtype=scipy.uint8 )
return (pixel_img*intensity[:,:,scipy.newaxis]).astype(scipy.uint8 )
def extract_dots( img_list, grid_size ):
'''
#arg img_list: usually a list of images, can be a single image
'''
# convert single array, into a 1-element list
if type(img_list) is scipy.ndarray:
img_list=[img_list,]
def get_dots( img ):
res=cv2.findCirclesGridDefault( img, grid_size)
if not res[0]: # sometimes, reversing the grid size will make the detection successful
return cv2.findCirclesGridDefault( img, grid_size[::-1] )
return res
all_dots=[ get_dots( img) for img in img_list]
#all_dots=[cv2.findCirclesGrid( img, grid_size[::-1] ) for img in img_list ]
full_captures=[x[1] for x in all_dots if x[0] ]
print "Kept {0} full captures out of {1} images".format( len(full_captures), len(img_list) )
if len(full_captures)<len(img_list):
print "\t", [x[0] for x in all_dots]
return [scipy.squeeze(x) for x in full_captures]
# convenience functions
def vec3_32(x,y,z):
return scipy.array( [x,y,z], dtype=scipy.float32 )
rvec=vec3_32
tvec=vec3_32
if __name__=="__main__":
basic_test()
The key issue is in the organization of the grid points passed in the first argument of cv2.calibrateCamera,
in the question the points are organized in column major order, so to speak, and need to be organized in row-major order:
def square_grid_xy_fixed( nx, ny, dx ):
''' Returns a square grid in the xy plane, useful
for defining test grids for camera calibration
'''
xvals=scipy.arange(nx)*dx
yvals=scipy.arange(ny)*dx
xvals=xvals-scipy.mean(xvals)
yvals=yvals-scipy.mean(yvals)
res=scipy.zeros( [3, nx*ny], dtype=scipy.float32 )
# need to have "x" be the most rapidly varying index, i.e.
# it must be the final argument to itertools.product
for (i,(y,x)) in enumerate( itertools.product(yvals, xvals)):
res[:,i]=scipy.array( [x,y,0.0] )
return res.transpose()
I want to bin the values of polygons to a fine regular grid.
For instance, I have the following coordinates:
data = 2.353
data_lats = np.array([57.81000137, 58.15999985, 58.13000107, 57.77999878])
data_lons = np.array([148.67999268, 148.69999695, 148.47999573, 148.92999268])
My regular grid looks like this:
delta = 0.25
grid_lons = np.arange(-180, 180, delta)
grid_lats = np.arange(90, -90, -delta)
llx, lly = np.meshgrid( grid_lons, grid_lats )
rows = lly.shape[0]
cols = llx.shape[1]
grid = np.zeros((rows,cols))
Now I can find the grid pixel that corresponds to the center of my polygon very easily:
centerx, centery = np.mean(data_lons), np.mean(data_lats)
row = int(np.floor( centery/delta ) + (grid.shape[0]/2))
col = int(np.floor( centerx/delta ) + (grid.shape[1]/2))
grid[row,col] = data
However, there are probably a couple of grid pixels that still intersect with the polygon. Hence, I would like to generate a bunch of coordinates inside my polygon (data_lons, data_lats) and find their corresponding grid pixel as before. Do you a suggestion to generate the coordinates randomly or systematically? I failed, but am still trying.
Note: One data set contains around ~80000 polygons so it has to be really fast (a couple of seconds). That is also why I chose this approach, because it does not account the area of overlap... (like my earlier question Data binning: irregular polygons to regular mesh which is VERY slow)
I worked on a quick and dirty solution by simply calculating the coordinates between corner pixels. Take a look:
dlats = np.zeros((data_lats.shape[0],4))+np.nan
dlons = np.zeros((data_lons.shape[0],4))+np.nan
idx = [0,1,3,2,0] #rearrange the corner pixels
for cc in range(4):
dlats[:,cc] = np.mean((data_lats[:,idx[cc]],data_lats[:,idx[cc+1]]), axis=0)
dlons[:,cc] = np.mean((data_lons[:,idx[cc]],data_lons[:,idx[cc+1]]), axis=0)
data_lats = np.column_stack(( data_lats, dlats ))
data_lons = np.column_stack(( data_lons, dlons ))
Thus, the red dots represent the original corners - the blue ones the intermediate pixels between them.
I can do this one more time and include the center pixel (geo[:,[4,9]])
dlats = np.zeros((data.shape[0],8))
dlons = np.zeros((data.shape[0],8))
for cc in range(8):
dlats[:,cc] = np.mean((data_lats[:,cc], geo[:,4]), axis=0)
dlons[:,cc] = np.mean((data_lons[:,cc], geo[:,9]), axis=0)
data_lats = np.column_stack(( data_lats, dlats, geo[:,4] ))
data_lons = np.column_stack(( data_lons, dlons, geo[:,9] ))
This works really nice, and I can assign each point directly to its corresponding grid pixel like this:
row = np.floor( data_lats/delta ) + (llx.shape[0]/2)
col = np.floor( data_lons/delta ) + (llx.shape[1]/2)
However the final binning now takes ~7sec!!! How can I speed this code up:
for ii in np.arange(len(data)):
for cc in np.arange(data_lats.shape[1]):
final_grid[row[ii,cc],col[ii,cc]] += data[ii]
final_grid_counts[row[ii,cc],col[ii,cc]] += 1
You'll need to test the following approach to see if it is fast enough. First, you should modify all your lats and lons into, to make them (possibly fractional) indices into your grid:
idx_lats = (data_lats - lat_grid_start) / lat_grid step
idx_lons = (data_lons - lon_grid_start) / lon_grid step
Next, we want to split your polygons into triangles. For any convex polygon, you could take the center of the polygon as one vertex of all triangles, and then the vertices of the polygon in consecutive pairs. But if your polygon are all quadrilaterals, it is going to be faster to divide them into only 2 triangles, using vertices 0, 1, 2 for the first, and 0, 2, 3 for the second.
To know if a certain point is inside a triangle, I am going to use the barycentric coordinates approach described here. This first function checks whether a bunch of points are inside a triangle:
def check_in_triangle(x, y, x_tri, y_tri) :
A = np.vstack((x_tri[0], y_tri[0]))
lhs = np.vstack((x_tri[1:], y_tri[1:])) - A
rhs = np.vstack((x, y)) - A
uv = np.linalg.solve(lhs, rhs)
# Equivalent to (uv[0] >= 0) & (uv[1] >= 0) & (uv[0] + uv[1] <= 1)
return np.logical_and(uv >= 0, axis=0) & (np.sum(uv, axis=0) <= 1)
Given a triangle by its vertices, you can get the lattice points inside it, by running the above function on the lattice points in the bounding box of the triangle:
def lattice_points_in_triangle(x_tri, y_tri) :
x_grid = np.arange(np.ceil(np.min(x_tri)), np.floor(np.max(x_tri)) + 1)
y_grid = np.arange(np.ceil(np.min(y_tri)), np.floor(np.max(y_tri)) + 1)
x, y = np.meshgrid(x_grid, y_grid)
x, y = x.reshape(-1), y.reshape(-1)
idx = check_in_triangle(x, y, x_tri, y_tri)
return x[idx], y[idx]
And for a quadrilateral, you simply call this last function twice :
def lattice_points_in_quadrilateral(x_quad, y_quad) :
return map(np.concatenate,
zip(lattice_points_in_triangle(x_quad[:3], y_quad[:3]),
lattice_points_in_triangle(x_quad[[0, 2, 3]],
y_quad[[0, 2, 3]])))
If you run this code on your example data, you will get two empty arrays returned: that's because the order of the quadrilateral points is a surprising one: indices 0 and 1 define one diagonal, 2 and 3 the other. My function above was expecting the vertices to be ordered around the polygon. If you really are doing things this other way, you need to change the second call to lattice_points_in_triangle inside lattice_points_in_quadrilateral so that the indices used are [0, 1, 3] instead of [0, 2, 3].
And now, with that change :
>>> idx_lats = (data_lats - (-180) ) / 0.25
>>> idx_lons = (data_lons - (-90) ) / 0.25
>>> lattice_points_in_quadrilateral(idx_lats, idx_lons)
[array([952]), array([955])]
If you change the resolution of your grid to 0.1:
>>> idx_lats = (data_lats - (-180) ) / 0.1
>>> idx_lons = (data_lons - (-90) ) / 0.1
>>> lattice_points_in_quadrilateral(idx_lats, idx_lons)
[array([2381, 2380, 2381, 2379, 2380, 2381, 2378, 2379, 2378]),
array([2385, 2386, 2386, 2387, 2387, 2387, 2388, 2388, 2389])]
Timing wise this approach is going to be, in my system, about 10x too slow for your needs:
In [8]: %timeit lattice_points_in_quadrilateral(idx_lats, idx_lons)
1000 loops, best of 3: 269 us per loop
So you are looking at over 20 sec. to process your 80,000 polygons.