Develop Reference

How to code up an image stitching software for these 'simple' images?

TLDR:
Need help trying to calculate overlap region between 2 graphs.
So I'm trying to stitch these 2 images:
Since I know that the images I will be stitching definitely come from the same image, I feel that I should be able to code this up myself. Using libraries like OpenCV feels a little like overkill for me for this task.
My current idea is that I can simplify this task by doing the following steps for each image:
Load image using PIL
Convert image to black and white (PIL image mode “L”)
[Optional: crop images to overlapping region by inspection by eye]
Create vector row_sum, which is a sum of each row
[Optional: log row_sum, to reduce the size of values we're working with]
Plot row_sum.
This would reduce the (potentially) (3*2)-dimensional problem, with 3 RGB channels for each pixel on the 2D image to a (1*2)-D problem with the black and white pixel for the 2D image instead. Then, summing across the rows reduces this to a 1D problem.
I used the following code to implement the above:
import matplotlib.pyplot as plt
import numpy as np
from PIL import Image
class Stitcher():
def combine_2(self, img1, img2):
# thr1, thr2 = self.get_cropped_bw(img1, 115, img2, 80)
thr1, thr2 = self.get_cropped_bw(img1, 0, img2, 0)
row_sum1 = np.log(thr1.sum(1))
row_sum2 = np.log(thr2.sum(1))
self.plot_4x4(thr1, thr2, row_sum1, row_sum2)
def get_cropped_bw(self, img1, img1_keep_from, img2, img2_keep_till):
im1 = Image.open(img1).convert("L")
im2 = Image.open(img2).convert("L")
data1 = (np.array(im1)[img1_keep_from:]
if img1_keep_from != 0 else np.array(im1))
data2 = (np.array(im2)[:img2_keep_till]
if img2_keep_till != 0 else np.array(im2))
return data1, data2
def plot_4x4(self, thr1, thr2, row_sum1, row_sum2):
fig, ax = plt.subplots(2, 2, sharey="row", constrained_layout=True)
ax[0, 0].imshow(thr1, cmap="Greys")
ax[0, 1].imshow(thr2, cmap="Greys")
ax[1, 0].plot(row_sum1, "k.")
ax[1, 1].plot(row_sum2, "r.")
ax[1, 0].set(
xlabel="Index Value",
ylabel="Row Sum",
)
plt.show()
imgs = (r"combine\imgs\test_image_part_1.jpg",
r"combine\imgs\test_image_part_2.jpg")
s = Stitcher()
s.combine_2(*imgs)
This gave me this graph:
(I've added in those yellow boxes, to indicate the overlap regions.)
This is the bit I'm stuck at. I want to find exactly:
the index value of the left-side of the yellow box for the 1st image and
the index value of the right-side of the yellow box for the 2nd image.
I define the overlap region as the longest range for which the end of the 1st graph 'matches' the start of the 2nd graph. For the method to find the overlap region, what should I do if the row sum values aren't exactly the same (what if one is the other scaled by some factor)?
I feel like this could be a problem that could use dot products to find the similarity between the 2 graphs? But I can't think of how to implement this.

I had a lot more fun with this than I expected. I wrote this using opencv, but that's just to load and show the image. Everything else is done with numpy so swapping this to PIL shouldn't be too difficult.
I'm using a brute-force matcher. I also wrote a random-start hillclimber that runs in much less time, but I can't guarantee it'll find the correct answer since the gradient space isn't smooth. I won't include it in my code since it's long and janky, but if you really need the time efficiency I can add it back in later.
I added a random crop and some salt and pepper noise to the images to test for robustness.
The brute-force matcher operates on the idea that we don't know which section of the two images overlap, so we need to convolve the smaller image over the larger image from left to right, top to bottom. This means our search space is:
horizontal = small_width + big_width
vertical = small_height + big_height
area = horizontal * vertical
This will grow very quickly with image size. I motivate the algorithm by giving it points for having a larger overlap, but it loses more points for having differences in color for the overlapped area.
Here are some pictures from an execution of this program
import cv2
import numpy as np
import random
# randomly snips edges
def randCrop(image, maxMargin):
c = [random.randint(0,maxMargin) for a in range(4)];
return image[c[0]:-c[1], c[2]:-c[3]];
# adds noise to image
def saltPepper(image, minNoise, maxNoise):
h,w = image.shape;
randNum = random.randint(minNoise, maxNoise);
for a in range(randNum):
x = random.randint(0, w-1);
y = random.randint(0, h-1);
image[y,x] = random.randint(0, 255);
return image;
# evaluate layout
def getScore(one, two):
# do raw subtraction
left = one - two;
right = two - one;
sub = np.minimum(left, right);
return np.count_nonzero(sub);
# return 2d random position within range
def randPos(img, big_shape):
th,tw = big_shape;
h,w = img.shape;
x = random.randint(0, tw - w);
y = random.randint(0, th - h);
return [x,y];
# overlays small image onto big image
def overlay(small, big, pos):
# unpack
h,w = small.shape;
x,y = pos;
# copy and place
copy = big.copy();
copy[y:y+h, x:x+w] = small;
return copy;
# calculates overlap region
def overlap(one, two, pos_one, pos_two):
# unpack
h1,w1 = one.shape;
h2,w2 = two.shape;
x1,y1 = pos_one;
x2,y2 = pos_two;
# set edges
l1 = x1;
l2 = x2;
r1 = x1 + w1;
r2 = x2 + w2;
t1 = y1;
t2 = y2;
b1 = y1 + h1;
b2 = y2 + h2;
# go
left = max(l1, l2);
right = min(r1, r2);
top = max(t1, t2);
bottom = min(b1, b2);
return [left, right, top, bottom];
# wrapper for overlay + getScore
def fullScore(one, two, pos_one, pos_two, big_empty):
# check positions
x,y = pos_two;
h,w = two.shape;
th,tw = big_empty.shape;
if y+h > th or x+w > tw or x < 0 or y < 0:
return -99999999;
# overlay
temp_one = overlay(one, big_empty, pos_one);
temp_two = overlay(two, big_empty, pos_two);
# get overlap
l,r,t,b = overlap(one, two, pos_one, pos_two);
temp_one = temp_one[t:b, l:r];
temp_two = temp_two[t:b, l:r];
# score
diff = getScore(temp_one, temp_two);
score = (r-l) * (b-t);
score -= diff*2;
return score;
# do brute force
def bruteForce(one, two):
# calculate search space
# unpack size
h,w = one.shape;
one_size = h*w;
h,w = two.shape;
two_size = h*w;
# small and big
if one_size < two_size:
small = one;
big = two;
else:
small = two;
big = one;
# unpack size
sh, sw = small.shape;
bh, bw = big.shape;
total_width = bw + sw * 2;
total_height = bh + sh * 2;
# set up empty images
empty = np.zeros((total_height, total_width), np.uint8);
# set global best
best_score = -999999;
best_pos = None;
# start scrolling
ybound = total_height - sh;
xbound = total_width - sw;
for y in range(ybound):
print("y: " + str(y) + " || " + str(empty.shape));
for x in range(xbound):
# get score
score = fullScore(big, small, [sw,sh], [x,y], empty);
# show
# prog = overlay(big, empty, [sw,sh]);
# prog = overlay(small, prog, [x,y]);
# cv2.imshow("prog", prog);
# cv2.waitKey(1);
# compare
if score > best_score:
best_score = score;
best_pos = [x,y];
print("best_score: " + str(best_score));
return best_pos, [sw,sh], small, big, empty;
# do a step of hill climber
def hillStep(one, two, best_pos, big_empty, step):
# make a step
new_pos = best_pos[1][:];
new_pos[0] += step[0];
new_pos[1] += step[1];
# get score
return fullScore(one, two, best_pos[0], new_pos, big_empty), new_pos;
# hunt around for good position
# let's do a random-start hillclimber
def randHill(one, two, shape):
# set up empty images
big_empty = np.zeros(shape, np.uint8);
# set global best
g_best_score = -999999;
g_best_pos = None;
# lets do 200 iterations
iters = 200;
for a in range(iters):
# progress check
print(str(a) + " of " + str(iters));
# start with random position
h,w = two.shape[:2];
pos_one = [w,h];
pos_two = randPos(two, shape);
# get score
best_score = fullScore(one, two, pos_one, pos_two, big_empty);
best_pos = [pos_one, pos_two];
# hill climb (only on second image)
while True:
# end condition: no step improves score
end_flag = True;
# 8-way
for y in range(-1, 1+1):
for x in range(-1, 1+1):
if x != 0 or y != 0:
# get score and update
score, new_pos = hillStep(one, two, best_pos, big_empty, [x,y]);
if score > best_score:
best_score = score;
best_pos[1] = new_pos[:];
end_flag = False;
# end
if end_flag:
break;
else:
# show
# prog = overlay(one, big_empty, best_pos[0]);
# prog = overlay(two, prog, best_pos[1]);
# cv2.imshow("prog", prog);
# cv2.waitKey(1);
pass;
# check for new global best
if best_score > g_best_score:
g_best_score = best_score;
g_best_pos = best_pos[:];
print("top score: " + str(g_best_score));
return g_best_score, g_best_pos;
# load both images
top = cv2.imread("top.jpg");
bottom = cv2.imread("bottom.jpg");
top = cv2.cvtColor(top, cv2.COLOR_BGR2GRAY);
bottom = cv2.cvtColor(bottom, cv2.COLOR_BGR2GRAY);
# randomly crop
top = randCrop(top, 20);
bottom = randCrop(bottom, 20);
# randomly add noise
saltPepper(top, 200, 1000);
saltPepper(bottom, 200, 1000);
# set up max image (assume no overlap whatsoever)
tw = 0;
th = 0;
h, w = top.shape;
tw += w;
th += h;
h, w = bottom.shape;
tw += w*2;
th += h*2;
# do random-start hill climb
_, best_pos = randHill(top, bottom, (th, tw));
# show
empty = np.zeros((th, tw), np.uint8);
pos1, pos2 = best_pos;
image = overlay(top, empty, pos1);
image = overlay(bottom, image, pos2);
# do brute force
# small_pos, big_pos, small, big, empty = bruteForce(top, bottom);
# image = overlay(big, empty, big_pos);
# image = overlay(small, image, small_pos);
# recolor overlap
h,w = empty.shape;
color = np.zeros((h,w,3), np.uint8);
l,r,t,b = overlap(top, bottom, pos1, pos2);
color[:,:,0] = image;
color[:,:,1] = image;
color[:,:,2] = image;
color[t:b, l:r, 0] += 100;
# show images
cv2.imshow("top", top);
cv2.imshow("bottom", bottom);
cv2.imshow("overlayed", image);
cv2.imshow("Color", color);
cv2.waitKey(0);
Edit: I added in the random-start hillclimber

Matplotlib: Rotate an ellipse in 3D issue [duplicate]

Short question
How can matplotlib 2D patches be transformed to 3D with arbitrary normals?
Long question
I would like to plot Patches in axes with 3d projection. However, the methods provided by mpl_toolkits.mplot3d.art3d only provide methods to have patches with normals along the principal axes. How can I add patches to 3d axes that have arbitrary normals?

Short answer
Copy the code below into your project and use the method
def pathpatch_2d_to_3d(pathpatch, z = 0, normal = 'z'):
"""
Transforms a 2D Patch to a 3D patch using the given normal vector.
The patch is projected into they XY plane, rotated about the origin
and finally translated by z.
"""
to transform your 2D patches to 3D patches with arbitrary normals.
from mpl_toolkits.mplot3d import art3d
def rotation_matrix(d):
"""
Calculates a rotation matrix given a vector d. The direction of d
corresponds to the rotation axis. The length of d corresponds to
the sin of the angle of rotation.
Variant of: http://mail.scipy.org/pipermail/numpy-discussion/2009-March/040806.html
"""
sin_angle = np.linalg.norm(d)
if sin_angle == 0:
return np.identity(3)
d /= sin_angle
eye = np.eye(3)
ddt = np.outer(d, d)
skew = np.array([[ 0, d[2], -d[1]],
[-d[2], 0, d[0]],
[d[1], -d[0], 0]], dtype=np.float64)
M = ddt + np.sqrt(1 - sin_angle**2) * (eye - ddt) + sin_angle * skew
return M
def pathpatch_2d_to_3d(pathpatch, z = 0, normal = 'z'):
"""
Transforms a 2D Patch to a 3D patch using the given normal vector.
The patch is projected into they XY plane, rotated about the origin
and finally translated by z.
"""
if type(normal) is str: #Translate strings to normal vectors
index = "xyz".index(normal)
normal = np.roll((1.0,0,0), index)
normal /= np.linalg.norm(normal) #Make sure the vector is normalised
path = pathpatch.get_path() #Get the path and the associated transform
trans = pathpatch.get_patch_transform()
path = trans.transform_path(path) #Apply the transform
pathpatch.__class__ = art3d.PathPatch3D #Change the class
pathpatch._code3d = path.codes #Copy the codes
pathpatch._facecolor3d = pathpatch.get_facecolor #Get the face color
verts = path.vertices #Get the vertices in 2D
d = np.cross(normal, (0, 0, 1)) #Obtain the rotation vector
M = rotation_matrix(d) #Get the rotation matrix
pathpatch._segment3d = np.array([np.dot(M, (x, y, 0)) + (0, 0, z) for x, y in verts])
def pathpatch_translate(pathpatch, delta):
"""
Translates the 3D pathpatch by the amount delta.
"""
pathpatch._segment3d += delta
Long answer
Looking at the source code of art3d.pathpatch_2d_to_3d gives the following call hierarchy
art3d.pathpatch_2d_to_3d
art3d.PathPatch3D.set_3d_properties
art3d.Patch3D.set_3d_properties
art3d.juggle_axes
The transformation from 2D to 3D happens in the last call to art3d.juggle_axes. Modifying this last step, we can obtain patches in 3D with arbitrary normals.
We proceed in four steps
Project the vertices of the patch into the XY plane (pathpatch_2d_to_3d)
Calculate a rotation matrix R that rotates the z direction to the direction of the normal (rotation_matrix)
Apply the rotation matrix to all vertices (pathpatch_2d_to_3d)
Translate the resulting object in the z-direction (pathpatch_2d_to_3d)
Sample source code and the resulting plot are shown below.
from mpl_toolkits.mplot3d import proj3d
from matplotlib.patches import Circle
from itertools import product
ax = axes(projection = '3d') #Create axes
p = Circle((0,0), .2) #Add a circle in the yz plane
ax.add_patch(p)
pathpatch_2d_to_3d(p, z = 0.5, normal = 'x')
pathpatch_translate(p, (0, 0.5, 0))
p = Circle((0,0), .2, facecolor = 'r') #Add a circle in the xz plane
ax.add_patch(p)
pathpatch_2d_to_3d(p, z = 0.5, normal = 'y')
pathpatch_translate(p, (0.5, 1, 0))
p = Circle((0,0), .2, facecolor = 'g') #Add a circle in the xy plane
ax.add_patch(p)
pathpatch_2d_to_3d(p, z = 0, normal = 'z')
pathpatch_translate(p, (0.5, 0.5, 0))
for normal in product((-1, 1), repeat = 3):
p = Circle((0,0), .2, facecolor = 'y', alpha = .2)
ax.add_patch(p)
pathpatch_2d_to_3d(p, z = 0, normal = normal)
pathpatch_translate(p, 0.5)

Very useful piece of code, but there is a small caveat: it cannot handle normals pointing downwards because it uses only the sine of the angle.
You need to use also the cosine:
from mpl_toolkits.mplot3d import Axes3D
from mpl_toolkits.mplot3d import art3d
from mpl_toolkits.mplot3d import proj3d
import numpy as np
def rotation_matrix(v1,v2):
"""
Calculates the rotation matrix that changes v1 into v2.
"""
v1/=np.linalg.norm(v1)
v2/=np.linalg.norm(v2)
cos_angle=np.dot(v1,v2)
d=np.cross(v1,v2)
sin_angle=np.linalg.norm(d)
if sin_angle == 0:
M = np.identity(3) if cos_angle>0. else -np.identity(3)
else:
d/=sin_angle
eye = np.eye(3)
ddt = np.outer(d, d)
skew = np.array([[ 0, d[2], -d[1]],
[-d[2], 0, d[0]],
[d[1], -d[0], 0]], dtype=np.float64)
M = ddt + cos_angle * (eye - ddt) + sin_angle * skew
return M
def pathpatch_2d_to_3d(pathpatch, z = 0, normal = 'z'):
"""
Transforms a 2D Patch to a 3D patch using the given normal vector.
The patch is projected into they XY plane, rotated about the origin
and finally translated by z.
"""
if type(normal) is str: #Translate strings to normal vectors
index = "xyz".index(normal)
normal = np.roll((1,0,0), index)
path = pathpatch.get_path() #Get the path and the associated transform
trans = pathpatch.get_patch_transform()
path = trans.transform_path(path) #Apply the transform
pathpatch.__class__ = art3d.PathPatch3D #Change the class
pathpatch._code3d = path.codes #Copy the codes
pathpatch._facecolor3d = pathpatch.get_facecolor #Get the face color
verts = path.vertices #Get the vertices in 2D
M = rotation_matrix(normal,(0, 0, 1)) #Get the rotation matrix
pathpatch._segment3d = np.array([np.dot(M, (x, y, 0)) + (0, 0, z) for x, y in verts])
def pathpatch_translate(pathpatch, delta):
"""
Translates the 3D pathpatch by the amount delta.
"""
pathpatch._segment3d += delta

Here's a more generalmethod that allows embedding in more complex ways than along a normal:
class EmbeddedPatch2D(art3d.PathPatch3D):
def __init__(self, patch, transform):
assert transform.shape == (4, 3)
self._patch2d = patch
self.transform = transform
self._path2d = patch.get_path()
self._facecolor2d = patch.get_facecolor()
self.set_3d_properties()
def set_3d_properties(self, *args, **kwargs):
# get the fully-transformed path
path = self._patch2d.get_path()
trans = self._patch2d.get_patch_transform()
path = trans.transform_path(path)
# copy across the relevant properties
self._code3d = path.codes
self._facecolor3d = self._patch2d.get_facecolor()
# calculate the transformed vertices
verts = np.empty(path.vertices.shape + np.array([0, 1]))
verts[:,:-1] = path.vertices
verts[:,-1] = 1
self._segment3d = verts.dot(self.transform.T)[:,:-1]
def __getattr__(self, key):
return getattr(self._patch2d, key)
To use this as desired in the question, we need a helper function
def matrix_from_normal(normal):
"""
given a normal vector, builds a homogeneous rotation matrix such that M.dot([1, 0, 0]) == normal
"""
normal = normal / np.linalg.norm(normal)
res = np.eye(normal.ndim+1)
res[:-1,0] = normal
if normal [0] == 0:
perp = [0, -normal[2], normal[1]]
else:
perp = np.cross(normal, [1, 0, 0])
perp /= np.linalg.norm(perp)
res[:-1,1] = perp
res[:-1,2] = np.cross(self.dir, perp)
return res
All together:
circ = Circle((0,0), .2, facecolor = 'y', alpha = .2)
# the matrix here turns (x, y, 1) into (0, x, y, 1)
mat = matrix_from_normal([1, 1, 0]).dot([
[0, 0, 0],
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]
])
circ3d = EmbeddedPatch2D(circ, mat)

I want to share my solution that extends the former proposals.
It enables both 3d elements and text to be added to a Axes3D presentation.
# creation of a rotation matrix that preserves the x-axis in an xy-plane of the original coordinate system
def rotationMatrix(normal):
norm = np.linalg.norm(normal)
if norm ==0: return Rotation.identity(None)
zDir = normal/norm
if np.abs(zDir[2])==1:
yDir = np.array([0,zDir[2],0])
else:
yDir = (np.array([0,0,1]) - zDir[2]*zDir)/math.sqrt(1-zDir[2]**2)
rotMat = np.empty((3,3))
rotMat[:,0] = np.cross(zDir,yDir)
rotMat[:,1] = yDir
rotMat[:,2] = -zDir
return Rotation.from_matrix(rotMat)
def toVector(vec):
if vec is None or not isinstance(vec,np.ndarray) : vec="z"
if isinstance(vec,str):
zdir = vec[0] if len(vec)>0 else "z"
if not zdir in "xyz": zdir="z"
index = "xyz".index(vec)
return np.roll((1.0,0,0), index)
else:
return vec
# Transforms a 2D Patch to a 3D patch using a pivot point and a the given normal vector.
def pathpatch_2d_to_3d(pathpatch, pivot=np.zeros(3), zDir='z'):
path = pathpatch.get_path() #Get the path and the associated transform
trans = pathpatch.get_patch_transform()
path = trans.transform_path(path) #Apply the transform
pathpatch.__class__ = mplot3d.art3d.PathPatch3D #Change the class
pathpatch._path2d = path #Copy the 2d path
pathpatch._code3d = path.codes #Copy the codes
pathpatch._facecolor3d = pathpatch.get_facecolor #Get the face color
# Get the 2D vertices and add the third dimension
verts3d = np.empty((path.vertices.shape[0],3))
verts3d[:,0:2] = path.vertices
verts3d[:,2] = pivot[2]
R = rotationMatrix(toVector(zDir))
pathpatch._segment3d = R.apply(verts3d - pivot) + pivot
return pathpatch
# places a 3D text element in axes with 3d projection.
def text3d(xyz, text, zDir="z", scalefactor=1.0, fp=FontProperties(), **kwargs):
pt = PathPatch(TextPath(xyz[0:2], text, size=scalefactor*fp.get_size(), prop=fp , usetex=False),**kwargs)
ax3D.add_patch(pathpatch_2d_to_3d(pt, xyz, zDir))
# places a 3D circle in axes with 3d projection.
def circle3d(center, radius, zDir='z', **kwargs):
pc = Circle(center[0:2], radius, **kwargs)
ax3D.add_patch(pathpatch_2d_to_3d(pc, center, zDir))

OR-Tools Python - AddNoOverlap2D not working - Stock Cutting Problem

Trying to cut given set of rectangles from a large rectangle. The program is running fine but it is not respecting the AddNoOverlap2D constraint.
The program outputs
0, 0 -> 2, 2
0, 0 -> 1, 3
0, 0 -> 4, 3
All the coordinates of rectangles output by program have (0,0) as first point and hence are overlaping. I want get the rectangles that are not overlaping?
I am using model.AddNoOverlap2D constraint and the objective I have set is to minimize the unused area of large rectangle. Complete Code:
from __future__ import print_function
import collections
from ortools.sat.python import cp_model
def StockCutter():
"""Cutting Stock problem."""
# Create the model
model = cp_model.CpModel()
# rect = [width, height]
rects_data = [
[2, 2],
[1, 3],
[4, 3]
]
rect_ids = range(len(rects_data))
# parent rect (to cut from)
horizon = [6, 6]
print("Horizon: ", horizon)
# Named tuple to store information about created variables
rect_type = collections.namedtuple('rect_type', 'x1 y1 x2 y2 x_interval y_interval')
all_vars = {}
# to save area of all small rects, to cut from parent rect
total_area = 0
# x_intervals holds the widths of each rect
x_intervals = collections.defaultdict(list)
# y_intervals holds the lengths of each rect
y_intervals = collections.defaultdict(list)
for rect_id, rect in enumerate(rects_data):
width = rect[0]
height = rect[1]
area = width * height
total_area += area
print(f"Rect: {width}x{height}, Area: {area}")
suffix = '_%i_%i' % (width, height)
# interval to represent width
x1_var = model.NewIntVar(0, horizon[0], 'x1' + suffix)
x2_var = model.NewIntVar(0, horizon[0], 'x2' + suffix)
x_interval_var = model.NewIntervalVar(x1_var, width, x2_var, 'x_interval' + suffix)
# interval to represent height
y1_var = model.NewIntVar(0, horizon[1], 'y1' + suffix)
y2_var = model.NewIntVar(0, horizon[1], 'y2' + suffix)
y_interval_var = model.NewIntervalVar(y1_var, height, y2_var, 'y_interval' + suffix)
all_vars[rect_id] = rect_type(
x1=x1_var,
y1=y1_var,
x2=x2_var,
y2=y2_var,
x_interval=x_interval_var,
y_interval=y_interval_var
)
x_intervals[rect_id].append(x_interval_var)
y_intervals[rect_id].append(y_interval_var)
# NOT WORKING???
for rect_id in rect_ids:
model.AddNoOverlap2D(x_intervals[rect_id], y_intervals[rect_id])
# objective: Area of parent (horizon) is max that the sum of all the rectangles' areas can have
obj_var = model.NewIntVar(0, horizon[0]*horizon[1], 'area')
# minimize the area not used
model.Minimize(obj_var - total_area)
# Solve model
solver = cp_model.CpSolver()
status = solver.Solve(model)
if status == cp_model.OPTIMAL:
# print coords
for rect_id, rect in enumerate(rects_data):
x1=solver.Value(all_vars[rect_id].x1)
y1=solver.Value(all_vars[rect_id].y1)
x2=solver.Value(all_vars[rect_id].x2)
y2=solver.Value(all_vars[rect_id].y2)
print(f"{x1}, {y1} -> {x2}, {y2}")
StockCutter()

You should only call AddNoOverlap2D once with the list of x_intervals and y_intervals:
# x_intervals holds the widths of each rect
x_intervals = []
# y_intervals holds the lengths of each rect
y_intervals = []
for rect_id, rect in enumerate(rects_data):
...
x_intervals.append(x_interval_var)
y_intervals.append(y_interval_var)
model.AddNoOverlap2D(x_intervals, y_intervals)

How to use vtkOBBTree and IntersectWithLine to find the intersection of a line and a PolyDataFilter in python using vtk?

I have an oriented cylinder generated with vtkCylinderSource and some transformations are applied on it to get the orientation that i want. Here is the code for creating this oriented-cylinder:
def cylinder_object(startPoint, endPoint, radius, my_color="DarkRed", opacity=1):
colors = vtk.vtkNamedColors()
# Create a cylinder.
# Cylinder height vector is (0,1,0).
# Cylinder center is in the middle of the cylinder
cylinderSource = vtk.vtkCylinderSource()
cylinderSource.SetRadius(radius)
cylinderSource.SetResolution(50)
# Generate a random start and end point
# startPoint = [0] * 3
# endPoint = [0] * 3
rng = vtk.vtkMinimalStandardRandomSequence()
rng.SetSeed(8775070) # For testing.8775070
# Compute a basis
normalizedX = [0] * 3
normalizedY = [0] * 3
normalizedZ = [0] * 3
# The X axis is a vector from start to end
vtk.vtkMath.Subtract(endPoint, startPoint, normalizedX)
length = vtk.vtkMath.Norm(normalizedX)
vtk.vtkMath.Normalize(normalizedX)
# The Z axis is an arbitrary vector cross X
arbitrary = [0] * 3
for i in range(0, 3):
rng.Next()
arbitrary[i] = rng.GetRangeValue(-10, 10)
vtk.vtkMath.Cross(normalizedX, arbitrary, normalizedZ)
vtk.vtkMath.Normalize(normalizedZ)
# The Y axis is Z cross X
vtk.vtkMath.Cross(normalizedZ, normalizedX, normalizedY)
matrix = vtk.vtkMatrix4x4()
# Create the direction cosine matrix
matrix.Identity()
for i in range(0, 3):
matrix.SetElement(i, 0, normalizedX[i])
matrix.SetElement(i, 1, normalizedY[i])
matrix.SetElement(i, 2, normalizedZ[i])
# Apply the transforms
transform = vtk.vtkTransform()
transform.Translate(startPoint) # translate to starting point
transform.Concatenate(matrix) # apply direction cosines
transform.RotateZ(-90.0) # align cylinder to x axis
transform.Scale(1.0, length, 1.0) # scale along the height vector
transform.Translate(0, .5, 0) # translate to start of cylinder
# Transform the polydata
transformPD = vtk.vtkTransformPolyDataFilter()
transformPD.SetTransform(transform)
transformPD.SetInputConnection(cylinderSource.GetOutputPort())
cylinderSource.Update()
# Create a mapper and actor for the arrow
mapper = vtk.vtkPolyDataMapper()
actor = vtk.vtkActor()
if USER_MATRIX:
mapper.SetInputConnection(cylinderSource.GetOutputPort())
actor.SetUserMatrix(transform.GetMatrix())
else:
mapper.SetInputConnection(transformPD.GetOutputPort())
actor.SetMapper(mapper)
actor.GetProperty().SetColor(colors.GetColor3d(my_color))
actor.GetProperty().SetOpacity(opacity)
return actor, transformPD
Now i want to ray cast a line with this oriented cylinder. unfortunately, using the vtkCylinderSource as the dataset for vtkOBBTree produces the wrong points as the result. how can i use ray-casting with a PolyDataFilter?
I came up with a solution where i export my oriented-cylinder to a .stl file and then read it again to implement the ray-casting algorithm using IntersectWithLine. The problem is i have thousands of these oriented-cylinders and this method (exporting and reading) makes my code extremely slow.
def ray_cast(filename, p_source, p_target):
'''
:param filename: STL file to perform ray casting on.
:param p_source: first point
:param p_target: second point
:return: code --> 0 : No intersection.
:return: code --> +1 : p_source lies OUTSIDE the closed surface.
:return; code --> -1 : p_source lies INSIDE closed surface
'''
reader = vtk.vtkSTLReader()
reader.SetFileName(filename)
reader.Update()
mesh = reader.GetOutput()
obbtree = vtk.vtkOBBTree()
obbtree.SetDataSet(mesh)
obbtree.BuildLocator()
pointsVTKIntersection = vtk.vtkPoints()
code = obbtree.IntersectWithLine(p_source, p_target, pointsVTKIntersection, None)
# Extracting data
pointsVTKIntersectionData = pointsVTKIntersection.GetData()
noPointsVTKIntersection = pointsVTKIntersectionData.GetNumberOfTuples()
pointsIntersection = []
for idx in range(noPointsVTKIntersection):
_tup = pointsVTKIntersectionData.GetTuple3(idx)
pointsIntersection.append(_tup)
return code, pointsIntersection, noPointsVTKIntersection
Below image shows the desired result using export-stl method. (the green spheres are intersection points)
I would appreciate any suggestion and help..

With vedo:
from vedo import *
cyl = Cylinder() # vtkActor
cyl.alpha(0.5).pos(3,3,3).orientation([2,1,1])
p1, p2 = (0,0,0), (4,4,5)
ipts_coords = cyl.intersectWithLine(p1, p2)
print('hit coords are', ipts_coords)
pts = Points(ipts_coords, r=10).color("yellow")
# print(pts.polydata()) # is the vtkPolyData object
origin = Point()
ln = Line(p1,p2)
show(origin, cyl, ln, pts, axes=True)

Can matplotlib contours match pixel edges?

How to outline pixel boundaries in matplotlib? For instance, for a semi-random dataset like the one below,
# the code block that follows is irrelevant
import numpy as np
k = []
for s in [2103, 1936, 2247, 2987]:
np.random.seed(s)
k.append(np.random.randint(0, 2, size=(2,6)))
arr = np.hstack([np.vstack(k)[:, :-1], np.vstack(k).T[::-1].T ])
image = np.zeros(shape=(arr.shape[0]+2, arr.shape[1]+2))
image[1:-1, 1:-1] = arr
it is quite clear that a contour matching the pixel edges of image would be preferred to the default behavior of the contour function, where the contour lines are effectively drawn across the diagonals of edge pixels.
import matplotlib.pyplot as plt
plt.contour(image[::-1], [0.5], colors='r')
How to make the contours align with the pixels? I'm looking for a solution within numpy and matplotlib libraries.

If the image has a resolution of 1 pixel per unit, how would you define the "edge" of a pixel? The notion of "edge" only makes sense in a frame of increased resolution compared to the pixel itself and contour cannot draw any edges if it is working with the same resoltion as the image itself.
On the other hand, it is of course possible to increase the resolution such that the notion "edge" carries a meaning. So let's say we increase the resolution by a factor of 100 we can easily draw the edges using a contour plot.
import matplotlib.pyplot as plt
import numpy as np
k = []
for s in [2103, 1936, 2247, 2987]:
np.random.seed(s)
k.append(np.random.randint(0, 2, size=(2,6)))
arr = np.hstack([np.vstack(k)[:, :-1], np.vstack(k).T[::-1].T ])
image = np.zeros(shape=(arr.shape[0]+2, arr.shape[1]+2))
image[1:-1, 1:-1] = arr
f = lambda x,y: image[int(y),int(x) ]
g = np.vectorize(f)
x = np.linspace(0,image.shape[1], image.shape[1]*100)
y = np.linspace(0,image.shape[0], image.shape[0]*100)
X, Y= np.meshgrid(x[:-1],y[:-1])
Z = g(X[:-1],Y[:-1])
plt.imshow(image[::-1], origin="lower", interpolation="none", cmap="Blues")
plt.contour(Z[::-1], [0.5], colors='r', linewidths=[3],
extent=[0-0.5, x[:-1].max()-0.5,0-0.5, y[:-1].max()-0.5])
plt.show()
For comparison, we can also draw the image itself in the same plot using imshow.

contour_rect_slow draws slingle lines at the boundaries between pixels with values 0 and 1. contour_rect is a more compact version, connecting longer lines to a single line.
Code:
import numpy as np
k = []
for s in [2103, 1936, 2247, 2987]:
np.random.seed(s)
k.append(np.random.randint(0, 2, size=(2,6)))
arr = np.hstack([np.vstack(k)[:, :-1], np.vstack(k).T[::-1].T ])
image = np.zeros(shape=(arr.shape[0]+2, arr.shape[1]+2))
image[1:-1, 1:-1] = arr[::1]
# image[1, 1] = 1
import matplotlib.pyplot as plt
plt.imshow(image, interpolation="none", cmap="Blues")
def contour_rect_slow(im):
"""Clear version"""
pad = np.pad(im, [(1, 1), (1, 1)]) # zero padding
im0 = np.abs(np.diff(pad, n=1, axis=0))[:, 1:]
im1 = np.abs(np.diff(pad, n=1, axis=1))[1:, :]
lines = []
for ii, jj in np.ndindex(im0.shape):
if im0[ii, jj] == 1:
lines += [([ii-.5, ii-.5], [jj-.5, jj+.5])]
if im1[ii, jj] == 1:
lines += [([ii-.5, ii+.5], [jj-.5, jj-.5])]
return lines
def contour_rect(im):
"""Fast version"""
lines = []
pad = np.pad(im, [(1, 1), (1, 1)]) # zero padding
im0 = np.abs(np.diff(pad, n=1, axis=0))[:, 1:]
im1 = np.abs(np.diff(pad, n=1, axis=1))[1:, :]
im0 = np.diff(im0, n=1, axis=1)
starts = np.argwhere(im0 == 1)
ends = np.argwhere(im0 == -1)
lines += [([s[0]-.5, s[0]-.5], [s[1]+.5, e[1]+.5]) for s, e
in zip(starts, ends)]
im1 = np.diff(im1, n=1, axis=0).T
starts = np.argwhere(im1 == 1)
ends = np.argwhere(im1 == -1)
lines += [([s[1]+.5, e[1]+.5], [s[0]-.5, s[0]-.5]) for s, e
in zip(starts, ends)]
return lines
lines = contour_rect(image)
for line in lines:
plt.plot(line[1], line[0], color='r', alpha=1)
Warning: This is significantly slower then mpl.contour for large images..