I'm doing some stuff with 2D opengl rendering.
Is there a way to render a vertex array object but have the data be passed through multiple shaders? For example, a shader that applies a normal map to the texture, and then a shader that blurs the image. It would be very difficult and unclean to combine the two shaders into one let alone potentially combining more than 2 shaders. This is my current code for creating the vertex array object:
# TEX_COORDS = [0, 1, 1, 1,
# 0, 0, 1, 0]
# TEX_INDICES = [0, 1, 2,
# 1, 2, 3]
# self.vertices looks something like this: [-1, -1, 1, -1, -1, 1, 1, 1], but with different coordinates
self.vbo = self.ctx.buffer(struct.pack("8f", *self.vertices))
self.uv_map = self.ctx.buffer(struct.pack("8f", *TEX_COORDS))
self.ibo = self.ctx.buffer(struct.pack("6I", *TEX_INDICES))
self.vao_content = [(self.vbo, "2f", "vertexPos"), (self.uv_map, "2f", "vertexTexCoord")]
self.vao = self.ctx.vertex_array(self.program, self.vao_content, self.ibo) # self.program is the shader program object
And I'm doing texture.use() (texture being a moderngl texture object) and then self.vao.render() to render it onto the screen.
A single rendering call will only ever use a single set of vertex, fragment, and other shaders. You cannot chain together shaders for a particular stage via the API; you must manufacture a single such shader that does those multiple things.
How you go about that process is up to you. You can have one shader that has all possible operations, with a bunch of uniform variables that define which operations will be applied. Or you can dynamically build shaders to fit particular needs.
I want to get the image in the x-axis and y-axis differential directions. But, I want to code using np.gradient function, without using cv2.Sobel filter.
So, I want to convert code like this
x_grad = cv2.Sobel(image1, cv2.CV_64F, 1, 0, ksize=5)
to something like this
x_grad = np.gradient(image1, axis=0)
But, the values are not the same. How should I fix it?
np.gradient has limited functionality, and only a few cases of cv2.Sobel can be somehow reproduced, namely the following:
import cv2
import numpy as np
image = cv2.imread('path/to/your/image.png', cv2.IMREAD_GRAYSCALE)
grad_cv2 = cv2.Sobel(image, cv2.CV_64F, 1, 0, ksize=1)
grad_np = 2 * np.gradient(image, axis=1)
grad_np[:, [0, -1]] = 0
print(np.all(grad_cv2 == grad_np))
# True
grad_cv2 = cv2.Sobel(image, cv2.CV_64F, 0, 1, ksize=1)
grad_np = 2 * np.gradient(image, axis=0)
grad_np[[0, -1], :] = 0
print(np.all(grad_cv2 == grad_np))
# True
grad_cv2 = cv2.Sobel(image, cv2.CV_64F, 1, 1, ksize=3)
grad_np = 2 * np.gradient(2 * np.gradient(image, axis=1), axis=0)
grad_np[:, [0, -1]] = 0
grad_np[[0, -1], :] = 0
print(np.all(grad_cv2 == grad_np))
# True
Factor 2 is needed, since np.gradient divides by 2 per default.
Setting the border(s) to zero must be done, because of the following (emphasis by me):
The gradient is computed using second order accurate central differences in the interior points and either first or second order accurate one-sides (forward or backwards) differences at the boundaries.
On the other hand, cv2.Sobel, per default, uses cv2.BORDER_REFLECT_101, such that you always get 0 for the borders.
Most other cases of cv2.Sobel will involve (Gaussian) smoothing, cf. the linked documentation:
The Sobel operators combine Gaussian smoothing and differentiation, so the result is more or less resistant to the noise.
That can't be attacked solely using np.gradient. If you want to have that, you're going to rewrite cv2.Sobel using NumPy methods.
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.9.1
NumPy: 1.20.2
OpenCV: 4.5.1
----------------------------------------
I have an arbitrary input curve, given as numpy array. I want to create a smoothed version of it, similar to a rolling mean, but which is strictly greater than the original and strictly smooth. I could use the rolling mean value but if the input curve has a negative peak, the smoothed version will drop below the original around that peak. I could then simply use the maximum of this and the original but that would introduce non-smooth spots where the transition occurs.
Furthermore, I would like to be able to parameterize the algorithm with a look-ahead and a look-behind for this resulting curve, so that given a large look-ahead and a small look-behind the resulting curve would rather stick to the falling edges, and with a large look-behind and a small look-ahead it would rather be close to rising edges.
I tried using the pandas.Series(a).rolling() facility to get rolling means, rolling maxima, etc., but up to now I found no way to generate a smoothed version of my input which in all cases stays above to input.
I guess there is a way to combine rolling maxima and rolling means somehow to achieve what I want, so here is some code for computing these:
import pandas as pd
import numpy as np
my input curve:
original = np.array([ 5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7 ])
This can be padded left (pre) and right (post) with the edge values as a preparation for any rolling function:
pre = 2
post = 3
padded = np.pad(original, (pre, post), 'edge')
Now we can apply a rolling mean:
smoothed = pd.Series(padded).rolling(
pre + post + 1).mean().get_values()[pre+post:]
But now the smoothed version is below the original, e. g. at index 4:
print(original[4], smoothed[4]) # 8 and 5.5
To compute a rolling maximum, you can use this:
maximum = pd.Series(padded).rolling(
pre + post + 1).max().get_values()[pre+post:]
But a rolling maximum alone would of course not be smooth in many cases and would display a lot of flat tops around the peaks of the original. I would prefer a smooth approach to these peaks.
If you have also pyqtgraph installed, you can easily plot such curves:
import pyqtgraph as pg
p = pg.plot(original)
p.plotItem.plot(smoothed, pen=(255,0,0))
(Of course, other plot libraries would do as well.)
What I would like to have as a result is a curve which is e. g. like the one formed by these values:
goal = np.array([ 5, 7, 7.8, 8, 8, 8, 7, 5, 3.5, 3, 4, 5.5, 6.5, 7 ])
Here is an image of the curves. The white line is the original (input), the red the rolling mean, the green is about what I would like to have:
EDIT: I just found the functions baseline() and envelope() of a module named peakutils. These two functions can compute polynomials of a given degree fitting the lower resp. upper peaks of the input. For small samples this can be a good solution. I'm looking for something which can also be applied on very large samples with millions of values; then the degree would need to be very high and the computation then also takes a considerate amount of time. Doing it piecewise (section for section) opens up a bunch of new questions and problems (like how to stitch properly while staying smooth and guaranteed above the input, performance when processing a massive amount of pieces etc.), so I'd like to avoid that if possible.
EDIT 2: I have a promising approach by a repetitively applying a filter which creates a rolling mean, shifts it slightly to the left and the right, and then takes the maximum of these two and the original sample. After applying this several times, it smoothes out the curve in the way I wanted it. Some unsmooth spots can remain, though, in deep valleys. Here is the code for this:
pre = 30
post = 30
margin = 10
s = [ np.array(sum([[ x ] * 100 for x in
[ 5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7 ]], [])) ]
for _ in range(30):
s.append(np.max([
pd.Series(np.pad(s[-1], (margin+pre, post), 'edge')).rolling(
1 + pre + post).mean().get_values()[pre+post:-margin],
pd.Series(np.pad(s[-1], (pre, post+margin), 'edge')).rolling(
1 + pre + post).mean().get_values()[pre+post+margin:],
s[-1]], 0))
This creates 30 iterations of applying the filter, plotting these can be done using pyqtplot so:
p = pg.plot(original)
for q in s:
p.plotItem.plot(q, pen=(255, 100, 100))
The resulting image looks like this:
There are two aspects I don't like about this approach: ① It needs iterating lots of time (which slows me down), ② it still has unsmooth parts in the valleys (although in my usecase this might be acceptable).
I have now played around quite a bit and I think I found two main answers which solve my direct need. I will give them below.
import numpy as np
import pandas as pd
from scipy import signal
import pyqtgraph as pg
These are just the necessary imports, used in all code blow. pyqtgraph is only used for displaying stuff of course, so you do not really need it.
Symmetrical Smoothing
This can be used to create a smooth line which is always above the signal, but it cannot distinguish between rising and falling edges, so the curve around a single peak will look symmetrical. In many cases this might be quite okay and since it is way less complex than the asymmetrical solution below (and also does not have any quirks I would know about).
s = np.repeat([5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7], 400) + 0.1
s *= np.random.random(len(s))
pre = post = 400
x = pd.Series(np.pad(s, (pre, post), 'edge')).rolling(
pre + 1 + post).max().get_values()[pre+post:]
y = pd.Series(np.pad(x, (pre, post), 'edge')).rolling(
pre + 1 + post, win_type='blackman').mean().get_values()[pre+post:]
p = pg.plot(s, pen=(100,100,100))
for c, pen in ((x, (0, 200, 200)),
(y, pg.mkPen((255, 255, 255), width=3, style=3))):
p.plotItem.plot(c, pen=pen)
Create a rolling maximum (x, cyan), and
create a windowed rolling mean of this (z, white dotted).
Asymmetrical Smoothing
My usecase called for a version which allowed to distinguish between rising and falling edges. The speed of the output should be different when falling or when rising.
Comment: Used as an envelope for a compressor/expander, a quickly rising curve would mean to dampen the effect of a sudden loud noise almost completely, while a slowly rising curve would mean to slowly compress the signal for a long time before the loud noise, keeping the dynamics when the bang appears. On the other hand, if the curve falls quickly after a loud noise, this would make quiet stuff shortly after a bang audible while a slowly falling curve would keep the dynamics there as well and only slowly expanding the signal back to normal levels.
s = np.repeat([5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7], 400) + 0.1
s *= np.random.random(len(s))
pre, post = 100, 1000
t = pd.Series(np.pad(s, (post, pre), 'edge')).rolling(
pre + 1 + post).max().get_values()[pre+post:]
g = signal.get_window('boxcar', pre*2)[pre:]
g /= g.sum()
u = np.convolve(np.pad(t, (pre, 0), 'edge'), g)[pre:]
g = signal.get_window('boxcar', post*2)[:post]
g /= g.sum()
v = np.convolve(np.pad(t, (0, post), 'edge'), g)[post:]
u, v = u[:len(v)], v[:len(u)]
w = np.min(np.array([ u, v ]),0)
pre = post = max(100, min(pre, post)*3)
x = pd.Series(np.pad(w, (pre, post), 'edge')).rolling(
pre + 1 + post).max().get_values()[pre+post:]
y = pd.Series(np.pad(x, (pre, post), 'edge')).rolling(
pre + 1 + post, win_type='blackman').mean().get_values()[pre+post:]
p = pg.plot(s, pen=(100,100,100))
for c, pen in ((t, (200, 0, 0)),
(u, (200, 200, 0)),
(v, (0, 200, 0)),
(w, (200, 0, 200)),
(x, (0, 200, 200)),
(y, pg.mkPen((255, 255, 255), width=3))):
p.plotItem.plot(c, pen=pen)
This sequence combines ruthlessly several methods of signal processing.
The input signal is shown in grey. It is a noisy version of the input mentioned above.
A rolling maximum is applied to this (t, red).
Then a specially designed convolution curve for the falling edges is created (g) and the convolution is computed (u, yellow).
This is repeated for the rising edges with a different convolution curve (g again) and the convolution is computed (v, green).
The minimum of u and v is a curve having the desired slopes but is not very smooth yet; especially it has ugly spikes when the falling and the rising slope reach into each other (w, purple).
On this the symmetrical method above is applied:
Create a rolling maximum of this curve (x, cyan).
Create a windowed rolling mean of this curve (y, white dotted).
As an initial stab at part of the problem, I've produced a function which fits a polynomial to the data by minimising the integral subject to constraints that the polynomial be strictly above the points. I suspect if you do this piecewise over your data, it may work for you.
import scipy.optimize
def upperpoly(xdata, ydata, order):
def objective(p):
"""Minimize integral"""
pint = np.polyint(p)
integral = np.polyval(pint, xdata[-1]) - np.polyval(pint, xdata[0])
return integral
def constraints(p):
"""Polynomial values be > data at every point"""
return np.polyval(p, xdata) - ydata
p0 = np.polyfit(xdata, ydata, order)
y0 = np.polyval(p0, xdata)
shift = (ydata - y0).max()
p0[-1] += shift
result = scipy.optimize.minimize(objective, p0,
constraints={'type':'ineq',
'fun': constraints})
return result.x
As pointed out in my note, the behaviour of your green line is inconsistent in the regions before and after the eight-high plateau. If the left region behavior is what you want, you could do something like this:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import interp1d
from scipy.spatial import ConvexHull
# %matplotlib inline # for interactive notebooks
y=np.array([ 5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7])
x=np.array(range(len(y)))
#######
# This essentially selects the vertices that you'd touch streatching a
# rubber band over the top of the function
vs = ConvexHull(np.asarray([x,y]).transpose()).vertices
indices_of_upper_hull_verts = list(reversed(np.concatenate([vs[np.where(vs == len(x)-1)[0][0]: ],vs[0:1]])))
newX = x[indices_of_upper_hull_verts]
newY = y[indices_of_upper_hull_verts]
#########
x_smooth = np.linspace(newX.min(), newX.max(),500)
f = interp1d(newX, newY, kind='quadratic')
y_smooth=f(x_smooth)
plt.plot (x,y)
plt.plot (x_smooth,y_smooth)
plt.scatter (x, y)
which yields:
UPDATE:
Here's an alternative that might better suit you. If instead of a rolling average you use a simple convolution centered around 1, the resulting curve will always be larger than the input. Wings of the convolution kernel can be adjusted for look-ahead/look-behind.
Like this:
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import interp1d
from scipy.ndimage.filters import convolve
## For interactive notebooks
#%matplotlib inline
y=np.array([ 5, 5, 5, 8, 8, 8, 2, 2, 2, 2, 2, 3, 3, 7]).astype(np.float)
preLength = 1
postLength = 1
preWeight = 0.2
postWeight = 0.2
kernal = [preWeight/preLength for i in range(preLength)] + [1] + [postWeight/postLength for i in range(postLength)]
output = convolve(y,kernal)
x=np.array(range(len(y)))
plt.plot (x,y)
plt.plot (x,output)
plt.scatter (x, y)
A drawback is that because the integrated kernel will typically be larger than one (which ensures that the output curve is smooth and never below the input), the output curve will always be larger than the input curve, e.g. on top of the large peak and not sitting right on top as you drew.
I wrote a Python script using matplotlib, to visualize and rotate polycubes together with planes or axes that reveal symmetry in the polycube. Up to a certain point I succeeded as you can see from the illustrations below. There you see one of seventy-seven possible 3D polycubes of six cubes. My script uses matplotlib and it seems to do a nice job: matplotlib allows me to rotate the polycube and view it at any wanted angle. There is a big problem with most polycubes, which is illustrated in the second image below. A little after I start rotating the polycube, matplotlib shows planes that are partly behind other planes and are therefore partly invisble and not to be drawn or only partly drawn.
I have been searching a lot on forums and with Google, but to no avail. There were hits suggesting that I should use mayavi instead of matplotlib. So I studied extensively on mayavi. I spent literally weeks trying to figure out how to get mayavi going. The hits on docs.enthought seemed promising at first, but although mayavi is clearly suited to the purpose and superb in visualizing objects, I cannot find understandable documentation. I could use a real programmers guide on vtk or tvtk. There is a lot of documentation, but mostly for designers, not programmers. If not available (?) I would also be happy with an example of a script with hexahedrons or irregular grids, that works in canopy's (1.6.2) implementation of python.
I've modified an example from the vtk documentation (http://www.vtk.org/gitweb?p=VTK.git;a=blob_plain;f=Examples/DataManipulation/Python/BuildUGrid.py)
This example can be turned into a function which can create the geometry from a point coordinate array and an element array.
I haven't used any guides for vtk, I usually refer to the python examples found here: http://www.vtk.org/Wiki/VTK/Examples/Python
import vtk
# Initialize the vtkPoints variable and set the number of points
points = vtk.vtkPoints()
points.SetNumberOfPoints(8)
# Add points to the variable, with the point number first, then the x, y, z coordinates.
# For demonstration purposes, I started numbering the ponts at 10 (normally they would start at 0).
points.InsertPoint(0, 0, 0, 0)
points.InsertPoint(1, 1, 0, 0)
points.InsertPoint(2, 1, 1, 0)
points.InsertPoint(3, 0, 1, 0)
points.InsertPoint(4, 0, 0, 1)
points.InsertPoint(5, 1, 0, 1)
points.InsertPoint(6, 1, 1, 1)
points.InsertPoint(7, 0, 1, 1)
points.InsertPoint(8, 0, 0, 1.1)
points.InsertPoint(9, 1, 0, 1.1)
points.InsertPoint(10, 1, 1, 1.1)
points.InsertPoint(11, 0, 1, 1.1)
points.InsertPoint(12, 0, 0, 2)
points.InsertPoint(13, 1, 0, 2)
points.InsertPoint(14, 1, 1, 2)
points.InsertPoint(15, 0, 1, 2.5)
# Define the hexahedron, then set the point Ids of the hexahedron cell/element.
# From the documentation: points (0,1,2,3) is the base of the hexahedron which, using the right hand rule, forms a
# quadrilaterial whose normal points in the direction of the opposite face (4,5,6,7)
aHexahedron1 = vtk.vtkHexahedron()
aHexahedron1.GetPointIds().SetId(0, 0) # Cell point 0 corresponds to point 0 which was defined above
aHexahedron1.GetPointIds().SetId(1, 1)
aHexahedron1.GetPointIds().SetId(2, 2)
aHexahedron1.GetPointIds().SetId(3, 3)
aHexahedron1.GetPointIds().SetId(4, 4)
aHexahedron1.GetPointIds().SetId(5, 5)
aHexahedron1.GetPointIds().SetId(6, 6)
aHexahedron1.GetPointIds().SetId(7, 7)
# Define a second hexahedron
aHexahedron2 = vtk.vtkHexahedron()
aHexahedron2.GetPointIds().SetId(0, 8) # Cell point 0 corresponds to point 8 which was defined above
aHexahedron2.GetPointIds().SetId(1, 9)
aHexahedron2.GetPointIds().SetId(2, 10)
aHexahedron2.GetPointIds().SetId(3, 11)
aHexahedron2.GetPointIds().SetId(4, 12)
aHexahedron2.GetPointIds().SetId(5, 13)
aHexahedron2.GetPointIds().SetId(6, 14)
aHexahedron2.GetPointIds().SetId(7, 15)
# Define an unstructured grid.
aHexahedronGrid = vtk.vtkUnstructuredGrid()
# Add the hexahedron to the unstructured grid
# Note: this operation defines the point ids, and not the actual point coordinates
aHexahedronGrid.InsertNextCell(aHexahedron1.GetCellType(), aHexahedron1.GetPointIds())
aHexahedronGrid.InsertNextCell(aHexahedron2.GetCellType(), aHexahedron2.GetPointIds())
# Set the points which includes the coordinates. The point ids defined in the line above correspond to the point ids
# that were defined earlier (i.e. points.InsertPoint(10, 0, 0, 0))
aHexahedronGrid.SetPoints(points)
# Now we have defined one hexahedron, and added it an unstructured grid.
# We could create more hexahedrons, and add them to the same unstructured grid.
# To view the unstructured grid, we need to define a mapper and set the unstructured grid as the input
aHexahedronMapper = vtk.vtkDataSetMapper()
aHexahedronMapper.SetInputData(aHexahedronGrid)
# Define an actor, and set the mapper as the input
aHexahedronActor = vtk.vtkActor()
aHexahedronActor.SetMapper(aHexahedronMapper)
# Create the usual rendering stuff.
ren = vtk.vtkRenderer()
renWin = vtk.vtkRenderWindow()
renWin.AddRenderer(ren)
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)
iren.SetInteractorStyle(vtk.vtkInteractorStyleTrackballCamera()) # Change the rotation type from the default to 'trackball'
ren.SetBackground(.1, .2, .4)
# Add the actor to the renderer to actually view the geometry
ren.AddActor(aHexahedronActor)
# Render the scene and start interaction.
iren.Initialize()
renWin.Render()
iren.Start()
I want to apply rigid body transformations to a large set of 2D image matrices. Ideally, I'd like to be able to just supply an affine transformation matrix specifying both the translation and rotation, apply this in one go, then do cubic spline interpolation on the output.
Unfortunately it seems that affine_transform in scipy.ndimage.interpolation doesn't do translation. I know I could use a combination of shift and rotate, but this is kind of messy and in involves interpolating the output multiple times.
I've also tried using the generic geometric_transformation like this:
import numpy as np
from scipy.ndimage.interpolation import geometric_transformation
# make the affine matrix
def maketmat(xshift,yshift,rotation,dimin=(0,0)):
# centre on the origin
in2orig = np.identity(3)
in2orig[:2,2] = -dimin[0]/2.,-dimin[1]/2.
# rotate about the origin
theta = np.deg2rad(rotation)
rotmat = np.identity(3)
rotmat[:2,:2] = [np.cos(theta),np.sin(theta)],[-np.sin(theta),np.cos(theta)]
# translate to new position
orig2out = np.identity(3)
orig2out[:2,2] = xshift,yshift
# the final affine matrix is just the product
tmat = np.dot(orig2out,np.dot(rotmat,in2orig))
# function that maps output space to input space
def out2in(outcoords,affinemat):
outcoords = np.asarray(outcoords)
outcoords = np.concatenate((outcoords,(1.,)))
incoords = np.dot(affinemat,outcoords)
incoords = tuple(incoords[0:2])
return incoords
def rbtransform(source,xshift,yshift,rotation,outdims):
# source --> target
forward = maketmat(xshift,yshift,rotation,source.shape)
# target --> source
backward = np.linalg.inv(forward)
# now we can use geometric_transform to do the interpolation etc.
tformed = geometric_transform(source,out2in,output_shape=outdims,extra_arguments=(backward,))
return tformed
This works, but it's horribly slow, since it's essentially looping over pixel coordinates! What's a good way to do this?
Can you use the scikit image?
If this is the case, you could try to apply an homography. An homography cab used to represent both translation and rotation at the same time through a 3x3 matrix.
You can use the skimage.transform.fast_homography function.
import numpy as np
import scipy
import skimage.transform
im = scipy.misc.lena()
H = np.asarray([[1, 0, 10], [0, 1, 20], [0, 0, 1]])
skimage.transform.fast_homography(im, H)
The transform took about 30 ms on my old Core 2 Duo.
About homography : http://en.wikipedia.org/wiki/Homography
I think affine_transform does do translation --- there's the offset parameter.