Currently, I'm playing around with a script generating Julia sets and the Mandelbrot set and then using pygame to render the points.
Essentially, the screen is mapped to a smaller coordinate system where its bounded by -2.5, 2.5 on the x axis and -1, 1 on the y axis. Each of the pixels in this mapped range is then passed to a function to check whether its complex number equivalent is in the given set. This function returns the number of iterations it took to calculate whether the number is in the set or not (or the max iterations).
Then, for each pixel, I know what colour to colour it based on this iteration score and render each of the pixels one by one. This part of the process is really intensive and takes a ~30 seconds to render but can be much more depending on the complexity of the set.
Here is the code for finding out if a passed complex number and complex coordinate are in Julia set, this doesn't take long to compute at all when checking 1920 * 1080 pixels:
max_iter = 45
def julia(z, c):
n = 0
while abs(z) <= 2 and n < max_iter:
z = z * z + c
n += 1
return n
Here is the code I use for pygame rendering, this is definitely where the problem lies:
size_ = 1920, 1080
re_ = -2.5, 2.5
im_ = -1, 1
surf = pygame.Surface(size)
colour_gradient1 = [c, c1, c2, c3, ...] # This is some list of colours generated by a gradient function
for x in range(0, size_[0]):
for y in range(0, size_[1]):
z = complex(re_[0] + (x / size_[0]) * (re_[1] - re_[0]),
im_[0] + (y / size_[1]) * (im_[1] - im_[0]))
m = julia(z, c)
colour = colour_gradient1[m]
pygame.draw.rect(surf,
colour,
(x, y, 1, 1))
I think I understand why this is performance intensive in that both pygame and python aren't really optimised for rendering stuff to the screen like this. I'm currently trying to learn C++ and I understand its better for stuff like this.
I also experimented with a zoom function where I could select a box with the mouse and the script would render this selected area but implementing this was where the problem stuck out. As the zoomed in fractals got more complex, the script took too long to use this function.
So my question, is there a better way to render something like this in close to real-time using python and maybe pygame? I'm open to using a different package but if it's possible through pygame that would be ideal.
Attached below are a couple of pictures of the generated sets:
Fractal generating algorithms always slow down the further in you zoom because there needs to be ever more iterations per pixel the deeper you go (or before the bail-out is reached).
This is never going to be particularly fast in an interpreted language. Sure you can tweak it to increase the speed a little, but it will never be "real time" (say < 1 second / image) for all zoom levels.
If you want to continue in Python, you will have to just agree with yourself that it's never going to be fast.
However. However, you could split the generation of each quadrant into separate processes which would each run on their own CPU/core. That will give you a N/cores speed up.
There are some optimisations that can be performed with detecting symmetry in the image, and only calculating say half the pixels because the other side is a mirror of it (like the horizontal axis through an zoomed-out Mandelbrot set). You could probably refer to the source of the venerable Fractint Program for examples of this.
...
Aside: I wrote one of these (drawing a Mandelbrot set) in C using the nVidia CUDA library which spreads the calculation over the 1200-ish "CPU"s on the video card (using a mid-range 2018 laptop). While it worked quite fast, for sufficiently large images, or deeply "zoomed-in" fractals, it still became slow. There's just so much number crunching involved.
(This question finally made me install PyOpenGL. So thanks!)
As far as I've seen, iterating over each pixel individually,
will never give good performance (not in C++/C/Assembly/).
Vectorization (in the CPU) will help. What will really help,
is using the GPU's ability to apply one operation(/kernel),
to a whole multi-dimensional array of elements, in parallel.
Specifically: Using a fragment shader, for calculating the
color of each pixel. But that means using a graphics API
like OpenGL(/Vulkan/Direct3D/), or a GPGPU/Compute API like
OpenCL(/CUDA/).
If the resulting image is used within the graphics pipeline,
then it can stay on the GPU & be displayed directly from
there. If the resulting image needs to be used e.g. in a
GUI, saved to disk, or similar, it needs to be brought from
GPU to CPU (maybe render-to-texture, read the framebuffer,
use off-screen buffers, or other options that I don't know).
import numpy as np
from OpenGL.GL import *
from OpenGL.GL import shaders
from OpenGL.GLUT import *
# Vertex shader: Pass through (no model-view-projection).
vsSrc = '''
#version 300 es
layout (location = 0) in vec4 posIn;
void main()
{
gl_Position = posIn;
}
'''
# Fragment shader: Compute fractal color, per-pixel.
# en.wikipedia.org/wiki/Mandelbrot_set#Computer_drawings
fsSrc = '''
#version 300 es
precision mediump float;
out vec4 colorOut;
vec2 mapLinear(
vec2 val,
vec2 srcMin, vec2 srcMax,
vec2 dstMin, vec2 dstMax
) {
vec2 valNorm = (val - srcMin) / (srcMax - srcMin);
return valNorm * (dstMax - dstMin) + dstMin;
}
void main()
{
// Debugging: Return fixed color; see which pixels get it.
//colorOut = vec4(0.0, 0.5, 0.0, 1.0);
//return;
// Originally, origin is top-left. Convert to Cartesian.
vec2 pixelMin = vec2(0.0f, 720.0f);
vec2 pixelMax = vec2(1280.0f, 0.0f);
vec2 mbMin = vec2(-2.5f, -1.0f);
vec2 mbMax = vec2(1.0f, 1.0f);
vec2 mbExtent = mbMax - mbMin;
vec2 mbCenter = mbMin + (mbExtent / 2.0f);
vec2 fragMapped = mapLinear(
gl_FragCoord.xy, pixelMin, pixelMax, mbMin, mbMax
);
float real = 0.0f;
float imag = 0.0f;
int iter = 0;
const int maxIter = 500;
while (
((real*real + imag*imag) < 4.0f) &&
(iter < maxIter)
) {
float realTemp = real*real - imag*imag + fragMapped.x;
imag = 2.0f*real*imag + fragMapped.y;
real = realTemp;
++iter;
}
// Using generated colors, instead of indexing a palette.
// (Don't remember anymore where this came from,
// or if it was a heuristic.)
vec3 chosenColor;
float iterNorm = float(iter) / float(maxIter);
if (iterNorm > 0.5f) {
float iterNormInverse = 1.0f - iterNorm;
chosenColor = vec3(
0.0f, iterNormInverse, iterNormInverse
);
}
else {
chosenColor = vec3(0.0f, iterNorm, iterNorm);
}
colorOut = vec4(chosenColor.xyz, 1.0f);
}
'''
def compileFractalShader():
vs = shaders.compileShader(vsSrc, GL_VERTEX_SHADER)
fs = shaders.compileShader(fsSrc, GL_FRAGMENT_SHADER)
return shaders.compileProgram(vs, fs)
# Geometry: Just 2 triangles, covering the display surface.
# (So that the fragment shader runs for all surface pixels.)
def drawTriangles():
topLeftTriangle = (
1.0, 1.0, 0.0,
-1.0, -1.0, 0.0,
-1.0, 1.0, 0.0
)
bottomRightTriangle = (
1.0, 1.0, 0.0,
-1.0, -1.0, 0.0,
1.0, -1.0, 0.0
)
verts = np.array(
topLeftTriangle + bottomRightTriangle,
dtype=np.float32
)
glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 0, verts)
glEnableVertexAttribArray(0)
glDrawArrays(GL_TRIANGLES, 0, 6)
def printShaderException(e):
errorMsg, shaderSrc, shaderType = e.args
print('Shader error message:')
for line in errorMsg.split('\\n'): print(line)
print('--')
#print('Shader source:')
#for line in shaderSrc[0].split(b'\n'): print(line)
#print('--')
print('Shader type:', shaderType)
WIDTH = 1280
HEIGHT = 720
glutInit()
glutInitWindowSize(WIDTH, HEIGHT)
glutCreateWindow('Fractals with fragment shaders.')
# Create shaders, after creating a window / opengl-context:
try: fractalShader = compileFractalShader()
except RuntimeError as e:
printShaderException(e)
exit()
glViewport(0, 0, WIDTH, HEIGHT)
glClearColor(0.5, 0.0, 0.5, 1.0)
def display():
glClear(GL_COLOR_BUFFER_BIT)
with fractalShader: drawTriangles()
glutSwapBuffers()
glutDisplayFunc(display)
glutMainLoop()
This is entirely unoptimized.
Also, as Kingsley wrote, zooming (not shown here)
was slowing things down even in the GPU (but: unoptimized).
Related
What I want to do is to loop over an image pixel by pixel using each pixel value to draw a circle in another corresponding image.
My approach is as follows:
it = np.nditer(pixels, flags=['multi_index'])
while not it.finished:
y, x = it.multi_index
color = it[0]
it.iternext()
center = (x*20 + 10, y*20 + 10) # corresponding circle center
cv2.circle(circles, center, int(8 * color/255), 255, -1)
Looping this way is somewhat slow. I tried adding the #njit decorator of numba, but apparently it has problems with opencv.
Input images are 32x32 pixels
They map to output images that are 32x32 circles
each circle is drawn inside a 20x20 pixels square
That is, the output image is 640x640 pixels
A single image takes around 100ms to be transformed to circles, and I was hoping to lower that to 30ms or lower
Any recommendations?
When:
Dealing with drawings
The number of possible options does not exceed a common sense value (in this case: 256)
Speed is important (I guess that's always the case)
There's no other restriction preventing this approach
the best way would be to "cache" the drawings (draw them upfront (or on demand depending on the needed overhead) in another array), and when the drawing should normally take place, simply take the appropriate drawing from the cache and place it in the target area (as #ChristophRackwitz stated in one of the comments), which is a very fast NumPy operation (compared to drawing).
As a side note, this is a generic method not necessarily limited to drawings.
But the results you claim you're getting: ~100 ms per one 32x32 image (to a 640x640 circles one), didn't make any sense to me (as OpenCV is also fast, and 1024 circles shouldn't be such a big deal), so I created a program to convince myself.
code00.py:
#!/usr/bin/env python
import itertools as its
import sys
import time
import cv2
import numpy as np
def draw_img_orig(arr_in, arr_out):
factor = round(arr_out.shape[0] / arr_in.shape[0])
factor_2 = factor // 2
it = np.nditer(arr_in, flags=["multi_index"])
while not it.finished:
y, x = it.multi_index
color = it[0]
it.iternext()
center = (x * factor + factor_2, y * factor + factor_2) # corresponding circle center
cv2.circle(arr_out, center, int(8 * color / 255), 255, -1)
def draw_img_regular_iter(arr_in, arr_out):
factor = round(arr_out.shape[0] / arr_in.shape[0])
factor_2 = factor // 2
for row_idx, row in enumerate(arr_in):
for col_idx, col in enumerate(row):
cv2.circle(arr_out, (col_idx * factor + factor_2, row_idx * factor + factor_2), int(8 * col / 255), 255, -1)
def draw_img_cache(arr_in, arr_out, cache):
factor = round(arr_out.shape[0] / arr_in.shape[0])
it = np.nditer(arr_in, flags=["multi_index"])
while not it.finished:
y, x = it.multi_index
yf = y * factor
xf = x *factor
arr_out[yf: yf + factor, xf: xf + factor] = cache[it[0]]
it.iternext()
def generate_input_images(shape, count, dtype=np.uint8):
return np.random.randint(256, size=(count,) + shape, dtype=dtype)
def generate_circles(shape, dtype=np.uint8, count=256, rad_func=lambda arg: int(8 * arg / 255), color=255):
ret = np.zeros((count,) + shape, dtype=dtype)
cy = shape[0] // 2
cx = shape[1] // 2
for idx, arr in enumerate(ret):
cv2.circle(arr, (cx, cy), rad_func(idx), color, -1)
return ret
def test_draw(imgs_in, img_out, count, draw_func, *draw_func_args):
print("\nTesting {:s}".format(draw_func.__name__))
start = time.time()
for i, e in enumerate(its.cycle(range(imgs_in.shape[0]))):
draw_func(imgs_in[e], img_out, *draw_func_args)
if i >= count:
break
print("Took {:.3f} seconds ({:d} images)".format(time.time() - start, count))
def test_speed(shape_in, shape_out, dtype=np.uint8):
imgs_in = generate_input_images(shape_in, 50, dtype=dtype)
#print(imgs_in.shape, imgs_in)
img_out = np.zeros(shape_out, dtype=dtype)
circles = generate_circles((shape_out[0] // shape_in[0], shape_out[1] // shape_in[1]))
count = 250
funcs_data = (
(draw_img_orig,),
(draw_img_regular_iter,),
(draw_img_cache, circles),
)
for func_data in funcs_data:
test_draw(imgs_in, img_out, count, func_data[0], *func_data[1:])
def test_accuracy(shape_in, shape_out, dtype=np.uint8):
img_in = np.arange(np.product(shape_in), dtype=dtype).reshape(shape_in)
circles = generate_circles((shape_out[0] // shape_in[0], shape_out[1] // shape_in[1]))
funcs_data = (
(draw_img_orig, "orig.png"),
(draw_img_regular_iter, "regit.png"),
(draw_img_cache, "cache.png", circles),
)
imgs_out = [np.zeros(shape_out, dtype=dtype) for _ in funcs_data]
for idx, func_data in enumerate(funcs_data):
func_data[0](img_in, imgs_out[idx], *func_data[2:])
cv2.imwrite(func_data[1], imgs_out[idx])
for idx, img in enumerate(imgs_out[1:], start=1):
if not np.array_equal(img, imgs_out[0]):
print("Image index different: {:d}".format(idx))
def main(*argv):
dt = np.uint8
shape_in = (32, 32)
factor_io = 20
shape_out = tuple(i * factor_io for i in shape_in)
test_speed(shape_in, shape_out, dtype=dt)
test_accuracy(shape_in, shape_out, dtype=dt)
if __name__ == "__main__":
print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
64 if sys.maxsize > 0x100000000 else 32, sys.platform))
rc = main(*sys.argv[1:])
print("\nDone.\n")
sys.exit(rc)
Notes:
Besides your implementation that uses np.nditer (which I placed in a function called draw_img_orig), I created 2 more:
One that iterates the input array Pythonicly (draw_img_regular_iter)
One that uses cached circles, and also iterates via np.nditer (draw_img_cache)
In terms of tests, there are 2 of them - each being performed on every of the 3 (above) approaches:
Speed: measure the time took to process a number of images
Accuracy: measure the output for a 32x32 input containing the interval [0, 255] (4 times)
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q071818080]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> dir /b
code00.py
[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.09_test0\Scripts\python.exe" code00.py
Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 064bit on win32
Testing draw_img_orig
Took 0.908 seconds (250 images)
Testing draw_img_regular_iter
Took 1.061 seconds (250 images)
Testing draw_img_cache
Took 0.426 seconds (250 images)
Done.
[prompt]>
[prompt]> dir /b
cache.png
code00.py
orig.png
regit.png
Above there are the speed test results: as seen, your approach took a bit less than a second for 250 images!!! So I was right, I don't know where your slowness comes from, but it's not from here (maybe you got the measurements wrong?).
The regular method is a bit slower, while the cached one is ~2X faster. I ran the code on my laptop:
Win 10 pc064
CPU: Intel i7 6820HQ # 2.70GHz (fairly old)
GPU: not relevant, as I didn't notice any spikes during execution
Regarding the accuracy test, all (3) output arrays are identical (there's no message saying otherwise), here's one saved image:
I'm fairly new to python programming and I'm struggling with the Vispy Library.
Basically, I have a Raspberry pi connected to 2 Arduinos accelerometers sensors. The raspberry is sending the X, Y and Z values from both of the sensors through UDP to my computer. Then my computer has to displays 9 graphs : 6 for the evolutions of x, y and z for both sensors and 3 for the differences between them (X1-X2, Y1-Y2 and Z1-Z2) and finally, it must be in real-time.
I wanted to use the Vispy library for that last point. After reading the documentation, I came up with the following code :
#!/usr/bin/env python3
import numpy as np
from vispy import app
from vispy import gloo
import socket
from itertools import count
# init x, y arrays
x1_vals = []
time_vals = []
#UDP connection from Raspberry pi
UDP_IP = ""
UDP_PORT = 5005
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
# Initialize the index and set it to 1
index = count()
next(index)
# Initialize the Canvas c
c = app.Canvas(keys='interactive')
vertex = """
attribute vec2 a_position;
void main (void)
{
gl_Position = vec4(a_position, 0.0, 1.0);
}
"""
fragment = """
void main()
{
gl_FragColor = vec4(0.0, 0.0, 15.0, 10.0);
}
"""
program = gloo.Program(vertex, fragment)
#c.connect
def on_resize(event):
gloo.set_viewport(0, 0, *event.size)
#c.connect
def on_draw(event):
gloo.clear((1,1,1,1))
program.draw('line_strip')
def on_timer(event):
# next index
cpt = next(index)
# Get data from UDP
recv, addr = sock.recvfrom(1024)
data = recv.decode('UTF-8').split(';')
# We want to display only 100 samples so the graph still readable.
# So we delete the first value of the x array if there are more than 100 values
if (cpt > 100):
del x1_vals[0]
time_vals = np.linspace(-1.0, +1.0, 100)
else:
time_vals = np.linspace(-1.0, +1.0, cpt)
# The values must be bound between -1.0 and 1.0
tmp = float(data[0])*0.5
if (tmp >= 1):
tmp = float(0.99)
elif (tmp <= -1):
tmp = float(-0.99)
x1_vals.append(tmp)
# Then we concatenate the arrays of x and y
program['a_position'] = np.c_[time_vals, x1_vals].astype(np.float32)
c.update()
c.timer = app.Timer('auto', connect=on_timer, start=True)
c.show()
app.run()
So as the comments describe it, it firstly intitializes the UDP connection and the canvas, then for each values received it updates the canvas with the newly added value. If the number of values exceed 100, the first value of the array is deleted to keep a constant number of samples.
It works well when I want to display only the X1 accelerometer sensors evolution. So now I picked the code from the Vispy documentation which demonstrates how to show multiple graphs, but the code is a bit too complex for my level.
Basically, in my code I receive all the sensors values in the data array. I pick the first value [0] (X1), but the complete data looks like this : [x1, y1, z1, dx, dy, dz, x2, y2, z2] where dx = x1 - x2, dy = y1 - y2 and dz = z1 - z2. (the difference has to be directly calculated on the raspberry).
So I tried to modify the code from the documentation as following :
# Number of cols and rows in the table.
nrows = 3
ncols = 3
# Number of signals.
m = nrows*ncols
# Number of samples per signal.
n = 100
Because I want 9 graphs and only 100 samples per graph.
I ignored the index, the color and deleted the amplitude has it is not required in my case. Basically, I almost kept the original code for the whole setting part, then I replaced the def on_timer with mine.
Now I'm trying to feed the a_position array from GLSL with my own data. But I'm not sure how to prepare the data to make it works properly with this code. I'm struggling to understand what does these lines do :
# GLSL C code
VERT_SHADER = """
// Compute the x coordinate from the time index.
float x = -1 + 2*a_index.z / (u_n-1);
vec2 position = vec2(x - (1 - 1 / u_scale.x), a_position);
// Find the affine transformation for the subplots.
vec2 a = vec2(1./ncols, 1./nrows)*.9;
vec2 b = vec2(-1 + 2*(a_index.x+.5) / ncols,
-1 + 2*(a_index.y+.5) / nrows);
// Apply the static subplot transformation + scaling.
gl_Position = vec4(a*u_scale*position+b, 0.0, 1.0);
"""
# Python code
def __init__(self):
self.program['a_position'] = y.reshape(-1, 1)
def on_timer(self, event):
k = 10
y[:, :-k] = y[:, k:]
y[:, -k:] = amplitudes * np.random.randn(m, k)
self.program['a_position'].set_data(y.ravel().astype(np.float32))
I deleted the surrounding code that I think I'm understanding.
Note that even if I'm starting with python, I'm aware that they are using a class definition for the Canvas when I'm using the bare object in my code. I understand the use of self and others.
How can I adapt the code from the realtime_signals documentation to my case ?
Disclaimer: Overall that realtime signals example is, in my opinion, a bit of a hack. It "cheats" to produce as many plots as it does, but in the end the result is fast.
What that bit of shader code is doing is trying to take the series of line vertices and figure out which "sub-plot" they should go in. All vertices of all the lines are going into the shader as one array. The shader code is trying to say "this vertex is 23rd in the array which means it must belong to sub-plot 5 and it is the 3rd point in that plot because we know we have 5 points per plot" (as an example). The shader does this mostly by the information in a_index. For example, this bit:
// Compute the x coordinate from the time index.
float x = -1 + 2*a_index.z / (u_n-1);
vec2 position = vec2(x - (1 - 1 / u_scale.x), a_position);
Is adjusting the x coordinate (a_position) based on which sub-plot the point falls in.
The next chunk:
// Find the affine transformation for the subplots.
vec2 a = vec2(1./ncols, 1./nrows)*.9;
vec2 b = vec2(-1 + 2*(a_index.x+.5) / ncols,
-1 + 2*(a_index.y+.5) / nrows);
// Apply the static subplot transformation + scaling.
gl_Position = vec4(a*u_scale*position+b, 0.0, 1.0);
Is trying to determine how big each subplot should be. So the first chunk was "what subplot does this point fall in" and this one is "where in that subplot does the point sit". This code it coming up with a linear affine transformation (y = m*x + b) to scale the line to the appropriate size so that all the subplots are the same size and don't overlap.
I'm not sure I can go into more detail without re-walking the whole script and trying to understand exactly what each value in a_index is.
Edit: Another suggestion, in the long run you may want to move the UDP recv code to a separate thread (QThread if using a Qt backend) that emits a signal with the new data when it is available. This way the GUI/main thread stays responsive and isn't hung up waiting for data to come in.
I need to rotate an image around its x-axis (or y-axis). I can easily create such an animation with avisynth, but now I need to implement that effect with Python's moviepy module. I can easily rotate an image with the following script but need some clues how to rotate it in 2D or 3D.
from moviepy.editor import *
clip = ImageClip('my_image.jpg')
rotated_clip = (clip.add_mask()
.fx(vfx.resize, width=300, height=300)
.fx(vfx.rotate, lambda t: 90*t, expand=False)
.set_duration(5))
final_clip = CompositeVideoClip([rotated_clip.set_pos("center")], size=(800,800), bg_color=3*[255])
final_clip.write_videofile("test.mp4", fps=25, codec="libx264")
Here is the avisynth script that actually generated that example image. Please note, it does require "QUAD" plugin.
function stars(clip c, int r) {
c.Overlay(x=rand(c.width),y=rand(c.height),BlankClip(c,width=1,height=1,color=$030301*rand(85)))
(r==0)? last : stars(r-1)
Trim(0,-1).Loop(c.Framecount, 0, 0)
}
width= 800
height=600
length=100000
Tcolor=$000040
Bcolor=$000018
StackVertical(BlankClip(length=length,width=2,height=1,color=TColor,pixel_type="RGB32"),BlankClip(length=length,width=2,height=1,color=BColor)).BilinearResize(width,2*height,src_top=0,src_height=2).Crop(0,height/2,0,-height/2).Stars(width*height/3072)
ImageSource("path_to_image.png", start=0, end=total_time, fps=300, pixel_type="RGB32")
#BlankClip(length=length,FPS=25,width=640,height=480,color=$000018,pixel_type="RGB32")
#ColorBars()
HALFCYCLE=10 # Frames in 1 HALF rotation (spinning clip)
NSPIN = 1 # Number of HALF rotations in spinning clip
NSTILL = 10 # Frames in STILL clip
V = 0.2 # Tilt/Yaw
tim = PI / HALFCYCLE
ScriptClip("""
c=last
t=tim*current_frame
t1x= 0.5 - 0.5 * cos(t) # BOTH Left
t2x= 0.5 + 0.5 * cos(t) # BOTH Right
#
t1y= 0.0 + V * sin(t) # ] both Top's opposite sign
t2y= 0.0 - V * sin(t) # ]
t3y= 1.0 + V * sin(t) # [ both Bottoms opposite sign
t4y= 1.0 - V * sin(t) # [
ResetMask
quad(t1x,t1y, t2x,t2y, t2x,t3y, t1x,t4y, normal=true)
#Overlay(c,last,mask=last.ShowAlpha())
""")
SPIN=Trim(0,-(NSPIN*HALFCYCLE +1)) # Spinning clip, + 1 to complete last spin
STILL=SPIN.Trim(SPIN.FrameCount-1,-1).Loop(NSTILL,0,0)
SPIN2=Trim((NSPIN%2 ==0)?0:HALFCYCLE,-(NSPIN*HALFCYCLE +1))
SPIN ++ STILL ++ SPIN2
Return Last
One way to do this is to use Vapory, another library by the author of MoviePy, which facilitates the operation of POV-Ray via Python. You can create a rectangle within a 3D scene and rotate it around whatever axis you like, saving frames into a MoviePy clip at intervals.
MoviePy + Vapory code
from moviepy.editor import concatenate, ImageClip, VideoClip
from vapory import *
img_path = './baseball.png'
img_clip = ImageClip(img_path)
W, H = img_clip.w, img_clip.h
AR = 1.0*W/H
# Set rotation rate by defining the period (in seconds) for 360 deg. revolution
t_rev = 2.0
t_half = t_rev/2.0 # The time required for a half revolution
t_still = 0.8 # How long (in seconds) to hold the half rotated image still
# Static POV-Ray objects
cam = Camera('location', [ 0, 0, -1],
'look_at', [ 0, 0, 0])
light = LightSource([0, 0, -1]) # Light at camera location
bg = Background('color', [0, 0, 0]) # Black background
def scene(t):
""" Returns the scene at time 't' (in seconds) """
s = Scene(camera = cam, objects = [light, bg])
# Add POV-Ray box with image textured on it
s = s.add_objects([
Box([0, 0, 0],
[W, H, 0],
Texture(Pigment(ImageMap('"{}"'.format(img_path), 'once')),
Finish('ambient', 1.0)),
'translate', [-0.5, -0.5, 0],
'scale', [AR, 1, 0],
'rotate', [0, (360/t_rev)*t, 0])]) # Can change axis of rotation here
return s
def make_frame(t):
return scene(t).render(width=W, height=H, antialiasing=0.1)
still_1 = VideoClip(make_frame).to_ImageClip(t=0).set_duration(t_still)
half_1 = VideoClip(make_frame).subclip(0, t_half)
still_2 = VideoClip(make_frame).to_ImageClip(t=t_half).set_duration(t_still)
half_2 = VideoClip(make_frame).subclip(t_half, t_rev)
final_clip = concatenate([still_1, half_1, still_2, half_2])
final_clip.write_gif("./baseball_rot.gif", fps=15)
Output GIF
Other thoughts:
The main things you might want to change are img_path, t_rev (the time for a full 360 degree revolution), t_still, and the output frame rate.
I removed one column of pixels from your example image to get it down to an even width (150 px). It's not important if you only want to make GIFs, but if you want to produce an x264-encoded MP4, you should probably use mod2 dimensions.
It seems like overkill to use a ray tracer for this problem, but it's the first working solution I came up with. I wanted to represent the image as a 2D rectangle in a 3D scene, where I could simply specify an angle of rotation and the 3D library would handle the rest.
It should be possible to solve this problem using a projective transform from scikit-image, as in this MoviePy example. Note specifically, the trapzWarp function near the middle of that code listing.
Using opencv for python I need to fit an ellipse (using cv2.fitEllipse) to the array of points returned by cv.FindCornerSubPix (here named 'features'). I have seen numerous examples of this on the internet, but I cannot figure it out.
I figured cv.FindCornerSubPix returns an array of tuples, and my code triggered an error asking me for a numpy array as argument for cv2.fitEllipse, so I tried to convert 'features' to a numpy array and the error is now:
'error: ......\src\opencv\modules\imgproc\src\contours.cpp:2019: error: (-215) points.checkVector(2) >= 0 && (points.depth() == CV_32F || points.depth() == CV_32S)'
on line 196 ('cv2.fitEllipse(ellipse)' at the end of my code), so I guess I am not feeding the right array format to cv2.fitEllipse. Would you please help me? The code below is just a modified version of the opencv sample lkdemo.py.
# search the good points
features = cv.GoodFeaturesToTrack (
grey, eig, temp,
MAX_COUNT,
quality, min_distance, mask, 10, 0, 0.04)
# refine the corner locations
features = cv.FindCornerSubPix (
grey,
features,
(win_size, win_size), (-1, -1),
(cv.CV_TERMCRIT_ITER | cv.CV_TERMCRIT_EPS, 20, 0.03))
elif features != []:
# we have points, so display them
# calculate the optical flow
features, status, track_error = cv.CalcOpticalFlowPyrLK (
prev_grey, grey, prev_pyramid, pyramid,
features,
(win_size, win_size), 3,
(cv.CV_TERMCRIT_ITER|cv.CV_TERMCRIT_EPS, 20, 0.03),
flags)
# set back the points we keep
features = [ p for (st,p) in zip(status, features) if st]
if add_remove_pt:
# we have a point to add, so see if it is close to
# another one. If yes, don't use it
def ptptdist(p0, p1):
dx = p0[0] - p1[0]
dy = p0[1] - p1[1]
return dx**2 + dy**2
if min([ ptptdist(pt, p) for p in features ]) < 25:
# too close
add_remove_pt = 0
# draw the points as green circles
for the_point in features:
cv.Circle (image, (int(the_point[0]), int(the_point[1])), 3, (0, 255, 0, 0), -1, 8, 0)
#Fit an ellipse
array = np.array([tuple(i) for i in features])
ellipse = np.asarray(array)
cv2.fitEllipse(ellipse)
This question is solved. Please look in the comments section. By the way, Stackoverflow asks for a delay of several hours for a newcomer to answer his own questions, which is why I put the answer in the comments.
Cheers
I have a set of points (3D) taken from a range scanner. Sample data can be found here: http://pastebin.com/RBfQLm56
I also have the following parameters for the scanner:
camera matrix
[3871.88184, 0, 950.736938;
0, 3871.88184, 976.1383059999999;
0, 0, 1]
distortion coeffs
[0.020208003; -1.41251862; -0.00355229038; -0.00438868301; 6.55825615]
camera to reference point (transform)
[0.0225656671, 0.0194614234, 0.9995559233, 1.2656986283;
-0.9994773883, -0.0227084301, 0.0230060289, 0.5798922567;
0.0231460759, -0.99955269, 0.0189388219, -0.2110195758;
0, 0, 0, 1]
I am trying to render these points properly using opengl but the rendering does not look right. What is the correct way to set openGL projection and modelview matrix? This is what I currently do -
znear = 0.00001
zfar = 100
K = array([[3871.88184, 0, 950.736938],[0, 3871.88184, 976.1383059999999],[0, 0, 1]])
Rt =array([[0.0225656671, 0.0194614234, 0.9995559233, 1.2656986283],[-0.9994773883, -0.0227084301, 0.0230060289, 0.5798922567],[0.0231460759, -0.99955269, 0.0189388219, -0.2110195758]])
ren.set_projection(K,zfar,znear)
ren.set_projection_from_camera(Rt)
The function being used are:
def set_projection(self,K,zfar,znear):
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
f_x = K[0,0]
f_y = K[1,1]
c_x = K[0,2]
c_y = K[1,2]
fovY = 1/(float(f_x)/height * 2);
aspectRatio = (float(width)/height) * (float(f_y)/f_x);
near = zfar
far = znear
frustum_height = near * fovY;
frustum_width = frustum_height * aspectRatio;
offset_x = (width/2 - c_x)/width * frustum_width * 2;
offset_y = (height/2 - c_y)/height * frustum_height * 2;
glFrustum(-frustum_width - offset_x, frustum_width - offset_x, -frustum_height - offset_y, frustum_height - offset_y, near, far);
def set_modelview_from_camera(self,Rt):
glMatrixMode(GL_MODELVIEW)
glLoadIdentity()
Rx = array([[1,0,0],[0,0,-1],[0,1,0]])
R = Rt[:,:3]
U,S,V = linalg.svd(R)
R = dot(U,V)
R[0,:]=-R[0,:]
t=Rt[:,3]
M=eye(4)
M[:3,:3]=dot(R,Rx)
M[:3,3]=t
M=M.T
m=M.flatten()
glLoadMatrixf(m)
Then I just render points (pasting snippet):
def renderLIDAR(self,filename):
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glPushMatrix();
glEnable(GL_DEPTH_TEST)
glClear(GL_DEPTH_BUFFER_BIT)
glPointSize(1.0)
f = open(filename,'r')
f.readline() #Contains number of particles
for line in f:
line = line.split(' ')
glBegin(GL_POINTS)
glColor3f (0.0,1.0,0.0);
x = float(line[0])
y = float(line[1])
z = float(line[2])
glVertex3f(x,y,z)
#print x,y,z
glEnd()
glPopMatrix();
The matrices you get back, most notably the last one in your question are what in OpenGL is the composition of projection and modelview, also called Modelviewprojection, i.e.
MVP = P · M
As long as you're not interested in performing illumination calculations, you can use just that in a vertex shader, i.e.
#version 330
uniform mat4 MVP;
in vec3 position;
void main()
{
gl_Position = MVP * vec4(position, 1);
}
BTW, OpenGL and probably the library you're using as well, are using column major order, i.e. the order of the elements in memory is
0 4 8 c
1 5 9 d
2 6 a e
3 7 b f
so what's written in source code must be thought as "transposed" (of course it is not). Since the matrix you wrote follows the same scheme you can just put it into the uniform as it is. The only question that remains are the boundaries of the NDC space used by the range scanner. But that could be taken care of with an additional matrix applied. OpenGL uses the range [-1, 1]^3 so the worst thing that can happen is, that if it's in the other popular NDC range [0, 1]^3, you'll see your geometry just squeezed into the upper left hand corner of your window, and maybe turned "inside out" if the Z axis goes into the other direction. Just try it, I'd say it already matches OpenGL.
Anyway, if you want to use it with illumination, you have to decompose it into a projection and a modelview part. Easier said than done, but a good starting point is to orthonormalize the upper left 3×3 submatrix, which yields the rotational part of the modelview 'M'. You then have to find a matrix P, that, when left multiplied with M yields the original matrix. That's an overdetermined set of linear equations, so a Gauss-Jordan scheme can do it. And if I'm not entirely mistaken, what you already got in form of that camera matrix is either the decomposed M or P (I'd go for M).
Once you got that you may want to get the translational part (the 4th column) into the modelview matrix as well.