I'm using pi3d to display an ImageSprite on the screen the texture of which comes form an image I'm loading.
displayTexture = pi3d.Texture("display/display.jpg", blend=True, mipmap=True)
displaySlide = pi3d.ImageSprite(texture=displayTexture, shader=shader, w=800, h=600)
This texture image is actually something I'm creating in-program. It's an openCV2 image and therefore just a numpy array. At the moment I'm saving it just to load it again as a texture, but is there a way to just constantly update the texture of the sprite with the changing numpy array values?
I looked into the openCV OpenGL support but from what I could see it only supports Windows at this stage and is therefore not suitable for this use.
Edit: Should have mentioned I'm happy for a lower level solution too. I'm currently trying to use .toString() on the image array and use the resulting byte list with glTexImage2D to produce a texture but no dice so far.
Yes you can pass a PIL.Image to pi3d.Texture and it will create a new Texture using that. There is a bit of work involved there so it will impact on frame rate if it's a big Texture. Also you need to update the pointer in the Buffer that holds the Texture array so the new Texture gets used.
There is a method to load a numpy array to a PIL.Image (Image.fromarray()) so this would be an easy route. However it's a bit convoluted as pi3d already converts the PIL.Image into a numpy array see https://github.com/tipam/pi3d/blob/master/pi3d/Texture.py#L163
The following works ok as a short-cut into the workings of pi3d.Texture but it's a bit of a hack calling the 'private' function _load_opengl. I might look at making a more robust method of doing this (i.e. for mapping videos to 3D objects etc)
#!/usr/bin/python
from __future__ import absolute_import, division, print_function, unicode_literals
import demo
import pi3d
import random
import numpy as np
from PIL import Image, ImageDraw
DISPLAY = pi3d.Display.create(x=150, y=150)
shader = pi3d.Shader("uv_flat")
im = Image.open("textures/PATRN.PNG")
#draw = ImageDraw.Draw(im) # there are various PIL libraries you could use
nparr = np.array(im)
tex = pi3d.Texture(im) # can pass PIL.Image rather than path as string
sprite = pi3d.ImageSprite(tex, shader, w=10.0, h=10.0)
mykeys = pi3d.Keyboard()
while DISPLAY.loop_running():
#draw.line((random.randint(0,im.size[0]),
# random.randint(0,im.size[1]),
# random.randint(0,im.size[0]),
# random.randint(0,im.size[1])), fill=128) # draw random lines
#nparr = np.array(im)
nparr += np.random.randint(-2, 2, nparr.shape) # random noise
tex.image = nparr
tex._load_opengl()
sprite.draw()
if mykeys.read() == 27:
mykeys.close()
DISPLAY.destroy()
break
PS I can't remember what version of pi3d the switch to numpy textures happened but it's quite recent so you probably have to upgrade
EDIT:
The switch from Texture.image being a bytes object to numpy array was v1.14 posted on 18Mar15
To clarify the steps to use a numpy array to initialise and refresh a changing image:
...
im = Image.fromarray(cv2im) # cv2im is a numpy array
tex = pi3d.Texture(im) # create Texture from PIL image
sprite = pi3d.ImageSprite(tex, shader, w=10.0, h=10.0)
...
tex.image = cv2im # set Texture.image to modified numpy array
tex._load_opengl() # re-run OpenGLESv2 routines
Related
I get that in Python OpenCV images are numpy arrays, that correspond to cv::Mat in c++.
This question is about what type-hint to put into python functions to properly restrict for OpenCV images (maybe even for a specific kind of OpenCV image).
What I do now is:
import numpy as np
import cv2
Mat = np.ndarray
def my_fun(image: Mat):
cv2.imshow('display', image)
cv2.waitKey()
Is there any better way to add typing information for OpenCV images in python?
You can specify it as numpy.typing.NDArray with an entry type. For example,
import numpy as np
Mat = np.typing.NDArray[np.uint8]
def my_fun(img: Mat):
pass
What's the size
import cv2
img = cv2.imread('./imgs/img.png', cv2.IMREAD_COLOR)
print(img.shape[0], img.shape[1]) # 182 277
print(img.size) # 151242
I thought img.size = img.width * img.width, but obviously wrong, check this example.
so, what's the definition of it?
It's all about what language you're working in.
The Python interface to OpenCV uses numpy, and numpy arrays have different methods from cv::Mat (C++). size means different things then.
Python: OpenCV uses numpy arrays. .size tells you the number of elements in the array. That is the product of all values in .shape.
C++: OpenCV uses its own cv::Mat. .size there is of type MatSize. It is roughly equivalent to .shape with numpy. Documentation indicates that it can also tell you the number of channels. It has an operator[] defined.
I have implemented pyqtgraph inside QGraphicsView in PyQt5. When I display the image the following way, it is stretched out and expands in the same aspect ratio as the screen. How do I fix this?
image = pg.ImageItem(asarray(Image.open('pic.png')) )
self.graphicsView.addItem(image)
image.rotate(270)
EDIT: found out how to rotate image, so I updated question with the solution. Now I am just trying to scale it properly.
You probably want something like:
import pyqtgraph as pg
from PIL import Image
from numpy import asarray
app = pg.mkQApp()
# Set up a window with ViewBox inside
gv = pg.GraphicsView()
vb = pg.ViewBox()
gv.setCentralItem(vb)
gv.show()
# configure view for images
vb.setAspectLocked()
vb.invertY()
# display image
img_data = asarray(Image.open('/home/luke/tmp/graph.png'))
image = pg.ImageItem(img_data, axisOrder='row-major')
vb.addItem(image)
The important pieces here that set the image scaling/orientation are:
using ImageItem(axisOrder='row-major') because image files are stored in row-major order
vb.invertY() because image files have the +y axis pointing downward
and vb.setAspectLocked() to keep the pixels square
I used np.rot90() instead, it's much faster and cythonable
image = pg.ImageItem(np.rot90(np.asarray(Image.open('pic.png'))))
I'm exploring SimpleCV as an imaging library in Python, and it seems pretty good. However, I'm stumped at how to perform an FFT on an image within SimpleCV. It seems I'd have to convert to an numpy array first, and then use the numpy facilities:
import SimpleCV as SV
im = Image('image.png')
img = im.getGrayNumpy()
imf = np.fft.fftshift(np.fft.fft2(img))
plt.imshow(log(abs(imf)+1),cmap=cm.gray)
Or maybe this is the best way? And of course if I want to convert the log of the fft spectrum into a SimpleCV image for later use, that's another issue...
You can bring numpy matrices back to SimpleCV using the Image() constructor:
import scipy
import numpy as np
import SimpleCV as scv
cam = scv.Camera()
disp = scv.Display()
while disp.isNotDone():
current = cam.getImage().resize(w=768)
matrix = current.getGrayNumpy()
spectrum = np.abs(np.log(np.fft.fftshift(np.fft.fft2(matrix))))
spectrum *= 255 / spectrum.max()
scv.Image(spectrum).show()
if disp.mouseLeft:
break
So, I'm generating a binary (well, really gray scale, 8bit, used as binary) image with python and opencv2, writing a small number of polygons to the image, and then dilating the image using a kernel. However, my source and destination image always end up the same, no matter what kernel I use. Any thoughts?
from matplotlib import pyplot
import numpy as np
import cv2
binary_image = np.zeros(image.shape,dtype='int8')
for rect in list_of_rectangles:
cv2.fillConvexPoly(binary_image, np.array(rect), 255)
kernel = np.ones((11,11),'int')
dilated = cv2.dilate(binary_image,kernel)
if np.array_equal(dilated, binary_image):
print("EPIC FAIL!!")
else:
print("eureka!!")
All I get is EPIC FAIL!
Thanks!
So, it turns out the problem was in the creation of both the kernel and the image. I believe that openCV expects 'uint8' as a data type for both the kernel and the image. In this particular case, I created the kernel with dtype='int', which defaults to 'int64'. Additionally, I created the image as 'int8', not 'uint8'. Somehow this did not trigger an exception, but caused the dilation to fail in a surprising fashion.
Changing the above two lines to
binary_image = np.zeros(image.shape,dtype='uint8')
kernel = np.ones((11,11),'uint8')
Fixed the problem, and now I get EUREKA! Hooray!