Is there a more idiomatic way to display a grid of images as in the below example?
import numpy as np
def gallery(array, ncols=3):
nrows = np.math.ceil(len(array)/float(ncols))
cell_w = array.shape[2]
cell_h = array.shape[1]
channels = array.shape[3]
result = np.zeros((cell_h*nrows, cell_w*ncols, channels), dtype=array.dtype)
for i in range(0, nrows):
for j in range(0, ncols):
result[i*cell_h:(i+1)*cell_h, j*cell_w:(j+1)*cell_w, :] = array[i*ncols+j]
return result
I tried using hstack and reshape etc, but could not get the right behaviour.
I am interested in using numpy to do this because there is a limit to how many images you can plot with matplotlib calls to subplot and imshow.
If you need sample data to test you can use your webcam like so:
import cv2
import matplotlib.pyplot as plt
_, img = cv2.VideoCapture(0).read()
plt.imshow(gallery(np.array([img]*6)))
import numpy as np
import matplotlib.pyplot as plt
def gallery(array, ncols=3):
nindex, height, width, intensity = array.shape
nrows = nindex//ncols
assert nindex == nrows*ncols
# want result.shape = (height*nrows, width*ncols, intensity)
result = (array.reshape(nrows, ncols, height, width, intensity)
.swapaxes(1,2)
.reshape(height*nrows, width*ncols, intensity))
return result
def make_array():
from PIL import Image
return np.array([np.asarray(Image.open('face.png').convert('RGB'))]*12)
array = make_array()
result = gallery(array)
plt.imshow(result)
plt.show()
yields
We have an array of shape (nrows*ncols, height, weight, intensity).
We want an array of shape (height*nrows, width*ncols, intensity).
So the idea here is to first use reshape to split apart the first axis into two axes, one of length nrows and one of length ncols:
array.reshape(nrows, ncols, height, width, intensity)
This allows us to use swapaxes(1,2) to reorder the axes so that the shape becomes
(nrows, height, ncols, weight, intensity). Notice that this places nrows next to height and ncols next to width.
Since reshape does not change the raveled order of the data, reshape(height*nrows, width*ncols, intensity) now produces the desired array.
This is (in spirit) the same as the idea used in the unblockshaped function.
Another way is to use view_as_blocks . Then you avoid to swap axes by hand :
from skimage.util import view_as_blocks
import numpy as np
def refactor(im_in,ncols=3):
n,h,w,c = im_in.shape
dn = (-n)%ncols # trailing images
im_out = (np.empty((n+dn)*h*w*c,im_in.dtype)
.reshape(-1,w*ncols,c))
view=view_as_blocks(im_out,(h,w,c))
for k,im in enumerate( list(im_in) + dn*[0] ):
view[k//ncols,k%ncols,0] = im
return im_out
This answer is based off #unutbu's, but this deals with HWC ordered tensors. Furthermore, it shows black tiles for any channels that do not factorize evenly into the given rows/columns.
def tile(arr, nrows, ncols):
"""
Args:
arr: HWC format array
nrows: number of tiled rows
ncols: number of tiled columns
"""
h, w, c = arr.shape
out_height = nrows * h
out_width = ncols * w
chw = np.moveaxis(arr, (0, 1, 2), (1, 2, 0))
if c < nrows * ncols:
chw = chw.reshape(-1).copy()
chw.resize(nrows * ncols * h * w)
return (chw
.reshape(nrows, ncols, h, w)
.swapaxes(1, 2)
.reshape(out_height, out_width))
Here's a corresponding detiling function for the reverse direction:
def detile(arr, nrows, ncols, c, h, w):
"""
Args:
arr: tiled array
nrows: number of tiled rows
ncols: number of tiled columns
c: channels (number of tiles to keep)
h: height of tile
w: width of tile
"""
chw = (arr
.reshape(nrows, h, ncols, w)
.swapaxes(1, 2)
.reshape(-1)[:c*h*w]
.reshape(c, h, w))
return np.moveaxis(chw, (0, 1, 2), (2, 0, 1)).reshape(h, w, c)
Related
I am trying to feed a very large image into Triton server. I need to divide the input image into patches and feed the patches one by one into a tensorflow model. The image has a variable size, so the number of patches N is variable for each call.
I think a Triton ensemble model that calls the following steps would do the job:
A python model (pre-process) to create the patches
The segmentation model
Finally another python model (post-process) to merge the output patches into a big output mask
However, for this, I would have to write a config. pbtxt file with 1:N and N:1 relation, meaning the ensemble scheduler needs to call the 2nd step multiple times and the 3rd once with the aggregated output.
Is this possible, or do I need to use some other technique?
Disclaimer
The below answer isn't the actual solution to the above question. I misunderstood the above query. But I'm leaving this response in case of future readers find it useful.
Input
import cv2
import matplotlib.pyplot as plt
input_img = cv2.imread('/content/2.jpeg')
print(input_img.shape) # (719, 640, 3)
plt.imshow(input_img)
Slice and Stitch
The following functionality is adopted from here. More details and discussion can be found here.. Apart from the original code, we bring together the necessary functionality and put them in a single class (ImageSliceRejoin).
# ref: https://github.com/idealo/image-super-resolution
class ImageSliceRejoin:
def pad_patch(self, image_patch, padding_size, channel_last=True):
""" Pads image_patch with padding_size edge values. """
if channel_last:
return np.pad(
image_patch,
((padding_size, padding_size),
(padding_size, padding_size), (0, 0)),
'edge',
)
else:
return np.pad(
image_patch,
((0, 0), (padding_size, padding_size), (padding_size, padding_size)),
'edge',
)
# function to split the image into patches
def split_image_into_overlapping_patches(self, image_array, patch_size, padding_size=2):
""" Splits the image into partially overlapping patches.
The patches overlap by padding_size pixels.
Pads the image twice:
- first to have a size multiple of the patch size,
- then to have equal padding at the borders.
Args:
image_array: numpy array of the input image.
patch_size: size of the patches from the original image (without padding).
padding_size: size of the overlapping area.
"""
xmax, ymax, _ = image_array.shape
x_remainder = xmax % patch_size
y_remainder = ymax % patch_size
# modulo here is to avoid extending of patch_size instead of 0
x_extend = (patch_size - x_remainder) % patch_size
y_extend = (patch_size - y_remainder) % patch_size
# make sure the image is divisible into regular patches
extended_image = np.pad(image_array, ((0, x_extend), (0, y_extend), (0, 0)), 'edge')
# add padding around the image to simplify computations
padded_image = self.pad_patch(extended_image, padding_size, channel_last=True)
xmax, ymax, _ = padded_image.shape
patches = []
x_lefts = range(padding_size, xmax - padding_size, patch_size)
y_tops = range(padding_size, ymax - padding_size, patch_size)
for x in x_lefts:
for y in y_tops:
x_left = x - padding_size
y_top = y - padding_size
x_right = x + patch_size + padding_size
y_bottom = y + patch_size + padding_size
patch = padded_image[x_left:x_right, y_top:y_bottom, :]
patches.append(patch)
return np.array(patches), padded_image.shape
# joing the patches
def stich_together(self, patches, padded_image_shape, target_shape, padding_size=4):
""" Reconstruct the image from overlapping patches.
After scaling, shapes and padding should be scaled too.
Args:
patches: patches obtained with split_image_into_overlapping_patches
padded_image_shape: shape of the padded image contructed in split_image_into_overlapping_patches
target_shape: shape of the final image
padding_size: size of the overlapping area.
"""
xmax, ymax, _ = padded_image_shape
# unpad patches
patches = patches[:, padding_size:-padding_size, padding_size:-padding_size, :]
patch_size = patches.shape[1]
n_patches_per_row = ymax // patch_size
complete_image = np.zeros((xmax, ymax, 3))
row = -1
col = 0
for i in range(len(patches)):
if i % n_patches_per_row == 0:
row += 1
col = 0
complete_image[
row * patch_size: (row + 1) * patch_size, col * patch_size: (col + 1) * patch_size, :
] = patches[i]
col += 1
return complete_image[0: target_shape[0], 0: target_shape[1], :]
Initiate Slicing
import numpy as np
isr = ImageSliceRejoin()
padding_size = 1
patches, p_shape = isr.split_image_into_overlapping_patches(
input_img,
patch_size=220,
padding_size=padding_size
)
patches.shape, p_shape, input_img.shape
((12, 222, 222, 3), (882, 662, 3), (719, 640, 3))
Verify
n = np.ceil(patches.shape[0] / 2)
plt.figure(figsize=(20, 20))
patch_size = patches.shape[1]
for i in range(patches.shape[0]):
patch = patches[i]
ax = plt.subplot(n, n, i + 1)
patch_img = np.reshape(patch, (patch_size, patch_size, 3))
plt.imshow(patch_img.astype("uint8"))
plt.axis("off")
Inference
I'm using the Image-Super-Resolution model for demonstration.
# import model
from ISR.models import RDN
model = RDN(weights='psnr-small')
# number of patches that will pass to model for inference:
# here, batch_size < len(patches)
batch_size = 2
for i in range(0, len(patches), batch_size):
# get some patches
batch = patches[i: i + batch_size]
# pass them to model to give patches output
batch = model.model.predict(batch)
# save the output patches
if i == 0:
collect = batch
else:
collect = np.append(collect, batch, axis=0)
Now, the collect holds the output of each patch from the model.
patches.shape, collect.shape
((12, 222, 222, 3), (12, 444, 444, 3))
Rejoin Patches
scale = 2
padded_size_scaled = tuple(np.multiply(p_shape[0:2], scale)) + (3,)
scaled_image_shape = tuple(np.multiply(input_img.shape[0:2], scale)) + (3,)
sr_img = isr.stich_together(
collect,
padded_image_shape=padded_size_scaled,
target_shape=scaled_image_shape,
padding_size=padding_size * scale,
)
Verify
print(input_img.shape, sr_img.shape)
# (719, 640, 3) (1438, 1280, 3)
fig, ax = plt.subplots(1,2)
fig.set_size_inches(18.5, 10.5)
ax[0].imshow(input_img)
ax[1].imshow(sr_img.astype('uint8'))
How do I downsample an image of any resolution to a quarter of the size by averaging the pixels in numpy?
What I came up through research only works for images that are square (i.e 512 X 512 down to 128 X 128) but will not work for images that are different dimensions (i.e 2400 X 1800 down to 600 X 450). In those cases I get a IndexError: index 450 is out of bounds for axis 1 with size 450.
I am trying to perform this task with numpy array manipulation and without installing other packages and libraries.
I researched a function
numpy.mean()
but I don't know how to use it in reference to this problem.
import cv2
import numpy as np
def quarter_res_avg(im):
original_width = im.shape[1]
original_height = im.shape[0]
width = original_width / 4
height = original_height / 4
resized_image = np.zeros(shape=(width, height, 3), dtype=np.uint8)
scale = 4
for i in range(width):
for j in range(height):
temp = np.array([0, 0, 0])
for x in range(scale):
for y in range(scale):
temp += im[i*scale+x, j*scale+y]
resized_image[i, j] = temp/(scale*scale)
return resized_image
im = cv2.imread('Lenna_test_image.png', 1)
cv2.imwrite('Lenna_test_image_avg.png', quarter_res_avg(im))
Any ideas are much appreciated.
Thanks.
import numpy as np
import skimage.measure
your_array = np.random.rand(2400, 800)
new_array = skimage.measure.block_reduce(your_array, (4,4), np.mean)
print(new_array.shape)
Out[18]: (600, 450)
First reshape your M x N image into a (M//4) x 4 x (N//4) x 4 array, then use np.mean in the second and last dimensions.
from typing import Tuple
import numpy as np
def downsample_by_averaging(img: np.ndarray, window_shape: Tuple[int, int]) -> np.ndarray:
return np.mean(
img.reshape((
*img.shape[:-2],
img.shape[-2] // window_shape[-2], window_shape[-2],
img.shape[-1] // window_shape[-1], window_shape[-1],
)),
axis=(-1, -3),
)
downsample_by_averaging(img, (4, 4))
The answer that worked for me with the help from #MarkSetchell in the comments of the question.
Without using np.mean()
def quarter_res_avg(im):
original_width = im.shape[1]
original_height = im.shape[0]
width = original_width / 4
height = original_height / 4
resized_image = np.zeros(shape=(height, width, 3), dtype=np.uint8)
scale = 4
for i in range(height):
for j in range(width):
temp = np.array([0, 0, 0])
for x in range(scale):
for y in range(scale):
temp += im[i*scale + x, j*scale + y]
resized_image[i, j] = temp/(scale*scale)
return resized_image
im = cv2.imread('Lenna_test_image.png', 1)
cv2.imwrite('Lenna_test_image_resized.png', quarter_res_avg(im))
By using np.mean() replace the for loops with:
for i in range(0, original_height, scale):
for j in range(0, original_width, scale):
resized_image[i/scale, j/scale] = np.mean(im[i:i + scale, j:j+scale], axis=(0,1))
I'm using PIL to load images and then transform them to NumPy arrays. Then I've to create a new image based on a list of images, so I append all theearrays to a list and then transform the list back to an array, so the shape for the list of images has 4 dimensions (n_images, height, width, rgb_channels). I'm using this code:
def gallery(array, ncols=4):
nindex, height, width, intensity = array.shape
nrows = nindex // ncols
# want result.shape = (height*nrows, width*ncols, intensity)
result = (array.reshape(nrows, ncols, height, width, intensity)
.swapaxes(1,2)
.reshape(height*nrows, width*ncols, intensity))
return result
def make_array(dim_x):
for i in range(dim_x):
print('series',i)
series = []
for j in range(TIME_STEP-1):
print('photo',j)
aux = np.asarray(Image.open(dirpath+'/images/pre_images /series_{0}_Xquakemap_{1}.jpg'.format(i,j)).convert('RGB'))
print(np.shape(aux))
series.append(aux)
print(np.shape(series))
im = Image.fromarray(gallery(np.array(series)))
im.save(dirpath+'/images/gallery/series_{0}_Xquakemap.jpg'.format(i))
im_shape = (im.size)
make_array(n_photos)
# n_photos is the total of photos in the dirpath
The problem is when the append on the series list happened, the shape of the image (the NumPy array added) gets lost. So when trying to reshape the array in the function gallery it causes a problem. A snippet of the output for the code above is this one:
...
series 2
photo 0
(585, 619, 3)
(1, 585, 619, 3)
photo 1
(587, 621, 3)
(2,)
photo 2
(587, 621, 3)
(3,)
photo 3
(587, 621, 3)
(4,)
...
As you can see, when appending the second photo the list loses a dimension. This is weird because the code works the first two iterations, which use fairly the same images. I tried using np.stack() but the error prevails.
I also find this issue on Github but I think it doesn't apply to this case even if the behavior is similar.
Working on Ubuntu 18, Python 3.7.3 and Numpy 1.16.2.
edit: added what #kwinkunks asked
In the second function, I think you need to move series = [] to before the outer loop.
Here's my reproduction of the problem:
import numpy as np
from PIL import Image
TIME_STEP = 3
def gallery(array, ncols=4):
"""Stitch images together."""
nindex, height, width, intensity = array.shape
nrows = nindex // ncols
result = array.reshape(nrows, ncols, height, width, intensity)
result = result.swapaxes(1,2)
result = result.reshape(height*nrows, width*ncols, intensity)
return result
def make_array(dim_x):
"""Make an image from a list of arrays."""
series = [] # <<<<<<<<<<< This is the line you need to check.
for i in range(dim_x):
for j in range(TIME_STEP - 1):
aux = np.ones((100, 100, 3)) * np.random.randint(0, 256, 3)
series.append(aux.astype(np.uint8))
im = Image.fromarray(gallery(np.array(series)))
return im
make_array(4)
This results in:
I'm trying to get texture properties from a GLCM I created using greycomatrix from skimage.feature. My input data is an image with multiple bands and I want the texture properties for each pixel (resulting in an image with the dimensions cols x rows x (properties *bands)), as it can be achieved using ENVI. But I'm too new to this to come to grips with greycomatrix and greycoprops. This is what I tried:
import numpy as np
from skimage import io
from skimage.feature import greycomatrix, greycoprops
array = io.imread('MYFILE.tif')
array = array.astype(np.int64)
props = ['contrast', 'dissimilarity', 'homogeneity', 'energy', 'correlation', 'ASM']
textures = np.zeros((array.shape[0], array.shape[1], array.shape[2] * len(props)), np.float32)
angles = [0, np.pi / 4, np.pi / 2, 3 * np.pi / 4]
bands = array.shape[2]
for b in range(bands):
glcm = greycomatrix(array[:, :, b], [1], angles, np.nanmax(array) + 1,
symmetric=True, normed=True)
for p, prop in enumerate(props):
textures[:, :, b] = greycoprops(glcm, prop)
Unfortunately, this gives me a 1 x 4 matrix per prop, which I guess is one value per angle FOR THE WHOLE IMAGE, but this is not what I want. I need it per pixel, like contrast for each single pixel, computed from its respective surroundings. What am I missing?
This snippet should get the job done:
import numpy as np
from skimage import io, util
from skimage.feature.texture import greycomatrix, greycoprops
img = io.imread('fourbandimg.tif')
rows, cols, bands = img.shape
radius = 5
side = 2*radius + 1
distances = [1]
angles = [0, np.pi/2]
props = ['contrast', 'dissimilarity', 'homogeneity']
dim = len(distances)*len(angles)*len(props)*bands
padded = np.pad(img, radius, mode='reflect')
windows = [util.view_as_windows(padded[:, :, band].copy(), (side, side))
for band in range(bands)]
feats = np.zeros(shape=(rows, cols, dim))
for row in range(rows):
for col in range(cols):
pixel_feats = []
for band in range(bands):
glcm = greycomatrix(windows[band][row, col, :, :],
distances=distances,
angles=angles)
pixel_feats.extend([greycoprops(glcm, prop).ravel()
for prop in props])
feats[row, col, :] = np.concatenate(pixel_feats)
The sample image has 128 rows, 128 columns and 4 bands (click here to download). At each image pixel a square local neighbourhood of size 11 is used to compute the grayscale matrices corresponding to the pixel to the right and the pixel above for each band. Then, contrast, dissimilarity and homogeneity are computed for those matrices. Thus we have 4 bands, 1 distance, 2 angles and 3 properties. Hence for each pixel the feature vector has 4 × 1 × 2 × 3 = 24 components.
Notice that in order to preserve the number of rows and columns the image has been padded using the image itself mirrored along the edge of the array. It this approach does not fit your needs you could simply ignore the outer frame of the image.
As a final caveat, the code could take a while to run.
Demo
In [193]: img.shape
Out[193]: (128, 128, 4)
In [194]: feats.shape
Out[194]: (128, 128, 24)
In [195]: feats[64, 64, :]
Out[195]:
array([ 1.51690000e+04, 9.50100000e+03, 1.02300000e+03,
8.53000000e+02, 1.25203577e+01, 9.38930575e+00,
2.54300000e+03, 1.47800000e+03, 3.89000000e+02,
3.10000000e+02, 2.95064854e+01, 3.38267222e+01,
2.18970000e+04, 1.71690000e+04, 1.21900000e+03,
1.06700000e+03, 1.09729371e+01, 1.11741654e+01,
2.54300000e+03, 1.47800000e+03, 3.89000000e+02,
3.10000000e+02, 2.95064854e+01, 3.38267222e+01])
In [196]: io.imshow(img)
Out[196]: <matplotlib.image.AxesImage at 0x2a74bc728d0>
Edit
You could cast your data to the type required by greycomatrix through NumPy's uint8 or scikit-images's img_as_ubyte.
need to read an image as an array and for each pixel select 7*7 neighbor pixels then reshape it and put as a first row of training set:
import numpy as np
from scipy import misc
face1=misc.imread('face1.jpg')
face1 dimensions are (288, 352, 3) , need to find 7*7 neighbor pixels for every pixel , so 49*3 color then reshape it as a (1,147) array and stack it into an array for all pixels , i took the following approach:
X_training=np.zeros([1,147] ,dtype=np.uint8)
for i in range(3, face1.shape[0]-3):
for j in range(3, face1.shape[1]-3):
block=face1[i-3:i+4,j-3:j+4]
pxl=np.reshape(block,(1,147))
X_training=np.vstack((pxl,X_training))
resulting X_training shape is (97572, 147)
and as last row contains all zeros then:
a = len(X_training)-1
X_training = X_training[:a]
above code works well for one picture but with Wall time: 5min 19s i have 2000 images, so it will take ages to do it for all the images. I am looking for a faster way to iterate over every pixel and do the above task.
Edit:
this is what i mean by neighbor pixels , for every pixel face1[i-3 : i+4 ,j-3:j+4]
An efficient way is to use stride_tricks to create a 2d rolling window over the image, then flatten it out:
import numpy as np
face1 = np.arange(288*352*3).reshape(288, 352, 3) # toy data
n = 7 # neighborhood size
h, w, d = face1.shape
s = face1.strides
tmp = np.lib.stride_tricks.as_strided(face1, strides=s[:2] + s,
shape=(h - n + 1, w - n + 1, n, n, d))
X_training = tmp.reshape(-1, n**2 * d)
X_training = X_training[::-1] # to get the rows into same order as in the question
tmp is a 5D view into the image, where tmp[x, y, :, :, c] is equivalent to the neigborhood face1[x:x+n, y:y+n, c] in color channel c.
The following is < 1s on my laptop:
import scipy as sp
im = sp.rand(300, 300, 3)
size = 3
ij = sp.meshgrid(range(size, im.shape[0]-size), range(size, im.shape[1]-size))
i = ij[0].T.flatten()
j = ij[1].T.flatten()
N = len(i)
L = (2*size + 1)**2
X_training = sp.empty(shape=[N, 3*L])
for pixel in range(N):
si = (slice(i[pixel]-size, i[pixel]+size+1))
sj = (slice(j[pixel]-size, j[pixel]+size+1))
X_training[pixel, :] = im[si, sj, :].flatten()
X_training = X_training[-1::-1, :]
I'm always a bit sad when I can't think of one-line vectorized version, but at least it's faster for you.
Using scikit-image:
import numpy as np
from skimage import util
image = np.random.random((288, 352, 3))
windows = util.view_as_windows(image, (7, 7, 3))
out = windows.reshape(-1, 7 * 7 * 3)