Related
I have several 3D images of shape (32,32,32) and I want to create 2D images from them. I want to do that by getting each slice in the z-axis and putting each of them in a square array in order, something like this:
Because I want the 2D image to be square I need to fill the missing slices with zeros (Black in the example).
This is what I did:
# I created an array of the desired dimensions
grid = np.zeros((6*32,6*32))
# Then, I assigned to each section of the grid the values of every slice of the 3d_image:
grid[0:32, 0:32] = 3d_image[:,:,0]
grid[0:32, 32:64] = 3d_image[:,:,1]
grid[0:32, 64:96] = 3d_image[:,:,2]
grid[0:32, 96:128] = 3d_image[:,:,3]
grid[0:32, 128:160] = 3d_image[:,:,4]
grid[0:32, 160:192] = 3d_image[:,:,5]
grid[32:64, 0:32] = 3d_image[:,:,6]
grid[32:64, 32:64] = 3d_image[:,:,7]
grid[32:64, 64:96] = 3d_image[:,:,8]
grid[32:64, 96:128] = 3d_image[:,:,9]
grid[32:64, 128:160] = 3d_image[:,:,10]
grid[32:64, 160:192] = 3d_image[:,:,11]
grid[64:96, 0:32] = 3d_image[:,:,12]
grid[64:96, 32:64] = 3d_image[:,:,13]
...
grid[160:192, 160:192] = 3d_image[:,:,31]
And It worked!! But I want to automate it, so I tried this:
d = [0, 32, 64, 96, 128, 160]
for j in range(6):
for i in d:
grid[0:32, i:i+32] = 3d_image[:,:,j]
But it didn't work, the slice index for 3d_image (j) is not changing, and I don't know how to change the index range for grid after every 6th slice.
Could you help me?
Assuming that that img is an array of the shape (32,32,32), this should work:
N = 32
a = np.vstack([img, np.zeros((4, N, N), dtype=img.dtype)])
grid = a.transpose(1, 0, 2).reshape(N, -1, 6*N).transpose(1, 0, 2).reshape(6*N, -1)
Here's an automated way to do it. Let's say your array with shape (32, 32, 32) is called n. Note that this method relies on all 3 dimensions having the same size.
num_layers = n.shape[0]
# num_across = how many images will go in 1 row or column in the final array.
num_across = int(np.ceil(np.sqrt(num_layers)))
# new_shape = how many numbers go in a row in the final array.
new_shape = num_across * num_layers
final_im = np.zeros((new_shape**2)).reshape(new_shape, new_shape)
for i in range(num_layers):
# Get what number row and column the image goes in (e.g. in the example,
# the image labelled 28 is in the 4th (3rd with 0-indexing) column and 5th
# (4th with 0-indexing) row.
col_num = i % num_across
row_num = i // num_across
# Put the image in the appropriate part of the final image.
final_im[row_num*num_layers:row_num*num_layers + num_layers, col_num*num_layers:col_num*num_layers + num_layers] = n[i]
final_im now contains what you want. Below is a representation where each image is a different color and the "black" areas are purple because matplotlib colormaps are funny like that:
Anyway, you can tell that the images go where they're supposed to and you get your empty space along the bottom.
I'm doing a project about stitching multiple images together(Panorama) in OpenCv. I tried multiple datasets, for some, I get the result however, most of the time I keep getting this error: index 0 is out of bounds for axis 1 with size 0.
**This the code part for the above errors
**
if __name__ == "__main__":
# Reading images.
Images = ReadImage("C:/Users/Ruchi Mishra/Documents/Project/InputImages/Temple")
BaseImage, _, _ = ProjectOntoCylinder(Images[0])
for i in range(1, len(Images)):
StitchedImage = StitchImages(BaseImage, Images[i])
BaseImage = StitchedImage.copy()
cv2.imwrite("Stitched_Panorama_Temple.png", BaseImage)
def StitchImages(BaseImage, SecImage):
# Applying Cylindrical projection on SecImage
SecImage_Cyl, mask_x, mask_y = ProjectOntoCylinder(SecImage)
# Getting SecImage Mask
SecImage_Mask = np.zeros(SecImage_Cyl.shape, dtype=np.uint8)
SecImage_Mask[mask_y, mask_x, : ] = 255
# Finding matches between the 2 images and their keypoints
Matches, BaseImage_kp, SecImage_kp = FindMatches(BaseImage, SecImage_Cyl)
# Finding homography matrix.
HomographyMatrix, Status = FindHomography(Matches, BaseImage_kp, SecImage_kp)
# Finding size of new frame of stitched images and updating the homography matrix
NewFrameSize, Correction, HomographyMatrix = GetNewFrameSizeAndMatrix(HomographyMatrix, SecImage_Cyl.shape[:2], BaseImage.shape[:2])
# Finally placing the images upon one another.
SecImage_Transformed = cv2.warpPerspective(SecImage_Cyl, HomographyMatrix, (NewFrameSize[1], NewFrameSize[0]))
SecImage_Transformed_Mask = cv2.warpPerspective(SecImage_Mask, HomographyMatrix, (NewFrameSize[1], NewFrameSize[0]))
BaseImage_Transformed = np.zeros((NewFrameSize[0], NewFrameSize[1], 3), dtype=np.uint8)
BaseImage_Transformed[Correction[1]:Correction[1]+BaseImage.shape[0], Correction[0]:Correction[0]+BaseImage.shape[1]] = BaseImage
StitchedImage = cv2.bitwise_or(SecImage_Transformed, cv2.bitwise_and(BaseImage_Transformed, cv2.bitwise_not(SecImage_Transformed_Mask)))
return StitchedImage
I understand the reason is that I am trying to access an index (here at 0th position), which is not there.
I have some code, which works as intended, however takes about 4 and a half hours to run, I understand that there are about 50 billion calculations my poor pc needs to do but I thought it would be worth asking!
This code gets an image, and wants to find every possible region of 331*331 pixels in the given image, and find how many black pixels there are in each, I will use this data to create a heatmap of black pixel density, and also a list of all of the values found:
image = Image.open(self.selectedFile)
pixels = list(image.getdata())
width, height = image.size
pixels = [pixels[i * width:(i+1) * width] for i in range(height)]
#print(pixels)
rightShifts = width - 331
downShifts = height - 331
self.totalRegionsLabel['text'] = f'Total Regions: {rightShifts * downShifts}'
self.blackList = [0 for i in range(0, rightShifts*downShifts)]
self.heatMap = [[] for i in range(0, downShifts)]
for x in range(len(self.heatMap)):
self.heatMap[x] = [0 for i in range(0, rightShifts)]
for x in range(rightShifts):
for y in range(downShifts):
blackCount = 0
for z in range(x + 331):
for w in range(y + 331):
if pixels[z][w] == 0:
blackCount += 1
self.blackList[x+1*y] = blackCount
self.heatMap[x][y] = blackCount
print(self.blackList)
You have several problems here, as I pointed out. Your z/w loops are always starting at the upper left, so by the time you get towards the end, you're summing the entire image, not just a 331x331 subset. You also have much confusion in your axes. In an image, [y] is first, [x] is second. An image is rows of columns. You need to remember that.
Here's an implementation as I suggested above. For each column, I do a full sum on the top 331x331 block. Then, for every row below, I just subtract the top row and add the next row below.
self.heatMap = [[0]*rightShifts for i in range(downShifts)]
for x in range(rightShifts):
# Sum up the block at the top.
blackCount = 0
for row in range(331):
for col in range(331):
if pixels[row][x+col] == 0:
blackCount += 1
self.heatMap[0][x] = blackCount
for y in range(1,downShifts):
# To do the next block down, we subtract the top row and
# add the bottom.
for col in range(331):
blackCount += pixels[y+330][x+col] - pixels[y-1][x+col]
self.heatMap[y][x] = blackCount
You could tweak this even more by alternating the columns. So, at the bottom of the first column, scoot to the right by subtracting the first column and adding the next new column. then scoot back up to the top. That's a lot more trouble.
The two innermost for-loops seem to be transformable to some numpy code if using this package is not an issue. It would give something like:
pixels = image.get_data() # it is probably already a numpy array
# Get an array filled with either True or False, with True whenever pixel is black:
pixel_is_black = (pixels[x:(x+331), y:(y+331)] == 0)
pixel_is_black *= 1 # Transform True and False to respectively 1 and 0. Maybe not needed
self.blackList[x+y] = pixel_is_black.sum() # self explanatory
This is the simplest optimization I can think of, you probably can do much better with clever numpy tricks.
I would recommend using some efficient vector computations through the numpy and opencv libraries.
First, binarize your image so that black pixels are set to zero, and any other color pixels (gray to white) are set to 1. Then, apply a 2D filter to the image of shape 331 x 331 where each value in the filter kernel is (1 / (331 x 331) - this will take the average of all the values in each 331x331 area and assign it to the center pixel.
This gives you a heatmap, where each pixel value is the proportion of non-black pixels in the surrounding 331 x 331 region. A darker pixel (value closer to zero) means more pixels in that region are black.
For some background, this approach uses image processing techniques called image binarization and box blur
Example code:
import cv2
import numpy as np
# setting up a fake image, with some white spaces, gray spaces, and black spaces
img_dim = 10000
fake_img = np.full(shape=(img_dim, img_dim), fill_value=255, dtype=np.uint8) # white
fake_img[: img_dim // 3, : img_dim // 3] = 0 # top left black
fake_img[2 * img_dim // 3 :, 2 * img_dim // 3 :] = 0 # bottom right black
fake_img[img_dim // 3 : 2 * img_dim // 3, img_dim // 3 : 2 * img_dim // 3] = 127 # center gray
# show the fake image
cv2.imshow("", fake_img)
cv2.waitKey()
cv2.destroyAllWindows()
# solution to your problem
binarized = np.where(fake_img == 0, 0, 1) # have 0 values where black, 1 values else
my_filter = np.full(shape=(331, 331), fill_value=(1 / (331 * 331))) # set up filter
heatmap = cv2.filter2D(fake_img, 1, my_filter) # apply filter, which takes average of values in 331x331 block
# show the heatmap
cv2.imshow("", heatmap)
cv2.waitKey()
cv2.destroyAllWindows()
I ran this on my laptop, with a huge (fake) image of 10000 x 10000 pixels, almost instantly.
Sorry I should have deleted this post before you all put the effort in, however, some of these workarounds are really smart and interesting, I ended up coming up with a solution independently that is the same as what Tim Robbers first suggested, I used the array I had and built a second one on which every item in a row is the number of black cells preceding it, and then for each row in a region instead of scanning every item, just scan the preceding value and the final value and you are good:
image = Image.open(self.selectedFile).convert('L') #convert to luminance mode as RGB information is irrelevant
pixels = list(image.getdata()) #get the value of every pixel in the image
width, height = image.size
pixels = [pixels[i * width:(i+1) * width] for i in range(height)] #split the pixels array into a two dimensional array with the dimensions to match the image
#This program scans every possible 331*331 square starting from the top left, so it will move right width - 331 pixels and down height - 331 pixels
rightShifts = width - 331
downShifts = height - 331
self.totalRegionsLabel['text'] = f'Total Regions: {rightShifts * downShifts}' #This wont update till the function has completed running
#The process of asigning new values to values in an array is faster than appending them so this is why I prefilled the arrays:
self.heatMap = [[] for i in range(0, downShifts)]
for x in range(len(self.heatMap)):
self.heatMap[x] = [0 for i in range(0, rightShifts)]
cumulativeMatrix = [] #The cumulative matrix replaces each value in each row with how many zeros precede it
for y in range(len(pixels)):
cumulativeMatrix.append([])
cumulativeMatrix[y].append(0)
count = 0
for x in range(len(pixels[y])):
if pixels[y][x] == 0:
count += 1
cumulativeMatrix[y].append(count)
regionCount = 0
maxValue = 0 #this is the lowest possible maximum value
minValue = 109561 #this is the largest possible minimum value
self.blackList = []
#loop through all possible regions
for y in range(downShifts):
for x in range(rightShifts):
blackPixels = 0
for regionY in range(y, y + 331):
lowerLimit = cumulativeMatrix[regionY][x]
upperLimit = cumulativeMatrix[regionY][x+332]
blackPixels += (upperLimit - lowerLimit)
if blackPixels > maxValue:
maxValue = blackPixels
if blackPixels < minValue:
minValue = blackPixels
self.blackList.append(blackPixels)
self.heatMap[y][x] = blackPixels
regionCount += 1
This brought run time to under a minute and thus solved my problem, however, thank you for your contributions I have learned a lot from reading them!
Try to look into the map() function. It uses C to streamline iterations.
You can speed up your for loops like this:
pixels = list(map(lambda i: x[i*width:(i+1)*width], range(height)))
This function receives a list of numpy arrays that consist of cropped parts of an image. The crops are all the same size, except for the right-most and bottom-most images which might be of smaller size.
predictions[2] would return the 3rd sub-image that was cropped from the original image. Each crop is a numpy array. There are WxH crops, enumerated from left to right, top to bottom (so if there are 4 sub-images constituting the width, the 5th image in predictions would be the first sub-image on the left from the 2nd row of sub-images).
crops contains the necessary information to find number of horizontal and vertical images that will constitute the reconstructed images. crops[2][3] will contain the 3rd from the top, 4th from the left image cropped.
The images contained by crops are of smaller dimension than the ones in predictions (I am basically making a model that increases the resolution of images). The reconstructed image if from the images in predictions, arranged in the same order as the ones in crops.
def reconstruct(predictions, crops):
if len(crops) != 0:
print("use crops")
# TODO: properly extract the size of the full image
width_length = 0
height_length = 0
full_image = np.empty(shape=(height_length, width_length))
print(full_image.shape)
# TODO: properly merge the crops back into a single image
for height in range(len(predictions[0])):
for width in range(len(predictions)):
# concatenate here
print(height, width)
return full_image
I was going to use numpy.concatenate, but according to other answers I've seen on SO it wouldn't be an efficient way of doing it (apparently numpy will just recreate a new variable in memory, copy the old one, and add the new data, etc.). So now I'm left wondering how to properly merge my multiple images into a single image. The current idea I was going for was to create a python list of the proper shape and progressively fill it with each numpy array's data, but even that I'm not sure if it's the proper idea.
Here is more or less the kind of bunch of images I'm trying to concatenate into a single image:
Here is the expected result:
And to help you out with understanding what more might be available to you, here is some more code:
def predict(args):
model = load_model(save_dir + '/' + args.model)
image = skimage.io.imread(tests_path + args.image)
predictions = []
images = []
crops = seq_crop(image) # crops into multiple sub-parts the image based on 'input_' constants
for i in range(len(crops)): # amount of vertical crops
for j in range(len(crops[0])): # amount of horizontal crops
current_image = crops[i][j]
images.append(current_image)
# Hack because GPU can only handle one image at a time
input_img = (np.expand_dims(images[p], 0)) # Add the image to a batch where it's the only member
predictions.append(model.predict(input_img)[0]) # returns a list of lists, one for each image in the batch
return predictions, image, crops
# adapted from: https://stackoverflow.com/a/52463034/9768291
def seq_crop(img):
"""
To crop the whole image in a list of sub-images of the same size.
Size comes from "input_" variables in the 'constants' (Evaluation).
Padding with 0 the Bottom and Right image.
:param img: input image
:return: list of sub-images with defined size
"""
width_shape = ceildiv(img.shape[1], input_width)
height_shape = ceildiv(img.shape[0], input_height)
sub_images = [] # will contain all the cropped sub-parts of the image
for j in range(height_shape):
horizontal = []
for i in range(width_shape):
horizontal.append(crop_precise(img, i*input_width, j*input_height, input_width, input_height))
sub_images.append(horizontal)
return sub_images
def crop_precise(img, coord_x, coord_y, width_length, height_length):
"""
To crop a precise portion of an image.
When trying to crop outside of the boundaries, the input to padded with zeros.
:param img: image to crop
:param coord_x: width coordinate (top left point)
:param coord_y: height coordinate (top left point)
:param width_length: width of the cropped portion starting from coord_x
:param height_length: height of the cropped portion starting from coord_y
:return: the cropped part of the image
"""
tmp_img = img[coord_y:coord_y + height_length, coord_x:coord_x + width_length]
return float_im(tmp_img) # From [0,255] to [0.,1.]
# from https://stackoverflow.com/a/17511341/9768291
def ceildiv(a, b):
"""
To get the ceiling of a division
:param a:
:param b:
:return:
"""
return -(-a // b)
if __name__ == '__main__':
preds, original, crops = predict(args) # returns the predictions along with the original
# TODO: reconstruct image
enhanced = reconstruct(preds, crops) # reconstructs the enhanced image from predictions
EDIT:
The answer worked. Here is the version I've used:
# adapted from https://stackoverflow.com/a/52733370/9768291
def reconstruct(predictions, crops):
# unflatten predictions
def nest(data, template):
data = iter(data)
return [[next(data) for _ in row] for row in template]
predictions = nest(predictions, crops)
H = np.cumsum([x[0].shape[0] for x in predictions])
W = np.cumsum([x.shape[1] for x in predictions[0]])
D = predictions[0][0]
recon = np.empty((H[-1], W[-1], D.shape[2]), D.dtype)
for rd, rs in zip(np.split(recon, H[:-1], 0), predictions):
for d, s in zip(np.split(rd, W[:-1], 1), rs):
d[...] = s
return recon
The most convenient is probably np.block
import numpy as np
from scipy import misc
import Image
# get example picture
data = misc.face()
# chop it up
I, J = map(np.arange, (200, 200), data.shape[:2], (200, 200))
chops = [np.split(row, J, axis=1) for row in np.split(data, I, axis=0)]
# do something with the bits
predictions = [chop-(i+j)*(chop>>3) for j, row in enumerate(chops) for i, chop in enumerate(row)]
# unflatten predictions
def nest(data, template):
data = iter(data)
return [[next(data) for _ in row] for row in template]
pred_lol = nest(predictions, chops)
# almost builtin reconstruction
def np_block_2D(chops):
return np.block([[[x] for x in row] for row in chops])
recon = np_block_2D(pred_lol)
Image.fromarray(recon).save('demo.png')
Reconstructed manipulated image:
But we can do faster than that by avoiding intermediary arrays. Instead, we copy into a preallocated array:
def speed_block_2D(chops):
H = np.cumsum([x[0].shape[0] for x in chops])
W = np.cumsum([x.shape[1] for x in chops[0]])
D = chops[0][0]
recon = np.empty((H[-1], W[-1], D.shape[2]), D.dtype)
for rd, rs in zip(np.split(recon, H[:-1], 0), chops):
for d, s in zip(np.split(rd, W[:-1], 1), rs):
d[...] = s
return recon
Timings, also including a generalized ND-ready variant of each method:
numpy 2D: 0.991 ms
prealloc 2D: 0.389 ms
numpy general: 1.021 ms
prealloc general: 0.448 ms
Code for general case and timings:
def np_block(chops):
d = 0
tl = chops
while isinstance(tl, list):
tl = tl[0]
d += 1
if d < tl.ndim:
def adjust_depth(L):
if isinstance(L, list):
return [adjust_depth(l) for l in L]
else:
ret = L
for j in range(d, tl.ndim):
ret = [ret]
return ret
chops = adjust_depth(chops)
return np.block(chops)
def speed_block(chops):
def line(src, i):
while isinstance(src, list):
src = src[0]
return src.shape[i]
def hyper(src, i):
src = iter(src)
fst = next(src)
if isinstance(fst, list):
res, dtype, szs = hyper(fst, i+1)
szs.append([res[i], *(line(s, i) for s in src)])
res[i] = sum(szs[-1])
return res, dtype, szs
res = np.array(fst.shape)
szs = [res[i], *(s.shape[i] for s in src)]
res[i] = sum(szs)
return res, fst.dtype, [szs]
shape, dtype, szs = hyper(chops, 0)
recon = np.empty(shape, dtype)
def cpchp(dst, src, i, szs=None):
szs = np.array(hyper(src, i)[2]) if szs is None else szs
dst = np.split(dst, np.cumsum(szs[-1][:-1]), i)
if isinstance(src[0], list):
szs = szs[:-1]
for ds, sr in zip(dst, src):
cpchp(ds, sr, i+1, szs)
szs = None
else:
for ds, sr in zip(dst, src):
ds[...] = sr
cpchp(recon, chops, 0, np.array(szs))
return recon
from timeit import timeit
T = (timeit(lambda: speed_block(pred_lol), number=1000),
timeit(lambda: np_block(pred_lol), number=1000),
timeit(lambda: speed_block_2D(pred_lol), number=1000),
timeit(lambda: np_block_2D(pred_lol), number=1000))
assert (np.all(speed_block(pred_lol)==np_block(pred_lol)) and
np.all(speed_block_2D(pred_lol)==np_block(pred_lol)) and
np.all(speed_block(pred_lol)==np_block_2D(pred_lol)))
print(f"""
numpy 2D: {T[3]:10.3f} ms
prealloc 2D: {T[2]:10.3f} ms
numpy general: {T[1]:10.3f} ms
prealloc general: {T[0]:10.3f} ms
""")
I want to run through a large tif stack +1500 frames and extract the coordinates of the local maxima for each frame. The code below does the job, however extremely slow for large files. When running on smaller bits (e.g. 20 frames) each frame is done almost instantly - when running on the whole dataset, each frame takes seconds.
Any solutions to run a faster code? I figure it is due to the loading of the large tiff file - however it should only be necessary one time initially?
I have the following code:
from pims import ImageSequence
from skimage.feature import peak_local_max
def cmask(index,array):
radius = 3
a,b = index
nx,ny = array.shape
y,x = np.ogrid[-a:nx-a,-b:ny-b]
mask = x*x + y*y <= radius*radius
return(sum(array[mask])) # number of pixels
images = ImageSequence('tryhard_red_small.tif')
frame_list = []
x = []
y = []
int_liposome = []
BG_liposome = []
for i in range(len(images[0])):
tmp_frame = images[0][i]
xy = pd.DataFrame(peak_local_max(tmp_frame, min_distance=8,threshold_abs=3000))
x.extend(xy[0].tolist())
y.extend(xy[1].tolist())
for j in range(len(xy)):
index = x[j],y[j]
int_liposome.append(cmask(index,tmp_frame))
frame_list.extend([i]*len(xy))
print "Frame: ", i, "of ",len(images[0])
features = pd.DataFrame(
{'lip_int':int_liposome,
'y' : y,
'x' : x,
'frame' : frame_list})
Have you tried profiling the code, say with %prun or %lprun in ipython? That'll tell you exactly where your slowdowns are occurring.
I can't make my own version of this without the tif stack, but I suspect the problem is the fact that you're using lists to store everything. Every time you do an append or an extension, python is having to allocate more memory. You could try getting the total count of maxima first, then allocating your output arrays, then rerunning to fill the arrays. Something like below
# run through once to get the count of local maxima
npeaks = (len(peak_local_max(f, min_distance=8, threshold_abs=3000))
for f in images[0])
total_peaks = sum(npeaks)
# allocate storage arrays and rerun
x = np.zeros(total_peaks, np.float)
y = np.zeros_like(x)
int_liposome = np.zeros_like(x)
BG_liposome = np.zeros_like(x)
frame_list = np.zeros(total_peaks, np.int)
index_0 = 0
for frame_ind, tmp_frame in enumerate(images[0]):
peaks = pd.DataFrame(peak_local_max(tmp_frame, min_distance=8,threshold_abs=3000))
index_1 = index_0 + len(peaks)
# copy the data from the DataFrame's underlying numpy array
x[index_0:index_1] = peaks[0].values
y[index_0:index_1] = peaks[1].values
for i, peak in enumerate(peaks, index_0):
int_liposome[i] = cmask(peak, tmp_frame)
frame_list[index_0:index_1] = frame_ind
# update the starting index
index_0 = index_1
print "Frame: ", frame_ind, "of ",len(images[0])