I am new to python, and I am trying to execute the following code, but I get the following error:
im[:,:,0] = f
ValueError: could not broadcast input array from shape (700,900,3) into shape (700,900)
Can someone assist me with it?
img = numpy.zeros((700, 900))
row_idx = 160
curve = []
count = 0
for i in range(0, 900):
contour.append((row_idx, i))
values, num_values = get_values(curve);
a = imread('20091016_seg1_26_18245948_1chop.png')
f = numpy.rot90(a, 2)
f = numpy.rot90(a, 2)
size_vec = numpy.shape(img)
im = numpy.zeros((size_vec[0],size_vec[1], 3));
im[:,:,0] = f
im[:,:,1] = f
im[:,:,2] = f
for i in range(0, num_values):
im[values[i].y, values[i].x, 0] = 0.0
im[values[i].y, values[i].x, 1] = 1.0
im[values[i].y, values[i].x, 2] = 0.0
imsave('OUTPUT.png', im)
(eliminated semicolons)
It should work with:
im[..., 0] = f[..., 0]
The problem is that you were trying to put the whole f into im[..., 0], giving the ValeError due to the dimension incompatibilty.
Related
I just learned about matrix convolution from Andrew Ng's deeplearning.ai coursera course and I wanted to try to make my own 2D matrix convolution using numpy but I keep getting the following error. (bottom of the code)
I don't understand what I did wrong as f = 100 and w = 0 for the first iteration so I don't get why aSlice = dataColor[vertStart:vertEnd, vertStart:vertEnd] is a 100 by 99 matrix. I tried to do a +1 to "horiEnd" but it becomes 100 by 101 instead. I also tried assert(dataColor.shape == (1200,1200)) but that doesn't seem to be the problem either. I think the problem starts after the for loops. Any help would be greatly apprieciated.
def convSingleStep(aSlice, F = np.full((100,100), 0.000001)):
#F = f*f filter
s = aSlice*F
Z = np.sum(s)
return Z
def convForward(dataColor, F = np.full((100,100), 0.000001)):
nH, nW = dataColor.shape #1200 x 1200
f, f = F.shape #100 x 100
theNewSlice = np.zeros((nH, nW)) #1200 x 1200
for h in range(nH): #nH = 1200
vertStart = h
vertEnd = h + f
for w in range(nW): #nW = 1200
horiStart = w
horiEnd = w + f
aSlice = dataColor[vertStart:vertEnd, horiStart:horiEnd]
theNewSlice[h, w] = convSingleStep(aSlice, F = np.full((100,100), 0.000001))
return theNewSlice
```----> 3 s = aSlice*F
ValueError: operands could not be broadcast together with shapes (100,99) (100,100)```
In the first step of your loop,
h = 0, , vertEnd = 100
w = 0, , horiEnd = 100
aSlice.shape = (100,100)
This is all good and acting as expected.
But getting to the ~1,100th step (11,101)
h = 1100, , vertEnd = 1200 # Note the index is 1200, which requires length of 1201
w = 0, , horiEnd = 100
And now you are setting the shape out of the range of your data, with vertEnd = 1200:
aSlice = dataColor[vertStart:vertEnd, horiStart:horiEnd]
aSlice.shape # 99, 100
Here,
aSlice[0] = dataColor[1101:1200] = dataColor[1101:] = dataColor[1101:1200]
While working on a ML project, we encountered the following value error:
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 118 and the array at index 1 has size 850.
The function that we have been working on is given below:
def function_FeatureExtract(path, indices, featureNum, classLabel):
extension = '.wav'
ListOfFrame2Vec = np.empty((0, frame_number, featureNum))
LabelOfTotalFrame2Vec = np.empty((0, 1, 1))
DictionaryFiletoIndex=[]
for root, dirs_list, files_list in os.walk(path):
for file_name in files_list:
#if os.path.splitext(file_name)[-1] == extension:
audiofile = os.path.join(root, file_name)
print(audiofile)
audio, s_rate = librosa.load(audiofile, sr=sample_rate)
#print(file_name)
segment_start_flag=0
start_seg = 0
while (start_seg + segment_length) < len(audio):
sound1 = audio[start_seg:(start_seg + segment_length)]
featureSet=function_FeatureExtractfromSinglewindow(sound1, hop_length, sample_rate)
if segment_start_flag==0:
SegAllFeat=featureSet
segment_start_flag=1
else:
SegAllFeat=np.vstack((SegAllFeat,featureSet))
start_seg = start_seg + overlappiong
if segment_start_flag==1:
SegAllFeat=normalize(SegAllFeat, norm = 'l2', axis=0)
SegAllFeat=SegAllFeat[:, indices[0:featureNum]]
ListOfFrame2Vec = np.append(ListOfFrame2Vec, array([SegAllFeat]), axis=0)
if classLabel == 1:
LabelOfTotalFrame2Vec=np.append(LabelOfTotalFrame2Vec,np.array([1]))
else:
LabelOfTotalFrame2Vec = np.append(LabelOfTotalFrame2Vec, np.array([0]))
#LabelOfTotalFrame2Vec
DictionaryFiletoIndex.append(file_name)
print(ListOfFrame2Vec.shape)
print(LabelOfTotalFrame2Vec.shape)
return ListOfFrame2Vec, LabelOfTotalFrame2Vec, DictionaryFiletoIndex
The line of code causing the error is:
ListOfFrame2Vec = np.append(ListOfFrame2Vec, array([SegAllFeat]), axis=0)
The shape of SegAllFeat is (850,80) and that of ListOfFrame2Vec is (0,118,80), but reshaping these arrays has not been successfull.
I'm working on a neural network where I am augmenting data via rotation and varying the size of each input volume.
Let me back up, the input to the network is a 3D volume. I generate variable size 3D volumes, and then pad each volume with zero's such that the input volume is constant. Check here for an issue I was having with padding (now resolved).
I generate a variable size 3D volume, append it to a list, and then convert the list into a numpy array. At this point, padding hasn't occured so converting it into a 4D tuple makes no sense...
input_augmented_matrix = []
label_augmented_matrix = []
for i in range(n_volumes):
if i % 50 == 0:
print ("Augmenting step #" + str(i))
slice_index = randint(0,n_input)
z_max = randint(5,n_input)
z_rand = randint(3,5)
z_min = z_max - z_rand
x_max = randint(75, n_input_x)
x_rand = randint(60, 75)
x_min = x_max - x_rand
y_max = randint(75, n_input_y)
y_rand = randint(60, 75)
y_min = y_max - y_rand
random_rotation = randint(1,4) * 90
for j in range(2):
temp_volume = np.empty((z_rand, x_rand, y_rand))
k = 0
for z in range(z_min, z_max):
l = 0
for x in range(x_min, x_max):
m = 0
for y in range(y_min, y_max):
if j == 0:
#input volume
try:
temp_volume[k][l][m] = input_matrix[z][x][y]
except:
pdb.set_trace()
else:
#ground truth volume
temp_volume[k][l][m] = label_matrix[z][x][y]
m = m + 1
l = l + 1
k = k + 1
temp_volume = np.asarray(temp_volume)
temp_volume = np.rot90(temp_volume,random_rotation)
if j == 0:
input_augmented_matrix.append(temp_volume)
else:
label_augmented_matrix.append(temp_volume)
input_augmented_matrix = np.asarray(input_augmented_matrix)
label_augmented_matrix = np.asarray(label_augmented_matrix)
The dimensions of input_augmented_matrix at this point is (N,)
Then I pad with the following code...
for i in range(n_volumes):
print("Padding volume #" + str(i))
input_augmented_matrix[i] = np.lib.pad(input_augmented_matrix[i], ((0,n_input_z - int(input_augmented_matrix[i][:,0,0].shape[0])),
(0,n_input_x - int(input_augmented_matrix[i][0,:,0].shape[0])),
(0,n_input_y - int(input_augmented_matrix[i][0,0,:].shape[0]))),
'constant', constant_values=0)
label_augmented_matrix[i] = np.lib.pad(label_augmented_matrix[i], ((0,n_input_z - int(label_augmented_matrix[i][:,0,0].shape[0])),
(0,n_input_x - int(label_augmented_matrix[i][0,:,0].shape[0])),
(0,n_input_y - int(label_augmented_matrix[i][0,0,:].shape[0]))),
'constant', constant_values=0)
At this point, the dimensions are still (N,) even though every element of the list is constant. For example input_augmented_matrix[0] = input_augmented_matrix[1]
Currently I just loop through and create a new array, but it takes too long and I would prefer some sort of method that automates this. I do it with the following code...
input_4d = np.empty((n_volumes, n_input_z, n_input_x, n_input_y))
label_4d = np.empty((n_volumes, n_input_z, n_input_x, n_input_y))
for i in range(n_volumes):
print("Converting to 4D tuple #" + str(i))
for j in range(n_input_z):
for k in range(n_input_x):
for l in range(n_input_y):
input_4d[i][j][k][l] = input_augmented_matrix[i][j][k][l]
label_4d[i][j][k][l] = label_augmented_matrix[i][j][k][l]
Is there a cleaner and faster way to do this?
As I understood this part
k = 0
for z in range(z_min, z_max):
l = 0
for x in range(x_min, x_max):
m = 0
for y in range(y_min, y_max):
if j == 0:
#input volume
try:
temp_volume[k][l][m] = input_matrix[z][x][y]
except:
pdb.set_trace()
else:
#ground truth volume
temp_volume[k][l][m] = label_matrix[z][x][y]
m = m + 1
l = l + 1
k = k + 1
You just want to do this
temp_input = input_matrix[z_min:z_max, x_min:x_max, y_min:y_max]
temp_label = label_matrix[z_min:z_max, x_min:x_max, y_min:y_max]
and then
temp_input = np.rot90(temp_input, random_rotation)
temp_label = np.rot90(temp_label, random_rotation)
input_augmented_matrix.append(temp_input)
label_augmented_matrix.append(temp_label)
Here
input_augmented_matrix[i] = np.lib.pad(
input_augmented_matrix[i],
((0,n_input_z - int(input_augmented_matrix[i][:,0,0].shape[0])),
(0,n_input_x - int(input_augmented_matrix[i][0,:,0].shape[0])),
(0,n_input_y - int(input_augmented_matrix[i][0,0,:].shape[0]))),
'constant', constant_values=0)
Better to do this, because shape property gives you size of array by all dimensions
ia_shape = input_augmented_matrix[i].shape
input_augmented_matrix[i] = np.lib.pad(
input_augmented_matrix[i],
((0, n_input_z - ia_shape[0]),
(0, n_input_x - ia_shape[1])),
(0, n_input_y - ia_shape[2]))),
'constant',
constant_values=0)
I guess now you're ready to refactor the last part of your code with magic indexing of NumPy.
My common suggestions:
use functions for repeated parts of code to avoid such indents like in your cascade of loops;
if you need so lot of nested loops, think about recursion, if you can't deal without them;
explore abilities of NumPy in official documentation: they're really exciting ;) For example, indexing is helpful for this task;
use PyLint and Flake8 packages to inspect quality of your code.
Do you want to write neural network by yourself, or you just want to solve some patterns recognition task? SciPy library may contain what you need and it's based on NumPy.
I want to write HDF5 format as 512x424(1channel) image database and multi-label in python (anaconda library) for image regression.
However, I have some error which is happened when memory allocation as array type.
import h5py
import numpy as np
cols = 512
rows = 424
channel = 1
total_image = sum(1 for line in open(data_label_list))
total_size = total_image*rows*cols
f_train = open(data_label_list)
lines = f_train.readlines()
data = np.empty((total_image, channel, cols, rows))
multi_label = np.empty((total_image, 1))
==========================================
data = np.empty((total_image, channel, cols, rows))
"MemoryError"
Can I solve this error for large scale database?
add source code :
cnt = 0
for line in lines:
line = line.rstrip('\n')
list_line = line.split(' ')
im = Image.open(list_line[0])
row, col = im.size
pixels = im.load()
for i in range(row):
for j in range(col):
data[cnt, 0, i, j] = pixels[i, j]
multi_label[cnt, 0] = list_line[1]
multi_label[cnt, 1] = list_line[2]
cnt += 1
Thanks,
In my code, I am trying to import a grayscale image (2D array) and then solve for the optical density (OD) based off an empirical formula I came up with. The optical density has a relationship with the grayscale value where OD = 0.51*((f-22.08)/(176.09-f))**(1./-1.519) where f is the grayscale value of each element in the array. Then, I converted it into an RGB image.
My problem is I am trying to run each individual element of the image array into an if statement. It does not enter the statement though. What I want to do is increase the intensity of each individual element or pixel value in R, G, and B based on what condition is met with the optical density. Say if it has an OD value that falls between b and c, it adds [128,0,0] to each element that satisfies that criteria.
t = Image.open("IMG_1.jpg").convert('L') #grayscale image
f = array(t) #Convert test image into an array
OD = 0.51*((f-22.08)/(176.09-f))**(1./-1.519) #Empirical Rodbard formula
OD[np.isnan(OD)] = 0
def to_rgb5(im):
OD.resize((OD.shape[0], OD.shape[1], 1))
return np.repeat(OD.astype(np.uint8), 3, 2)
cmap = plt.get_cmap('jet')
rgba_img = cmap(OD)
rgb_img = np.delete(rgba_img, 3, 2)
a = 0.08
b = 0.11
c = 0.15
if np.all(OD < a):
background_noise = rgb_img
if np.all(OD < b):
small = rgb_img + [128, 0, 0]
elif np.all(OD >= c):
large = rgb_img + [0, 0, 128]
Red = f + small
Green = f
Blue = f + large
I was able to get it work using the np.where suggested by #Stuart
t = Image.open("IMG_1.jpg").convert('L') #grayscale image
f = array(t) #Convert test image into an array
OD = 0.51*((f-22.08)/(176.09-f))**(1./-1.519) #Empirical Rodbard formula
OD[np.isnan(OD)] = 0
thB = 0.02
ththin = 0.11
ththick = 0.15
GSb = 162
GSthin = 150
GSthick = 145
if np.where(OD < ththin):
thresholded = threshold(f, GSthin, GSb)
def to_rgb1(thresholded):
thresholded.resize((thresholded.shape[0], thresholded.shape[1], 1))
return np.repeat(thresholded.astype(np.uint8), 3, 2)
cmap = plt.get_cmap('jet')
rgba_img1 = cmap(thresholded)
rgb_img1 = np.delete(rgba_img1, 3, 2)
view = rgb_img1[:, :, 2]
view[view < 0.1] += 128
thin = rgb_img1
if np.where(OD > ththick):
thresholded2 = threshold(f, threshmax = GSthick)
def to_rgb2(thresholded2):
thresholded2.resize((thresholded2.shape[0], thresholded2.shape[1], 1))
return np.repeat(thresholded2.astype(np.uint8), 3, 2)
cmap = plt.get_cmap('jet')
rgba_img2 = cmap(thresholded2)
rgb_img2 = np.delete(rgba_img2, 3, 2)
view2 = rgb_img2[:, :, 0]
view2[view2 > 0] += 128
thick = rgb_img2