I want to get some pixel values from an image which are not equal with a specified value. But I want to get back in RGB format and not as a long vector. How can I do it?
import cv2
import numpy as np
image = cv2.imread('image.jpg')
sought = [36,255,12]
result = image[image!=sought]
import sys
np.set_printoptions(threshold=sys.maxsize)
print(result)
And I've got:
[20 103 75 21 98 70 16 100 72 18 101 73 19 97 69 15 95 66
15 95 67 13 101 73 19 104 77 21 96 69 13 94 65 8 99 69
14 98 68 13 94 63 10 88 66 24 92 69 24 92 67 23 93 67
13 93 67 13 93 67 13 97 72 16 96 70 16 93 66 15 96 68
.....
99 69 14 96 66 11 91 67 25 88 65 20 92 68 14 96 69 18
96 70 16 91 64 13 95 67 13 92 64 10 90 63]
But I want something like this:
[[[R,G,B], [R,G,B], [R,G,B], [R,G,B]],
......
[[R,G,B], [R,G,B], [R,G,B], [R,G,B]]]
What did I miss here?
If the wanted output is a list of pixels, then after the component-wise comparison, you must check what pixels differ on any of the R,G or B with .any(axis = 2):
image[(image != sought).any(axis=2)]
output of the form:
array([[ 22, 136, 161],
[197, 141, 153],
[173, 122, 65],
[137, 189, 67],
...
[166, 205, 238],
[207, 99, 129],
[ 44, 76, 97]])
With result = image[image != sought] you lost the shape of image.
The solution is to get a mask (image != sought) and work on the image with that mask (e.g. using np.where)
Generate some data:
import numpy as np
H, W, C = 8, 8, 3
sought = [255, 0, 0]
colors = np.array(
[sought, [0, 0, 255], [0, 255, 0], [0, 255, 255], [255, 0, 255], [255, 255, 0]]
)
colors_idxs = np.random.choice(np.arange(len(colors)), size=(H, W))
image = colors[colors_idxs]
Compute mask (note the keepdims=True for np.where to work easier):
mask = np.any(image != sought, axis=-1, keepdims=True)
# Get color back into the mask
mask_inpaint_pos = np.where(mask, image, 0)
mask_inpaint_neg = np.where(mask, 0, image)
Plot:
import matplotlib.pyplot as plt
fig, (ax_im, ax_mask, ax_mask_pos, ax_mask_neg) = plt.subplots(ncols=4, sharey=True)
ax_im.set_title("Original")
ax_im.imshow(image)
ax_mask.set_title("Mask binary")
ax_mask.imshow(mask.astype(int), cmap="gray")
ax_mask_pos.set_title("Mask True RGB")
ax_mask_pos.imshow(mask_inpaint_pos)
ax_mask_neg.set_title("Mask False RGB")
ax_mask_neg.imshow(mask_inpaint_neg)
plt.show()
I hope that someone can help me with my problem since I'm not used to python and numpy yet. I have the following array with 24 elements:
load = np.array([10, 12, 9, 13, 17, 23, 25, 28, 26, 24, 22, 20, 18, 20, 22, 24, 26, 28, 23, 24, 21, 18, 16, 13])
I want to create a new array with the same length as "load" and calculate for each element in the array the sum of the current and the next two numbers, so that my objective array would look like this:
[31, 34, 39, 53, 65, 76, 79, 78, 72, 66, 60, 58, 60, 66, 72, 78, 77, 75, 68, 63, 55, 47, 29, 13]
I tried to solve this with the following code:
output = np.empty(len(load))
for i in range((len(output))-2):
output[i] = load[i]+load[i+1]+load[i+2]
print(output)
The output array looks like this:
array([31. , 34. , 39. , 53. , 65. , 76. , 79. , 78. , 72. , 66. , 60. ,
58. , 60. , 66. , 72. , 78. , 77. , 75. , 68. , 63. , 55. , 47. ,
6. , 4.5])
The last two numbers are not right. For the 23th element I want the sum of just 16 and 13 and for the last number to stay 13 since the array ends there. I don't unterstand how python calculated these numbers. Also I would prefer the numbers to be integers without the dot.
Does anyone have a better solution in mind? I know that this probably is easy to solve, I just don't know all the functionalities of numpy.
Thank you very much!
np.empty creates an array containing uninitialized data. In your code, you initialize an array output of length 24 but assign only 22 values to it. The last 2 values contain arbitrary values (i.e. garbage). Unless performance is of importance, np.zeros is usually the better choice for initializing arrays since all values will have a consistent value of 0.
You can solve this without a for loop by padding the input array with zeros, then computing a vectorized sum.
import numpy as np
load = np.array([10, 12, 9, 13, 17, 23, 25, 28, 26, 24, 22, 20, 18, 20, 22, 24, 26, 28, 23, 24, 21, 18, 16, 13])
tmp = np.pad(load, [0, 2])
output = load + tmp[1:-1] + tmp[2:]
print(output)
Output
[31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
If the array is not super long, and you don't care too much about memory utilization you could use:
from itertools import zip_longest
output = [sum([x, y, z]) for x, y, z in zip_longest(load, load[1:], load[2:], fillvalue=0)]
Output is:
[31, 34, 39, 53, 65, 76, 79, 78, 72, 66, 60, 58, 60, 66, 72, 78, 77, 75, 68, 63, 55, 47, 29, 13]
I'll address the question "How Python calculated those two numbers" in your source: they were not calculated by your program.
If you notice, your main loop runs until the end of the array but the last two elements. The value of those was not touched. For this reason it corresponds to the data that was in the memory at the position corresponding to the memory allocated by np.empty(). In fact, np.empty() will only acquire the ownership of the memory without initialization (i.e. without changing its content).
A simple approach is to loop through and sum different views of the original array:
def sum_next_k_loop(arr, k):
result = arr.copy()
for i in range(1, k):
result[:-i] += arr[i:]
return result
This is quite fast for relatively small values of k, but as k gets larger one may want to avoid the relatively slow explicit looping.
One way to do this is to use strides to create a view of the array that can be used to sum along an extra dimension.
This approach leaves behind the partial sums at the end of the input.
One could either start with a zero-padded input:
import numpy as np
import numpy.lib.stride_tricks
def sum_next_k_strides(arr, k):
n = arr.size
result = np.zeros(arr.size + k - 1, dtype=arr.dtype)
result[:n] = arr
window = (k,) * result.ndim
window_size = k ** result.ndim
reduced_shape = tuple(dim - k + 1 for dim, k in zip(result.shape, window))
view = np.lib.stride_tricks.as_strided(
result, shape=reduced_shape + window, strides=arr.strides * 2, writeable=False)
result = np.sum(view, axis=-1)
return result
or, more memory efficiently, construct the tail afterwards with np.cumsum():
import numpy as np
import numpy.lib.stride_tricks
def sum_next_k_strides_cs(arr, k):
n = arr.size
window = (k,) * arr.ndim
window_size = k ** arr.ndim
reduced_shape = tuple(dim - k + 1 for dim, k in zip(arr.shape, window))
view = np.lib.stride_tricks.as_strided(
arr, shape=reduced_shape + window, strides=arr.strides * 2, writeable=False)
result = np.empty_like(arr)
result[:n - k + 1] = np.sum(view, axis=-1)
result[n - k:] = np.cumsum(arr[-1:-(k + 1):-1])[::-1]
return result
Note that looping through the input size instead of k is not going to be fast, no matter the inputs, because k is limited by the size of the input.
Alternatively, one could use np.convolve(), which computes exactly what you are after but with both tails, so that you just need to slice out the starting tail:
def sum_next_k_conv(arr, k):
return np.convolve(arr, (1,) * k)[(k - 1):]
Finally, one could write a fully explicit looping solution accelerated with Numba:
import numpy as np
import numba as nb
#nb.njit
def running_sum_nb(arr, k):
n = arr.size
m = n - k + 1
o = k - 1
result = np.zeros(n, dtype=arr.dtype)
# : fill bulk
for j in range(m):
tot = arr[j]
for i in range(1, k):
tot += arr[j + i]
result[0 + j] = tot
# : fill tail
for j in range(o):
tot = 0
for i in range(j, o):
tot += arr[m + i]
result[m + j] = tot
return result
To check that all the solutions give the same result as the expected output:
funcs = running_sum_loop, running_sum_strides, running_sum_strides_cs, running_sum_conv, running_sum_nb
load = np.array([10, 12, 9, 13, 17, 23, 25, 28, 26, 24, 22, 20, 18, 20, 22, 24, 26, 28, 23, 24, 21, 18, 16, 13])
tgt = np.array([31, 34, 39, 53, 65, 76, 79, 78, 72, 66, 60, 58, 60, 66, 72, 78, 77, 75, 68, 63, 55, 47, 29, 13])
print(f"{'Input':>24} {load}")
print(f"{'Target':>24} {tgt}")
for i, func in enumerate(funcs, 1):
print(f"{func.__name__:>24} {func(load, 3)}")
Input [10 12 9 13 17 23 25 28 26 24 22 20 18 20 22 24 26 28 23 24 21 18 16 13]
Target [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
running_sum_loop [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
running_sum_strides_cs [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
running_sum_strides [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
running_sum_conv [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
running_sum_nb [31 34 39 53 65 76 79 78 72 66 60 58 60 66 72 78 77 75 68 63 55 47 29 13]
Benchmarking all these for varying input size:
import pandas as pd
timeds_n = {}
for p in range(6):
n = 10 ** p
k = 3
arr = np.array(load.tolist() * n)
print(f"N = {n * len(load)}")
base = funcs[0](arr, k)
timeds_n[n] = []
for func in funcs:
res = func(arr, k)
timed = %timeit -r 8 -n 8 -q -o func(arr, k)
timeds_n[n].append(timed.best)
print(f"{func.__name__:>24} {np.allclose(base, res)} {timed.best:.9f}")
pd.DataFrame(data=timeds_n, index=[func.__name__ for func in funcs]).transpose().plot()
and varying k:
timeds_k = {}
for p in range(1, 10):
n = 10 ** 5
k = 2 ** p
arr = np.array(load.tolist() * n)
print(f"k = {k}")
timeds_k[k] = []
base = funcs[0](arr, k)
for func in funcs:
res = func(arr, k)
timed = %timeit -q -o func(arr, k)
timeds_k[k].append(timed.best)
print(f"{func.__name__:>24} {np.allclose(base, res)} {timed.best:.9f}")
pd.DataFrame(data=timeds_k, index=[func.__name__ for func in funcs]).transpose().plot()
I have a Pandas dataframe df which is of the form:
pk id_column date_column sales_column
0 111 03/10/19 23
1 111 04/10/19 24
2 111 05/10/19 25
3 111 06/10/19 26
4 112 07/10/19 27
5 112 08/10/19 28
6 112 09/10/19 29
7 112 10/10/19 30
8 113 11/10/19 31
9 113 12/10/19 32
10 113 13/10/19 33
11 113 14/10/19 34
12 114 15/10/19 35
13 114 16/10/19 36
14 114 17/10/19 37
15 114 18/10/19 38
How do I get a new dictionary which contains data from id_column and sales_column as its value like below in the order of date_column.
{
111: [23, 24, 25, 26],
112: [27, 28, 29, 30],
113: ...,
114: ...
}
First create Series of lists in groupby with list and then convert to dictionary by Series.to_dict:
If need sorting by id_column and date_column first convert values to datetimes and then use DataFrame.sort_values:
df['date_column'] = pd.to_datetime(df['date_column'], dayfirst=True)
df = df.sort_values(['id_column','date_column'])
d = df.groupby('id_column')['sales_column'].apply(list).to_dict()
print (d)
{111: [23, 24, 25, 26], 112: [27, 28, 29, 30], 113: [31, 32, 33, 34], 114: [35, 36, 37, 38]}
I'm trying to return a list of list of vertical, horizontal and diagonal nearest neighbors of every item of a 2D numpy array
import numpy as np
import copy
tilemap = np.arange(99).reshape(11, 9)
print(tilemap)
def get_neighbor(pos, array):
x = copy.deepcopy(pos[0])
y = copy.deepcopy(pos[1])
grid = copy.deepcopy(array)
split = []
split.append([grid[y-1][x-1]])
split.append([grid[y-1][x]])
split.append([grid[y-1][x+1]])
split.append([grid[y][x - 1]])
split.append([grid[y][x+1]])
split.append([grid[y+1][x-1]])
split.append([grid[y+1][x]])
split.append([grid[y+1][x+1]])
print("\n Neighbors of ITEM[{}]\n {}".format(grid[y][x],split))
cordinates = [5, 6]
get_neighbor(pos=cordinates, array=tilemap)
i would want a list like this:
first item = 0
[[1],[12],[13],
[1,2], [12,24],[13,26],
[1,2,3], [12,24,36], [13,26,39]....
till it get to the boundaries completely then proceeds to second item = 1
and keeps adding to the list. if there is a neighbor above it should be add too..
MY RESULT
[[ 0 1 2 3 4 5 6 7 8]
[ 9 10 11 12 13 14 15 16 17]
[18 19 20 21 22 23 24 25 26]
[27 28 29 30 31 32 33 34 35]
[36 37 38 39 40 41 42 43 44]
[45 46 47 48 49 50 51 52 53]
[54 55 56 57 58 59 60 61 62]
[63 64 65 66 67 68 69 70 71]
[72 73 74 75 76 77 78 79 80]
[81 82 83 84 85 86 87 88 89]
[90 91 92 93 94 95 96 97 98]]
Neighbors of ITEM[59]
[[49], [50], [51], [58], [60], [67], [68], [69]]
Alright, what about a using a function like this? This takes the array, your target index, and the "radius" of the elements to be included.
def get_idx_adj(arr, idx, radius):
num_rows, num_cols = arr.shape
idx_row, idx_col = idx
slice_1 = np.s_[max(0, idx_row - radius):min(num_rows, idx_row + radius + 1)]
slice_2 = np.s_[max(0, idx_col - radius):min(num_cols, idx_col + radius + 1)]
return arr[slice_1, slice_2]
I'm currently trying to find the best way to transform the index of the element, so that the function can be used on its own output successively to get all the subarrays of various sizes.