Related
I have a numpy array of x and y coordinates and want to make it regular. The array is sorted based on its x values (first column):
import numpy as np
Irregular_points = np.array([[1.1,5.], [0.85,7.1], [0.9,9], [1.1,11], [1.,13.1],
[1.9,5.2], [2.,6.9], [1.95,9], [2.1,11.1], [2.,13.1],
[3.0,5.1], [3.1,7.0], [3.,9], [3.0,11.], [3.1,12.8]])
I want to firtly find out which points have almost the same x values: it will be the first five rows, middle five rows and last five rows. One signal for finding these points is that y value decreases when I go to the next group. Then, I want to replace the x values of each group with the average value. For example in the fisrt five rows x values are 1.1, 0.85, 0.9, 1.1 and 1. and the average is 0.98. I want to do the same for next two parts.
For y values I again want to find similar ones which fall into five groups and then replace them with average of each group. y values of the first group are 5., 5.2 and 5.1 and average is 5.1. Finally, my points should be like the following array:
Regular_points = np.array([[0.98,5.1], [0.98,7.0], [0.98,9.0], [0.98,11.03], [0.98,13.0],
[1.98,5.1], [1.98,7.0], [1.98,9.0], [1.98,11.03], [1.98,13.0],
[3.04,5.1], [3.04,7.0], [3.04,9.0], [3.04,11.03], [3.04,13.0]])
I tried to round numbers but it did not work for real cases and I need to make these averages. I very much appreciate any help. The figure clearly shows what I want. Red dots are irregular points but by replacing averages, blue dots can be resulted.
Since you're averaging rows and columns, you'll need to use a different shape. Then separate x and y coords, average them by different axis and use np.transpose + np.meshgrid for nice display:
irregular_points = np.array([[1.1,5.], [0.85,7.1], [0.9,9], [1.1,11], [1.,13.1],
[1.9,5.2], [2.,6.9], [1.95,9], [2.1,11.1], [2.,13.1],
[3.0,5.1], [3.1,7.0], [3.,9], [3.0,11.], [3.1,12.8]])
points_reshape = irregular_points.reshape(3, 5, 2)
x, y = np.transpose(points_reshape)
x_mean = x.mean(axis=0)
y_mean = y.mean(axis=1)
regular_points = np.transpose(np.meshgrid(x_mean, y_mean))
regular_points
>>>
array([[[ 0.99 , 5.1 ],
[ 0.99 , 7. ],
[ 0.99 , 9. ],
[ 0.99 , 11.03333333],
[ 0.99 , 13. ]],
[[ 1.99 , 5.1 ],
[ 1.99 , 7. ],
[ 1.99 , 9. ],
[ 1.99 , 11.03333333],
[ 1.99 , 13. ]],
[[ 3.04 , 5.1 ],
[ 3.04 , 7. ],
[ 3.04 , 9. ],
[ 3.04 , 11.03333333],
[ 3.04 , 13. ]]])
You could use a cluster algorithm like KMeans:
import numpy as np
from sklearn.cluster import KMeans
irregular_points = np.array([[1.1,5.], [0.85,7.1], [0.9,9], [1.1,11], [1.,13.1],
[1.9,5.2], [2.,6.9], [1.95,9], [2.1,11.1], [2.,13.1],
[3.0,5.1], [3.1,7.0], [3.,9], [3.0,11.], [3.1,12.8]])
kmeans_x = KMeans(n_clusters=3).fit(irregular_points[:, 0, np.newaxis])
kmeans_y = KMeans(n_clusters=5).fit(irregular_points[:, 1, np.newaxis])
clusters_x = kmeans_x.predict(irregular_points[:, 0, np.newaxis])
clusters_y = kmeans_y.predict(irregular_points[:, 1, np.newaxis])
regular_points_x = kmeans_x.cluster_centers_[clusters_x]
regular_points_y = kmeans_y.cluster_centers_[clusters_y]
regular_points = np.asarray([[regular_points_x[i], regular_points_y[i]] for i in range(irregular_points.shape[0])])
>>>
array([[[ 0.99 , 5.1 ],
[ 0.99 , 7. ],
[ 0.99 , 9. ],
[ 0.99 , 11.03333333],
[ 0.99 , 13. ]],
[[ 1.99 , 5.1 ],
[ 1.99 , 7. ],
[ 1.99 , 9. ],
[ 1.99 , 11.03333333],
[ 1.99 , 13. ]],
[[ 3.04 , 5.1 ],
[ 3.04 , 7. ],
[ 3.04 , 9. ],
[ 3.04 , 11.03333333],
[ 3.04 , 13. ]]])
I am trying to calculate a dot product between two matrices, for each couple of rows.
I have matrix D with (u x 2) dimensions and matrix R with (u*2 x c) dimensions.
Below an example:
D = np.array([[0.02747092, 0.11233295],
[0.02747092, 0.07295284],
[0.01245856, 0.19935923],
[0.01245856, 0.13520913],
[0.11233295, 0.07295284]])
R = np.array([[-3. , 0. , 1. , -1. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0. , -2. , 2. , -4. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0.66666667, -3.33333333, 2.66666667, -4.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-3. , 0. , 1. , -1. ]])
The result should be matrix M with dimensions (u x c) as follows (example of first row):
M = np.array([[-0.2185, 0.0825, 0.2195, -0.1645],
[...]])
Which is result of dot product between the first row of D and first two rows of matrix R as such:
D_ = np.array([[0.027, 0.11]])
R_ = np.array([[-3., 0., 1., -1.],
[-1.25, 0.75, 1.75, -1.25]])
D_.dot(R_)
I tried various ways of np.tensordot after reshaping the D matrix into tensor, but without any luck. I am looking for vectorized solution and to avoid loops (which is my current solution, quite slow).
Reshape R to 3D and use np.einsum -
np.einsum('ijk,ij->ik',R.reshape(len(D),2,-1),D)
I have a set of RGB values in an array rgb_array of the form
[255.000, 56,026, 0.000]
[246.100, 60,000, 0.000]
...
>>> print(rbg_array)
1000, 3
that I'd like to plot similarly to the color gradient shown above.
How can I best use matpotlib's imshow to achieve this?
Supposing your array has N rows where each row contains 3 floats between 0 and 255, you can create an image as follows. First convert it to a numpy array of integers, and reshape it to (1, N, 3). This will make it a 1xN image. Then, display the image using imshow. You need to set an extent to get the x and y axes as in your example, or just set them to [0, 1, 0, 1]. Also the aspect ratio needs to be controlled, as otherwise the pixels would be considered "square".
import numpy as np
import matplotlib.pyplot as plt
rgb_array = [[255.000, 56.026 + (255 - 56.026) * i / 400, 255 * i / 400] for i in range(400)]
rgb_array += [[255 - 255 * i / 600, 255 - 255 * i / 600, 255] for i in range(600)]
img = np.array(rgb_array, dtype=int).reshape((1, len(rgb_array), 3))
plt.imshow(img, extent=[0, 16000, 0, 1], aspect='auto')
plt.show()
Don't use this method - #JohanC provides a much superior solution of creating an image rather than making a bar-graph.
I'm not so good on Matplotlib, but came up with this. There may be more efficient methods, so someone correct me please if this is the wrong approach.
#!/usr/bin/env python3
import numpy as np
import matplotlib.pyplot as plt
NSAMPLES = 100
# Synthesize R, G, B and A channels with dummy data
# The thing to note is that the samples are REAL and in range [0..1]
r = np.linspace(0,1,NSAMPLES).astype(np.float)
g = 1.0 - r
b = np.full(NSAMPLES,0.5,np.float)
a = np.full(NSAMPLES,1,np.float)
# Merge into a single array, 4 deep
RGBA = np.dstack((r,g,b,a))
# Plot
height, width = 40, 1
plt.bar(np.arange(NSAMPLES), height, width, color=rgba.reshape(-1,4))
plt.title("Some Funky Barplot")
plt.show()
The array RGBA looks like this:
array([[[0. , 1. , 0.5 , 1. ],
[0.01010101, 0.98989899, 0.5 , 1. ],
[0.02020202, 0.97979798, 0.5 , 1. ],
[0.03030303, 0.96969697, 0.5 , 1. ],
[0.04040404, 0.95959596, 0.5 , 1. ],
[0.05050505, 0.94949495, 0.5 , 1. ],
[0.06060606, 0.93939394, 0.5 , 1. ],
[0.07070707, 0.92929293, 0.5 , 1. ],
[0.08080808, 0.91919192, 0.5 , 1. ],
[0.09090909, 0.90909091, 0.5 , 1. ],
[0.1010101 , 0.8989899 , 0.5 , 1. ],
[0.11111111, 0.88888889, 0.5 , 1. ],
[0.12121212, 0.87878788, 0.5 , 1. ],
[0.13131313, 0.86868687, 0.5 , 1. ],
[0.14141414, 0.85858586, 0.5 , 1. ],
[0.15151515, 0.84848485, 0.5 , 1. ],
[0.16161616, 0.83838384, 0.5 , 1. ],
[0.17171717, 0.82828283, 0.5 , 1. ],
[0.18181818, 0.81818182, 0.5 , 1. ],
[0.19191919, 0.80808081, 0.5 , 1. ],
[0.2020202 , 0.7979798 , 0.5 , 1. ],
[0.21212121, 0.78787879, 0.5 , 1. ],
[0.22222222, 0.77777778, 0.5 , 1. ],
[0.23232323, 0.76767677, 0.5 , 1. ],
[0.24242424, 0.75757576, 0.5 , 1. ],
[0.25252525, 0.74747475, 0.5 , 1. ],
[0.26262626, 0.73737374, 0.5 , 1. ],
[0.27272727, 0.72727273, 0.5 , 1. ],
[0.28282828, 0.71717172, 0.5 , 1. ],
[0.29292929, 0.70707071, 0.5 , 1. ],
[0.3030303 , 0.6969697 , 0.5 , 1. ],
[0.31313131, 0.68686869, 0.5 , 1. ],
[0.32323232, 0.67676768, 0.5 , 1. ],
[0.33333333, 0.66666667, 0.5 , 1. ],
[0.34343434, 0.65656566, 0.5 , 1. ],
[0.35353535, 0.64646465, 0.5 , 1. ],
[0.36363636, 0.63636364, 0.5 , 1. ],
[0.37373737, 0.62626263, 0.5 , 1. ],
[0.38383838, 0.61616162, 0.5 , 1. ],
[0.39393939, 0.60606061, 0.5 , 1. ],
[0.4040404 , 0.5959596 , 0.5 , 1. ],
[0.41414141, 0.58585859, 0.5 , 1. ],
[0.42424242, 0.57575758, 0.5 , 1. ],
[0.43434343, 0.56565657, 0.5 , 1. ],
[0.44444444, 0.55555556, 0.5 , 1. ],
[0.45454545, 0.54545455, 0.5 , 1. ],
[0.46464646, 0.53535354, 0.5 , 1. ],
[0.47474747, 0.52525253, 0.5 , 1. ],
[0.48484848, 0.51515152, 0.5 , 1. ],
[0.49494949, 0.50505051, 0.5 , 1. ],
[0.50505051, 0.49494949, 0.5 , 1. ],
[0.51515152, 0.48484848, 0.5 , 1. ],
[0.52525253, 0.47474747, 0.5 , 1. ],
[0.53535354, 0.46464646, 0.5 , 1. ],
[0.54545455, 0.45454545, 0.5 , 1. ],
[0.55555556, 0.44444444, 0.5 , 1. ],
[0.56565657, 0.43434343, 0.5 , 1. ],
[0.57575758, 0.42424242, 0.5 , 1. ],
[0.58585859, 0.41414141, 0.5 , 1. ],
[0.5959596 , 0.4040404 , 0.5 , 1. ],
[0.60606061, 0.39393939, 0.5 , 1. ],
[0.61616162, 0.38383838, 0.5 , 1. ],
[0.62626263, 0.37373737, 0.5 , 1. ],
[0.63636364, 0.36363636, 0.5 , 1. ],
[0.64646465, 0.35353535, 0.5 , 1. ],
[0.65656566, 0.34343434, 0.5 , 1. ],
[0.66666667, 0.33333333, 0.5 , 1. ],
[0.67676768, 0.32323232, 0.5 , 1. ],
[0.68686869, 0.31313131, 0.5 , 1. ],
[0.6969697 , 0.3030303 , 0.5 , 1. ],
[0.70707071, 0.29292929, 0.5 , 1. ],
[0.71717172, 0.28282828, 0.5 , 1. ],
[0.72727273, 0.27272727, 0.5 , 1. ],
[0.73737374, 0.26262626, 0.5 , 1. ],
[0.74747475, 0.25252525, 0.5 , 1. ],
[0.75757576, 0.24242424, 0.5 , 1. ],
[0.76767677, 0.23232323, 0.5 , 1. ],
[0.77777778, 0.22222222, 0.5 , 1. ],
[0.78787879, 0.21212121, 0.5 , 1. ],
[0.7979798 , 0.2020202 , 0.5 , 1. ],
[0.80808081, 0.19191919, 0.5 , 1. ],
[0.81818182, 0.18181818, 0.5 , 1. ],
[0.82828283, 0.17171717, 0.5 , 1. ],
[0.83838384, 0.16161616, 0.5 , 1. ],
[0.84848485, 0.15151515, 0.5 , 1. ],
[0.85858586, 0.14141414, 0.5 , 1. ],
[0.86868687, 0.13131313, 0.5 , 1. ],
[0.87878788, 0.12121212, 0.5 , 1. ],
[0.88888889, 0.11111111, 0.5 , 1. ],
[0.8989899 , 0.1010101 , 0.5 , 1. ],
[0.90909091, 0.09090909, 0.5 , 1. ],
[0.91919192, 0.08080808, 0.5 , 1. ],
[0.92929293, 0.07070707, 0.5 , 1. ],
[0.93939394, 0.06060606, 0.5 , 1. ],
[0.94949495, 0.05050505, 0.5 , 1. ],
[0.95959596, 0.04040404, 0.5 , 1. ],
[0.96969697, 0.03030303, 0.5 , 1. ],
[0.97979798, 0.02020202, 0.5 , 1. ],
[0.98989899, 0.01010101, 0.5 , 1. ],
[1. , 0. , 0.5 , 1. ]]])
I have a bunch of points and need to select a subset of them, add a value to the x coordinates and store the information in the original points.
I need to do it without loops or intermediate assignments.
import numpy as np
points=np.array([[100. , 100. , 100. ],
[ 0. , -2.75, 0. ],
[ 0. , -2.75, 5. ],
[ 0. , -1.9 , 3.15],
[ 0. , -1.9 , 3.35]])
then trying:
points[[3,4,0]][:,[0]]+=2
or
points[[3,4,0]][:,[0]]=points[[3,4,0]][:,[0]]+2
the original points variable does not change.
Any ideas? I suspect I am missing some stupid stuff...
If you are looking to edit first column of those rows use:
points[[3,4,0], 0] += 2
points
#[[ 102. 100. 100. ]
# [ 0. -2.75 0. ]
# [ 0. -2.75 5. ]
# [ 2. -1.9 3.15]
# [ 2. -1.9 3.35]]
Currently I have an array as follows:
myArray = np.array(
[[ 976.77 , 152.95 , 105.62 , 53.44 , 0 ],
[ 987.61 , 156.63 , 105.53 , 51.1 , 0 ],
[1003.74 , 151.31 , 104.435, 52.86 , 0 ],
[ 968. , 153.41 , 106.24 , 58.98 , 0 ],
[ 978.66 , 152.19 , 103.28 , 57.97 , 0 ],
[1001.9 , 152.88 , 105.08 , 58.01 , 0 ],
[1024.93 , 146.59 , 107.06 , 59.94 , 0 ],
[1020.01 , 148.05 , 109.96 , 58.67 , 0 ],
[1034.01 , 152.69 , 107.64 , 59.74 , 0 ],
[ 0. , 154.88 , 102. , 58.96 , 0 ],
[ 0. , 147.46 , 100.69 , 54.95 , 0 ],
[ 0. , 149.7 , 102.439, 53.91 , 0 ]]
)
I would like the fill in the zeros in the first column with the previous last value (1034.01) however if the 0's start from index 0, for it to remain as 0.
Example of end result:
myArrayEnd = np.array(
[[ 976.77 , 152.95 , 105.62 , 53.44 , 0 ],
[ 987.61 , 156.63 , 105.53 , 51.1 , 0 ],
[1003.74 , 151.31 , 104.435, 52.86 , 0 ],
[ 968. , 153.41 , 106.24 , 58.98 , 0 ],
[ 978.66 , 152.19 , 103.28 , 57.97 , 0 ],
[1001.9 , 152.88 , 105.08 , 58.01 , 0 ],
[1024.93 , 146.59 , 107.06 , 59.94 , 0 ],
[1020.01 , 148.05 , 109.96 , 58.67 , 0 ],
[1034.01 , 152.69 , 107.64 , 59.74 , 0 ],
[1034.01 , 154.88 , 102. , 58.96 , 0 ],
[1034.01 , 147.46 , 100.69 , 54.95 , 0 ],
[1034.01 , 149.7 , 102.439, 53.91 , 0 ]]
)
I would like the code to be applicable to any array not just this one, where the situation may be different. (Column 3 might be all 0's and Column 4 might have 0's in the middle which should be filled with the last previous value).
Here's a vectorised way with pandas. This is also possible with numpy. In any case, you should not need explicit loops for this task.
import pandas as pd
import numpy as np
df = pd.DataFrame(myArray)\
.replace(0, np.nan)\
.ffill().fillna(0)
res = df.values
print(res)
[[ 976.77 152.95 105.62 53.44 0. ]
[ 987.61 156.63 105.53 51.1 0. ]
[ 1003.74 151.31 104.435 52.86 0. ]
[ 968. 153.41 106.24 58.98 0. ]
[ 978.66 152.19 103.28 57.97 0. ]
[ 1001.9 152.88 105.08 58.01 0. ]
[ 1024.93 146.59 107.06 59.94 0. ]
[ 1020.01 148.05 109.96 58.67 0. ]
[ 1034.01 152.69 107.64 59.74 0. ]
[ 1034.01 154.88 102. 58.96 0. ]
[ 1034.01 147.46 100.69 54.95 0. ]
[ 1034.01 149.7 102.439 53.91 0. ]]
Staying within numpy:
for k, c in enumerate(myArray.T):
idx = np.flatnonzero(c == 0)
if idx.size > 0 and idx[0] > 0:
myArray[idx, k] = myArray[idx[0] - 1, k]
Assuming I've understood you correctly, this should do the trick:
def fill_zeroes(array):
temp_array = array
for i in xrange(1, len(temp_array)):
if temp_array[i][0] == 0:
temp_array[i][0] = temp_array[i-1][0]
return temp_array
How about something like this (in psuedo code)?
for each col in array
for each row in col
if array[col,row] == 0 && row>0
array[col,row] = array[col,row-1]
edit: Combined with #ukemi, who has a quicker solution, but does not loop over the various columns. Also, you need to make sure to not try to index array[0][-1].
The code below requires testing:
values = myArray.to_list() # don't remember if nd_array.to_list is a method or property
result = []
last = None
for i,item in enumerate(values):
if i == 0 and item[0] == 0:
last = item
elif item[0] == 0 and last is not None:
item[0] = last
else:
last = item[0]
result.append(item)