I need help with making this so I can convert a string into a number in a list, I have done this but if I wanted to do it this way I would have to wright a dictionary with 100 definitions which I do not want to do. The code is just to show what I found all ready. As you can see this would take 100 definitions if I were to do it this way.
x1 = [0,0,0,0,0,0,0,0,0,0]
x2 = [0,0,0,0,0,0,0,0,0,0]
x3 = [0,0,0,0,0,0,0,0,0,0]
x4 = [0,0,0,0,0,0,0,0,0,0]
x5 = [0,0,0,0,0,0,0,0,0,0]
x6 = [0,0,0,0,0,0,0,0,0,0]
x7 = [0,0,0,0,0,0,0,0,0,0]
x8 = [0,0,0,0,0,0,0,0,0,0]
x9 = [0,0,0,0,0,0,0,0,0,0]
x10 = [0,0,0,0,0,0,0,0,0,0]
my_dict_grid = {
'x2[3]' : x2[3]
}
x = 'x2[3]'
print(my_dict_grid[x])
If you have multiple arrays you are managing all at once, create a multi-dimensional array:
x = [
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0]
]
In that case, you can just index by row then column:
x[2][3]
Based on your comment, you want to randomly change values in the array. In that case, the approach above is not at all what you want. You want to pick two random numbers, and index to them in x to change them:
import random
for _ in range(5):
updated = False
while not updated:
i = random.randrange(10)
j = random.randrange(10)
if x[i][j] == 0:
x[i][j] = 1
updated = True
Original answer to the initial question:
(this is here more as an interesting thing, not as a viable approach)
Okay. Assuming that you have to do it the way you have described, you can generate a dictionary with all of the string keys:
my_dict_grid = {
f"x{i + 1}[{j}]": arr[j]
for i, arr in enumerate([x1, x2, x3, x4, x5, x6, x7, x8, x9, x10])
for j in range(10)
}
However, I have to stress that this is not a good idea.
3 different ways to solve this with oneliners, depending of the output you want:
my_list = [[ 0 for _ in range(10)] for _ in range(10)]
my_dict = {"x"+str(i+1):[ 0 for _ in range(10)] for i in range(10)}
my_dict2 = {"x"+str((i+1)%10)+"["+str(int((i+1)/10))+"]": 0 for i in range(100)}
print(my_list) #[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0,...
print(my_dict) #{'x10': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'x9': [0,...
print(my_dict2)#{'x4[3]': 0, 'x1[9]': 0, 'x6[6]': 0, 'x2[8]': 0,...
A more mathematical, but very practical way to do this with numpy:
import numpy as np
grid_shape = [10, 10] # define 10 x 10 grid
num_ones = 5
cells = np.zeros(grid_shape[0]*grid_shape[1]) # define 10*10 = 100 cells as flat array
cells[0:num_ones] = 1 # Set the first 5 entries to 1
np.random.shuffle(cells) # Shuffle the entries, such that the 1's are at random position
grid = cells.reshape(grid_shape) # shape the grid into the desired shape
Running the code above and will e.g. result in grid=
[[0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]
[0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]]
Note that, by changing grid_shape you can resize your grid, and by changing num_ones, you will adapt the number of ones in your grid. Also, it is guaranteed that there will always be num_ones ones in your grid (given that num_ones is smaller or equal the number of elements in the grid).
Related
I have coordinates of some points that I need to plot and then convert plot to black & white bitmap:
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from PIL import Image
plt.scatter(x,y)
plt.tight_layout()
fig1 = plt.gcf()
plt.show()
type(fig1)
matplotlib.figure.Figure
How to get from this figure black & white bitmap as numpy array similar to this one:
side = 5
image = np.random.choice([0, 1], size=side*side, p=[.1, .9])
image = image.reshape(side,side)
image = np.expand_dims(image, axis=-1)
print("image.shape: ",image.shape)
plt.imshow(image, cmap=plt.get_cmap('gray'))
image.shape: (5, 5, 1)
print(image.reshape(side,side))
[[1 1 1 0 1]
[1 1 1 1 1]
[1 0 1 1 0]
[1 1 1 1 0]
[1 1 1 1 1]]
Update 1
I need also to get resulting bitmap as a numpy array. How to get it?
In case I use solution given by Zephyr:
fig, ax = plt.subplots(figsize = (5,5))
ax.hist2d(x, y, cmap = 'Greys', cmin = 0, cmax = 1)
plt.show()
I get image different from the scatter plot. And they should be similar:
You can create a grid and use it to define a map were the closest points will be white. My try with random data, range 0 to 1:
import matplotlib.pyplot as plt
import numpy as np
n_points = 10
# create random coordinates
x, y = np.random.rand(n_points,2).T
fig, ax = plt.subplots()
ax.scatter(x,y)
ax.set_xlim([0,1])
ax.set_ylim([0,1])
ax.set_aspect(1.0)
# create a grid
grid_points = 10
grid_x = np.linspace(0,1,grid_points)
grid_y = grid_x.copy()
# initiate array of ones (white)
image = np.ones([grid_points, grid_points])
for xp, yp in zip(x,y):
# selecing the closest point in grid
index_x = np.argmin(np.abs(xp - grid_x))
index_y = np.argmin(np.abs(yp - grid_y))
# setting to black
image[index_x,index_y] = 0
# you need to transpose it so x is represented
# by the columns and y by the rows
fig, ax = plt.subplots()
ax.imshow(
image.T,
origin='lower',
cmap=plt.get_cmap('gray'))
Note that the closest may not be always good. It gets better with a more refined grid.
First of all, I generate random N points within range (x_min, x_max) and (y_min, y_max):
np.random.seed(42)
N = 10
x_min = 0
x_max = 40
y_min = -20
y_max = 20
x = np.random.uniform(x_min, x_max, N)
y = np.random.uniform(y_min, y_max, N)
Then I prepare:
a grid (the bitmap) of (size, size) dimension
two vectors x_grid and y_grid which resample (x_min, x_max) and (y_min, y_max) in size + 1 points, so size inverval: one interval for each grid cell
size = 10
grid = np.zeros((size, size))
x_grid = np.linspace(x_min, x_max, size + 1)
y_grid = np.linspace(y_min, y_max, size + 1)
Then I loop over each grid cells; in each iteration I check if there is at least 1 point of (x, y) which stay within limits of that cell. If so, I set the correspondent value of grid to 1:
for i in range(size):
for j in range(size):
for x_i, y_i in zip(x, y):
if (x_grid[i] < x_i <= x_grid[i + 1]) and (y_grid[j] < y_i <= y_grid[j + 1]):
grid[i, j] = 1
break
Resulting numpy matrix:
[[0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]
[0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]
[0. 0. 1. 0. 0. 0. 0. 0. 1. 0.]
[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]]
Complete Code
import matplotlib.pyplot as plt
import numpy as np
np.random.seed(42)
N = 10
x_min = 0
x_max = 40
y_min = -20
y_max = 20
x = np.random.uniform(x_min, x_max, N)
y = np.random.uniform(y_min, y_max, N)
size = 10
grid = np.zeros((size, size))
x_grid = np.linspace(x_min, x_max, size + 1)
y_grid = np.linspace(y_min, y_max, size + 1)
for i in range(size):
for j in range(size):
for x_i, y_i in zip(x, y):
if (x_grid[i] < x_i <= x_grid[i + 1]) and (y_grid[j] < y_i <= y_grid[j + 1]):
grid[i, j] = 1
break
fig, ax = plt.subplots(1, 2, figsize = (10, 5))
ax[0].scatter(x, y)
ax[0].set_xlim(x_min, x_max)
ax[0].set_ylim(y_min, y_max)
ax[0].grid()
ax[0].set_xticks(x_grid)
ax[0].set_yticks(y_grid)
ax[1].imshow(grid.T, cmap = 'Greys', extent = (x_min, x_max, y_min, y_max))
ax[1].invert_yaxis()
plt.show()
NOTE
Pay attention to the fact that in ax.imshow you need to transpose the matrix (grid.T) and then invert y axis in order to be able to compare the ax.imshow with ax.scatter.
If you want grid matrix to match ax.imshow, then you need to rotate it counterclockwise by 90°:
grid = np.rot90(grid, k=1, axes=(0, 1))
Rotated grid, which correspond to the above plot:
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]
[0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]
[0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]
[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 1. 0. 1. 0. 0.]
[0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]]
I'm using numpy's ndarray, and I'm wondering is there a way that allows me to insert multiple elements to different locations all at once?
For example, I have an image, and I want to pad the image with 0s. This is what I currently have:
def zero_padding(self):
padded = self.copy()
padded.img = np.insert(self.img, 0, 0, axis = 0)
padded.img = np.insert(padded.img, padded.img.shape[0], 0, axis = 0)
padded.img = np.insert(padded.img, 0, 0, axis = 1)
padded.img = np.insert(padded.img, padded.img.shape[1], 0, axis = 1)
return padded
where padded is an instance of the image.
Sure, you can use the fancy indexing techinque of NumPy as follows:
import numpy as np
if __name__=='__main__':
A = np.zeros((5, 5))
A[[1, 2], [0, 3]] = 1
print(A)
Output:
[[0. 0. 0. 0. 0.]
[1. 0. 0. 0. 0.]
[0. 0. 0. 1. 0.]
[0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0.]]
Cheers
I am new in numpy, and I am having troubles with simple managment of numpy arrays.
I am doing a task in which it said that loops has to be avoid as much as possible, and I need to edit the values of an array through another array of indexes.
indexes # [3, 16]
y # [0. 1. 1. 1. 0. 1. 0. 0. 0. 0. 1. 1. 1. 0. 1. 0. 0. 0. 1. 1.]
y[indexes] = 2 # [0. 1. 1. 2. 0. 1. 0. 0. 0. 0. 1. 1. 1. 0. 1. 0. 2. 0. 1. 1.]
But I don't need change the value simply by 2. I need make a conditional change. This what I have got, but I would need something like
y[indexes] = 0 if y[indexes] == 1 else 0
>>> [0. 1. 1. 0. 0. 1. 0. 0. 0. 0. 1. 1. 1. 0. 1. 0. 1. 0. 1. 1.]
And the line above should be the results.
This is the loop way answer, but I need a numpy way if exists:
for index in indexes:
y[index] = 1 if y[index] == 0 else 0
Thanks in advance.
I don't know if I understood your question. But I hope this helps you.
tip 01
import numpy as np
indexes = [1, 5, 7] # index list
y = np.array([9,10,11,12,13,14,15,16,17,18,19,20,21,22,23]) #array example
y[indexes][2] #3rd(0,1,>>2<<) item of y array (1,5,>>7<<).
In this case it is y[7] equal 16.
tip 02
This can also be useful.
y = np.array([0,1,1,0,3,0,1,0,1,0])
y
array([0, 1, 1, 0, 3, 0, 1, 0, 1, 0])
np.where(y != 1, y, 0)
y
array([0, 0, 0, 0, 3, 0, 0, 0, 0, 0])
I have a training data like this:
x_train = np.random.randint(100, size=(1000, 25))
where each row is a sample and thus we have 1000 samples.
Now I need to have the training data such that for each of the sample/row there can be at max 3 non-zero elements out of 25.
Can you all please suggest how I can implement that? Thanks!
I am assuming that you want to turn a majority of your data into zeros, except that 0 to 3 non-zero elements are retained (randomly) for each row. If this is the case, a possible way to do this is as follows.
Code
import numpy as np
max_ = 3
nrows = 1000
ncols = 25
np.random.seed(7)
X = np.zeros((nrows,ncols))
data = np.random.randint(100, size=(nrows, ncols))
# number of max non-zeros to be generated for each column
vmax = np.random.randint(low=0, high=4, size=(nrows,))
for i in range(nrows):
if vmax[i]>0:
#index for setting non-zeros
col = np.random.randint(low=0, high=ncols, size=(1,vmax[i]))
#set non-zeros elements
X[i][col] = data[i][col]
print(X)
Output
[[ 0. 68. 25. ... 0. 0. 0.]
[ 0. 0. 0. ... 0. 0. 0.]
[ 0. 0. 0. ... 0. 0. 0.]
...
[ 0. 0. 0. ... 0. 0. 0.]
[88. 0. 0. ... 0. 0. 0.]
[ 0. 0. 0. ... 0. 0. 0.]]
I want to perform a specific operation. Namely, from a matrix:
A = np.array([[1,2],
[3,4]])
To the following
B = np.array([[1, 0, 0, 2, 0, 0],
[0, 1, 0, 0, 2, 0],
[0, 0, 1, 0, 0, 2],
[3, 0, 0, 4, 0, 0],
[0, 3, 0, 0, 4, 0],
[0, 0, 3, 0, 0, 4]])
Or in words: multiply every entry by the identity matrix and keep the same order.
Now I have accomplished this by using numpy, using the following code. Here N and M are the dimensions of the starting matrix, and the dimension of the identity matrix.
l_slice = 3
n_slice = 2
A = np.reshape(np.arange(1, 1+N ** 2), (N, N))
B = np.array([i * np.eye(M) for i in A.flatten()])
C = B.reshape(N, N, M, M).reshape(N, N * M, M).transpose([0, 2, 1]).reshape((N * M, N * M))
where C has my desired properties.
But now I want do this modification in Keras/Tensorflow, where the matrix A is the outcome of one of my layers.
However, I am not sure yet if I will be able to properly create matrix B. Especially when batches are involved, I think I will somehow mess up the dimensions of my problem.
Can anyone with more Keras/Tensorflow experience comment on this 'reshape' and how he/she sees this happening within Keras/Tensorflow?
Here is a way to do that with TensorFlow:
import tensorflow as tf
data = tf.placeholder(tf.float32, [None, None])
n = tf.placeholder(tf.int32, [])
eye = tf.eye(n)
mult = data[:, tf.newaxis, :, tf.newaxis] * eye[tf.newaxis, :, tf.newaxis, :]
result = tf.reshape(mult, n * tf.shape(data))
with tf.Session() as sess:
a = sess.run(result, feed_dict={data: [[1, 2], [3, 4]], n: 3})
print(a)
Output:
[[1. 0. 0. 2. 0. 0.]
[0. 1. 0. 0. 2. 0.]
[0. 0. 1. 0. 0. 2.]
[3. 0. 0. 4. 0. 0.]
[0. 3. 0. 0. 4. 0.]
[0. 0. 3. 0. 0. 4.]]
By the way, you can do basically the same in NumPy, which should be faster than your current solution:
import numpy as np
data = np.array([[1, 2], [3, 4]])
n = 3
eye = np.eye(n)
mult = data[:, np.newaxis, :, np.newaxis] * eye[np.newaxis, :, np.newaxis, :]
result = np.reshape(mult, (n * data.shape[0], n * data.shape[1]))
print(result)
# The output is the same as above
EDIT:
I'll try to give some intuition about why/how this works, sorry if it's too long. It is not that hard but I think it's sort of tricky to explain. Maybe it is easier to see how the following multiplication works
import numpy as np
data = np.array([[1, 2], [3, 4]])
n = 3
eye = np.eye(n)
mult1 = data[:, :, np.newaxis, np.newaxis] * eye[np.newaxis, np.newaxis, :, :]
Now, mult1 is a sort of "matrix of matrices". If I give two indices, I will get the diagonal matrix for the corresponding element in the original one:
print(mult1[0, 0])
# [[1. 0. 0.]
# [0. 1. 0.]
# [0. 0. 1.]]
So you could say this matrix could be visualize like this:
| 1 0 0 | | 2 0 0 |
| 0 1 0 | | 0 2 0 |
| 0 0 1 | | 0 0 2 |
| 3 0 0 | | 4 0 0 |
| 0 3 0 | | 0 4 0 |
| 0 0 3 | | 0 0 4 |
However this is deceiving, because if you try to reshape this to the final shape the result is not the right one:
print(np.reshape(mult1, (n * data.shape[0], n * data.shape[1])))
# [[1. 0. 0. 0. 1. 0.]
# [0. 0. 1. 2. 0. 0.]
# [0. 2. 0. 0. 0. 2.]
# [3. 0. 0. 0. 3. 0.]
# [0. 0. 3. 4. 0. 0.]
# [0. 4. 0. 0. 0. 4.]]
The reason is that reshaping (conceptually) "flattens" the array first and then gives the new shape. But the flattened array in this case is not what you need:
print(mult1.ravel())
# [1. 0. 0. 0. 1. 0. 0. 0. 1. 2. 0. 0. 0. 2. 0. ...
You see, it first traverses the first submatrix, then the second, etc. What you want though is for it to traverse first the first row of the first submatrix, then the first row of the second submatrix, then second row of first submatrix, etc. So basically you want something like:
Take the first two submatrices (the ones with 1 and 2)
Take all the first rows ([1, 0, 0] and [2, 0, 0]).
Take the first of these ([1, 0, 0])
Take each of its elements (1, 0 and 0).
And then continue for the rest. So if you think about it, we traversing first the axis 0 (row of "matrix of matrices"), then 2 (rows of each submatrix), then 1 (column of "matrix of matrices") and finally 3 (columns of submatrices). So we can just reorder the axis to do that:
mult2 = mult1.transpose((0, 2, 1, 3))
print(np.reshape(mult2, (n * data.shape[0], n * data.shape[1])))
# [[1. 0. 0. 2. 0. 0.]
# [0. 1. 0. 0. 2. 0.]
# [0. 0. 1. 0. 0. 2.]
# [3. 0. 0. 4. 0. 0.]
# [0. 3. 0. 0. 4. 0.]
# [0. 0. 3. 0. 0. 4.]]
And it works! So in the solution I posted, to avoid the tranposing, I just make the multiplication so the order of the axes is exactly that:
mult = data[
:, # Matrix-of-matrices rows
np.newaxis, # Submatrix rows
:, # Matrix-of-matrices columns
np.newaxis # Submatrix columns
] * eye[
np.newaxis, # Matrix-of-matrices rows
:, # Submatrix rows
np.newaxis, # Matrix-of-matrices columns
: # Submatrix columns
]
I hope that makes it slightly clearer. To be honest, in this case in particular I could came up with the solution quickly because I had to solve a similar problem not too long ago, and I guess you end up building an intuition of these things.
Another way to achieve the same effect in numpy is to use the following:
A = np.array([[1,2],
[3,4]])
B = np.repeat(np.repeat(A, 3, axis=0), 3, axis=1) * np.tile(np.eye(3), (2,2))
Then, to replicate it in tensorflow, we can use tf.tile, but there is no tf.repeat, however someone has provided this function on tensorflow tracker.
def tf_repeat(tensor, repeats):
"""
Args:
input: A Tensor. 1-D or higher.
repeats: A list. Number of repeat for each dimension, length must be the same as the number of dimensions in input
Returns:
A Tensor. Has the same type as input. Has the shape of tensor.shape * repeats
"""
with tf.variable_scope("repeat"):
expanded_tensor = tf.expand_dims(tensor, -1)
multiples = [1] + list(repeats)
tiled_tensor = tf.tile(expanded_tensor, multiples=multiples)
repeated_tesnor = tf.reshape(tiled_tensor, tf.shape(tensor) * repeats)
return repeated_tesnor
and thus the tensorflow implementation will look like the following. Here I also consider that the first dimension represents batches, and thus we do not operate on it.
N = 2
M = 3
nbatch = 2
Ain = np.reshape(np.arange(1, 1 + N*N*nbatch), (nbatch, N, N))
A = tf.placeholder(tf.float32, shape=(nbatch, N, N))
B = tf.tile(tf.eye(M), [N, N]) * tf_repeat(A, [1, M, M])
with tf.Session() as sess:
print(sess.run(C, feed_dict={A: Ain}))
and the result:
[[[1. 0. 0. 2. 0. 0.]
[0. 1. 0. 0. 2. 0.]
[0. 0. 1. 0. 0. 2.]
[3. 0. 0. 4. 0. 0.]
[0. 3. 0. 0. 4. 0.]
[0. 0. 3. 0. 0. 4.]]
[[5. 0. 0. 6. 0. 0.]
[0. 5. 0. 0. 6. 0.]
[0. 0. 5. 0. 0. 6.]
[7. 0. 0. 8. 0. 0.]
[0. 7. 0. 0. 8. 0.]
[0. 0. 7. 0. 0. 8.]]]