I am trying to insert a column into a 2D array.
Currently I have a 2D array generated using itertools.
sample_points=[-1.5, -.8]
base_points = itertools.combinations_with_replacement(sample_points, 3)
base_points_list=list(base_points)
base_points_array=np.asarray(base_points_list)
Then I get an array which looks like this:
>>> base_points_array
array([[-1.5, -1.5, -1.5],
[-1.5, -1.5, -0.8],
[-1.5, -0.8, -0.8],
[-0.8, -0.8, -0.8]])
I want to add a column at the beginning so that the array looks like this:
[[1 -1.5 -1.5 -1.5]
[1 -1.5 -1.5 -0.8]
[1 -1.5 -0.8 -0.8]
[1 -0.8 -0.8 -0.8]]
So I used the command:
np.insert(base_points_array,0,1,1)
Because it should be able to do that using broadcasting.
but I get something completely different. the number of rows have changes:
array([[ 1. , -1.5, -1.5, -1.5, -0.8],
[ 1. , -1.5, -1.5, -0.8, -0.8],
[ 1. , -1.5, -0.8, -0.8, -0.8]])
What am I doing wrong?
Using the np.append . But if your array to insert is 1D array
insert_array= [1, 1, 1, 1]
You need to expand the dimension of your inserting array by 1 first, you can do it with
insert_array= np.expand_dims(insert_array, 1)
And then you can use the append method
base_points_array= np.append(insert_array, base_points_array, 1)
Related
I'm sorry if this question isn't framed well. So I would rather explain with an example.
I have a numpy matrix:
a = np.array([[0.5, 0.8, 0.1], [0.6, 0.9, 0.3], [0.7, 0.4, 0.8], [0.8, 0.7, 0.6]])
And another numpy array as shown:
b = np.array([1, 0, 2, 2])
With the given condition that values in b will be in the range(a.shape[1]) and that b.shape[1] == a.shape[0]. Now this is the operation I need to perform.
For every index i of a, and the corresponding index i of b, I need to subtract 1 from the index j of a[i] where j== b[i]
So in my example, a[0] == [0.5 0.8 0.1] and b[0] == 1. Therefore I need to subtract 1 from a[0][b[0]] so that a[0] = [0.5, -0.2, 0.1]. This has to be done for all rows of a. Any direct solution without me having to iterate through all rows or columns one by one?
Thanks.
Use numpy indexing. See this post for a nice introduction:
import numpy as np
a = np.array([[0.5, 0.8, 0.1], [0.6, 0.9, 0.3], [0.7, 0.4, 0.8], [0.8, 0.7, 0.6]])
b = np.array([1, 0, 2, 2])
a[np.arange(a.shape[0]), b] -= 1
print(a)
Output
[[ 0.5 -0.2 0.1]
[-0.4 0.9 0.3]
[ 0.7 0.4 -0.2]
[ 0.8 0.7 -0.4]]
As an alternative use substract.at:
np.subtract.at(a, (np.arange(a.shape[0]), b), 1)
print(a)
Output
[[ 0.5 -0.2 0.1]
[-0.4 0.9 0.3]
[ 0.7 0.4 -0.2]
[ 0.8 0.7 -0.4]]
The main idea is that:
np.arange(a.shape[0]) # shape[0] is equals to the numbers of rows
generates the indices of the rows:
[0 1 2 3]
I have the following torch tensor:
tensor([[-0.2, 0.3],
[-0.5, 0.1],
[-0.4, 0.2]])
and the following numpy array: (I can convert it to something else if necessary)
[1 0 1]
I want to get the following tensor:
tensor([0.3, -0.5, 0.2])
i.e. I want the numpy array to index each sub-element of my tensor. Preferably without using a loop.
Thanks in advance
You may want to use torch.gather - "Gathers values along an axis specified by dim."
t = torch.tensor([[-0.2, 0.3],
[-0.5, 0.1],
[-0.4, 0.2]])
idxs = np.array([1,0,1])
idxs = torch.from_numpy(idxs).long().unsqueeze(1)
# or torch.from_numpy(idxs).long().view(-1,1)
t.gather(1, idxs)
tensor([[ 0.3000],
[-0.5000],
[ 0.2000]])
Here, your index is numpy array so you have to convert it to LongTensor.
Just simply, use a range(len(index)) for the first dimension.
import torch
a = torch.tensor([[-0.2, 0.3],
[-0.5, 0.1],
[-0.4, 0.2]])
c = [1, 0, 1]
b = a[range(3),c]
print(b)
I am trying to make a program faster and I found this post and I want to implement a solution that resembles the fourth case given in that question.
Here is the relevant part of the code I am using:
count = 0
hist_dat = np.zeros(r**2)
points = np.zeros((r**2, 2))
for a in range(r):
for b in range(r):
for i in range(N):
for j in range(N):
hist_dat[count] += retval(a/r, (a+1)/r, data_a[i][j])*retval(b/r, (b+1)/r, data_b[i][j])/N
points[count][0], points[count][1] = (a+0.5)/r, (b+0.5)/r
count += 1
What this code does is generate the values of a normalized 2D histogram (with "r" divisions in each direction) and the coordinates for those values as numpy.ndarray.
As you can see in the other question linked, I am currently using the second worst possible solution and it takes several minutes to run.
For starters I want to change what the code is doing for the points array (I think that once I can see how that is done I could figure something out for hist_dat). Which is basically this:
In the particular case I am working on, both A and B are the same. So for example, it could be like going from array([0, 0.5, 1]) to array([[0,0], [0,0.5], [0,1], [0.5,0], [0.5,0.5], [0.5,1], [1,0], [1,0.5], [1,1]])
Is there any method for numpy.ndarray or an operation with the np.arange() that does what the above diagram shows without requiring for loops?
Or is there any alternative that can do this as fast as what the linked post showed for the np.arange()?
You can use np.c_ to combine the result of np.repeat and np.tile:
import numpy as np
start = 0.5
end = 5.5
step = 1.0
points = np.arange(start, end, step) # [0.5, 1.5, 2.5, 3.5, 4.5]
output = np.c_[np.repeat(points, n_elements), np.tile(points, n_elements)]
print(output)
Output:
[[0.5 0.5]
[0.5 1.5]
[0.5 2.5]
[0.5 3.5]
[0.5 4.5]
[1.5 0.5]
[1.5 1.5]
[1.5 2.5]
[1.5 3.5]
[1.5 4.5]
[2.5 0.5]
[2.5 1.5]
[2.5 2.5]
[2.5 3.5]
[2.5 4.5]
[3.5 0.5]
[3.5 1.5]
[3.5 2.5]
[3.5 3.5]
[3.5 4.5]
[4.5 0.5]
[4.5 1.5]
[4.5 2.5]
[4.5 3.5]
[4.5 4.5]]
maybe np.mgird would help?
import numpy as np
np.mgrid[0:2:.5,0:2:.5].reshape(2,4**2).T
Output:
array([[0. , 0. ],
[0. , 0.5],
[0. , 1. ],
[0. , 1.5],
[0.5, 0. ],
[0.5, 0.5],
[0.5, 1. ],
[0.5, 1.5],
[1. , 0. ],
[1. , 0.5],
[1. , 1. ],
[1. , 1.5],
[1.5, 0. ],
[1.5, 0.5],
[1.5, 1. ],
[1.5, 1.5]])
I have some sparse indices:
[[0 0]
[0 1]
[1 0]
[1 1]
[1 2]
[2 0]]
The corresponding value of each index is:
[[0.1 0.2 0.3]
[0.4 0.5 0.6]
[0.7 0.8 0.9]
[1.0 1.1 1.2]
[1.3 1.4 1.5]
[1.6 1.7 1.8]]
How to convert the 6x3 value tensor to 3x3x3 dense tensor in tensorflow? The value for indices not specified in indices is zero vector [0. 0. 0.]. The dense tensor is just like this:
[[[0.1 0.2 0.3]
[0.4 0.5 0.6]
[0.0 0.0 0.0]]
[[0.7 0.8 0.9]
[1.0 1.1 1.2]
[1.3 1.4 1.5]]
[[1.6 1.7 1.8]
[0.0 0.0 0.0]
[0.0 0.0 0.0]]]
You can do that with tf.scatter_nd:
import tensorflow as tf
with tf.Graph().as_default(), tf.Session() as sess:
indices = tf.constant(
[[0, 0],
[0, 1],
[1, 0],
[1, 1],
[1, 2],
[2, 0]])
values = tf.constant(
[[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6],
[0.7, 0.8, 0.9],
[1.0, 1.1, 1.2],
[1.3, 1.4, 1.5],
[1.6, 1.7, 1.8]])
out = tf.scatter_nd(indices, values, [3, 3, 3])
print(sess.run(out))
Output:
[[[0.1 0.2 0.3]
[0.4 0.5 0.6]
[0. 0. 0. ]]
[[0.7 0.8 0.9]
[1. 1.1 1.2]
[1.3 1.4 1.5]]
[[1.6 1.7 1.8]
[0. 0. 0. ]
[0. 0. 0. ]]]
There is no definite way to do it in Tensorflow using any reshape kind of function. I could only think about an iterative solution by creating a list and converting it back to a Tensor. This is perhaps not the most efficient solution, but this might work for your code.
# list of indices
idx=[[0,0],[0,1], [1,0],[1,1], [1,2], [2,0]]
# Original Tensor to reshape
dense_tensor=tf.Variable([[0.1, 0.2 ,0.3],[0.4, 0.5, 0.6], [0.7, 0.8, 0.9], [1.0,1.1,1.2],[1.3,1.4,1.5], [1.6,1.7,1.8]])
# creating a temporary list to later convert to Tensor
c=np.zeros([3,3,3]).tolist()
for i in range(3):
count=0
for j in range(3):
if([i,j] in idx):
c[i][j]=dense_tensor[count]
count=count+1
else:
c[i][j]=tf.Variable([0,0,0], dtype=tf.float32)
# Convert obtained list to Tensor
converted_tensor = tf.convert_to_tensor(c, dtype=tf.float32)
You can define the ranges depending upon the size of Tensor you want. For your case, I have chosen 3 as you wanted a 3x3x3 Tensor. I hope this helps!
I am confused about doing vectorization using numpy.
In particular, I have a matrix of this form:
of type <type 'list'>
[[0.0, 0.0, 0.0, 0.0], [0.02, 0.04, 0.0325, 0.04], [1, 2, 3, 4]]
How do I make it look like the following using numpy?
[[ 0.0 0.0 0.0 0.0 ]
[ 0.02 0.04 0.0325 0.04 ]
[ 1 2 3 4 ]]
Yes, I know I can do it using:
np.array([[0.0, 0.0, 0.0, 0.0], [0.02, 0.04, 0.0325, 0.04], [1, 2, 3, 4]])
But I have a very long matrix, and I can't just type out each rows like that. How can I handle the case when I have a very long matrix?
This is not a matrix of type list, it is a list that contains lists. You may think of it as matrix, but to Python it is just a list
alist = [[0.0, 0.0, 0.0, 0.0], [0.02, 0.04, 0.0325, 0.04], [1, 2, 3, 4]]
arr = np.array(alist)
works just the same as
arr = np.array([[0.0, 0.0, 0.0, 0.0], [0.02, 0.04, 0.0325, 0.04], [1, 2, 3, 4]])
This creates 2d array, with shape (3,4) and dtype float
In [212]: arr = np.array([[0.0, 0.0, 0.0, 0.0], [0.02, 0.04, 0.0325, 0.04], [1, 2, 3, 4]])
In [213]: arr
Out[213]:
array([[ 0. , 0. , 0. , 0. ],
[ 0.02 , 0.04 , 0.0325, 0.04 ],
[ 1. , 2. , 3. , 4. ]])
In [214]: print(arr)
[[ 0. 0. 0. 0. ]
[ 0.02 0.04 0.0325 0.04 ]
[ 1. 2. 3. 4. ]]
Assuming you start with a large array, why not split it into arrays of the right size (n):
splitted = [l[i:i + n] for i in range(0, len(array), n)]
and make the matrix from that:
np.array(splitted)
If you're saying you have a list of lists stored in Python object A, all you need to do is call np.array(A) which will return a numpy array using the elements of A. Otherwise, you need to specify what form your data is in right now to clarify how you want to load your data.