I have a list of numbers that represent the flattened output of a matrix or array produced by another program, I know the dimensions of the original array and want to read the numbers back into either a list of lists or a NumPy matrix. There could be more than 2 dimensions in the original array.
e.g.
data = [0, 2, 7, 6, 3, 1, 4, 5]
shape = (2,4)
print some_func(data, shape)
Would produce:
[[0,2,7,6],
[3,1,4,5]]
Cheers in advance
Use numpy.reshape:
>>> import numpy as np
>>> data = np.array( [0, 2, 7, 6, 3, 1, 4, 5] )
>>> shape = ( 2, 4 )
>>> data.reshape( shape )
array([[0, 2, 7, 6],
[3, 1, 4, 5]])
You can also assign directly to the shape attribute of data if you want to avoid copying it in memory:
>>> data.shape = shape
If you dont want to use numpy, there is a simple oneliner for the 2d case:
group = lambda flat, size: [flat[i:i+size] for i in range(0,len(flat), size)]
And can be generalized for multidimensions by adding recursion:
import operator
def shape(flat, dims):
subdims = dims[1:]
subsize = reduce(operator.mul, subdims, 1)
if dims[0]*subsize!=len(flat):
raise ValueError("Size does not match or invalid")
if not subdims:
return flat
return [shape(flat[i:i+subsize], subdims) for i in range(0,len(flat), subsize)]
For those one liners out there:
>>> data = [0, 2, 7, 6, 3, 1, 4, 5]
>>> col = 4 # just grab the number of columns here
>>> [data[i:i+col] for i in range(0, len(data), col)]
[[0, 2, 7, 6],[3, 1, 4, 5]]
>>> # for pretty print, use either np.array or np.asmatrix
>>> np.array([data[i:i+col] for i in range(0, len(data), col)])
array([[0, 2, 7, 6],
[3, 1, 4, 5]])
Without Numpy we can do as below as well..
l1 = [1,2,3,4,5,6,7,8,9]
def convintomatrix(x):
sqrt = int(len(x) ** 0.5)
matrix = []
while x != []:
matrix.append(x[:sqrt])
x = x[sqrt:]
return matrix
print (convintomatrix(l1))
[list(x) for x in zip(*[iter(data)]*shape[1])]
(found this post searching for how this works)
Related
I have a function that returns an array, I want to append several of these arrays and get the mean of each row.
Basic code I want:
arr = []
for i in range(0,3):
b = get_array()
arr.append(b)
b_mean = np.mean(b, axis=0)
What I hope to achieve with it:
'''
arr = np.array([])
b0 = np.array([3, 3, 3, 5])
b1 = np.array([6, 6, 5, 7])
b3 = np.array([1, 2, 3, 4])
desired_result = np.array([[3, 3, 3, 5],
[6, 6, 5, 7],
[3, 2, 7, 3]])
desired_result_average = np.array([4, 3.66, 5, 5])
'''
I think I can achieve this by first making the array a regular list, and later converting the list back to a numpy array so I can get the mean. But this seems like a weird way...
I tried np.concatenate, np.insert, np.vstack. But these all require an initial array of the dimensions as the other arrays..
No need to be aware of the shape of b.
arrs = []
for i in range(0,3):
b = get_array()
arrs.append(b)
arr = np.vstack(arrs)
b_mean = np.mean(arr, axis=0)
I have the following grid(actually a dataframe):
params = pd.DataFrame(np.array([(alpha, gamma) for alpha in np.linspace(0,1,10) for gamma in np.linspace(0,2,10)]),
columns = ['alpha','gamma'])
Then I use apply
params['res'] = params.apply(lambda row: func(x=params['alpha'],y=params['gamma'],axis=1)
How do I unpack the above into a matrix/dataframe below?
pd.DataFrame(elements of params['res'],
index = np.linspace(0,1,10),
columns = np.linspace(0,2,10))
You can first trasform the res Series to a numpy array, then use the reshape method:
result_df = pd.DataFrame(params['res'].to_numpy().reshape(10,10),
index = np.linspace(0,1,10),
columns = np.linspace(0,2,10))
Using a smaller size example:
# Simulating res
res = np.random.randint(0,10, 9)
res
array([9, 3, 5, 9, 3, 1, 4, 0, 6])
res.reshape(3,3)
array([[9, 3, 5],
[9, 3, 1],
[4, 0, 6]])
If this is not your expected result, you can traspose it:
res.reshape(3,3).T
array([[9, 9, 4],
[3, 3, 0],
[5, 1, 6]])
You are looking for pivot:
params.pivot(index='alpha', columns='gamma', values='res')
If the question only ask the fastest way to make a 10 by 10 numpy 2d array/matrix from the res column, then I think this solution is the one:
params.res.values.reshape(10,10)
I have an array A of size [1, x] of values and an array B of size [1, y] (y > x) of indexes corresponding to array A. I want as result an array C of size [1,y] filled with values of A.
Here is an example of inputs and outputs:
>>> A = [6, 7, 8]
>>> B = [0, 2, 0, 0, 1]
>>> C = #Some operations
>>> C
[6, 8, 6, 6, 7]
Of course I could solve it like that:
>>> C = []
>>> for val in B:
>>> C.append(A[val])
But I was actually expected a nicer way to do it. Especially because I want to use it as an argument of another function. An expression looking like A[B] (but a working one) would be ideal. I don't mind solution using NumPy or pandas.
Simple with a list comprehension:
A = [6, 7, 8]
B = [0, 2, 0, 0, 1]
C = [A[i] for i in B]
print(C)
This yields
[6, 8, 6, 6, 7]
For fetching multiple items operator.itemgetter comes in handy:
from operator import itemgetter
A = [6, 7, 8]
B = [0, 2, 0, 0, 1]
itemgetter(*B)(A)
# (6, 8, 6, 6, 7)
Also as you've mentioned numpy, this could be done directly by indexing the array as you've specified, i.e. A[B]:
import numpy as np
A = np.array([6, 7, 8])
B = np.array([0, 2, 0, 0, 1])
A[B]
# array([6, 8, 6, 6, 7])
Another option is to use np.take:
np.take(A,B)
# array([6, 8, 6, 6, 7])
This is one way, using numpy ndarrays:
import numpy as np
A = [6, 7, 8]
B = [0, 2, 0, 0, 1]
C = list(np.array(A)[B]) # No need to convert B into an ndarray
# list() is for converting ndarray back into a list,
# (if that's what you finally want)
print (C)
Explanation
Given a numpy ndarray (np.array(A)), we can index into it using an
array of integers (which happens to be exactly what your preferred
form of solution is): The array of integers that you use for
indexing into the ndarray, need not be another ndarray. It can even
be a list, and that suits us too, since B happens to a list. So,
what we have is:
np.array(A)[B]
The result of such an indexing would be another ndarray, having the
same shape (dimensions) as the array of indexes. So, in our case, as
we are indexing into an ndarray using a list of integer indexes, the
result of that indexing would be a one-dimensional ndarray of the
same length as the list of indexes.
Finally, if we want to convert the above result, from a
one-dimensional ndarray back into a list, we can pass it as an
argument to list():
list(np.array(A)[B])
You could do it with list comprehension:
>>> A = [6, 7, 8]
>>> B = [0, 2, 0, 0, 1]
>>> C = [A[x] for x in B]
>>> print(C)
[6, 8, 6, 6, 7]
I think you need a generator (list comprehension):
A = [1, 2, 3]
B = [0, 2, 0, 0, 1]
C = [A[i] for i in B]
Once you're using numpy.array you're able to do exactly what you want with syntax you expect:
>>> a = array([6, 7, 8])
>>> b = array([0, 2, 0, 0, 1])
>>> a[b]
array([6, 8, 6, 6, 7])
Let's say I have a list of elements X and one of indices Y.
X = [1, 2, 3, 4, 5, 6, 7]
Y = [0, 3, 4]
Is there a function in Python that allows one to extract elements from X based on the indices provided in Y? After execution, X would be:
X = [1, 4, 5]
X = [X[index] for index in Y]
This is a list comprehension; you can look up that topic to learn more.
The list comprehension provided by #Prune is the way to go in pure python. If you don't mind numpy, it might be easier just use their indexing scheme:
import numpy as np
>>> np.array(X)[Y]
array([1, 4, 5])
You can use list.__getitem__ with map:
X = [1, 2, 3, 4, 5, 6, 7]
Y = [0, 3, 4]
res = list(map(X.__getitem__, Y)) # [1, 4, 5]
Or, if you are happy to use a 3rd party library, you can use NumPy:
import numpy as np
X = np.array([1, 2, 3, 4, 5, 6, 7])
res = X[Y] # array([1, 4, 5])
Need to take the values from one array, put them through a function and put them in another array. It is meant to be done using a pair of nested for loops. Please help. Complete beginner here.
EDIT: Ok to clarify, I have a 2-d array with various values in it. I want to apply a function to all of these values and have a 2-d array returned with the values after they have gone through the function. I am working in python. Thanks for the quick responses and any help you can give!
EDIT3: Example code:
import numpy as N
def makeGrid(dim):
''' Function to return a grid of distances from the centre of an array.
This version uses loops to fill the array and is thus slow.'''
tabx = N.arange(dim) - float(dim/2.0) + 0.5
taby = N.arange(dim) - float(dim/2.0) + 0.5
grid = N.zeros((dim,dim), dtype='float')
for y in range(dim):
for x in range(dim):
grid[y,x] = N.sqrt(tabx[x]**2 + taby[y]**2)
return grid
import math
def BigGrid(dim):
l= float(raw_input('Enter a value for lambda: '))
p= float(raw_input('Enter a value for phi: '))
a = makeGrid
b= N.zeros ((10,10),dtype=float) #Create an arry to take the returned values
for i in range(10):
for j in range (10):
b[i,j] = a[i][j]*2
if __name__ == "__main__":
''' Module test code '''
size = 10 #Dimension of the array
newGrid = BigGrid(size)
newGrid = N.round(newGrid, decimals=2)
print newGrid
def map_row(row):
return map(some_function,row)
map(map_row,my_2d_list)
Is probably how I would do it...
Based on your question, it appears you're using Numpy. If you're not too concerned about speed, you can simply call the function with a numpy array; the function will operate on the entire array for you.
There's no need to write the iteration explicitly, though if you can find a way to take advantage of numpy's special features, that will be faster than using a function designed to operate on one element at a time. Unless you're working with a very large dataset, though, this should be fine:
import numpy as np
>>> g = np.array( [ [1,2,3], [ 4,5,6] ] )
array([[1, 2, 3],
[4, 5, 6]])
>>> def myfunc( myarray ):
... return 2 * myarray
...
>>> myfunc(g)
array([[ 2, 4, 6],
[ 8, 10, 12]])
First, you have a bug in your code in the following line:
a = makeGrid
You are setting a to be a function, not an array. You should have the following:
a = makeGrid(dim)
That is why you had the TypeError when you tried the answer by #abought.
Now, to apply an operation element-wise in numpy there are many possibilities. If you want to perform the same operation for every element in the array, the simplest way is to use array operations:
b = a * 2
(Note that you don't need to declare b beforehand. And you also don't need any loops.) Numpy has also many C-optimised functions that perform the same operation on each element of an array. These are called ufuncs. You can combine ufuncs to get complex expressions evaluated element-wise. For example:
b = N.sin(a**2) + N.log(N.abs(a))
Your a array from makeGrid() can also be much more efficiently created using array operations and numpy's mgrid:
grid = N.mgrid[-dim//2 + 1:dim//2:0.5, -dim//2 + 1:dim//2:0.5]
grid = N.sqrt(grid[0]**2 + grid[1]**2)
If you want to perform different operations on each array element, things get more complicated and it may not be possible to avoid loops. For these cases, numpy has a way to decompose loops on a nD array using ndenumerate or ndidex. Your example with ndenumerate:
for index, x in N.ndenumerate(a):
b[index] = x * 2
This is faster than multiple loops, but the array operations should be used whenever possible.
From what I can get in terms of context from the question and what a 2d-array typically means it looks like you are trying to do the following:
>>>> array2d = [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]
>>> def add_two( v ):
... return v + 2
...
>>> [ [ add_two( v ) for v in row ] for row in array2d ]
[[2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6], [2, 3, 4, 5, 6]]
The above uses a list comprehension which is the same as using the two nested for loops and in this case more readable and involves less direct interaction of the list methods as you're describing what the list is rather than building it.
Here is a one-line with double map
map(lambda x:map(func, x), l)
Example:
l=[[1,2,3],[4,3,1]]
map(lambda x:map(lambda x:x*10,x),l)
[[10, 20, 30], [40, 30, 10]]
Easy to do it with a nested loop:
def my_function(n): # n will become y from the next part
new_num = # do whatever you want with it
return new_num
my_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] # just an example
new_list, final_list = [], [] # multiple assignment
for x in my_list:
print x
new_list = []
for y in x:
# y is now the first value of the first value of my_list--- 1.
my_num = my_function(y)
new_list.append(my_num)
final_list.append(new_list)
print final_list
That should do it.
Returns: [[2, 3, 4], [5, 6, 7], [8, 9, 10]].
for(int i; i < x; i++)
for(int j; j < y; j++)
array2[i][j] = func(array2[i][j])
Something like that?