Good night,
I need to convert the result of a operation and I don´t how:
C = np.matmul(A, B)
print(C)
I get:
[[1 3 22 6]
[5 14 5 3]
[44 31 24 33]]
But I need:
[[1 5 44 3]
[14 31 22 5]
[24 6 3 33]]
How i can convert the first result to get the second one?
Thanks
np.matmul multiples backwards
from how you would imagine: try C = np.matmul(B, A). It's harder given we don't have the matrices A or B !
Transpose the matrix, reshape to a 12-element vector (or 1x12 matrix), and then reshape to the final form.
Related
How can I write a function named split which accepts three parameters a, b, c and then do the following.
create a n dimensional array 'x' having first a natural numbers (use np.arange method).
change the shape of x to (c, b) and assign to new array y.
split the array y horizontally into two arrays, then assign it to i and j.
display i and j.
I tried using hsplit and array_split methods and then assign it to i and j. But the output is not matching as given below.
import numpy as np
x=np.arange(20)
y = np.array(x)
z= y.reshape(10,2)
#a = np.hsplit(z,2)
(a,b)=np.array_split(z,2,axis=0)
print(a)
print(b)
Actual output:-
[[0 1]
[2 3]
[4 5]
[6 7]
[8 9]]
[[10 11]
[12 13]
[14 15]
[16 17]
[18 19]]
Desired output:-
[[ 0 1 2 3 4]
[10 11 12 13 14]]
[[ 5 6 7 8 9]
[15 16 17 18 19]]
You were right with hsplit, the problem is just the shape is the other way around to get the desired output:
import numpy as np
x=np.arange(20)
y = np.array(x)
z= y.reshape(2,10)
a,b = np.hsplit(z,2)
print(a)
print(b)
output:
[[ 0 1 2 3 4]
[10 11 12 13 14]]
[[ 5 6 7 8 9]
[15 16 17 18 19]]
I have a 2D array of shape (50,50). I need to subtract a value from each column of this array skipping the first), which is calculated based on the index of the column. For example, using a for loop it would look something like this:
for idx in range(1, A[0, :].shape[0]):
A[0, idx] -= idx * (...) # simple calculations with idx
Now, of course this works fine, but it's very slow and performance is critical for my application. I've tried computing the values to be subtracted using np.fromfunction() and then subtracting it from the original array, but results are different than those obtained by the for loop iteractive subtraction:
func = lambda i, j: j * (...) #some simple calculations
subtraction_matrix = np.fromfunction(np.vectorize(func), (1,50))
A[0, 1:] -= subtraction_matrix
What am I doing wrong? Or is there some other method that would be better? Any help is appreciated!
All your code snippets indicate that you require the subtraction to happen only in the first row of A (though you've not explicitly mentioned that). So, I'm proceeding with that understanding.
Referring to your use of from_function(), you can use the subtraction_matrix as below:
A[0,1:] -= subtraction_matrix[1:]
Testing it out (assuming shape (5,5) instead of (50,50)):
import numpy as np
A = np.arange(25).reshape(5,5)
print (A)
func = lambda j: j * 10 #some simple calculations
subtraction_matrix = np.fromfunction(np.vectorize(func), (5,), dtype=A.dtype)
A[0,1:] -= subtraction_matrix[1:]
print (A)
Output:
[[ 0 1 2 3 4] # print(A), before subtraction
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
[[ 0 -9 -18 -27 -36] # print(A), after subtraction
[ 5 6 7 8 9]
[ 10 11 12 13 14]
[ 15 16 17 18 19]
[ 20 21 22 23 24]]
If you want the subtraction to happen in all the rows of A, you just need to use the line A[:,1:] -= subtraction_matrix[1:], instead of the line A[0,1:] -= subtraction_matrix[1:]
i'm tring to do a simple linear regression using pyfinance package and using PandasRollingOLS to have rolling regression beta (rolling with min_window option).
it works but i would like to have a min_window in the function.
i would like to have min_window in the rollingOLS function, because if we have a window of 90 it does not perform OLS on first 90 values. i would like to perform a OLS expanding until 90 observations starting when there is at least 12 observation (min_window), then rolling of 90 (window)
i tried to understand the code of the package but i'm not able to include min_window in the code.
i would like this kind of function (this is init of PandasRollingOLS class):
def __init__(self, y, x=None, window=None, **min_window=None**, has_const=False, use_const=True):
i think i should update the code on utils.rolling_windows posted below, can someone help me please?
def rolling_windows(a, window):
"""Creates rolling-window 'blocks' of length `window` from `a`.
Note that the orientation of rows/columns follows that of pandas.
Example
-------
import numpy as np
onedim = np.arange(20)
twodim = onedim.reshape((5,4))
print(twodim)
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]
[12 13 14 15]
[16 17 18 19]]
print(rwindows(onedim, 3)[:5])
[[0 1 2]
[1 2 3]
[2 3 4]
[3 4 5]
[4 5 6]]
print(rwindows(twodim, 3)[:5])
[[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
[[ 4 5 6 7]
[ 8 9 10 11]
[12 13 14 15]]
[[ 8 9 10 11]
[12 13 14 15]
[16 17 18 19]]]
"""
if window > a.shape[0]:
raise ValueError('Specified `window` length of {0} exceeds length of'
' `a`, {1}.'.format(window, a.shape[0]))
if isinstance(a, (Series, DataFrame)):
a = a.values
if a.ndim == 1:
a = a.reshape(-1, 1)
shape = (a.shape[0] - window + 1, window) + a.shape[1:]
strides = (a.strides[0],) + a.strides
windows = np.squeeze(np.lib.stride_tricks.as_strided(a, shape=shape,
strides=strides))
# In cases where window == len(a), we actually want to "unsqueeze" to 2d.
# I.e., we still want a "windowed" structure with 1 window.
if windows.ndim == 1:
windows = np.atleast_2d(windows)
return windows
thank you all!
Alessandro
I am struggling with this myself at the moment using PandasRollingOLS. I came to the temporary conclusion to simply take care of it before the regression, i.e. delete every column with below min_window value before running regressions.
min_window = 3
df.loc[:,~(df.rolling(min_window).count() < min_window).all()]
Note that it requires that your dataframe has NaNs (which is why I guess you want to have a min_window):
NaN NaN
0.5 NaN
0.8 NaN
0.7 0.5
0.6 0.4
This might be a temporary (ugly) solution until a Python guru stumbles upon your post.
I have a 3D matrix of dimensions, 549x19x50 I need to create a 2D matrix which gets me a 549x950 matrix.
What i did so far is using tensorflow;
#data_3d is the 3D matrix
data_2d = tf.reshape(data_3d,[549,-1])
This prints out all the values of data_3d in the prompt and when I try to access data_2d it gives me an NameError
data_3d is a list of list of lists. Not a tensor or a ndarray. If we cant do this for lists, is there any way to easily convert lists to ndarrays?
Thanks in advance,
Bhashithe
There is a simple way to do so using numpy:
import numpy as np
data_3d = np.arange(27).reshape((3,3,3))
data_2d = data_3d.swapaxes(1,2).reshape(3,-1)
Ouput:
data_2d
[[ 0 3 6 1 4 7 2 5 8]
[ 9 12 15 10 13 16 11 14 17]
[18 21 24 19 22 25 20 23 26]]
print data_3d
[[[ 0 1 2]
[ 3 4 5]
[ 6 7 8]]
[[ 9 10 11]
[12 13 14]
[15 16 17]]
[[18 19 20]
[21 22 23]
[24 25 26]]]
Note: swapaxes(1,2) is the main thing here - you need to define which axes you want to swap.
There are 2D arrays of numbers as outputs of some numerical processes in the form of 1x1, 3x3, 5x5, ... shaped, that correspond to different resolutions.
In a stage an average i.e., 2D array value in the shape nxn needs to be produced.
If the outputs were in consistency of shape i.e., say all in 11x11 the solution was obvious, so:
element_wise_mean_of_all_arrays.
For the problem of this post however the arrays are in different shapes so the obvious way does not work!
I thought it might be some help by using kron function however it didn't. For example, if array is in shape of 17x17 how to make it 21x21. So for all others from 1x1,3x3,..., to build a constant-shaped array, say 21x21.
Also it can be the case that the arrays are smaller and bigger in shape compared to the target shape. That is an array of 31x31 to be shruk into 21x21.
You could imagine the problem as a very common task for images, being shrunk or extended.
What are possible efficient approaches to do the same jobs on 2D arrays, in Python, using numpy, scipy, etc?
Updates:
Here is a bit optimized version of the accepted answer bellow:
def resize(X,shape=None):
if shape==None:
return X
m,n = shape
Y = np.zeros((m,n),dtype=type(X[0,0]))
k = len(X)
p,q = k/m,k/n
for i in xrange(m):
Y[i,:] = X[i*p,np.int_(np.arange(n)*q)]
return Y
It works perfectly, however do you all agree it is the best choice in terms of the efficiency? If not any improvement?
# Expanding ---------------------------------
>>> X = np.array([[1,2,3],[4,5,6],[7,8,9]])
[[1 2 3]
[4 5 6]
[7 8 9]]
>>> resize(X,[7,11])
[[1 1 1 1 2 2 2 2 3 3 3]
[1 1 1 1 2 2 2 2 3 3 3]
[1 1 1 1 2 2 2 2 3 3 3]
[4 4 4 4 5 5 5 5 6 6 6]
[4 4 4 4 5 5 5 5 6 6 6]
[7 7 7 7 8 8 8 8 9 9 9]
[7 7 7 7 8 8 8 8 9 9 9]]
# Shrinking ---------------------------------
>>> X = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])
[[ 1 2 3 4]
[ 5 6 7 8]
[ 9 10 11 12]
[13 14 15 16]]
>>> resize(X,(2,2))
[[ 1 3]
[ 9 11]]
Final note: that the code above easily could be translated to Fortran for the highest performance possible.
I'm not sure I understand exactly what you are trying but if what I think the simplest way would be:
wanted_size = 21
a = numpy.array([[1,2,3],[4,5,6],[7,8,9]])
b = numpy.zeros((wanted_size, wanted_size))
for i in range(wanted_size):
for j in range(wanted_size):
idx1 = i * len(a) / wanted_size
idx2 = j * len(a) / wanted_size
b[i][j] = a[idx1][idx2]
You could maybe replace the b[i][j] = a[idx1][idx2] with some custom function like the average of a 3x3 matrix centered in a[idx1][idx2] or some interpolation function.