Related
I'm struggling with the result of a matrix multiplication in pytorch and I don't know how to solve it, in particular:
I'm multiplying these two matrices
tensor([[[[209.5000, 222.7500],
[276.5000, 289.7500]],
[[208.5000, 221.7500],
[275.5000, 288.7500]]]], dtype=torch.float64)
and
tensor([[[[ 0., 1., 2., 5., 6., 7., 10., 11., 12.],
[ 2., 3., 4., 7., 8., 9., 12., 13., 14.],
[10., 11., 12., 15., 16., 17., 20., 21., 22.],
[12., 13., 14., 17., 18., 19., 22., 23., 24.]],
[[25., 26., 27., 30., 31., 32., 35., 36., 37.],
[27., 28., 29., 32., 33., 34., 37., 38., 39.],
[35., 36., 37., 40., 41., 42., 45., 46., 47.],
[37., 38., 39., 42., 43., 44., 47., 48., 49.]],
[[50., 51., 52., 55., 56., 57., 60., 61., 62.],
[52., 53., 54., 57., 58., 59., 62., 63., 64.],
[60., 61., 62., 65., 66., 67., 70., 71., 72.],
[62., 63., 64., 67., 68., 69., 72., 73., 74.]]]],
dtype=torch.float64)
with the following line of code A.view(2,-1) # B, and then I reshape the result with result.view(2, 3, 3, 3).
The resulting matrix is
tensor([[[[ 6687.5000, 7686.0000, 8684.5000],
[11680.0000, 12678.5000, 13677.0000],
[16672.5000, 17671.0000, 18669.5000]],
[[ 6663.5000, 7658.0000, 8652.5000],
[11636.0000, 12630.5000, 13625.0000],
[16608.5000, 17603.0000, 18597.5000]],
[[31650.0000, 32648.5000, 33647.0000],
[36642.5000, 37641.0000, 38639.5000],
[41635.0000, 42633.5000, 43632.0000]]],
[[[31526.0000, 32520.5000, 33515.0000],
[36498.5000, 37493.0000, 38487.5000],
[41471.0000, 42465.5000, 43460.0000]],
[[56612.5000, 57611.0000, 58609.5000],
[61605.0000, 62603.5000, 63602.0000],
[66597.5000, 67596.0000, 68594.5000]],
[[56388.5000, 57383.0000, 58377.5000],
[61361.0000, 62355.5000, 63350.0000],
[66333.5000, 67328.0000, 68322.5000]]]], dtype=torch.float64)
Instead I want
tensor([[[[ 6687.5000, 7686.0000, 8684.5000],
[11680.0000, 12678.5000, 13677.0000],
[16672.5000, 17671.0000, 18669.5000]],
[[31650.0000, 32648.5000, 33647.0000],
[36642.5000, 37641.0000, 38639.5000],
[41635.0000, 42633.5000, 43632.0000]],
[[56612.5000, 57611.0000, 58609.5000],
[61605.0000, 62603.5000, 63602.0000],
[66597.5000, 67596.0000, 68594.5000]]],
[[[ 6663.5000, 7658.0000, 8652.5000],
[11636.0000, 12630.5000, 13625.0000],
[16608.5000, 17603.0000, 18597.5000]],
[[31526.0000, 32520.5000, 33515.0000],
[36498.5000, 37493.0000, 38487.5000],
[41471.0000, 42465.5000, 43460.0000]],
[[56388.5000, 57383.0000, 58377.5000],
[61361.0000, 62355.5000, 63350.0000],
[66333.5000, 67328.0000, 68322.5000]]]], dtype=torch.float64)
Can someone help me? Thanks
This is a common but interesting problem because it involves a combination of torch.reshapes and torch.transpose to solve it. More specifically, you will need
Apply an initial reshape to restructure the tensor and expose the axes you want to swap;
Then do so using a transpose operation;
Lastly apply a second reshape to get to the desired format.
In your case, you could do:
>>> result.reshape(3,2,3,3).transpose(0,1).reshape(2,3,3,3)
tensor([[[[ 6687.5000, 7686.0000, 8684.5000],
[11680.0000, 12678.5000, 13677.0000],
[16672.5000, 17671.0000, 18669.5000]],
[[31650.0000, 32648.5000, 33647.0000],
[36642.5000, 37641.0000, 38639.5000],
[41635.0000, 42633.5000, 43632.0000]],
[[56612.5000, 57611.0000, 58609.5000],
[61605.0000, 62603.5000, 63602.0000],
[66597.5000, 67596.0000, 68594.5000]]],
[[[ 6663.5000, 7658.0000, 8652.5000],
[11636.0000, 12630.5000, 13625.0000],
[16608.5000, 17603.0000, 18597.5000]],
[[31526.0000, 32520.5000, 33515.0000],
[36498.5000, 37493.0000, 38487.5000],
[41471.0000, 42465.5000, 43460.0000]],
[[56388.5000, 57383.0000, 58377.5000],
[61361.0000, 62355.5000, 63350.0000],
[66333.5000, 67328.0000, 68322.5000]]]], dtype=torch.float64)
I encourage you to look a the intermediate results to get an idea of how the method works so you can apply it on other use cases in the future.
I would like to replace values in a NumpyArray, in only one column, on several selected rows only, using putmask. I wish to use indexing on the array to be modified as well as the mask used. Therefor I create a nd.array, a mask and and array of desired replacements. as follows:
import numpy as np
a = np.linspace(1,30,30)
a.shape(10,3)
mask = np.random.randint(2, size=8)
replacements = a[[2,4,5,6,7,8],0]*a[[2,4,5,6,7,8],1]
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
mask
array([0, 1, 0, 0, 1, 0, 1, 1])
replacements
array([ 56., 182., 272., 380., 506., 650.])
np.putmask(a[[2,4,5,6,7,8],2], mask[2::], replacements)
My expected result would look like this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 272.],
[19., 20., 21.],
[22., 23., 506.],
[25., 26., 650.],
[28., 29., 30.]])
But instead I get this:
a
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.],
[10., 11., 12.],
[13., 14., 15.],
[16., 17., 18.],
[19., 20., 21.],
[22., 23., 24.],
[25., 26., 27.],
[28., 29., 30.]])
Anybody has an idea maybe?
Note that you are using fancy indexing, so when using np.putmask you are modifying a copy rather than a sliced view, and thus the original array remains unchanged. You can check this by trying to index using slice notation, np.putmask(a[2:8,2], mask[2::], replacements) for instance, which would in this case modify the values in a.
What you could do is use np.where and reassign the values to the corresponding indices in a:
a[[2,4,5,6,7,8],2] = np.where(mask[2::], replacements, a[[2,4,5,6,7,8],2])
Output
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 56.],
[ 10., 11., 12.],
[ 13., 14., 182.],
[ 16., 17., 272.],
[ 19., 20., 380.],
[ 22., 23., 506.],
[ 25., 26., 650.],
[ 28., 29., 30.]])
I have the array
A = array([[ 1., 2., 3., 10., 11., 12.],
[ 4., 5., 6., 13., 14., 15.],
[ 7., 8., 9., 16., 17., 18.],
[ 19., 20., 21., 28., 29., 30.],
[ 22., 23., 24., 31., 32., 33.],
[ 25., 26., 27., 34., 35., 36.]])
I would like to reshape it in order to obtain
B = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]
I have tried
>>> B = A.reshape(1,36)
array([[ 1., 2., 3., 10., 11., 12., 4., 5., 6., 13., 14.,
15., 7., 8., 9., 16., 17., 18., 19., 20., 21., 28.,
29., 30., 22., 23., 24., 31., 32., 33., 25., 26., 27.,
34., 35., 36.]])
But, obviously, I didn't reach the result. My real data differs from the example, so I can't sort the array A to obtain B.
I suppose I need more reshapes...
Split each of those two axes such that the remaining ones are of lengths 2 each with a reshape giving us a 4D array and then swap the middle two axes with np.swapaxes() and finally flatten with np.ravel() -
A.reshape(2,3,2,3).swapaxes(1,2).ravel()
Generically put -
m,n = A.shape
A.reshape(2,m//2,2,n//2).swapaxes(1,2).ravel()
Sample run -
In [15]: A
Out[15]:
array([[ 1., 2., 3., 10., 11., 12.],
[ 4., 5., 6., 13., 14., 15.],
[ 7., 8., 9., 16., 17., 18.],
[ 19., 20., 21., 28., 29., 30.],
[ 22., 23., 24., 31., 32., 33.],
[ 25., 26., 27., 34., 35., 36.]])
In [16]: A.reshape(2,3,2,3).swapaxes(1,2).ravel()
Out[16]:
array([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11.,
12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.,
23., 24., 25., 26., 27., 28., 29., 30., 31., 32., 33.,
34., 35., 36.])
I'm trying to perform a fourier transform of a data set that I have and subsequently writing its real and imaginary parts separately.
This is my code:
import sys,string
import numpy as np
from math import *
import fileinput
from scipy.fftpack import fft, ifft
temparray = []
for i in range(200000):
line = sys.stdin.readline() ## reads data from a text file
fields = map(float,line.split())
temparray.append(fields)
acf = sum(temparray,[]) ## I need to do this as the appended array from above is nested
y = np.fft.fft(acf)
z = (y.real,y.imag)
print z
The output that I get is as follows:
(array([ 2600.36368107, 2439.50426935, 1617.52631545, ..., 1575.78483016, 1617.52631545, 2439.50426935]), array([ 0. , -767.19967198, -743.75183367, ..., 726.45052092, 743.75183367, 767.19967198]))
It looks like its only printing the first few and last few values, completely skipping everything in between. Can anybody please tell me why this is happening?
Thanks
As others have indicated, include a modified version of
>>> np.set_printoptions(edgeitems=5,linewidth=80,precision=2,suppress=True,threshold=10)
>>> a = np.arange(0,100.)
>>>
>>> a
array([ 0., 1., 2., 3., 4., ..., 95., 96., 97., 98., 99.])
>>> np.set_printoptions(edgeitems=5,linewidth=80,precision=2,suppress=True,threshold=100)
>>> a
array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11.,
12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.,
24., 25., 26., 27., 28., 29., 30., 31., 32., 33., 34., 35.,
36., 37., 38., 39., 40., 41., 42., 43., 44., 45., 46., 47.,
48., 49., 50., 51., 52., 53., 54., 55., 56., 57., 58., 59.,
60., 61., 62., 63., 64., 65., 66., 67., 68., 69., 70., 71.,
72., 73., 74., 75., 76., 77., 78., 79., 80., 81., 82., 83.,
84., 85., 86., 87., 88., 89., 90., 91., 92., 93., 94., 95.,
96., 97., 98., 99.])
np.set_printoptions(edgeitems=3,linewidth=80,precision=2,suppress=True,threshold=5)
perhaps setting threshold to some very large number.
Addendum
I would be remiss if I didn't state that the simplest solution to the above is to simple use
>>> list(a)
should you not care whether an array is visually returned or not.
I have following problem:
I try to solve the equilation by using linalg.solv and it seems to work. But if i try to check it by inserting the aquired coefficients and one of the required points i get a difference of about 30% to the original data. Have i done a mistake, which i dont get? Or do i have do use a different methode to get more accurate datasets. If yes, which one?
Further if i use different values from which i entered while calculating the coefficients, i get strangly high results
data = np.genfromtxt("data1.csv",dtype=float,delimiter=";")
to = data[0,2:]
tc = data[1:,0]
y = data[1:,2:]
a = np.array([[1, to[0], tc[0], to[0]**2, to[0]*tc[0], tc[0]**2, to[0]**3, tc[0]*to[0]**2, to[0]*tc[0]**2, tc[0]**3],
[1, to[1], tc[1], to[1]**2, to[1]*tc[1], tc[1]**2, to[1]**3, tc[1]*to[1]**2, to[1]*tc[1]**2, tc[1]**3],
[1, to[2], tc[2], to[2]**2, to[2]*tc[2], tc[2]**2, to[2]**3, tc[2]*to[2]**2, to[2]*tc[2]**2, tc[2]**3],
[1, to[3], tc[3], to[3]**2, to[3]*tc[3], tc[3]**2, to[3]**3, tc[3]*to[3]**2, to[3]*tc[3]**2, tc[3]**3],
[1, to[4], tc[4], to[4]**2, to[4]*tc[4], tc[4]**2, to[4]**3, tc[4]*to[4]**2, to[4]*tc[4]**2, tc[4]**3],
[1, to[5], tc[5], to[5]**2, to[5]*tc[5], tc[5]**2, to[5]**3, tc[5]*to[5]**2, to[5]*tc[5]**2, tc[5]**3],
[1, to[6], tc[6], to[6]**2, to[6]*tc[6], tc[6]**2, to[6]**3, tc[6]*to[6]**2, to[6]*tc[6]**2, tc[6]**3],
[1, to[7], tc[7], to[7]**2, to[7]*tc[7], tc[7]**2, to[7]**3, tc[7]*to[7]**2, to[7]*tc[7]**2, tc[7]**3],
[1, to[8], tc[8], to[8]**2, to[8]*tc[8], tc[8]**2, to[8]**3, tc[8]*to[8]**2, to[8]*tc[8]**2, tc[8]**3],
[1, to[9], tc[9], to[9]**2, to[9]*tc[9], tc[9]**2, to[9]**3, tc[9]*to[9]**2, to[9]*tc[9]**2, tc[9]**3]])
b = np.array([y[0,0],y[1,1],y[2,2],y[3,3],y[4,4],y[5,5],y[6,6],y[7,7],y[8,8],y[9,9]])
c = np.linalg.solve(a, b)
ges_to = 10
ges_tc = 35
ges_y = c[0] + c[1]*ges_to + c[2]*ges_tc + c[3]*ges_to**2 + c[4]*ges_to*ges_tc + c[5]*ges_tc**2 + c[6]*ges_to**3 \
+ c[7]*ges_tc*ges_to**2 + c[8]*ges_to*ges_tc**2 + c[9]*ges_tc**3
Here are the values I use to calculate
('to:', array([ 15., 10., 5., 0., -5., -10., -15., -20., -25., -30., -35.]))
('tc:', array([ 30., 35., 40., 45., 50., 55., 60., 65., 70., 80., 90.]))
('b', array([ 24., 31., 35., 36., 35., 33., 30., 25., 21., 18.]))
('y:', array([[ 24., 26., 27., 27., 26., 25., 23., 20., 18., 15., 13.],
[ 30., 31., 31., 30., 29., 27., 24., 21., 18., 16., 14.],
[ 35., 35., 35., 33., 31., 29., 26., 22., 19., 16., 15.],
[ 40., 40., 38., 36., 33., 30., 27., 23., 20., 16., 15.],
[ 45., 44., 41., 39., 35., 32., 28., 24., 20., 17., 16.],
[ 49., 47., 44., 41., 37., 33., 29., 25., 20., 17., 16.],
[ 53., 51., 47., 43., 39., 34., 30., 25., 21., 17., 16.],
[ 57., 54., 50., 45., 40., 35., 30., 25., 21., 17., 16.],
[ 61., 57., 52., 47., 41., 36., 31., 26., 21., 17., 16.],
[ 64., 60., 54., 59., 53., 37., 32., 27., 22., 18., 19.],
[ 67., 63., 56., 61., 55., 59., 34., 29., 24., 18., 19.]]))
('ges_y:', 49.0625)
The floating point arithmetic is leading you astray. If you look at the determinant of that matrix a, it's something incredibly small like 1.551864434916621e-51. If you compute the determinate with the entries as integers (and avoid floating point arithmetic weirdness) you'll see it's actually 0, and the rank of your matrix is 5. So it's singular, and in general equations like ax = b may not have any solution.
Another quick thing you can do to see this is np.dot(a, np.linalg.inv(a)) is nowhere close to the identity matrix. Similarly, np.dot(a, c) is nowhere close to b.
There may or may not be an actual solution to ax = b, but np.linalg.lstsq(a,b) will get you an approximate solution in either case, if that's sufficient for you rneeds.