I am trying to create a plot with numpy and I am running into a Value Error. I have checked the shapes of the arrays before they enter the loop and somehow they are coming out miss-shaped relative to one another. Here is my current code:
sol = np.zeros((partition, partition))
solA = np.zeros((partition, partition))
for i in range(partition):
sol[:, i] = odeint(SolveMe, ic, timeSpace, args=(zeemanSpace[i],))[:, 1]
solA[:, i] = (1/(time[-1]))*It.cumtrapz(sol[i:, 1], timeSpace)
With the previous declarations:
partition = 100
time = [ti, tf]
zman = [qi, qf]
zeemanSpace = np.linspace(zman[0], zman[-1], partition)
timeSpace = np.linspace(time[0], time[-1], partition)
I commented out the solA[:, i] in my for loop and run this test:
print(sol.shape)
print(solA.shape)
Which produces
runfile('/Users/taylor/Library/Mobile Documents/com~apple~CloudDocs/Documents/Academia/Student Material/OU/Research/DPT of BEC/Work/Numerical/Gen X. & P/Zhang et al. Results/Scratch.py', wdir='/Users/taylor/Library/Mobile Documents/com~apple~CloudDocs/Documents/Academia/Student Material/OU/Research/DPT of BEC/Work/Numerical/Gen X. & P/Zhang et al. Results')
(100, 100)
(100, 100)
When the previous line in the for loop is uncommented, the following error is produced:
ValueError: could not broadcast input array from shape (99,) into shape (100,)
Any help would be greatly appreciated!
Related
I am attempting to do a nested loop in order to find the mean-squared for a variety of different sized distributions. I keep getting an error that reads: "ValueError: could not broadcast input array from shape (0,) into shape (1000,)".
I am a beginner coder so I know this may be trivial for some...
My code:
#%% Initialize variables.
rng = np.random.default_rng()
rand = rng.random
num_steps = 1000
num_walks = 1000
x_step = np.zeros((num_steps, num_walks))
y_step = np.zeros((num_steps, num_walks))
x_final = np.zeros((1, num_walks))
y_final = np.zeros((1, num_walks))
displacement = np.zeros((num_walks, 1))
mean_squared_displacement = np.zeros(10)
#%% Find the mean-squared displacement for a variety of step numbers.
step_variation = np.linspace(0, 10000, 11)
for n in range(np.size(step_variation)-1):
for m in range(num_walks):
x_step[:,m] = np.cumsum(2*(rand(int(step_variation[n]))<.5)-1) # ERROR APPEARS ON THIS LINE
y_step[:,m] = np.cumsum(2*(rand(int(step_variation[n]))<.5)-1)
x_final[0,m] = x_step[-1,m]
y_final[0,m] = y_step[-1,m]
displacement[m,0] = np.sqrt(x_final[0,m]**2 + y_final[0,m]**2)
mean_squared_displacement[n] = np.mean(displacement[m,0]**2)
What steps did you take to debug this? Any? or did you just throw your hands up in despair, not understanding that the error means?
Did you examine the problem line? Test pieces in it?
x_step[:,m] = np.cumsum(2*(rand(int(step_variation[n]))<.5)-1)
The first value of step_variation is 0 (from linspace). rand(0) produces a (0,) shape array. The rest of that expression is thus also (0,) shape.
In [13]: rand(0)
Out[13]: array([], dtype=float64)
x_step is (1000,1000), so x_step[:,m] is (1000,) shape. The error tells us/you that it can't put a (0,) (no values) array into that (1000,) shape slot.
I have three 1D vectors. Let's say T with 100k element array, f and df each with 200 element array:
T = [T0, T1, ..., T100k]
f = [f0, f1, ..., f200]
df = [df0, df1, ..., df200]
For each element array, I have to calculate a function such as the following:
P = T*f + T**2 *df
My first instinct was to use the NumPy outer to find the function with each combination of f and df
P1 = np.outer(f,T)
P2 = np.outer(df,T**2)
P = np.add.outer(P1, P2)
However, in this case, I am facing the ram issue and receiving the following error:
Unable to allocate 2.23 PiB for an array with shape (200, 100000, 200,
100000) and data type float64
Is there a good way that I can calculate this?
My attempt using for loops
n=100
f_range = 5e-7
df_range = 1.5e-15
fsrc = np.arange(f - n * f_range, f + n * f_range, f_range) #array of 200
dfsrc = np.arange(df - n * df_range, df + n * df_range, df_range) #array of 200
dfnus=pd.DataFrame(fsrc)
numf=dfnus.shape[0]
dfnudots=pd.DataFrame(dfsrc)
numfdot=dfnudots.shape[0]
test2D = np.zeros([numf,(numfdot)])
for indexf, f in enumerate(fsrc):
for indexfd, fd in enumerate(dfsrc):
a=make_phase(T,f,fd) #--> this is just a function that performs T*f + T**2 *df
zlauf2d=z_n(a, n=1, norm=1) #---> And this is just another function that takes this 1D "a" and gives another 1D element array
test2D[indexf, indexfd]=np.copy(zlauf2d) #---> I do this so I could make a contour plot at the end. It just copys the same thing to 2D
Now my test2D has the shape of (200,200). This is what I want, however the floor loop is taking ages and I want somehow reduce two for loop to at least one.
Using broadcasting:
P1 = (f[:, np.newaxis] * T).sum(axis=-1)
P2 = (df[:, np.newaxis] * T**2).sum(axis=-1)
P = P1[:, np.newaxis] + P2
Alternatively, using outer:
P1 = (np.outer(f, T)).sum(axis=-1)
P2 = (np.outer(df, T**2)).sum(axis=-1)
P = P1[..., np.newaxis] + P2
This produces an array of shape (f.size, df.size) == (200, 200).
Generally speaking, if the final output array size is very large, one can either:
Reduce the size of the datatypes. One way is to change the datatypes of the arrays used to calculate the final output via P1.astype(np.float32). Alternatively, some operations allow one to pass in a dtype=np.float32 as a parameter.
Chunk the computation and work with smaller subsections of the result.
Based on the most recent edit, compute an array a with shape (200, 200, 100000). Then, take its element-wise norm along the last axis to produce an array z with shape (200, 200).
a = (
f[:, np.newaxis, np.newaxis] * T
+ df[np.newaxis, :, np.newaxis] * T**2
)
# L1 norm along last axis.
z = np.abs(a).sum(axis=-1)
This produces an array of shape (f.size, df.size) == (200, 200).
I have a for loop with a range of 2000 in this for loop I have to create an array called Array
out of two other arrays, let's call them ArrayOfPositionSatellite with a size of (3,38) and the other array called ArrayOfPositionMassPoint with a size of (38, 3, 4412). The size of Array is (38,3,4412) and the size of PositonOfSatellite and PointsOfMassPoint is (3, ). My attempt to overwrite the ArrayOfMassPoint with to for-loops :
ArrayOfPositionSatellite= ArrayOfPositionSatellite.T
Array = ArrayOfPositionMassPoint
for i in range(38):
for k in range(4412):
PositionOfSatellite = ArrayOfPositionSatellite[:,i]
PositionOfMassPoint= ArrayOfPositionMassPoint[i,:,k]
ElementOfA = -Gravitationalconstant* (PositionOfSatellite - PositionOfMassPoint)/(np.linalg.norm( PositionOfSatellite - PositionOfMassPoint)**3)
Array[i,:,k] = ElementOfArray
Problem
My problem is that it takes around 3 hours to run the code and this is too long. Is there some way to make it more time-efficient?
If something is unclear please leave a comment and I will add more details.
You can vectorize your calculations. Like:
import numpy as np
ArrayOfPositionSatellite = np.random.randn(3, 38)
ArrayOfPositionMassPoint = np.random.randn(38, 3, 4412)
Gravitationalconstant = 6.67430e-11
# This is the difference vector
v = ArrayOfPositionMassPoint - ArrayOfPositionSatellite.T[:,:,None]
# This is norm of the difference vector
norm = np.linalg.norm(v, axis=1) ** 3
# This is normalized vector
norm_v = v / norm[:, None, :]
# This is the result
array = norm_v * -Gravitationalconstant
array.shape
>>> (38, 3, 4412)
This takes around ~40ms on my machine, instead of 3 hours.
Hoping this is an easy problem and I just don't know the correct syntax.
I currently have a small 3D volume that is defined by a numpy array of 100,100,100.
For the problem I am testing I want to put this volume into a larger array (doesn't matter how big right now but I am testing on a 1000,1000,100 array).
Currently I am just making an empty numpy array using the following:
BigArray = np.zeros((1000,1000,100),np.float16)
Then I have my smaller array that for the purpose of this example can just be a randomly filled array.:
SmallArray = np.random.rand(100,100,100)
From here I want to loop through and fill the 1000,1000,100 array with the 100,100,100 array placing each cube next to one another. The large array starts with '0' values so it should be as simple as just adding the small array to the correct coordinates of the larger array however have no idea the syntax to do this. Could someone help?
Thanks
This should do it -- just use a standard nested for loop and numpy array assignment syntax:
small = np.random.rand(100, 100, 100)
big = np.zeros((1000, 1000, 100), dtype=np.int16)
for i in range(0, 1000, 100):
for j in range(0, 1000, 100):
big[i:i+100, j:j+100, :] = small
For generic sized 3D arrays:
def inset_into(small, big):
sx, sy, sz = small.shape
bx, by, bz = big.shape
# make sure values work
assert bx % sx == 0
assert by % sy == 0
assert bz == sz
for i in range(0, bx, sx):
for j in range(0, by, sy):
big[i:i+sx, j:j+sy, :] = small
return big
This should just be numpy slicing.
small = np.random.rand(100, 100, 100)
big = np.zeros((1000, 1000, 100), dtype=np.int16)
If you want to make big out of a bunch of smalls here is another way.
big = np.concatenate([small] * (big.shape[0] // small.shape[0]), axis=1)
big = np.concatenate([big] * (big.shape[1] // small.shape[1]), axis=0)
There is a speed difference. Looping is better.
I'm trying to rebuild a song in python, but I cannot concatenate the notes of the same.
I get this error:
ValueError: operands could not be broadcast together with shapes (0)
(1250)
Here's my code:
import numpy as np, matplotlib.pyplot as plt
def nota(f,d):
ts = 0.0002
t = np.arange(0, d, ts)
X = 5500*np.cos(2*np.pi*f*t)
return X
# II.2.b)
pausa = nota(0,0)
La = nota(440,0.25)
Mi = nota(659.26,0.25)
Do = nota(253.25,0.25)
Sol = nota(783.99,0.25)
Si = nota(493.88,0.25)
Solbemol = nota(830.61,0.25)
def FurElise():
musica = np.array((pausa,pausa,La,Mi,La,pausa,pausa,Mi,Mi,Solbemol, \
pausa,pausa,La,Mi,La,pausa,pausa,pausa,La,Mi,La, \
pausa,pausa,Mi,Mi,Solbemol,pausa,pausa,La,Mi,La, \
pausa,Do,Sol,Do,pausa,pausa,Sol,Sol,Si,pausa,pausa, \
La,Mi,La,pausa,pausa,Mi,Mi,Mi,pausa))
y=0
for x in musica:
z=np.hstack((x,y))
y = y+x
z=np.hstack((x,y))
plt.plot(z)
plt.show()
FurElise()
As #filmor notes, x and y are of different shapes, and the reason for that is your definition of pausa = nota(0,0). By using a d value of 0, the resulting array is of length 0 while all other arrays are of length 1250, and y = y+x will eventually throw the error you're seeing (e.g. after 3 iterations, given your current definition of musica).
Assuming you want the pause to be of the same length as all other notes, you can re-define pausa so as to get rid of the error:
pausa = nota(0,0.25)
You create musica (assuming that you've fixed pausa as described by #fgb) by stacking the 51 notes as rows in a 2D array. So, musica.shape is (51, 1250)
I think that you want z to be a long 1D array where all the notes are in one row, instead of each in their own row. There are two solutions. One, is to say:
musica = np.array((pausa,pausa,La,Mi,La,...))
z = musica.flatten()
plt.plot(z)
and completely remove all of this:
y=0
for x in musica:
z=np.hstack((x,y))
y = y+x
z=np.hstack((x,y))
A better solution is to make musica a 1D array when you create it:
musica = np.hstack((pausa,pausa,La,Mi,La,...))
plt.plot(musica)
This takes all the notes (pausa, La, etc) and stacks them horizontally (so that they're all in one row together), so you never need to flatten or concatenate anything.