I got a problem I don't really understand.
I have a .bin-File with a timeseries of signals and noise in it. I have the exact timedata, to cut out only the interesting parts.
My problem is that sometimes the amplitudes are way to high, and sometimes they are like expected. I think I broke down the problem to the following:
sampling_rate = 2e6
dt = np.dtype(np.int32)
# get Timedata
start_raw_L1 = 261.2 # good_signal
count_raw_L1 = 1.315
# start_raw_L1 = 261.4 bad_signal
# count_raw_L1 = 1.315
start_L1 = np.int64(start_raw_L1*sampling_rate*4)
count_L1 = np.int64(count_raw_L1 * sampling_rate)
# L1
bin_data = open(bin_file, "rb")
bin_data.seek(start_L1, os.SEEK_SET)
data_L1 = np.fromfile(bin_data, dtype=dt, count=count_L1, sep='')
bin_data.close()
# Plot
plt.plot(data_L1)
So it looks like that it matters a lot which time I choose? If I just change the start time a little bit, the signal changes in the amplitude height, I dont get why? Maybe someone can help me out.
Thanks a lot!
Best regards
Bastian
good_signal
bad_signal
int(261.4*2e6*4) gives 2091199999. That is not a multiple of 4. The problem is that 261.4*2e6 gives 522799999.99999994, not 522800000 as you might have expected.
Move the multiplication by 4 outside of the conversion to an integer: 4*int(261.4*2e6) gives 2091199996. You might prefer 4*round(261.4*2e6), which gives 2091200000. In your code, that means using, say,
start_L1 = 4*np.int64(start_raw_L1*sampling_rate)
Related
I am working in a RK4 orbital propagator, and I have followed the basic structure of:
k1=dt×f(tn,yn)
k2=dt×f(tn+dt/2,yn+k1/2)
k3=dt×f(tn+dt/2,yn+k2/2)
k4=dt×f(tn+dt,yn+k3)
yn+1=yn+1/6(k1+2k2+2k3+k4)
tn+1=tn+dt
I have bassed this procces in a simpler one, created by me for RK1 implementation:
#NOTE: sat is an array with the structure [x,y,z,vx,vy,vz,ax,ay,az]
def pasosimple(sat,dt):
#physics constants
G = 6.6742e-20
m_s = 1.989e30
mu=G*m_s
#Position Change from previous values of velocity and acceleration
dx=sat[3]*dt+sat[6]*(dt**2)/2
dy=sat[4]*dt+sat[7]*(dt**2)/2
dz=sat[5]*dt+sat[8]*(dt**2)/2
#Velocity change due to previous acceleration
dvx=sat[6]*dt
dvy=sat[7]*dt
dvz=sat[8]*dt
#xyz update
x=dx+sat[0]
y=dy+sat[1]
z=dz+sat[2]
#With the xyz update, we calculate new accelerations
ax=(-mu*(x)/(np.sqrt((x)**2+(y)**2+(z)**2)**3))
ay=(-mu*(y)/(np.sqrt((x)**2+(y)**2+(z)**2)**3))
az=(-mu*(z)/(np.sqrt((x)**2+(y)**2+(z)**2)**3))
#Substraction to obtain the difference acceleration
dax=ax-sat[6]
day=ay-sat[7]
daz=az-sat[8]
dsat=np.array([dx,dy,dz,dvx,dvy,dvz,dax,day,daz])
sat=np.array([x,y,z,dvx+sat[3],dvy+sat[4],dvz+sat[5],ax,ay,az])
return dsat,sat
This code works, as far as I know, and I have already tested it.
Now, for imlementing the RK4, I'm doing this:
def rk4(sat,dt):
d1,s1=pasosimple(sat,dt)
d2,s2=pasosimple(sat+d1/2,dt+0.5*dt)
d3,s3=pasosimple(sat+d2/2,dt+0.5*dt)
d4,s4=pasosimple(sat+d3,dt+dt)
sat=sat+(1/6)*(d1+d2*2+d3*2+d4)
return sat
Which does not work.
I was hoping that anyone could give some insight about what i am doing wrong.
Thank you all.
EDIT:
I have tried this alternate configuration for pasosimple, following (or trying to follow) the comments below. The result did not work either. However, using only the new pasosimple code and not nesting it into rk4works nicely, which makes me think the problem is now in rk4.
def pasosimple(sat,dt):
G = 6.6742e-20
m_t=5.972e24
m_s = 1.989e30
m_m=6.39e23
mu=G*m_s
mu_t=G*m_t
mu_m=G*m_m
ax=(-mu*(sat[0])/(np.sqrt((sat[0])**2+(sat[1])**2+(sat[2])**2)**3))
ay=(-mu*(sat[1])/(np.sqrt((sat[0])**2+(sat[1])**2+(sat[2])**2)**3))
az=(-mu*(sat[2])/(np.sqrt((sat[0])**2+(sat[1])**2+(sat[2])**2)**3))
x_punto=np.array([sat[3],sat[4],sat[5],ax,ay,az])
x=np.array([sat[0]+x_punto[0]*dt,sat[1]+x_punto[1]*dt,sat[2]+x_punto[2]*dt,sat[3]+x_punto[3]*dt,sat[4]+x_punto[4]*dt,sat[5]+x_punto[5]*dt])
dsat=np.array([x_punto[0]*dt,x_punto[1]*dt,x_punto[2]*dt,x_punto[3]*dt,x_punto[4]*dt,x_punto[5]*dt,ax-sat[6],ay-sat[7],ay-sat[8]])
sat=np.array([x[0],x[1],x[2],x[3],x[4],x[5],ax,ay,az])
return dsat,sat
Suppose I have an object Trace, say trace, already and I want to have an amplitude data at the time 30 sec.
I think I can do it like below, assuming begin time is 0 for simplicity,
delta = trace.stats.delta
count = int(30 / delta)
target_value = trace.data[count]
Do they prepare a good way to do it?
something like....
trace.foo(time=30)
Right now we do not have such a convenience method, but I agree that it would be a good addition. You could make a feature request in our GitHub issue tracker: https://github.com/obspy/obspy/issues/new
In the meantime, you could make use of the sample times convenience functions mixed with some numpy to achieve what you want..
If you're looking for times relative to start of trace:
from obspy import read
tr = read()[0]
my_time = 4 # 4 seconds after start of trace
times = tr.times()
index = times.searchsorted(my_time)
print(times[index])
print(tr.data[index])
4.0
204.817965896
If you're looking for absolute times:
from obspy import read, UTCDateTime
tr = read()[0]
my_time = UTCDateTime("2009-08-24T00:20:12.0Z") # 4 seconds after start of trace
times = tr.times('utcdatetime')
index = times.searchsorted(my_time)
print(times[index])
print(tr.data[index])
2009-08-24T00:20:12.000000Z
156.68994731
If you're not matching the exact time of one sample, you will have to manually interpolate.
Hope this gets you started.
"""Some simulations to predict the future portfolio value based on past distribution. x is
a numpy array that contains past returns.The interpolated_returns are the returns
generated from the cdf of the past returns to simulate future returns. The portfolio
starts with a value of 100. portfolio_value is filled up progressively as
the program goes through every loop. The value is multiplied by the returns in that
period and a dollar is removed."""
portfolio_final = []
for i in range(10000):
portfolio_value = [100]
rand_values = np.random.rand(600)
interpolated_returns = np.interp(rand_values,cdf_values,x)
interpolated_returns = np.add(interpolated_returns,1)
for j in range(1,len(interpolated_returns)+1):
portfolio_value.append(interpolated_returns[j-1]*portfolio_value[j-1])
portfolio_value[j] = portfolio_value[j]-1
portfolio_final.append(portfolio_value[-1])
print (np.mean(portfolio_final))
I couldn't find a way to write this code using numpy. I was having a look at iterations using nditer but I was unable to move ahead with that.
I guess the easiest way to figure out how you can vectorize your stuff would be to look at the equations that govern your evolution and see how your portfolio actually iterates, finding patterns that could be vectorized instead of trying to vectorize the code you already have. You would have noticed that the cumprod actually appears quite often in your iterations.
Nevertheless you can find the semi-vectorized code below. I included your code as well such that you can compare the results. I also included a simple loop version of your code which is much easier to read and translatable into mathematical equations. So if you share this code with somebody else I would definitely use the simple loop option. If you want some fancy-pants vectorizing you can use the vector version. In case you need to keep track of your single steps you can also add an array to the simple loop option and append the pv at every step.
Hope that helps.
Edit: I have not tested anything for speed. That's something you can easily do yourself with timeit.
import numpy as np
from scipy.special import erf
# Prepare simple return model - Normal distributed with mu &sigma = 0.01
x = np.linspace(-10,10,100)
cdf_values = 0.5*(1+erf((x-0.01)/(0.01*np.sqrt(2))))
# Prepare setup such that every code snippet uses the same number of steps
# and the same random numbers
nSteps = 600
nIterations = 1
rnd = np.random.rand(nSteps)
# Your code - Gives the (supposedly) correct results
portfolio_final = []
for i in range(nIterations):
portfolio_value = [100]
rand_values = rnd
interpolated_returns = np.interp(rand_values,cdf_values,x)
interpolated_returns = np.add(interpolated_returns,1)
for j in range(1,len(interpolated_returns)+1):
portfolio_value.append(interpolated_returns[j-1]*portfolio_value[j-1])
portfolio_value[j] = portfolio_value[j]-1
portfolio_final.append(portfolio_value[-1])
print (np.mean(portfolio_final))
# Using vectors
portfolio_final = []
for i in range(nIterations):
portfolio_values = np.ones(nSteps)*100.0
rcp = np.cumprod(np.interp(rnd,cdf_values,x) + 1)
portfolio_values = rcp * (portfolio_values - np.cumsum(1.0/rcp))
portfolio_final.append(portfolio_values[-1])
print (np.mean(portfolio_final))
# Simple loop
portfolio_final = []
for i in range(nIterations):
pv = 100
rets = np.interp(rnd,cdf_values,x) + 1
for i in range(nSteps):
pv = pv * rets[i] - 1
portfolio_final.append(pv)
print (np.mean(portfolio_final))
Forget about np.nditer. It does not improve the speed of iterations. Only use if you intend to go one and use the C version (via cython).
I'm puzzled about that inner loop. What is it supposed to be doing special? Why the loop?
In tests with simulated values these 2 blocks of code produce the same thing:
interpolated_returns = np.add(interpolated_returns,1)
for j in range(1,len(interpolated_returns)+1):
portfolio_value.append(interpolated_returns[j-1]*portfolio[j-1])
portfolio_value[j] = portfolio_value[j]-1
interpolated_returns = (interpolated_returns+1)*portfolio - 1
portfolio_value = portfolio_value + interpolated_returns.tolist()
I assuming that interpolated_returns and portfolio are 1d arrays of the same length.
This question may be a little specialist, but hopefully someone might be able to help. I normally use IDL, but for developing a pipeline I'm looking to use python to improve running times.
My fits file handling setup is as follows:
import numpy as numpy
from astropy.io import fits
#Directory: /Users/UCL_Astronomy/Documents/UCL/PHASG199/M33_UVOT_sum/UVOTIMSUM/M33_sum_epoch1_um2_norm.img
with fits.open('...') as ima_norm_um2:
#Open UVOTIMSUM file once and close it after extracting the relevant values:
ima_norm_um2_hdr = ima_norm_um2[0].header
ima_norm_um2_data = ima_norm_um2[0].data
#Individual dimensions for number of x pixels and number of y pixels:
nxpix_um2_ext1 = ima_norm_um2_hdr['NAXIS1']
nypix_um2_ext1 = ima_norm_um2_hdr['NAXIS2']
#Compute the size of the images (you can also do this manually rather than calling these keywords from the header):
#Call the header and data from the UVOTIMSUM file with the relevant keyword extensions:
corrfact_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
coincorr_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
#Check that the dimensions are all the same:
print(corrfact_um2_ext1.shape)
print(coincorr_um2_ext1.shape)
print(ima_norm_um2_data.shape)
# Make a new image file to save the correction factors:
hdu_corrfact = fits.PrimaryHDU(corrfact_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_corrfact]).writeto('.../M33_sum_epoch1_um2_corrfact.img')
# Make a new image file to save the corrected image to:
hdu_coincorr = fits.PrimaryHDU(coincorr_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_coincorr]).writeto('.../M33_sum_epoch1_um2_coincorr.img')
I'm looking to then apply the following corrections:
# Define the variables from Poole et al. (2008) "Photometric calibration of the Swift ultraviolet/optical telescope":
alpha = 0.9842000
ft = 0.0110329
a1 = 0.0658568
a2 = -0.0907142
a3 = 0.0285951
a4 = 0.0308063
for i in range(nxpix_um2_ext1 - 1): #do begin
for j in range(nypix_um2_ext1 - 1): #do begin
if (numpy.less_equal(i, 4) | numpy.greater_equal(i, nxpix_um2_ext1-4) | numpy.less_equal(j, 4) | numpy.greater_equal(j, nxpix_um2_ext1-4)): #then begin
#UVM2
corrfact_um2_ext1[i,j] == 0
coincorr_um2_ext1[i,j] == 0
else:
xpixmin = i-4
xpixmax = i+4
ypixmin = j-4
ypixmax = j+4
#UVM2
ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax])
xvec_UVM2 = ft*ima_UVM2sum
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2*xvec_UVM2) + (a3*xvec_UVM2*xvec_UVM2*xvec_UVM2) + (a4*xvec_UVM2*xvec_UVM2*xvec_UVM2*xvec_UVM2)
Ctheory_UVM2 = - alog(1-(alpha*ima_UVM2sum*ft))/(alpha*ft)
corrfact_um2_ext1[i,j] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum)
coincorr_um2_ext1[i,j] = corrfact_um2_ext1[i,j]*ima_sk_um2[i,j]
The above snippet is where it is messing up, as I have a mixture of IDL syntax and python syntax. I'm just not sure how to convert certain aspects of IDL to python. For example, the ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax]) I'm not quite sure how to handle.
I'm also missing the part where it will update the correction factor and coincidence correction image files, I would say. If anyone could have the patience to go over it with a fine tooth comb and suggest the neccessary changes I need that would be excellent.
The original normalised image can be downloaded here: Replace ... in above code with this file
One very important thing about numpy is that it does every mathematical or comparison function on an element-basis. So you probably don't need to loop through the arrays.
So maybe start where you convolve your image with a sum-filter. This can be done for 2D images by astropy.convolution.convolve or scipy.ndimage.filters.uniform_filter
I'm not sure what you want but I think you want a 9x9 sum-filter that would be realized by
from scipy.ndimage.filters import uniform_filter
ima_UVM2sum = uniform_filter(ima_norm_um2_data, size=9)
since you want to discard any pixel that are at the borders (4 pixel) you can simply slice them away:
ima_UVM2sum_valid = ima_UVM2sum[4:-4,4:-4]
This ignores the first and last 4 rows and the first and last 4 columns (last is realized by making the stop value negative)
now you want to calculate the corrections:
xvec_UVM2 = ft*ima_UVM2sum_valid
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2**2) + (a3*xvec_UVM2**3) + (a4*xvec_UVM2**4)
Ctheory_UVM2 = - np.alog(1-(alpha*ima_UVM2sum_valid*ft))/(alpha*ft)
these are all arrays so you still do not need to loop.
But then you want to fill your two images. Be careful because the correction is smaller (we inored the first and last rows/columns) so you have to take the same region in the correction images:
corrfact_um2_ext1[4:-4,4:-4] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum_valid)
coincorr_um2_ext1[4:-4,4:-4] = corrfact_um2_ext1[4:-4,4:-4] *ima_sk_um2
still no loop just using numpys mathematical functions. This means it is much faster (MUCH FASTER!) and does the same.
Maybe I have forgotten some slicing and that would yield a Not broadcastable error if so please report back.
Just a note about your loop: Python's first axis is the second axis in FITS and the second axis is the first FITS axis. So if you need to loop over the axis bear that in mind so you don't end up with IndexErrors or unexpected results.
Hey so I am just working on some coding homework for my Python class using JES. Our assignment is to take a sound, add some white noise to the background and to add an echo as well. There is a bit more exacts but I believe I am fine with that. There are four different functions that we are making: a main, an echo equation based on a user defined length of time and amount of echos, a white noise generation function, and a function to merge the noises.
Here is what I have so far, haven't started the merging or the main yet.
#put the following line at the top of your file. This will let
#you access the random module functions
import random
#White noise Generation functiton, requires a sound to match sound length
def whiteNoiseGenerator(baseSound) :
noise = makeEmptySound(getLength(baseSound))
index = 0
for index in range(0, getLength(baseSound)) :
sample = random.randint(-500, 500)
setSampleValueAt(noise, index, sample)
return noise
def multipleEchoesGenerator(sound, delay, number) :
endSound = getLength(sound)
newEndSound = endSound +(delay * number)
len = 1 + int(newEndSound/getSamplingRate(sound))
newSound = makeEmptySound(len)
echoAmplitude = 1.0
for echoCount in range (1, number) :
echoAmplitude = echoAmplitude * 0.60
for posns1 in range (0, endSound):
posns2 = posns1 + (delay * echoCount)
values1 = getSampleValueAt(sound, posns1) * echoAmplitude
values2 = getSampleValueAt(newSound, posns2)
setSampleValueAt (newSound, posns2, values1 + values2)
return newSound
I receive this error whenever I try to load it in.
The error was:
Inappropriate argument value (of correct type).
An error occurred attempting to pass an argument to a function.
Please check line 38 of C:\Users\insanity180\Desktop\Work\Winter Sophomore\CS 140\homework3\homework_3.py
That line of code is:
setSampleValueAt (newSound, posns2, values1 + values2)
Anyone have an idea what might be happening here? Any assistance would be great since I am hoping to give myself plenty of time to finish coding this assignment. I have gotten a similar error before and it was usually a syntax error however I don't see any such errors here.
The sound is made before I run this program and I defined delay and number as values 1 and 3 respectively.
Check the arguments to setSampleValueAt; your sample value must be out of bounds (should be within -32768 - 32767). You need to do some kind of output clamping for your algorithm.
Another possibility (which indeed was the error, according to further input) is that your echo will be out of the range of the sample - that is, if your sample was 5 seconds long, and echo was 0.5 seconds long; or the posns1 + delay is beyond the length of the sample; the length of the new sound is not calculated correctly.