Related
I am having trouble getting my scipy optimization to work, and I think it is because of my constraint. The goal of my code is to find the best x and y coordinates to place a label for a scatterplot point, without this label being close to an old label. My optimization is the Euclidean distance between the scatterplot point and the label point (so that the label is as close as possible to the point). My constraint checks all the used x and y values, and makes sure that the new label x and y values (nx and ny), are not close to any of the old x and y values. If either the x or y is within 1, then the constraint returns 1 instead of 0. This should be the correct way I think because the constraint is an "eq".
However, I am guessing the problem is that the constraint is not a smooth equation, so the optimization has no idea how to handle it.
Is there any alternative way I can solve this problem?
#SECTION 1: Scatterplot: Labeling Points
#optimize (minimize) Euclidean Distance
usedx = []
usedy = []
#bounds
b = (0,10)
bnds = (b,b)
#objective
def objective(x, y):
nx = x[0]
ny = x[1]
ox = y[0]
oy = y[1]
p1 = (ox,oy)
p2 = (nx, ny)
return distance.euclidean(p1, p2)
#constraint
def constraint(p, x, y):
test = 0
nx = p[0]
ny = p[1]
for i in x:
if abs(i - nx) < 1:
test = 1
for i in y:
if abs(i - ny) < 1:
test = 1
return test
const = {'type': 'eq', 'fun': constraint, 'args': (usedx, usedy)}
for i, txt in enumerate(selectedFirmCurrent.sht_fund_name):
originaly = selectedFirmCurrent.iloc[i]['category_score']
originalx = selectedFirmCurrent.iloc[i]['fund_score']
originalpoint = [originalx, originaly]
#initial guess
p0 = (originalx, originaly -.25)
#optimization
solution = minimize(objective, args = originalpoint, x0 = p0,bounds = bnds,constraints = const)
#variable assign
newx = solution.x[0]
newy = solution.x[1]
usedx.append(newx)
usedy.append(newy)
print(originalx, ": ", newx)
print(originaly, ": ", newy+0.25)
#implement
ax.annotate(txt, (originalx,originaly), (newx,newy), arrowprops= dict(arrowstyle = '->'))
I'm generating coordinates of a bi-cone model in spherical coordinates. I'm using a series of nested for loops, as show here:
theta_in = 30.0 * np.pi/180.0
theta_out = 60.0 * np.pi/180.0
phi = 2*np.pi # rotation
R = 1.0
sampling = 100
theta = np.linspace(theta_in,theta_out,sampling)
phi = np.linspace(0,phi,sampling)
r = np.linspace(-R,R,sampling)
x = []
y = []
z = []
for ri in r:
for pi in phi:
for ti in theta:
xi = ri*np.cos(pi)*np.sin(ti)
yi = ri*np.sin(pi)*np.sin(ti)
zi = ri*np.cos(ti)
x.append(xi)
y.append(yi)
z.append(zi)
This generates the following desired output:
The use of these nested for loops to generate coordinate lists is not very efficient. My question is: what would be the most efficient way to do this while possibly avoiding the use of nested for loops?
Create open grids off those inputs and then perform the same operations -
RI,PI,TI = np.ix_(r,phi,theta) # get open grids
X = RI*np.cos(PI)*np.sin(TI)
Y = RI*np.sin(PI)*np.sin(TI)
Z = np.repeat(RI*np.cos(TI),sampling,axis=1)
Alternative #1 : Those open grids could also be constructed with explicit axis addition, like so -
RI,PI,TI = r[:,None,None], phi[:,None], theta
Alternative #2 : We can pre-compute np.sin(TI) and re-use it at two steps.
Alternative #3 : Also, for a read-only broadcatsed version of Z, we can use np.broadcast_to -
Z = np.broadcast_to(RI*np.cos(TI),(sampling,sampling,sampling))
Alternative #4 : Leverage multi-cores with numexpr module -
import numexpr as ne
X = ne.evaluate('RI*cos(PI)*sin(TI)')
# similarly Y and Z
Note that the outputs would be 3D arrays. So, to get equivalent ones, you might want to flatten those. For the same, we can use .ravel() on the outputs.
theta_in = 30.0 * np.pi/180.0
theta_out = 60.0 * np.pi/180.0
phi = 2*np.pi # rotation
R = 1.0
sampling = 100
theta = np.linspace(theta_in,theta_out,sampling)
phi = np.linspace(0,phi,sampling)
r = np.linspace(-R,R,sampling)
x=list(map(lambda ti,pi,ri:ri*np.cos(pi)*np.sin(ti),theta,phi,r))
y=list(map(lambda ti,pi,ri:ri*np.sin(pi)*np.sin(ti),theta,phi,r))
z=list(map(lambda ti,ri:ri*np.cos(ti),theta,r))
It is the first time I am trying to write a Poincare section code at Python.
I borrowed the piece of code from here:
https://github.com/williamgilpin/rk4/blob/master/rk4_demo.py
and I have tried to run it for my system of second order coupled odes. The problem is that I do not see what I was expecting to. Actually, I need the Poincare section when x=0 and px>0.
I believe that my implementation is not the best out there. I would like to:
Improve the way that the initial conditions are chosen.
Apply the correct conditions (x=0 and px>0) in order to acquire the correct Poincare section.
Create one plot with all the collected poincare section data, not four separate ones.
I would appreciate any help.
This is the code:
from matplotlib.pyplot import *
from scipy import *
from numpy import *
# a simple Runge-Kutta integrator for multiple dependent variables and one independent variable
def rungekutta4(yprime, time, y0):
# yprime is a list of functions, y0 is a list of initial values of y
# time is a list of t-values at which solutions are computed
#
# Dependency: numpy
N = len(time)
y = array([thing*ones(N) for thing in y0]).T
for ii in xrange(N-1):
dt = time[ii+1] - time[ii]
k1 = dt*yprime(y[ii], time[ii])
k2 = dt*yprime(y[ii] + 0.5*k1, time[ii] + 0.5*dt)
k3 = dt*yprime(y[ii] + 0.5*k2, time[ii] + 0.5*dt)
k4 = dt*yprime(y[ii] + k3, time[ii+1])
y[ii+1] = y[ii] + (k1 + 2.0*(k2 + k3) + k4)/6.0
return y
# Miscellaneous functions
n= 1.0/3.0
kappa1 = 0.1
kappa2 = 0.1
kappa3 = 0.1
def total_energy(valpair):
(x, y, px, py) = tuple(valpair)
return .5*(px**2 + py**2) + (1.0/(1.0*(n+1)))*(kappa1*np.absolute(x)**(n+1)+kappa2*np.absolute(y-x)**(n+1)+kappa3*np.absolute(y)**(n+1))
def pqdot(valpair, tval):
# input: [x, y, px, py], t
# takes a pair of x and y values and returns \dot{p} according to the Hamiltonian
(x, y, px, py) = tuple(valpair)
return np.array([px, py, -kappa1*np.sign(x)*np.absolute(x)**n+kappa2*np.sign(y-x)*np.absolute(y-x)**n, kappa2*np.sign(y-x)*np.absolute(y-x)**n-kappa3*np.sign(y)*np.absolute(y)**n]).T
def findcrossings(data, data1):
# returns indices in 1D data set where the data crossed zero. Useful for generating Poincare map at 0
prb = list()
for ii in xrange(len(data)-1):
if (((data[ii] > 0) and (data[ii+1] < 0)) or ((data[ii] < 0) and (data[ii+1] > 0))) and data1[ii] > 0:
prb.append(ii)
return array(prb)
t = linspace(0, 1000.0, 100000)
print ("step size is " + str(t[1]-t[0]))
# Representative initial conditions for E=1
E = 1
x0=0
y0=0
init_cons = [[x0, y0, np.sqrt(2*E-(1.0*i/10.0)*(1.0*i/10.0)-2.0/(n+1)*(kappa1*np.absolute(x0)**(n+1)+kappa2*np.absolute(y0-x0)**(n+1)+kappa3*np.absolute(y0)**(n+1))), 1.0*i/10.0] for i in range(-10,11)]
outs = list()
for con in init_cons:
outs.append( rungekutta4(pqdot, t, con) )
# plot the results
fig1 = figure(1)
for ii in xrange(4):
subplot(2, 2, ii+1)
plot(outs[ii][:,1],outs[ii][:,3])
ylabel("py")
xlabel("y")
title("Full trajectory projected onto the plane")
fig1.suptitle('Full trajectories E = 1', fontsize=10)
# Plot Poincare sections at x=0 and px>0
fig2 = figure(2)
for ii in xrange(4):
subplot(2, 2, ii+1)
xcrossings = findcrossings(outs[ii][:,0], outs[ii][:,3])
yints = [.5*(outs[ii][cross, 1] + outs[ii][cross+1, 1]) for cross in xcrossings]
pyints = [.5*(outs[ii][cross, 3] + outs[ii][cross+1, 3]) for cross in xcrossings]
plot(yints, pyints,'.')
ylabel("py")
xlabel("y")
title("Poincare section x = 0")
fig2.suptitle('Poincare Sections E = 1', fontsize=10)
show()
You need to compute the derivatives of the Hamiltonian correctly. The derivative of |y-x|^n for x is
n*(x-y)*|x-y|^(n-2)=n*sign(x-y)*|x-y|^(n-1)
and the derivative for y is almost, but not exactly (as in your code), the same,
n*(y-x)*|x-y|^(n-2)=n*sign(y-x)*|x-y|^(n-1),
note the sign difference. With this correction you can take larger time steps, with correct linear interpolation probably even larger ones, to obtain the images
I changed the integration of the ODE to
t = linspace(0, 1000.0, 2000+1)
...
E_kin = E-total_energy([x0,y0,0,0])
init_cons = [[x0, y0, (2*E_kin-py**2)**0.5, py] for py in np.linspace(-10,10,8)]
outs = [ odeint(pqdot, con, t, atol=1e-9, rtol=1e-8) ) for con in init_cons[:8] ]
Obviously the number and parametrization of initial conditions may change.
The computation and display of the zero-crossings was changed to
def refine_crossing(a,b):
tf = -a[0]/a[2]
while abs(b[0])>1e-6:
b = odeint(pqdot, a, [0,tf], atol=1e-8, rtol=1e-6)[-1];
# Newton step using that b[0]=x(tf) and b[2]=x'(tf)
tf -= b[0]/b[2]
return [ b[1], b[3] ]
# Plot Poincare sections at x=0 and px>0
fig2 = figure(2)
for ii in xrange(8):
#subplot(4, 2, ii+1)
xcrossings = findcrossings(outs[ii][:,0], outs[ii][:,3])
ycrossings = [ refine_crossing(outs[ii][cross], outs[ii][cross+1]) for cross in xcrossings]
yints, pyints = array(ycrossings).T
plot(yints, pyints,'.')
ylabel("py")
xlabel("y")
title("Poincare section x = 0")
and evaluating the result of a longer integration interval
I am trying to define the archimedean spiral: when I'm trying to define the inclination angle (incl) of the tangent vector to the orbit ( i.e: tan(incl))
I'm getting an error:
'numpy.ufunc' object does not support item assignment"
and "can't assign to function call"
the same error when I want to calculate cos(incl), and sin(incl).
Any suggestions and helps.
My code is:
T = 100
N = 10000
dt = float(T)/N
D = 2
DII = 10
a = 2.
v = 0.23
omega = 0.2
r0 = v/omega
t = np.linspace(0,T,N+1)
r = v*t
theta = a + r/r0
theta = omega*t
x = r * np.cos(omega*t)
y = r * np.sin(omega*t)
dxdr = np.cos(theta) - (r/r0)*np.sin(theta)
dydr = np.sin(theta) + (r/r0)*np.cos(theta)
dydx = (r0*np.sin(theta) + r*np.cos(theta))/r0*np.cos(theta) - r*np.sin(theta)
np.tan[incl] = dydx
incl = np.arctan((dydx))
### Calculate cos(incl) ,sin(incl) :
np.sin[np.incl] = np.tan(np.incl)/np.sqrt(1 + np.tan(np.incl)*2)
np.cos[incl] = 1/np.sqrt(1 + np.tan(incl)*2)
p1, = plt.plot(xx, yy)
i= 0 # this is the first value of the array
Bx = np.array([np.cos(i), -np.sin(i)])
By = np.array([np.sin(i), np.cos(i)])
n = 1000
seed(2)
finalpositions = []
for number in range(0, 10):
x = []
y = []
x.append(0)
y.append(0)
for i in range(n):
s = np.random.normal(0, 1, 2)
deltaX = Bx[0]*np.sqrt(2*DII*dt)*s[0] + Bx[1]*np.sqrt(2*D*dt)*s[1]
deltaY = By[0]*np.sqrt(2*DII*dt)*s[0] + By[1]*np.sqrt(2*D*dt)*s[1]
x.append(x[-1] + deltaX)
y.append(y[-1] + deltaY)
finalpositions.append([x[-1], y[-1]])
p2, = plt.plot(finalpositions[:,0],finalpositions[:,1],'*')
plt.show()
The error message is correct, you are trying to assign to a function! I think you're trying to compute a value that represents the sin, cos or tan of a value, but that doesn't mean you need to assign to np.sin, etc. What you want is to calculate the value which represents the trig function, and then use the inverse trig function to get the angle:
## np.tan[incl]= dydx ## np.tan is a function, so you cannot index it like an array, and you should not assign to it.
incl = np.arctan((dydx)) ## this is all you need to get "incl"
### Calculate cos(incl) ,sin(incl) :
## NOTE: you already have the angle you need!! No need for a complicated formulate to compute the sin or cos!
sin_incl = np.sin(incl)
cos_incl = np.cos(incl)
EDIT: One additional comment...np is a module that contains lots of numeric methods. When you calculate incl, it is not part of np! So there is no need to reference it like np.incl. Just use incl.
EDIT2: Another problem I found is this line:
dydx = (r0*np.sin(theta) + r*np.cos(theta))/r0*np.cos(theta) - r*np.sin(theta)
To calculate dydx, you're just dividing dydr by dxdr, but that's not what your code does! You need parens around the denominator like this:
dydx = (r0*np.sin(theta) + r*np.cos(theta))/(r0*np.cos(theta) - r*np.sin(theta))
I am not sure how to take derivatives of the u and v components of the wind in satellite data. I thought I could use numpy.gradient in this way:
from netCDF4 import Dataset
import numpy as np
import matplotlib.pyplot as plt
GridSat = Dataset('analysis_20040713_v11l30flk.nc4','r',format='NETCDF4')
missing_data = -9999.0
lat = GridSat.variables['lat']
lat = lat[:]
lat[np.where(lat==missing_data)] = np.nan
lat[np.where(lat > 90.0)] = np.nan
lon = GridSat.variables['lon']
lon = lon[:]
lon[np.where(lon==missing_data)] = np.nan
uwind_data = GridSat.variables['uwnd']
uwind = GridSat.variables['uwnd'][:]
uwind_sf = uwind_data.scale_factor
uwind_ao = uwind_data.add_offset
miss_uwind = uwind_data.missing_value
uwind[np.where(uwind==miss_uwind)] = np.nan
vwind_data = GridSat.variables['vwnd']
vwind = GridSat.variables['vwnd'][:]
vwind_sf = vwind_data.scale_factor
vwind_ao = vwind_data.add_offset
miss_vwind = vwind_data.missing_value
vwind[np.where(vwind==miss_vwind)] = np.nan
uwind = uwind[2,:,:]
vwind = vwind[2,:,:]
dx = 28400.0 # meters calculated from the 0.25 degree spatial gridding
dy = 28400.0 # meters calculated from the 0.25 degree spatial gridding
dv_dx, dv_dy = np.gradient(vwind, [dx,dy])
du_dx, du_dy = np.gradient(uwind, [dx,dy])
File "<ipython-input-229-c6a5d5b09224>", line 1, in <module>
np.gradient(vwind, [dx,dy])
File "/Users/anaconda/lib/python2.7/site-packages/nump/lib/function_base.py", line 1040, in gradient
out /= dx[axis]
ValueError: operands could not be broadcast together with shapes (628,1440) (2,) (628,1440)
Honestly, I am not sure how to calculate central differences of satellite data with (0.25x0.25) degree spacing. I dont think my dx and dy are correct either. I would really appreciate if someone had a good idea on approaching these types of calculations in satellite data. Thank you!!
As #moarningsun commented, changing how you call np.gradient should correct the ValueError
dv_dx, dv_dy = np.gradient(vwind, dx,dy)
du_dx, du_dy = np.gradient(uwind, dx,dy)
How you got vwind from the file is not particularly important, especially since we don't have access to that file. The shape of vwind would have been useful, though we can guess that from the error message. The reference in the error to a (2,) array is to [dx,dy]. When you get broadcasting errors, check the shapes of the various arguments.
np.gradient code is straight forward, only complicated by the fact that it can handle 1, 2, 3d and higher data. Basically it doing calculations like
(z[:,2:]-z[:,:-2])/2
(z[2:,:]-z[:-2,:])/2
for the inner values, and 1 item steps for the boundary values.
I'll leave the question of deriving a curl from the gradients (or not) to others.
As mentioned, there is the issue of having to implement a discrete curl operator of some kind. This is presumably a routine concern in atmospheric physics so you could check a textbook on that.
Another approach might be to fit a spline to the data so that you can use continuous operations. For example
bspl = scipy.interpolate.SmoothBivariateSpline(x,y,z,s=0)
s here is a smoothing factor which you should play with; if the data are very precise s=0 gives best results; if they have substantial scatter you will want some smoothing.Now you can compute the curl directly:
curl = bspl.integral(x0,x1,y0,y1) / ((x1-x0)*(y1-y0))
EDIT:
The above expression does not give the curl, but the basic idea is sound.
The code below can be ran on Matlab wind dataset, the file wind.mat is in
http://bioinformatics.intec.ugent.be/MotifSuite/INCLUSive_for_users/CPU_64/Matlab_Compiler_Runtime/v79/toolbox/matlab/demos/
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
import scipy.io as sio
def curl(x,y,z,u,v,w):
dx = x[0,:,0]
dy = y[:,0,0]
dz = z[0,0,:]
dummy, dFx_dy, dFx_dz = np.gradient (u, dx, dy, dz, axis=[1,0,2])
dFy_dx, dummy, dFy_dz = np.gradient (v, dx, dy, dz, axis=[1,0,2])
dFz_dx, dFz_dy, dummy = np.gradient (w, dx, dy, dz, axis=[1,0,2])
rot_x = dFz_dy - dFy_dz
rot_y = dFx_dz - dFz_dx
rot_z = dFy_dx - dFx_dy
l = np.sqrt(np.power(u,2.0) + np.power(v,2.0) + np.power(w,2.0));
m1 = np.multiply(rot_x,u)
m2 = np.multiply(rot_y,v)
m3 = np.multiply(rot_z,w)
tmp1 = (m1 + m2 + m3)
tmp2 = np.multiply(l,2.0)
av = np.divide(tmp1, tmp2)
return rot_x, rot_y, rot_z, av
mat = sio.loadmat('wind.mat')
x = mat['x']; y = mat['y']; z = mat['z']
u = mat['u']; v = mat['v']; w = mat['w']
rot_x, rot_y, rot_z, av = curl(x,y,z,u,v,w)
# plot a small area of the wind
i=5;j=7;k=8;S = 3
x1 = x[i-S:i+S, j-S:j+S, k-S:k+S];
y1 = y[i-S:i+S, j-S:j+S, k-S:k+S];
z1 = z[i-S:i+S, j-S:j+S, k-S:k+S];
u1 = u[i-S:i+S, j-S:j+S, k-S:k+S];
v1 = v[i-S:i+S, j-S:j+S, k-S:k+S];
w1 = w[i-S:i+S, j-S:j+S, k-S:k+S];
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.view_init(elev=47, azim=-145)
ax.quiver(x1, y1, z1, u1, v1, w1, length=0.05, color = 'black')
i=5;j=7;k=8;
x0=x[i,j,k]
y0=y[i,j,k]
z0=z[i,j,k]
cx0=rot_x[i,j,k]
cy0=rot_y[i,j,k]
cz0=rot_z[i,j,k]
ax.quiver(x0, y0, z0, 0, cy0, cz0, length=1.0, color = 'blue')
plt.show()