Related
I have multiple GPS Coordinate Points I would like to create a line from in python. The points aren't in a straight line but are exact enough to connect them with straight lines.
I know how to connect one point to another, but not how I would connect multiple of these singular lines to a longer one and then get a point based on the percentage of the whole line.
I use this code to get the percentage of a singular line:
def pointAtPercent(p0, p1, percent):
if p0.x != p1.x:
x = p0.x + percent * (p1.x - p0.x)
else:
x = p0.x;
if p0.y != p1.y:
y = p0.y + percent * (p1.y - p0.y)
else:
y = p0.y
p = point()
p.x = x
p.y = y
return p;
Here is an example list:
[ 10.053417,
53.555737,
10.053206,
53.555748,
10.052497,
53.555763,
10.051125,
53.555757,
10.049193,
53.555756,
10.045511,
53.555762,
10.044863,
53.555767,
10.044319,
53.555763,
10.043685,
53.555769,
10.042765,
53.555759,
10.04201,
53.555756,
10.041919,
53.555757,
10.041904,
53.555766
]
You could create a list of x,y pairs and access the GPS point based on the length of the list:
points = [
10.053417, 53.555737, 10.053206, 53.555748, 10.052497, 53.555763, 10.051125,
53.555757, 10.049193, 53.555756, 10.045511, 53.555762, 10.044863, 53.555767,
10.044319, 53.555763, 10.043685, 53.555769, 10.042765, 53.555759, 10.04201,
53.555756, 10.041919, 53.555757, 10.041904, 53.555766
]
points = [(points[i], points[i + 1]) for i in range(0, len(points) - 1, 2)]
def pointAtPercent(points, percent):
lstIndex = int(
len(points) / 100. * percent
) # possibly creates some rounding issues !
print(points[lstIndex - 1])
pointAtPercent(points, 10)
pointAtPercent(points, 27.5)
pointAtPercent(points, 50)
pointAtPercent(points, 100)
Out:
(10.053417, 53.555737)
(10.052497, 53.555763)
(10.045511, 53.555762)
(10.041904, 53.555766)
The basic algorithm is this:
Determine the lengths of each segment. You are in spherical polar coordinates (assuming the Earth is a sphere, which it isn't (see WGS 84 if you need more precision)), so you can do this.
def great_circle_distance(lat_0, lon_0, lat_1, lon_1):
return math.acos(
math.sin(lat_0) * math.sin(lat_1)
+ math.cos(lat_0) * math.cos(lat_1) * math.cos(lon_1 - lon_0)
)
radian_points = [(math.radians(p.x), math.radians(p.y)) for p in points]
lengths = [
great_circle_distance(*p0, *p1) for p0, p1 in zip(radian_points, radian_points[1:])
]
path_length = sum(lengths)
Given a percentage, you can work out how far along the path it is.
distance_along = percentage * path_length
Find the index of the correct segment.
# Inefficient but easy (consider bisect.bisect with a stored list of sums)
index = max(
next(i for i in range(len(lengths) + 1) if sum(lengths[:i]) >= distance_along) - 1,
0,
)
Then use your original algorithm.
point = pointAtPercent(
points[index],
points[index + 1],
(distance_along - sum(lengths[:index])) / lengths[index],
)
Given a line with coordinates 'start' and 'end' and the coordinates of a point 'pnt' find the shortest distance from pnt to the line. I have tried the below code.
import math
def dot(v,w):
x,y,z = v
X,Y,Z = w
return x*X + y*Y + z*Z
def length(v):
x,y,z = v
return math.sqrt(x*x + y*y + z*z)
def vector(b,e):
x,y,z = b
X,Y,Z = e
return (X-x, Y-y, Z-z)
def unit(v):
x,y,z = v
mag = length(v)
return (x/mag, y/mag, z/mag)
def distance(p0,p1):
return length(vector(p0,p1))
def scale(v,sc):
x,y,z = v
return (x * sc, y * sc, z * sc)
def add(v,w):
x,y,z = v
X,Y,Z = w
return (x+X, y+Y, z+Z)
def pnt2line(pnt, start, end):
line_vec = vector(start, end)
pnt_vec = vector(start, pnt)
line_len = length(line_vec)
line_unitvec = unit(line_vec)
pnt_vec_scaled = scale(pnt_vec, 1.0/line_len)
t = dot(line_unitvec, pnt_vec_scaled)
if t < 0.0:
t = 0.0
elif t > 1.0:
t = 1.0
nearest = scale(line_vec, t)
dist = distance(nearest, pnt_vec)
nearest = add(nearest, start)
return (dist, nearest)
The resolution can be explained by the figure, which shows the locus of the points at a given distance from a segment. It is made of two half circles and two line segments, which are separated by the two perpendiculars at the endpoints.
We can simplify the discussion by ensuring that the segment is in a canonical position, with endpoints (0, 0) and (L, 0). For any segment, we can apply a similarity transformation to bring it in the canonical position (see below), and move the target point accordingly.
Now the computation of the distance amounts to
X < 0 -> √[X² + Y²]
0 ≤ X ≤ L -> |Y|
L < X -> √[(X-L)² + Y²]
Subtract the coordinates of one endpoint from all points to bring the segment to the origin.
Compute the length L.
Normalize the vector to the second endpoint to obtain a unit vector, let U.
Transform the target point with X' = Ux.X + Uy.Y, Y' = Ux.Y - Uy.X.
Technical remark:
The geometric analysis proves that the output function is the square root of a piecewise quadratic function and it takes one or two comparisons to tell the active piece and this cannot be avoided. If I am right, the algebraic expressions cannot be much simpified.
I have this code that have three function called ( Bx,Bhalo, Bdisk)these three the functions only accept arrays (e.g. shape (1000)):
import numpy as np
import logging
import warnings
import gmf
signum = lambda x: (x < 0.) * -1. + (x >= 0) * 1.
pi = np.pi
#Class with analytical functions that describe the GMF according to the model of JF12
class GMF(object):
def __init__(self): # self:is automatically set to reference the newly created object that needs to be initialized
self.Rsun = -8.5 # position of the sun along the x axis in kpc
############################################################################
# Disk Parameters
############################################################################
self.bring, self.bring_unc = 0.1,0.1 # floats, field strength in ring at 3 kpc < r < 5 kpc
self.hdisk, self.hdisk_unc = 0.4, 0.03 # float, disk/halo transition height
self.wdisk, self.wdisk_unc = 0.27,0.08 # floats, transition width
self.b = np.array([0.1,3.,-0.9,-0.8,-2.0,-4.2,0.,2.7]) # (8,1)-dim np.arrays, field strength of spiral arms at 5 kpc
self.b_unc = np.array([1.8,0.6,0.8,0.3,0.1,0.5,1.8,1.8]) # uncertainty
self.rx = np.array([5.1,6.3,7.1,8.3,9.8,11.4,12.7,15.5])# (8,1)-dim np.array,dividing lines of spiral lines coordinates of neg. x-axes that intersect with arm
self.idisk = 11.5 * pi/180. # float, spiral arms pitch angle
#############################################################################
# Halo Parameters
#############################################################################
self.Bn, self.Bn_unc = 1.4,0.1 # floats, field strength northern halo
self.Bs, self.Bs_unc = -1.1,0.1 # floats, field strength southern halo
self.rn, self.rn_unc = 9.22,0.08 # floats, transition radius south, lower limit
self.rs, self.rs_unc = 16.7,0. # transition radius south, lower limit
self.whalo, self.whalo_unc = 0.2,0.12 # floats, transition width
self.z0, self.z0_unc = 5.3, 1.6 # floats, vertical scale height
##############################################################################
# Out of plaxe or "X" component Parameters
##############################################################################
self.BX0, self.BX_unc = 4.6,0.3 # floats, field strength at origin
self.ThetaX0, self.ThetaX0_unc = 49. * pi/180., pi/180. # elev. angle at z = 0, r > rXc
self.rXc, self.rXc_unc = 4.8, 0.2 # floats, radius where thetaX = thetaX0
self.rX, self.rX_unc = 2.9, 0.1 # floats, exponential scale length
# striated field
self.gamma, self.gamma_unc = 2.92,0.14 # striation and/or rel. elec. number dens. rescaling
return
##################################################################################
##################################################################################
# Transition function given by logistic function eq.5
##################################################################################
def L(self,z,h,w):
if np.isscalar(z):
z = np.array([z]) # scalar or numpy array with positions (height above disk, z; distance from center, r)
ones = np.ones(z.shape[0])
return 1./(ones + np.exp(-2. *(np.abs(z)- h)/w))
####################################################################################
# return distance from center for angle phi of logarithmic spiral
# r(phi) = rx * exp(b * phi) as np.array
####################################################################################
def r_log_spiral(self,phi):
if np.isscalar(phi): #Returns True if the type of num is a scalar type.
phi = np.array([phi])
ones = np.ones(phi.shape[0])
# self.rx.shape = 8
# phi.shape = p
# then result is given as (8,p)-dim array, each row stands for one rx
# vstack : Take a sequence of arrays and stack them vertically to make a single array
# tensordot(a, b, axes=2):Compute tensor dot product along specified axes for arrays >=1D.
result = np.tensordot(self.rx , np.exp((phi - 3.*pi*ones) / np.tan(pi/2. - self.idisk)),axes = 0)
result = np.vstack((result, np.tensordot(self.rx , np.exp((phi - pi*ones) / np.tan(pi/2. - self.idisk)),axes = 0) ))
result = np.vstack((result, np.tensordot(self.rx , np.exp((phi + pi*ones) / np.tan(pi/2. - self.idisk)),axes = 0) ))
return np.vstack((result, np.tensordot(self.rx , np.exp((phi + 3.*pi*ones) / np.tan(pi/2. - self.idisk)),axes = 0) ))
#############################################################################################
# Disk component in galactocentric cylindrical coordinates (r,phi,z)
#############################################################################################
def Bdisk(self,r,phi,z):
# Bdisk is purely azimuthal (toroidal) with the field strength b_ring
"""
r: N-dim np.array, distance from origin in GC cylindrical coordinates, is in kpc
z: N-dim np.array, height in kpc in GC cylindrical coordinates
phi:N-dim np.array, polar angle in GC cylindircal coordinates, in radian
Bdisk: (3,N)-dim np.array with (r,phi,z) components of disk field for each coordinate tuple
Bdisk|: N-dim np.array, absolute value of Bdisk for each coordinate tuple
"""
if (not r.shape[0] == phi.shape[0]) and (not z.shape[0] == phi.shape[0]):
warnings.warn("List do not have equal shape! returning -1", RuntimeWarning)
return -1
# Return a new array of given shape and type, filled with zeros.
Bdisk = np.zeros((3,r.shape[0])) # Bdisk vector in r, phi, z
ones = np.ones(r.shape[0])
r_center = (r >= 3.) & (r < 5.1)
r_disk = (r >= 5.1) & (r <= 20.)
Bdisk[1,r_center] = self.bring
# Determine in which arm we are
# this is done for each coordinate individually
if np.sum(r_disk):
rls = self.r_log_spiral(phi[r_disk])
rls = np.abs(rls - r[r_disk])
arms = np.argmin(rls, axis = 0) % 8
# The magnetic spiral defined at r=5 kpc and fulls off as 1/r ,the field direction is given by:
Bdisk[0,r_disk] = np.sin(self.idisk)* self.b[arms] * (5. / r[r_disk])
Bdisk[1,r_disk] = np.cos(self.idisk)* self.b[arms] * (5. / r[r_disk])
Bdisk *= (ones - self.L(z,self.hdisk,self.wdisk)) # multiplied by (1-L)
return Bdisk, np.sqrt(np.sum(Bdisk**2.,axis = 0)) # the Bdisk, the normalization
# axis=0 : sum over index 0(row)
# axis=1 : sum over index 1(columns)
##############################################################################################
# Halo component
###############################################################################################
def Bhalo(self,r,z):
# Bhalo is purely azimuthal (toroidal), i.e. has only a phi component
if (not r.shape[0] == z.shape[0]):
warnings.warn("List do not have equal shape! returning -1", RuntimeWarning)
return -1
Bhalo = np.zeros((3,r.shape[0])) # Bhalo vector in r, phi, z rows: r, phi and z component
ones = np.ones(r.shape[0])
m = ( z != 0. )
# SEE equation 6.
Bhalo[1,m] = np.exp(-np.abs(z[m])/self.z0) * self.L(z[m], self.hdisk, self.wdisk) * \
( self.Bn * (ones[m] - self.L(r[m], self.rn, self.whalo)) * (z[m] > 0.) \
+ self.Bs * (ones[m] - self.L(r[m], self.rs, self.whalo)) * (z[m] < 0.) )
return Bhalo , np.sqrt(np.sum(Bhalo**2.,axis = 0))
##############################################################################################
# BX component (OUT OF THE PLANE)
###############################################################################################
def BX(self,r,z):
#BX is purely ASS and poloidal, i.e. phi component = 0
if (not r.shape[0] == z.shape[0]):
warnings.warn("List do not have equal shape! returning -1", RuntimeWarning)
return -1
BX= np.zeros((3,r.shape[0])) # BX vector in r, phi, z rows: r, phi and z component
m = np.sqrt(r**2. + z**2.) >= 1.
bx = lambda r_p: self.BX0 * np.exp(-r_p / self.rX) # eq.7
thetaX = lambda r,z,r_p: np.arctan(np.abs(z)/(r - r_p)) # eq.10
r_p = r[m] *self.rXc/(self.rXc + np.abs(z[m] ) / np.tan(self.ThetaX0)) # eq. 9
m_r_b = r_p > self.rXc # region with constant elevation angle
m_r_l = r_p <= self.rXc # region with varying elevation angle
theta = np.zeros(z[m].shape[0])
b = np.zeros(z[m].shape[0])
r_p0 = (r[m])[m_r_b] - np.abs( (z[m])[m_r_b] ) / np.tan(self.ThetaX0) # eq.8
b[m_r_b] = bx(r_p0) * r_p0/ (r[m])[m_r_b] # the field strength in the constant elevation angle (b_x(r_p)r_p/r)
theta[m_r_b] = self.ThetaX0 * np.ones(theta.shape[0])[m_r_b]
b[m_r_l] = bx(r_p[m_r_l]) * (r_p[m_r_l]/(r[m])[m_r_l] )**2. # the field strength with varying elevation angle (b_x(r_p)(r_p/r)**2)
theta[m_r_l] = thetaX((r[m])[m_r_l] ,(z[m])[m_r_l] ,r_p[m_r_l])
mz = (z[m] == 0.)
theta[mz] = np.pi/2.
BX[0,m] = b * (np.cos(theta) * (z[m] >= 0) + np.cos(pi*np.ones(theta.shape[0]) - theta) * (z[m] < 0))
BX[2,m] = b * (np.sin(theta) * (z[m] >= 0) + np.sin(pi*np.ones(theta.shape[0]) - theta) * (z[m] < 0))
return BX, np.sqrt(np.sum(BX**2.,axis=0))
now I want to add these three function together; Btotal= Bx+ Bhalo+ Bdisk,these function is a vector field in cylindrical coordinate (r,theta,z), I convert from cylindrical to Cartesian(x,y,z) and I defined a function called vectors to calculate new two vector called n1,n2 (to get two perpendicular vector to Btotal).to calculate the diffusion of particle using stochastic differential equations for many particles(~10000) in the code below, I am trying to call the three functions (form above code) by defined r, theta, z to use it the diffusion equation and plot the result
i.e:
in the for loop I have to get one position in shape (3,1)[I put the range from (0,1)] in (r,theta,z) then convert the value to Cartesian(x,y,z) to use it to find n1,n2,d eltaX,deltaY,deltaZ, respectively! :
import scipy as sp
import numpy as np
import numpy.random as npr
from numpy.lib.scimath import logn # to logartnim scale
import math as math
from random import seed,random, choice
from pylab import *
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import gmf
###########################################################
gmfm = gmf.GMF()
#############################################################
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.set_xlabel(r'$\ X$',size=16)
ax.set_ylabel(r'$\ Y$',size=16)
ax.set_zlabel(r'$\ Z$',size=16)
ax.set_title(' Diffusion Particles in GMF in 3D ')
#############################################################
def vectors(b):
b = b/np.sqrt(np.sum(b**2.,axis=0))
b = b/np.linalg.norm(b)
z = np.array([0.,0.,1.])
n1 = np.cross(z,b,axis=0)
n1 = n1/np.linalg.norm(n1)
n2 = np.cross(b,n1,axis=0)
n2 = n2/np.linalg.norm(n2)
return n1,n2
#############################################################
def CylindricalToCartesian(r,theta,z):
x= r*np.cos(theta)
y= r*np.sin(theta)
z= z
return np.array([x, y, z])
############################################################
T=1 # 100
N=10000
dt=float(T)/N
D= 1 # 2
DII=10
n= 1000
seed(3)
finalpositions=[]
###############################################################
for number in range(0,1):
finalpositions.append([])
r=[]
theta=[]
z=[]
r.append(0)
theta.append(0)
z.append(0)
x=[]
y=[]
x.append(8.5)
y.append(0)
for i in range(n):
Bdisk, Babs_d = gmfm.Bdisk(r,theta,z)
Bhalo, Babs_h = gmfm.Bhalo(r,z)
BX, Babs_x = gmfm.BX(r,z)
Btotal = Bdisk + Bhalo + BX
FieldInXYZ = CylindricalToCartesian(r[-1],theta[-1],z[-1])
FieldInXYZ =Btotal(x[-1],y[-1],z[-1])
localB = Btotal(x[-1],y[-1],z[-1])
print 'FieldInXYZ:', FieldInXYZ
#print 'localB:',localB
n1, n2 = vectors(localB)
s = np.random.normal(0, 1, 3)
finalpositions[-1].append(x)
finalpositions[-1].append(y)
finalpositions[-1].append(z)
allxes = []
allyes = []
allzes = []
for p in finalpositions:
allxes.append(p[0][-1])
allyes.append(p[1][-1])
allzes.append(p[1][-1])
plt.plot(allxes, allyes,allzes, 'o')
plt.show()
but I am getting an error:
AttributeError Traceback (most recent call last)
/usr/lib/python2.7/dist-packages/IPython/utils/py3compat.pyc in execfile(fname, *where)
202 else:
203 filename = fname
--> 204 __builtin__.execfile(filename, *where)
/home/January.py in <module>()
71 for i in range(n):
72
---> 73 Bdisk, Babs_d = gmfm.Bdisk(r,theta,z)
74 Bhalo, Babs_h = gmfm.Bhalo(r,z)
75 BX, Babs_x = gmfm.BX(r,z)
/home/gmf.py in Bdisk(self, r, phi, z)
80 Bdisk|: N-dim np.array, absolute value of Bdisk for each coordinate tuple
81 """
---> 82 if (not r.shape[0] == phi.shape[0]) and (not z.shape[0] == phi.shape[0]):
83 warnings.warn("List do not have equal shape! returning -1", RuntimeWarning)
84 return -1
AttributeError: 'list' object has no attribute 'shape'
I don't know what I did wrong?! Any help would be appreciate
The traceback tells you exactly where the problem lives: you are using r.shape where type(r) is list while it should be numpy.ndarray instead. Tracing the problem back reveals that you declare r = [] and then you pass r to gmfm.Bdisk.
Instead you could do:
Bdisk, Babs_d = gmfm.Bdisk(numpy.ndarray(r),theta,z)
(line number 73); (If you have other lists that you treat as numpy.ndarrays then you need to convert them accordingly of course.)
I need to convert latitude and longitude values to a point in the 3-dimensional space. I've been trying this for about 2 hours now, but I do not get the correct results.
The Equirectangular coordinates come from openflights.org. I've tried several combinations of cos and sin, but the result did never look like our little beloved earth.
In the following, you can see the result of applying the conversion Wikipedia suggests. I think one can guess from context what c4d.Vector is.
def llarToWorld(latit, longit, altid, rad):
x = math.sin(longit) * math.cos(latit)
z = math.sin(longit) * math.sin(latit)
y = math.cos(longit)
v = c4d.Vector(x, y, z)
v = v * altid + v * rad
return v
Red: X, Green: Y, Blue: Z
One can indeed identify North- and South America, especially the land around the Gulf of Mexico. However, it looks somewhat squished and kind of in the wrong place..
As the result looks somewhat rotated, I think, I tried swapping latitude and longitude. But that result is somewhat awkward.
def llarToWorld(latit, longit, altid, rad):
temp = latit
latit = longit
longit = temp
x = math.sin(longit) * math.cos(latit)
z = math.sin(longit) * math.sin(latit)
y = math.cos(longit)
v = c4d.Vector(x, y, z)
v = v * altid + v * rad
return v
This is what the result looks like without converting the values.
def llarToWorld(latit, longit, altid, rad):
return c4d.Vector(math.degrees(latit), math.degrees(longit), altid)
Question: How can I convert the longitude and latitude correctly?
Solution
Thanks to TreyA, I found this page on mathworks.com. The code that does it's work is the following:
def llarToWorld(lat, lon, alt, rad):
# see: http://www.mathworks.de/help/toolbox/aeroblks/llatoecefposition.html
f = 0 # flattening
ls = atan((1 - f)**2 * tan(lat)) # lambda
x = rad * cos(ls) * cos(lon) + alt * cos(lat) * cos(lon)
y = rad * cos(ls) * sin(lon) + alt * cos(lat) * sin(lon)
z = rad * sin(ls) + alt * sin(lat)
return c4d.Vector(x, y, z)
Actually, I switched y and z because the earth was rotated then, however, it works! That's the result:
I've reformatted the code that was previously mentioned here, but more importantly you have left out some of the equations mentioned in the link provided by Niklas R
def LLHtoECEF(lat, lon, alt):
# see http://www.mathworks.de/help/toolbox/aeroblks/llatoecefposition.html
rad = np.float64(6378137.0) # Radius of the Earth (in meters)
f = np.float64(1.0/298.257223563) # Flattening factor WGS84 Model
cosLat = np.cos(lat)
sinLat = np.sin(lat)
FF = (1.0-f)**2
C = 1/np.sqrt(cosLat**2 + FF * sinLat**2)
S = C * FF
x = (rad * C + alt)*cosLat * np.cos(lon)
y = (rad * C + alt)*cosLat * np.sin(lon)
z = (rad * S + alt)*sinLat
return (x, y, z)
Comparison output: finding ECEF for Los Angeles, CA (34.0522, -118.40806, 0 elevation)
My code:
X = -2516715.36114 meters or -2516.715 km
Y = -4653003.08089 meters or -4653.003 km
Z = 3551245.35929 meters or 3551.245 km
Your Code:
X = -2514072.72181 meters or -2514.072 km
Y = -4648117.26458 meters or -4648.117 km
Z = 3571424.90261 meters or 3571.424 km
Although in your earth rotation environment your function will produce right geographic region for display, it will NOT give the right ECEF equivalent coordinates. As you can see some of the parameters vary by as much as 20 KM which is rather a large error.
Flattening factor, f depends on the model you assume for your conversion. Typical, model is WGS 84; however, there are other models.
Personally, I like to use this link to Naval Postgraduate School for sanity checks on my conversions.
you're not doing what wikipedia suggests. read it again carefully.
they say:
x = r cos(phi) sin(theta)
y = r sin(phi) sin(theta)
z = r cos(theta)
and then:
theta == latitude
phi == longitude
and, in your case, r = radius + altitude
so you should be using:
r = radius + altitude
x = r cos(long) sin(lat)
y = r sin(long) sin(lat)
z = r cos(lat)
note that the final entry is cos(lat) (you are using longitude).
As TreyA statet, LLA to ECEF is the solution. See http://www.mathworks.de/help/toolbox/aeroblks/llatoecefposition.html
I have a R data.frame containing longitude, latitude which spans over the entire USA map. When X number of entries are all within a small geographic region of say a few degrees longitude & a few degrees latitude, I want to be able to detect this and then have my program then return the coordinates for the geographic bounding box. Is there a Python or R CRAN package that already does this? If not, how would I go about ascertaining this information?
I was able to combine Joran's answer along with Dan H's comment. This is an example ouput:
The python code emits functions for R: map() and rect(). This USA example map was created with:
map('state', plot = TRUE, fill = FALSE, col = palette())
and then you can apply the rect()'s accordingly from with in the R GUI interpreter (see below).
import math
from collections import defaultdict
to_rad = math.pi / 180.0 # convert lat or lng to radians
fname = "site.tsv" # file format: LAT\tLONG
threshhold_dist=50 # adjust to your needs
threshhold_locations=15 # minimum # of locations needed in a cluster
def dist(lat1,lng1,lat2,lng2):
global to_rad
earth_radius_km = 6371
dLat = (lat2-lat1) * to_rad
dLon = (lng2-lng1) * to_rad
lat1_rad = lat1 * to_rad
lat2_rad = lat2 * to_rad
a = math.sin(dLat/2) * math.sin(dLat/2) + math.sin(dLon/2) * math.sin(dLon/2) * math.cos(lat1_rad) * math.cos(lat2_rad)
c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a));
dist = earth_radius_km * c
return dist
def bounding_box(src, neighbors):
neighbors.append(src)
# nw = NorthWest se=SouthEast
nw_lat = -360
nw_lng = 360
se_lat = 360
se_lng = -360
for (y,x) in neighbors:
if y > nw_lat: nw_lat = y
if x > se_lng: se_lng = x
if y < se_lat: se_lat = y
if x < nw_lng: nw_lng = x
# add some padding
pad = 0.5
nw_lat += pad
nw_lng -= pad
se_lat -= pad
se_lng += pad
# sutiable for r's map() function
return (se_lat,nw_lat,nw_lng,se_lng)
def sitesDist(site1,site2):
#just a helper to shorted list comprehension below
return dist(site1[0],site1[1], site2[0], site2[1])
def load_site_data():
global fname
sites = defaultdict(tuple)
data = open(fname,encoding="latin-1")
data.readline() # skip header
for line in data:
line = line[:-1]
slots = line.split("\t")
lat = float(slots[0])
lng = float(slots[1])
lat_rad = lat * math.pi / 180.0
lng_rad = lng * math.pi / 180.0
sites[(lat,lng)] = (lat,lng) #(lat_rad,lng_rad)
return sites
def main():
sites_dict = {}
sites = load_site_data()
for site in sites:
#for each site put it in a dictionary with its value being an array of neighbors
sites_dict[site] = [x for x in sites if x != site and sitesDist(site,x) < threshhold_dist]
results = {}
for site in sites:
j = len(sites_dict[site])
if j >= threshhold_locations:
coord = bounding_box( site, sites_dict[site] )
results[coord] = coord
for bbox in results:
yx="ylim=c(%s,%s), xlim=c(%s,%s)" % (results[bbox]) #(se_lat,nw_lat,nw_lng,se_lng)
print('map("county", plot=T, fill=T, col=palette(), %s)' % yx)
rect='rect(%s,%s, %s,%s, col=c("red"))' % (results[bbox][2], results[bbox][0], results[bbox][3], results[bbox][2])
print(rect)
print("")
main()
Here is an example TSV file (site.tsv)
LAT LONG
36.3312 -94.1334
36.6828 -121.791
37.2307 -121.96
37.3857 -122.026
37.3857 -122.026
37.3857 -122.026
37.3895 -97.644
37.3992 -122.139
37.3992 -122.139
37.402 -122.078
37.402 -122.078
37.402 -122.078
37.402 -122.078
37.402 -122.078
37.48 -122.144
37.48 -122.144
37.55 126.967
With my data set, the output of my python script, shown on the USA map. I changed the colors for clarity.
rect(-74.989,39.7667, -73.0419,41.5209, col=c("red"))
rect(-123.005,36.8144, -121.392,38.3672, col=c("green"))
rect(-78.2422,38.2474, -76.3,39.9282, col=c("blue"))
Addition on 2013-05-01 for Yacob
These 2 lines give you the over all goal...
map("county", plot=T )
rect(-122.644,36.7307, -121.46,37.98, col=c("red"))
If you want to narrow in on a portion of a map, you can use ylim and xlim
map("county", plot=T, ylim=c(36.7307,37.98), xlim=c(-122.644,-121.46))
# or for more coloring, but choose one or the other map("country") commands
map("county", plot=T, fill=T, col=palette(), ylim=c(36.7307,37.98), xlim=c(-122.644,-121.46))
rect(-122.644,36.7307, -121.46,37.98, col=c("red"))
You will want to use the 'world' map...
map("world", plot=T )
It has been a long time since I have used this python code I have posted below so I will try my best to help you.
threshhold_dist is the size of the bounding box, ie: the geographical area
theshhold_location is the number of lat/lng points needed with in
the bounding box in order for it to be considered a cluster.
Here is a complete example. The TSV file is located on pastebin.com. I have also included an image generated from R that contains the output of all of the rect() commands.
# pyclusters.py
# May-02-2013
# -John Taylor
# latlng.tsv is located at http://pastebin.com/cyvEdx3V
# use the "RAW Paste Data" to preserve the tab characters
import math
from collections import defaultdict
# See also: http://www.geomidpoint.com/example.html
# See also: http://www.movable-type.co.uk/scripts/latlong.html
to_rad = math.pi / 180.0 # convert lat or lng to radians
fname = "latlng.tsv" # file format: LAT\tLONG
threshhold_dist=20 # adjust to your needs
threshhold_locations=20 # minimum # of locations needed in a cluster
earth_radius_km = 6371
def coord2cart(lat,lng):
x = math.cos(lat) * math.cos(lng)
y = math.cos(lat) * math.sin(lng)
z = math.sin(lat)
return (x,y,z)
def cart2corrd(x,y,z):
lon = math.atan2(y,x)
hyp = math.sqrt(x*x + y*y)
lat = math.atan2(z,hyp)
return(lat,lng)
def dist(lat1,lng1,lat2,lng2):
global to_rad, earth_radius_km
dLat = (lat2-lat1) * to_rad
dLon = (lng2-lng1) * to_rad
lat1_rad = lat1 * to_rad
lat2_rad = lat2 * to_rad
a = math.sin(dLat/2) * math.sin(dLat/2) + math.sin(dLon/2) * math.sin(dLon/2) * math.cos(lat1_rad) * math.cos(lat2_rad)
c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a));
dist = earth_radius_km * c
return dist
def bounding_box(src, neighbors):
neighbors.append(src)
# nw = NorthWest se=SouthEast
nw_lat = -360
nw_lng = 360
se_lat = 360
se_lng = -360
for (y,x) in neighbors:
if y > nw_lat: nw_lat = y
if x > se_lng: se_lng = x
if y < se_lat: se_lat = y
if x < nw_lng: nw_lng = x
# add some padding
pad = 0.5
nw_lat += pad
nw_lng -= pad
se_lat -= pad
se_lng += pad
#print("answer:")
#print("nw lat,lng : %s %s" % (nw_lat,nw_lng))
#print("se lat,lng : %s %s" % (se_lat,se_lng))
# sutiable for r's map() function
return (se_lat,nw_lat,nw_lng,se_lng)
def sitesDist(site1,site2):
# just a helper to shorted list comprehensioin below
return dist(site1[0],site1[1], site2[0], site2[1])
def load_site_data():
global fname
sites = defaultdict(tuple)
data = open(fname,encoding="latin-1")
data.readline() # skip header
for line in data:
line = line[:-1]
slots = line.split("\t")
lat = float(slots[0])
lng = float(slots[1])
lat_rad = lat * math.pi / 180.0
lng_rad = lng * math.pi / 180.0
sites[(lat,lng)] = (lat,lng) #(lat_rad,lng_rad)
return sites
def main():
color_list = ( "red", "blue", "green", "yellow", "orange", "brown", "pink", "purple" )
color_idx = 0
sites_dict = {}
sites = load_site_data()
for site in sites:
#for each site put it in a dictionarry with its value being an array of neighbors
sites_dict[site] = [x for x in sites if x != site and sitesDist(site,x) < threshhold_dist]
print("")
print('map("state", plot=T)') # or use: county instead of state
print("")
results = {}
for site in sites:
j = len(sites_dict[site])
if j >= threshhold_locations:
coord = bounding_box( site, sites_dict[site] )
results[coord] = coord
for bbox in results:
yx="ylim=c(%s,%s), xlim=c(%s,%s)" % (results[bbox]) #(se_lat,nw_lat,nw_lng,se_lng)
# important!
# if you want an individual map for each cluster, uncomment this line
#print('map("county", plot=T, fill=T, col=palette(), %s)' % yx)
if len(color_list) == color_idx:
color_idx = 0
rect='rect(%s,%s, %s,%s, col=c("%s"))' % (results[bbox][2], results[bbox][0], results[bbox][3], results[bbox][1], color_list[color_idx])
color_idx += 1
print(rect)
print("")
main()
I'm doing this on a regular basis by first creating a distance matrix and then running clustering on it. Here is my code.
library(geosphere)
library(cluster)
clusteramounts <- 10
distance.matrix <- (distm(points.to.group[,c("lon","lat")]))
clustersx <- as.hclust(agnes(distance.matrix, diss = T))
points.to.group$group <- cutree(clustersx, k=clusteramounts)
I'm not sure if it completely solves your problem. You might want to test with different k, and also perhaps do a second run of clustering of some of the first clusters in case they are too big, like if you have one point in Minnesota and a thousand in California.
When you have the points.to.group$group, you can get the bounding boxes by finding max and min lat lon per group.
If you want X to be 20, and you have 18 points in New York and 22 in Dallas, you must decide if you want one small and one really big box (20 points each), if it is better to have have the Dallas box include 22 points, or if you want to split the 22 points in Dallas to two groups. Clustering based on distance can be good in some of these cases. But it of course depend on why you want to group the points.
/Chris
A few ideas:
Ad-hoc & approximate: The "2-D histogram". Create arbitrary "rectangular" bins, of the degree width of your choice, assign each bin an ID. Placing a point in a bin means "associate the point with the ID of the bin". Upon each add to a bin, ask the bin how many points it has. Downside: doesn't correctly "see" a cluster of points that stradle a bin boundary; and: bins of "constant longitudinal width" actually are (spatially) smaller as you move north.
Use the "Shapely" library for Python. Follow it's stock example for "buffering points", and do a cascaded union of the buffers. Look for globs over a certain area, or that "contain" a certain number of original points. Note that Shapely is not intrinsically "geo-savy", so you'll have to add corrections if you need them.
Use a true DB with spatial processing. MySQL, Oracle, Postgres (with PostGIS), MSSQL all (I think) have "Geometry" and "Geography" datatypes, and you can do spatial queries on them (from your Python scripts).
Each of these has different costs in dollars and time (in the learning curve)... and different degrees of geospatial accuracy. You have to pick what suits your budget and/or requirements.
if you use shapely, you could extend my cluster_points function
to return the bounding box of the cluster via the .bounds property of the shapely geometry , for example like this:
clusterlist.append(cluster, (poly.buffer(-b)).bounds)
maybe something like
def dist(lat1,lon1,lat2,lon2):
#just return normal x,y dist
return sqrt((lat1-lat2)**2+(lon1-lon2)**2)
def sitesDist(site1,site2):
#just a helper to shorted list comprehensioin below
return dist(site1.lat,site1.lon,site2.lat,site2.lon)
sites_dict = {}
threshhold_dist=5 #example dist
for site in sites:
#for each site put it in a dictionarry with its value being an array of neighbors
sites_dict[site] = [x for x in sites if x != site and sitesDist(site,x) < threshhold_dist]
print "\n".join(sites_dict)