Develop Reference

Get New GPS with Known start GPS, bearing and distance in feet

I am brand new to Python in general and have a limited but aged knowledge of Javascript. I am trying to accomplish a small project using Python, but I have gotten stuck with my own comprehension.
I have a GPS coordinate in decimal degrees as a starting point (30.456025341663068, -86.41408883615411), a distance in feet (86 feet) from the start point to the expected endpoint and I have a bearing in degrees (0 to 360) from start point to endpoint. Given these values, I am attempting to simply return the GPS coordinate in decimal degrees of the endpoint.
I found one StackExchange post that appears to give me a viable option using GeoPy, but I'm simply getting lost in trying to implement it. Can someone assist with how to accomplish this? GeoPy is not a requirement for me, I just imported it based on the answer in the other StackExchange question:
calculating a gps coordinate given a point, bearing and distance

Perhaps you could do something like this:
import geopy
import geopy.distance
lat = 30.456025341663068
lon = -86.41408883615411
distance_ft = 86
bearing = 0
start_point = geopy.Point(lat, lon)
end_point = geopy.distance.geodesic(feet=distance_ft).destination(start_point, bearing)
print(end_point.latitude, end_point.longitude)
This should output something like:
30.456261790886277 -86.41408883615411
You can then also use the geodesic method to calculate the distance between the points:
print(geopy.distance.geodesic(start_point, end_point).feet)
And get something like:
86.0000000020017

Alternative answer which is not using geopy
from pyproj import Geod
g = Geod(ellps="WGS84")
lat = 30.456025341663068
long = -86.41408883615411
feet_in_meters = .3048
distance_in_feet = 86
distance_in_meters = distance_in_feet * feet_in_meters
long_destination, lat_destination, azi_back = g.fwd(lons=long, lats=lat, az=0, dist=distance_in_meters)
which print(lat_destination, long_destination)
30.456261790886273 -86.41408883615411

Calculate the distance between each GPS points of two lists in python

Let assume that we have a pandas dataframe contain of two columns as ("longitude" and "latitude"), which split by (comma) for example:
longitude
latitude
[116.415642, 116.41832, 116.418976, 116.419029, 116.421791, 116.426666, 116.429077, 116.43174, 116.434334, 116.436806]
[39.897133, 39.897213, 39.898766, 39.900906, 39.900486, 39.90062, 39.900681, 39.900738, 39.900749, 39.900818]
I am stuck with how can I calculate the distance between each long/lat point to get a new column as distance gap (KM) between each of them as (''gap_dist''). The outputs should be like:
I have found numerous tutorials just to find the distance between a single pair of coordinates (source and destination). So what is the best way to achieve this in python?.
Thanks in advance

if you want the gap between each point and the first point, it goes as this:
from math import dist
lats = [float(lat) for lat in df['latitude'][0]]
longs = [float(long) for long in df['longtitude'][0]]
p0 = [lats[0], longs[0]]
dists = [dist(p0, [x,y]) for x, y in zip(lats, longs)]

Clustering latitude longitude points in Python with fixed number of clusters

kmeans does not work properly for geospatial coordinates - even when changing the distance function to haversine as stated here.
I had a look at DBSCAN which doesn
t let me set a fixed number of clusters.
Is there any algorithm (in python if possible) that has the same input values as kmeans? or
Can I easily convert latitude, longitude to euclidean coordinates (x,y,z) as done here and do the calculation on my data?
It does not have to perfectly accurate, but it would nice if it would.

Using just lat and longitude leads to problems when your geo data spans a large area. Especially since the distance between longitudes is less near the poles. To account for this it is good practice to first convert lon and lat to cartesian coordinates.
If your geo data spans the united states for example you could define an origin from which to calculate distance from as the center of the contiguous united states. I believe this is located at Latitude 39 degrees 50 minutes and Longitude 98 degrees 35 minute.
TO CONVERT lat lon to CARTESIAN coordinates- calculate the distance using haversine, from every location in your dataset to the defined origin. Again, I suggest Latitude 39 degrees 50 minutes and Longitude 98 degrees 35 minute.
You can use haversine in python to calculate these distances:
from haversine import haversine
origin = (39.50, 98.35)
paris = (48.8567, 2.3508)
haversine(origin, paris, miles=True)
Now you can use k-means on this data to cluster, assuming the haversin model of the earth is adequate for your needs. If you are doing data analysis and not planning on launching a satellite I think this should be okay.

Have you tried kmeans? The issue raised in the linked question seems to be with points that are close to 180 degrees. If your points are all close enough together (like in the same city or country for example) then kmeans might work OK for you.

Find the intersection between two geographical data points

I have two pairs of lat/lon (expressed in decimal degrees) along with their radius (expressed in meters). What I am trying to achieve is to find if an intersect between these two points exits (of course, it is obvious that this doesn't hold here but the plan is to try this algorithm in many other data points). In order to check this I am using Shapely's intersects() function. My question however is how should I deal with the different units? Should I make some sort of transformation \ projection first (same units for both lat\lon and radius)?
48.180759,11.518950,19.0
47.180759,10.518950,10.0
EDIT:
I found this library here (https://pypi.python.org/pypi/utm) which seems helpfull. However, I am not 100% sure if I apply it correctly. Any ideas?
X = utm.from_latlon(38.636782, 21.414384)
A = geometry.Point(X[0], X[1]).buffer(30.777)
Y = utm.from_latlon(38.636800, 21.414488)
B = geometry.Point(Y[0], Y[1]).buffer(23.417)
A.intersects(B)
SOLUTION:
So, I finally managed to solve my problem. Here are two different implementations that both solve the same problem:
X = from_latlon(48.180759, 11.518950)
Y = from_latlon(47.180759, 10.518950)
print(latlonbuffer(48.180759, 11.518950, 19.0).intersects(latlonbuffer(47.180759, 10.518950, 19.0)))
print(latlonbuffer(48.180759, 11.518950, 100000.0).intersects(latlonbuffer(47.180759, 10.518950, 100000.0)))
X = from_latlon(48.180759, 11.518950)
Y = from_latlon(47.180759, 10.518950)
print(geometry.Point(X[0], X[1]).buffer(19.0).intersects(geometry.Point(Y[0], Y[1]).buffer(19.0)))
print(geometry.Point(X[0], X[1]).buffer(100000.0).intersects(geometry.Point(Y[0], Y[1]).buffer(100000.0)))

Shapely only uses the Cartesian coordinate system, so in order to make sense of metric distances, you would need to either:
project the coordinates into a local projection system that uses distance units in metres, such as a UTM zone.
buffer a point from (0,0), and use a dynamic azimuthal equidistant projection centered on the lat/lon point to project to geographic coords.
Here's how to do #2, using shapely.ops.transform and pyproj
import pyproj
from shapely.geometry import Point
from shapely.ops import transform
from functools import partial
WGS84 = pyproj.Proj(init='epsg:4326')
def latlonbuffer(lat, lon, radius_m):
proj4str = '+proj=aeqd +lat_0=%s +lon_0=%s +x_0=0 +y_0=0' % (lat, lon)
AEQD = pyproj.Proj(proj4str)
project = partial(pyproj.transform, AEQD, WGS84)
return transform(project, Point(0, 0).buffer(radius_m))
A = latlonbuffer(48.180759, 11.518950, 19.0)
B = latlonbuffer(47.180759, 10.518950, 10.0)
print(A.intersects(B)) # False
Your two buffered points don't intersect. But these do:
A = latlonbuffer(48.180759, 11.518950, 100000.0)
B = latlonbuffer(47.180759, 10.518950, 100000.0)
print(A.intersects(B)) # True
As shown by plotting the lon/lat coords (which distorts the circles):

Python KD Tree Nearest Neigbour where distance is greater than zero

I am trying to implement a Nearest neighbour search for Lat and Lon data. Here is the Data.txt
61.3000183105 -21.2500038147 0
62.299987793 -23.750005722 1
66.3000488281 -28.7500038147 2
40.8000183105 -18.250005722 3
71.8000183105 -35.7500038147 3
39.3000183105 -19.7500019073 4
39.8000183105 -20.7500038147 5
41.3000183105 -20.7500038147 6
The problem is, when I want to do the nearest neighbour for each of the Lat and Lon on the data set, it is searching it self. e.g Nearest Neighbour of (-21.2500038147,61.3000183105) will be (-21.2500038147,61.3000183105) and the resulting distance will be 0.0. I am trying to avoid this but with no luck. I tried doing if not (array_equal) but still...
Below is my python code
import numpy as np
from numpy import *
import decimal
from scipy import spatial
from scipy.spatial import KDTree
from math import radians,cos,sin,sqrt,exp
Lat =[]
Lon =[]
Day =[]
nja = []
Data = np.loadtxt('Data.txt',delimiter=" ")
for i in range(0,len(Data)):
Lon.append(Data[i][:][0])
Lat.append(Data[i][:][1])
Day.append(Data[i][:][2])
tree =spatial.KDTree(zip(Lon,Lat) )
print "Lon :",len(Lon)
print "Tree :",len(tree.data)
for i in range(0,len(tree.data)):
pts = np.array([tree.data[i][0],tree.data[i][1]])
nja.append(pts)
for i in range(0, len(nja)):
if not (np.array_equal(nja,tree.data)):
nearest = tree.query(pts,k=1,distance_upper_bound =9)
print nearest

For each point P[i] in your data set, you're asking "Which is the point nearest to P[i] in my data set?" and you get the answer "It is P[i]".
If you ask a different question, "Which are the TWO points nearest to P[i]?", i.e., tree.query(pts,k=2) (the difference with your code being s/k=1/k=2/)
you will get P[i] and also a P[j], the second nearest point, that is the result you want.
Side note:
I'd recommend that you project your data before building the tree, cause in your range of latitudes there is a large fluctuation in what is meant by a 1 degree distance in longitude.

How'bout a low-tech solution? If you have a large number of points (say 10000 or more), this is no more reasonable, but for a smaller number this brute force solution might be useful:
import numpy as np
dist = (Lat[:,None]-Lat[None,:])**2 + (Lon[:,None]-Lon[None,:])**2
Now you have an NxN array (N is the number of points) with distances (or squares of distances, to be more precise) between all point pairs. Finding the shortest distance for each point is then a matter of finding the smallest value on each row. To exclude the point itself you may set the diagonal to NaN and use nanargmax:
np.fill_diagonal(dist, np.nan)
closest = np.nanargmin(dist, axis=1)
This approach is very simple and guaranteed to find the closest points, but has two significant downsides:
It is O(n^2), and at 10000 points it takes around one second
Ot consumes a lot of memory (800 MB for the above mentioned case)
The latter problem can of course be avoided by doing this piecewise, but the first problem excludes large point sets.
This can be carried out also by using scipy.spatial.distance.pdist:
dist=scipy.spatial.distance.pdist(np.column_stack((Lon, Lat)))
This is a bit faster (by half at least), but the output matrix is in the condensed form, see the documentation for scipy.spatial.distance.squareform.
If you need to calculate the real distances, then this is a good alternative, as pdist can handle distances on a sphere.
Then, again, you may use your KDtree approach by just extending your query to two closest point:
nearest = tree.query(pts, k=2, distance_upper_bound=9)
Then nearest[1][0] has the point itself ("me, myself, and I"), nearest[1][1] the real nearest neighbour (or inf if there is nothing near enough).
The best solution depends on the number of points you have. Also, you might want to use something else than cartesian 2D distances if your map points are not close to each other on the globe.
Just a note about using latitudes and longitudes in finding distances: If you just try to pretend they are 2D Cartesian points, you get it wrong. At 60°N one degree of latitude is 1111 km, whereas one degree of longitude is 555 km. So, at least you will have to divide the longitudes by cos(latitude). And even with that trick you will end up in trouble when the longitudes change from east to west.
Probably the easiest way out of this trouble is to calculate the coordinate points into Cartesian 3D points:
x = cos(lat) * cos(lon)
y = cos(lat) * sin(lon)
z = sin(lat)
If you then calculate the shortest distances between these points, you will get the correct results. (Just note that the distances are not the same as real shortest distances on the surface of the globe.)