I have an array (lons) of longitude values in the range [-180, 180]. I need to find the mean of the time series. This is easily done with
np.mean(lons)
This straight forward mean, of course, doesn't work if the series contains values either side of the dateline. What is the correct way of calculating the mean for all possible cases? Note, I would rather not have a condition that treats dateline crossing cases differently.
I've played around with np.unwrap after converting from degrees to rad, but I know my calculations are wrong because a small percentage of cases are giving me mean longitudes somewhere near 0 degrees (the meridian) over Africa. These aren't possible as this is an ocean data set.
Thanks.
EDIT: I now realise a more precise way of calculating the mean [lat, lon] position of a time series might be to convert to a cartesian grid. I may go down this route.
This is an application for directional statistics, where the angular mean is computed in the complex plane (see this section). The result is a complex number, whose imaginary part represents the mean angle:
import numpy as np
def angular_mean(angles_deg):
N = len(angles_deg)
mean_c = 1.0 / N * np.sum(np.exp(1j * angles_deg * np.pi/180.0))
return np.angle(mean_c, deg=True)
lons = [
np.array([-175, -170, 170, 175]), # broad distribution
np.random.rand(1000) # narrow distribution
]
for lon in lons:
print angular_mean(lon), np.mean(lon)
As you can see, arithmetic mean and angular mean are quite similar for a narrow distribution, whereas they differ significantly for a broad distribution.
Using cartesian coordinates is not appropriate, as the center of mass will be located within the earth, but since you are using surface data I assume you want it to be located on the surface.
Here is my solution. Note that I calculate the mean latitude and longitude, but also the mean distance (mean_dist) of the [lat, lon] coordinates from the calculated mean latitude (lat_mean) and mean longitude (lon_mean). The reason is that I'm also interested in how much variation there is from the central [lat, lon]. I believe this is correct but I'm open to discussion!
lat_size = np.size(lats)
lon_rad = np.deg2rad(lons) # lons in degrees [-180, 180]
lat_rad = np.deg2rad(lats) # lats in degrees [-90, 90]
R = 6371 # Approx radius of Earth (km)
x = R * np.cos(lat_rad) * np.cos(lon_rad)
y = R * np.cos(lat_rad) * np.sin(lon_rad)
z = R * np.sin(lat_rad)
x_mean = np.mean(x)
y_mean = np.mean(y)
z_mean = np.mean(z)
lat_mean = np.rad2deg(np.arcsin(z_mean / R))
lon_mean = np.rad2deg(np.arctan2(y_mean, x_mean))
# Calculate distance from centre point for each [lat, lon] pair
dist_list = np.empty(lat_size)
dist_list.fill(np.nan)
p = 0
for lat, lon in zip(lats, lons):
coords_1 = (lat, lon)
coords_2 = (lat_mean, lon_mean )
dist_list[p] = geopy.distance.vincenty(coords_1, coords_2).km
p = p + 1
mean_dist = np.mean(dist_list)
return lat_mean, lon_mean, mean_dist
Related
I downloaded weather data and it has longitude (in decimal), latitude (in decimal), and elevation (in m) values. There is no information about the coordinate system used. How I can convert it to cartesian coordinates ?. My attempts are below. But, my problem is to find the right formulas
def cartesian(self,longitude,latitude, elevation):
R = 6378137.0 + elevation # relative to centre of the earth
X = R * math.cos(longitude) * math.sin(latitude)
Y = R * math.sin(longitude) * math.sin(latitude)
Z = R * math.cos(latitude)
def cartesian3(self,longitude,latitude, elevation):
X = longitude * 60 * 1852 * math.cos(latitude)
Y = latitude * 60 * 1852
Z = elevation
return X,Y,Z
An answer here by Daphna Shezaf uses different formulas. However, it does not use elevations.
I would appreciate if someone could clear my confusion, should elevation be used in converting from long/lat or not ?. What are the right formulas ?. I have tried to compare the result of my codes on this website by using specific long, lat, elev. My both methods above have results that are far from the obtained result from the website
UPDATE
I would like to share the solution to my problem. I have implemented lla2ecef function from Matlab as here in python. It allows to convert radian longitude, latitude, and elevation (height in m) to cartesian. I only need to convert latitude and longitude to radian iff they are in decimal by :
latitude = (lat * math.pi) / 180 #latitude in radian, and lat in decimal
To verify my calculations. I compared the conversion result to the website above (website) and this one as well. Both give me almost same result.
Note: If you consider for simplicity earth is sphere, you can use def cartesian (I updated it; thanks to Sasha for correction). If you consider earth is ellipsoid (WGS 84 Geodetic System), you can implement the conversion as in lla2ecef. def cartesian is for cartographic projection (Thanks for rodrigo)
Elevation is measured from sealevel. Radius connects the center of the earth to you geographic location. This means that R = 6371km + elevation. This fixed point can vary and the exact value should be specified by the data provider. Your first function seems to be correct, just replace the R calculation.
To be blunt: Without radius (elevation), it is not possible to convert from spherical to cartesian coordinates. Least you could do is use the "sea level height", but this will only give you the coordinates on a planet which is a perfect sphere. Which Earth isn't.
For example, on the website you provided, you can select the ellipsoid. For WGS 84 standard, I found the following in wikipedia;
The WGS 84 datum surface is an oblate spheroid (ellipsoid) with major (equatorial) radius a = 6378137 m at the equator and flattening f = 1/298.257223563.[6] The polar semi-minor axis b then equals a times (1−f), or 6356752.3142 m
I would like to be able to plot two lines using direction and distance. It is a Drillhole trace, so I have the data in this format right now,
The depth is actually distance down the hole, not vertical depth. Azimuth is from magnetic north. Dip is based on 0 being horizontal. I want to plot two lines from the same point (0,0,0 is fine) and see how they differ, based on this kind of info.
I have no experience with Matplotlib but am comfortable with Python and would like to get to know this plotting tool. I have found this page and it helped to understand the framework, but I still can't figure out how to plot lines with 3d vectors. Can someone give me some pointers on how to do this or where to find the directions I need? Thank you
A script converting your coordinates to cartesian and plotting it with matplotlib with the comments included:
import numpy as np
import matplotlib.pyplot as plt
# import for 3d plot
from mpl_toolkits.mplot3d import Axes3D
# initializing 3d plot
fig = plt.figure()
ax = fig.add_subplot(111, projection = '3d')
# several data points
r = np.array([0, 14, 64, 114])
# get lengths of the separate segments
r[1:] = r[1:] - r[:-1]
phi = np.array([255.6, 255.6, 261.7, 267.4])
theta = np.array([-79.5, -79.5, -79.4, -78.8])
# convert to radians
phi = phi * 2 * np.pi / 360.
# in spherical coordinates theta is measured from zenith down; you are measuring it from horizontal plane up
theta = (90. - theta) * 2 * np.pi / 360.
# get x, y, z from known formulae
x = r*np.cos(phi)*np.sin(theta)
y = r*np.sin(phi)*np.sin(theta)
z = r*np.cos(theta)
# np.cumsum is employed to gradually sum resultant vectors
ax.plot(np.cumsum(x),np.cumsum(y),np.cumsum(z))
For a drillhole with 500 m you may use minimum curvature method, otherwise the position error will be really large. I implemented this in a python module for geostatistics (PyGSLIB). An example showing a complete desurvey process for a real drillhole database, including positions at assay/lithology intervals is shown at:
http://nbviewer.ipython.org/github/opengeostat/pygslib/blob/master/pygslib/Ipython_templates/demo_1.ipynb
This also shows how to export drillholes in VTK format to lad it in paraview.
Results shown in Paraview
The code in Cython to desurvey one interval is as follows:
cpdef dsmincurb( float len12,
float azm1,
float dip1,
float azm2,
float dip2):
"""
dsmincurb(len12, azm1, dip1, azm2, dip2)
Desurvey one interval with minimum curvature
Given a line with length ``len12`` and endpoints p1,p2 with
direction angles ``azm1, dip1, azm2, dip2``, this function returns
the differences in coordinate ``dz,dn,de`` of p2, assuming
p1 with coordinates (0,0,0)
Parameters
----------
len12, azm1, dip1, azm2, dip2: float
len12 is the length between a point 1 and a point 2.
azm1, dip1, azm2, dip2 are direction angles azimuth, with 0 or
360 pointing north and dip angles measured from horizontal
surface positive downward. All these angles are in degrees.
Returns
-------
out : tuple of floats, ``(dz,dn,de)``
Differences in elevation, north coordinate (or y) and
east coordinate (or x) in an Euclidean coordinate system.
See Also
--------
ang2cart,
Notes
-----
The equations were derived from the paper:
http://www.cgg.com/data//1/rec_docs/2269_MinimumCurvatureWellPaths.pdf
The minimum curvature is a weighted mean based on the
dog-leg (dl) value and a Ratio Factor (rf = 2*tan(dl/2)/dl )
if dl is zero we assign rf = 1, which is equivalent to balanced
tangential desurvey method. The dog-leg is zero if the direction
angles at the endpoints of the desurvey intervals are equal.
Example
--------
>>> dsmincurb(len12=10, azm1=45, dip1=75, azm2=90, dip2=20)
(7.207193374633789, 1.0084573030471802, 6.186459064483643)
"""
# output
cdef:
float dz
float dn
float de
# internal
cdef:
float i1
float a1
float i2
float a2
float DEG2RAD
float rf
float dl
DEG2RAD=3.141592654/180.0
i1 = (90 - dip1) * DEG2RAD
a1 = azm1 * DEG2RAD
i2 = (90 - dip2) * DEG2RAD
a2 = azm2 * DEG2RAD
# calculate the dog-leg (dl) and the Ratio Factor (rf)
dl = acos(cos(i2-i1)-sin(i1)*sin(i2)*(1-cos(a2-a1)))
if dl!=0.:
rf = 2*tan(dl/2)/dl # minimum curvature
else:
rf=1 # balanced tangential
dz = 0.5*len12*(cos(i1)+cos(i2))*rf
dn = 0.5*len12*(sin(i1)*cos(a1)+sin(i2)*cos(a2))*rf
de = 0.5*len12*(sin(i1)*sin(a1)+sin(i2)*sin(a2))*rf
return dz,dn,de
I have a problem that I cannot seem to work out. I also cannot find a solution already given on any prior posts.
I am working in a metric coordinate system where all of the variables are negative values (example: origin = -2,-2; north = -2,-1; east = -1,-2; south = -2, -3, west = -3,-2). It's a southern hemisphere coordinate system. I need to calculate the azimuth orientation and slope of a line that passes through two points, given that the first point is the origin point.
I have been able to write a script using Python that calculates the orientations (0-360 degrees) for each pair of points, but a number of the values are 180 degrees opposite, according to a reference data set that I am comparing my results against, which already has these values calculated.
If I use ATAN2 and then convert radians to degrees does it matter which quadrant on a 2D graph the line passes through? DO I need to add or subtract 0,90,180,270, or 360 depending on the quadrant? I think this is my problem, but I am not sure.
Lastly, the above assumes that I am making the calculations for orientation and slope in 2D spaces, respectively. Is there a more parsimonious way to calculate these variables within 3D space?
I've attached my current block of code that includes the calculation of the azimuth angles per quadrant. I would really appreciate any help you all can provide.
dn = north_2 - north_1
de = east_2 - east_1
x = x + 1
if dn<0 and de<=0:
q = "q3"
theta = math.degrees(math.atan2(dn,de))
orientation = 90- theta
if dn>=0 and de <0:
q = "q4"
theta = math.degrees(math.atan2(dn,de))
orientation = 270-theta
if dn>0 and de>=0:
q = "q1"
theta = math.degrees(math.atan2(dn,de))
orientation = 270-theta
if dn<=0 and de>0:
q = "q2"
theta = math.degrees(math.atan2(dn,de))
orientation = 90-theta
I have a set of GPS coordinates in decimal notation, and I'm looking for a way to find the coordinates in a circle with variable radius around each location.
Here is an example of what I need. It is a circle with 1km radius around the coordinate 47,11.
What I need is the algorithm for finding the coordinates of the circle, so I can use it in my kml file using a polygon. Ideally for python.
see also Adding distance to a GPS coordinate for simple relations between lat/lon and short-range distances.
this works:
import math
# inputs
radius = 1000.0 # m - the following code is an approximation that stays reasonably accurate for distances < 100km
centerLat = 30.0 # latitude of circle center, decimal degrees
centerLon = -100.0 # Longitude of circle center, decimal degrees
# parameters
N = 10 # number of discrete sample points to be generated along the circle
# generate points
circlePoints = []
for k in xrange(N):
# compute
angle = math.pi*2*k/N
dx = radius*math.cos(angle)
dy = radius*math.sin(angle)
point = {}
point['lat']=centerLat + (180/math.pi)*(dy/6378137)
point['lon']=centerLon + (180/math.pi)*(dx/6378137)/math.cos(centerLat*math.pi/180)
# add to list
circlePoints.append(point)
print circlePoints
Use the formula for "Destination point given distance and bearing from start point" here:
http://www.movable-type.co.uk/scripts/latlong.html
with your centre point as start point, your radius as distance, and loop over a number of bearings from 0 degrees to 360 degrees. That will give you the points on a circle, and will work at the poles because it uses great circles everywhere.
It is a simple trigonometry problem.
Set your coordinate system XOY at your circle centre. Start from y = 0 and find your x value with x = r. Then just rotate your radius around origin by angle a (in radians). You can find the coordinates of your next point on the circle with Xi = r * cos(a), Yi = r * sin(a). Repeat the last 2 * Pi / a times.
That's all.
UPDATE
Taking the comment of #poolie into account, the problem can be solved in the following way (assuming the Earth being the right sphere). Consider a cross section of the Earth with its largest diameter D through our point (call it L). The diameter of 1 km length of our circle then becomes a chord (call it AB) of the Earth cross section circle. So, the length of the arc AB becomes (AB) = D * Theta, where Theta = 2 * sin(|AB| / 2). Further, it is easy to find all other dimensions.
I use Python:
I have 2 arrays of GPS points - lon and lat (more than 500 000 points).
I have 1 array of date-time.
lon = numpy.array(lon)
lat = numpy.array(lat)
dt = numpy.array(dt)
I have a location error (GPS sensor error). For example 15 meters.
GPS_sensor_error = 0.015
I need to exclude a GPS_sensor_error from coordinates that there were no asterisks on a track.
(I don't draw a point with identical coordinates)
How I am able to do it?
Now:
I calculate distance between points.
I find the minimum distance, if it less GPS_sensor_error, then I average lon, lat.
repeat 1.
repeat 2.
repeat until all distances won't be more GPS_sensor_error
Update:
lon = numpy.array()
lat = numpy.array()
flag = True
while flag:
lon1 = lon[:-1]
lon2 = lon[1:]
lat1 = lat[:-1]
lat2 = lat[1:]
'''distance'''
x = (lon2 - lon1)
y = (lat2 - lat1)
d = numpy.sqrt(x * x + y * y)
min = numpy.min(d)
if min < GPS_sensor_error:
j = numpy.where(d == min)[0][0]
lon[j] = (lon[j] + lon[j + 1]) / 2
lat[j] = (lat[j] + lat[j + 1]) / 2
lon = numpy.delete(lon, j + 1)
lat = numpy.delete(lat, j + 1)
else:
flag = False
Bypass on all points works at a pure python very long...
Prompt please, how to implement it using scipy, numpy?
Thanks
P.s. probably already there is a GPS filter in scipy, numpy?
From a data science perspective what you are doing is not correct. You cant just use the average error distance as a cutoff and think your data will be more correct. The two points you are comparing can have an error more or less than 15 m they can shift toward each other or move away of each other. And if you don't have another exact dataset there is no way of telling what would be the correct point. You can't make this dataset more precise.
However I think you goal is to simplify your dataset, not to make it more accurate. For that you can use the Douglas–Peucker algorithm.
I would suggest you load your data in an Postgis enabled database (Postgresql + postgis) and then use the simplify function. This will require some db setup time, but then it will speed you up greatly. However if you want it in pure python this SO question has a very nice snippet.
BTW. If your are doing distance calculations with lat,lon do not use Pythagoras. It is not valid since lat,lon is not Euclidean. Use the haversine algorithm.
You can easily do all your calculations using numpy primitives only and no python looping.
First define your distance function as a function that operates on numpy arrays (I assume you did that already ..):
def dist(lon1, lat1, lon2, lat2):
"""Compute the distance between (lon1, lat1) and (lon2, lat2).
Both may be numpy arrays."""
...
Then apply it to your data as so:
d = dist(lon[:-1], lat[:-1], lon[1:], lat[1:])
This notation means you will compare the ith point to the i+1th.
Next find the indices where d is greater than your threshold:
I = d > GPS_sensor_error
Now keep only those and the first point!
lon_out = numpy.hstack([[lon[0]], lon[1:][I]]) # could also use numpy.where
lat_out = numpy.hstack([[lat[0]], lat[1:][I]])
Update:
If you want to keep the same number of points, ie set lon[i] to the last good value, use the following trick instead of the previous two lines:
idx, = numpy.where(I)
idx = numpy.hstack([[0], idx])
J = numpy.cumsum(I) # the trick
lon_out = lon[idx[J]]
lat_out = lat[idx[J]]