Develop Reference

project local coordinates to global GPS with reference point

I have a bunch of shapes (e.g. shapely LineStrings or Polygons) in a geopandas GeoDataFrame.
The shapes specify coordinates in a local 200x200 meters grid, i.e. all coordinates are between (0, 0) and (200, 200).
I now would like to "place" these lines globally.
For this, I want to specify a GPS Point (with a given lat/lon) as a reference.
My first (naive) approach would be to use geographiclib, take all shapes' coords (in local X/Y) and apply the following transformation and "recreate" the shape:
# Convert coordinates to GPS location
from shapely.geometry import LineString
from geographiclib.geodesic import Geodesic
geod = Geodesic.WGS84 # the base geodesic (i.e. the world)
origin = (48.853772345870176, 2.350983211585546) # this is somewhere in Paris, for example
def local_to_latlong(x, y, orientation=0, scale=1):
""" Two step process.
- First walk x meters to east from origin.
- Then, from that point, walk y meters north from origin.
Optional:
- orientation allows to "spin" the coordinates
- scale allows to grow/shrink the distances
"""
go_X = geod.Direct(*origin, orientation + 90, x * scale) # x is East-coordinate
go_Y = geod.Direct(go_X["lat2"], go_X["lon2"], orientation + 0, y * scale) # y is North-coordinate
return go_Y["lat2"], go_Y["lon2"]
original_line = LineString([(0,0), (100,100), (200,100)])
global_line = LineString([local_to_latlong(x, y) for y, x in original_line.coords])
However, I hope that this is not the smartest way to do it, and that there are smarter ways out there...
I would like to apply such a transformation onto any shape within a GeoDataFrame. Ideally, it would work using a "to_crs", but I am not sure how to transform the shapes so they are "in reference to a origin" and which crs to use.

given your origin is EPSG:4326, you can estimate the UTM zone
with this you can get UTM zone coordinates of origin
translate your custom 200x200 metre zone into co-ordinates of UTM zone
finally use to_crs() to transform into EPSG:4326
import shapely.geometry
import geopandas as gpd
import pandas as pd
import numpy as np
# generate some polygons (squares), where grid is 200*200
gdf = gpd.GeoDataFrame(
geometry=pd.DataFrame(
np.repeat(np.sort(np.random.randint(0, 200, [20, 2]), axis=1), 2, axis=1)
).apply(lambda d: shapely.geometry.box(*d), axis=1)
)
# chage to linestrings, clearer when we plot
gdf["geometry"] = gdf["geometry"].exterior
origin = (2.350983211585546, 48.853772345870176) # this is somewhere in Paris, for example
# work out utm crs of point. utm is in metres
gdf_o = gpd.GeoDataFrame(geometry=[shapely.geometry.Point(origin)], crs="EPSG:4326")
crs = gdf_o.estimate_utm_crs()
# where is origin in utm zone
xo,yo = gdf_o.to_crs(crs).loc[0,"geometry"].xy
# translate custom zone to co-ordinates of utm zone
# assume point is center of 200x200 grid (hence subtract 100)
gdf_gps = gdf["geometry"].translate(xoff=xo[0]-100, yoff=yo[0]-100).set_crs(crs).to_crs("epsg:4326")
# plot on map to show it has worked...
m = gdf_gps.explore()
m = gdf_o.explore(m=m, color="red", marker_kwds={"radius":20})
m

Fastest way to produce a grid of points that fall within a polygon or shape?

I am using shapely in python and trying to generate evenly spaced points in a grid that fall within a shape in the fastest O(n) time. The shape may be any closed polygon, not just a square or circle. My current approach is:
Find min/max y & x to build a rectangle.
Build a grid of points given a spacing parameter (resolution)
Verify one-by-one if the points fall within the shape.
Is there a faster way to do this?
# determine maximum edges
polygon = shape(geojson['features'][i]['geometry'])
latmin, lonmin, latmax, lonmax = polygon.bounds
# construct a rectangular mesh
points = []
for lat in np.arange(latmin, latmax, resolution):
for lon in np.arange(lonmin, lonmax, resolution):
points.append(Point((round(lat,4), round(lon,4))))
# validate if each point falls inside shape
valid_points.extend([i for i in points if polygon.contains(i)])

I saw that you answered your question (and seems to be happy with using intersection) but also note that shapely (and the underlying geos library) have prepared geometries for more efficient batch operations on some predicates (contains, contains_properly, covers, and intersects).
See Prepared geometry operations.
Adapted from the code in your question, it could be used like so:
from shapely.prepared import prep
# determine maximum edges
polygon = shape(geojson['features'][i]['geometry'])
latmin, lonmin, latmax, lonmax = polygon.bounds
# create prepared polygon
prep_polygon = prep(polygon)
# construct a rectangular mesh
points = []
for lat in np.arange(latmin, latmax, resolution):
for lon in np.arange(lonmin, lonmax, resolution):
points.append(Point((round(lat,4), round(lon,4))))
# validate if each point falls inside shape using
# the prepared polygon
valid_points.extend(filter(prep_polygon.contains, points))

The best i can think is do this:
X,Y = np.meshgrid(np.arange(latmin, latmax, resolution),
np.arange(lonmin, lonmax, resolution))
#create a iterable with the (x,y) coordinates
points = zip(X.flatten(),Y.flatten())
valid_points.extend([i for i in points if polygon.contains(i)])

if you want to generate n points in a shapely.geometry.Polygon, there is a simple iterative function to do it. Manage tol (tolerance) argument to speed up the points generation.
import numpy as np
from shapely.geometry import Point, Polygon
def gen_n_point_in_polygon(self, n_point, polygon, tol = 0.1):
"""
-----------
Description
-----------
Generate n regular spaced points within a shapely Polygon geometry
-----------
Parameters
-----------
- n_point (int) : number of points required
- polygon (shapely.geometry.polygon.Polygon) : Polygon geometry
- tol (float) : spacing tolerance (Default is 0.1)
-----------
Returns
-----------
- points (list) : generated point geometries
-----------
Examples
-----------
>>> geom_pts = gen_n_point_in_polygon(200, polygon)
>>> points_gs = gpd.GeoSeries(geom_pts)
>>> points_gs.plot()
"""
# Get the bounds of the polygon
minx, miny, maxx, maxy = polygon.bounds
# ---- Initialize spacing and point counter
spacing = polygon.area / n_point
point_counter = 0
# Start while loop to find the better spacing according to tolérance increment
while point_counter <= n_point:
# --- Generate grid point coordinates
x = np.arange(np.floor(minx), int(np.ceil(maxx)), spacing)
y = np.arange(np.floor(miny), int(np.ceil(maxy)), spacing)
xx, yy = np.meshgrid(x,y)
# ----
pts = [Point(X,Y) for X,Y in zip(xx.ravel(),yy.ravel())]
# ---- Keep only points in polygons
points = [pt for pt in pts if pt.within(polygon)]
# ---- Verify number of point generated
point_counter = len(points)
spacing -= tol
# ---- Return
return points

Oh why hell yes. Use the intersection method of shapely.
polygon = shape(geojson['features'][i]['geometry'])
lonmin, latmin, lonmax, latmax = polygon.bounds
# construct rectangle of points
x, y = np.round(np.meshgrid(np.arange(lonmin, lonmax, resolution), np.arange(latmin, latmax, resolution)),4)
points = MultiPoint(list(zip(x.flatten(),y.flatten())))
# validate each point falls inside shapes
valid_points.extend(list(points.intersection(polygon)))

Calculating the rectangle area from coordinates

If I have two points (51.9925734, 5.65038093), (51.99226769, 5.64991222)
I want to get the rectangle earth area.
I tried this method from other resources:
from area import area
obj_field = {'type':'Polygon','coordinates':[[[51.9925734,5.65038093],
[51.9925734,5.64991222],
[51.99226769,5.64991222],
[51.99226769,5.65038093]]]}
field_area_2 = area(obj_field)
print(f'The total field area is: {field_area_2} square meters')
The total field area is: 1767.018812772391 square meters
But I'm not sure if it is correct.

For small areas, like your example, the simplest approach is to convert lat/lon coordinates to UTM. Then you can use basic math, like calculating the area of the rectangle, as in this snippet:
import utm
a_geo = 51.9925734, 5.65038093
b_geo = 51.99226769, 5.64991222
# Convert lat, lon to utm
a_easting, a_northing, zone, _ = utm.from_latlon(*a_geo)
b_easting, b_northing, _, _ = utm.from_latlon(*b_geo, force_zone_number=zone)
area = abs((a_easting - b_easting) * (a_northing - b_northing))

Convert latitude, longitude to distance from equator in kilometers and round to nearest kilometer

For each coordinate I have, I find the distance from the equator in kilometers giving me two distances:
from pyproj import Geod
wgs84_geod = Geod(ellps='WGS84')
_,_, lon_dist = wgs84_geod.inv(0, 0,lon, 0)
_,_, lat_dist = wgs84_geod.inv(0, 0,0, lat)
As a sanity check,I can recalculate the original coordinate from these values as follows (assume the direction from the equator coordinate (0,0) is North and West:
_, new_lat, _ = wgs84_geod.fwd(0,0, 0, lat_dist)
new_lon, _, _ = wgs84_geod.fwd(0, 0, 90, lon_dist)
This gives me back the same coordinates I started with.
Now I want to find the closest kilometer point to my coordinate. I round the lon_dist and lat_dist to kilometers from the equator values.
lat_km_dist = round(lat_dist/1000)*1000 #to nearest km and back to meters
lon_km_dist = round(lon_dist/1000)*1000
I get coordinates using these distances in the same way as before
_, km_lat, _ = wgs84_geod.fwd(0,0, 0, lat_km_dist)
km_lon, _, _ = wgs84_geod.fwd(0, 0, 90, lon_km_dist)
The logic should be that for multiple coordinates in the same area, the closest distance between any km_lat, km_lon pair should be 1km.
This is true in the North/South axis, but for longitudes the distance varies depending on which latitude I'm at.
I'm attaching two screenshots to visualize the problem where the km_lat, km_lon coordinates are represented by black circles at the center of polygons with an area of 1km.
How can I correct for this?

What this algorithm is essentially doing is that it constructs an equidistant mesh (with points 1km apart) on the equator (lat=0) and the main meridian (lon=0). It then effectively constructs a grid on the ellipsoid as a Cartesian product of these points.
However, the lat/lon coordinates do not form a Cartesian frame, the resulting parallels/meridians generated by these grid points define "squares" the size of which depends not only on the particular longitude, but latitude as well. On a perfect sphere, this would work in the north-south direction since then an equidistant (in terms of great-circle distance) grid on lon=0 is also equidistant in the latitude (difference in latitude being equal to the difference in distance over the radius of the sphere).
In other words, if you fix two latitudes lat1, lat2 and for a particular longitude lon move from (lat1, lon), (lat2, lon) 1km in, say, westward direction, then these newly obtained points won't have the same longitude...
I am not fully sure what you are trying to achieve but if the goal is to obtain some representative points not too close to each other, then perhaps hierarchical clustering in terms of the great-circle distance could provide reasonable results...
EDIT:
As an approximative workaround, you could get most likely away by choosing another reference point than (0, 0) - the new reference point shouldn't be too far away from the area that you are trying to describe (something like a "bottom-left" corner of the area of interest). If the entire area of interest doesn't cover a significant part of the globe (large span of latitudes) then the discrepancies will be quite small so that they will be probably almost invisible in the GoogleMaps visualization...
So if you are interested in Denmark (judging by the screenshots), then something like the following might work:
lat_ref, lon_ref = 53.637976, 6.694138
_,_, lon_dist = wgs84_geod.inv(lon_ref,lat_ref, lon, 0)
_,_, lat_dist = wgs84_geod.inv(lon_ref,lat_ref, 0, lat)
lat_km_dist = round(lat_dist/1000)*1000 #to nearest km and back to meters
lon_km_dist = round(lon_dist/1000)*1000
_, km_lat, _ = wgs84_geod.fwd(lon_ref,lat_ref, 0, lat_km_dist)
km_lon, _, _ = wgs84_geod.fwd(lon_ref,lat_ref, 90, lon_km_dist)

Automatically center matplotlib basemap onto data

I would like a solution to automatically center a basemap plot on my coordinate data.
I've got things to automatically center, but the resulting area is much larger than the area actually used by my data. I would like the plot to be bounded by the plot coordinates, rather than an area drawn from the lat/lon boundaries.
I am using John Cook's code for calculating the distance between two points on (an assumed perfect) sphere.
First Try
Here is the script I started with. This was causing the width and height to bee small too small for the data area, and the center latitude (lat0) too far south.
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
import sys
import csv
import spheredistance as sd
print '\n'
if len(sys.argv) < 3:
print >>sys.stderr,'Usage:',sys.argv[0],'<datafile> <#rows to skip>'
sys.exit(1)
print '\n'
dataFile = sys.argv[1]
dataStream = open(dataFile, 'rb')
dataReader = csv.reader(dataStream,delimiter='\t')
numRows = sys.argv[2]
dataValues = []
dataLat = []
dataLon = []
print 'Plotting Data From: '+dataFile
dataReader.next()
for row in dataReader:
dataValues.append(row[0])
dataLat.append(float(row[1]))
dataLon.append(float(row[2]))
# center and set extent of map
earthRadius = 6378100 #meters
factor = 1.00
lat0new = ((max(dataLat)-min(dataLat))/2)+min(dataLat)
lon0new = ((max(dataLon)-min(dataLon))/2)+min(dataLon)
mapH = sd.distance_on_unit_sphere(max(dataLat),lon0new,
min(dataLat),lon0new)*earthRadius*factor
mapW = sd.distance_on_unit_sphere(lat0new,max(dataLon),
lat0new,min(dataLon))*earthRadius*factor
# setup stereographic basemap.
# lat_ts is latitude of true scale.
# lon_0,lat_0 is central point.
m = Basemap(width=mapW,height=mapH,
resolution='l',projection='stere',\
lat_0=lat0new,lon_0=lon0new)
#m.shadedrelief()
m.drawcoastlines(linewidth=0.2)
m.fillcontinents(color='white', lake_color='aqua')
#plot data points (omitted due to ownership)
#x, y = m(dataLon,dataLat)
#m.scatter(x,y,2,marker='o',color='k')
# draw parallels and meridians.
m.drawparallels(np.arange(-80.,81.,20.), labels=[1,0,0,0], fontsize=10)
m.drawmeridians(np.arange(-180.,181.,20.), labels=[0,0,0,1], fontsize=10)
m.drawmapboundary(fill_color='aqua')
plt.title("Example")
plt.show()

After generating some random data, it was obvious that the bounds that I chose did not work with this projection (red lines). Using map.drawgreatcircle(), I first visualized where I wanted the bounds while zoomed over the projection of random data.
I corrected the longitude by using the longitudinal difference at the southern most latitude (blue horizontal line).
I determined the latitudinal range using the Pythagorean theorem to solve for the vertical distance, knowing the distance between the northern most longitudinal bounds, and the central southernmost point (blue triangle).
def centerMap(lats,lons,scale):
#Assumes -90 < Lat < 90 and -180 < Lon < 180, and
# latitude and logitude are in decimal degrees
earthRadius = 6378100.0 #earth's radius in meters
northLat = max(lats)
southLat = min(lats)
westLon = max(lons)
eastLon = min(lons)
# average between max and min longitude
lon0 = ((westLon-eastLon)/2.0)+eastLon
# a = the height of the map
b = sd.spheredist(northLat,westLon,northLat,eastLon)*earthRadius/2
c = sd.spheredist(northLat,westLon,southLat,lon0)*earthRadius
# use pythagorean theorom to determine height of plot
mapH = pow(pow(c,2)-pow(b,2),1./2)
arcCenter = (mapH/2)/earthRadius
lat0 = sd.secondlat(southLat,arcCenter)
# distance between max E and W longitude at most souther latitude
mapW = sd.spheredist(southLat,westLon,southLat,eastLon)*earthRadius
return lat0,lon0,mapW*scale,mapH*scale
lat0center,lon0center,mapWidth,mapHeight = centerMap(dataLat,dataLon,1.1)
The lat0 (or latitudinal center) in this case is therefore the point half-way up the height of this triangle, which I solved using John Cooks method, but for solving for an unknown coordinate while knowing the first coordinate (the median longitude at the southern boundary) and the arc length (half that of the total height).
def secondlat(lat1, arc):
degrees_to_radians = math.pi/180.0
lat2 = (arc-((90-lat1)*degrees_to_radians))*(1./degrees_to_radians)+90
return lat2
Update:
The above function, as well as the distance between two coordinates can be achieved with higher accuracy using the pyproj Geod class methods geod.fwd() and geod.inv(). I found this in Erik Westra's Python for Geospatial Development, which is an excellent resource.
Update:
I have now verified that this also works for Lambert Conformal Conic (lcc) projections.