I am trying to generate a python code that takes geojson containing polygon as input an then outputs a polygon that is scaled by exact value in meters. For example lets say that my polygon is a square representing a house and I want to get an output of another square that is exactly 20m bigger on each side. This would be simple with square but it is quite hard with complex shapes.
Heres is what I use now (found here Scale GeoJSON to find latitude and longitude points nearby):
import json
from shapely import affinity
from shapely.geometry import shape, Point, mapping
def handler(event, context):
with open('polygon.geojson', encoding='utf-8') as f:
js = json.load(f)
polygon = shape(js['features'][0]['geometry'])
polygon_nearby = affinity.scale(polygon, xfact=1.1, yfact=1.1)
print(json.dumps({"type": "FeatureCollection", "features": [{"type": "Feature", 'properties': {}, 'geometry': mapping(polygon)}]}))
print(json.dumps({"type": "FeatureCollection", "features": [{"type": "Feature", 'properties': {}, 'geometry': mapping(polygon_nearby)}]}))
if __name__ == '__main__':
handler(None, None)
However as you can see this scales by realtive value (10%) and I would like to change that to absolute value in meters.
Thank you in forward for any suggestions
Related
Goal: Ultimately, to convert .geojson to .wkt. Here, I want to extract all coordinates, each as a list.
In my.geojson, there are n many: {"type":"Polygon","coordinates":...
Update: I've successfully extracted the first coordinates. However, this file has two coordinates.
Every .geojson has at least 1 coordinates, but may have more.
How can I dynamically extract all key-values of many coordinates?
Code:
from pathlib import Path
import os
import geojson
import json
from shapely.geometry import shape
ROOT = Path('path/')
all_files = os.listdir(ROOT)
geojson_files = list(filter(lambda f: f.endswith('.geojson'), all_files))
for gjf in geojson_files:
with open(f'{str(ROOT)}/{gjf}') as f:
gj = geojson.load(f)
o = dict(coordinates = gj['features'][0]['geometry']['coordinates'], type = "Polygon")
geom = shape(o)
wkt = geom.wkt
Desired Output:
1 .wkt for all corrdinates in geojson
To convert a series of geometries in GeoJSON files to WKT, the shape() function can convert the GeoJSON geometry to a shapely object which then can be formatted as WKT and/or projected to a different coordinate reference system.
If want to access the coordinates of polygon once it's in a shapely object, usex,y = geo.exterior.xy.
If just want to convert a series of GeoJSON files into one .wkt file per GeoJSON file then try this:
from pathlib import Path
import json
from shapely.geometry import shape
ROOT = Path('path')
for f in ROOT.glob('*.geojson'):
with open(f) as fin, open(f.with_suffix(".wkt"), "w") as fout:
features = json.load(fin)["features"]
for feature in features:
geo = shape(feature["geometry"])
# format geometry coordinates as WKT
wkt = geo.wkt
print(wkt)
fout.write(wkt + "\n")
This output uses your example my.geojson file.
Output:
POLYGON ((19372 2373, 19322 2423, ...
POLYGON ((28108 25855, 27755 26057, ...
If need to convert the coordinates to EPSG:4327 (WGS-84) (e.g. 23.314208, 37.768469), you can use pyproj.
Full code to convert collection of GeoJSON files to a new GeoJSON file in WGS-84.
from pathlib import Path
import json
import geojson
from shapely.geometry import shape, Point
from shapely.ops import transform
from pyproj import Transformer
ROOT = Path('wkt/')
features = []
# assume we're converting from 3857 to 4327
# and center point is at lon=23, lat=37
c = Point(23.676757000000002, 37.9914205)
local_azimuthal_projection = f"+proj=aeqd +R=6371000 +units=m +lat_0={c.y} +lon_0={c.x}"
aeqd_to_wgs84 = Transformer.from_proj(local_azimuthal_projection,
'+proj=longlat +datum=WGS84 +no_defs')
for f in ROOT.glob('*.geojson'):
with open(f) as fin:
features = json.load(fin)["features"]
for feature in features:
geo = shape(feature["geometry"])
poly_wgs84 = transform(aeqd_to_wgs84.transform, geo)
features.append(geojson.Feature(geometry=poly_wgs84))
# Output new GeoJSON file
with open("out.geojson", "w") as fp:
fc = geojson.FeatureCollection(features)
fp.write(geojson.dumps(fc))
Assuming the conversion is from EPSG:3857 to EPSG:4327 and center point is at lon=23, lat=37, the output GeoJSON file will look like this:
{"features": [{"type": "Polygon", "geometry": {"coordinates": [[[23.897879, 38.012554], ...
in the below code i want to calculate the distance from a point to the nearest edge of a polygon.as shown in the results section below, the coordinates are provided.the code posted below shows how i find the distance from a point to the neatrest edge of a polygon.
at run time, and as shown below in restults section, for the give point and geometry, the distance from postgis is equal to 4.32797817574802 while the one calculated from geopandas gives 3.8954865274727614e-05
please let me know how to find the distance from a point to nearest edge of a polygon.
code
poly = wkt.loads(fieldCoordinatesAsTextInWKTInEPSG25832)
pt = wkt.loads(centerPointointAsTextInWKTInEPSG25832)
print(poly.distance(pt)))
results:
queryPostgreSQLForDistancesFromPointsToPolygon:4.32797817574802#result from postgis using st_distance operator
centerPointointAsTextInWKTInEPSG4326:POINT(6.7419520458647835 51.08427961641239)
centerPointointAsTextInWKTInEPSG25832:POINT(341849.5 5661622.5)
centerPointointAsTextInWKTInEPSG4326:POINT(6.7419520458647835 51.08427961641239)
fieldCoordinatesAsTextInWKTInEPSG25832:POLYGON ((5622486.93624152 1003060.89945681,5622079.52632924 1003170.95198635,5622126.00418918 1003781.73122161,5622444.73987453 1003694.55868486,5622486.93624152 1003060.89945681))
fieldCoordinatesAsTextInWKTInEPSG4326:POLYGON((6.741879696309871 51.08423775429969,6.742907378503366 51.08158745820981,6.746964018740842 51.08233499299334,6.746152690693346 51.08440763989611,6.741879696309871 51.08423775429969))
poly.distance(pt):3.8954865274727614e-05#result from geopandas
your code works. It's approx 7000km from Belgium to Ethiopia
are you sure your data is correct? Have built a plotly graph to show where buffered polygon, polygon centroid and point are located in EPSG:4326 CRS
from shapely import wkt
import geopandas as gpd
import plotly.express as px
import json
# queryPostgreSQLForDistancesFromPointsToPolygon:4.32797817574802#result from postgis using st_distance operator
centerPointointAsTextInWKTInEPSG4326 = "POINT(6.7419520458647835 51.08427961641239)"
centerPointointAsTextInWKTInEPSG25832 = "POINT(341849.5 5661622.5)"
centerPointointAsTextInWKTInEPSG4326 = "POINT(6.7419520458647835 51.08427961641239)"
fieldCoordinatesAsTextInWKTInEPSG25832 = "POLYGON ((5622486.93624152 1003060.89945681,5622079.52632924 1003170.95198635,5622126.00418918 1003781.73122161,5622444.73987453 1003694.55868486,5622486.93624152 1003060.89945681))"
fieldCoordinatesAsTextInWKTInEPSG4326 = "POLYGON((6.741879696309871 51.08423775429969,6.742907378503366 51.08158745820981,6.746964018740842 51.08233499299334,6.746152690693346 51.08440763989611,6.741879696309871 51.08423775429969))"
# poly.distance(pt):3.8954865274727614e-05#result from geopandas
poly = wkt.loads(fieldCoordinatesAsTextInWKTInEPSG25832)
pt = wkt.loads(centerPointointAsTextInWKTInEPSG25832)
print(poly.distance(pt)/10**3)
# let's visualize it....
gpoly = (
gpd.GeoDataFrame(geometry=[poly], crs="EPSG:25832")
.buffer(10 ** 6)
.to_crs("EPSG:4326")
)
gpoly.plot()
gpt = gpd.GeoDataFrame(geometry=[pt, poly.centroid], crs="EPSG:25832").to_crs(
"EPSG:4326"
)
px.scatter_mapbox(
gpt.assign(dist=poly.distance(pt)/10**3),
lat=gpt.geometry.y,
lon=gpt.geometry.x,
hover_data={"dist":":.0f"},
).update_layout(
mapbox={
"style": "carto-positron",
"zoom": 4,
"layers": [
{
"source": json.loads(gpoly.to_json()),
"below": "traces",
"type": "fill",
"color": "red",
}
],
}
)
I have been trying to render geoJSON in Plotly by converting shapefiles from https://geoportal.statistics.gov.uk/datasets/local-authority-districts-december-2019-boundaries-uk-bfc.
The Python Plotly docs for plotly.graph_objects.Choroplethmapbox mention that in the geoJSON an id field is required for each feature. I have tried both creating an artificial id and using the plotly featurekeyid field but neither of them are working. When I do use the id key, I have checked that the id key is in the correct location and have tried both as int64 and string.
Sometimes the base mapbox layer will render but no polygons and others the code will run and then hang.
I have also tried reducing the size of the .shp file using mapshaper's various algorithms then saving that to geoJSON format and skipping the conversion step in Python from .shp to geoJSON but again to no avail. Also changing the tolerance in the shapely manipulation does not seem to change the output.
What I am expecting is a map projection with a mapbox base layer with the local authority district polygons on top and filled. The below link shows the polygons and was created on mapshaper.org:
Polygons of Local Authority District
My mapbox access token is valid.
This is an example of trying to render the Local Authority Boundaries polygons by adding in an id field and converting the .shp file to geoJSON and then creating the trace:
import geopandas as gpd
from shapely.geometry import LineString, MultiLineString
import plotly.graph_objs as go
# load in shp files
lad_shp = gpd.read_file('zip://../../data/external/Local_Authority_Districts_(December_2019)_Boundaries_UK_BFC-shp.zip', encoding='utf-8')
# using empet code to convert .shp to geoJSON
def shapefile_to_geojson(gdf, index_list, tolerance=0.025):
# gdf - geopandas dataframe containing the geometry column and values to be mapped to a colorscale
# index_list - a sublist of list(gdf.index) or gdf.index for all data
# tolerance - float parameter to set the Polygon/MultiPolygon degree of simplification
# returns a geojson type dict
geo_names = list(gdf[f'lad19nm']) # name of authorities
geojson = {'type': 'FeatureCollection', 'features': []}
for index in index_list:
geo = gdf['geometry'][index].simplify(tolerance)
if isinstance(geo.boundary, LineString):
gtype = 'Polygon'
bcoords = np.dstack(geo.boundary.coords.xy).tolist()
elif isinstance(geo.boundary, MultiLineString):
gtype = 'MultiPolygon'
bcoords = []
for b in geo.boundary:
x, y = b.coords.xy
coords = np.dstack((x,y)).tolist()
bcoords.append(coords)
else: pass
feature = {'type': 'Feature',
'id' : index,
'properties': {'name': geo_names[index]},
'geometry': {'type': gtype,
'coordinates': bcoords},
}
geojson['features'].append(feature)
return geojson
geojsdata = shapefile_to_geojson(lad_shp, list(lad_shp.index))
# length to generate synthetic data for z attribute
L = len(geojsdata['features'])
# check id key is there
geojsdata['features'][0].keys()
>> dict_keys(['type', 'id', 'properties', 'geometry'])
# example of authroity name
geojsdata['features'][0]['properties']['name']
>> 'Hartlepool'
# check id
k=5
geojsdata['features'][k]['id']
>> '5'
trace = go.Choroplethmapbox(z=np.random.randint(10, 75, size=L), # synthetic data
locations=[geojsdata['features'][k]['id'] for k in range(L)],
colorscale='Viridis',
colorbar=dict(thickness=20, ticklen=3),
geojson=geojsdata,
text=regions,
marker_line_width=0.1, marker_opacity=0.7)
layout = go.Layout(title_text='UK LAD Choropleth Demo',
title_x=0.5,
width=750,
height=700,
mapbox=dict(center=dict(lat=54, lon=-2),
accesstoken=mapbox_access_token,
zoom=3))
fig=go.Figure(data=[trace], layout =layout)
fig.show()
The geoJSON output from the above shapefile_to_geojson function can be found here: https://www.dropbox.com/s/vuf3jtrr2boq5eg/lad19-geo.json?dl=0
Does anyone have any idea what could be causing the issue? I'm assuming the .shp files are good as they are rendered fine on mapshaper.org and QGis. Any help would be greatly appreciated.
Thank you.
Simply changing the projection system corrected the error. Doing this before conversion to geoJSON:
lad_shp = lad_shp.to_crs(epsg=4326)
I'm having two shape files which is contains polygons in it. I'm trying to find delta out of it.
Trying to do this by following code but not working the way i expected.
Following is two shape files blue one is buffer shape file, I need to remove that buffer area which intersecting with blue buffer.i.e. need to get difference of geometry the same as Qgis difference function
import fiona
from shapely.geometry import shape, mapping, Polygon
green = fiona.open(
"/home/gulve/manual_geo_ingestion/probe-data/images/r/shape_out/dissolved.shp")
blue = fiona.open(
"/home/gulve/manual_geo_ingestion/probe-data/images/g/shape/shape.shp")
print([not shape(i['geometry']).difference(shape(j['geometry'])).is_empty for i, j in zip(list(blue), list(green))])
schema = {'geometry': 'Polygon',
'properties': {}}
crs = {'init': u'epsg:3857'}
with fiona.open(
'/home/gulve/manual_geo_ingestion/probe-data/images/r/shape_out/diff.shp', 'w',
driver='ESRI Shapefile', crs=crs, schema=schema
) as write_shape:
for geom in [shape(i['geometry']).difference(shape(j['geometry'])) for i, j in zip(list(blue), list(green))]:
if not geom.empty:
write_shape.write({'geometry': mapping((shape(geom))), 'properties': {}})
Expected output:
After you imported the shapefiles into PostgreSQL, just execute this query:
CREATE TABLE not_intersects AS
SELECT * FROM shape
WHERE id NOT IN (SELECT DISTINCT shape.id
FROM buffer,shape
WHERE ST_Intersects(buffer.geom,shape.geom));
This query will create a third table (called here not_intersects) containing the polygons that do not intersect between the two tables (shape files).
The result is represented in yellow.
I'm able to solve this with shapely functions
Here's my code..
import fiona
from shapely.geometry import shape, mapping, Polygon
from shapely.ops import unary_union
buffered_shape = fiona.open(
"dissolved.shp", 'r', encoding='UTF-8')
color_shape = fiona.open(
"shape.shp", 'r', encoding='UTF-8')
print([not shape(i['geometry']).difference(shape(j['geometry'])).is_empty for i, j in
zip(list(color_shape), list(buffered_shape))])
outmulti = []
for pol in color_shape:
green = shape(pol['geometry'])
for pol2 in buffered_shape:
red = shape(pol2['geometry'])
if red.intersects(green):
# If they intersect, create a new polygon that is
# essentially pol minus the intersection
intersect = green.intersection(red)
nonoverlap = green.symmetric_difference(intersect)
outmulti.append(nonoverlap)
else:
outmulti.append(green)
finalpol = unary_union(outmulti)
schema = {'geometry': 'MultiPolygon',
'properties': {}}
crs = {'init': u'epsg:4326'}
with fiona.open(
'shape_out/diff.shp', 'w',
driver='ESRI Shapefile', crs=crs, schema=schema
) as write_shape:
for geom in finalpol:
# if not geom.empty:
write_shape.write({'geometry': mapping(Polygon(shape(geom))), 'properties': {}})
I'm having difficulty loading the following JSON containing GIS data (https://data.cityofnewyork.us/resource/5rqd-h5ci.json) into a GeoDataFrame.
The following code fails when I try to set the geometry.
import requests
import geopandas as gpd
data = requests.get("https://data.cityofnewyork.us/resource/5rqd-h5ci.json")
gdf = gpd.GeoDataFrame(data.json())
gdf = gdf.set_geometry('the_geom')
gdf.head()
For people who are using web mapping libraries...
If the GeoJSON is wrapped in a FeatureCollection, as they often are when exported to a GeoJSON string by web mapping libraries (in my case, Leaflet), then all you need to do is pass the list at features to from_features() like so:
import geopandas as gpd
study_area = json.loads("""
{"type": "FeatureCollection", "features": [{"type": "Feature", "properties": {}, "geometry": {"type": "Polygon", "coordinates": [[[36.394272, -18.626726], [36.394272, -18.558391], [36.489716, -18.558391], [36.489716, -18.626726], [36.394272, -18.626726]]]}}]}
""")
gdf = gpd.GeoDataFrame.from_features(study_area["features"])
print(gdf.head())
Output:
geometry
0 POLYGON ((36.394272 -18.626726, 36.394272 -18....
Easy peasy.
Setting the geometry fails because the geopandas.GeoDataFrame constructor doesn't appear to be built to handle JSON objects as python data structures. It therefore complains about the argument not being a valid geometry object. You have to parse it into something that geopandas.GeoDataFrame can understand, like a shapely.geometry.shape. Here's what ran without error on my side, Python 3.5.4:
#!/usr/bin/env python3
import requests
import geopandas as gpd
from shapely.geometry import shape
r = requests.get("https://data.cityofnewyork.us/resource/5rqd-h5ci.json")
r.raise_for_status()
data = r.json()
for d in data:
d['the_geom'] = shape(d['the_geom'])
gdf = gpd.GeoDataFrame(data).set_geometry('the_geom')
gdf.head()
A disclaimer: I know absolutely nothing about Geo anything. I didn't even know these libraries and this kind of data existed until I installed geopandas to tackle this bounty and read a little bit of online documentation.
Combining the above answers, this worked for me.
import pandas as pd
import geopandas as gpd
from shapely.geometry import shape
nta = pd.read_json( r'https://data.cityofnewyork.us/resource/93vf-i5bz.json' )
nta['the_geom'] = nta['the_geom'].apply(shape)
nta_geo = gpd.GeoDataFrame(nta).set_geometry('geometry')
A more idiomatic way that uses regular dataframe functions inherited from pandas, and the native GeoDataFrame.from_features:
gdf = gpd.GeoDataFrame(data.json())
# features column does not need to be stored, this is just for illustration
gdf['features'] = gdf['the_geom'].apply(lambda x: {'geometry': x, 'properties': {}})
gdf2 = gpd.GeoDataFrame.from_features(gdf['features'])
gdf = gdf.set_geometry(gdf2.geometry)
gdf.head()