How can I check if a geopoint is within the area of a given shapefile?
I managed to load a shapefile in python, but can't get any further.
Another option is to use Shapely (a Python library based on GEOS, the engine for PostGIS) and Fiona (which is basically for reading/writing files):
import fiona
import shapely
with fiona.open("path/to/shapefile.shp") as fiona_collection:
# In this case, we'll assume the shapefile only has one record/layer (e.g., the shapefile
# is just for the borders of a single country, etc.).
shapefile_record = fiona_collection.next()
# Use Shapely to create the polygon
shape = shapely.geometry.asShape( shapefile_record['geometry'] )
point = shapely.geometry.Point(32.398516, -39.754028) # longitude, latitude
# Alternative: if point.within(shape)
if shape.contains(point):
print "Found shape for point."
Note that doing point-in-polygon tests can be expensive if the polygon is large/complicated (e.g., shapefiles for some countries with extremely irregular coastlines). In some cases it can help to use bounding boxes to quickly rule things out before doing the more intensive test:
minx, miny, maxx, maxy = shape.bounds
bounding_box = shapely.geometry.box(minx, miny, maxx, maxy)
if bounding_box.contains(point):
...
Lastly, keep in mind that it takes some time to load and parse large/irregular shapefiles (unfortunately, those types of polygons are often expensive to keep in memory, too).
This is an adaptation of yosukesabai's answer.
I wanted to ensure that the point I was searching for was in the same projection system as the shapefile, so I've added code for that.
I couldn't understand why he was doing a contains test on ply = feat_in.GetGeometryRef() (in my testing things seemed to work just as well without it), so I removed that.
I've also improved the commenting to better explain what's going on (as I understand it).
#!/usr/bin/python
import ogr
from IPython import embed
import sys
drv = ogr.GetDriverByName('ESRI Shapefile') #We will load a shape file
ds_in = drv.Open("MN.shp") #Get the contents of the shape file
lyr_in = ds_in.GetLayer(0) #Get the shape file's first layer
#Put the title of the field you are interested in here
idx_reg = lyr_in.GetLayerDefn().GetFieldIndex("P_Loc_Nm")
#If the latitude/longitude we're going to use is not in the projection
#of the shapefile, then we will get erroneous results.
#The following assumes that the latitude longitude is in WGS84
#This is identified by the number "4326", as in "EPSG:4326"
#We will create a transformation between this and the shapefile's
#project, whatever it may be
geo_ref = lyr_in.GetSpatialRef()
point_ref=ogr.osr.SpatialReference()
point_ref.ImportFromEPSG(4326)
ctran=ogr.osr.CoordinateTransformation(point_ref,geo_ref)
def check(lon, lat):
#Transform incoming longitude/latitude to the shapefile's projection
[lon,lat,z]=ctran.TransformPoint(lon,lat)
#Create a point
pt = ogr.Geometry(ogr.wkbPoint)
pt.SetPoint_2D(0, lon, lat)
#Set up a spatial filter such that the only features we see when we
#loop through "lyr_in" are those which overlap the point defined above
lyr_in.SetSpatialFilter(pt)
#Loop through the overlapped features and display the field of interest
for feat_in in lyr_in:
print lon, lat, feat_in.GetFieldAsString(idx_reg)
#Take command-line input and do all this
check(float(sys.argv[1]),float(sys.argv[2]))
#check(-95,47)
This site, this site, and this site were helpful regarding the projection check. EPSG:4326
Here is a simple solution based on pyshp and shapely.
Let's assume that your shapefile only contains one polygon (but you can easily adapt for multiple polygons):
import shapefile
from shapely.geometry import shape, Point
# read your shapefile
r = shapefile.Reader("your_shapefile.shp")
# get the shapes
shapes = r.shapes()
# build a shapely polygon from your shape
polygon = shape(shapes[0])
def check(lon, lat):
# build a shapely point from your geopoint
point = Point(lon, lat)
# the contains function does exactly what you want
return polygon.contains(point)
i did almost exactly what you are doing yesterday using gdal's ogr with python binding. It looked like this.
import ogr
# load the shape file as a layer
drv = ogr.GetDriverByName('ESRI Shapefile')
ds_in = drv.Open("./shp_reg/satreg_etx12_wgs84.shp")
lyr_in = ds_in.GetLayer(0)
# field index for which i want the data extracted
# ("satreg2" was what i was looking for)
idx_reg = lyr_in.GetLayerDefn().GetFieldIndex("satreg2")
def check(lon, lat):
# create point geometry
pt = ogr.Geometry(ogr.wkbPoint)
pt.SetPoint_2D(0, lon, lat)
lyr_in.SetSpatialFilter(pt)
# go over all the polygons in the layer see if one include the point
for feat_in in lyr_in:
# roughly subsets features, instead of go over everything
ply = feat_in.GetGeometryRef()
# test
if ply.Contains(pt):
# TODO do what you need to do here
print(lon, lat, feat_in.GetFieldAsString(idx_reg))
Checkout http://geospatialpython.com/2011/01/point-in-polygon.html and http://geospatialpython.com/2011/08/point-in-polygon-2-on-line.html
One way to do this is to read the ESRI Shape file using the OGR
library Link and then use the GEOS geometry
library http://trac.osgeo.org/geos/ to do the point-in-polygon test.
This requires some C/C++ programming.
There is also a python interface to GEOS at http://sgillies.net/blog/14/python-geos-module/ (which I have never used). Maybe that is what you want?
Another solution is to use the http://geotools.org/ library.
That is in Java.
I also have my own Java software to do this (which you can download
from http://www.mapyrus.org plus jts.jar from http://www.vividsolutions.com/products.asp ). You need only a text command
file inside.mapyrus containing
the following lines to check if a point lays inside the
first polygon in the ESRI Shape file:
dataset "shapefile", "us_states.shp"
fetch
print contains(GEOMETRY, -120, 46)
And run with:
java -cp mapyrus.jar:jts-1.8.jar org.mapyrus.Mapyrus inside.mapyrus
It will print a 1 if the point is inside, 0 otherwise.
You might also get some good answers if you post this question on
https://gis.stackexchange.com/
If you want to find out which polygon (from a shapefile full of them) contains a given point (and you have a bunch of points as well), the fastest way is using postgis. I actually implemented a fiona based version, using the answers here, but it was painfully slow (I was using multiprocessing and checking bounding box first). 400 minutes of processing = 50k points. Using postgis, that took less than 10seconds. B tree indexes are efficient!
shp2pgsql -s 4326 shapes.shp > shapes.sql
That will generate a sql file with the information from the shapefiles, create a database with postgis support and run that sql. Create a gist index on the geom column. Then, to find the name of the polygon:
sql="SELECT name FROM shapes WHERE ST_Contains(geom,ST_SetSRID(ST_MakePoint(%s,%s),4326));"
cur.execute(sql,(x,y))
Related
I want to select grid cells from ERA5 gridded data (surface level only) that are inside geographical masks for North- and South-Switzerland (plus the radar buffer), to calculate regional means.
The 4 masks (masks) are given as polygons/multipolygons (polygons) in a shapefile and so far for 2 of the masks I was able to use salem roi to get what I want:
radar_north = salem.read_shapefile('radar_north140.shp')
file_radar_north = file.salem.roi(shape=radar_north)
file_radar_north.cape.mean(dim='time').salem.quick_map()
However, for the radar_south and alpensuedseite shapefiles the code didn´t work at the beginning (wrong selection or shows no data), and now the nothing works anymore (?). I don´t know why, as I have not changed anything from the first time to the second.
If someone sees the issue or knows a different way to mask the ERA data (which is maybe quicker) I would be grateful! (I was unsuccessfull with the answers from similar questions here).
Best
Lena
This could work if you are working on netcdf files
import geopandas as gpd
import xarray as xr
import rioxarray
from shapely.geometry import mapping
# load shapefile with geopandas
radar_north = gpd.read_file('radar_north140.shp')
# load ERA5 netcdf with xarray
era = xr.open_dataset('ERA5.nc')
# add projection system to nc
era = era.rio.write_crs("EPSG:4326", inplace=True)
# mask ERA5 data with shapefile
era_radar_north = era.rio.clip(radar_north.geometry.apply(mapping), radar_north.crs)
I have big problems by using GDAL in my Anaconda Spyder, but I need to do the following coordinate transformations:
src_spatialReference = osr.SpatialReference()#input coordinate system
src_spatialReference.ImportFromEPSG(inputCoordSystem) #import coordinate system from EPSG code
dst_spatialReference = osr.SpatialReference()#output coordinate system
dst_spatialReference.ImportFromEPSG(outputCoordSystem) #import coordinate system from EPSG code
transform_coord = osr.CoordinateTransformation(src_spatialReference,dst_spatialReference)
#transform geometry object
geomObj = ogr.CreateGeometryFromWkt(geomObj.wkt) #get geometric Object as WKT (well known text) and transform it to org geoemtry
geomObj.Transform(transform_coord) #transform object to other coordinate system
geomObjTransf=shapely.wkt.loads(geomObj.ExportToWkt()) #create shapely object form org geometry objekt
Does someone has an idea how to do it without GDAL?
I tried it with transform from shapely but I read it is only for points and I need to transform shapefiles and it is just for points.
I am using Spyder 3.2.8 in Anaconda and Python 3.6.4
Thank you very much for your help!
Maybe it is not the best solution but this one wors for me now:
from shapely.geometry import Point
from shapely.geometry import LineString
from pyproj import Proj, transform
df_all = df
# store the measured points in a list
measuredpoints_list = np.array([df_all.LON20Hz.tolist(), df_all.LAT20Hz.tolist()]).T
# transform them to a proj. coord. system
measuredpoints_listtrans = []
for i in range(len(measuredpoints_list)):
measuredpoints_listtrans.append(PyProjTransform(4326, 3581, measuredpoints_list[i]))
cl_trans = []
for i in range(len(cl)):
cl_trans.append(PyProjTransform(4326, 3581, cl[i]))
With shapely it is possible to transform a shapefile point wise, after that, you can create a new array which contains the transformed points. With this, you can start further calculations.
I want to reduce the number of triangles in a mesh (STL file). Here is my code:
import vtk
filename = 'E://stl_file.stl'
reader = vtk.vtkSTLReader()
reader.SetFileName('filename.stl')
##convert polygonal mesh into triangle mesh
tri = vtk.vtkTriangleFilter()
tri.SetInputConnection(reader.GetOutputPort());
##decimate triangle
deci = vtk.vtkDecimatePro()
deci.SetInputConnection(tri.GetOutputPort())
deci.SetTargetReduction(0.9)
deci.PreserveTopologyOn()
it seems to work (at least it runs without errors). Now how can I extract points and triangles of the mesh?
Just as with most other vtk filters, deci.GetOuptut() will give you the result, which in this case should be a vtkPolyData that is a decimated version of your input mesh. you can get the points from the output object by output.GetPoints(), triangles by output.GetPolys() etc., see the documentation pages at http://www.vtk.org/doc/nightly/html/classvtkPolyData.html
BTW, there is a whole page with examples of VTK filters that would have given you the anwer, e.g. http://www.vtk.org/Wiki/VTK/Examples/Cxx/Meshes/Decimation. It's in C++ but it works in python the same way.
I have files in ESRI Shapefile format. May I know how can I use QGIS or python gdal library to extract the latitude and longitude? I have installed QGIS and add layer of the shapefile into it, I can view the (lat, long) on the map but I do not know how to extract the (lat,long) out.. Do advice me how to extract the (lat,long) out. TQ
Right click your layer, select "Save As". The dialog "Save vector layer as..." will open. Select the following options:
Format: Comma Separated Value [CSV]
File name: Select a file name
CRS: EPSG:4326, WGS84
In the lower part, expand "Select fields to export" and click "Deselect All" if you just want the coordinates, or check the available items (columns from the attribute table) if you need place names, etc. By choosing WGS84 as the output CRS you'll receive your coordinates in decimal degrees (latitude, longitude).
Example output:
X,Y
12.4533865449718,41.9032821799601
12.4417701578001,43.936095834768
9.51666947290727,47.1337237742936
If you want to do it with python, you don't need gdal for that. I find geopandas (with Shapely) a much simpler library for working with shapefiles. Though at first it might take a few minutes longer than clicking through QGIS interface, learning how to manipulate shapefiles in Python is very powerful knowledge in a long run. Here is a sample of your task, performed with python:
import geopandas as gpd
# read shapefile (attributes and geometry) into geodataframe
shp = gpd.read_file("sample_shapefile.shp")
# convert "geometry" field into separate X and Y fields
shp["X"]=shp["geometry"].apply(lambda geom: geom.x)
shp["Y"]=shp["geometry"].apply(lambda geom: geom.y)
#save X,Y into csv file
shp[["X","Y"]].to_csv("coords.csv",header=True,index=False,sep=",")
Whick will produce the desired output in coords.csv:
X,Y
425070.0,80330.0
408390.0,81150.0
405370.0,85860.0
410880.0,82850.0
415310.0,80630.0
418610.0,80350.0
I used a builders' level to get x,y,z coordinates on a 110' x 150' building lot.
They are not in equally spaced rows and columns, but are randomly placed.
I have found a lot of info on mapping and I'm looking forward to learning about GIS. And how to use the many free software utilities out there.
Where should I start?
Now the data is in a csv file format, but I could change that.
It seems that I want to get the information I have into a "shapefile" or a raster format.
I supose I could look up the formats and do this, but it seems that I havn't come accross the proper utility for this part of the process.
Thank You Peter
You can convert your coordinate into a shapefile to display them in QGIS, ArcMAP, or similar GIS programs. You probably want a polygon shapefile.
One easy way to do this is with the PySAL
>>> import pysal
>>> coords = [(0,0), (10,0), (10,10), (0,10), (0,0)]
>>> pts = map(pysal.cg.Point, coords)
>>> polygon = pysal.cg.Polygon(pts)
>>> shp = pysal.open('myPolygon.shp','w')
>>> shp.write(polygon)
>>> shp.close()
Note: pysal currently doesn't support (Z coordinates) but there are plenty of similar libraries that do.
Also notice the first and last point are the same, indicating a closed polygon.
If your X,Y,Z coordinates are GPS coordinates you'll be able to align your data with other GIS data easily by telling the GIS what projection your data is in (WGS84, UTM Zone #, etc). If your coordinates are in local coordinates (not tied to a grid like UTM, etc) you'll need to "Georeference" you coordinates in order to align them with other data.
Finally using the ogr2ogr command you can easilly export your data from shapefile to other formats like KML,
ogr2ogr -f KML myPolygon.kml myPolygon.shp
You can convert a CSV file into any OGR supported format. All you need is a header file for the CSV file.
Here you have an example:
<ogrvrtdatasource>
<ogrvrtlayer name="bars">
<srcdatasource>bars.csv</srcdatasource>
<geometrytype>wkbPoint</geometrytype>
<layersrs>EPSG:4326</layersrs>
<geometryfield encoding="PointFromColumns" x="longitude" y="latitude">
</geometryfield>
</ogrvrtlayer>
</ogrvrtdatasource>
In the datasource field you set the CSV file name.
In your case, you have points, so the example is ok.
The field layersrs indicates the projection of the coordinates. If you have longitude and latitude, this one is ok.
The geometryfields must contain the x and y properties, that define the columns in the CSV file that containt the coordinates. The CSV file must have a first line defining the field names.
Save the file with a .vrt extension.
Once you have this, use the ogr2ogr program, which you have if GDAL is installed.
If you want to convert the file to a Shapefile, just type in a console:
ogr2ogr -f "ESRI Shapefile" bars.shp bars.vrt
If your question is what to do with the data, you can check the gdal_grid utility program, which converts scattered data (as yours) to raster data. You can use the CSV with the vrt header file as the input, without changing the format.