I have a dataset coming from a shape file (.shp extention) with coordinates. They should look something like this:
-70.62 -33.43
-70.59 -33.29
And so on. I already have developed a way to plot this data with pyplot, where each green dot represents a tree and each line a street, which looks like this:
pyplot streets & trees
However, I need to draw a grid over it and color it's blocks depending on the amount of trees on each section. That way, the blocks with more trees would be colored with a stronger green, whereas the ones with less amount of trees would be a light green/yellow/red. Of course, these colors should be partially transparent, so the map isn't covered completely.
This is my code:
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import cartopy.io.shapereader as shpreader
import shapely.geometry as sg
wgs84 = ccrs.Geodetic()
utm19s = ccrs.UTM(19, southern_hemisphere=True)
p_a = [-70.637, -33.449]
p_b = [-70.58, -33.415]
LL = utm19s.transform_point(p_a[0], p_a[1], wgs84)
UR = utm19s.transform_point(p_b[0], p_b[1], wgs84)
ax = plt.axes(projection=utm19s)
ax.set_extent([LL[0], UR[0], LL[1], UR[1]], crs=utm19s)
rds = shpreader.Reader('roadsUTM.shp')
trees = shpreader.Reader('treesUTM.shp')
rect = sg.box(LL[0], UR[0], LL[1], UR[1])
rds_sel = [r for r in rds.geometries() if r.intersects(rect)]
trees_sel = [t for t in trees.geometries() if t.intersects(rect)]
ax.add_geometries(rds_sel, utm19s, linestyle='solid', facecolor='none')
ax.scatter([t.x for t in trees_sel], [t.y for t in trees_sel], color = "green", edgecolor = "black", transform=utm19s)
plt.show()
TL;DR: A way to use shapefile encripted position data as plain numbers would solve part of my problem. Thanks.
EDIT: So I discovered that the data was already given in the UTM19S format. Should have researched a little bit before asking.
However, I still need to plot said grid over the map.
Related
I'd like to know how to fill in a map of U.S. counties by value (i.e., a chloropleth map), using Python 3 and Cartopy, and I haven't yet found anything online to guide me in that. That filled value could be, for instance, highest recorded tornado rating (with counties left blank for no recorded tornadoes), or even something arbitrary such as whether I've visited (=1) or lived (=2) in the county. I found a helpful MetPy example to get the county boundaries on a map:
https://unidata.github.io/MetPy/latest/examples/plots/US_Counties.html
What I envision is somehow setting a list (or dictionary?) of county names to a certain value, and then each value would be assigned to a particular fill color. This is my current script, which generates a nice blank county map of the CONUS/lower 48 (though I'd eventually also like to add Alaska/Hawaii insets).
import cartopy
import cartopy.crs as ccrs
import matplotlib as mpl
import matplotlib.pyplot as plt
from metpy.plots import USCOUNTIES
plot_type = 'png'
borders = cartopy.feature.BORDERS
states = cartopy.feature.NaturalEarthFeature(category='cultural', scale='10m', facecolor='none', name='admin_1_states_provinces_lakes')
oceans = cartopy.feature.OCEAN
lakes = cartopy.feature.LAKES
mpl.rcParams['figure.figsize'] = (12,10)
water_color = 'lightblue'
fig = plt.figure()
ax = plt.axes(projection=ccrs.LambertConformal(central_longitude=-97.5, central_latitude=38.5, standard_parallels=(38.5,38.5)))
ax.set_extent([-120, -74, 23, 50], ccrs.Geodetic())
ax.coastlines()
ax.add_feature(borders, linestyle='-')
ax.add_feature(states, linewidth=0.50, edgecolor='black')
ax.add_feature(oceans, facecolor=water_color)
ax.add_feature(lakes, facecolor=water_color, linewidth=0.50, edgecolor='black')
ax.add_feature(USCOUNTIES.with_scale('500k'), linewidth=0.10, edgecolor='black')
plt.savefig('./county_map.'+plot_type)
plt.close()
Any ideas or tips on how to assign values to counties and fill them accordingly?
So Cartopy's shapereader.Reader can give you access to all of the records in the shapefile, including their attributes. Putting this together with MetPy's get_test_data to get access to the underlying shapefile you can get what you want, assuming you have a dataset that maps e.g. FIPSCODE to EF rating:
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection=ccrs.PlateCarree())
cmap = plt.get_cmap('magma')
norm = plt.Normalize(0, 5)
# Fake tornado dataset with a value for each county code
tor_data = dict()
# This will only work (have access to the shapefile's database of
# attributes after it's been download by using `USCOUNTIES` or
# running get_test_data() for the .shx and .dbf files as well.
for rec in shpreader.Reader(get_test_data('us_counties_20m.shp',
as_file_obj=False)).records():
# Mimic getting data, but actually getting a random number
# GEOID seems to be the FIPS code
max_ef = tor_data.get(rec.attributes['GEOID'], np.random.randint(0, 5))
# Normalize the data to [0, 1] and colormap manually
color = tuple(cmap(norm(max_ef)))
# Add the geometry to the plot, being sure to specify the coordinate system
ax.add_geometries([rec.geometry], crs=ccrs.PlateCarree(), facecolor=color)
ax.set_extent((-125, -65, 25, 48))
That gives me:
I'm not sure about passing in a dict, but you can pass in a list to facecolor.
ax.add_feature(USCOUNTIES.with_scale('500k'), linewidth=0.10, edgecolor='black', facecolor=["red", "blue", "green"])
If you know how many counties there are you can make a list that long by:
import matplotlib.cm as cm
import numpy as np
number_of_counties = 3000
color_scale = list(cm.rainbow(np.linspace(0, 1, number_of_counties)))
ax.add_feature(USCOUNTIES.with_scale('500k'), linewidth=.10, edgecolor="black", facecolor=color_scale)
but they didn't make it easy to extract the names from USCOUNTIES. You can see where it is defined in your source code:
from metpy import plots
print(plots.__file__)
If you go inside the directory printed there is a file named cartopy_utils.py and inside the class definition for class MetPyMapFeature(Feature): you will see USCOUNTIES. You might have better luck than I did mapping county names to the geometric shapes.
EDIT: Also, I just used cm.rainbow as an example, you can choose from any color map https://matplotlib.org/stable/tutorials/colors/colormaps.html. Not sure if it even goes up to 3000, but you get the idea.
I have a dataframe that contains thousands of points with geolocation (longitude, latitude) for Washington D.C. The following is a snippet of it:
import pandas as pd
df = pd.DataFrame({'lat': [ 38.897221,38.888100,38.915390,38.895100,38.895100,38.901005,38.960491,38.996342,38.915310,38.936820], 'lng': [-77.031048,-76.898480,-77.021380,-77.036700,-77.036700 ,-76.990784,-76.862907,-77.028131,-77.010403,-77.184930]})
If you plot the points in the map you can see that some of them are clearly within some buildings:
import folium
wash_map = folium.Map(location=[38.8977, -77.0365], zoom_start=10)
for index,location_info in df.iterrows():
folium.CircleMarker(
location=[location_info["lat"], location_info["lng"]], radius=5,
fill=True, fill_color='red',).add_to(wash_map)
wash_map.save('example_stack.html')
import webbrowser
import os
webbrowser.open('file://'+os.path.realpath('example_stack.html'), new=2)
My goal is to exclude all the points that are within buildings. For that, I first download bounding boxes for the city buildings and then try to exclude points within those polygons as follows:
import osmnx as ox
#brew install spatialindex this solves problems in mac
%matplotlib inline
ox.config(log_console=True)
ox.__version__
tags = {"building": True}
gdf = ox.geometries.geometries_from_point([38.8977, -77.0365], tags, dist=1000)
gdf.shape
For computational simplicity I have requested the shapes of all buildings around the White house with a radius of 1 km. On my own code I have tried with bigger radiuses to make sure all the buildings are included.
In order to exclude points within the polygons I developed the following function (which includes the shape obtention):
def buildings(df,center_point,dist):
import osmnx as ox
#brew install spatialindex this solves problems in mac
%matplotlib inline
ox.config(log_console=True)
ox.__version__
tags = {"building": True}
gdf = ox.geometries.geometries_from_point(center_point, tags,dist)
from shapely.geometry import Point,Polygon
# Next step is to put our coordinates in the correct shapely format: remember to run the map funciton before
#df['within_building']=[]
for point in range(len(df)):
if gdf.geometry.contains(Point(df.lat[point],df.lng[point])).all()==False:
df['within_building']=False
else :
df['within_building']=True
buildings(df,[38.8977, -77.0365],1000)
df['within_building'].all()==False
The function always returns that points are outside building shapes although you can clearly see in the map that some of them are within. I don't know how to plot the shapes over my map so I am not sure if my polygons are correct but for the coordinates they appear to be so. Any ideas?
The example points you provided don't seem to fall within those buildings' footprints. I don't know what your points' coordinate reference system is, so I guessed EPSG4326. But to answer your question, here's how you would exclude them, resulting in gdf_points_not_in_bldgs:
import geopandas as gpd
import matplotlib.pyplot as plt
import osmnx as ox
import pandas as pd
# the coordinates you provided
df = pd.DataFrame({'lat': [38.897221,38.888100,38.915390,38.895100,38.895100,38.901005,38.960491,38.996342,38.915310,38.936820],
'lng': [-77.031048,-76.898480,-77.021380,-77.036700,-77.036700 ,-76.990784,-76.862907,-77.028131,-77.010403,-77.184930]})
# create GeoDataFrame of point geometries
geom = gpd.points_from_xy(df['lng'], df['lat'])
gdf_points = gpd.GeoDataFrame(geometry=geom, crs='epsg:4326')
# get building footprints
tags = {"building": True}
gdf_bldgs = ox.geometries_from_point([38.8977, -77.0365], tags, dist=1000)
gdf_bldgs.shape
# get all points that are not within a building footprint
mask = gdf_points.within(gdf_bldgs.unary_union)
gdf_points_not_in_bldgs = gdf_points[~mask]
print(gdf_points_not_in_bldgs.shape) # (10, 1)
# plot buildings and points
ax = gdf_bldgs.plot()
ax = gdf_points.plot(ax=ax, c='r')
plt.show()
# zoom in to see better
ax = gdf_bldgs.plot()
ax = gdf_points.plot(ax=ax, c='r')
ax.set_xlim(-77.04, -77.03)
ax.set_ylim(38.89, 38.90)
plt.show()
Starting with a shapefile I obtained from https://s3.amazonaws.com/nyc-tlc/misc/taxi_zones.zip, I'd like to plot the borough of Manhattan, and have outlines for each taxi-zone.
This code rotates each individual taxi zone individually instead of all at once.
import geopandas as gpd
from matplotlib import pyplot as plt
fname = "path_to_shapefile.shp"
df = gpd.read_file(fname)
df = df[df['borough'] == "Manhattan"]
glist = gpd.GeoSeries([g for g in df['geometry']])
glist = glist.rotate(90)
glist.plot()
[EDIT]
I have further refined this to be able to rotate the image programmatically. However, if I add a legend, then that is also rotated, which is not desirable. Still looking for a better solution.
Note, there is also this stackoverflow post (How can I rotate a matplotlib plot through 90 degrees?), however, the solutions that rotate the plot, and not the image, only work with 90 degree rotations.
import geopandas as gpd
from matplotlib import pyplot as plt
import numpy as np
from scipy import ndimage
from matplotlib import transforms
fname = "path_to_shapefile.shp"
df = gpd.read_file(fname)
df = df[df['borough'] == "Manhattan"]
df.plot()
plt.axis("off")
plt.savefig("test.png")
img = plt.imread('test.png')
rotated_img = ndimage.rotate(img, -65)
plt.imshow(rotated_img, cmap=plt.cm.gray)
plt.axis('off')
plt.show()
[EDIT2]
A simple modification to the answer given below by #PMende solved it.
df = gpd.read_file(fname)
df = df[df['borough'] == "Manhattan"]
glist = gpd.GeoSeries([g for g in df['geometry']])
glist = glist.rotate(-65, origin=(0,0))
glist.plot()
The key was rotating all of the objects around a single point, instead of around their individual origins.
[EDIT 3] If anyone is trying to do this, and needs to save the resulting rotated geoseries to a dataframe (say for instance, to color the geometry based on an additional column), you need to create a new one, simply writing
df['geometry'] = glist
does not work. I'm not sure why at the moment. However, the following code worked for me.
new_dataframe = gpd.GeoDataFrame(glist)
new_dataframe = new_dataframe.rename(columns={0:'geometry'}).set_geometry('geometry')
new_dataframe.plot()
If I'm understanding GeoPandas' documentation correctly, you can specify the origin of the rotation of each of your geometries (which by default is the center of each geometry). To get your desired behavior, you can rotate each shape about the same origin.
For example:
import geopandas as gpd
from matplotlib import pyplot as plt
fname = "path_to_shapefile.shp"
df = gpd.read_file(fname)
df = df[df['borough'] == "Manhattan"]
center = df["geometry"].iloc[0].centroid()
glist = gpd.GeoSeries([g for g in df['geometry']])
glist = glist.rotate(90, origin=center)
glist.plot()
I can't test this myself, but it should hopefully get you started in the right direction.
(Though I also agree with #martinfeleis' point about not necessarily wanting to rotate the geometry, but rather the plot.)
I'm trying to plot data around the Antarctica while masking the continent. While I'm using basemap and it has an option to easily mask continents using map.fillcontinents(), the continent considered by basemap includes the ice shelves, which I do not want to mask.
I tried using geopandas from a code I found on the Internet. This works, except the coastline produces an undesired line in what I assume is the beginning/end of the polygon for the Antarctica:
import numpy as np
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
from matplotlib.collections import PatchCollection
import geopandas as gpd
import shapely
from descartes import PolygonPatch
lats = np.arange(-90,-59,1)
lons = np.arange(0,361,1)
X, Y = np.meshgrid(lons, lats)
data = np.random.rand(len(lats),len(lons))
world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
fig=plt.figure(dpi=150)
ax = fig.add_subplot(111)
m = Basemap(projection='spstere',boundinglat=-60,lon_0=180,resolution='i',round=True)
xi, yi = m(X,Y)
cf = m.contourf(xi,yi,data)
patches = []
selection = world[world.name == 'Antarctica']
for poly in selection.geometry:
if poly.geom_type == 'Polygon':
mpoly = shapely.ops.transform(m, poly)
patches.append(PolygonPatch(mpoly))
elif poly.geom_type == 'MultiPolygon':
for subpoly in poly:
mpoly = shapely.ops.transform(m, poly)
patches.append(PolygonPatch(mpoly))
else:
print(poly, 'blah')
ax.add_collection(PatchCollection(patches, match_original=True,color='w',edgecolor='k'))
The same line appears when I try to use other shapefiles, such as the land one that is available to download for free from Natural Earth Data. So I edited this shapefile in QGIS to remove the borders of the Antarctica. The problem now is that I don't know how to mask everything that's inside the shapefile (and couldn't find how to do it either). I also tried combining the previous code with geopandas by setting the linewidth=0, and adding on top the shapefile I created. The problem is that they are not exactly the same:
Any suggestion on how to mask using a shapefile, or with geopandas but without the line?
Edit: Using Thomas Khün's previous answer with my edited shapefile produces a well masked Antarctica/continents, but the coastline goes outside the round edges of the map:
I uploaded here the edited shapefile I used, but it's the Natural Earth Data 50m land shapefile without the line.
Here an example of how to achieve what you want. I basically followed the Basemap example how to deal with shapefiles and added a bit of shapely magic to restrict the outlines to the map boundaries. Note that I first tried to extract the map outline from ax.patches, but that somehow didn't work, so I defined a circle which has a radius of boundinglat and transformed it using the Basemap coordinate transformation functionality.
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap
from matplotlib.collections import PatchCollection
from matplotlib.patches import Polygon
import shapely
from shapely.geometry import Polygon as sPolygon
boundinglat = -40
lats = np.arange(-90,boundinglat+1,1)
lons = np.arange(0,361,1)
X, Y = np.meshgrid(lons, lats)
data = np.random.rand(len(lats),len(lons))
fig, ax = plt.subplots(nrows=1, ncols=1, dpi=150)
m = Basemap(
ax = ax,
projection='spstere',boundinglat=boundinglat,lon_0=180,
resolution='i',round=True
)
xi, yi = m(X,Y)
cf = m.contourf(xi,yi,data)
#adjust the path to the shapefile here:
result = m.readshapefile(
'shapefiles/AntarcticaWGS84_contorno', 'antarctica',
zorder = 10, color = 'k', drawbounds = False)
#defining the outline of the map as shapely Polygon:
rim = [np.linspace(0,360,100),np.ones(100)*boundinglat,]
outline = sPolygon(np.asarray(m(rim[0],rim[1])).T)
#following Basemap tutorial for shapefiles
patches = []
for info, shape in zip(m.antarctica_info, m.antarctica):
#instead of a matplotlib Polygon, create first a shapely Polygon
poly = sPolygon(shape)
#check if the Polygon, or parts of it are inside the map:
if poly.intersects(outline):
#if yes, cut and insert
intersect = poly.intersection(outline)
verts = np.array(intersect.exterior.coords.xy)
patches.append(Polygon(verts.T, True))
ax.add_collection(PatchCollection(
patches, facecolor= 'w', edgecolor='k', linewidths=1., zorder=2
))
plt.show()
The result looks like this:
Hope this helps.
For anyone still trying to figure out a simple way to mask a grid from a shapefile, here is a gallery example from the python package Antarctic-Plots which makes this simple.
from antarctic_plots import maps, fetch, utils
import pyogrio
# fetch a grid and shapefile
grid = fetch.bedmachine(layer='surface')
shape = fetch.groundingline()
# subset the grounding line from the coastline
gdf = pyogrio.read_dataframe(shape)
groundingline = gdf[gdf.Id_text == "Grounded ice or land"]
# plot the grid
fig = maps.plot_grd(grid)
# plot the shapefile
fig.plot(groundingline, pen='1p,red')
fig.show()
# mask the inside region
masked_inside = utils.mask_from_shp(
shapefile=groundingline, xr_grid=grid, masked=True)
masked_inside.plot()
# mask the outside region
masked_outside = utils.mask_from_shp(
shapefile=groundingline, xr_grid=grid, masked=True, invert=False)
masked_outside.plot()
I'm very new to Python but have been learning lots over the last few months. I'm trying to plot NOAA swell height data from a grib2 file located here: ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20140122/nww3.t06z.grib.grib2
I use Basemap and a tutorial that I found on a Basemap forum.
A minimum working example is below, but I'm getting some strange white boxes around the coastline.
import Nio
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
f = Nio.open_file('nww3.t12z.grib(2).grib2')
lons = f.variables['lon_0'][:]
lats = f.variables['lat_0'][::-1] # flip latitudes so data goes S-->N
times = f.variables['forecast_time0'][:]
ntime = 5
data = f.variables['HTSGW_P0_L1_GLL0'][ntime,::-1]
fig = plt.figure(figsize=(16,16))
m = Basemap(llcrnrlon=-35.,llcrnrlat=42.,urcrnrlon=5.,urcrnrlat=65.,
projection='lcc',lat_1=10.,lat_2=15.,lon_0=10.,
resolution ='h',area_thresh=1000.)
x, y = m(*np.meshgrid(lons, lats))
m.fillcontinents(color='#477519')
m.drawcoastlines(linewidth=0.5, color='k', antialiased=1, ax=None, zorder=None )
m.contourf(x, y, data, np.arange(0,9.9,0.1))
plt.show()
This is the result (the top panel; I would like it to look like the bottom panel): http://oi43.tinypic.com/s2s3m0.jpg
Sorry I don't have enough points to post images.
Thanks in advance,
Al