Develop Reference

Python folium synchronizing 2 markerclusters

I have a situation, where I decided to distinguish my markers slightly by adding the MarkerCluster to the existing Marker.
The clustering works, although isn't synchronized. It means, that if I switch one layer off, just the first markercluster disappears, whereas the second criterion defined as CircleMarker still appears like shown below.
My code is:
df = pd.read_csv("or_geo.csv")
fo=FeatureGroup(name="OR",overlay = True)
openreach_cluster = MarkerCluster(name="OR").add_to(map)
openreach_status = MarkerCluster(control=False,
visible=True
).add_to(map)
for i,row in df.iterrows():
lat =df.at[i, 'lat']
lng = df.at[i, 'lng']
sp = df.at[i, 'sp']
stat = df.at[i,'status']
popup = df.at[i,'sp'] +'<br>' + str(df.at[i, 'street']) + '<br>' + str(df.at[i, 'post code']) + '<br>{}'.format(style)
or_marker = folium.Marker(location=[lat,lng], tooltip='<strong>Job details</strong>', popup=popup, icon = folium.Icon(
color='blue', icon='glyphicon-calendar'))
or_stat_marker = folium.CircleMarker(
location=[lat,lng],
radius=10,
color=or_color(stat),
fill_color=or_color(stat),
fill_opacity=0.5)
openreach_cluster.add_child(or_marker)
openreach_status.add_child(or_stat_marker)
Is there any way to combine these markerClusters together or sync them?
UPDATE:
The first approach from the answer below, unfortunately, doubles the jobs up and the user can't see them until clicks on any of them shown below:
This option would be fantastic if the behavior could be the same as in the image above.
UPDATE II:
The second approach is still not what I am looking for, because the clusters are doubled again and after clicking the circlemarker falls almost in the opposite direction as presented in the image above.
I need to have the behaviors exactly like those displayed on the top-left image. The circlemarker should be integrated with the point marker.

If I got you right, I think there are two possibilities:
Use the same cluster
Add your markers from or_stat_marker to the openreach_cluster and not to another cluster to de-/activate them at the same time with the same button
# was
openreach_cluster.add_child(or_marker)
openreach_status.add_child(or_stat_marker)
# try this
openreach_cluster.add_child(or_marker)
openreach_cluster.add_child(or_stat_marker)
Use marker subgroups
In this case you will have three checkmarks, one parent and two childs for each markercluster which gives full control to visibility
UPDATE: If you add the control=False option to the subgroup you will only see the parent group in LayerControl which then show/hide both groups. But the matter of markers "spreading for visibility" still is a problem I think
Another UPDATE: You are able to deactivate the clustering according to the map zoom level or even at all by using disableClusteringAtZoom option (use True or an int number for zoom level). See here for reference
# markergroups in layercontrol
mc = folium.plugins.MarkerCluster(name='OR',
overlay=True,
control=True,
show=True,
disableClusteringAtZoom=15) # choose zoom lvl to your needs
mc.add_to(map)
sub1 = folium.plugins.FeatureGroupSubGroup(mc, name='openreach_cluster', control=False, show=True) # False --> deactivated at start
sub1.add_to(map)
sub2 = folium.plugins.FeatureGroupSubGroup(mc, name='openreach_status', control=False)
sub2.add_to(map)
# the layercontrol itself
lc = folium.map.LayerControl(collapsed=False)
lc.add_to(map)
# ...
for i, row in df.iterrows():
# ...
or_marker = folium.Marker(...)
or_marker.add_to(sub1)
or_stat_marker = folium.CircleMarker(...)
or_stat_marker.add_to(sub2)
My Result:
By following the second approach described I get a map shown here. There is one checkmark "Segment Markers" which shows/hides the markers including the circles. They don't move around when clicked and are fully shown when zoomed in to a specific lvl by using disableClusteringAtZoom.
Sorry this didn't work for your problem, I really don't know why ..

Weird "demonic" xtick in matplotlib (jpeg artifacts? No way...)

So I'm comparing NBA betting lines between different sportsbooks over time
Procedure:
Open pickle file of scraped data
Plot the scraped data
The pickle file is a dictionary of NBA betting lines over time. Each of the two teams are their own nested dictionary. Each key in these team-specific dictionaries represents a different sportsbook. The values for these sportsbook keys are lists of tuples, representing timeseries data. It looks roughly like this:
dicto = {
'Time': <time that the game starts>,
'Team1': {
Market1: [ (time1, value1), (time2, value2), etc...],
Market2: [ (time1, value1), (time2, value2), etc...],
etc...
}
'Team2': {
<SAME FORM AS TEAM1>
}
}
There are no issues with scraping or manipulating this data. The issue comes when I plot it. Here is the code for the script that unpickles and plots these dictionaries:
import matplotlib.pyplot as plt
import pickle, datetime, os, time, re
IMAGEPATH = 'Images'
reg = re.compile(r'[A-Z]+#[A-Z]+[0-9|-]+')
noDate = re.compile(r'[A-Z]+#[A-Z]+')
# Turn 1 into '01'
def zeroPad(num):
if num < 10:
return '0' + str(num)
else:
return num
# Turn list of time-series tuples into an x list and y list
def unzip(lst):
x = []
y = []
for i in lst:
x.append(f'{i[0].hour}:{zeroPad(i[0].minute)}')
y.append(i[1])
return x, y
# Make exactly 5, evenly spaced xticks
def prune(xticks):
last = len(xticks)
first = 0
mid = int(len(xticks) / 2) - 1
upMid = int( mid + (last - mid) / 2)
downMid = int( (mid - first) / 2)
out = []
count = 0
for i in xticks:
if count in [last, first, mid, upMid, downMid]:
out.append(i)
else:
out.append('')
count += 1
return out
def plot(filename, choice):
IMAGEPATH = 'Images'
IMAGEPATH = os.path.join(IMAGEPATH, choice)
with open(filename, 'rb') as pik:
dicto = pickle.load(pik)
fig, axs = plt.subplots(2)
gameID = noDate.search(filename).group(0)
tm = dicto['Time']
fig.suptitle(gameID + '\n' + str(tm))
i = 0
for team in dicto.keys():
axs[i].set_title(team)
if team == 'Time':
continue
for market in dicto[team].keys():
lst = dicto[team][market]
x, y = unzip(lst)
axs[i].plot(x, y, label= market)
axs[i].set_xticks(prune(x))
axs[i].set_xticklabels(rotation=45, labels = x)
i += 1
plt.tight_layout()
#Finish
outputFile = reg.search(filename).group(0)
date = (datetime.datetime.today() - datetime.timedelta(hours = 6)).date()
fig.savefig(os.path.join(IMAGEPATH, str(date), f'{outputFile}.png'))
plt.close()
Here is the image that results from calling the plot function on one of the dictionaries that I described above. It is pretty much exactly as I intended it, except for one very strange and bothersome problem.
You will notice that the bottom right tick looks haunted, demonic, jpeggy, whatever you want to call it. I am highly suspicious that this problem occurs in the prune function, which I use to set the xtick values of the plot.
The reason that I prune the values with a function like this is because these dictionaries are continuously updated, so setting a static number of xticks would not work. And if I don't prune the xticks, they end up becoming unreadable due to overlapping one another.
I am quite confused as to what could cause an xtick to look like this. It happens consistently, for every dictionary, every time. Before I added the prune function (when the xticks unbound, overlapping one another), this issue did not occur. So when I say I'm suspicious that the prune function is the cause, I am really quite certain.
I will be happy to share an instance of one of these dictionaries, but they are saved as .pickle files, and I'm pretty sure it's bad practice to share pickle files over the internet. I have been warned about potential malware, so I'll just stay away from that. But if you need to see the dictionary, I can take the time to prettily print one and share a screenshot. Any help is greatly appreciated!

Matplotlib does this when there are many xticks or yticks which are plotted on the same value. It is normal. If you can limit the number of times the specific value is plotted - you can make it appear indistinguishable from the rest of the xticks.
Plot a simple example to test this out and you will see for yourself.

Find neighbouring polygons in Python QGIS

I am using code I found and slightly modified for my purposes. The problem is, it is not doing exactly what I want, and I am stuck with what to change to fix it.
I am searching for all neighbouring polygons, that share common borded (a line), that is not a point
My goal: 135/12 is neigbour with 319/2 135/4, 317 but not with 320/1
What I get in my QGIS table after I run my script
NEIGBOURS are the neighbouring polygons,
SUM is the number of neighbouring polygons
The code I use also includes 320/1 as a neighbouring polygon. How to fix it?
from qgis.utils import iface
from PyQt4.QtCore import QVariant
_NAME_FIELD = 'Nr'
_SUM_FIELD = 'calc'
_NEW_NEIGHBORS_FIELD = 'NEIGHBORS'
_NEW_SUM_FIELD = 'SUM'
layer = iface.activeLayer()
layer.startEditing()
layer.dataProvider().addAttributes(
[QgsField(_NEW_NEIGHBORS_FIELD, QVariant.String),
QgsField(_NEW_SUM_FIELD, QVariant.Int)])
layer.updateFields()
feature_dict = {f.id(): f for f in layer.getFeatures()}
index = QgsSpatialIndex()
for f in feature_dict.values():
index.insertFeature(f)
for f in feature_dict.values():
print 'Working on %s' % f[_NAME_FIELD]
geom = f.geometry()
intersecting_ids = index.intersects(geom.boundingBox())
neighbors = []
neighbors_sum = 0
for intersecting_id in intersecting_ids:
intersecting_f = feature_dict[intersecting_id]
if (f != intersecting_f and
not intersecting_f.geometry().disjoint(geom)):
neighbors.append(intersecting_f[_NAME_FIELD])
neighbors_sum += intersecting_f[_SUM_FIELD]
f[_NEW_NEIGHBORS_FIELD] = ','.join(neighbors)
f[_NEW_SUM_FIELD] = neighbors_sum
layer.updateFeature(f)
layer.commitChanges()
print 'Processing complete.'

I have found somewhat a workaround for it. Before using my script, I create a small (for my purposes, 0,01 m was enough) buffer around all joints. Later, I use a Difference tool to remove the buffer areas from my main layer, thus removing not-needed neighbouring polygons. Using the code now works fine

Count number of points in multipolygon shapefile using Python

I have a polygon shapefile of the U.S. made up of individual states as their attribute values. In addition, I have arrays storing latitude and longitude values of point events that I am also interested in. Essentially, I would like to 'spatial join' the points and polygons (or perform a check to see which polygon [i.e., state] each point is in), then sum the number of points in each state to find out which state has the most number of 'events'.
I believe the pseudocode would be something like:
Read in US.shp
Read in lat/lon points of events
Loop through each state in the shapefile and find number of points in each state
print 'Here is a list of the number of points in each state: '
Any libraries or syntax would be greatly appreciated.
Based on what I can tell, the OGR library is what I need, but I am having trouble with the syntax:
dsPolygons = ogr.Open('US.shp')
polygonsLayer = dsPolygons.GetLayer()
#Iterating all the polygons
polygonFeature = polygonsLayer.GetNextFeature()
k=0
while polygonFeature:
k = k + 1
print "processing " + polygonFeature.GetField("STATE") + "-" + str(k) + " of " + str(polygonsLayer.GetFeatureCount())
geometry = polygonFeature.GetGeometryRef()
#Read in some points?
geomcol = ogr.Geometry(ogr.wkbGeometryCollection)
point = ogr.Geometry(ogr.wkbPoint)
point.AddPoint(-122.33,47.09)
point.AddPoint(-110.11,33.33)
#geomcol.AddGeometry(point)
print point.ExportToWkt()
print point
numCounts=0.0
while pointFeature:
if pointFeature.GetGeometryRef().Within(geometry):
numCounts = numCounts + 1
pointFeature = pointsLayer.GetNextFeature()
polygonFeature = polygonsLayer.GetNextFeature()
#Loop through to see how many events in each state

I like the question. I doubt I can give you the best answer, and definitely can't help with OGR, but FWIW I'll tell you what I'm doing right now.
I use GeoPandas, a geospatial extension of pandas. I recommend it — it's high-level and does a lot, giving you everything in Shapely and fiona for free. It is in active development by twitter/#kajord and others.
Here's a version of my working code. It assumes you have everything in shapefiles, but it's easy to generate a geopandas.GeoDataFrame from a list.
import geopandas as gpd
# Read the data.
polygons = gpd.GeoDataFrame.from_file('polygons.shp')
points = gpd.GeoDataFrame.from_file('points.shp')
# Make a copy because I'm going to drop points as I
# assign them to polys, to speed up subsequent search.
pts = points.copy()
# We're going to keep a list of how many points we find.
pts_in_polys = []
# Loop over polygons with index i.
for i, poly in polygons.iterrows():
# Keep a list of points in this poly
pts_in_this_poly = []
# Now loop over all points with index j.
for j, pt in pts.iterrows():
if poly.geometry.contains(pt.geometry):
# Then it's a hit! Add it to the list,
# and drop it so we have less hunting.
pts_in_this_poly.append(pt.geometry)
pts = pts.drop([j])
# We could do all sorts, like grab a property of the
# points, but let's just append the number of them.
pts_in_polys.append(len(pts_in_this_poly))
# Add the number of points for each poly to the dataframe.
polygons['number of points'] = gpd.GeoSeries(pts_in_polys)
The developer tells me that spatial joins are 'new in the dev version', so if you feel like poking around in there, I'd love to hear how that goes! The main problem with my code is that it's slow.

import geopandas as gpd
# Read the data.
polygons = gpd.GeoDataFrame.from_file('polygons.shp')
points = gpd.GeoDataFrame.from_file('points.shp')
# Spatial Joins
pointsInPolygon = gpd.sjoin(points, polygons, how="inner", op='intersects')
# Add a field with 1 as a constant value
pointsInPolygon['const']=1
# Group according to the column by which you want to aggregate data
pointsInPolygon.groupby(['statename']).sum()
**The column ['const'] will give you the count number of points in your multipolygons.**
#If you want to see others columns as well, just type something like this :
pointsInPolygon = pointsInPolygon.groupby('statename').agg({'columnA':'first', 'columnB':'first', 'const':'sum'}).reset_index()
[1]: https://geopandas.org/docs/user_guide/mergingdata.html#spatial-joins
[2]: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html

How do you assign a short name to a variable in the netCDF4 module for python?

I'm making a netCDF file using code along the lines of:
from netCDF4 import Dataset
outfile = Dataset('output.nc', 'w', format = 'NETCDF4')
level = outfile.createDimension('level', 1)
time = outfile.createDimension('t', None)
lats = outfile.createDimension('latitude', 141)
lons = outfile.createDimension('longitude', 121)
precips = outfile.createVariable('Precipitation', 'f4',('t','level','latitude','longitude'))
times = outfile.createVariable('t','f8',('t',))
levels = outfile.createVariable('level','i4',('level',))
latitudes = outfile.createVariable('latitude','f4',('latitude',))
longitudes = outfile.createVariable('longitude','f4',('longitude',))
latitudes.units = "degrees east"
longitudes.units = "degrees north"
levels.units = "surface"
precips.units = "mm/day"
times.calendar = "gregorian"
That all works fine (once you fill the variables with data and call outfile.close()), but how do you assign a short name to a variable? I would expect something along the lines of:
precips.shortFieldName = "prec"
But, having tried a number of variations on that, and scanned the docs, and rifled through the source code, I'm no closer to a solution.
Any ideas?

prec=precips
^^Then there are pointers on this not two objects
=> 2 names which are both usable