I have a dataframe that denotes events that occured in particular locations.
I am aware that folium does not allow dynamic display of the appearance of the events so I was thinking about basically iterate through the dates and save a png of each folium map created.
Unfortunately I am mentally stuck in a 2 part problem:
1) how to loop through a ranges of dates (for example one map for each month)
2) an appropriate way to save the generated images for each loop.
This is a dataframe sample for this example:
since = ['2019-07-05', '2019-07-17', '2014-06-12', '2016-03-11']
lats = [38.72572, 38.71362, 38.79263, 38.71931]
longs = [-9.13412, -9.14407, -9.40824, -9.13143]
since_map = {'since' : pd.Series(since), 'lats' : pd.Series(lats), 'longs' : pd.Series(longs)}
since_df = pd.DataFrame(since_map)
I was able to create the base map:
lat_l = 38.736946
long_l = -9.142685
base_l = folium.Map(location=[lat_l,long_l], zoom_start=12)
neigh = folium.map.FeatureGroup()
And add some of the markers to the folium map:
for lati, longi in zip(since_df.longs, since_df.lats):
neigh.add_child(folium.CircleMarker([longi, lati], radius = 2, color = 'blue', fill = True))
base_l.add_child(neigh)
I am struggling to visualize how to loop through ranges of the dates and save each file. From what I saw here:
https://github.com/python-visualization/folium/issues/35 I actually have to open the saved html and then save it as png for each image.
If you could point me to an example or documentation that could demonstrate how this can be accomplished I would be very appreciative.
If you think I am overcomplicating it or you have a better alternative to what I am thinking I have an open ear to suggestions.
Thank you for your help.
Related
I have a MariaDB Table containing a MEDIUMBLOB Column. There are several entries in this table corresponding to one photo each.
When querying the data to PowerBI using the MariaDB connector, I get the Data in the format "Binary"
However, when querying the same data in Python (IDE or PowerBI) the format is different:
The bigger picture is to use this code to split the image in bits as PBI has a character-limit on their data elements:
Source = MariaDB.Contents("XXX.XXX.XXX.XXX:YYYY", "ZZZZZ"),
Query1= Source{[Name="Query1",Kind="Table"]}[Data],
#"Removed Top Rows" = Table.Skip(qc_westernblot_Table,1),
//Remove unnecessary columns
RemoveOtherColumns = Table.SelectColumns(#"Removed Top Rows",{"picture", "batchname"}),
//Creates Splitter function
SplitTextFunction = Splitter.SplitTextByRepeatedLengths(30000),
//Converts table of files to list
ListInput = Table.ToRows(RemoveOtherColumns),
//Function to convert binary of photo to multiple
//text values
ConvertOneFile = (InputRow as list) =>
let
BinaryIn = InputRow{0},
FileName = InputRow{1},
BinaryText = Binary.ToText(BinaryIn, BinaryEncoding.Base64),
SplitUpText = SplitTextFunction(BinaryText),
AddFileName = List.Transform(SplitUpText, each {FileName,_})
in
AddFileName,
//Loops over all photos and calls the above function
ConvertAllFiles = List.Transform(ListInput, each ConvertOneFile(_)),
//Combines lists together
CombineLists = List.Combine(ConvertAllFiles),
//Converts results to table
ToTable = #table(type table[Name=text,Pic=text],CombineLists),
//Adds index column to output table
AddIndexColumn = Table.AddIndexColumn(ToTable, "Index", 0, 1)
in
AddIndexColumn
As I am a beginner on this topic, I am confident there is a straight-forward conversion missing here but I couldn't figure it out so far myself.
I greatly appreciate any help. Thank you!
What are you planning to do with this data? PBI doesn't support binary data although you can see it in Power Query. It must be converted to something else before it can be loaded to the PBI data model.
I suspect the Python version is just the binary already converted to text. If you click the two arrows in the top right of the picture column for the PBI version, do you not get the same output?
From an stack of geotiff (time-series of NDVI) in a file like 'NDVI_TS.tif' I want to get individual band names. For example: 'Band 086: 20190803T004719'. I can see that when I load the stack into QGIS for example. I need to trace back the dates in the name, in the example above would be 2019-08-03. However, I can't find a way to access it from Python. I can access the bands by index but that doesn't help me finding from which date they are from.
from osgeo import gdal
NDVI = gdal.Open('NDVI_TS.tif', gdal.GA_ReadOnly)
#for example I can get data from band 86 as and array with:
band86 = NDVI.GetRasterBand(86).ReadAsArray()
I feel there should be some easy solution for this but failed to find.
I don't know if this is efficient, but is the way I solve this (only tested with small images):
bands = {NDVI.GetRasterBand(i).GetDescription(): i for i in range(1, NDVI.RasterCount + 1)}
You get a dictionary with the band name (the one you see in QGIS) and their respective index.
{ ...,
'Band 086: 20190803T004719': 86,
...
}
You can create a small function to parse the band name and get the dates as you want (didn't test it properly):
import datetime
import re
def string2date(string):
date_string = re.match('.* (.*)T', string).group(1)
return datetime.datetime.strptime(date_string, '%Y%m%d').strftime('%Y-%m-%d')
And then you can apply to the previous dict:
bands = {string2date(NDVI.GetRasterBand(i).GetDescription()): i for i in range(1, NDVI.RasterCount + 1)}
Hope it helps
I'm making a map of the different bus routes in my city, I have the [lat,lon] of every station and point but the problem is that I can't seem to find a way to name the routes that I created.
Here's the code:
>df = pd.read_excel (r'Bus.xlsx')
>points= pd.DataFrame(index=np.arange(len(df)), columns=np.arange(2))
>points.columns = ["Latitude", "Longitude"]
>names = list()
>for i in range(len(df)):
> points.iloc[i,1] = df.iloc[i,3]
> points.iloc[i,0] = df.iloc[i,4]
> names.append(df.iloc[i,2])
>folium.PolyLine(points, color="red", weight=2.5, opacity=1, tooltip=names).add_to(my_map)
but when I open the map, the name that appears when I hover the mouse is the whole list as in ['xxx','yyyy','zzzz']
Any help is appreciated !!
(Just cleaning up your prep:)
df = pd.read_excel (r'Bus.xlsx')
points = df.iloc[:,[3,4]]
points.columns = ["Latitude", "Longitude"]
names = df.iloc[:,2].to_list()
This is the problematic line, as I understand it.
folium.PolyLine(points, color="red", weight=2.5, opacity=1, tooltip=names).add_to(my_map)
If I understand correctly, you're passing a table of points and drawing one polyline -- and then the tooltip for this line is your list of names. I believe you instead need to generate a list/DataFrame of pairs of points and corresponding names for route between pairs of points, and then for each pair, add that as a PolyLine to your map.
is it possible to create a chart in python with multiple categories?
For example:
I copy pasted this in powerpoint, loaded it and tried this:
presentation.slides[0].shapes[0].chart.plots[0].categories.flattened_labels
which gave me all the labels in a tuple format -->
(('Oct-2019', 'Advertiser'),('Oct-2019','25th percentile), etc ...)
if I try printing :
presentation.slides[0].shapes[0].chart.plots[0].categories[i]
for i in 0 to 3
I get the values 'Advertiser','25th percentile' etc, but I can't find a way to access the 'Oct-2019' value.
When creating the ChartData, I saw that I can also add_category, but this adds the label to the categories I'm currently accessing (e.g. Advertiser, 25th percentile, etc), but I would like to add 'Nov 2019' which is in another hierarchy level.
This is a bit of a roundabout way of asking if anyone has created a multi-category chart with python pptx, and how they would do it from the chartdata level/what this would look like.
Thank you!
#data
pivot_digital_monthly_imp_laydown = pd.pivot_table(digital_monthly_imp_laydown, values='Original Value', index=['Year','Month'],
columns=['Campaign'], aggfunc=np.sum,fill_value=0,sort=False).reset_index().rename_axis(None, axis=1)
pivot_digital_monthly_imp_laydown.Year=pivot_digital_monthly_imp_laydown.Year.astype(str)
#inserting Impressions laydown in slide
prs=input_ppt
digital_monthly_imp_laydown_chart = prs.slides[3].shapes[6].chart
digital_monthly_imp_laydown_data =CategoryChartData()
#multilevel Categories
cat=list(pivot_digital_monthly_imp_laydown['Year'])
subcat=list(pivot_digital_monthly_imp_laydown['Month'])
b={}
for i,j in zip(cat,subcat):
key=str(i)
b.setdefault(key,[])
b[key].append(j)
main_cat=list(b.keys())
for i in range(len(main_cat)):
ear=digital_monthly_imp_laydown_data.add_category(str(main_cat[i]))
for sub in b[main_cat[i]]:
ear.add_sub_category(str(sub))
#add series data
for col in pivot_digital_monthly_imp_laydown.columns[2:]:
new = list(pivot_digital_monthly_imp_laydown[col])
digital_monthly_imp_laydown_data.add_series(str(col),new)
new=[]
digital_monthly_imp_laydown_chart.replace_data(digital_monthly_imp_laydown_data)
#saving
prs.save("./Output/Sub Brand Output.pptx")
I'm trying to get NDVI mean in every polygon in a feature collection with earth engine python API.
I think that I succeeded getting the result (a feature collection in a feature collection), but then I don't know how to get data from it.
The data I want is IDs from features and ndvi mean in each feature.
import datetime
import ee
ee.Initialize()
#Feature collection
fc = ee.FeatureCollection("ft:1s57dkY_Sg_E_COTe3sy1tIR_U-5Gw-BQNwHh4Xel");
fc_filtered = fc.filter(ee.Filter.equals('NUM_DECS', 1))
#Image collection
Sentinel_collection1 = (ee.ImageCollection('COPERNICUS/S2')).filterBounds(fc_filtered)
Sentinel_collection2 = Sentinel_collection1.filterDate(datetime.datetime(2017, 1, 1),datetime.datetime(2017, 8, 1))
# NDVI function to use with ee map
def NDVIcalc (image):
red = image.select('B4')
nir = image.select('B8')
ndvi = nir.subtract(red).divide(nir.add(red)).rename('NDVI')
#NDVI mean calculation with reduceRegions
MeansFeatures = ndvi.reduceRegions(reducer= ee.Reducer.mean(),collection= fc_filtered,scale= 10)
return (MeansFeatures)
#Result that I don't know to get the information: Features ID and NDVI mean
result = Sentinel_collection2.map(NDVIcalc)
If the result is small, you pull them into python using result.getInfo(). That will give you a python dictionary containing a list of FeatureCollection (which are more dictionaries). However, if the results are large or the polygons cover large regions, you'll have to Export the collection instead.
That said, there are probably some other things you'll want to do first:
1) You might want to flatten() the collection, so it's not nested collections. It'll be easier to handle that way.
2) You might want to add a date to each result so you know what time the result came from. You can do that with a map on the result, inside your NDVIcalc function
return MeansFeatures.map(lambda f : f.set('date', image.date().format())
3) If what you really want is a time-series of NDVI over time for each polygon (most common), then restructuring your code to map over polygons first will be easier:
Sentinel_collection = (ee.ImageCollection('COPERNICUS/S2')
.filterBounds(fc_filtered)
.filterDate(ee.Date('2017-01-01'),ee.Date('2017-08-01')))
def GetSeries(feature):
def NDVIcalc(img):
red = img.select('B4')
nir = img.select('B8')
ndvi = nir.subtract(red).divide(nir.add(red)).rename(['NDVI'])
return (feature
.set(ndvi.reduceRegion(ee.Reducer.mean(), feature.geometry(), 10))
.set('date', img.date().format("YYYYMMdd")))
series = Sentinel_collection.map(NDVIcalc)
// Get the time-series of values as two lists.
list = series.reduceColumns(ee.Reducer.toList(2), ['date', 'NDVI']).get('list')
return feature.set(ee.Dictionary(ee.List(list).flatten()))
result = fc_filtered.map(GetSeries)
print(result.getInfo())
4) And finally, if you're going to try to Export the result, you're likely to run into an issue where the columns of the exported table are selected from whatever columns the first feature has, so it's good to provide a "header" feature that has all columns (times), that you can merge() with the result as the first feature:
# Get all possible dates.
dates = ee.List(Sentinel_collection.map(function(img) {
return ee.Feature(null, {'date': img.date().format("YYYYMMdd") })
}).aggregate_array('date'))
# Make a default value for every date.
header = ee.Feature(null, ee.Dictionary(dates, ee.List.repeat(-1, dates.size())))
output = header.merge(result)
ee.batch.Export.table.toDrive(...)