Develop Reference

Datashader on Google Colab - blank output

I'm trying out Datashader on Google Colab to visualise a large dataset of longitudes and latitudes colored logarithmically with the colorcet.fire colormaps, but my code throws a completely blank output.
Code in text:
import datashader as ds
import pandas as pd
import colorcet
data = pd.read_csv('hab.csv', usecols=['longitude','latitude'])
cvs = ds.Canvas()
agg = cvs.points(data, 'latitude', 'longitude')
ds.tf.set_background(ds.tf.shade(agg, cmap=colorcet.fire, how='log'))
What I see on Colab:

I'm not a collab user, but yes, when I run your code locally with the five datapoints shown I get a blank plot. In my local version, it's because the code is specifying a colormap whose highest value is white, and for a few scattered points each of them are at the highest value. The code uses set_background, perhaps trying to set the background to black as would be suitable for that colormap, but it doesn't specify "black" and so the set_background call does nothing. If I specify the background color and add Datashader spreading so that these single datapoints are easier to see, I do get a plot from your code:
cvs = ds.Canvas()
agg = cvs.points(data, 'latitude', 'longitude')
ds.tf.set_background(ds.tf.shade(ds.tf.spread(agg, px=10), cmap=colorcet.fire, how='log'), "black")
You may have some other problem as well, though, since the plot you showed wasn't just white, it appeared to be transparent. And if your dataset is indeed large, you should see output anyway, because data points would then overlap and use all the colors in the colormap.

Is there a way to plot many markers in Folium?

I am trying to use Folium for geographical information reading from a pandas dataframe.
The code I have is this one:
import folium
from folium import plugins
import pandas as pd
...operations on dataframe df...
map_1 = folium.Map(location=[51.5073219, -0.1276474],
zoom_start=12,
tiles='Stamen Terrain')
markerCluster = folium.plugins.MarkerCluster().add_to(map_1)
lat=0.
lon=0.
for index,row in df.iterrows():
lat=row['lat]
lon=row['lon']
folium.Marker([lat, lon], popup=row['name']).add_to(markerCluster)
map_1
df is a dataframe with longitude, latitude and name information. Longitude and latitude are float.
I am using jupyter notebook and the map does not appear. Just a white empty box.
I have also tried to save the map:
map_1.save(outfile='map_1.html')
but also opening the file doesn't work (using Chrome, Firefox or Explorer).
I have tried to reduce the number of markers displayed and below 300 markers the code works and the map is correctly displayed. If there are more than 300 Markers the map returns to be be blank.
The size of the file is below 5 MB and should be processed correctly by Chrome.
Is there a way around it (I have more than 2000 entries in the dataframe and I would like to plot them all)? Or to set the maximum number of markers shown in folium?
Thanks

This might be too late but I stumbled upon the same problem and found a solution that worked for me without having to remove the popups so I figured if anybody has the same issue they can try it out. Try replacing popup=row['name'] with popup=folium.PopUp(row['name'], parse_html=True) and see whether it works. You can read more about it here https://github.com/python-visualization/folium/issues/726

Python PPTX workaround function for rotating chart data labels

I intend to create the following chart using Python PPTX.
Below code achieve the color setting, font size and number format. However, I am not yet able to rotate the data label, as I believe this API is not yet available in python-pptx 0.6.5
lbl = plot.data_labels
lbl.font.size = config["DATA_LABEL_FONT_SIZE"]
lbl.font.color.rgb = config["DATA_LABEL_FONT_COLOR"]
lbl.number_format = config["DATA_LABEL_NUMBER_FORMAT"]
lbl.position = config["DATA_LABEL_POSITION"]
To get started, I have created two minimal slides before and after rotating, and use opc-diag tool to find the diff.
<a:bodyPr rot="-5400000" spcFirstLastPara="1" vertOverflow="ellipsis"
vert="horz" wrap="square" lIns="38100" tIns="19050" rIns="38100"
bIns="19050" anchor="ctr" anchorCtr="1">\n
<a:spAutoFit/>\n </a:bodyPr>\n
I believe I need to add rot="-5400000" XML element to lbl (plot.data_labels), but not clear on how to achieve this. I have used dir(), ._element and .xml on the chart and its children but not able to find <a:bodyPr> tag.

I tried below and it works.
if config["DATA_LABEL_VERTICAL"]:
txPr = lbl._element.get_or_add_txPr()
txPr.bodyPr.set('rot','-5400000')

Big data multiple plots, multiple web pages Plotly

I am trying to work with big data plotting around 20 plots on plotly and embed them on web page. I can very well plot individual plots with username and one api_key that found in the profile.
The Problem comes is when: I have to rerun all the 20 plots with python program after interval of every 15 mins and every time I am getting new windows. Instead I need the same plot to update/redraw.
How do I get that? I tried reading the plot.ly document and also few tutorials outside. Cannot find how to get it done. Can anyone please help me with steps or refer me to some document where I can know how to work with multiple plots that will update at same time.
I am following the steps given in plotly tutorial not sure if I should use stream_ids ? Or can I create a new api_key for every plot ?Confused !!! Thanks in Advance for the suggestions.
Edit: I could make access tokens and Initiate the credentials from the following tutorial.
The code below works perfect: But now I am looking for required fixing in the below code by trying to minimize the code with annotations and where to include the streaming API Access Tokens while having sizable scatter plots ?
import plotly.plotly as py
import plotly.tools as tls
from plotly.graph_objs import *
import csv
import pandas as pd
import numpy as np
df = pd.read_csv('finally.csv')
df1=df[['NAME','COUNT']]
sizemode='area'
sizeref=df1['COUNT'].max()/1000
def Trace(X,PLACE,sizes):
return Scatter(
x=X['NAME'],
y=X['COUNT'].sum(),
name=PLACE,
mode='marker',
marker=Marker(
line=Line(width=0.9),
size=sizes,
sizeref=sizeref,
opacity=0.9,
)
)
data=Data()
for PLACE, X in df1.groupby('NAME'):
sizes=X['COUNT'].sum()/1000
data.append(Trace(X,PLACE,sizes))
title = "Fig 1.1 : All NAMES"
x_title = "Names".format()
y_title = "Count"
# Define a dictionary of axis style options
axis_style = dict(
zeroline=False, # remove thick zero line
gridcolor='#FFFFFF', # white grid lines
ticks='outside', # draw ticks outside axes
ticklen=8, # tick length
tickwidth=1.5 # and width
)
# Make layout object
layout = Layout(
title=title, # set plot title
plot_bgcolor='#EFECEA', # set plot color to grey
xaxis=XAxis(
axis_style, # add axis style dictionary
title=x_title, # x-axis title
),
yaxis=YAxis(
axis_style, # add axis style dictionary
title=y_title, # y-axis title
),
showlegend=False,
)
fig = Figure(data=data,layout=layout)
plot_url=py.plot(fig,filename=' plotting')

In plot/ iplot there is 'fileopt' option which should help you. For example, if you would want to add new traces to your existing data you can run
plot_url = py.plot(fig, filename='my-file', fileopt='append')
You're right it is not well documented yet. But if you run help(py.plot) you would get a small document on it as follow:
plot(figure_or_data, validate=True, **plot_options)
Create a unique url for this plot in Plotly and optionally open url.
plot_options keyword agruments:
filename (string) -- the name that will be associated with this figure
fileopt ('new' | 'overwrite' | 'extend' | 'append') -- 'new' creates a
'new': create a new, unique url for this plot
'overwrite': overwrite the file associated with `filename` with this
'extend': add additional numbers (data) to existing traces
'append': add additional traces to existing data lists
world_readable (default=True) -- make this figure private/public
auto_open (default=True) -- Toggle browser options
True: open this plot in a new browser tab
False: do not open plot in the browser, but do return the unique url

How to create a PDF document with differing page sizes in reportlab, python

Is it possible to create a PDF document with differing page sizes in reportlab?
I would like to create a document where the first page has a different size then the other pages. Can anyone help?

Yes, this should be possible, since PDF supports this, it's just a question of how to make it happen in ReportLab. I've never done this, but the following should work:
c = reportlab.pdfgen.canvas.Canvas("test.pdf")
# draw some stuff on c
c.showPage()
c.setPageSize((700, 500)) #some page size, given as a tuple in points
# draw some more stuff on c
c.showPage()
c.save()
And your document should now have two pages, one with a default size page and one with a page of size 700 pt by 500 pt.
If you are using PLATYPUS you should be able to achieve the same sort of thing, but will probably require getting fancy in a BaseDocTemplate subclass to handle changing page sizes, since I'm pretty sure the PageTemplate machinery doesn't already support this since each PageTemplate is mainly a way of changing the way frames are laid out on each page. But it is technically possible, it just isn't documented and you'll probably have to spend some time reading and understanding how PLATYPUS works internally.

my use case was to create a big table inside pdf. as table was big it was getting cropped on both sides. this is how to create a pdf with custom size.i am using platypus from reportlab.
from reportlab.platypus import SimpleDocTemplate
from reportlab.lib.units import mm, inch
pagesize = (20 * inch, 10 * inch) # 20 inch width and 10 inch height.
doc = SimpleDocTemplate('sample.pdf', pagesize=pagesize)
data = [['sarath', 'indiana', 'usa'],
['jose', 'indiana', 'shhs']]
table = Table(data)
elems = []
elems.append(table)
doc.build(elems)
one downside of this technique is the size is same for all pages.But will help people looking to create pdf with custom size (same for all pages)