The code works if result=random.random(), but would like to dynamically Read Excel Spreadsheets with Python, Flask without reload the html page
Excel file will change at a specified interval and HTML keeping static
.py
#app.route("/rel")
def rel_country():
z = "test"
book = load_workbook("rel/" + str(z) + ".xlsx")
sheet = book.active
return jsonify(result=sheet)
HTML:
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.4/jquery.min.js"></script>
<script type="text/javascript">
var $SCRIPT_ROOT = {{ request.script_root|tojson|safe }};
var intervalID = setInterval(update_values, 1000);
function update_values() {
$.getJSON($SCRIPT_ROOT + '/rel',
function (data) {
$('#result').text(data.result);
console.log(data)
});
};
function stopTextColor() {
clearInterval(intervalID);
}
</script>
<title>Excel To HTML Table</title>
</head>
<body onload="update_values();">
<h1>Dynamic Update</h1>
<p>
<span id="result">?</span>
<script>
document.getElementById("result").innerHTML;
</script>
<button onclick="stopTextColor();">Stop</button>
</body>
Error:
raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")
TypeError: Object of type Worksheet is not JSON serializable
Use pandas to read the xlsx file. The dataframe that results from that is json serializable.
df = pd.read_excel('tmp.xlsx', index_col=0)
result = df.to_json(orient="records")
Related
Is there ever a way possible to get the actual creation / modification time of the file being uploaded, using JavaScript?
As for PHP, using filectime() and filemtime(), it only shows the date / time the file is uploaded, and not the time the file is actually created / modified on the source.
In short, what I want is to check the m-time of a file before/during/after upload (where-ever possible) and decide whether or not to store the file on the server, and report the same back to the client.
If you're talking about the file date/time on the user's machine, you can get that via the File API (support), which provides lastModified, which is the date/time as a number of milliseconds since The Epoch (if you want a Date, you can pass that into new Date). (There's also the deprecated lastModifiedDate, but that is deprecated and not supported on Safari [at least].) The File API is universally supported in modern browsers (the particular feature you'd be using is the File object). You'd get the value from the File object and include that information in a separate (for instance, hidden) field.
Here's a rough-but-complete example of reading the last modified date (live copy):
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8">
<title>Show File Modified</title>
<style type='text/css'>
body {
font-family: sans-serif;
}
</style>
<script type='text/javascript'>
function showFileModified() {
var input, file;
// Testing for 'function' is more specific and correct, but doesn't work with Safari 6.x
if (typeof window.FileReader !== 'function' &&
typeof window.FileReader !== 'object') {
write("The file API isn't supported on this browser yet.");
return;
}
input = document.getElementById('filename');
if (!input) {
write("Um, couldn't find the filename element.");
}
else if (!input.files) {
write("This browser doesn't seem to support the `files` property of file inputs.");
}
else if (!input.files[0]) {
write("Please select a file before clicking 'Show Modified'");
}
else {
file = input.files[0];
write("The last modified date of file '" + file.name + "' is " + new Date(file.lastModified));
}
function write(msg) {
var p = document.createElement('p');
p.innerHTML = msg;
document.body.appendChild(p);
}
}
</script>
</head>
<body>
<form action='#' onsubmit="return false;">
<input type='file' id='filename'>
<input type='button' id='btnShowModified' value='Show Modified' onclick='showFileModified();'>
</form>
</body>
</html>
The reason you couldn't get the time from the uploaded file on the server is that only the content of the file is transmitted in the request, not the client's filesystem metadata.
JavaScript does not have access to the local filesystem, so you can't get to this information without using Flash, Java or Active-x.
Perhaps you could use javascript to get the last modified time, then use that in some other javacript to sort on that. This time will be in GMT.
var xmlhttp = createXMLHTTPObject();
xmlhttp.open("HEAD", "http://myurl/interesting_image.jpg" ,true);
xmlhttp.onreadystatechange=function() {
if (xmlhttp.readyState==4) {
alert("Last modified: "+
var lastModTimeForInterestingImage = xmlhttp.getResponseHeader("Last-Modified"))
}
}
xmlhttp.send(null);
I would like to parse an XML file from a URL.
By doing the following:
req = requests.get('https://www.forbes.com/news_sitemap.xml')
Instead of getting the proper XML file, I get:
<!doctype html>
<html lang="en">
<head>
<meta http-equiv="Content-Language" content="en_US">
<script type="text/javascript">
(function () {
function isValidUrl(toURL) {
// Regex taken from welcome ad.
return (toURL || '').match(/^(?:https?:?\/\/)?(?:[^.(){}\\\/]*)?\.?forbes\.com(?:\/|\?|$)/i);
}
function getUrlParameter(name) {
name = name.replace(/[\[]/, '\\[').replace(/[\]]/, '\\]');
var regex = new RegExp('[\\?&]' + name + '=([^&#]*)');
var results = regex.exec(location.search);
return results === null ? '' : decodeURIComponent(results[1].replace(/\+/g, ' '));
};
function consentIsSet(message) {
console.log(message);
var result = JSON.parse(message.data);
if(result.message == "submit_preferences"){
var toURL = getUrlParameter("toURL");
if(!isValidUrl(toURL)){
toURL = "https://www.forbes.com/";
}
location.href=toURL;
}
}
var apiObject = {
PrivacyManagerAPI:
{
action: "getConsent",
timestamp: new Date().getTime(),
self: "forbes.com"
}
};
var json = JSON.stringify(apiObject);
window.top.postMessage(json,"*");
window.addEventListener("message", consentIsSet, false);
})();
</script>
</head>
<div id='teconsent'>
<script async="async" type="text/javascript" crossorigin src='//consent.truste.com/notice?domain=forbes.com&c=teconsent'></script>
</div>
<body>
</body>
</html>
Is there also a better way to handle the XML file (for example, if it is compressed, or by parsing it recursively if the file is too big...)? Thanks!
This site checks a cookie for GDPR if you give that cookie to request you can get XML file.
Try this code, works fine to me.
import requests
url = "https://www.forbes.com/news_sitemap.xml"
news_sitemap = requests.get(url, headers={"Cookie": "notice_gdpr_prefs=0,1,2:1a8b5228dd7ff0717196863a5d28ce6c"})
print(news_sitemap.text)
Using requests module I get the xml file. You can then use an xml parser library to do what you want.
import requests
url = "https://www.forbes.com/news_sitemap.xml"
x = requests.get(url)
print(x.text)
I have recently come across the Pyodide project.
I have built a little demo using Pyodide, but although I've spent a lot of time looking at the source, it is not obvious (yet) to me how to redirect print output from python (other than modifying the CPython source), and also, how to redirect output from matplotlib.pyplot to the browser.
From the source code, FigureCanvasWasm does have a show() method with the appropriate backend for plotting to the browser canvas - however, it is not clear to me how to instantiate this class and invoke it's show() method or indeed, if there is another more obvious way of redirecting plots to canvas.
My questions therefore are:
How do I redirect print() messages
How do I force pyodide to plot matplotlib figures in the browser?
Here is my test page:
<!doctype html>
<meta charset="utf-8">
<html lang="en">
<html>
<head>
<title>Demo</title>
<script src="../../pyodide/build/pyodide.js"></script>
</head>
<body>
</body>
<script type="text/javascript">
languagePluginLoader.then(() => {
pyodide.loadPackage(['matplotlib']).then(() => {
pyodide.runPython(`
import matplotlib.pyplot as plt
plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
#fig = plt.gcf()
#fig.savefig(imgdata, format='png')
print('Done from python!')`
);
//var image = pyodide.pyimport('imgdata');
//console.log(image);
});});
</script>
<html>
First of all let's see if we can get just anything to show up in the browser; e.g. a normal string. Python variables are stored in the pyodide.globals attribute. Hence we can take the python object from there and place it into a <div> element on the page.
<!doctype html>
<meta charset="utf-8">
<html>
<head>
<title>Demo</title>
<script src="../pyodide/pyodide.js"></script>
</head>
<body>
</body>
<script type="text/javascript">
languagePluginLoader.then(() => {
pyodide.runPython(`my_string = "This is a python string." `);
document.getElementById("textfield").innerText = pyodide.globals.my_string;
});
</script>
<div id="textfield"></div>
<html>
Now I guess we can do the same with a matplotlib figure. The following would show a saved png image in the document.
<!doctype html>
<meta charset="utf-8">
<html lang="en">
<html>
<head>
<title>Demo</title>
<script src="../pyodide/pyodide.js"></script>
</head>
<body>
</body>
<script type="text/javascript">
languagePluginLoader.then(() => {
pyodide.loadPackage(['matplotlib']).then(() => {
pyodide.runPython(`
import matplotlib.pyplot as plt
import io, base64
fig, ax = plt.subplots()
ax.plot([1,3,2])
buf = io.BytesIO()
fig.savefig(buf, format='png')
buf.seek(0)
img_str = 'data:image/png;base64,' + base64.b64encode(buf.read()).decode('UTF-8')`
);
document.getElementById("pyplotfigure").src=pyodide.globals.img_str
});});
</script>
<div id="textfield">A matplotlib figure:</div>
<div id="pyplotdiv"><img id="pyplotfigure"/></div>
<html>
I haven't looked into the backends.wasm_backend yet, so that may allow for a more automated way of the above.
When using the wasm backend, the canvas property of a figure is an instance of FigureCanvasWasm. Calling the show() method of the canvas should be sufficient to display the figure in the browser. Unfortunately a minor bug in the create_root_element() method of the canvas prevents the figure from being displayed. This method creates a div element that will contain the figure. It tries first to create an iodide output div element. If that fails a plain HTML div element is created. This element however is never appended to the document and remains therefore invisible.
Below are the lines of code from FigureCanvasWasm were it happens
def create_root_element(self):
# Designed to be overridden by subclasses for use in contexts other
# than iodide.
try:
from js import iodide
return iodide.output.element('div')
except ImportError:
return document.createElement('div')
The comment suggests the non-iodide code is a stub that needs to be extended, by overriding the method. This would require subclassing FigureCanvasWasm, installing it as a pyodide module and configuring matplotlib to use that backend.
There is a shortcut however, because python allows overriding a method of an instance, without modifying the class, as per question 394770. Putting the following code in your HTML document gives a figure in the browser
import numpy as np
from matplotlib import pyplot as plt
from js import document
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)
f = plt.figure()
plt.plot(x,y)
# ordinary function to create a div
def create_root_element1(self):
div = document.createElement('div')
document.body.appendChild(div)
return div
#ordinary function to find an existing div
#you'll need to put a div with appropriate id somewhere in the document
def create_root_element2(self):
return document.getElementById('figure1')
#override create_root_element method of canvas by one of the functions above
f.canvas.create_root_element = create_root_element1.__get__(
create_root_element1, f.canvas.__class__)
f.canvas.show()
Initially the toolbar did not show icons. I had to download, unzip and install fontawesome alongside pyodide and include the following line in the header to get those
<link rel="stylesheet" href="font-awesome-4.7.0/css/font-awesome.min.css">
Edit:
About the first part of your question, redirecting the output stream to the browser, you could take a look at how it is done in pyodide's console.html.
It replaces sys.stdout by a StringIO object
pyodide.runPython(`
import sys
import io
sys.stdout = io.StringIO()
`);
Then run the python code (that can be completely oblivious to the fact that it is running in a wasm context)
pyodide.runPython(`
print("Hello, world!")
`);
Finally, send the contents of the stdout buffer to an output element
var stdout = pyodide.runPython("sys.stdout.getvalue()")
var div = document.createElement('div');
div.innerText = stdout;
document.body.appendChild(div);
To show print() calls form pyodide you can use the parameters on loadPyodide to redirect stdout:
var paragraph = document.getElementById("p");
pyodide = await loadPyodide({
indexURL : "https://cdn.jsdelivr.net/pyodide/v0.18.1/full/",
stdin: window.prompt,
stdout: (text) => {paragraph.textContent += text;},
stderr: (text) => {paragraph.textContent += text;}
});
https://github.com/pyodide/pyodide/blob/main/src/js/pyodide.js
I created a simple interactive shell for Python. Read my tutorial if you need more detailed information.
const output = document.getElementById("output")
const code = document.getElementById("code")
code.addEventListener("keydown", function (event) {
if (event.ctrlKey && event.key === "Enter") {
evaluatePython()
}
})
function addToOutput(s) {
output.value += `>>>${code.value}\n${s}\n`
output.scrollTop = output.scrollHeight
code.value=''
}
output.value = 'Initializing...\n'
// init pyodide
languagePluginLoader.then(() => { output.value += 'Ready!\n' })
function evaluatePython() {
pyodide.runPythonAsync(code.value)
.then(output => addToOutput(output))
.catch((err) => { addToOutput(err) })
}
<!DOCTYPE html>
<head>
<script type="text/javascript">
// this variable should be changed if you load pyodide from different source
window.languagePluginUrl = 'https://pyodide-cdn2.iodide.io/v0.15.0/full/';
</script>
<script src="https://pyodide-cdn2.iodide.io/v0.15.0/full/pyodide.js"></script>
</head>
<body>
Output:
</div>
<textarea id='output' style='width: 100%;' rows='10' disabled></textarea>
<textarea id='code' value='' rows='2'></textarea>
<button id='run' onclick='evaluatePython()'>Run</button>
<p>You can execute any Python code. Just enter something in the box above and click the button (or Ctrl+Enter).</p>
<div><a href='https://github.com/karray/truepyxel/demo.html'>Source code</a></div>
</body>
</html>
Here is the example for matplotlib. Note that this will load a bunch of dependencies which will take up to several minutes.
let python_code = `
from js import document
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
import io, base64
def generate_plot_img():
# get values from inputs
mu = int(document.getElementById('mu').value)
sigma = int(document.getElementById('sigma').value)
# generate an interval
x = np.linspace(mu - 3*sigma, mu + 3*sigma, 100)
# calculate PDF for each value in the x given mu and sigma and plot a line
plt.plot(x, stats.norm.pdf(x, mu, sigma))
# create buffer for an image
buf = io.BytesIO()
# copy the plot into the buffer
plt.savefig(buf, format='png')
buf.seek(0)
# encode the image as Base64 string
img_str = 'data:image/png;base64,' + base64.b64encode(buf.read()).decode('UTF-8')
# show the image
img_tag = document.getElementById('fig')
img_tag.src = img_str
buf.close()
`
languagePluginLoader.then(()=>pyodide.runPythonAsync(python_code).then(()=>document.getElementById('status').innerHTML='Done!'))
<!DOCTYPE html>
<head>
<script type="text/javascript">
// this variable should be changed if you load pyodide from different source
window.languagePluginUrl = 'https://pyodide-cdn2.iodide.io/v0.15.0/full/';
</script>
<script src="https://pyodide-cdn2.iodide.io/v0.15.0/full/pyodide.js"></script>
</head>
<body>
Status: <strong id='status'>Initializing...</strong>
<br><br>
mu:
<input id='mu' value='1' type="number">
<br><br>
sigma:
<input id='sigma' value='1' type="number">
<br><br>
<button onclick='pyodide.globals.generate_plot_img()'>Plot</button>
<br>
<img id="fig" />
</body>
</html>
I am using AJAX to get the contents of a file when I press a button(I am very new to AJAX.) , Here's the HTML:
<!DOCTYPE html>
<html>
<head>
<script>
function loadXMLDoc() {
var xmlhttp;
if (window.XMLHttpRequest) { // code for IE7+, Firefox, Chrome, Opera, Safari
xmlhttp = new XMLHttpRequest();
} else { // code for IE6, IE5
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
document.getElementById("myDiv").innerHTML = xmlhttp.responseText;
}
}
xmlhttp.open("GET", "data.dat", true);
xmlhttp.send();
}
</script>
</head>
<body>
<div id="myDiv">
<p>- - -</p>
</div>
<button type="button" onclick="loadXMLDoc()">Change Content</button>
</body>
</html>
And here is the python to change the file(This is not the python code i am using, but it still does the same thing, almost):
from time import *
a = 0
while True:
print(a)
file = open("data.dat","w")
file.write("<p>"+str(a)+"</p>")
file.close()
sleep(1)
a+=1
I would like to get the file contents every second, How would I Do that? Any help is good.
You could use setInterval() to periodically run the function which updates your document.
var intervalID = setInterval(loadXMLDoc, 1000); // Every 1s
I am trying to put a simple python script (here I've used a random two-word generator which works fine) in a div on a webpage with Ajax and have a button beneath that reloads it. The page successfully loads the script... however, I haven't quite got my head around the missing part to call the script again to reload the random two words made by the python script (I understand why the code below is wrong but I can't work out how to make it right!). Pointer much appreciated!
(N.B. Yes I am using Python 2.4 because my web host hasn't upgraded yet - they will soon! And yes I saw this question but it didn't work for me...)
HTML:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html;charset=UTF-8">
<title>Find a sentence</title>
<script src="http://code.jquery.com/jquery-2.0.3.js"></script>
<script>
$(function()
{
$.ajax({
url: "cgi-bin/test.py",
type: "GET",
data: {foo: 'bar', bar: 'foo'},
success: function(response){
$("#div").html(response);
}
});
});
</script>
</head>
<body>
<div id="div"></div>
<form name="input" action="cgi-bin/test.py" method="get">
<input type="submit" value="Regenerate!">
</form>
</body>
PYTHON:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# enable debugging
import cgitb
cgitb.enable()
import linecache
from random import randrange
print "Content-Type: text/html"
print
# file length counter
def file_len(fname):
f = open(fname)
try:
for i, l in enumerate(f):
pass
finally:
f.close()
return i + 1
# range sizes
one_size = file_len('text/one.csv')
two_size = file_len('text/two.csv')
# random lines
one_line = randrange(one_size) + 1
two_line = randrange(two_size) + 1
# outputs
one_out = linecache.getline('text/one.csv', one_line)
two_out = linecache.getline('text/two.csv', two_line)
# output
sentence = one_out.strip('\n') + " " + two_out.strip('\n') + " "
print sentence
Well, I would assume that one has to click on the "Regenerate" button for the form to reload. My idea would be to have the ajax logic in a separate function. I don't think you need the form there since your reload call is now ajax call.
<head>
<script>
function reload() {
$.ajax({
url: "cgi-bin/test.py",
type: "GET",
data: {foo: 'bar', bar: 'foo'},
dataType : "html",
success: function(response){
$("#div").html(response);
}
});
}
$(window).load(function() {
reload();
$("#reload_button").click(function() {
reload();
});
});
</script>
</head>
<body>
<div id="div"></div>
<input type="button" id="reload_button" value="Regenerate!">
</body>