I am making a data-plotting tool using google app engine and google docs/gdata and have run into a python/js/html communication issue.
I want a very simple layout for the webpage -- just a series of dropdown menu sections and then the final graphs. My current dilemma involves the dropdown menus.
There are total of 4 dropdown menus, and the fields of each one depends on the previous selection. I have found JS/HTML code that has all of the needed logic, but it assumes that you have pre-loaded all of the selection fields. Due to the scale of the project, it is impossible to load everything on start-up, and the data is dynamic/changing, so I can't just declare static files with the menu selection data contained.
Currently, I have methods to get all of the menu selections/data I need from google docs (using gdata). Ideally, I would like to just call each one in series (html selection1 -> python method1 to get next selection set -> html selection2 -> python method2 to get next selection set-> etc) but from what I can gather it seems this is not possible (return to python, ie 'post', without reloading the entire page).
So, is there a way to access python variables/methods in a google-app-engine JS code AFTER the initial template render or without 'posting'? Or, more open-endedly, if anyone has insight/suggestions/ideas I'd appreciate it.
Thanks
Two step process:
Create some server side code to return your data as JSON - the defacto method of transferring data on the web.
In your javascript code, call your server (from step 1) and then parse the result. Use the result to populate your dropdown.
The good news is that Google already provides json as an alternate format for their gdata APIs. See this link for a primer on how to use gdata and return JSON.
For the second part, almost all javascript client side libraries (and you really should be using one) provide an easy way to request data using ajax. As this is a common task, there are plenty of questions on StackOverflow on this - this one in particular has an example that shows you how to populate a dropdown using jquery and ajax.
Assuming you can use javascript and replicate whatever functionality your Python code is providing, the above will work for you. If you have some specific logic that you don't want to transfer to the client side, then your Python code will have to return json (in effect, you would be acting like the gdata api for your javascript code).
Since you didn't highlight which Python framework you are using - here is an example using flask, a simple framework for Python web development:
from flask import Flask, jsonify
app = Flask(__name__)
#app.route('/the-question')
def the_answer():
return jsonify(answer=42)
A more comprehensive example is available in the documentation under AJAX with jQuery.
If you are using webapp2, Google provides excellent documentation to do the same thing.
Related
I have a machine learning model which is trained to perform classification. I want to provide the test data through a front end(I have web based front end in my mind if there is any other option then please recommend.). So i would require an html page to take the input , pass it to the backend so that calculations can be performed in python and return it back to the html page. How can i do this?
[EDIT] I started with django. I am able to render the html page but how do i pass the data to a python file?
There are a lot of ways you can implement an front end for python but probably the two easiest are:
Using Flask which is an easy to learn web microframework for python.
Using the Django Web Framework which is harder to learn than Flask but provides more functionality.
I have an HTML webpage. It has a search textbox. I want to allow the user to search within a dataset. The dataset is represented by a bunch of files on my server. I wrote a python script which can make that search.
Unfortunately, I'm not familiar with how can I unite the HTML page and a Python script.
The task is to put a python script into the html file so, that:
Python code will be run on the server side
Python code can somehow take the values from the HTML page as input
Python code can somehow put the search results to the HTML webpage as output
Question 1 : How can I do this?
Question 2 : How the python code should be stored on the website?
Question 3 : How it should take HTML values as input?
Question 4 : How can it output the results to the webpage? Do I need to install/use any additional frameworks?
Thanks!
There are too many things to get wrong if you try to implement that by yourself with only what the standard library provides.
I would recommend using a web framework, like flask or django. I linked to the quickstart sections of the comprehensive documentation of both. Basically, you write code and URL specifications that are mapped to the code, e.g. an HTTP GET on /search is mapped to a method returning the HTML page.
You can then use a form submit button to GET /search?query=<param> with the being the user's input. Based on that input you search the dataset and return a new HTML page with results.
Both frameworks have template languages that help you put the search results into HTML.
For testing purposes, web frameworks usually come with a simple webserver you can use. For production purposes, there are better solutions like uwsgi and gunicorn
Also, you should consider putting the data into a database, parsing files for each query can be quite inefficient.
I'm sure you will have more questions on the way, but that's what stackoverflow is for, and if you can ask more specific questions, it is easier to provide more focused answers.
I would look at the cgi library in python.
You should check out Django, its a very flexible and easy Python web-framework.
I'm looking to add search capability into an existing entirely static website. Likely, the new search functionality itself would need to be dynamic, as the search index would need to be updated periodically (as people make changes to the static content), and the search results will need to be dynamically produced when a user interacts with it. I'd hope to add this functionality using Python, as that's my preferred language, though am open to ideas.
The Google Web Search API won't work in this case because the content being indexed is on a private network. Django haystack won't work for this case, as that requires that the content be stored in Django models. A tool called mnoGoSearch might be an option, as I think it can spider a website like Google does, but I'm not sure how active that project is anymore; the project site seems a bit dated.
I'm curious about using tools like Solr, ElasticSearch, or Whoosh, though I believe that those tools are only the indexing engine and don't handle the parsing of search content. Does anyone have any recommendations as to how one may index static html content for retrieving as a set of search results? Thanks for reading and for any feedback you have.
With Solr, you would write code that retrieves content to be indexed, parses out the target portions from the each item then sends it to Solr for indexing.
You would then interact with Solr for search, and have it return either the entire indexed document an ID or some other identifying information about the original indexed content, using that to display results to the user.
So this is somewhat similar to
What's easiest way to get Python script output on the web?
and
Matplotlib: interactive plot on a web server
but none of those used d3.js and I don't think they achieve the same level of interactive-ness.
So I am a newcomer to d3.js and frankly have no clue where I should start in this instance.
Project Flow:
Ask user name on the front end. Send this to backend python
get graph data (object + edges) for a depth of x where x is distance away from a starting node by Python calls to 3rd party website API
run some machine learning on the data (python)
display the graph + some numbers (maybe in the bottom right corner) in d3.js
loop:
Have an interactive d3.js graph so that if I click a node, it will redo the calculation with that as the starting node
have it so that d3.js shows part of the text (first 10 chars?) of each node and then shows the full text once you mouse over.
(7) Bonus updating embeddable picture? Like those profile signatures?
Questions:
How would I achieve this bidirectional communication?
Where should I start?
Is there a better way I can do this?
Sidenote: I'm planning to do this project using Google App Engine, I don't know if that lets you use it as a 'vm' though.
Thanks!
Edit: This looks a lot like Python at backend JS at frontend integration, but I don't know where to even start and I think they suggest more art-oriented tools as a result of his project.
It sounds like you need to write a web service in Python that consumes and returns JSON. I'm not sure if the data is originating on the client or the server, so for this example, I'm assuming it comes from the client.
POST your graph data in JSON format to your webservice
Do processing server-side
return JSON representing the results of the processing to the client
Render it with d3.js
User interaction on the client creates a new or modified data structure
Go back to step 1
Maybe your server-side web service method looks like this (this is pseudo-code based on web.py):
import simplejson
class ml(object):
def POST(self,data=None):
# turn the JSON string into a Python data structure
d = simplejson.loads(data)
# do stuff
results = alter_data(d)
web.header('Content-Type','application/json')
# send JSON back to the client
return simplejson.dumps(results)
And your client-side code might look something like this (again, just a rough sketch). This example assumes that you're using jQuery to do your AJAX stuff.
function render(data,textStatus) {
d3
.select('svg')
.selectAll('circle')
.data(data)
.enter()
.append('circle')
.on('click',function() {
$.getJSON(
'your_webservice_url',
// POST an updated data structure to your service
{ 'data' : alter_data(data) },
// Call the render method again when new data is received
render);
});
}
d3.js' enter and exit methods make updating the visual representation of your data very easy. I recommend reading the documentation for those.
This example might give you some ideas about how to represent your graph data with JSON, and how to render it.
Check out the on method for an idea about how to trigger a POST to your web service when a node is clicked.
I'm going to skip the really specific stuff, like displaying text with d3.js. I'm sure you can figure out how to do that by perusing the d3.js documentation.
This is a late answer and I've just stumbled across this python package: Bokeh
To cover points 1-6 of your requirements it seems you pretty much only have to write python code which will be a plus for someone not used to writing java script.
But then this package is still quite new and might have some limitations I'm not yet aware of at the moment.
I am learning python. I have created some scripts that I use to parse various websites that I run daily (as their stats are updated), and look at the output in the Python interpreter. I would like to create a website to display the results. What I want to do is run my script when I go to the site, and display a sortable table of the results.
I have looked at Django and am part way through the tutorial, but it seems like an awful lot of overhead for what should be a simple problem. I know that I could just write a Python script to output simple HTML, but is that really the best way? I would like to be able to sort the table by various columns.
I have years of programming experience (C, Java, etc.), but have very little web development experience.
Thanks in advance.
Have you considered Flask? Like Tornado, it is both a "micro-framework" and a simple web server, so it has everything you need right out of the box. http://flask.pocoo.org/
This example (right off the homepage) pretty much sums up how simple the code can be:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run()
If you are creating non-interactive pages, you can easily setup any modern web server to execute your python script as a CGI. Instead of loading a static file, your web server will return the output of your python script.
This isn't very sophisticated, but if you are simply returning the output without needing browser submitted date, this is the easiest way (scaling under load is a different story).
You don't even need the "cgi" module from python, if you aren't receiving any data from the browser. Anything more complicated than this and you should use a web framework.
Examples and other methods
Simple Example: hardest part is webserver configuration
mod_python: Cut down on CGI overhead (otherwise, apache execs the python interpreter for each hit)
python module cgi: sending data to your python script from the browser.
Sorting
Javascript side sorting: I've used this javascript library to add sortable tables. This is the easiest way to add sorting without requiring additional work or another HTTP GET.
Instructions:
Download this file
Add to your HTML
Add class="sortable" to any table you'd like to make sortable
Click on the headers to sort
You might consider Tornado if Django is too much overhead. I've used both and agree that, if you have something simple/small to do and don't already know Django, it's going to exponentially increase your time to production. On the other hand, you can 'get' Tornado in a couple of hours and get something relatively simple done in a day or two with no prior experience with it. At least, that's been my experience with it.
Note that Tornado is still a tradeoff: you get a lot of simplicity in exchange for the huge cornucopia of features and shortcuts you get w/ Django.
PS - in addition to being a 'micro-framework', Tornado is also its own web server, so there's no mucking with wsgi/mod-cgi/fcgi.... just write your request handlers and run it. Be sure to see the demos included in the distribution.
Have you seen bottle framework? It is a micro framework and very simple.
If I correctly understood your requirements you might find Wooey very interesting.
Wooey is a A Django app that creates automatic web UIs for Python scripts:
http://wooey.readthedocs.org
Here you can check a demo:
https://wooey.herokuapp.com/
Django is a big webframework, meant to include loads of things becaus eyou often needs them, even though sometimes you don't.
Look at Pyramid, earlier known as BFG. It's much smaller.
http://pypi.python.org/pypi/pyramid/1.0a1
Other microframeworks to check out are here: http://wiki.python.org/moin/WebFrameworks
On the other hand, in this case it's probably also overkill. sounds like you can run the script once every ten minites, and write a static HTML file, and just use Apache.
If you are not willing to write your own tool, there is a pretty advanced tool for executing your scripts: http://rundeck.org/
It's pretty simple to start and can be configured for complex scenarios as well.
For the requirement of custom view (with sortable results), I believe you can implement a simple plugin for translating script output into html elements.
Also, for simple setups I could recommend my own tool: https://github.com/bugy/script-server. It doesn't have tons of features, but very easy for end-users and supports interactive execution.
If you don't need any input from the browser, this sounds like an almost-static webpage that just happens to change once a day. You'll only need some way to get html out of your script, in a place where your webserver can access it.)
So you'd use some form of templating; if you'll need some structure above the single page, there's static site / blog generators that you can feed your output in, say, Markdown format, and call their make html or the like.
You can use DicksonUI https://dicksonui.gitbook.io
DicksonUI is better
Or Remi gui(search in google)
DicksonUI is better.
I am the author of DicksonUI