I am trying to create a Python web server that takes user input and makes a request to a third-party API. The user input is obtained via a simple form I created.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Test Document</title>
</head>
<body>
<form action="server.py" method="post">
<label for="firstname">First Name</label>
<input type="text" name="firstname" id="firstname"><br>
<label for="lastname">Last Name</label>
<input type="text" name="lastname" id="lastname"><br>
<label for="url">URL</label>
<input type="text" name="url" id="application-url"><br>
<input type="submit" value="Submit">
</form>
</body>
</html>
I named this file index.html. My server.py file looks like this:
import cgi
from http.server import HTTPServer, CGIHTTPRequestHandler
class TestServerHandler(CGIHTTPRequestHandler):
def do_POST(self):
if self.path == '/':
self.path = './index.html'
try:
form = cgi.FieldStorage()
firstname = form.getvalue('firstname')
lastname = form.getvalue('lastname')
url = form.getvalue('url')
print(firstname + ' ' + lastname + ' ' + url)
output=firstname + lastname + url
except:
self.send_error(404, 'Bad request submitted.')
self.end_headers()
self.wfile.write(bytes(output, 'utf-8'))
test_server = HTTPServer(('localhost', 8080), TestServerHandler)
test_server.serve_forever()
I then run the server.py file on my terminal and go to the http://localhost:8080/ url in my browser. However, when I submit the form I get an error in the browser. The terminal output shows an error that says 'cannot concatenate a 'str' with a 'nonetype'. Based on this error, the form values aren't being passed to this page. Either that or the HTTP server is passing them as query parameters. In any case, would I be able to use the cgi.FieldStorage class inside my HTTP server to access the form field values?
After about a week of looking for a solution to this I found that the http.server Python module is not really compatible with the cgi module. The cgi module is used inside of Python scripts that are passed form values from some HTML document on the web server (i.e. a form on the index.html page of a web server with the "action" attribute set to the Python script in question). However, in order for the cgi module to have access to the form values passed to that script (via the cgi.FieldStorage() call), the script must be running inside a web server. The problem with my code example above is that the server.py script I created IS a web server itself. Specifically, it creates an instance of HTTPServer using a custom request handler class I create (TestServerHandler). My custom class subclasses the CGIHTTPRequestHandler class contained in the http.server module. This class contains the Do_POST method. When implementing this method, any form data passed to the Python script is contained inside the self.rfile instance variable. To access this variable and get to the form data I wrote code similar to this.
content_length = int(self.headers['Content-Length'])
data_input = bytes.decode(self.rfile.read(content_length))
After you have the form_data stored in the data_input variable, you can then use a URL parser to access the values you need from the form.
The FieldStorage class has a constructor defined on this page. It's important parameters are fp, headers, and environ. OP is right in that cgi.FieldStorage() is usually called without parameters in a CGI script, in which case the parameters are filled in for you. However, we can create a FieldStorage object with variables that CGIHTTPRequestHandler provides for us:
form = cgi.FieldStorage(
fp=self.rfile,
headers=self.headers,
environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': self.headers['Content-Type'],
}
Related
I have a project where I save a lot of data in the sqlite3 database. I want to create a simple application in Flask - displaying appropriate data from the database.
New records are added to the database every few minutes - I want this application to refresh automatically. I prepared some information about it and found information that it can be done somehow using socketio Can flask framework send real-time data from server to client browser? or using AJAX. I tried to do something like this: or Python Flask date update real-time but I didn't manage to do it.
here's the code I'm using right now:
from flask import Flask, request, render_template, session, redirect
import numpy as np
import pandas as pd
import sqlite3
con = sqlite3.connect(r'path')
app = Flask(__name__)
df = pd.read_sql('select * from table1', con)
df1 = pd.read_sql('select * from table2', con)
#app.route('/', methods=("POST", "GET"))
def html_table():
return render_template('app.html',
tables=[df.to_html(classes='data1'),df1.to_html(classes='data2')],
titles=df.columns.values)
if __name__ == '__main__':
app.run()
Template:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Title</title>
</head>
<body>
{% for table in tables %}
{{titles[loop.index]}}
{{ table|safe }}
{% endfor %}
</body>
</html>
in this case, everything is displayed, but it doesn't work in real time.
I'm asking for some instructions or some advice on how to do it? Can I use the links I posted here somehow? This is my first application in Flask so I don't quite know how to achieve such a thing.
If i got your question what you want to do is load the data without loading the page. As you mentioned AJAX and socket.io, AJAX helps in getting the data on the same page but it requires reloading.It get different data on the same route without going to another route.Socketio will help you in achieving that.As it is used commonly in real time chat apps that is why we dont need to refresh the page to see the messages.In flask, use flask_socketio library to use it and you also need Javascript and by incorporating these two, you can do what you want.
The idea is that user push the button and Flask should add some content on the page.
In my case python and Flask is not an aleatory choice, because Python make a lot of backdoor work (that for example js can't).
And perfectly it should be just one page without redirection on another page.
So my question is:
How to get the moment when user push the button and run the function I want
How to make this function add some generated content on the page without reloading.
As one of the commenters said, you are missing some key concepts in the web development stack you are working with. Lemme try to explain.
Flask is a backend framework/server. It receives requests from your browser and generates and sends a response. Your browser can make a request and load the response as a new page.
JavaScript runs in the browser (or to use a more generic term frontend). It can make changes to the way the current page looks and behaves without reloading. It can make requests to the backend, process the results it receives and act on them.
There are two basic ways browser makes a request to the backend. When you load a page. And when your Javascript code makes an XHR request.
There are three basic ways your frontend does things in response to user actions.
Run some javascript and do something without making any requests. For example, highlight a button user is hovering his mouse over, autocomplete what user is typing from some pre-defined list, switch UI tabs, hide/show elements etc.
Make a request to the backend and load the results as a new page. For example, follow a link to a new page on the site, submit a form, reload the page etc. This is how most web applications used to work 10-15 years ago.
Run some javascript and use it to make a request to the backend in the background. Use javascript to process the results and update the page dynamically without reloading. This is how most web applications work now.
Basically, there are two ways to do what you want, the old way, where you have your Flask app generate the whole page with your new content and send it to the browser. This will make the browser reload the page. And the new way, where your Flask app just provides the data you want to change and the javascript in teh browser makes an XHR request in the background and uses the data to update the page. This can happen without a reload and this is the way most modern web apps work.
I hope this clears things up for you a bit.
app.py
from flask import Flask,render_template,request
app = Flask(__name__)
#app.route("/add_item")
def add_item():
#in reality you are doing something else i imagine
return "<li>%s</li>"%(request.args.get('name'))
#app.route('/')
def hello_world():
return render_template('index.html',items=["Item 1","Item 2","Item 3"])
if __name__ == '__main__':
app.run()
templates/index.html
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Document</title>
<script src="https://code.jquery.com/jquery-3.2.1.js"></script>
</head>
<body>
<ul id="items_holder">
{% for item in items %}
<li>{{ item }}</li>
{% endfor %}
</ul>
<button id="do_something">Do Something</button>
</body>
<script src="/static/app.js"></script>
</html>
static/app.js
$(function(){
$("#do_something").click(function(){
var item_name = prompt("enter item name")
$.get("/add_item?name="+item_name,function(result_data){
$('#items_holder').append(result_data)
})
})
})
I'm using Google Books API to integrate with an E-commerce site that I'm developing. The idea is to let the users search for books using a search bar, and then calling the Google API to output the number of books corresponding to the search keyword.
However, I get a 403 Forbidden error after I click submit after entering my query in the form. This is strange, because this never happened when I was testing my application on the localhost. Here's the code for my application:
main.py
class SearchHandler(Handler):
def get(self):
self.render("search.html")
def post(self):
keey = self.request.get('keey')
finaal = "https://www.googleapis.com/books/v1/volumes?q=" + keey + "&key=MY_APP_KEY"
f = urllib2.urlopen(finaal).read()
self.render("jsony.html", finaal = finaal)
app = webapp2.WSGIApplication(('/search', SearchHandler)], debug=True)
search.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
Book Name:<input type = "text" name = "keey">
<input type = "submit">
</form>
</body>
</html>
jsony.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
{{finaal}}
</form>
</body>
Now, the jsony.html is still incomplete. All I'm doing now is displaying the URL which contains the outputted json in it's raw, unprocessed form.
What seems to be causing this 403 error to arise after I deploy my application ?
EDIT 1:
The problem resolves when I remove the following line from my main python file:
f = urllib2.urlopen(finaal).read()
However, I would be needing my API's URL in order to extract data from its source code. What's happening ?
Try adding &country=US to the URL query string.
I tried to upload blobs to Google App Engine's blobstore using the following HTML form:
<!DOCTYPE html>
<html>
<head>
<meta charset=utf-8>
</head>
<body>
<form id=upload action={{upload_url}} method=post enctype=multipart/form-data>
Name: <input type=text name=name>
Your photo: <input type=file name=image required=required><br><br>
<input type=submit value=submit>
</form>
</body>
</html>
The value of the template variable {{upload_url}} is obtained by upload_url = blobstore.create_upload_url('/upload') on the server side. The post-handling script is as follows:
class Test(ndb.Model):
name = StringProperty()
image = StringProperty()
test = Test()
test.name = self.request.get('name')
image = self.get_uploads('image')[0]
test.image = str(image.key())
test.put()
Usually, the name field will be filled with non-English characters (E.g., Chinese). The above programs works fine on my local SDK. However, the name is incorrectly coded when the program is run on Google App Engine. What's the problem then?
Don't you have to put quotations around the meta tag parameter: <meta charset="UTF-8">? Also, try: <meta http-equiv="content-type" content="text/html; charset=utf-8" />. And, make sure you are saving your template's text document in UTF-8 encoding.
Just found out that this is an old bug for years, see here. There are two solutions:
(1) Add the following statements into app.yaml:
libraries:
- name: webob
version: "1.2.3"
(2) Add the file appengine_config.yaml with the following content:
# -*- coding: utf-8 -*-
from webob import multidict
def from_fieldstorage(cls, fs):
"""Create a dict from a cgi.FieldStorage instance.
See this for more details:
http://code.google.com/p/googleappengine/issues/detail?id=2749
"""
import base64
import quopri
obj = cls()
if fs.list:
# fs.list can be None when there's nothing to parse
for field in fs.list:
if field.filename:
obj.add(field.name, field)
else:
# first, set a common charset to utf-8.
common_charset = 'utf-8'
# second, check Content-Transfer-Encoding and decode
# the value appropriately
field_value = field.value
transfer_encoding = field.headers.get('Content-Transfer-Encoding', None)
if transfer_encoding == 'base64':
field_value = base64.b64decode(field_value)
if transfer_encoding == 'quoted-printable':
field_value = quopri.decodestring(field_value)
if field.type_options.has_key('charset') and field.type_options['charset'] != common_charset:
# decode with a charset specified in each
# multipart, and then encode it again with a
# charset specified in top level FieldStorage
field_value = field_value.decode(field.type_options['charset']).encode(common_charset)
# TODO: Should we take care of field.name here?
obj.add(field.name, field_value)
return obj
multidict.MultiDict.from_fieldstorage = classmethod(from_fieldstorage)
I have multiple forms in a html file, which all call the same python cgi script. For example:
<html>
<body>
<form method="POST" name="form1" action="script.cgi" enctype="multipart/data-form">
....
</form>
...
<form method="POST" name="form2" action="script.cgi" enctype="multipart/data-form">
...
</form>
...
</body>
</html>
And in my cgi script I do the following:
#!/usr/bin/python
import os
import cgi
print "content-type: text/html; charset=utf-8\n\n"
form = cgi.FieldStorate();
...
I am unable to get the data from the second from. I have tried to call FieldStorage multiple times, but that did not seem to work. So my question is how do I access different forms in the same cgi script?
You cannot. The browser submits one form, or the other, but not both.
If you need data from both forms, merge the forms into one <form> tag instead.
First, FieldStorage() consumes standard input, so it should only be instantiated once.
Second, only the data in the submitted form is sent to the server. The other forms may
as well not exist.
So while you can use the same cgi script to process both forms, if you need process both forms at the same time, as Martijn suggested, merge the forms into one <form>.