I'm using Google Books API to integrate with an E-commerce site that I'm developing. The idea is to let the users search for books using a search bar, and then calling the Google API to output the number of books corresponding to the search keyword.
However, I get a 403 Forbidden error after I click submit after entering my query in the form. This is strange, because this never happened when I was testing my application on the localhost. Here's the code for my application:
main.py
class SearchHandler(Handler):
def get(self):
self.render("search.html")
def post(self):
keey = self.request.get('keey')
finaal = "https://www.googleapis.com/books/v1/volumes?q=" + keey + "&key=MY_APP_KEY"
f = urllib2.urlopen(finaal).read()
self.render("jsony.html", finaal = finaal)
app = webapp2.WSGIApplication(('/search', SearchHandler)], debug=True)
search.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
Book Name:<input type = "text" name = "keey">
<input type = "submit">
</form>
</body>
</html>
jsony.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
{{finaal}}
</form>
</body>
Now, the jsony.html is still incomplete. All I'm doing now is displaying the URL which contains the outputted json in it's raw, unprocessed form.
What seems to be causing this 403 error to arise after I deploy my application ?
EDIT 1:
The problem resolves when I remove the following line from my main python file:
f = urllib2.urlopen(finaal).read()
However, I would be needing my API's URL in order to extract data from its source code. What's happening ?
Try adding &country=US to the URL query string.
Related
I have a project where I save a lot of data in the sqlite3 database. I want to create a simple application in Flask - displaying appropriate data from the database.
New records are added to the database every few minutes - I want this application to refresh automatically. I prepared some information about it and found information that it can be done somehow using socketio Can flask framework send real-time data from server to client browser? or using AJAX. I tried to do something like this: or Python Flask date update real-time but I didn't manage to do it.
here's the code I'm using right now:
from flask import Flask, request, render_template, session, redirect
import numpy as np
import pandas as pd
import sqlite3
con = sqlite3.connect(r'path')
app = Flask(__name__)
df = pd.read_sql('select * from table1', con)
df1 = pd.read_sql('select * from table2', con)
#app.route('/', methods=("POST", "GET"))
def html_table():
return render_template('app.html',
tables=[df.to_html(classes='data1'),df1.to_html(classes='data2')],
titles=df.columns.values)
if __name__ == '__main__':
app.run()
Template:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Title</title>
</head>
<body>
{% for table in tables %}
{{titles[loop.index]}}
{{ table|safe }}
{% endfor %}
</body>
</html>
in this case, everything is displayed, but it doesn't work in real time.
I'm asking for some instructions or some advice on how to do it? Can I use the links I posted here somehow? This is my first application in Flask so I don't quite know how to achieve such a thing.
If i got your question what you want to do is load the data without loading the page. As you mentioned AJAX and socket.io, AJAX helps in getting the data on the same page but it requires reloading.It get different data on the same route without going to another route.Socketio will help you in achieving that.As it is used commonly in real time chat apps that is why we dont need to refresh the page to see the messages.In flask, use flask_socketio library to use it and you also need Javascript and by incorporating these two, you can do what you want.
I am trying to create a Python web server that takes user input and makes a request to a third-party API. The user input is obtained via a simple form I created.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Test Document</title>
</head>
<body>
<form action="server.py" method="post">
<label for="firstname">First Name</label>
<input type="text" name="firstname" id="firstname"><br>
<label for="lastname">Last Name</label>
<input type="text" name="lastname" id="lastname"><br>
<label for="url">URL</label>
<input type="text" name="url" id="application-url"><br>
<input type="submit" value="Submit">
</form>
</body>
</html>
I named this file index.html. My server.py file looks like this:
import cgi
from http.server import HTTPServer, CGIHTTPRequestHandler
class TestServerHandler(CGIHTTPRequestHandler):
def do_POST(self):
if self.path == '/':
self.path = './index.html'
try:
form = cgi.FieldStorage()
firstname = form.getvalue('firstname')
lastname = form.getvalue('lastname')
url = form.getvalue('url')
print(firstname + ' ' + lastname + ' ' + url)
output=firstname + lastname + url
except:
self.send_error(404, 'Bad request submitted.')
self.end_headers()
self.wfile.write(bytes(output, 'utf-8'))
test_server = HTTPServer(('localhost', 8080), TestServerHandler)
test_server.serve_forever()
I then run the server.py file on my terminal and go to the http://localhost:8080/ url in my browser. However, when I submit the form I get an error in the browser. The terminal output shows an error that says 'cannot concatenate a 'str' with a 'nonetype'. Based on this error, the form values aren't being passed to this page. Either that or the HTTP server is passing them as query parameters. In any case, would I be able to use the cgi.FieldStorage class inside my HTTP server to access the form field values?
After about a week of looking for a solution to this I found that the http.server Python module is not really compatible with the cgi module. The cgi module is used inside of Python scripts that are passed form values from some HTML document on the web server (i.e. a form on the index.html page of a web server with the "action" attribute set to the Python script in question). However, in order for the cgi module to have access to the form values passed to that script (via the cgi.FieldStorage() call), the script must be running inside a web server. The problem with my code example above is that the server.py script I created IS a web server itself. Specifically, it creates an instance of HTTPServer using a custom request handler class I create (TestServerHandler). My custom class subclasses the CGIHTTPRequestHandler class contained in the http.server module. This class contains the Do_POST method. When implementing this method, any form data passed to the Python script is contained inside the self.rfile instance variable. To access this variable and get to the form data I wrote code similar to this.
content_length = int(self.headers['Content-Length'])
data_input = bytes.decode(self.rfile.read(content_length))
After you have the form_data stored in the data_input variable, you can then use a URL parser to access the values you need from the form.
The FieldStorage class has a constructor defined on this page. It's important parameters are fp, headers, and environ. OP is right in that cgi.FieldStorage() is usually called without parameters in a CGI script, in which case the parameters are filled in for you. However, we can create a FieldStorage object with variables that CGIHTTPRequestHandler provides for us:
form = cgi.FieldStorage(
fp=self.rfile,
headers=self.headers,
environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': self.headers['Content-Type'],
}
I have a script which takes uploaded data, munges it together, turns it into a plot (using Bokeh) and then exports it to a directory as JSON.
At some point in the future, a user can hit the right URL and the appropriate plot should be displayed to the user as part of the HTML template.
I can generate the plot. I can save it as JSON. I can get the URL to retrieve it as JSON, but I cannot get the JSON plot to render within the template.
I've had a dig around the Bokeh documentation and examples, but they all seem to use a flask app to serve the pages.
I think I'm on the right track, using views.py to find and return JSON as part of a render() response, and then have Bokeh.embed.embed_items() do the work in the template to make it look right, but it's not working out - everything but the plot is displayed.
1) Create the plot and puts it in the directory for later use (app/results/1)
create plot.py
import os
import json
from django.conf import settings
from bokeh.embed import json_item
from bokeh.plotting import figure
x=[1,2,3,4,5]
y=[0,-1,-2,3,4]
p=figure(title="test_example")
p.line(x, y)
#json_export = json_item(p, "result")
json_export = json_item(p)
with open(os.path.join(settings.RESULTS_DIR,"1", "test.json"), 'w') as fp:
fp.write(json.dumps(json_export))
2) Set up the url
urls.py
urlpatterns=[
path('result/<int:pk>', views.resultdetailview, name='result-detail'),
]
3) Take the request, use the pk to find the plot json and render it all in the appropriate template.
views.py
def resultdetailview(request, pk):
results=str(pk)
with open(os.path.join(settings.RESULTS_DIR, results, "test.json"), 'r') as fp:
#data=json.load(fp)
data=json.loads(fp.read())
#context={'result':data}
#return render(request, 'app/sandbox_detail.html', context)
return render(request=request,
context={'json_object':data, 'resources':CDN.render()})
NB: If I instead use return JsonResponse(data, safe=False) then the url returns the json successfully ...
I think therefore that the issue is in the template.
4) Show the wonderous plot to the user
sandbox_detail.html
<header>
<link rel="stylesheet" href="http://cdn.bokeh.org./bokeh/release/bokeh-0.11.1.min.css" type="text/css" >
<script type="text/javascript" src="http://cdn.bokeh.org./bokeh/release/bokeh-0.11.1.min.js"> </script>
</header>
<h1> Title: Test </h1>
<div>Test</div>
<body>
<div id="result"></div>
<script>
Bokeh.embed.embed_item({{json_object}}, "result");
</script>
</body>
This template renders everything but the 'result' div.
What have I missed?
This is what I see so far:
FIRST: You are mixing 2 methods for injecting plot json data into the page.
According to documentation you can do it using either of these two methods:
1) specify the div directly:
Python: json_data = json.dumps(json_item(p, "myplot"))
JavaScript: Bokeh.embed.embed_item(item);
2) specify the div in embed_item function:
Python: json_data = json.dumps(json_item(p))
JavaScript: Bokeh.embed.embed_item(item, "myplot");
But not both of them at the same time. Could this be the problem?
SECOND: Preferably don't insert Bokeh resources by hand: rather use CDN.render() or INLINE.render() to automatically include all that your script needs:
import json
from bokeh.resources import CDN
return render(request = request,
template_name = 'app/sandbox_detail.html',
context = { json_object = json.loads(json_string),
resources = CDN.render() } )
sandbox_detail.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
{{ resources }}
</head>
<body>
<div id="result"></div>
<script>
Bokeh.embed.embed_item({{ json_object }}, "result");
</script>
</body>
</html>
THIRD: Make sure what you embed in the page is json object not a string (see variable naming above)
It helps when you debug your rendered template in your browser's debug tool.
I tried a very similar approach and found one large flaw: My browser noted that it could not find the None object. The reason here is that python stores the empty value as None, while JavaScript expects a null object.
The solution? Python already translates None to null, when you run json.dumps. To keep it that way, read the json string as a string. So instead of your data=json.loads(fp.read()) use data=fp.read().
I need to know it to correctly get filedata from POST dictionary.
I want to use Agile Uploader (resizes images on client side before upload on the server) in my project. It has process.php file as example of handling image on server. In that file:
$tmp_name = $_FILES["Filedata"]["file_name"]; // where "tmp_name" is file name
As Pyramid doesn't have FILES dictionary, I supose I have to find image in POST. But when I try to upload image POST is empty...
So where it send that image and how to find it on server side?
HTML (most part of html code taken from their demo):
<form action="/test" method="post" id="singularDemo" enctype="multipart/form-data">
<div id="single"></div>
</form>
Submit
<script type="text/javascript">
$('#single').agileUploaderSingle({
submitRedirect:'',
formId:'singularDemo',
progressBarColor:'#3b5998',
flashVars:{
firebug:true,
form_action:'/test'
}
});
</script>
Python (Pyramid code), just for testing - simple view:
def test(request):
if request.method == 'POST':
pass
return {}
Thanks!
Since the $_FILES global contains files submitted by a POST request, you can access them using request.POST:
# access the filename
filename = request.POST['Filedata'].filename
# access the actual file
input_file = request.POST['Filedata'].file
This is an exact equivalent of the PHP $_FILES variable, so if it doesn't work, something else must be wrong.
The Pyramid cookbook has more information about file uploads.
After login, I want the text "Welcome, Niklas" to display but after logging in I have to reload the page and I didn't understand how to make the page display the text from the server variable current_user. If I login and press reload then the correct welcome message appears. Can you help me achieve what I want? Why is there no simple working example for FB python + javascript? Can I implement facebook connect without javascript? If so, do I have to use and set the cookie myself? Thank you
{% load i18n %}
<!DOCTYPE html>
<html xmlns:fb="https://www.facebook.com/2008/fbml">
<head>
<title>
Test Facebook SDK
</title>
</head>
<body>
<div id="fb-root"></div>
<script>
window.fbAsyncInit = function() {
FB.init({
appId : '164355773607006', // App ID
channelURL : '//WWW.KOOLBUSINESS.COM/static/channel.html', // Channel File
status : true, // check login status
cookie : true, // enable cookies to allow the server to access the session
oauth : true, // enable OAuth 2.0
xfbml : true // parse XFBML
});
};
// Load the SDK Asynchronously
(function(d){
var js, id = 'facebook-jssdk'; if (d.getElementById(id)) {return;}
js = d.createElement('script'); js.id = id; js.async = true;
js.src = "//connect.facebook.net/en_US/all.js";
d.getElementsByTagName('head')[0].appendChild(js);
}(document));
</script>
<fb:login-button autologoutlink="true"></fb:login-button>
{% if current_user %}
<div id="user-ident">
<span>{% trans "Welcome," %} <b>{{ current_user.name|escape }}</b></span>
</div>
{% endif %}
</body>
</html>
Here's how I get the variable current_user
#property
def current_user(self):
if not hasattr(self, "_current_user"):
self._current_user = None
cookie = facebook.get_user_from_cookie(
self.request.cookies, facebookconf.FACEBOOK_APP_ID, facebookconf.FACEBOOK_APP_SECRET)
logging.debug("logging cookie"+str(cookie))
if cookie:
# Store a local instance of the user data so we don't need
# a round-trip to Facebook on every request
user = FBUser.get_by_key_name(cookie["uid"])
logging.debug("user "+str(user))
logging.debug("username "+str(user.name))
if not user:
graph = facebook.GraphAPI(cookie["access_token"])
profile = graph.get_object("me")
user = FBUser(key_name=str(profile["id"]),
id=str(profile["id"]),
name=profile["name"],
profile_url=profile["link"],
access_token=cookie["access_token"])
user.put()
elif user.access_token != cookie["access_token"]:
user.access_token = cookie["access_token"]
user.put()
self._current_user = user
return self._current_user
When using Oauth 2.0
FB.Init(...oauth=true)
the login button is setting a cookie into your domain named
fbsr_(appid)=....
which is encoded using the app secret key, and which do not contain the user token anymore.
If you really want to avoid the use of java script client side (which is the most simple way) you can leverage the presence of this cookie to know if the user is connected to facebook and then perform any authorization check or dynamically display a welcome message.
I do not use python, so i do not have working example, but this give you a way to search.
Hope this help.