I am using flask.ext.rest to build a api. I want return some chinese string. However, every time I receive "\u7231"(This is a string of length 8). What should I do to receive 爱 ?
from flask import Flask
from flask.ext.restful import reqparse, abort, Api, Resource
class E2C(Resource): # English to Chinglish
def get(self):
chinese = u'爱'
type(chinese) # unicode
return chinese
The get method should return a Response instance. see docs here.
The code should be:
from flask import Flask, make_response
from flask.ext.restful import reqparse, abort, Api, Resource
class E2C(Resource): # English to Chinglish
def get(self):
chinese = u'爱'
type(chinese) # unicode
return make_response(chinese)
'\u7231' is indeed the character you seek, the problem is with the rendering of that character by whatever device you're using to display it.
So your browser page probably needs to include a meta tag to render UTF-8
<head>
<meta charset="UTF-8">
</head>
cURL, on the other hand, given a quick google for you, it sounds like it receives the unicode char ok by default so it's only a question of what you're using to store/display the results... you need to prevent the terminal or file system or program, or whatever you're using, from converting that unicode char back into it's numerical representation. So if you save it to file you need to ensure that file gets a utf-8 character encoding; if you render it to screen you need ensure your screen is capable and expecting it.
make_response can actually solve the problem.
My case is slightly different, as I have a dictionary object and it hasn't not been encoded to utf-8 yet. so I modify the solution from #Xing Shi, in case there are someone else having similar problem as I did.
def get(self):
return make_response(
dumps({"similar": "爱“, "similar_norm": ”this－thing"},
ensure_ascii=False).decode('utf-8'))
Related
It seems that when I try to decode some bytes that where decoded from base 64 it gives an Input Format not Supported. I cannot isolate the issue, as when I bring the decoded logic alone into a new file, the error will not happen, making me think that this is something to do with the way flask passes arguments to the functions.
Code:
from flask import Flask
import base64
import lzma
from urllib.parse import quote, unquote
app = Flask('app')
#app.route('/')
def hello_world():
return 'Hello, World!<br><button onclick = "var base = \'https://Text-Viewer-from-Bsace-64-URL.inyourface3445.repl.co/encode\';location.href = `${base}/${prompt(\'What do you want to send?\')}`" >Use</button>'
newline = '/n'
#app.route('/view/<path:b64>')
def viewer(b64):
print(type(b64))
s1 = base64.b64decode(b64.encode() + b'==')
s2 = lzma.decompress(s1).decode()
s3 = unquote(s2).replace(newline, '<br>')
return f'<div style="overflow-x: auto;">{s3}</div>'
#app.route('/encode/<path:txt>')
def encode(txt):
quote_text = quote(txt, safe = "")
compressed_text = lzma.compress(quote_text.encode())
base_64_txt = base64.b64encode(compressed_text).decode()
return f'text link '
app.run(host='0.0.0.0', port=8080, debug=True)
Can someone explain what I am doing wrong?
You are passing a base64-encoded string as a part of the URL, and that string may contain characters that gets mangled in the process.
For example, visiting /encode/hello will give the following URL:
https://text-viewer-from-bsace-64-url.inyourface3445.repl.co/view//Td6WFoAAATm1rRGAgAhARYAAAB0L+WjAQAEaGVsbG8AAAAAsTe52+XaHpsAAR0FuC2Arx+2830BAAAAAARZWg==
Several characters could go wrong:
The first character is /, and as a result Flask will redirect from view//TD6... to view/TD6...: in other words the first character gets deleted
Depending on how URL-encoding is performed by the browser and URL-decoding is performed by Flask, the + character may be decoded into a space
To avoid these issues, I would suggest using base64.urlsafe_b64encode / base64.urlsafe_b64decode which are versions of the base64 encoding where the output can be used in URLs without being mangled.
The following changes on your code seems to do the trick:
s1 = base64.urlsafe_b64decode(b64.encode()) in viewer
base_64_txt = base64.urlsafe_b64encode(compressed_text).decode() in encode
I'm working on some code that pulls course info from Canvas. As pure python, it works fine. If I try to incorporate it with Flask, I get the following error
requests.exceptions.MissingSchema: Invalid URL 'run/api/v1/courses/1234567': No schema supplied. Perhaps you meant http://run/api/v1/courses/1234567?
This is the code in question:
Canvas file
import sys
from canvasapi import Canvas
def getinfo():
canvasurl = "https://canvas.instructure.com/";
canvastoken = #Redacted for this example
try:
canvastoken = sys.argv[1];
canvasurl = sys.argv[2];
except:
print()
#Create a new canvas object passing in the newly aquired url and token
canvas = Canvas(canvasurl, canvastoken);
#print(canv)
# Create a new course oject -- passing in course number as a parameter
# Course number is currently hard coded
print(canvas.get_course(1234567))
Flask file code (the file that I'm trying to run):
from flask import Flask
import canvas
canvas.getinfo()
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
No schema provided usually means you haven't specified the http:// or https:// in the URL.
In the code you provided, I don't see any reference to a run/api/v1/courses/1234567. One possibility is if you are using the url_for method from requests anywhere in your code, try setting _external=True:
url = url_for('relativeURL', _external=True)
This allows Flask to construct an absolute URL (i.e., a URL with domain included).
If you aren't using url_for, check other places in your code where you might be omitting the http or https from the URL.
If you update your question to include the part that refers to the offending URL, we might be able to provide more specific help.
This is probably a stupid question/problem, but i could not find an answer for it. Also, it may not realy be twisted specific.
I am trying to write a resource for a twisted.web webserver, which should serve a page containing non-ascii characters.
According to this discusson, all i need to do is to set the Content-Type HTTP-Header and return an encoded string.
Unfortunately, the page shows invalid characters.
Here is the code (as a .rpy):
"""a unicode test"""
from twisted.web.resource import Resource
class UnicodeTestResource(Resource):
"""A unicode test resource."""
isLeaf = True
encoding = "utf-8"
def render_GET(self, request):
text = u"unicode test\n ä ö ü ß"
raw = u"<HTML><HEAD><TITLE>Unicode encoding test</TITLE><HEAD><BODY><P>{t}</P></BODY></HTML>".format(t=text)
enc = raw.encode(self.encoding)
request.setHeader("Content-Type", "text/html; charset=" + self.encoding)
return enc
resource = UnicodeTestResource()
The result (without the html) is: unicode test ä ö ü Ã.
Is this caused by an encoding mismatch between the server and the client?
I am using python 2.7.12 and twisted 17.1.0. The page has been accessed using firefox.
Sorry for my terrible english.
Thanks
EDIT: I found the problem. I assumed that twisted.web.static.File with a ResourceScript processor would use the encoding specified in the file in which the reactor is running.
Apparently this is not the case.
Adding # -*- coding: utf-8 -*- on the top of each file fixed the problem.
I'm making a small flask app where I had something like this:
#app.route('/bye')
def logout():
session.pop('logged_in', None)
flash('Adiós')
return redirect('/index')
Needless to say when I ran the application and I navigated to '/bye' it gave me a UnicodeDecodeError. Well, now it gives me the same unicodedecodeerror on every page that extends the base template (which renders the messages) even after restarting the application. and always with the same dump() despite removing that flash in the source code. All I can think of is what the crap? Help please.
Well I had to restart my computer to clear the stupid session cache or something.
I think that flash() actually creates a session called session['_flashes']. See this code here. So you will probably have to either:
clear/delete the cookie
OR
session.pop('_flashes', None)
Flask flashing stores the messages in a session cookie until they are succesfully "consumed".
If you get a UnicodeDecodeError (https://wiki.python.org/moin/UnicodeDecodeError) in this case the messages is not consumed, so you get the error again and again.
My solution was to delete the cookie from the browser
Since I had the problem when using localization, I solved the cause now by installing my translation object like:
trans = gettext.GNUTranslations(...)
trans.install(unicode=True)
and having UTF-8 encoding in my python source files and "Content-Type: text/plain; charset=UTF-8\n" in the translation file (.pot)
You're using an non ascii string "adiós", so you need to ensure that python will process strings as unicode, not as ascii.
Add this to the header of your python file. This will tell the compiler that your file contains utf8 strings
#!/usr/bin/env python
# -*- coding: utf-8 -*-
so your code will be something like this:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from flask import Flask
app = Flask()
#app.route('/bye')
def logout():
session.pop('logged_in', None)
flash('Adiós')
return redirect('/index')
I have a webapp that export reports in PDF. Everything is fine when the query returns less than 100 values. When the number of records raise above 100 the server raise a 502 Proxy Error. The report outputs fine in HTML. The process that hangs up the server is the conversion from html to PDF.
I'm using xhtml2pdf (AKA pisa 3.0) to generate the PDF. The algorythm is something like this:
def view1(request, **someargs):
queryset = someModel.objects.get(someargs)
if request.GET['pdf']:
return pdfWrapper('template.html',queryset,'filename')
else:
return render_to_response('template.html',queryset)
def pdfWrapper(template_src, context_dict, filename):
################################################
#
# The code comented below is an older version
# I updated the code according the comment recived
# The function still works for short HTML documents
# and produce the 502 for larger onese
#
################################################
##import cStringIO as StringIO
import ho.pisa as pisa
from django.template.loader import get_template
from django.template import Context
from django.http import HttpResponse
##from cgi import escape
template = get_template(template_src)
context = Context(context_dict)
html = template.render(context)
response = HttpResponse()
response['Content-Type'] ='application/pdf'
response['Content-Disposition']='attachment; filename=%s.pdf'%(filename)
pisa.CreatePDF(
src=html,
dest=response,
show_error_as_pdf=True)
return response
##result = StringIO.StringIO()
##pdf = pisa.pisaDocument(
## StringIO.StringIO(html.encode("ISO-8859-1")),
## result)
##if not pdf.err:
## response = HttpResponse(
## result.getvalue(),
## mimetype='application/pdf')
## response['Content-Disposition']='attachement; filename=%s.pdf'%(filename)
## return response
##return HttpResponse('Hubo un error<pre>%s</pre>' % escape(html))
I've put some thought about creating a buffer so the server can free some memory but I didn't find anything yet.
Anyone could help? please?
I can't tell you exactly what causes your problem - it could be caused by buffering problems in StringIO.
However, you are wrong if you assume that this code would actually stream the generated PDF data: StringIO.getvalue() returns the content of the string buffer at the time this method is called, not an output stream (see http://docs.python.org/library/stringio.html#StringIO.StringIO.getvalue).
If you want to stream the output, you can treat the HttpResponse instance as a file-like object (see http://docs.djangoproject.com/en/1.2/ref/request-response/#usage).
Secondly, I don't see any reason to make use of StringIO here. According to the documentation of Pisa I found (which calls this function CreatePDF, by the way) the source can be a string or a unicode object.
Personally, I would try the following:
Create the HTML as unicode string
Create and configure the HttpResponse object
Call the PDF generator with the string as input and the response as output
In outline, this could look like this:
html = template.render(context)
response = HttpResponse()
response['Content-Type'] ='application/pdf'
response['Content-Disposition']='attachment; filename=%s.pdf'%(filename)
pisa.CreatePDF(
src=html,
dest=response,
show_error_as_pdf=True)
#response.flush()
return response
However, I did not try if this actually works. (I did this sort of PDF streaming only in Java, so far.)
Update: I just looked at the implementation of HttpResponse. It implements the file interface by collecting the chunks of strings written to it in a list. Calling response.flush() is pointless, because it does nothing. Also, you can set response parameters like Content-Type even after the response has been accessed as file-object.
Your original problem may also be related to the fact you never closed the StringIO objects. The underlying buffer of a StringIO object is not released before close() is called.