Develop Reference

How to test file upload in Django rest framework using PUT?

I would like to write a unit test for a view on a Django REST Framework application. The test should upload a file using PUT, in essence equivalent to
http -a malkarouri PUT http://localhost:8000/data-packages/upload/ka #tmp/hello.py
The code I have written so far is
factory = APIRequestFactory()
request = factory.put( '/data-packages/upload/ka',
data,
content_type='application/octet-stream',
content_disposition="attachment; filename=data.dump")
force_authenticate(request, user)
view = PackageView.as_view()
response = view(request, "k.py")
which, obviously, does not upload a file. The specific error when running the test is 400:
{u'detail': u'Missing filename. Request should include a Content-Disposition header with a filename parameter.'}
Notably, I am using a request factory to test the view rather than a full client. That is what makes solutions such as the one in this question not work for me.
What is the correct way to set the content disposition header?

Hi you need to use the SimpleUploadedFile wrapper for that :
from django.core.files.uploadedfile import SimpleUploadedFile
from django.core.files import File
data = File(open('path/bond-data.dump', 'rb'))
upload_file = SimpleUploadedFile('data.dump', data.read(),content_type='multipart/form-data')
request = factory.put( '/data-packages/upload/ka',
{'file':upload_file,other_params},
content_type='application/octet-stream',
content_disposition="attachment; filename=data.dump")
Ps : I am using APITestCase

from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
from rest_framework.test import APIClient
class MyTest(TestCase):
client_class = APIClient
def test_it(self):
file = SimpleUploadedFile("file.txt", b"abc", content_type="text/plain")
payload = {"file": file}
response = self.client.post("/some/api/path/", payload, format="multipart")
self.assertEqual(response.status_code, 201)
# If you do more calls in this method with the same file then seek to zero
file.seek(0)

Python restplus API to upload and dowload files

With python flask_restplus what is correct way to have a post and get methods to get and push a file e.g. xlsx to the server ?
Does the marshaling need to be used for this ?
reference: https://philsturgeon.uk/api/2016/01/04/http-rest-api-file-uploads/
This answer give general info but not in the python>flask>restplus context: REST API File Upload

First you need to configure a parser
# parsers.py
import werkzeug
from flask_restplus import reqparse
file_upload = reqparse.RequestParser()
file_upload.add_argument('xls_file',
type=werkzeug.datastructures.FileStorage,
location='files',
required=True,
help='XLS file')
Then add a new resource to your api namespace
# api.py
import …
import parsers
#api.route('/upload/')
class my_file_upload(Resource):
#api.expect(parsers.file_upload)
def post(self):
args = parsers.file_upload.parse_args()
if args['xls_file'].mimetype == 'application/xls':
destination = os.path.join(current_app.config.get('DATA_FOLDER'), 'medias/')
if not os.path.exists(destination):
os.makedirs(destination)
xls_file = '%s%s' % (destination, 'custom_file_name.xls')
args['xls_file'].save(xls_file)
else:
abort(404)
return {'status': 'Done'}
I hope this helps.

self.async_callback missing from classes that inherit from RequestHandler

So I'm learning the Tornado web framework right now, and following a few examples from a book I managed to get authentication with OAuth2/Google almost working. It redirects to Google to prompt a user to sign in, but after signing in it throws the following error at me:
Traceback (most recent call last):
File "C:\python27\lib\site-packages\tornado\web.py", line 1288, in _stack_context_handle_exception
raise_exc_info((type, value, traceback))
File "C:\python27\lib\site-packages\tornado\web.py", line 1475, in wrapper
result = method(self, *args, **kwargs)
File "C:\Users\enricojr\Downloads\Github\LearningTornado\grumble_login.py", line 8, in get
self.get_authenticated_user(callback=self.async_callback(self._on_auth))
AttributeError: 'LoginHandler' object has no attribute 'async_callback'
Upon checking the Tornado source code on Github, it does appear indeed that the async_callback() method is gone. Any idea on how to handle this? Every example of authentication using Tornado's auth module I've seen thus far calls for it to be used, and since I'm new to the whole 'asynchronous web programming' thing I'm not quite sure at this point how else I can do it.
My code is below, Python 2.7 and Tornado 4.0.2:
############################
# grumble_login.py
############################
from tornado.auth import GoogleMixin
from tornado.web import RequestHandler, asynchronous
class LoginHandler(RequestHandler, GoogleMixin):
#asynchronous
def get(self):
if self.get_argument("openid.mode", None):
self.get_authenticated_user(callback=self.async_callback(self._on_auth))
return
else:
self.authenticate_redirect()
def _on_auth(self, user):
if not user:
self.clear_all_cookies()
raise tornado.web.HTTPError(500, "Google auth failed.")
else:
# the user id serves as the basis for
# all the data we store.
self.set_secure_cookie('user_id', user['id'])
self.redirect("/")
class LogoutHandler(RequestHandler):
def get(self):
self.clear_all_cookies()
self.render("logout.html")
############################
# main.py
############################
import os
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from tornado.web import Application
from tornado.options import define, options
from grumble_handlers import *
from grumble_user_handlers import *
from grumble_login import *
# /user/profile/<user> - leads to profile page
# /user/posts - leads to posts made by user
# /post/<id> - leads to a specific post
# /post/add - add post
# /post/edit - edit post
# /post/delete - delete post
# D$oP5lz3D$oP5lz3
TEMPLATE_PATH = os.path.join(os.path.dirname(__file__), "templates")
STATIC_PATH = os.path.join(os.path.dirname(__file__), "static")
if __name__ == "__main__":
define("port", default=8000, help="run on the given port", type=int)
settings_dict = {
'cookie_secret': "GENERATE NEW KEY HERE",
'template_path': TEMPLATE_PATH,
'static_path': STATIC_PATH,
'debug': True,
'login_url': "/login",
}
general_handlers = [
(r"/", IndexHandler),
(r"/login", LoginHandler),
(r"/logout", LogoutHandler)
]
post_handler_list =[
(r"/post/(\d+)", PostFetchHandler), # leads to a single post page
(r"/post/add", PostAddHandler),
(r"/post/edit", PostEditHandler),
(r"/post/delete", PostDeleteHandler),
]
user_handler_list = [
(r"/user/(\d+)/posts", UserPostsHandler), # lists out all the posts by a given user
(r"/user/(\d+)/profile", UserProfileHandler),
]
master_handler_list = general_handlers + post_handler_list + user_handler_list
app = Application(
handlers=master_handler_list,
**settings_dict
)
http_server = HTTPServer(app)
http_server.listen(options.port)
# GO TEAM GO
IOLoop.instance().start()
############################
# grumble_handlers.py
############################
...
class IndexHandler(NotImplementedHandler):
#authenticated
def get(self):
pass

The book is, unfortunately, long outdated. async_callback was removed in Tornado 4 since it is no longer required. You can simply do:
self.get_authenticated_user(callback=self._on_auth)
Once you have that working, I recommend following the coroutine documents and learn how to use "yield":
user = yield self.get_authenticated_user()

How do I handle logouts with repoze.who (and bottle.py)?

I'm trying to make bottle.py work with repoze.who, and so far have managed to put together the following very simplistic program to get it working, using a combination of various examples I've found. Obviously this isn't something I'd run in production, I'm just trying to make the least complicated code I can so that I can learn how to use this - but unfortunately, tutorials for using bottle.py with repoze.who are very few and far between.
This example below works, and allows someone to login with username/password of admin/admin. What am I supposed to do with repoze.who to make the logout() function work? I gather there's a forget function that might be for this purpose, but I can't work out how I'm meant to call it.
Thanks.
from bottle import route, run, app, get, abort, request
from StringIO import StringIO
import repoze
from repoze.who.middleware import PluggableAuthenticationMiddleware
from repoze.who.interfaces import IIdentifier
from repoze.who.interfaces import IChallenger
from repoze.who.plugins.basicauth import BasicAuthPlugin
from repoze.who.plugins.auth_tkt import AuthTktCookiePlugin
from repoze.who.plugins.cookie import InsecureCookiePlugin
from repoze.who.plugins.form import FormPlugin
from repoze.who.plugins.htpasswd import HTPasswdPlugin
from repoze.who.classifiers import default_request_classifier
from repoze.who.classifiers import default_challenge_decider
import logging, sys
import pprint
#route('/')
def root():
if request.environ.get('repoze.who.identity') is None:
abort(401, "Not authenticated")
return "Authenticated"
#route('/hello')
def index():
identity = request.environ.get('repoze.who.identity')
if identity == None:
abort(401, "Not authenticated")
user = identity.get('repoze.who.userid')
return '<b>Hello %s!</b>' % user
#route('/logout')
def logout():
# I have no idea what to put here
pass
io = StringIO()
salt = 'aa'
for name, password in [ ('admin', 'admin'), ('paul', 'paul') ]:
io.write('%s:%s\n' % (name, password))
io.seek(0)
def cleartext_check(password, hashed):
return password == hashed
htpasswd = HTPasswdPlugin(io, cleartext_check)
basicauth = BasicAuthPlugin('repoze.who')
auth_tkt = AuthTktCookiePlugin('secret', 'auth_tkt')
form = FormPlugin('__do_login', rememberer_name='auth_tkt')
form.classifications = { IIdentifier:['browser'],
IChallenger:['browser'] }
identifiers = [('form', form),('auth_tkt',auth_tkt),('basicauth',basicauth)]
authenticators = [('htpasswd', htpasswd)]
challengers = [('form',form), ('basicauth',basicauth)]
mdproviders = []
log_stream = None
import os
if os.environ.get('WHO_LOG'):
log_stream = sys.stdout
middleware = PluggableAuthenticationMiddleware(
app(),
identifiers,
authenticators,
challengers,
mdproviders,
default_request_classifier,
default_challenge_decider,
log_stream = log_stream,
log_level = logging.DEBUG
)
if __name__ == '__main__':
run(app=middleware, host='0.0.0.0', port=8080, reloader=True)
else:
application = middleware
run(host='0.0.0.0', port=8080)

If you can I would use RedirectingFormPlugin rather than FormPlugin. RedirectingFormPlugin allows you to register a logout URL. With it you do not have to implement the /logout handler as such as the RedirectingFormPlugin intercepts the request and handles the calls to forget etc. for you. I have used this with Bobo and appengine and it works well.

if you still want to do it the unpreferred way in old repoze.who v1, following worked for me:
from bottle import response # , redirect
# ...
#route('/logout')
def logout():
identity = request.environ.get('repoze.who.identity')
if identity:
for (i_name, i) in identifiers:
hdrs = i.forget(request.environ, identity)
[ response.add_header(*h) for h in hdrs ]
## following would be nice, but does not work,
## since redirect is not using defined response headers
# rfr = request.get_header('referer', '/')
# redirect(rfr)
## so we do just this:
return "you have been hopefully logged out"

Bottle.py error routing

Bottle.py ships with an import to handle throwing HTTPErrors and route to a function.
Firstly, the documentation claims I can (and so do several examples):
from bottle import error
#error(500)
def custom500(error):
return 'my custom message'
however, when importing this statement error is unresolved but on running the application ignores this and just directs me to the generic error page.
I found a way to get around this by:
from bottle import Bottle
main = Bottle()
#Bottle.error(main, 500)
def custom500(error):
return 'my custom message'
But this code prevents me from embedding my errors all in a separate module to control the nastiness that would ensue if I kept them in my main.py module because the first argument has to be a bottle instance.
So my questions:
Has anyone else experienced this?
why doesn't error seem to resolve in only my case (I installed from pip install bottle)?
Is there a seamless way to import my error routing from a separate python module into the main application?

If you want to embed your errors in another module, you could do something like this:
error.py
def custom500(error):
return 'my custom message'
handler = {
500: custom500,
}
app.py
from bottle import *
import error
app = Bottle()
app.error_handler = error.handler
#app.route('/')
def divzero():
return 1/0
run(app)

This works for me:
from bottle import error, run, route, abort
#error(500)
def custom500(error):
return 'my custom message'
#route("/")
def index():
abort("Boo!")
run()

In some cases I find it's better to subclass Bottle. Here's an example of doing that and adding a custom error handler.
#!/usr/bin/env python3
from bottle import Bottle, response, Route
class MyBottle(Bottle):
def __init__(self, *args, **kwargs):
Bottle.__init__(self, *args, **kwargs)
self.error_handler[404] = self.four04
self.add_route(Route(self, "/helloworld", "GET", self.helloworld))
def helloworld(self):
response.content_type = "text/plain"
yield "Hello, world."
def four04(self, httperror):
response.content_type = "text/plain"
yield "You're 404."
if __name__ == '__main__':
mybottle = MyBottle()
mybottle.run(host='localhost', port=8080, quiet=True, debug=True)