I'm trying to understand how to use WebTest to do integration testing and I'm stuck on the first example.
I've tried to follow the instructions. First I created a module that contains the code I want to test:
# functions.py
def application(environ, start_response):
"""docstring for application"""
# I added body, otherwise you get an undefined variable error
body = 'foobar'
headers = [('Content-Type', 'text/html; charset=utf8'),
('Content-Length', str(len(body)))]
start_response('200 OK', headers)
return [body]
Then I created a test runner file:
# test.py
from webtest import TestApp
from functions import application
app = TestApp(application)
resp = app.get('/')
assert resp.status == '200 OK'
assert resp.status_int == 200
When I execute test.py, I get the following error:
AssertionError: Iterator returned a non- object: 'foobar'.
What do I need to do to make this sample code from the WebTest documentation run?
In WSGI the body must be an iterable of bytes:
body = b'foobar'
Related
After the first Python3 application is created in a new RedHat OpenShift account, wsgi.py looks like this template:
def application(environ, start_response):
ctype = 'text/plain'
if environ['PATH_INFO'] == '/health':
response_body = "1"
elif environ['PATH_INFO'] == '/env':
response_body = ['%s: %s' % (key, value)
for key, value in sorted(environ.items())]
response_body = '\n'.join(response_body)
else:
ctype = 'text/html'
response_body = '<lots of html>'
response_body = response_body.encode('utf-8')
status = '200 OK'
response_headers = [('Content-Type', ctype),
('Content-Length', str(len(response_body)))]
#
start_response(status, response_headers)
return [response_body ]
#
# Below for testing only
#
if __name__ == '__main__':
from wsgiref.simple_server import make_server
httpd = make_server('localhost', 8051, application)
# Wait for a single request, serve it and quit.
httpd.handle_request()
My question is this: During the execution of this code, the following information appears in the log, "[28/Jun/2016 18:35:08] "GET / HTTP/1.1" 200 14". How do I get at that information from within the python program? Have examined the available variables but can not see it. Is there a function call I can make?
Many thanks for your insights.
.... later ....
From reader fat fantasma I see that I should provide additional information.
The user connects to the website using a URL that looks like this:
http://www.website.com/help?
The "help?" appears after the GET in "GET / HTTP/1.1", when the extension to the base URL appears. The GET phrase appears in the system log but I need access to it within wsgi.py's app() function. I do not see how to do that.
I'm trying to route certain URLs to a grafted WSGI app and also route sub URLs to a normal cherrypy page handler.
I need the following routes to work. All the other routes should return 404.
/api -> WSGI
/api?wsdl -> WSGI
/api/goodurl -> Page Handler
/api/badurl -> 404 Error
The WSGI app mounted at /api is a legacy SOAP based application. It needs to accept ?wsdl parameters but that is all.
I'm writing a new RESTful api at /api/some_resource.
The issue I'm having is that if the resource doesn't exist, it ends up sending the bad request to the legacy soap application. The final example "/api/badurl" ends up going to the WSGI app.
Is there a way to tell cherrypy to only send the first two routes to the WSGI app?
I wrote up a simple example of my issue:
import cherrypy
globalConf = {
'server.socket_host': '0.0.0.0',
'server.socket_port': 8080,
}
cherrypy.config.update(globalConf)
class HelloApiWsgi(object):
def __call__(self, environ, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return ['Hello World from WSGI']
class HelloApi(object):
#cherrypy.expose
def index(self):
return "Hello from api"
cherrypy.tree.graft(HelloApiWsgi(), '/api')
cherrypy.tree.mount(HelloApi(), '/api/hello')
cherrypy.engine.start()
cherrypy.engine.block()
Here's some unit tests:
import unittest
import requests
server = 'localhost:8080'
class TestRestApi(unittest.TestCase):
def testWsgi(self):
r = requests.get('http://%s/api?wsdl'%(server))
self.assertEqual(r.status_code, 200)
self.assertEqual(r.text, 'Hello World from WSGI')
r = requests.get('http://%s/api'%(server))
self.assertEqual(r.status_code, 200)
self.assertEqual(r.text, 'Hello World from WSGI')
def testGoodUrl(self):
r = requests.get('http://%s/api/hello'%(server))
self.assertEqual(r.status_code, 200)
self.assertEqual(r.text, 'Hello from api')
def testBadUrl(self):
r = requests.get('http://%s/api/badurl'%(server))
self.assertEqual(r.status_code, 404)
Outputs:
nosetests test_rest_api.py
F..
======================================================================
FAIL: testBadUrl (webserver.test_rest_api.TestRestApi)
----------------------------------------------------------------------
Traceback (most recent call last):
File line 25, in testBadUrl
self.assertEqual(r.status_code, 404)
AssertionError: 200 != 404
-------------------- >> begin captured stdout << ---------------------
Hello World from WSGI
Preface: I can not avoid mentioning that I wish everyone would ask questions in such a complete form with means to validate the answer :-)
Solutions out of CherryPy's scope:
do URL pre-processing at front-end server, e.g. nginx
create own WSGI middleware, i.e. wrap you legacy WSGI app in another app that will filter URLs
The latter is probably the preferred way to do it, but here's CherryPy's way. Documentation section host a foreign WSGI application in CherryPy says:
You cannot use tools with a foreign WSGI application.
Also you cannot set a custom dispatcher. But you can subclass the application tree.
#!/usr/bin/env python
import cherrypy
class Tree(cherrypy._cptree.Tree):
def __call__(self, environ, start_response):
# do more complex check likewise
if environ['PATH_INFO'].startswith('/api/badurl'):
start_response('404 Not Found', [])
return []
return super(Tree, self).__call__(environ, start_response)
cherrypy.tree = Tree()
globalConf = {
'server.socket_host': '0.0.0.0',
'server.socket_port': 8080,
}
cherrypy.config.update(globalConf)
class HelloApiWsgi:
def __call__(self, environ, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return ['Hello World from WSGI']
class HelloApi:
#cherrypy.expose
def index(self):
return "Hello from api"
cherrypy.tree.graft(HelloApiWsgi(), '/api')
cherrypy.tree.mount(HelloApi(), '/api/hello')
if __name__ == '__main__':
cherrypy.engine.signals.subscribe()
cherrypy.engine.start()
cherrypy.engine.block()
I don't see a password prompt in IE/Firefox even when I send the 401 status code with Tornado:-
import tornado.ioloop
import tornado.web
class UserHandler(tornado.web.RequestHandler):
def get(self, user_id):
self.set_header('WWW-Authenticate', 'Basic realm="users"')
self.send_error(status_code=401)
application = tornado.web.Application([
(r"/users/(\w+)", UserHandler),
],debug=True)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
Moreover the WWW-Authenticate header won't show in the response headers. When I don't send the 401 status, it does show in the header but it doesn't show the password prompt still.
This works although I am not sure why.
import tornado.ioloop
import tornado.web
class UserHandler(tornado.web.RequestHandler):
def get(self, user_id):
self.set_status(401)
self.set_header('WWW-Authenticate', 'Basic realm=Users')
application = tornado.web.Application([
(r"/users/(\w+)", UserHandler),
],debug=True)
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
The solution you found (using set_status() instead of send_error()) works because send_error() calls clear(), which "[r]esets all headers and content", as can be seen in the source. It then calls write_error() to send new content and headers based on the status code.
If you're fine with handling errors by setting all content and headers manually (or with your own methods), just never use send_error().
I found that you can also modify RequestHandler's methods to make send_error() work for you instead of against you. The WWW-Authenticate header is an expected part of any 401 response, so it would be nice if send_error(401) would automatically add it. That can be done by overriding write_error():
class BaseHandler(RequestHandler):
def write_error(self, status_code, **kwargs):
if status_code == 401:
self.add_header(
'WWW-Authenticate',
'Basic realm=' + json.dumps(self.realm)) # adds quotes
super(BaseHandler, self).write_error(status_code, **kwargs)
class UserHandler(BaseHandler):
realm = 'Users'
# ...
I'm using WSGI and trying to access the get/post data, using this code:
import os
import cgi
from traceback import format_exception
from sys import exc_info
def application(environ, start_response):
try:
f = cgi.FieldStorage(fp=os.environ['wsgi.input'], environ=os.environ)
output = 'Test: %s' % f['test'].value
except:
output = ''.join(format_exception(*exc_info()))
status = '200 OK'
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]
However I get the following error:
Traceback (most recent call last):
File "/srv/www/vm/custom/gettest.wsgi", line 9, in application
f = cgi.FieldStorage(fp=os.environ['wsgi.input'], environ=os.environ)
File "/usr/lib64/python2.4/UserDict.py", line 17, in __getitem__
def __getitem__(self, key): return self.data[key]
KeyError: 'wsgi.input'
Is it because wsgi.input does not exist in my version?
You're misusing the WSGI API.
Please create a minimal ("hello world") function that shows this error so we can comment on your code. [Don't post your entire application, it may be too big and unwieldy for us to comment on.]
The os.environ is not what you should be using. WSGI replaces this with an enriched environment. A WSGI application gets two arguments: one is a dictionary that includes 'wsgi.input'.
In your code...
def application(environ, start_response):
try:
f = cgi.FieldStorage(fp=os.environ['wsgi.input'], environ=os.environ)
Per the WSGI API specification (http://www.python.org/dev/peps/pep-0333/#specification-details), don't use os.environ. Use environ, the first positional parameter to your application.
The environ parameter is a dictionary
object, containing CGI-style
environment variables. This object
must be a builtin Python dictionary
(not a subclass, UserDict or other
dictionary emulation), and the
application is allowed to modify the
dictionary in any way it desires. The
dictionary must also include certain
WSGI-required variables (described in
a later section), and may also include
server-specific extension variables,
named according to a convention that
will be described below.
I'm searching a wsgi middleware which I can warp around a wsgi applications and which lets me monitor incoming and outgoing http requests and header fields.
Something like firefox live headers, but for the server side.
The middleware
from wsgiref.util import request_uri
import sys
def logging_middleware(application, stream=sys.stdout):
def _logger(environ, start_response):
stream.write('REQUEST\n')
stream.write('%s %s\n' %(
environ['REQUEST_METHOD'],
request_uri(environ),
))
for name, value in environ.items():
if name.startswith('HTTP_'):
stream.write(' %s: %s\n' %(
name[5:].title().replace('_', '-'),
value,
))
stream.flush()
def _start_response(code, headers):
stream.write('RESPONSE\n')
stream.write('%s\n' % code)
for data in headers:
stream.write(' %s: %s\n' % data)
stream.flush()
start_response(code, headers)
return application(environ, _start_response)
return _logger
The test
def application(environ, start_response):
start_response('200 OK', [
('Content-Type', 'text/html')
])
return ['Hello World']
if __name__ == '__main__':
logger = logging_middleware(application)
from wsgiref.simple_server import make_server
httpd = make_server('', 1234, logger)
httpd.serve_forever()
See also the werkzeug debugger Armin wrote, it's usefull for interactive debugging.
That shouldn't be too hard to write yourself as long as you only need the headers. Try that:
import sys
def log_headers(app, stream=None):
if stream is None:
stream = sys.stdout
def proxy(environ, start_response):
for key, value in environ.iteritems():
if key.startswith('HTTP_'):
stream.write('%s: %s\n' % (key[5:].title().replace('_', '-'), value))
return app(environ, start_response)
return proxy
If you want Apache-style logs, try paste.translogger
But for something more complete, though not in a very handy or stable location (maybe copy it into your source) is wsgifilter.proxyapp.DebugHeaders
And writing one using WebOb:
import webob, sys
class LogHeaders(object):
def __init__(self, app, stream=sys.stderr):
self.app = app
self.stream = stream
def __call__(self, environ, start_response):
req = webob.Request(environ)
resp = req.get_response(self.app)
print >> self.stream, 'Request:\n%s\n\nResponse:\n%s\n\n\n' % (req, resp)
return resp(environ, start_response)
The mod_wsgi documentation provides various tips on debugging which are applicable to any WSGI hosting mechanism and not just mod_wsgi. See:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
This includes an example WSGI middleware that captures request and response.
My WebCore project has a bit of middleware that logs the entire WSGI environment (thus Beaker sessions, headers, etc.) for the incoming request, headers for outbound responses, as well as performance information to a MongoDB database. Average overhead is around 4ms.
The module has been removed from the core package, but hasn’t yet been integrated into its own. The current version as of this answer is available in the Git history:
http://github.com/GothAlice/WebCore/blob/cd1d6dcbd081323869968c51a78eceb1a32007d8/web/extras/cprofile.py