Develop Reference

Webob / Pyramid query string parameters out of order upon receipt

I am running Pyramid as my API server. Recently we started getting query string parameters out of order when handed to the RESTful API server. For example, a GET to /v1/finishedGoodRequests?exact=true&id=39&join=OR&exact=false&name=39
is logged by the RESTful api module upon init as request.url:
v1/finishedGoodRequests?join=OR&name=39&exact=true&exact=false&id=39
with request.query_string: join=OR&name=39&exact=true&exact=false&id=39
I process the query params in order to qualify the search, in this case id exactly 39 or 39 anywhere in the name. What kind of possible server setting or bug could have crept in to the server code to cause such a thing? It is still a MultiDict...

As a simple example, this works fine for me, and the MultiDict has always preserved the order and so I suspect something is getting rewritten by something you're using in your stack.
from pyramid.config import Configurator
from pyramid.view import view_config
from waitress import serve
#view_config(renderer='json')
def view(request):
return list(request.GET.items())
config = Configurator()
config.scan(__name__)
app = config.make_wsgi_app()
serve(app, listen='127.0.0.1:8080')
$ curl http://localhost:8080\?join=OR\&name=39\&exact=true\&exact=false\&id=39
[["join", "OR"], ["name", "39"], ["exact", "true"], ["exact", "false"], ["id", "39"]]
Depending on which WSGI server you are using, often you can view environ vars to see the original url which may be handy. Waitress does not, so instead just put something high up in the pipeline (wsgi middleware) that can log out the environ['QUERY_STRING'] and see if it doesn't match somewhere lower down in your stack.

Setting multiple handlers with Python App Engine

I just cannot seem to get multiple handlers working. I'm using Google App Engine with Python. The base URL returns "Hello world!", as expected, but I keep getting a 404 error when I try to visit "/girl".
As far as I can tell I'm doing exactly what is specified in the docs:
https://cloud.google.com/appengine/docs/python/config/appconfig
And in this similar question:
YAML file url and script in GAE python
and yet trying all variants I can think of on these models does not resolve my 404 problem. I am a beginner and don't really understand how the app.yaml file works, thus I'm pretty sure that I'm not specifying the handlers correctly. But I don't know how to fix it.
app.yaml
application: multiapp
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: ./girl/.*
script: girl.app
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.2"
main.py
import webapp2
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hello world!')
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
girl.py
import webapp2
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hey girl!')
app = webapp2.WSGIApplication([
('/',MainHandler),
('/girl/', MainHandler)
], debug=True)
Added: Different organization of the project could definitely avoid having to solve this problem, but I would also like to know why setting these multiple handlers isn't working to begin with.

I know this is an old thread, and that you sort of got your answer, but I want to give a better explanation of what was happening, as I have just found the answer to your fundamental question: How to split handlers into different files?
The key problem is that you are working with regular expressions, both when you declare your WSGIApplication
app = webapp2.WSGIApplication([
('/', MainHandler),
('/girl/', GirlHandler)
], debug=True)
and in your yaml file
handlers:
- url: ./girl/.*
script: girl.app
- url: .*
script: main.app
The path in your yaml url: regex must be the full path from your webapp domain. So if your domain is localhost:8080 then in your yaml file yo should add the url from localhost:8080 onwards. If you type url: /girl/.*
you're asking the browser to match domain+regex: localhost:8080/girl/.*
This means:
match this part exactly: localhost:8080/girl/
match any characters that follow the first part (as that's what ".*"
means in a regex)
So the following yaml statement:
handlers:
- url: /girl/.*
script: girl.app
means that for any url of the form domain/girl/ anything (or nothing) use the the app variable in girl.py file app = webapp2.WSGIApplication(...)
The first implication of this, is there is no purpose in writing a handler for a url that girl.py will never handle, such as when you coded:
girl.py
import webapp2
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hey girl!')
app = webapp2.WSGIApplication([
('/',MainHandler),
], debug=True)
You will never use this main handler, as it will only 'activate' when you browse for domain/, but you said in your yaml file that you only wanted girl.py to handler urls of the form domain/girl/something. It's a contradiction
This means that for your setup to work, in girl.py you should only write handlers for urls that match the regex in your yaml. in this case, any regex that is also accepted by /girl/.*.
As a sidenote, if you wanted your girlHandler to work on either domain/girl and domain/girl/ you should use this regex in your yaml file:
handlers:
- url: ./girl(?:/.*)?
script: girl.app
- url: .*
script: main.app
as this makes the /.* part optional
Hope this helps anyone reaching this question, as an insight of how webapp2 calls each handler depending on the url given

You should bundle all your routes together in one file, and have different handlers for different routes.
main.py
import webapp2
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hello world!')
class GirlHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hey Girl!')
app = webapp2.WSGIApplication([
('/', MainHandler),
('/girl/', GirlHandler)
], debug=True)
then in your app.yaml you only have to link to main.app
application: multiapp
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.2"
You don't need multiple webapp2.WSGIApplication instances.

You could have central routing while keeping handlers in separate files with something along these lines in main.py:
app = webapp2.WSGIApplication([
('/girl/.*', "girl.MainHandler")
('/.*', MainHandler),
], debug=True)
Another possibility of running largely independent "apps" while still be able to share some info across them (like authentication) would be to make them modules of the main app. While offering more flexibility in the long run they do have a non-neglijible learning curve (and docs aren't always up2date for them)

Organizational issues aside, the reason this wasn't working was that I didn't include a trailing slash.
"/girl" (no slash) returns 404;
"/girl/" (with slash) renders the page.
Also the lines in app.yaml should be:
handlers:
- url: /girl/.*
script: girl.app

App.yaml on app engine is not forwarding wildcards properly

I tried recently to enable html5mode browsing in angular with
// use the HTML5 History API
$locationProvider.html5Mode(true);
Suddenly though, when I try to go to one of my client side routes, app engine seems not to even forward my request to my webapp2 handler.
For example, localhost:8080/#/myRoute routes properly when entered directly but localhost:8080/myRoute gives me a 404.
I recognize # indicates a client side route, but as long as my request is forwarded to my index.html angular should handle the request. I didn't explicitly create a route for /#/ so it seems like it's handling those as wildcards but not other routes.
I tried these routes in my app.yaml:
- url: /
script: server.web_server.main.app
- url: .*
script: server.web_server.main.app
- url: /.*
script: server.web_server.main.app
My server.web_server.main.app routing setup looks like:
app = webapp2.WSGIApplication([
# Main SPA handler
webapp2.Route('/', MainHandler, name='main'),
], debug=True)
MainHandler never even sees a request when I don't use # and the server logs indicate indeed they did have a 404. I could understand if both # and nothing caused a 404 then my wildcards would not be working properly, but why would /#/ work and / not work if I haven't put any special routing for /#/?
Can someone explain what I'm doing wrong?
Thanks in advance!
-- Jp.

MainHandler isn't seeing a request for /myRoute because only / is being routed (within the app) to that handler, even though your app.yaml is routing requests to the app.
Try adding something like
webapp2.Route('/myRoute', MainHandler, name='myroutemain'),
to the WSGIApplication.

So the post above is totally correct, but I thought I would add an even simpler solution solution which doesn't require you to hand specify each client side route.
You can add a wildcard regex by doing the following:
class MainHandler(BaseHandler):
def get(self, opt=None):
self.render("index.html", {"title": "My Angular App"})
app = webapp2.WSGIApplication([
# Main SPA handler
webapp2.Route(r'/<:.*>', MainHandler, name='main')
], debug=True)
I didn't quite have the syntax right for the regex when I tried originally. The captured value gets passed to get which you can use if you want.

How do I make bottle work with google app engine?

This is the code,
import webapp2
from framework import bottle
from framework.bottle import route, template, request, error, debug
#route('/')
def root():
return 'hello world'
class MainHandler(webapp2.RequestHandler):
def get(self):
root()
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
All the dependencies are there (framework, bottle, etc), however, when I deploy it using GAE, I just get an empty page!
Also I tried these and none of them worked, perhaps GAE changed its settings:
http://blog.rutwick.com/use-bottle-python-framework-with-google-app-engine
problems deploying bottle application with google app engine

You have not followed the advice in those links. Most obviously, you are simply calling root without actually returning its result back as the response. In Python, you need to explicitly use return to send a value back from a function, which you don't do in get.
You are also hopelessly confused with setting up the handlers. If you're using bottle, use it: there's no need to have webapp in the mix as well. Webapp is an alternative to bottle, not something that is baked into GAE. Your links show exactly how to do this.

Another solution that worked perfectly for me: https://github.com/GoogleCloudPlatform/appengine-bottle-skeleton

Be aware to use:
app.run(server='gae')
Otherwise bottle will try to access your system and GAE will fail

The Bottle class implements WSGI which works for GAE if you make it a global variable in your main script handler.
main.py
from bottle import default_app, route, run
#route('/')
def root():
return 'hello world'
app = default_app()
app.yaml
runtime: python39
handlers:
- url: .*
script: main.app
requirements.txt
bottle
And that's it!
I tested this on Google Cloud Shell with Bottle 0.12.19 and deploying to Google App Engine Standard.

Configuring Script Handlers Google App Engine

I am trying to make a simple application using Google App Engine.
Below is my code
helloworld.py
print "hello"
class helloworld():
def myfunc(self):
st = "inside class"
return st
test.py
import helloworld
hw_object = helloworld.helloworld()
print hw_object.myfunc()
app.yaml
handlers:
- url: /.*
script: helloworld.py
- url: /.*
script: test.py
When I run my application via http://localhost:10000 it prints only hello whereas my expected output is hello and inside class.
My directory structure
E:\helloworld>dir
app.yaml helloworld.py test.py
I am pretty sure this has something to do with Script Handlers.So, what is the correct way to define handlers and what is wrong in my way of defining them.

When your first handler pattern /.* matches http://localhost:10000, the remaining handlers are all ignored.
You can updated your app.yaml
handlers:
- url: /hello
script: helloworld.py
- url: /test
script: test.py
And browse http://localhost:10000/test

Please walk through the Getting Started guide from the appengine documentation. It will help you get through the initial setup problems like this.
http://code.google.com/appengine/docs/python/gettingstarted/helloworld.html
Here is the sample Handler from that documentation.
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class MainPage(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, webapp World!')
application = webapp.WSGIApplication(
[('/', MainPage)],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
Note that the class extends webapp.RequestHandler, the method name is get (or post if you are responding to a http post request) Also the extra code at the bottom for setting up the application. You can add extra URL's to the application by adding arguments to the WSGIApplication. For example:
application = webapp.WSGIApplication(
[('/', MainPage)],
[('/help/', HelpPage)],
debug=True)
Also note that in your app.yaml as both scripts refer to the same url pattern, there is no way that any request will ever get to test.py. The normal pattern is to have specific url patterns at the top and a catch-all patter last.
Good Luck.

I had a similar problem too. Expanding on Hamish's answer, and correcting the last part where the square brackets are:
application = webapp.WSGIApplication([
('/', MainPage),
('/help/', HelpPage)],
debug=True)
Reference:
https://webapp-improved.appspot.com/guide/routing.html
** Edit I also had an extra closing bracket in my code above. Changed that now.