Develop Reference

Implementing Google Directory API users watch with Python

I'm having some trouble understanding and implementing the Google Directory API's users watch function and push notification system (https://developers.google.com/admin-sdk/reports/v1/guides/push#creating-notification-channels) in my Python GAE app. What I'm trying to achieve is that any user (admin) who uses my app would be able to watch user changes within his own domain.
I've verified the domain I want to use for notifications and implemented the watch request as follows:
directoryauthdecorator = OAuth2Decorator(
approval_prompt='force',
client_id='my_client_id',
client_secret='my_client_secret',
callback_path='/oauth2callback',
scope=['https://www.googleapis.com/auth/admin.directory.user'])
class PushNotifications(webapp.RequestHandler):
#directoryauthdecorator.oauth_required
def get(self):
auth_http = directoryauthdecorator.http()
service = build("admin", "directory_v1", http=auth_http)
uu_id=str(uuid.uuid4())
param={}
param['customer']='my_customer'
param['event']='add'
param['body']={'type':'web_hook','id':uu_id,'address':'https://my-domain.com/pushNotifications'}
watchUsers = service.users().watch(**param).execute()
application = webapp.WSGIApplication(
[
('/pushNotifications',PushNotifications),
(directoryauthdecorator.callback_path, directoryauthdecorator.callback_handler())],
debug=True)
Now, the receiving part is what I don't understand. When I add a user on my domain and check the app's request logs I see some activity, but there's no usable data. How should I approach this part?
Any help would be appreciated. Thanks.

The problem
It seems like there's been some confusion in implementing the handler. Your handler actually sets up the notifications channel by sending a POST request to the Reports API endpoint. As the docs say:
To set up a notification channel for messages about changes to a particular resource, send a POST request to the watch method for the resource.
source
You should only need to send this request one time to set up the channel, and the "address" parameter should be the URL on your app that will receive the notifications.
Also, it's not clear what is happening with the following code:
param={}
param['customer']='my_customer'
param['event']='add'
Are you just breaking the code in order to post it here? Or is it actually written that way in the file? You should actually preserve, as much as possible, the code that your app is running so that we can reason about it.
The solution
It seems from the docs you linked - in the "Receiving Notifications" section, that you should have code inside the "address" specified to receive notifications that will inspect the POST request body and headers on the notification push request, and then do something with that data (like store it in BigQuery or send an email to the admin, etc.)

Managed to figure it out. In the App Engine logs I noticed that each time I make a change, which is being 'watched', on my domain I get a POST request from Google's API, but with a 302 code. I discovered that this was due to the fact I had login: required configured in my app.yaml for the script, which was handling the requests and the POST request was being redirected to the login page, instead of the processing script.

App.yaml on app engine is not forwarding wildcards properly

I tried recently to enable html5mode browsing in angular with
// use the HTML5 History API
$locationProvider.html5Mode(true);
Suddenly though, when I try to go to one of my client side routes, app engine seems not to even forward my request to my webapp2 handler.
For example, localhost:8080/#/myRoute routes properly when entered directly but localhost:8080/myRoute gives me a 404.
I recognize # indicates a client side route, but as long as my request is forwarded to my index.html angular should handle the request. I didn't explicitly create a route for /#/ so it seems like it's handling those as wildcards but not other routes.
I tried these routes in my app.yaml:
- url: /
script: server.web_server.main.app
- url: .*
script: server.web_server.main.app
- url: /.*
script: server.web_server.main.app
My server.web_server.main.app routing setup looks like:
app = webapp2.WSGIApplication([
# Main SPA handler
webapp2.Route('/', MainHandler, name='main'),
], debug=True)
MainHandler never even sees a request when I don't use # and the server logs indicate indeed they did have a 404. I could understand if both # and nothing caused a 404 then my wildcards would not be working properly, but why would /#/ work and / not work if I haven't put any special routing for /#/?
Can someone explain what I'm doing wrong?
Thanks in advance!
-- Jp.

MainHandler isn't seeing a request for /myRoute because only / is being routed (within the app) to that handler, even though your app.yaml is routing requests to the app.
Try adding something like
webapp2.Route('/myRoute', MainHandler, name='myroutemain'),
to the WSGIApplication.

So the post above is totally correct, but I thought I would add an even simpler solution solution which doesn't require you to hand specify each client side route.
You can add a wildcard regex by doing the following:
class MainHandler(BaseHandler):
def get(self, opt=None):
self.render("index.html", {"title": "My Angular App"})
app = webapp2.WSGIApplication([
# Main SPA handler
webapp2.Route(r'/<:.*>', MainHandler, name='main')
], debug=True)
I didn't quite have the syntax right for the regex when I tried originally. The captured value gets passed to get which you can use if you want.

How do I redirect to the www. version of my Flask site on Heroku?

I've got a Python Flask app running on Heroku (Cedar stack) with two custom domains (one with and one without the www subdomain). I'd like to redirect all incoming requests to the www. version of the resource requested (the inverse of this question). I think I need some WSGI middleware for this but I can't find a good example.
How do I do this?

An easier solution than to create a separate Heroku app would be a before_request function.
from urllib.parse import urlparse, urlunparse
#app.before_request
def redirect_nonwww():
"""Redirect non-www requests to www."""
urlparts = urlparse(request.url)
if urlparts.netloc == 'example.com':
urlparts_list = list(urlparts)
urlparts_list[1] = 'www.example.com'
return redirect(urlunparse(urlparts_list), code=301)
This will redirect all non-www requests to www using a "HTTP 301 Moved Permanently" response.

According to the Heroku Docs, you've got the right idea about using the www subdomain (eg www.foo.com) vs apex domain (eg foo.com). Their suggestion for dealing with this is to use a DNS layer redirect:
To quote:
Subdomain redirection
Subdomain redirection results in a 301 permanent redirect to the
specified subdomain for all requests to the apex domain so all current
and future requests are properly routed and the full www hostname is
displayed in the user’s location field.
Almost all DNS providers offer domain redirection services - sometimes
also called domain forwarding. DNSimple provides a convenient URL
redirect seen here redirecting from the heroku-sslendpoint.com apex
domain to the www.heroku-sslendpoint.com subdomain.
Source:
http://devcenter.heroku.com/articles/avoiding-apex-domains-dns-arecords#subdomain_redirection
Hope that helps!

One possible approach would be to add a function to listen on request_started, and do the appropriate redirection.
This signal is sent before any request processing started but when the
request context was set up. Because the request context is already
bound, the subscriber can access the request with the standard global
proxies such as request.

What I ended up doing was creating a second Heroku app, assigning the non-www hostname to that one and using a catch all Flask route to redirect to the www version keeping the path intact.

Try this:
#app.route('/', methods=['GET', 'POST'], subdomain="www")
#app.route('/<string:path>', methods=['GET', 'POST'], subdomain="www")
def www(path=''):
return redirect(url_for('app.index')+path)

Getting the Server URL in Google App Engine using python

How do I get App Engine to generate the URL of the server it is currently running on?
If the application is running on development server it should return
http://localhost:8080/
and if the application is running on Google's servers it should return
http://application-name.appspot.com

You can get the URL that was used to make the current request from within your webapp handler via self.request.url or you could piece it together using the self.request.environ dict (which you can read about on the WebOb docs - request inherits from webob)
You can't "get the url for the server" itself, as many urls could be used to point to the same instance.
If your aim is really to just discover wether you are in development or production then use:
'Development' in os.environ['SERVER_SOFTWARE']

Here is an alternative answer.
from google.appengine.api import app_identity
server_url = app_identity.get_default_version_hostname()
On the dev appserver this would show:
localhost:8080
and on appengine
your_app_id.appspot.com

If you're using webapp2 as framework chances are that you already using URI routing in you web application.
http://webapp2.readthedocs.io/en/latest/guide/routing.html
app = webapp2.WSGIApplication([
webapp2.Route('/', handler=HomeHandler, name='home'),
])
When building URIs with webapp2.uri_for() just pass _full=True attribute to generate absolute URI including current domain, port and protocol according to current runtime environment.
uri = uri_for('home')
# /
uri = uri_for('home', _full=True)
# http://localhost:8080/
# http://application-name.appspot.com/
# https://application-name.appspot.com/
# http://your-custom-domain.com/
This function can be used in your Python code or directly from templating engine (if you register it) - very handy.
Check webapp2.Router.build() in the API reference for a complete explanation of the parameters used to build URIs.

Pylons - Redirects will drop from HTTPS to HTTP unless I specify the protocol... Is there a way to fix this?

On my Pylons website, I have my login form sending it's data to 'https://mysite.com'. Upon a successful login, a redirect takes place to send them to their profile page.
redirect(url(controller='profile'))
This sends the user to http://mysite.com/profile instead of https://mysite.com/profile. The only way I've found to fix this is to change the redirect to:
redirect(url(controller='profile', protocol='https'))
The problem I have with this is "what if, for whatever reason, my cert goes away and I have to drop SSL" I don't want to have to go through my entire code looking for all redirects I specify the 'https' protocol in. I want my login to send the user to HTTPS and that's it...
Is there a reason the redirect drops to HTTP? Is there a way to stop it? :/

Since I spent a couple of hours wading through the pylons/routes/beaker/etc. source I thought I'd share my solution.
First a bit of context. I'm using an elastic load balancer (ELB) on AWS with SSL termination. The application is built to run solely over https; this is a post-firesheep world after all. It's layered like so:
ELB -> nginx -> pasteWSGI -> pylons
ELB is jolly good in terms of simplicity but any call to pylons.controllers.util.redirect would trigger a 302 Redirect to "http://mysite/". The ELB would not change that on the way back (no reason to) and so my browser would be sent back to port 80 and there is no ELB listening on that port.
I've tried updating the Mapper as suggested above.
it did not work,
I wanted my redirects to be relative. Switching to https in pylons means that the URL generator goes and fetches the host to create a new URL (https://localhost/....)
Note that Mapper.redirect_to works out of the box and uses relative redirects so there is no need to mess with that. The fundamental problem is that controllers.redirect uses a slightly different code path. In particular, in Routes, the controllers.util.redirect is not a redirect (there's an "if routes and routes.redirect" which evals to False).
My solution: replace all calls to redirect by a new controller method (called redirect too) to change redirects from absolute to relative redirects.
The code is as follows:
lib/helpers.py
def relative_redirect(to_url, start_response):
"""Returns a redirect that is compatible with AWS ELB (no absolute http responses)
Using pylons.controllers.util.redirect triggers an exception that'll be turned into a 302
But with an absolute path so the response does not contains https but simple http
"""
start_response("302 Found", [("Content-Type", "text/plain; charset=utf-8"), ("Location", url(to_url))])
return ["You are being redirected to {0}".format(url(to_url))]
With that bit called from the base class of my controllers:
class BaseController(WSGIController):
...
def redirect(self, to_url):
"""Force a relative redirection, to work with AWS ELB"""
return relative_redirect(to_url, self.start_response)

I'd customize the Mapper so that every call to "url" would force the correct protocol...
Inside routing.py:
class CustomMapper(Mapper):
def generate(self, *args, **kwargs):
kwargs["protocol"] = "https"
return Mapper.generate(self, *args, **kwargs)
def make_map(config):
"""Create, configure and return the routes Mapper"""
map = CustomMapper(directory=config['pylons.paths']['controllers'],
always_scan=config['debug'])