Develop Reference

Generate the AWS HTTP signature from boto3

I am working with the AWS Transcribe streaming service that boto3 does not support yet, so to make HTTP/2 requests, I need to manually setup the authorization header with the "AWS Signature Version 4"
I've found some example implementation, but I was hoping to just call whatever function boto3/botocore have implemented using the same configuration object.
Something like
session = boto3.Session(...)
auth = session.generate_signature('POST', '/stream-transcription', ...)
Any pointers in that direction?

Contrary to the AWS SDKs for most other programming languages, boto3/botocore don't offer the functionality to sign arbitrary requests using "AWS Signature Version 4" yet. However there is at least already an open feature request for that: https://github.com/boto/botocore/issues/1784
In this feature request, existing alternatives are discussed as well. One is the third-party Python library aws-requests-auth, which provides a thin wrapper around botocore and requests to sign HTTP-requests. That looks like the following:
import requests
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth
auth = BotoAWSRequestsAuth(aws_host="your-service.domain.tld",
aws_region="us-east-1",
aws_service="execute-api")
response = requests.get("https://your-service.domain.tld",
auth=auth)
Another alternative presented in the feature request is to implement the necessary glue-code on your own, as shown in the following gist: https://gist.github.com/rhboyd/1e01190a6b27ca4ba817bf272d5a5f9a.

Did you check this SDK? Seems very recent but might do what you need.
https://github.com/awslabs/amazon-transcribe-streaming-sdk/tree/master
It looks like it handles the signing: https://github.com/awslabs/amazon-transcribe-streaming-sdk/blob/master/amazon_transcribe/signer.py

I have not tested this, but you can likely accomplish this by following along with with this SigV4 unit test:
https://github.com/boto/botocore/blob/master/tests/unit/test_auth_sigv4.py
Note, this constructs a request using the botocore.awsrequest.AWSRequest helper. You'll likely need to dig around to figure out how to send the actual HTTP request (perhaps with httpsession.py)

How to use google python oauth libraries to implement OpenID Connect?

I am evaluating different options for authentication in a python App Engine flex environment, for apps that run within a G Suite domain.
I am trying to put together the OpenID Connect "Server flow" instructions here with how google-auth-library-python implements the general OAuth2 instructions here.
I kind of follow things up until 4. Exchange code for access token and ID token, which looks like flow.fetch_token, except it says "response to this request contains the following fields in a JSON array," and it includes not just the access token but the id token and other things. I did see this patch to the library. Does that mean I could use some flow.fetch_token to create an IDTokenCredentials (how?) and then use this to build an OpenID Connect API client (and where is that API documented)? And what about validating the id token, is there a separate python library to help with that or is that part of the API library?
It is all very confusing. A great deal would be cleared up with some actual "soup to nuts" example code but I haven't found anything anywhere on the internet, which makes me think (a) perhaps this is not a viable way to do authentication, or (b) it is so recent the python libraries have not caught up? I would however much rather do authentication on the server than in the client with Google Sign-In.
Any suggestions or links to code are much appreciated.

It seems Google's python library contains a module for id token validation. This can be found at google.oauth2.id_token module. Once validated, it will return the decoded token which you can use to obtain user information.
from google.oauth2 import id_token
from google.auth.transport import requests
request = requests.Request()
id_info = id_token.verify_oauth2_token(
token, request, 'my-client-id.example.com')
if id_info['iss'] != 'https://accounts.google.com':
raise ValueError('Wrong issuer.')
userid = id_info['sub']
Once you obtain user information, you should follow authentication process as described in Authenticate the user section.

OK, I think I found my answer in the source code now.
google.oauth2.credentials.Credentials exposes id_token:
Depending on the authorization server and the scopes requested, this may be populated when credentials are obtained and updated when refresh is called. This token is a JWT. It can be verified and decoded [as #kavindu-dodanduwa pointed out] using google.oauth2.id_token.verify_oauth2_token.
And several layers down the call stack we can see fetch_token does some minimal validation of the response JSON (checking that an access token was returned, etc.) but basically passes through whatever it gets from the token endpoint, including (i.e. if an OpenID Connect scope is included) the id token as a JWT.
EDIT:
And the final piece of the puzzle is the translation of tokens from the (generic) OAuthSession to (Google-specific) credentials in google_auth_oauthlib.helpers, where the id_token is grabbed, if it exists.
Note that the generic oauthlib library does seem to implement OpenID Connect now, but looks to be very recent and in process (July 2018). Google doesn't seem to use any of this at the moment (this threw me off a bit).

What is Google API Discovery?

I am having trouble understanding the concept of “API discovery” as used in Google products/services. Here’s some Python code that uses the said discovery service to access Google Cloud Vision:
from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials
…
API_DISCOVERY_FILE = 'https://vision.googleapis.com/$discovery/rest?version=v1'
hlh = httplib2.Http()
credentials = GoogleCredentials.get_application_default().create_scoped(
['https://www.googleapis.com/auth/cloud-platform'])
credentials.authorize(hlh)
service = build(serviceName='vision', version='v1', http=hlh, discoveryServiceUrl=API_DISCOVERY_FILE)
service_request = service.images().annotate(body={ <more JSON code here> })
Here’s another bit of Python code that also accesses Google Cloud Vision, but does not use API discovery and works just fine:
import requests
…
ENDPOINT_URL = 'https://vision.googleapis.com/v1/images:annotate'
response = requests.post(ENDPOINT_URL,
data=make_image_data(image_filenames),
params={'key': api_key},
headers={'Content-Type': 'application/json'})
What I can’t wrap my head around is this question: You need to know the details of the API that you are going to be calling so that you can tailor the call; this is obvious. So, how would API discovery help you at the time of the call, after you have already prepared the code for calling that API?
PS: I did look at the following resources prior to posting this question:
https://developers.google.com/discovery/v1/getting_started
https://developers.google.com/discovery/v1/using
I did see this answered question but would appreciate additional insight.

Note: I lack the knowledge of the Google API you are mentioning.
You need to know the details of the API that you are going to be
calling so that you can tailor the call; this is obvious.
In theory to me this is only required for the very first call, which would be the call to a starting service which would for instance list a number of resources. From there on you can have branches to underlying resources and their allowed methods (verbs if you will). So this example illustrated a tree like structure. If you provide a GUI to navigate through this discoverability in a generic way, a person would then be able to decide what to do.
In practice this is kind of hard to do once you have vast amounts of resources and all sorts of interrelationships between them.

Is there a better way to access my public api?

I am new to Flask.
I have a public api, call it api.example.com.
#app.route('/api')
def api():
name = request.args.get('name')
...
return jsonify({'address':'100 Main'})
I am building an app on top of my public api (call it www.coolapp.com), so in another app I have:
#app.route('/make_request')
def index():
params = {'name':'Fred'}
r = requests.get('http://api.example.com', params=params)
return render_template('really_cool.jinja2',address=r.text)
Both api.example.com and www.coolapp.com are hosted on the same server. It seems inefficient the way I have it (hitting the http server when I could access the api directly). Is there a more efficient way for coolapp to access the api and still be able to pass in the params that api needs?

Ultimately, with an API powered system, it's best to hit the API because:
It's user testing the API (even though you're the user, it's what others still access);
You can then scale easily - put a pool of API boxes behind a load balancer if you get big.
However, if you're developing on the same box you could make a virtual server that listens on localhost on a random port (1982) and then forwards all traffic to your api code.
To make this easier I'd abstract the API_URL into a setting in your settings.py (or whatever you are loading in to Flask) and use:
r = requests.get(app.config['API_URL'], params=params)
This will allow you to make a single change if you find using this localhost method isn't for you or you have to move off one box.
Edit
Looking at your comments you are hoping to hit the Python function directly. I don't recommend doing this (for the reasons above - using the API itself is better). I can also see an issue if you did want to do this.
First of all we have to make sure the api package is in your PYTHONPATH. Easy to do, especially if you're using virtualenvs.
We from api import views and replace our code to have r = views.api() so that it calls our api() function.
Our api() function will fail for a couple of reasons:
It uses the flask.request to extract the GET arg 'name'. Because we haven't made a request with the flask WSGI we will not have a request to use.
Even if we did manage to pass the request from the front end through to the API the second problem we have is using the jsonify({'address':'100 Main'}). This returns a Response object with an application type set for JSON (not just the JSON itself).
You would have to completely rewrite your function to take into account the Response object and handle it correctly. A real pain if you do decide to go back to an API system again...

Depending on how you structure your code, your database access, and your functions, you can simply turn the other app into package, import the relevant modules and call the functions directly.
You can find more information on modules and packages here.
Please note that, as Ewan mentioned, there's some advantages to using the API. I would advise you to use requests until you actually need faster requests (this is probably premature optimization).
Another idea that might be worth considering, depending on your particular code, is creating a library that is used by both applications.

Is there a working OAuth library for Python 3?

What's the most current form of Oauth for Python 3?
I'm trying to create a stock screener using my broker's API, which uses Oauth. Most of the info I find is out of date or conflicting. I've seen the following modules referenced:
Oauth - Seems to be the original, now outdated. I get an error of "'module' object has no attribute 'Consumer'"
Oauth2 - Newer version, apparently also outdated? The one most referenced one online. Glitches out in pip/can't figure out how to install it.
Oauthlib - IIRC, claims to be the new replacement for Oauth and Oauth2
Rauth.OAuth2Service - Also potentially replacement for Oauth and Oauth2?
Requests - ?
Oauth_hook - ?
pyoauth2 - I get an error about not having a module named "client" in pyoauth2's init.
None of them seem to work as expected, and I have a feeling that this is due to low Oauth 3 support. Have you gotten OAuth to work in Python 3? If so, how did you do it?

It looks like Requets_oauthlib works. Here's code I used that works in Python 3. I'm posting it because most of the example code I found used formats that I couldn't get working.
from requests_oauthlib import OAuth1
client_key = ''
client_secret = ''
resource_owner_key = ''
resource_owner_secret = ''
def query(queryurl):
headeroauth = OAuth1(client_key, client_secret, resource_owner_key,
resource_owner_secret, signature_type = 'auth_header')
return requests.get(queryurl, auth = headeroauth)
query('http://website.com')

Author of rauth here: rauth is a client library which currently does not officially support Python 3.
However, we are working on it, and there's an active branch (aptly named "python-3") over at GitHub which works. You're free to use it, but bear in mind that things may change slightly when we officially release support for it later on. With that said, it would be great to have people out in the real world testing it so that we can make changes to accommodate the Python 3 ecosystem.
Also note: oauthlib is not a replacement for rauth and not a client library. It attempts to be a generic solution, much like python-oauth2 was, but it doesn't provide a client, unlike python-oauth2.