Develop Reference

Coinbase API standard python example returns "invalid signature"

I am trying to use the Coinbase API with Python, starting of from the standard example on the Coinbase Developer API page: https://developers.coinbase.com/docs/wallet/api-key-authentication#
This piece of code gave me only errors, like:
TypeError: key: expected bytes or bytearray, but got 'str'
After some googling I made some adjustments that got me a little further, but I'm still not there. Could it be that the API documentation page gives an outdated example?
The main message that is returned to me now is this: {"id":"authentication_error","message":"invalid signature"}
I am using the following code (replacing my keys with xxxxxx):
import json, hmac, hashlib, time, requests
from requests.auth import AuthBase
# Before implementation, set environmental variables with the names API_KEY and API_SECRET
API_KEY = 'xxxxxxx'
API_SECRET = b'xxxxxxx'
def get_timestamp():
return int(time.time() * 1000)
# Create custom authentication for Coinbase API
class CoinbaseWalletAuth(AuthBase):
def __init__(self, api_key, secret_key):
self.api_key = api_key
self.secret_key = secret_key
def __call__(self, request):
timestamp = str(int(time.time()))
print(timestamp)
message = timestamp + request.method + request.path_url + (request.body or b'').decode()
hmac_key = base64.b64decode(self.secret_key)
signature = hmac.new(hmac_key, message.encode(), hashlib.sha256)
signature_b64 = base64.b64encode(signature.digest()).decode()
request.headers.update({
'CB-ACCESS-SIGN': signature_b64,
'CB-ACCESS-TIMESTAMP': timestamp,
'CB-ACCESS-KEY': self.api_key,
'Content-Type': 'application/json'
})
return request
api_url = 'https://api.coinbase.com/v2/'
auth = CoinbaseWalletAuth(API_KEY, API_SECRET)
# Get current user
r = requests.get(api_url + 'user', auth=auth)
print(r.json())
# {u'data': {u'username': None, u'resource': u'user', u'name': u'User'...
Hope you can help me fix this. It seems like this is just getting the basics right...

The first thing that is important to mention, the code in their page that you linked is python2 and if you run their code with it it works off the bat.
I had the same issue today and realized that your code has multiple problems (because I assume you tried everything to fix it) so I fixed those for you and that the requests library in python3 or the API as of today (04/28/21) has an issue. I came up with a fix.
import json, hmac, hashlib, time, requests
from requests.auth import AuthBase
# Before implementation, set environmental variables with the names API_KEY and API_SECRET
API_KEY = 'xxxxxxx'
API_SECRET = 'xxxxxxx'
# Create custom authentication for Coinbase API
class CoinbaseWalletAuth(AuthBase):
def __init__(self, api_key, secret_key):
self.api_key = api_key
self.secret_key = secret_key
def __call__(self, request):
timestamp = str(int(time.time()))
# the following try statement will fix the bug
try:
body = request.body.decode()
if body == "{}":
request.body = b""
body = ''
except AttributeError:
request.body = b""
body = ''
message = timestamp + request.method + request.path_url + body
signature = hmac.new(self.secret_key.encode(), message.encode(), hashlib.sha256).hexdigest()
request.headers.update({
'CB-ACCESS-SIGN': signature,
'CB-ACCESS-TIMESTAMP': timestamp,
'CB-ACCESS-KEY': self.api_key,
})
return request
api_url = 'https://api.coinbase.com/v2/'
auth = CoinbaseWalletAuth(API_KEY, API_SECRET)
# Get current user
r = requests.get(api_url + 'user', auth=auth)
print(r.json())
# {u'data': {u'username': None, u'resource': u'user', u'name': u'User'...
now save this as test.py and run with python3 test.py, given that you have the libraries installed in your environment/system.
as of why there is a bug?
troubleshooting I found out that the request.body must be empty for the API to like it (by empty I mean '') as it expects no body for that specific endpoint. This is an assumption about the request library, when you either specify content type set to JSON or use the json= in the call and it sees that you have an empty body it will automatically populate the body with {}. This makes sense since you don't not want to troll the server by saying you are going to send json but did not send anything, so request in python3 is just being nice to the server. In python2 the request library does not automatically do that. This becomes an issue for us since the API likes the body completely empty if the endpoint does not take a body.
Why this fixes it?
I have no idea, as this is not a sign authentication issue. I checked both the working python2 version and python3 version and the signatures come out the same, therefore not an authentication issue. It seems like the server is not flexible with their request body being an empty JSON object.
So as a recap to solve this we must clear the body of the request when we see the body set to {}.
TLDR:
The server misleads into believing it's a sign/auth issue when in reality is a bad request. look at the code.

requests-mock: how can I match POSTed payload in a mocked endpoint

What I've Done
I've written an authentication class for obtaining an application's bearer token from Twitter using the application's API Key and its API key secret as demonstrated in the Twitter developer docs.
I've mocked the appropriate endpoint using requests_mock this way:
#pytest.fixture
def mock_post_bearer_token_endpoint(
requests_mock, basic_auth_string, bearer_token
):
requests_mock.post(
"https://api.twitter.com/oauth2/token",
request_headers={
"Authorization": f"Basic {basic_auth_string}",
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
},
json={"token_type": "bearer", "access_token": f"{bearer_token}"},
)
And my test method is :
#pytest.mark.usefixtures("mock_post_bearer_token_endpoint")
def test_basic_auth(api_key, api_key_secret, bearer_token):
response = requests.post(
'https://api.twitter.com/oauth2/token',
data={"grant_type": "client_credentials"},
auth=TwitterBasicAuth(api_key, api_key_secret),
)
assert response.json()['access_token'] == bearer_token
(Where TwitterBasicAuth is the authentication class I wrote, and the fixture basic_auth_string is a hardcoded string that would be obtained from transforming the fixtures api_key and api_key_secret appropriately).
And it works.
The Problem
But I'm really bothered by the fact that the mocked endpoint doesn't check the payload. In this particular case, the payload is vital to obtain a bearer token.
I've combed through the documentation for requests_mock (and responses, too) but haven't figured out how to make the endpoint respond with a bearer token only when the correct payload is POSTed.
Please help.

I think the misconception here is that you need to put everything in the matcher and let NoMatchException be the thing to tell you if you got it right.
The matcher can be the simplest thing it needs to be in order to return the right response and then you can do all the request/response checking as part of your normal unit test handling.
additional_matchers is useful if you need to switch the response value based on the body of the request for example, and typically true/false is sufficient there.
eg, and i made no attempt to look up twitter auth for this:
import requests
import requests_mock
class TwitterBasicAuth(requests.auth.AuthBase):
def __init__(self, api_key, api_key_secret):
self.api_key = api_key
self.api_key_secret = api_key_secret
def __call__(self, r):
r.headers['x-api-key'] = self.api_key
r.headers['x-api-key-secret'] = self.api_key_secret
return r
with requests_mock.mock() as m:
api_key = 'test'
api_key_secret = 'val'
m.post(
"https://api.twitter.com/oauth2/token",
json={"token_type": "bearer", "access_token": "token"},
)
response = requests.post(
'https://api.twitter.com/oauth2/token',
data={"grant_type": "client_credentials"},
auth=TwitterBasicAuth(api_key, api_key_secret),
)
assert response.json()['token_type'] == "bearer"
assert response.json()['access_token'] == "token"
assert m.last_request.headers['x-api-key'] == api_key
assert m.last_request.headers['x-api-key-secret'] == api_key_secret
https://requests-mock.readthedocs.io/en/latest/history.html

Updated Answer
I went with gold_cy's comment and wrote a custom matcher that takes a request and returns an appropriately crafted OK response if the request has the correct url path, headers and json payload. It returns a 403 response otherwise, as I'd expect from the Twitter API.
#pytest.fixture
def mock_post_bearer_token_endpoint(
requests_mock, basic_auth_string, bearer_token
):
def matcher(req):
if req.path != "/oauth2/token":
# no mock address
return None
if req.headers.get("Authorization") != f"Basic {basic_auth_string}":
return create_forbidden_response()
if (
req.headers.get("Content-Type")
!= "application/x-www-form-urlencoded;charset=UTF-8"
):
return create_forbidden_response()
if req.json().get("grant_type") != "client_credentials":
return create_forbidden_response()
resp = requests.Response()
resp._content = json.dumps(
{"token_type": "bearer", "access_token": f"{bearer_token}"}
).encode()
resp.status_code = 200
return resp
requests_mock._adapter.add_matcher(matcher)
yield
def create_forbidden_response():
resp = requests.Response()
resp.status_code = 403
return resp
Older Answer
I went with gold_cy's comment and wrote an additional matcher that takes the request and checks for the presence of the data of interest in the payload.
#pytest.fixture(name="mock_post_bearer_token_endpoint")
def fixture_mock_post_bearer_token_endpoint(
requests_mock, basic_auth_string, bearer_token
):
def match_grant_type_in_payload(request):
if request.json().get("grant_type") == "client_credentials":
return True
resp = Response()
resp.status_code = 403
resp.raise_for_status()
requests_mock.post(
"https://api.twitter.com/oauth2/token",
request_headers={
"Authorization": f"Basic {basic_auth_string}",
"Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
},
json={"token_type": "bearer", "access_token": f"{bearer_token}"},
additional_matcher=match_grant_type_in_payload,
)
I opted to raise an Http403 error (instead of just returning False) in order to reduce the cognitive load of determining the reason exceptions are raised — returning False would lead to a requests_mock.exceptions.NoMockAddress being raised, which I don't think is descriptive enough in this case.
I still think there's a better way around this, and I'll keep searching for it.

Python requests parameters not going through

I'm trying to make a post request to Quizlet following their OAuth flow from these instructions https://quizlet.com/api/2.0/docs/authorization-code-flow. I'm running into a problem where on Step 2, I have to make a post request with a token I generated from their server, but I'm not having success passing in the token to the url. I know it was generated correctly, but I'm having trouble passing it in and not getting a 400 response.
More directly, my question is, is there another way of including the grant_type and code parameters that I'm trying to pass in through the url in the post request such as passing them in through the header of the post request? I've looked at the documentation for requests but I've had no luck.
#app.route('/')
#app.route('/index')
def index():
code = request.args.get('code')
state = request.args.get('state')
print("code is " + code)
r = requests.post("https://api.quizlet.com/oauth/token?grant_type=authorization_code&code=" + code)
return render_template('index.html')

You must specify the required headers Authorization, Content-Type.
import requests
from requests.auth import _basic_auth_str
client_id = 'YOUR CLIENT ID'
secret = 'YOUR CLIENT SECRET'
code = 'CODE FROM STEP 1'
headers = {
'Authorization': _basic_auth_str(client_id, secret),
'Content-Type': 'application/x-www-form-urlencoded'
}
r = requests.post('https://api.quizlet.com/oauth/token?grant_type=authorization_code&code={0}'.format(
code), headers=headers)
print r.status_code
print r.content

Using Python to test HTTP APIs

I'm fairly new to Python programming and I don't know all the libraries needed for the following.
I would like to use Python to test some HTTP APIs. Mainly I want to use OAuth and make a few JSON calls. The APIs in question can be found on: https://developers.trustpilot.com/authentication and the generate product review link (I can only use one link)
I want to authenticate myself and then generate a product review link in one step. So far I've been using the Advanced REST client (ARC) to make these calls individually. I could also use .arc files if you think it's easier.
The idea would be make these calls successively in one go. So it would be something along the lines:
1) Make the authentication call.
The HTTP Method looks like this:
https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken
Method Post:
Header
Authorization: Basic Base64encode(APIkey:Secret)
Content-Type: application/x-www-form-urlencoded
Payload:
grant_type=password&username=user#mail.com&password=SomePass
Translate this bit into Python basically.
1.a) Add a header to the call
Header Authorization: base64encode hash Content-Type: application/x-www-form-urlencoded
1.b) Add a payload to the call
Payload: grant_type=password&username
4) Receive the token from call made in step 1) (Result is format)
"access token": Auth_token
5) Take the token and use it in creating a product review.
5.a) Add the token in the header
Header: Authorization: Bearer Auth_token
6.a) Add a JSON payload to the call made in step 5.
Here's the code I have so far:
Import requests
header = {'Authorization: Basic NnNrQUprTWRHTU5VSXJGYXBVRGxack1oT01oTUFRZHI6QTFvOGJjRUNDdUxBTmVqUQ==}','Content-Type: application/x-www-form-urlencoded'}
payload = {'grant_type=password&username=email#address.com&password=SomePassword'}
r = requests.post('https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken', headers=header, params=payload )
Ideally I want to create the requests.post(url, header, payload) and then return what the server answers in JSON format. I think that print r.text would do the last part.
So this is the code I have writtent (that works now):
import requests
import getpass
import json
from requests.auth import HTTPBasicAuth
header = {'grant_type':'password' , 'username':'mail#maildomain.com', 'password':'YourPassword'}
username= "YOURAPIKEY" #APIKey
password= "YOURSECRET" #Secret
res = requests.post(
'URL/v1/oauth/oauth-business-users-for-applications/accesstoken',
auth=HTTPBasicAuth(username, password), # basic authentication
data=header)
#print(res.content) #See content of the call result.
data = res.json() # get response as parsed json (will return a dict)
auth_token = data.get('access_token')

requests can do all what you ask without any work from your part.
See the doc for authentication, parameters, json output, json input
Make the authentication call.
import requests
import getpass
from requests.auth import HTTPBasicAuth
username = raw_input('Username: ')
password = getpass.getpass('Password: ')
res = requests.post(
'https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken',
auth=HTTPBasicAuth(username, password), # basic authentication
params={ # url parameters
'grant_type': 'password',
'username': 'email#address.com',
'password': 'SomePassword'
})
Receive the token from call made in step 1) (Result is format)
# res = requests.post.....
data = res.json() # get response as parsed json (will return a dict)
auth_token = data.get('access token')
Take the token and use it in creating a product review.
request.post(
'.../product_review',
headers={
'Authorization': 'Bearer ' + auth_token
},
json={'my': 'payload'}) # send data as json

How to use a refresh_token to get a new access_token (using Flask-OAuthLib)?

I'm building a website + backend with the FLask Framework in which I use Flask-OAuthlib to authenticate with google. After authentication, the backend needs to regularly scan the user his Gmail. So currently users can authenticate my app and I store the access_token and the refresh_token. The access_token expires after one hour, so within that one hour I can get the userinfo like so:
google = oauthManager.remote_app(
'google',
consumer_key='xxxxxxxxx.apps.googleusercontent.com',
consumer_secret='xxxxxxxxx',
request_token_params={
'scope': ['https://www.googleapis.com/auth/userinfo.email', 'https://www.googleapis.com/auth/gmail.readonly'],
'access_type': 'offline'
},
base_url='https://www.googleapis.com/oauth2/v1/',
request_token_url=None,
access_token_method='POST',
access_token_url='https://accounts.google.com/o/oauth2/token',
authorize_url='https://accounts.google.com/o/oauth2/auth'
)
token = (the_stored_access_token, '')
userinfoObj = google.get('userinfo', token=token).data
userinfoObj['id'] # Prints out my google id
Once the hour is over, I need to use the refresh_token (which I've got stored in my database) to request a new access_token. I tried replacing the_stored_access_token with the_stored_refresh_token, but this simply gives me an Invalid Credentials-error.
In this github issue I read the following:
regardless of how you obtained the access token / refresh token (whether through an authorization code grant or resource owner password credentials), you exchange them the same way, by passing the refresh token as refresh_token and grant_type set to 'refresh_token'.
From this I understood I had to create a remote app like so:
google = oauthManager.remote_app(
'google',
# also the consumer_key, secret, request_token_params, etc..
grant_type='refresh_token',
refresh_token=u'1/xK_ZIeFn9quwvk4t5VRtE2oYe5yxkRDbP9BQ99NcJT0'
)
But this leads to a TypeError: __init__() got an unexpected keyword argument 'refresh_token'. So from here I'm kinda lost.
Does anybody know how I can use the refresh_token to get a new access_token? All tips are welcome!

This is how I get a new access_token for google:
from urllib2 import Request, urlopen, URLError
from webapp2_extras import json
import mimetools
BOUNDARY = mimetools.choose_boundary()
def refresh_token()
url = google_config['access_token_url']
headers = [
("grant_type", "refresh_token"),
("client_id", <client_id>),
("client_secret", <client_secret>),
("refresh_token", <refresh_token>),
]
files = []
edata = EncodeMultiPart(headers, files, file_type='text/plain')
headers = {}
request = Request(url, headers=headers)
request.add_data(edata)
request.add_header('Content-Length', str(len(edata)))
request.add_header('Content-Type', 'multipart/form-data;boundary=%s' % BOUNDARY)
try:
response = urlopen(request).read()
response = json.decode(response)
except URLError, e:
...
EncodeMultipart function is taken from here:
https://developers.google.com/cloud-print/docs/pythonCode
Be sure to use the same BOUNDARY

Looking at the source code for OAuthRemoteApp. The constructor does not take a keyword argument called refresh_token. It does however take an argument called access_token_params which is an optional dictionary of parameters to forward to the access token url.
Since the url is the same, but the grant type is different. I imagine a call like this should work:
google = oauthManager.remote_app(
'google',
# also the consumer_key, secret, request_token_params, etc..
grant_type='refresh_token',
access_token_params = {
refresh_token=u'1/xK_ZIeFn9quwvk4t5VRtE2oYe5yxkRDbP9BQ99NcJT0'
}
)

flask-oauthlib.contrib contains an parameter named auto_refresh_url / refresh_token_url in the remote_app which does exactely what you wanted to wanted to do. An example how to use it looks like this:
app= oauth.remote_app(
[...]
refresh_token_url='https://www.douban.com/service/auth2/token',
authorization_url='https://www.douban.com/service/auth2/auth',
[...]
)
However I did not manage to get it running this way. Nevertheless this is possible without the contrib package. My solution was to catch 401 API calls and redirect to a refresh page if a refresh_token is available.
My code for the refresh endpoint looks as follows:
#app.route('/refresh/')
def refresh():
data = {}
data['grant_type'] = 'refresh_token'
data['refresh_token'] = session['refresh_token'][0]
data['client_id'] = CLIENT_ID
data['client_secret'] = CLIENT_SECRET
# make custom POST request to get the new token pair
resp = remote.post(remote.access_token_url, data=data)
# checks the response status and parses the new tokens
# if refresh failed will redirect to login
parse_authorized_response(resp)
return redirect('/')
def parse_authorized_response(resp):
if resp is None:
return 'Access denied: reason=%s error=%s' % (
request.args['error_reason'],
request.args['error_description']
)
if isinstance(resp, dict):
session['access_token'] = (resp['access_token'], '')
session['refresh_token'] = (resp['refresh_token'], '')
elif isinstance(resp, OAuthResponse):
print(resp.status)
if resp.status != 200:
session['access_token'] = None
session['refresh_token'] = None
return redirect(url_for('login'))
else:
session['access_token'] = (resp.data['access_token'], '')
session['refresh_token'] = (resp.data['refresh_token'], '')
else:
raise Exception()
return redirect('/')
Hope this will help. The code can be enhanced of course and there surely is a more elegant way than catching 401ers but it's a start ;)
One other thing: Do not store the tokens in the Flask Session Cookie. Rather use Server Side Sessions from "Flask Session" which I did in my code!

This is how i got my new access token.
from urllib2 import Request, urlopen, URLError
import json
import mimetools
BOUNDARY = mimetools.choose_boundary()
CRLF = '\r\n'
def EncodeMultiPart(fields, files, file_type='application/xml'):
"""Encodes list of parameters and files for HTTP multipart format.
Args:
fields: list of tuples containing name and value of parameters.
files: list of tuples containing param name, filename, and file contents.
file_type: string if file type different than application/xml.
Returns:
A string to be sent as data for the HTTP post request.
"""
lines = []
for (key, value) in fields:
lines.append('--' + BOUNDARY)
lines.append('Content-Disposition: form-data; name="%s"' % key)
lines.append('') # blank line
lines.append(value)
for (key, filename, value) in files:
lines.append('--' + BOUNDARY)
lines.append(
'Content-Disposition: form-data; name="%s"; filename="%s"'
% (key, filename))
lines.append('Content-Type: %s' % file_type)
lines.append('') # blank line
lines.append(value)
lines.append('--' + BOUNDARY + '--')
lines.append('') # blank line
return CRLF.join(lines)
def refresh_token():
url = "https://oauth2.googleapis.com/token"
headers = [
("grant_type", "refresh_token"),
("client_id", "xxxxxx"),
("client_secret", "xxxxxx"),
("refresh_token", "xxxxx"),
]
files = []
edata = EncodeMultiPart(headers, files, file_type='text/plain')
#print(EncodeMultiPart(headers, files, file_type='text/plain'))
headers = {}
request = Request(url, headers=headers)
request.add_data(edata)
request.add_header('Content-Length', str(len(edata)))
request.add_header('Content-Type', 'multipart/form-data;boundary=%s' % BOUNDARY)
response = urlopen(request).read()
print(response)
refresh_token()
#response = json.decode(response)
#print(refresh_token())

With your refresh_token, you can get a new access_token like:
from google.oauth2.credentials import Credentials
from google.auth.transport import requests
creds = {"refresh_token": "<goes here>",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"client_id": "<YOUR_CLIENT_ID>.apps.googleusercontent.com",
"client_secret": "<goes here>",
"scopes": ["https://www.googleapis.com/auth/userinfo.email"]}
cred = Credentials.from_authorized_user_info(creds)
cred.refresh(requests.Request())
my_new_access_token = cred.token