I've been asked to deal with an external REST API (Zendesk's, in fact) whose credentials need to be formatted as {email}/token:{security_token} -- a single value rather than the usual username/password pair. I'm trying to use the Python requests module for this task, since it's Pythonic and doesn't hurt my brain too badly, but I'm not sure how to format the authentication credentials. The Zendesk documentation only gives access examples using curl, which I'm unfamiliar with.
Here's how I currently have requests.auth.AuthBase subclassed:
class ZDTokenAuth(requests.auth.AuthBase):
def __init__(self,username,token):
self.username = username
self.token = token
def __call__(self,r):
auth_string = self.username + "/token:" + self.token
auth_string = auth_string.encode('utf-8')
r.headers['Authorization'] = base64.b64encode(auth_string)
return r
I'm not sure that the various encodings are required, but that's how someone did it on github (https://github.com/skipjac/Zendesk-python-api/blob/master/zendesk-ticket-delete.py) so why not. I've tried it without the encoding too, of course - same result.
Here's the class and methods I'm using to test this:
class ZDStats(object):
api_base = "https://mycompany.zendesk.com/api/v2/"
def __init__(self,zd_auth):
self.zd_auth = zd_auth # this is assumed to be a ZDTokenAuth object
def testCredentials(self):
zd_users_url = self.api_base + "users.json"
zdreq = requests.get(zd_users_url, auth=self.zdauth)
return zdreq
This is called with:
credentials = ZDTokenAuth(zd_username,zd_apitoken)
zd = ZDStats(credentials)
users = zd.testCredentials()
print(users.status_code)
print(users.text)
The status code I'm getting back is a 401, and the text is simply {"error":"Couldn't authenticate you."}. Clearly I'm doing something wrong here, but I don't think I know enough to know what it is I'm doing wrong, if that makes sense. Any ideas?
What you're missing is the auth type. Your Authorization header should be created like this:
r.headers['Authorization'] = b"Basic " + base64.b64encode(auth_string)
You can also achieve the same passing by a tuple as auth parameter with:
requests.get(url, auth=(username+"/token", token))
Related
I have looked through the FTX api documentation found here: https://docs.ftx.us/#overview
And I've looked at example code found in this repo: https://github.com/ftexchange/ftx/tree/master/rest
I can't 'get' or 'post' anything that requires the Authentication. I am using the api key on my account that has 'full trade permissions', and when I look at: print(request.headers) the headers look like they are in the right format.
I've tried: using google colab instead of vs code, updating all my libraries, generating a new api key, restarting kernel and computer. I can pull something like 'markets' because it doesn't need the Authentication.
Let me know if you need any more information, below is a portion of the code I have that isolates the problem and returns {'success': False, 'error': 'Not logged in'}
import time
import urllib.parse
from typing import Optional, Dict, Any, List
from requests import Request, Session, Response
import hmac
ep = 'https://ftx.us/api/wallet/balances'
ts = int(time.time() * 1000)
s = Session()
request = Request('GET', ep)
prepared = request.prepare()
signature_payload = f'{ts}{prepared.method}{prepared.path_url}'.encode()
if prepared.body:
signature_payload += prepared.body
signature = hmac.new(secret.encode(), signature_payload, 'sha256').hexdigest()
request.headers['FTX-KEY'] = key
request.headers['FTX-SIGN'] = signature
request.headers['FTX-TS'] = str(ts)
response = s.send(prepared)
data = response.json()
print(data)
I've faced with the same problem.
You need to change this part:
prepared.headers['FTX-KEY'] = key
prepared.headers['FTX-SIGN'] = signature
prepared.headers['FTX-TS'] = str(ts)
PS. I believe that the FTX needs to fix their API documentation
PSS. I've checked the a part of https://github.com/ftexchange/ftx/tree/master/rest code. I beleave FTX guys just do a copy-paste into docs this code but originally it belongs to more a sophisticated object oriented solution that will work correctly because they pass into method an already created request and use a prepared variable just to calculate path_url and method
For ftx.us, you need to use different headers:
prepared.headers['FTXUS-KEY'] = key
prepared.headers['FTXUS-TS'] = str(ts)
prepared.headers['FTXUS-SIGN'] = signature
Amazon provides iOS, Android, and Javascript Cognito SDKs that offer a high-level authenticate-user operation.
For example, see Use Case 4 here:
https://github.com/aws/amazon-cognito-identity-js
However, if you are using python/boto3, all you get are a pair of primitives: cognito.initiate_auth and cognito.respond_to_auth_challenge.
I am trying to use these primitives along with the pysrp lib authenticate with the USER_SRP_AUTH flow, but what I have is not working.
It always fails with "An error occurred (NotAuthorizedException) when calling the RespondToAuthChallenge operation: Incorrect username or password." (The username/password pair work find with the JS SDK.)
My suspicion is I'm constructing the challenge response wrong (step 3), and/or passing Congito hex strings when it wants base64 or vice versa.
Has anyone gotten this working? Anyone see what I'm doing wrong?
I am trying to copy the behavior of the authenticateUser call found in the Javascript SDK:
https://github.com/aws/amazon-cognito-identity-js/blob/master/src/CognitoUser.js#L138
but I'm doing something wrong and can't figure out what.
#!/usr/bin/env python
import base64
import binascii
import boto3
import datetime as dt
import hashlib
import hmac
# http://pythonhosted.org/srp/
# https://github.com/cocagne/pysrp
import srp
bytes_to_hex = lambda x: "".join("{:02x}".format(ord(c)) for c in x)
cognito = boto3.client('cognito-idp', region_name="us-east-1")
username = "foobar#foobar.com"
password = "123456"
user_pool_id = u"us-east-1_XXXXXXXXX"
client_id = u"XXXXXXXXXXXXXXXXXXXXXXXXXX"
# Step 1:
# Use SRP lib to construct a SRP_A value.
srp_user = srp.User(username, password)
_, srp_a_bytes = srp_user.start_authentication()
srp_a_hex = bytes_to_hex(srp_a_bytes)
# Step 2:
# Submit USERNAME & SRP_A to Cognito, get challenge.
response = cognito.initiate_auth(
AuthFlow='USER_SRP_AUTH',
AuthParameters={ 'USERNAME': username, 'SRP_A': srp_a_hex },
ClientId=client_id,
ClientMetadata={ 'UserPoolId': user_pool_id })
# Step 3:
# Use challenge parameters from Cognito to construct
# challenge response.
salt_hex = response['ChallengeParameters']['SALT']
srp_b_hex = response['ChallengeParameters']['SRP_B']
secret_block_b64 = response['ChallengeParameters']['SECRET_BLOCK']
secret_block_bytes = base64.standard_b64decode(secret_block_b64)
secret_block_hex = bytes_to_hex(secret_block_bytes)
salt_bytes = binascii.unhexlify(salt_hex)
srp_b_bytes = binascii.unhexlify(srp_b_hex)
process_challenge_bytes = srp_user.process_challenge(salt_bytes,
srp_b_bytes)
timestamp = unicode(dt.datetime.utcnow().strftime("%a %b %d %H:%m:%S +0000 %Y"))
hmac_obj = hmac.new(process_challenge_bytes, digestmod=hashlib.sha256)
hmac_obj.update(user_pool_id.split('_')[1].encode('utf-8'))
hmac_obj.update(username.encode('utf-8'))
hmac_obj.update(secret_block_bytes)
hmac_obj.update(timestamp.encode('utf-8'))
challenge_responses = {
"TIMESTAMP": timestamp.encode('utf-8'),
"USERNAME": username.encode('utf-8'),
"PASSWORD_CLAIM_SECRET_BLOCK": secret_block_hex,
"PASSWORD_CLAIM_SIGNATURE": hmac_obj.hexdigest()
}
# Step 4:
# Submit challenge response to Cognito.
response = cognito.respond_to_auth_challenge(
ClientId=client_id,
ChallengeName='PASSWORD_VERIFIER',
ChallengeResponses=challenge_responses)
There are many errors in your implementation. For example:
pysrp uses SHA1 algorithm by default. It should be set to SHA256.
_ng_const length should be 3072 bits and it should be copied from amazon-cognito-identity-js
There is no hkdf function in pysrp.
The response should contain secret_block_b64, not secret_block_hex.
Wrong timestamp format. %H:%m:%S means "hour:month:second" and +0000 should be replaced by UTC.
Has anyone gotten this working?
Yes. It's implemented in the warrant.aws_srp module.
https://github.com/capless/warrant/blob/master/warrant/aws_srp.py
from warrant.aws_srp import AWSSRP
USERNAME='xxx'
PASSWORD='yyy'
POOL_ID='us-east-1_zzzzz'
CLIENT_ID = '12xxxxxxxxxxxxxxxxxxxxxxx'
aws = AWSSRP(username=USERNAME, password=PASSWORD, pool_id=POOL_ID,
client_id=CLIENT_ID)
tokens = aws.authenticate_user()
id_token = tokens['AuthenticationResult']['IdToken']
refresh_token = tokens['AuthenticationResult']['RefreshToken']
access_token = tokens['AuthenticationResult']['AccessToken']
token_type = tokens['AuthenticationResult']['TokenType']
authenticate_user method supports only PASSWORD_VERIFIER challenge. If you want to respond to other challenges, just look into the authenticate_user and boto3 documentation.
Unfortunately it's a hard problem since you don't get any hints from the service with regards to the computations (it mainly says not authorized as you mentioned).
We are working on improving the developer experience when users are trying to implement SRP on their own in languages where we don't have an SDK. Also, we are trying to add more SDKs.
As daunting as it sounds, what I would suggest is to take the Javascript or the Android SDK, fix the inputs (SRP_A, SRP_B, TIMESTAMP) and add console.log statements at various points in the implementation to make sure your computations are similar. Then you would run these computations in your implementation and make sure you are getting the same. As you have suggested, the password claim signature needs to be passed as a base64 encoded string to the service so that might be one of the issues.
Some of the issues I encountered while implementing this was related to BigInteger library differences (the way they do byte padding and transform negative numbers to byte arrays and inversely).
I use Python Social Auth - Django to log in my users.
My backend is Microsoft, so I can use Microsoft Graph but I don't think that it is relevant.
Python Social Auth deals with authentication but now I want to call the API and for that, I need a valid access token.
Following the use cases I can get to this:
social = request.user.social_auth.get(provider='azuread-oauth2')
response = self.get_json('https://graph.microsoft.com/v1.0/me',
headers={'Authorization': social.extra_data['token_type'] + ' '
+ social.extra_data['access_token']})
But the access token is only valid for 3600 seconds and so I need to refresh, I guess I can do it manually but there must be a better solution.
How can I get an access_token refreshed?
.get_access_token(strategy) refresh the token automatically if it's expired. You can use it like that:
from social_django.utils import load_strategy
#...
social = request.user.social_auth.get(provider='google-oauth2')
access_token = social.get_access_token(load_strategy())
Using load_strategy() at social.apps.django_app.utils:
social = request.user.social_auth.get(provider='azuread-oauth2')
strategy = load_strategy()
social.refresh_token(strategy)
Now the updated access_token can be retrieved from social.extra_data['access_token'].
The best approach is probably to check if it needs to be updated (customized for AzureAD Oauth2):
def get_azuread_oauth2_token(user):
social = user.social_auth.get(provider='azuread-oauth2')
if social.extra_data['expires_on'] <= int(time.time()):
strategy = load_strategy()
social.refresh_token(strategy)
return social.extra_data['access_token']
This is based on the method get_auth_tokenfrom AzureADOAuth2. I don't think this method is accessible outside the pipeline, please answer this question if there is any way to do it.
Updates
Update 1 - 20/01/2017
Following an Issue to request an extra data parameter with the time of the access token refresh, it is now possible to check if the access_token needs to be updated in every backend.
In future versions (>0.2.1 for the social-auth-core) there will be a new field in extra data:
'auth_time': int(time.time())
And so this works:
def get_token(user, provider):
social = user.social_auth.get(provider=provider)
if (social.extra_data['auth_time'] + social.extra_data['expires']) <= int(time.time()):
strategy = load_strategy()
social.refresh_token(strategy)
return social.extra_data['access_token']
Note: According to OAuth 2 RFC all responses should (it's a RECOMMENDED param) provide an expires_in but for most backends (including the azuread-oauth2) this value is being saved as expires. Be careful to understand how your backend behaves!
An Issue on this exists and I will be update the answer with the relevant info when it exists.
Update 2 - 17/02/17
Additionally, there is a method in UserMixin called access_token_expired (code) that can be used to assert if the token is valid or not (note: this method doesn't work for race conditions, as pointed out in this anwser by #SCasey).
Update 3 - 31/05/17
In Python Social Auth - Core v1.3.0 get_access_token(self, strategy) was introduced in storage.py.
So now:
from social_django.utils import load_strategy
social = request.user.social_auth.get(provider='azuread-oauth2')
response = self.get_json('https://graph.microsoft.com/v1.0/me',
headers={'Authorization': '%s %s' % (social.extra_data['token_type'],
social.get_access_token(load_strategy())}
Thanks #damio for pointing it out.
#NBajanca's update is almost correct for version 1.0.1.
extra_data['expires_in']
is now
extra_data['expires']
So the code is:
def get_token(user, provider):
social = user.social_auth.get(provider=provider)
if (social.extra_data['auth_time'] + social.extra_data['expires']) <= int(time.time()):
strategy = load_strategy()
social.refresh_token(strategy)
return social.extra_data['access_token']
I'd also recommend subtracting an arbitrary amount of time from that calc, so that we don't run into a race situation where we've checked the token 0.01s before expiry and then get an error because we sent the request after expiry. I like to add 10 seconds just to be safe, but it's probably overkill:
def get_token(user, provider):
social = user.social_auth.get(provider=provider)
if (social.extra_data['auth_time'] + social.extra_data['expires'] - 10) <= int(time.time()):
strategy = load_strategy()
social.refresh_token(strategy)
return social.extra_data['access_token']
EDIT
#NBajanca points out that expires_in is technically correct per the Oauth2 docs. It seems that for some backends, this may work. The code above using expires is what works with provider="google-oauth2" as of v1.0.1
I am struggling getting a Rest API Post to work with a vendor api and hope someone can give me a pointer.
The intent is to feed a cli command to the post body and pass to a device which returns the output.
The call looks like this : ( this works for all other calls but this is different because of posting to body)
def __init__(self,host,username,password,sid,method,http_meth):
self.host=host
self.username= username
self.password= password
self.sid=sid
self.method=method
self.http_meth=http_meth
def __str__(self):
self.url = 'http://' + self.host + '/rest/'
self.authparams = urllib.urlencode({ "session_id":self.sid,"method": self.method,"username": self.username,
"password": self.password,
})
call = urllib2.urlopen(self.url.__str__(), self.authparams).read()
return (call)
No matter how I have tried this I cant get it to work properly. Here is an excerpt from the API docs that explains how to use this method:
To process these APIs, place your CLI commands in the HTTP post buffer, and then place the
method name, session ID, and other parameters in the URL.
Can anyone give me an idea of how to properly do this. I am not a developer and am trying to learn this correctly. For example if I wanted to send the command "help" in the post body?
Thanks for any guidance
Ok this was ridiculously simple and I was over-thinking this. I find that I can sometimes look at a much higher level than a problem really is and waist time. Anyway this is how it should work:
def cli(self,method):
self.url = ("http://" + str(self.host) + "/rest//?method=" +str(method)+ "&username=" +str(self.username)+ "&password=" +str(self.password)+ "&enable_password=test&session_id="+str(self.session_id))
data="show ver"
try:
req = urllib2.Request(self.url)
response = urllib2.urlopen(req,data)
result = response.read()
print result
except urllib2.URLError, e:
print e.reason
The cli commands are just placed in the buffer and not encoded....
In this post, Nick suggested a decoartor:
Python/WebApp Google App Engine - testing for user/pass in the headers
I'm writing an API to expose potentially dozens of methods as web-services, so the decorator sounds like a great idea.
I tried to start coding one based on this sample:
http://groups.google.com/group/google-appengine/browse_thread/thread/ac51cc32196d62f8/aa6ccd47f217cb9a?lnk=gst&q=timeout#aa6ccd47f217cb9a
I need it compatible with Python 2.5 to run under Google App Engine (GAE).
Here's my attempt. Please just point the way to if I'm on the right track or not.
Currently getting an error "Invalid Syntax" on this line:
class WSTest(webapp.RequestHandler):
My idea is to pass an array of roles to the decorator. These are the only roles (from my db that should have access to each various web service).
def BasicAuthentication(roles=[]):
def _decorator(func):
def _wrapper(*args, **kwds):
logging.info("\n\n BasicAuthentication:START:__call__ \n\n")
auth = None
if 'Authorization' in self.request.headers:
auth = self.request.headers['Authorization']
if not auth:
self.response.headers['WWW-Authenticate'] = 'Basic realm="MYREALM"'
self.response.set_status(401)
self.response.out.write("Authorization required")
logging.info ("\n\n Authorization required \n\n")
return
(username, password) = base64.b64decode(auth.split(' ')[1]).split(':')
logging.info ("\n\n username = " + username + " password=" + password + "\n\n")
isValidUserPass = False
usersSimulatedRole = "Admin"
#check against database here...
if user == "test12" and password == "test34":
isValidUserPass = True
isValidRole = False
if usersSimulatedRole in roles:
isValidRole = True
#next check that user has one of the roles
# TODO
if not isValidUserPass:
self.response.set_status(403)
self.response.out.write("Forbidden: Userid/password combination failed")
logging.info("\n\n BasicAuthentication:END:__call__ \n\n")
return func(*args, **kwds)
return _wrapper
return _decorator
#BasicAuthentication(["Admin","Worker"]) #list of roles that can run this function
class WSTest(webapp.RequestHandler):
def get(self):
logging.info("\n\n\n WSTest \n\n")
...etc...
Thanks,
Neal Walters
You need to write a method decorator, not a class decorator: As lost-theory points out, class decorators don't exist in Python 2.5, and they wouldn't work very well in any case, because the RequestHandler class isn't initialized with request data until after it's constructed. A method decorator also gives you more control - eg, you could allow GET requests unauthenticated, but still require authentication for POST requests.
Other than that, your decorator looks fine - just apply it to the relevant methods. The only change I would really suggest is replacing the .set_status() calls with .error() calls and remove the response.write calls; this allows you to override .error() on the RequestHandler class to output a nice error page for each possible status code.
Class decorators were added in Python 2.6.
You'll have to manually wrap the class or think of another solution to work under 2.5. How about writing a decorator for the get method instead?