The answer to a previous question showed that Nexus implement a custom authentication helper called "NxBASIC".
How do I begin to implement a handler in python?
Update:
Implementing the handler per Alex's suggestion looks to be the right approach, but fails trying to extract the scheme and realm from the authreq.
The returned value for authreq is:
str: NxBASIC realm="Sonatype Nexus Repository Manager API""
AbstractBasicAuthHandler.rx.search(authreq) is only returning a single tuple:
tuple: ('NxBASIC', '"', 'Sonatype Nexus Repository Manager API')
so scheme,realm = mo.groups() fails. From my limited regex knowledge it looks like the standard regex from AbstractBasicAuthHandler should match scheme and realm, but it seems not to.
The regex is:
rx = re.compile('(?:.*,)*[ \t]*([^ \t]+)[ \t]+'
'realm=(["\'])(.*?)\\2', re.I)
Update 2:
From inspection of AbstractBasicAuthHandler, the default processing is to do:
scheme, quote, realm = mo.groups()
Changing to this works. I now just need to set the password against the correct realm. Thanks Alex!
If, as described, name and description are the only differences between this "NxBasic" and good old "Basic", then you could essentially copy-paste-edit some code from urllib2.py (which unfortunately doesn't expose the scheme name as easily overridable in itself), as follows (see urllib2.py's online sources):
import urllib2
class HTTPNxBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
def http_error_auth_reqed(self, authreq, host, req, headers):
# host may be an authority (without userinfo) or a URL with an
# authority
# XXX could be multiple headers
authreq = headers.get(authreq, None)
if authreq:
mo = AbstractBasicAuthHandler.rx.search(authreq)
if mo:
scheme, realm = mo.groups()
if scheme.lower() == 'nxbasic':
return self.retry_http_basic_auth(host, req, realm)
def retry_http_basic_auth(self, host, req, realm):
user, pw = self.passwd.find_user_password(realm, host)
if pw is not None:
raw = "%s:%s" % (user, pw)
auth = 'NxBasic %s' % base64.b64encode(raw).strip()
if req.headers.get(self.auth_header, None) == auth:
return None
req.add_header(self.auth_header, auth)
return self.parent.open(req)
else:
return None
As you can see by inspection, I've just changed two strings from "Basic" to "NxBasic" (and the lowercase equivalents) from what's in urrlib2.py (in the abstract basic auth handler superclass of the http basic auth handler class).
Try using this version -- and if it's still not working, at least having it be your code can help you add print/logging statements, breakpoints, etc, to better understand what's breaking and how. Best of luck! (Sorry I can't help further but I don't have any Nexus around to experiment with).
Related
I have a DataFrame common_ips containing IPs as shown below.
I need to achieve two basic tasks:
Identify private and public IPs.
Check organisation for public IPs.
Here is what I am doing:
import json
import urllib
import re
baseurl = 'http://ipinfo.io/' # no HTTPS supported (at least: not without a plan)
def isIPpublic(ipaddress):
return not isIPprivate(ipaddress)
def isIPprivate(ipaddress):
if ipaddress.startswith("::ffff:"):
ipaddress=ipaddress.replace("::ffff:", "")
# IPv4 Regexp from https://stackoverflow.com/questions/30674845/
if re.search(r"^(?:10|127|172\.(?:1[6-9]|2[0-9]|3[01])|192\.168)\..*", ipaddress):
# Yes, so match, so a local or RFC1918 IPv4 address
return True
if ipaddress == "::1":
# Yes, IPv6 localhost
return True
return False
def getipInfo(ipaddress):
url = '%s%s/json' % (baseurl, ipaddress)
try:
urlresult = urllib.request.urlopen(url)
jsonresult = urlresult.read() # get the JSON
parsedjson = json.loads(jsonresult) # put parsed JSON into dictionary
return parsedjson
except:
return None
def checkIP(ipaddress):
if (isIPpublic(ipaddress)):
if bool(getipInfo(ipaddress)):
if 'bogon' in getipInfo(ipaddress).keys():
return 'Private IP'
elif bool(getipInfo(ipaddress).get('org')):
return getipInfo(ipaddress)['org']
else:
return 'No organization data'
else:
return 'No data available'
else:
return 'Private IP'
And applying it to my common_ips DataFrame with
common_ips['Info'] = common_ips.IP.apply(checkIP)
But it's taking longer than I expected. And for some IPs, it's giving incorrect Info.
For instance:
where it should have been AS19902 Department of Administrative Services as I cross-checked it by
and
What am I missing here ? And how can I achieve these tasks in a more Pythonic way ?
A blanket except: is basically always a bug. You are returning None instead of handling any anomalous or error response from the server, and of course the rest of your code has no way to recover.
As a first debugging step, simply take out the try/except handling. Maybe then you can find a way to put back a somewhat more detailed error handler for some cases which you know how to recover from.
def getipInfo(ipaddress):
url = '%s%s/json' % (baseurl, ipaddress)
urlresult = urllib.request.urlopen(url)
jsonresult = urlresult.read() # get the JSON
parsedjson = json.loads(jsonresult) # put parsed JSON into dictionary
return parsedjson
Perhaps the calling code in checkIP should have a try/except instead, and e.g. retry after sleeping for a bit if the server indicates that you are going too fast.
(In the absence of an authorization token, it looks like you are using the free version of this service, which is probably not in any way guaranteed anyway. Also maybe look at using their recommended library -- I haven't looked at it in more detail, but I would imagine it at the very least knows better how to behave in the case of a server-side error. It's almost certainly also more Pythonic, at least in the sense that you should not reinvent things which already exist.)
I develop class that manages connections to external resources. Its responsibility is reading credentials from given source and returning authenticated connections, like:
def get_svn_connection(self):
username, pwd=self.get_credentials("SVN")
client=self.get_svn_client(username, pwd)
return client
def get_db_connection(self):
username, pwd, url, schema=self.get_credentials("DB")
client=self.get_db_client(username, pwd, url, schema)
return client
Today I met new use case: sometimes user needs to get only credentials without connection. So, following current solution, I have to create methods get_svn_credentials, get_db_credentials and so on. It looks very redundant so I want to refactor this structure.
My ideas are:
Create subclasses for every connection type and for each of them define methods get_connection and get_credentials. The problems are: 1) I want one class to rule all the connections; 2) I would have to use multi-inheritance (possible in Python, but would not work in, for example, Java - if I'll meet this problem in future)
Create methods get_connection(type) and get_credentials(type) and for every known connection type specified by string return corresponding instance. Problem is I have to create a switch which considered bad in OOP.
Use metaprogramming to generate methods for every connection types. Not obvious and Python-specific.
How would you design this?
You could have a Credentials class, which can be used to authenticate another function call:
class SvnCredentials:
def __init__(username, password):
self._username = username
self._password = password
def authenticate(function):
return function(self._username, self._password)
Usage:
>>> credentials = SvnCredentials('AzureDiamond ', 'hunter2')
>>> client = credentials.authenticate(get_svn_client)
Just to add on Jean-Francois Fabre answer, you can do one function the following way
def get_svn_connection(self, decision_str):
if decision_str = "DB":
return self.get_connection("DB",self.get_db_client)
if decision_str = "SVN":
return self.get_connection(decision_str, self.get_svn_client)
I don't think you can do it any more concisely than that, and I am not convinced it is better than Jean's way...
def get_svn_connection(self, credentials=False):
username, pwd=self.get_credentials("SVN")
if credentials:
return (username, pwd)
client=self.get_svn_client(username, pwd)
return client
def get_db_connection(self, credentials=False):
username, pwd, url, schema=self.get_credentials("DB")
if credentials:
return (username, pwd, url, schema)
client=self.get_db_client(username, pwd, url, schema)
return client
I've been asked to deal with an external REST API (Zendesk's, in fact) whose credentials need to be formatted as {email}/token:{security_token} -- a single value rather than the usual username/password pair. I'm trying to use the Python requests module for this task, since it's Pythonic and doesn't hurt my brain too badly, but I'm not sure how to format the authentication credentials. The Zendesk documentation only gives access examples using curl, which I'm unfamiliar with.
Here's how I currently have requests.auth.AuthBase subclassed:
class ZDTokenAuth(requests.auth.AuthBase):
def __init__(self,username,token):
self.username = username
self.token = token
def __call__(self,r):
auth_string = self.username + "/token:" + self.token
auth_string = auth_string.encode('utf-8')
r.headers['Authorization'] = base64.b64encode(auth_string)
return r
I'm not sure that the various encodings are required, but that's how someone did it on github (https://github.com/skipjac/Zendesk-python-api/blob/master/zendesk-ticket-delete.py) so why not. I've tried it without the encoding too, of course - same result.
Here's the class and methods I'm using to test this:
class ZDStats(object):
api_base = "https://mycompany.zendesk.com/api/v2/"
def __init__(self,zd_auth):
self.zd_auth = zd_auth # this is assumed to be a ZDTokenAuth object
def testCredentials(self):
zd_users_url = self.api_base + "users.json"
zdreq = requests.get(zd_users_url, auth=self.zdauth)
return zdreq
This is called with:
credentials = ZDTokenAuth(zd_username,zd_apitoken)
zd = ZDStats(credentials)
users = zd.testCredentials()
print(users.status_code)
print(users.text)
The status code I'm getting back is a 401, and the text is simply {"error":"Couldn't authenticate you."}. Clearly I'm doing something wrong here, but I don't think I know enough to know what it is I'm doing wrong, if that makes sense. Any ideas?
What you're missing is the auth type. Your Authorization header should be created like this:
r.headers['Authorization'] = b"Basic " + base64.b64encode(auth_string)
You can also achieve the same passing by a tuple as auth parameter with:
requests.get(url, auth=(username+"/token", token))
I have 2 files compiled by django-pipeline along with s3boto: master.css and master.js. They are set to "Public" in my buckets. However, when I access them, sometimes master.css is served, sometimes it errs with SignatureDoesNotMatch. The same with master.js. This doesn't happen on Chrome. What could I be missing?
EDIT: It now happens on Chrome too.
Happened to me too...
Took a few hours to find, but I figured it out eventually.
Turns out that if the right signature is :
ssCNsAOxLf5vA80ldAI3M0CU2%2Bw=
Then AWS will NOT accept:
ssCNsAOxLf5vA80ldAI3M0CU2+w=
Where the only difference is the translation of %2B to '+'.
S3BotoStorage actually yields it correctly but the encoding happens on CachedFilesMixin in the final line of the url method (return unquote(final_url)).
To fix it, I derived a new CachedFilesMixin to undo the "damage" (I should mention that I don't know why this unquote exists in the first place, so undoing it might cause other problems)
class MyCachedFilesMixin(CachedFilesMixin):
def url(self, *a, **kw):
s = super(MyCachedFilesMixin, self).url(*a, **kw)
if isinstance(s, unicode):
s = s.encode('utf-8', 'ignore')
scheme, netloc, path, qs, anchor = urlparse.urlsplit(s)
path = urllib.quote(path, '/%')
qs = urllib.quote_plus(qs, ':&=')
return urlparse.urlunsplit((scheme, netloc, path, qs, anchor))
Where I used the code I found here.
Hope this helps...
I had a similar issue causing SignatureDoesNotMatch errors when downloading files using an S3 signed URL and the python requests HTTP library.
My problem ended up being a bad content-type. The documentation at AWS on Authenticating REST Requests helped me figure it out, and has examples in Python.
I was struggling with this for a while, and I didn't like the idea of messing up with CachedFilesMixin (seemed like an overkill to me).
Until a proper fix is issued to the django platform, I've found quoting the signature two times is a good option. I know it's not pretty, but it works and it's simple.
So you'll just have to do something like this:
signature = urllib.quote_plus(signature.strip())
signature = urllib.quote_plus(signature.strip())
Hope it helps!
This article on Flask is a good resource on getting your signatures right: https://devcenter.heroku.com/articles/s3-upload-python
#app.route('/sign_s3/')
def sign_s3():
AWS_ACCESS_KEY = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
S3_BUCKET = os.environ.get('S3_BUCKET')
object_name = request.args.get('s3_object_name')
mime_type = request.args.get('s3_object_type')
expires = int(time.time()+10)
amz_headers = "x-amz-acl:public-read"
put_request = "PUT\n\n%s\n%d\n%s\n/%s/%s" % (mime_type, expires, amz_headers, S3_BUCKET, object_name)
signature = base64.encodestring(hmac.new(AWS_SECRET_KEY,put_request, sha).digest())
signature = urllib.quote_plus(signature.strip())
url = 'https://%s.s3.amazonaws.com/%s' % (S3_BUCKET, object_name)
return json.dumps({
'signed_request': '%s?AWSAccessKeyId=%s&Expires=%d&Signature=%s' % (url, AWS_ACCESS_KEY, expires, signature),
'url': url
})
Simple workaround for me was to generate a new access key with only alphanumeric characters (ie no special characters such as "/", "+", etc. which AWS sometimes adds to the keys).
In this post, Nick suggested a decoartor:
Python/WebApp Google App Engine - testing for user/pass in the headers
I'm writing an API to expose potentially dozens of methods as web-services, so the decorator sounds like a great idea.
I tried to start coding one based on this sample:
http://groups.google.com/group/google-appengine/browse_thread/thread/ac51cc32196d62f8/aa6ccd47f217cb9a?lnk=gst&q=timeout#aa6ccd47f217cb9a
I need it compatible with Python 2.5 to run under Google App Engine (GAE).
Here's my attempt. Please just point the way to if I'm on the right track or not.
Currently getting an error "Invalid Syntax" on this line:
class WSTest(webapp.RequestHandler):
My idea is to pass an array of roles to the decorator. These are the only roles (from my db that should have access to each various web service).
def BasicAuthentication(roles=[]):
def _decorator(func):
def _wrapper(*args, **kwds):
logging.info("\n\n BasicAuthentication:START:__call__ \n\n")
auth = None
if 'Authorization' in self.request.headers:
auth = self.request.headers['Authorization']
if not auth:
self.response.headers['WWW-Authenticate'] = 'Basic realm="MYREALM"'
self.response.set_status(401)
self.response.out.write("Authorization required")
logging.info ("\n\n Authorization required \n\n")
return
(username, password) = base64.b64decode(auth.split(' ')[1]).split(':')
logging.info ("\n\n username = " + username + " password=" + password + "\n\n")
isValidUserPass = False
usersSimulatedRole = "Admin"
#check against database here...
if user == "test12" and password == "test34":
isValidUserPass = True
isValidRole = False
if usersSimulatedRole in roles:
isValidRole = True
#next check that user has one of the roles
# TODO
if not isValidUserPass:
self.response.set_status(403)
self.response.out.write("Forbidden: Userid/password combination failed")
logging.info("\n\n BasicAuthentication:END:__call__ \n\n")
return func(*args, **kwds)
return _wrapper
return _decorator
#BasicAuthentication(["Admin","Worker"]) #list of roles that can run this function
class WSTest(webapp.RequestHandler):
def get(self):
logging.info("\n\n\n WSTest \n\n")
...etc...
Thanks,
Neal Walters
You need to write a method decorator, not a class decorator: As lost-theory points out, class decorators don't exist in Python 2.5, and they wouldn't work very well in any case, because the RequestHandler class isn't initialized with request data until after it's constructed. A method decorator also gives you more control - eg, you could allow GET requests unauthenticated, but still require authentication for POST requests.
Other than that, your decorator looks fine - just apply it to the relevant methods. The only change I would really suggest is replacing the .set_status() calls with .error() calls and remove the response.write calls; this allows you to override .error() on the RequestHandler class to output a nice error page for each possible status code.
Class decorators were added in Python 2.6.
You'll have to manually wrap the class or think of another solution to work under 2.5. How about writing a decorator for the get method instead?