I am trying to encode an auth token and pass it to a REST API, this works fine with powershell but applying the same method to python script throws 'unauthorized' exception.
I suspect there is a problem in encoded value. Not able to figure out the solution. Any ideas ?
The rest endpoint is IBM uDeploy.
Powershell
$tokenEncoded = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes( "PasswordIsAuthToken:{`"token`":`"$pass`"}" ))
$basicAuthValue = "Basic $tokenEncoded"
$headers = #{}
$headers.Add("Authorization", $basicAuthValue)
$response = Invoke-RestMethod -Method Put -Headers $headers -Uri $requestUri -Body $jsonRequest
Python
epass = base64.b64encode("PasswordIsAuthToken:{\"token\":\"$password\"}")
print 'base64 encoded: ' + epass
opener = urllib2.build_opener(urllib2.HTTPHandler)
req = urllib2.Request(reqUrl,json.dumps(json_data))
req.add_header('Authorization', 'Basic '+epass)
req.get_method = lambda: 'PUT'
resp = opener.open(req)
You are sending the literal string $password as the token, not the contents of a variable named password.
You only need to include the PasswordIsAuthToken and your token in a Basic Auth HTTP header (PasswordIsAuthToken forms the username, and token the password):
epass = base64.b64encode("PasswordIsAuthToken:" + password)
opener = urllib2.build_opener(urllib2.HTTPHandler)
req = urllib2.Request(reqUrl,json.dumps(json_data))
req.add_header('Authorization', 'Basic ' + epass)
If you reall need to wrap the token in a JSON-like structure then you'd need to use string formatting (which the Powershell code also does, but you omitted):
epass = base64.b64encode('PasswordIsAuthToken:{"token":"%s"}' % password)
or you could use the json module:
epass = base64.b64encode('PasswordIsAuthToken:' + json.dumps({'token': password}))
However, I believe the system should accept the token unwrapped.
I strongly recommend you use the requests library instead, which makes using a REST API vastly cleaner:
import requests
auth = ('PasswordIsAuthToken', password)
# or alternatively
# auth = ('PasswordIsAuthToken', '{"token":"%s"}' % password)
response = requests.put(json=json_data, auth=auth)
Note that there is no need to encode the JSON body yourself, nor do you have to encode the Basic Auth header.
Related
I am writing a Python3 script to trigger an Axis camera via their HTTP API. The string they need in the url to trigger the Digital Input (with authorization) is:
"http://{ip}/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig={open/closed}"
If I put this in a browser it works AKA response 200 - OK. When I run this on my Linux machine using the urllib request.
#encode user,pass
values = { 'username': username,'password': password }
data = urllib.urlencode(values)
#Axis param.cgi action
action = "update"
trig = "closed"
cgi_args = {"action": action, "IOPort.I0.Input.Trig": trig}
cgi_data = urllib.urlencode(cgi_args)
url = "http://{ip}/axis-cgi/param.cgi?{data}".format(ip=ip, data=cgi_data)
req = urllib2.Request(url, data, headers={'Content-type': 'text/html'})
print(req.get_full_url())
response = urllib2.urlopen(req)
result = response.read()
print (result)
The output is:
http://192.168.50.191/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig=closed
action must be specified
I know that I am authenticated otherwise I get Unauthorized response from server.
As you can see for debugging I print the req.get_full_url() to make sure I've built the url string correctly, however the server responds with action must be specified which is what I get in the browser when I just have http://192.168.50.191/axis-cgi/param.cgi? in the address bar. So the action portion of the URL appears to not be making it to the server.
I've tried:
Using %3F as the ? character, but I get 404 error
Embedding the data in the data parameter does not work either
Anything I am missing here?
used pycurl with digest authorization for a GET request with auth:
def configCurl():
username = "user"
password = "pass"
auth_mode = pycurl.HTTPAUTH_DIGEST
curl = pycurl.Curl()
curl.setopt(pycurl.HTTPAUTH, auth_mode)
curl.setopt(pycurl.USERPWD, "{}:{}".format(username, password))
curl.setopt(pycurl.WRITEFUNCTION, lambda x: None)
return curl
curl = configCurl()
curl.setopt(curl.URL, url)
curl.perform()
curl.close()
I'm trying to send a get request with python 2.7. The server I'm trying to access has basic authentication and the url I'm accessing is just displaying a string I want my script to print that string.
So basically my code is sending a get request to this server and it saves the string it receives from the server, which also has basic auth.
The problem is that it prints out the html page for the login page not the string.
username = 'username'
password = 'password'
server = "http://someserver/update"
def get_auth():
request = urllib2.Request(server)
base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
# print(result)
def get_string():
f = urllib2.urlopen(server)
print f.read()
def main():
get_auth()
get_string()
if __name__ == '__main__':
main()
They way you are using get_string() never actually sends the authentication header.
Rather than fiddling with urllib2, you might want to try the requests library instead::
import requests
requests.get('http://someserver/update', auth=('username', 'password'))
see: http://docs.python-requests.org/en/latest/user/authentication/
I need to HTTP Basic Auth for a REST call. In the username I have to provide a domain (which has a hyphen) and then a backslash to separate it from the username, like this: DOM-AIN\user_name. Then the password is pretty benign.
This works fine with curl:
curl 'https://DOM-AIN\user_name:password#myurl.com'
I need to put this into Python now, but I've tried with requests and urllib/2/3...they don't like the \ : or the #. Even when I URL encode to %40, etc., those get interpreted as an actual : and urllib thinks I'm trying to define a port and I get an error: Invalid socket, I think, I forgot.
So I tried passing the username and password in the header using urllib3, but I get an unauthorized access error and I suspect it's because I need to somehow encode the username in the header to account for the backslash (%5C), but that doesn't seem to be working either.
Here is some code that doesn't work:
# Attempt 1
http = urllib3.PoolManager()
url1 = https://ws.....
headers = urllib3.util.make_headers(basic_auth='DOM-AIN\user_name:password')
r1 = http.request('GET', url1, headers=headers)
response = r1.data
# Attempt 2
passwordManager = urllib2.HTTPPasswordMgrWithDefaultRealm()
passwordManager.add_password(None, url, 'DOM-AIN\user_name, password)
authenticationHandler = urllib2.HTTPBasicAuthHandler(passwordManager)
opener = urllib2.build_opener(authenticationHandler)
data = opener.open(url1)
There were some other attempts with request, but I don't have those anymore. I can get the errors of these if it would be useful, but if there is already a known thing I'm doing wrong that would be great...
Backslash should be escaped in Python string literals:
username = 'DOM-AIN\\user_name' # OR
username = r'DOM-AIN\user_name' # raw-string literal
Example:
import urllib2, base64
request = urllib2.Request('https://example.com')
credentials = base64.b64encode(username + b':' + password)
request.add_header('Authorization', b'Basic ' + credentials)
response = urllib2.urlopen(request)
Note: unlike HTTPBasicAuthHandler code; it always sends the credentials without waiting for 401 response with WWW-Authenticate header.
First convert your DOM-AIN\user_name into base64 string. Lets say its XXXXYYYYYYY. Now place this base64 string into the http header like below code with urllib2.
headers = { 'Authorization:' : 'Basic XXXXYYYYYYY' }
req = urllib2.Request(url, data, headers)
I found a way using urllib, with this post's mention of FancyURLopener sending me down the right path. This was the closest I could come to replicating the way it worked in curl, although looking at Sabuj's answer there might be a way to use headers properly, but I haven't tried his method.
import urllib
opener = urllib.FancyURLopener()
data = opener.open('https://DOM-AIN**%5C%user_name:password#url.com?whatever_parameters')
response = data.read()
It works when I only URL encoded the backslash. Didn't work when I encoded the other characters like : and #.
This is part of my unit test in Flask-RESTful.
self.app = application.app.test_client()
rv = self.app.get('api/v1.0/{0}'.format(ios_sync_timestamp))
eq_(rv.status_code,200)
Within the command line I could use curl to send the username:password to the service:
curl -d username:password http://localhost:5000/api/v1.0/1234567
How do I achieve the same within my unit test's get() ?
Since my get/put/post require authentication otherwise the test would fail.
From RFC 1945, Hypertext Transfer Protocol -- HTTP/1.0
11.1 Basic Authentication Scheme
...
To receive authorization, the client sends the user-ID and password,
separated by a single colon (":") character, within a base64 [5]
encoded string in the credentials.string.
...
If the user agent wishes to send the user-ID "Aladdin" and password
open sesame", it would use the following header field:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
So if you really use http basic authentication you can solution like below, although your curl usage suggests some other authentication scheme.
from base64 import b64encode
headers = {
'Authorization': 'Basic ' + b64encode("{0}:{1}".format(username, password)).decode('utf-8')
}
rv = self.app.get('api/v1.0/{0}'.format(ios_sync_timestamp), headers=headers)
An alternative solution - All credit goes to Doug Black
def request(self, method, url, auth=None, **kwargs):
headers = kwargs.get('headers', {})
if auth:
headers['Authorization'] = 'Basic ' + base64.b64encode(auth[0] + ':' + auth[1])
kwargs['headers'] = headers
return self.app.open(url, method=method, **kwargs)
and then use this method in your tests:
resp = self.request('GET', 'api/v1.0/{0}'.format(ios_sync_timestamp), auth=(username, password))
For Python 3, try the following example:
from base64 import b64encode
headers = {
'Authorization': 'Basic %s' % b64encode(b"username:password").decode("ascii")
}
self.app.get("foo/", headers=headers)
If you'd like to use dynamic variables for username and password, then try something like:
'Basic %s' % b64encode(bytes(username + ':' + password, "utf-8")).decode("ascii")
See also: Python, HTTPS GET with basic authentication
Update: based on Lee's comment I decided to condense my code to a really simple script and run it from the command line:
import urllib2
import sys
username = sys.argv[1]
password = sys.argv[2]
url = sys.argv[3]
print("calling %s with %s:%s\n" % (url, username, password))
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, username, password)
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPBasicAuthHandler(passman)))
req = urllib2.Request(url)
f = urllib2.urlopen(req)
data = f.read()
print(data)
Unfortunately it still won't generate the Authorization header (per Wireshark) :(
I'm having a problem sending basic AUTH over urllib2. I took a look at this article, and followed the example. My code:
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, "api.foursquare.com", username, password)
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPBasicAuthHandler(passman)))
req = urllib2.Request("http://api.foursquare.com/v1/user")
f = urllib2.urlopen(req)
data = f.read()
I'm seeing the following on the Wire via wireshark:
GET /v1/user HTTP/1.1
Host: api.foursquare.com
Connection: close
Accept-Encoding: gzip
User-Agent: Python-urllib/2.5
You can see the Authorization is not sent, vs. when I send a request via curl: curl -u user:password http://api.foursquare.com/v1/user
GET /v1/user HTTP/1.1
Authorization: Basic =SNIP=
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4 OpenSSL/0.9.8k zlib/1.2.3
Host: api.foursquare.com
Accept: */*
For some reason my code seems to not send the authentication - anyone see what I'm missing?
thanks
-simon
The problem could be that the Python libraries, per HTTP-Standard, first send an unauthenticated request, and then only if it's answered with a 401 retry, are the correct credentials sent. If the Foursquare servers don't do "totally standard authentication" then the libraries won't work.
Try using headers to do authentication:
import urllib2, base64
request = urllib2.Request("http://api.foursquare.com/v1/user")
base64string = base64.b64encode('%s:%s' % (username, password))
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
Had the same problem as you and found the solution from this thread: http://forums.shopify.com/categories/9/posts/27662
(copy-paste/adapted from https://stackoverflow.com/a/24048772/1733117).
First you can subclass urllib2.BaseHandler or urllib2.HTTPBasicAuthHandler, and implement http_request so that each request has the appropriate Authorization header.
import urllib2
import base64
class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
'''Preemptive basic auth.
Instead of waiting for a 403 to then retry with the credentials,
send the credentials if the url is handled by the password manager.
Note: please use realm=None when calling add_password.'''
def http_request(self, req):
url = req.get_full_url()
realm = None
# this is very similar to the code from retry_http_basic_auth()
# but returns a request object.
user, pw = self.passwd.find_user_password(realm, url)
if pw:
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
req.add_unredirected_header(self.auth_header, auth)
return req
https_request = http_request
Then if you are lazy like me, install the handler globally
api_url = "http://api.foursquare.com/"
api_username = "johndoe"
api_password = "some-cryptic-value"
auth_handler = PreemptiveBasicAuthHandler()
auth_handler.add_password(
realm=None, # default realm.
uri=api_url,
user=api_username,
passwd=api_password)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
Here's what I'm using to deal with a similar problem I encountered while trying to access MailChimp's API. This does the same thing, just formatted nicer.
import urllib2
import base64
chimpConfig = {
"headers" : {
"Content-Type": "application/json",
"Authorization": "Basic " + base64.encodestring("hayden:MYSECRETAPIKEY").replace('\n', '')
},
"url": 'https://us12.api.mailchimp.com/3.0/'}
#perform authentication
datas = None
request = urllib2.Request(chimpConfig["url"], datas, chimpConfig["headers"])
result = urllib2.urlopen(request)
The second parameter must be a URI, not a domain name. i.e.
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, "http://api.foursquare.com/", username, password)
I would suggest that the current solution is to use my package urllib2_prior_auth which solves this pretty nicely (I work on inclusion to the standard lib.