I use a python code to connect to an API to retrieve information in json format. A password is used for the authentication to the API. The authentication was working fine so far, but since I changed my password into one with special characters, things went wrong. i guess the password is not well handled either by the API or the code I use.
This is the piece of code that seems to cause the errors:
# Create connection and request header.
# This class does not perform any verification of the server`s certificate.
conn = HTTPSConnection(ADDRESS)
auth_header = 'Basic %s' % (':'.join([USER_NAME, PASSWORD]).encode('Base64').strip('\r\n'))
request_header = {'Authorization':auth_header.decode(),
'Content-Type': content_type}
The error I get after executing the code is the following:
response = self.perform_request(log, 'GET', 'network', params=query)
File "/usr/local/lib/python2.7/query_code.py", line 103, in perform_request
auth_header = 'Basic %s' % str(":".join([USER_NAME, PASSWORD]).encode('Base64').strip('\r\n'))
TypeError: sequence item 1: expected string or Unicode, NoneType found
I should have had a successful connection to the API with code 200 and results returned in json format.
I have difficulties spotting the origin of all this, and would like to have your opinion on what I think it is, and how to solve it.
Try this instead:
def basic_token(key, secret):
return base64.b64encode(bytes(f'{key}:{secret}', 'utf-8')).decode('utf-8')
auth_header = f'basic {basic_token(USERNAME, PASSWORD)}'
If you are using a python version < 3.6, you will need to use .format instead of f-strings:
def basic_token(key, secret):
return base64.b64encode(bytes('{}:{}'.format(key, secret), 'utf-8')).decode('utf-8')
auth_header = 'basic {}'.format(basic_token(USERNAME, PASSWORD))
Related
This is the code I wrote to get user balance from BingX API.
I think I do everything correct but it doesn't work properly.
import urllib.request
import json
import base64
import hmac
import time
APIURL = "https://open-api.bingx.com"
APIKEY = "MyApiKey"
SECRETKEY = "MySecretKey"
def genSignature(paramsStr):
return hmac.new(SECRETKEY.encode("utf-8"),
paramsStr.encode("utf-8"), digestmod="sha256").digest()
def post(url, body):
req = urllib.request.Request(url, headers={
'User-Agent': 'Mozilla/5.0',
'X-BX-APIKEY': APIKEY,
}, method="GET")
return urllib.request.urlopen(req).read()
def getBalance():
paramsMap = {
"timestamp": int(time.time()*1000)
}
paramsStr = "&".join(["%s=%s" % (k, paramsMap[k]) for k in paramsMap])
paramsStr += "&signature=" + genSignature(paramsStr).hex()
url = "%s/openApi/swap/v2/user/balance?%s" % (APIURL, paramsStr)
return post(url, paramsStr)
def main():
print(getBalance())
if __name__ == "__main__":
main()
But when I run it I get this:
b'{"code":100001,"msg":"","success":false,"timestamp":1675069039381}'
This is the doc link
The response from the API is indicating that the request was unsuccessful and returned a code of 100001 with a success value of false. This means that there was some sort of signature authentication error in the request that was made.
The 100001 error code means that the signature authentication has failed. The signature is used to verify the authenticity of the request, so if the signature is incorrect, the request will fail.
There are a few things that could be causing the signature to fail:
Incorrect calculation of the signature: Make sure the code for generating the signature is correct and follows the requirements of the BingX API.
Incorrect encoding: Make sure the signature is properly encoded before being added to the request as a query parameter.
Incorrect secret key: Make sure the secret key used to generate the signature is correct and up-to-date.
Incorrect time stamp: Make sure the time stamp included in the request is correct and in the correct format.
You should carefully review the code and the API documentation to ensure that the signature is being generated correctly and that all required information is included in the request. If the issue persists, you may also want to reach out to the BingX API support team for additional assistance.
import requests
# Replace YOUR_API_KEY with your actual Binance API key
headers = {'X-MBX-APIKEY': 'YOUR_API_KEY'}
# Make a GET request to the Binance account endpoint
response = requests.get('https://api.binance.com/api/v3/account', headers=headers)
# Check if the request was successful
if response.status_code == 200:
# Parse the JSON response
data = response.json()
# Get the user's available balance for the specified asset
asset_balance = [balance for balance in data['balances'] if balance['asset'] == 'BTC'][0]['free']
print('User balance:', asset_balance)
else:
# Handle the error
print('Error:', response.text)
I need to HTTP Basic Auth for a REST call. In the username I have to provide a domain (which has a hyphen) and then a backslash to separate it from the username, like this: DOM-AIN\user_name. Then the password is pretty benign.
This works fine with curl:
curl 'https://DOM-AIN\user_name:password#myurl.com'
I need to put this into Python now, but I've tried with requests and urllib/2/3...they don't like the \ : or the #. Even when I URL encode to %40, etc., those get interpreted as an actual : and urllib thinks I'm trying to define a port and I get an error: Invalid socket, I think, I forgot.
So I tried passing the username and password in the header using urllib3, but I get an unauthorized access error and I suspect it's because I need to somehow encode the username in the header to account for the backslash (%5C), but that doesn't seem to be working either.
Here is some code that doesn't work:
# Attempt 1
http = urllib3.PoolManager()
url1 = https://ws.....
headers = urllib3.util.make_headers(basic_auth='DOM-AIN\user_name:password')
r1 = http.request('GET', url1, headers=headers)
response = r1.data
# Attempt 2
passwordManager = urllib2.HTTPPasswordMgrWithDefaultRealm()
passwordManager.add_password(None, url, 'DOM-AIN\user_name, password)
authenticationHandler = urllib2.HTTPBasicAuthHandler(passwordManager)
opener = urllib2.build_opener(authenticationHandler)
data = opener.open(url1)
There were some other attempts with request, but I don't have those anymore. I can get the errors of these if it would be useful, but if there is already a known thing I'm doing wrong that would be great...
Backslash should be escaped in Python string literals:
username = 'DOM-AIN\\user_name' # OR
username = r'DOM-AIN\user_name' # raw-string literal
Example:
import urllib2, base64
request = urllib2.Request('https://example.com')
credentials = base64.b64encode(username + b':' + password)
request.add_header('Authorization', b'Basic ' + credentials)
response = urllib2.urlopen(request)
Note: unlike HTTPBasicAuthHandler code; it always sends the credentials without waiting for 401 response with WWW-Authenticate header.
First convert your DOM-AIN\user_name into base64 string. Lets say its XXXXYYYYYYY. Now place this base64 string into the http header like below code with urllib2.
headers = { 'Authorization:' : 'Basic XXXXYYYYYYY' }
req = urllib2.Request(url, data, headers)
I found a way using urllib, with this post's mention of FancyURLopener sending me down the right path. This was the closest I could come to replicating the way it worked in curl, although looking at Sabuj's answer there might be a way to use headers properly, but I haven't tried his method.
import urllib
opener = urllib.FancyURLopener()
data = opener.open('https://DOM-AIN**%5C%user_name:password#url.com?whatever_parameters')
response = data.read()
It works when I only URL encoded the backslash. Didn't work when I encoded the other characters like : and #.
This is part of my unit test in Flask-RESTful.
self.app = application.app.test_client()
rv = self.app.get('api/v1.0/{0}'.format(ios_sync_timestamp))
eq_(rv.status_code,200)
Within the command line I could use curl to send the username:password to the service:
curl -d username:password http://localhost:5000/api/v1.0/1234567
How do I achieve the same within my unit test's get() ?
Since my get/put/post require authentication otherwise the test would fail.
From RFC 1945, Hypertext Transfer Protocol -- HTTP/1.0
11.1 Basic Authentication Scheme
...
To receive authorization, the client sends the user-ID and password,
separated by a single colon (":") character, within a base64 [5]
encoded string in the credentials.string.
...
If the user agent wishes to send the user-ID "Aladdin" and password
open sesame", it would use the following header field:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
So if you really use http basic authentication you can solution like below, although your curl usage suggests some other authentication scheme.
from base64 import b64encode
headers = {
'Authorization': 'Basic ' + b64encode("{0}:{1}".format(username, password)).decode('utf-8')
}
rv = self.app.get('api/v1.0/{0}'.format(ios_sync_timestamp), headers=headers)
An alternative solution - All credit goes to Doug Black
def request(self, method, url, auth=None, **kwargs):
headers = kwargs.get('headers', {})
if auth:
headers['Authorization'] = 'Basic ' + base64.b64encode(auth[0] + ':' + auth[1])
kwargs['headers'] = headers
return self.app.open(url, method=method, **kwargs)
and then use this method in your tests:
resp = self.request('GET', 'api/v1.0/{0}'.format(ios_sync_timestamp), auth=(username, password))
For Python 3, try the following example:
from base64 import b64encode
headers = {
'Authorization': 'Basic %s' % b64encode(b"username:password").decode("ascii")
}
self.app.get("foo/", headers=headers)
If you'd like to use dynamic variables for username and password, then try something like:
'Basic %s' % b64encode(bytes(username + ':' + password, "utf-8")).decode("ascii")
See also: Python, HTTPS GET with basic authentication
I have the following code, which should perform the first part of creating a new download at github. It should send the json-data with POST.
jsonstring = '{"name": "test", "size": "4"}'
req = urllib2.Request("https://api.github.com/repos/<user>/<repo>/downloads")
req.add_header('Authorization', 'token ' + '<token>')
result = urllib2.urlopen(req, jsonstring)
If I remove the , jsonstring from the urlopen(), it does not fail, and gives me the list of available downloads. However, if I try to POST the json-string, I get 404 error.
The problem has to be with the json, or in the way I send it, but I can't figure out what the problem is.
The strings at <...> are right in the actual code, I just removed them from the post
I tried roughly the same with curl on the command-line, with slightly different method of authentication, and it worked.
Tested:
Works(returns the wanted json):
curl -u "user:password" --data "json..." https://api.github.com/repos/<user>/<repo>/downloads
Works:
curl -H 'Authorization: token <token>' https://api.github.com/
Does not work (returns "invalid credentials"):
curl -H 'Authorization: token <invalid_token>' https://api.github.com/
Does not work ("not found"):
curl -H 'Authorization: token <valid_token>' --data "json..." https://api.github.com/repos/<user>/<repo>/downloads
This does not seem to be an issue specific to the python code. The json POST data seems to be fine, and the OAuth token authorization seems to be (atleast partly) working. But when these are put together, it stops working.
I was finally able to work out, why it did not work.
I did not have the autorization scopes for the authorization token set correctly. The token I was using, was not "authorized" to do any modifications, and every action using it, that tried to modify something (add a download), failed.
I had to add the correct scopes to the authorization to make it work.
I have provided an answer as to how to POST JSON data to the V3 API without any authentication, but seen as you have identified that the original problem was with not setting up your OAUTH tokens correctly, I thought I would provide a programatic solution to getting one (this implementation gets a token every time the script is run, whereas in practice it would be done just once, and the token would be stored localy).
import urllib2
import json
import getpass
import base64
# Generate a token from the username and password.
# NOTE: this is a naive implementation. Store pre-retrieved tokens if possible.
username = 'pelson'
passwd = getpass.getpass() # <- this just puts a string in passwd (plaintext)
req = urllib2.Request("https://api.github.com/authorizations")
# add the username and password info to the request
base64string = base64.encodestring('%s:%s' % (username, passwd)).replace('\n', '')
req.add_header("Authorization", "Basic %s" % base64string)
data = json.dumps({"scopes":["repo"], "note":"Access to your repository."})
result = urllib2.urlopen(req, data)
result = json.loads('\n'.join(result.readlines()))
token = result['token']
Once you have this token, it can be used for any "repo" scope actions. Lets add a new issue to a repository:
# add an issue to the tracker using the new token
repo = 'name_of_repo'
data = json.dumps({'title': 'My automated issue.'})
req = urllib2.Request("https://api.github.com/repos/%s/%s/issues" % (username, repo))
req.add_header("Authorization", "token %s" % token)
result = urllib2.urlopen(req, data)
result = json.loads('\n'.join(result.readlines()))
print result['number']
Hope that helps somebody.
The v3 github api has a nice feature where it can convert markdown to html. No authentication is needed to request this information, hence it makes a nice example to try before delving into the more tricky world of authentication.
The API docs state:
Render an arbritrary Markdown document
POST /markdown Input
text Required string - The Markdown text to render
And even give an example of a markdown string to be converted:
{"text": "Hello world github/linguist#1 **cool**, and #1!"}
Given this information, lets build an urllib2 Request for the html-ified version of that markdown.
import urllib2
import json
data = {"text": "Hello world github/linguist#1 **cool**, and #1!"}
json_data = json.dumps(data)
req = urllib2.Request("https://api.github.com/markdown")
result = urllib2.urlopen(req, json_data)
print '\n'.join(result.readlines())
And the result is some html representing the markdown.
Given this example, it is not the JSON which is causing the 404 when posting your request, but more likely the authentication (as you have already answered).
Isn't the data meant to be URL-encoded, not JSON? Try:
data = {"name": "test", "size": "4"}
req = urllib2.Request("https://api.github.com/repos/<user>/<repo>/downloads")
req.add_header('Authorization', 'token ' + '<token>')
result = urllib2.urlopen(req, urllib.parse.urlencode(data))
I need to perform preemptive basic authentication against an HTTP server, i.e., authenticate right away without waiting on a 401 response. Can this be done with httplib2?
Edit:
I solved it by adding an Authorization header to the request, as suggested in the accepted answer:
headers["Authorization"] = "Basic {0}".format(
base64.b64encode("{0}:{1}".format(username, password)))
Add an appropriately formed 'Authorization' header to your initial request.
This also works with the built-in httplib (for anyone wishing to minimize 3rd-party libs/modules). I am using it to authenticate with our Jenkins server using the API Token that Jenkins can create for each user.
>>> import base64, httplib
>>> headers = {}
>>> headers["Authorization"] = "Basic {0}".format(
base64.b64encode("{0}:{1}".format('<username>', '<jenkins_API_token>')))
>>> ## Enable the job
>>> conn = httplib.HTTPConnection('jenkins.myserver.net')
>>> conn.request('POST', '/job/Foo-trunk/enable', None, headers)
>>> resp = conn.getresponse()
>>> resp.status
302
>>> ## Disable the job
>>> conn = httplib.HTTPConnection('jenkins.myserver.net')
>>> conn.request('POST', '/job/Foo-trunk/disable', None, headers)
>>> resp = conn.getresponse()
>>> resp.status
302
I realize this is old, but I figured I'd throw in the solution if you're using Python 3 with httplib2 since I haven't been able to find it anywhere else. I'm also authenticating against a Jenkins server using the API Token for each Jenkins user. If you're not concerned with Jenkins, simply substitute the actual user's password for the API Token.
b64encode is expecting an binary string of ASCII characters. With Python 3 a TypeError will be raised if a plain string is passed in. To get around this, the "user:api_token" portion of the header must be encoded using either 'ascii' or 'utf-8', passed to b64encode, then the resulting byte string must be decoded to a plain string before being placed in the header. The following code did what I needed:
import httplib2, base64
cred = base64.b64encode("{0}:{1}".format(
<user>, <api_token>).encode('utf-8')).decode()
headers = {'Authorization': "Basic %s" % cred}
h = httplib2.Http('.cache')
response, content = h.request("http://my.jenkins.server/job/my_job/enable",
"GET", headers=headers)