The API docs are here
The only code example is in Java, here
Every time I try to authenticate I get:
{
"error": "Authorization field missing, malformed or invalid"
}
I have been through the auth docs many times over and still no luck.
Here is my code:
import requests
import secrets
import codecs
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
import hashlib
import hmac
import base64
import urllib.parse
key = '<API_KEY>'
secret = '<API_SECRET>'
# Getting current time
now = datetime.now()
stamp = mktime(now.timetuple())
# Formats time into this format --> Mon, 25 Jul 2016 16:36:07 GMT
formated_time = format_date_time(stamp)
# Generates a secure random string for the nonce
nonce = secrets.token_urlsafe(30)
# Combines date and nonce into a single string that will be signed
signature_string = 'date' + ':' + formated_time + '\n' + 'x-mod-nonce' + ':' + nonce
# Expected output example --> date: Mon, 25 Jul 2016 16:36:07 GMT\nx-mod-nonce: 28154b2-9c62b93cc22a-24c9e2-5536d7d
# Encodes secret and message into a format that can be signed
secret = bytes(secret, encoding='utf-8')
message = bytes(signature_string,encoding='utf-8')
# Signing process
digester = hmac.new(secret, message, hashlib.sha1)
# Converts to hex
hex_code = digester.hexdigest()
# Decodes the signed string in hex into base64
b64 = codecs.encode(codecs.decode(hex_code, 'hex'), 'base64').decode()
# Encodes the string so it is safe for URL
url_safe_code = urllib.parse.quote(b64,safe='')
# Adds the key and signed response
authorization = f'Signature keyId="{key}",algorithm="hmac-sha1",headers="date x-mod-nonce",signature="{url_safe_code}"'
account_id = 'A120BU48'
url = f'https://api-sandbox.modulrfinance.com/api-sandbox/accounts/{account_id}'
headers = {
'Authorization': authorization, # Authorisation header
'Date' : formated_time, # Date header
'x-mod-nonce': nonce, # Addes nonce
'accept': 'application/json',
}
response = requests.get(url,headers=headers)
print(response.text)
I am not sure where the process is going wrong, as far as I know, the signature is being signed correctly as I added in the test data from the authentication example and I get the expected string.
If you want to try with real API keys, register for access here
The docs for the API endpoint I am trying to call is here
The docs you linked has a space between the colon and the values.
signature_string = 'date' + ':' + formated_time + '\n' + 'x-mod-nonce' + ':' + nonce
should be:
signature_string = 'date' + ': ' + formated_time + '\n' + 'x-mod-nonce' + ': ' + nonce
or (simpler):
signature_string = 'date: ' + formated_time + '\n' + 'x-mod-nonce: ' + nonce
Update
I registered to see what is going on. I also ran your code on the example given in the documentation and saw that the signature is not entirely correct.
In addition to the change I suggested above, a further change was necessary.
After changing the line
b64 = codecs.encode(codecs.decode(hex_code, 'hex'), 'base64').decode()
to
b64 = codecs.encode(codecs.decode(hex_code, 'hex'), 'base64').decode().strip()
the signature of the example matched.
After this I was able to connect to the API with my own keys.
Here is the complete working code:
import codecs
import hashlib
import hmac
import secrets
import urllib.parse
from datetime import datetime
from time import mktime
from wsgiref.handlers import format_date_time
import requests
key = '<key>'
secret = '<secret>'
account_id = '<account id>'
url = f'https://api-sandbox.modulrfinance.com/api-sandbox/accounts/{account_id}'
# Getting current time
now = datetime.now()
stamp = mktime(now.timetuple())
# Formats time into this format --> Mon, 25 Jul 2016 16:36:07 GMT
formatted_time = format_date_time(stamp)
# Generates a secure random string for the nonce
nonce = secrets.token_urlsafe(30)
# Combines date and nonce into a single string that will be signed
signature_string = 'date' + ': ' + formatted_time + '\n' + 'x-mod-nonce' + ': ' + nonce
# Encodes secret and message into a format that can be signed
secret = bytes(secret, encoding='utf-8')
message = bytes(signature_string, encoding='utf-8')
# Signing process
digester = hmac.new(secret, message, hashlib.sha1)
# Converts to hex
hex_code = digester.hexdigest()
# Decodes the signed string in hex into base64
b64 = codecs.encode(codecs.decode(hex_code, 'hex'), 'base64').decode().strip()
# Encodes the string so it is safe for URL
url_safe_code = urllib.parse.quote(b64, safe='')
# Adds the key and signed response
authorization = f'Signature keyId="{key}",algorithm="hmac-sha1",headers="date x-mod-nonce",signature="{url_safe_code}"'
headers = {
'Authorization': authorization, # Authorisation header
'Date': formatted_time, # Date header
'x-mod-nonce': nonce, # Adds nonce
'accept': 'application/json',
}
response = requests.get(url, headers=headers)
print(response.text)
Related
I am trying to post a record into Netsuite and it works well with the Postman, but throwing invalid login Error with Python.
Below is the Code.
import requests
url = "https://5559796-sb1.suitetalk.api.netsuite.com/services/rest/record/v1/qbc_pqr_1234"
payload="{\"aaa\":\"AN\",\"bbb\":\"OA\",\"ccc\":1,\"ddd\":false,\"eee\":114.01,\"fff\":\"OAOTWH\",\"ggg\":\"hhh\",\"hhh\":18,\"iii\":2,\"jjj\":\"2021-07-31\",\"kkk\":3257.29,\"lll\":\"Intra-Brand Stay - 3.5%\"}"
headers = {
'Content-Type': 'application/json',
'Authorization': 'OAuth realm="5559796_SB1",oauth_consumer_key="xxxxx",oauth_token="abcdefgh",oauth_signature_method="HMAC-SHA256",oauth_timestamp="1630905651",oauth_nonce="xxxxx",oauth_version="1.0",oauth_signature="xxxxx"'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
And the error is :
{"type":"https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2","title":"Unauthorized","status":401,"o:errorDetails":[{"detail":"Invalid login attempt. For more details, see the Login Audit Trail in the NetSuite UI at Setup > Users/Roles > User Management > View Login Audit Trail.","o:errorCode":"INVALID_LOGIN"}]}
Also, how can I read the payload from a json file with multiple records.
While hitting NetSuite API you need to create a signature which is responsible to establish your connection below is the code which will help you.
import hashlib
import hmac
import json
import requests
import base64
import time
import random
import urllib.parse
def _generateTimestamp():
return str(int(time.time()))
def _generateNonce(length=11):
"""Generate pseudorandom number"""
return ''.join([str(random.randint(0, 9)) for i in range(length)])
def _generateSignature(method, url, consumerKey, Nonce, currentTime, token, consumerSecret,
tokenSecret, offset):
signature_method = 'HMAC-SHA256'
version = '1.0'
base_url = url
encoded_url = urllib.parse.quote_plus(base_url)
collected_string = None
if type(offset) == int:
collected_string = '&'.join(['oauth_consumer_key=' + consumerKey, 'oauth_nonce=' + Nonce,
'oauth_signature_method=' + signature_method, 'oauth_timestamp=' + currentTime,
'oauth_token=' + token, 'oauth_version=' + version, 'offset=' + str(offset)])
else:
collected_string = '&'.join(['oauth_consumer_key=' + consumerKey, 'oauth_nonce=' + Nonce,
'oauth_signature_method=' + signature_method, 'oauth_timestamp=' + currentTime,
'oauth_token=' + token, 'oauth_version=' + version])
encoded_string = urllib.parse.quote_plus(collected_string)
base = '&'.join([method, encoded_url, encoded_string])
key = '&'.join([consumerSecret, tokenSecret])
digest = hmac.new(key=str.encode(key), msg=str.encode(base), digestmod=hashlib.sha256).digest()
signature = base64.b64encode(digest).decode()
return urllib.parse.quote_plus(signature)
def import_customer():
nsAccountID = "YOUR_ACCOUNT_ID"
consumerKey = "YOUR_CONSUMER_KEY"
consumerSecret = "YOUR_CONSUMER_SECRET"
token = "YOUR_TOKEN"
tokenSecret = "YOUR_TOKEN_SECRET"
base_url = "https://nsAccountID.suitetalk.api.netsuite.com/services/rest/record/v1/customer"
Nonce = self._generateNonce(length=11)
currentTime = self._generateTimestamp()
signature = self._generateSignature('GET', base_url, consumerKey, Nonce, currentTime, token, consumerSecret, tokenSecret, offset)
payload = ""
oauth = "OAuth realm=\"" + nsAccountID + "\"," \
"oauth_consumer_key=\"" + consumerKey + "\"," \
"oauth_token=\"" + token + "\"," \
"oauth_signature_method=\"HMAC-SHA256\"," \
"oauth_timestamp=\"" + currentTime + "\"," \
"oauth_nonce=\"" + Nonce + "\"," \
"oauth_version=\"1.0\"," \
"oauth_signature=\"" + signature + "\""
headers = {
'Content-Type': "application/json",
'Authorization': oauth,
'cache-control': "no-cache",
}
response = requests.request("GET", base_url + '?offset=' + str(offset), data=payload, headers=headers)
return json.loads(response.text)
print(import_customer())
And here you can see the example of import customer. In every API hit you need to generate signature with every required information
I am trying to convert this part into Python3 and just can't get it to work.
import urllib
import datetime
import urllib2
import httplib
import hmac
from hashlib import sha256
TIMEOUT=60
def getDimMetrics(options):
now = datetime.datetime.now()
# create a dictionary of the arguments to the API call
post_dict = {}
if options.group:
post_dict['group'] = options.group
# start and end are dates, the format will be specified on the command line
# we will simply pass those along in the API
if options.start:
post_dict['start'] = options.start
if options.end:
post_dict['end'] = options.end
# encode the data
post_data = urllib.urlencode(post_dict)
protocol = 'https'
if 'localhost' in options.host:
# for testing
protocol = 'http'
url = protocol + '://' + options.host + options.url + '.' + options.type.lower()
# create the request
request = urllib2.Request(url, post_data)
# add a date header (optional)
request.add_header('Date', str(now))
# calculate the authorization header and add it
hashString = 'POST' + request.get_selector() + request.get_header('Date') + 'application/x-www-form-urlencoded' + str(len(request.get_data()))
calc_sig = hmac.new(str(options.secret), hashString,
sha256).hexdigest()
request.add_header('Authorization', 'Dimo %s:%s' %(options.key, calc_sig))
print 'url=', url
print 'post_data=', post_data
print 'headers=', request.headers
This is what I have in Python3. When I run it I get an error message saying 400 Malformed Authorization header. How can I fix this error so that I can get this part running in python 3.
from requests_oauthlib import OAuth1Session
CONSUMER_KEY = "aaaaaaaaaaaaaaaaa"
CONSUMER_SECRET = "bbbbbbbbbbbbbbbbbbbb"
host = "api.dimo.com"
uri = "/api/Visits.json"
oauthRequest = OAuth1Session(CONSUMER_KEY,
client_secret=CONSUMER_SECRET)
url = 'https://' + host + uri
headers = {
'Accept': "application/json",
'Accept-Encoding': "application/x-www-form-urlencoded",
'Authorization': "dimo bbbbbbbbbbbbbbb"
}
response = oauthRequest.post(url, headers=headers)
print(response.status_code)
print(response.content)
ERROR
400
b'Malformed Authorization header.\n'
Problem
You're experiencing issues converting python 2.7 code to python 3.x.
Your authorization header in your original python 3 converted code isn't valid.
Solution
Run python 2to3 converter with some additional cleanup based on PEP 8 -- Style Guide for Python ( max line length, etc.).
Example
import datetime
import urllib.request
import urllib.error
import urllib.parse
import http.client
import hmac
from hashlib import sha256
TIMEOUT = 60
def getDimMetrics(options):
now = datetime.datetime.now()
# create a dictionary of the arguments to the API call
post_dict = {}
if options.group:
post_dict['group'] = options.group
# start and end are dates, the format will be specified on the command line
# we will simply pass those along in the API
if options.start:
post_dict['start'] = options.start
if options.end:
post_dict['end'] = options.end
# encode the data
post_data = urllib.parse.urlencode(post_dict)
protocol = 'https'
if 'localhost' in options.host:
# for testing
protocol = 'http'
url = f'{protocol}://{options.host}{options.url}.' \
f'{options.type.lower()}'
# create the request
request = urllib.request.Request(url, post_data)
# add a date header (optional)
request.add_header('Date', str(now))
# calculate the authorization header and add it
hashString = f'POST{request.get_selector()}' \
f'{request.get_header('Date')}' \
f'application/x-www-form-urlencoded' \
f'{str(len(request.get_data()))}'
calc_sig = hmac.new(
str(options.secret), hashString, sha256).hexdigest()
request.add_header('Authorization', 'Dimo {options.key}:{calc_sig}')
print(f'url={url}')
print(f'post_data={post_data}')
print(f'headers={request.headers}')
References
Python 2to3: https://docs.python.org/3/library/2to3.html
I am trying to make authenticated post request to place an order on the testnet. I have been trying to get it going for couple of days, but unable to figure out why I get "Signature not Valid"
import json
import requests
import aiohttp
import asyncio
import urllib
import time
import hashlib, hmac
import urllib.parse
import json
api_key = "YOUR_API_KEY"
api_secret = "YOUR_SECRET_KEY"
base_url = 'https://testnet.bitmex.com'
method = 'POST'
data = '{"symbol":"XBTUSD","quantity":1,"price":395.01}'
path = '/api/v1/order'
url = base_url + path
print(url)
nonce = int(time.time() * 1000)
print(nonce)
message = bytes(method + path + str(nonce) + data, 'utf-8')
print(message)
def generate_signature(secret, verb, url, nonce, data):
"""Generate a request signature compatible with BitMEX."""
# Parse the url so we can remove the base and extract just the path.
parsedURL = urllib.parse.urlparse(url)
path = parsedURL.path
if parsedURL.query:
path = path + '?' + parsedURL.query
message = bytes(verb + path + str(nonce) + data, 'utf-8')
# print("Computing HMAC: %s" % message)
signature = hmac.new(bytes(secret, 'utf-8'), message, digestmod=hashlib.sha256).hexdigest()
return signature
# signature = hmac.new(bytes(api_secret, 'utf-8'), message, digestmod=hashlib.sha256).hexdigest()
signature = generate_signature(api_secret, method, path, nonce, data)
print(signature)
headers = {'api-expires':str(nonce),'api-key':api_key,'api-signature':signature, 'Content-Type': 'aplication/json','Accept': 'application/json', 'X-Requested-With': 'XMLHttpRequest'}
print(headers)
r = requests.post(url, data=data, headers=headers)
print(r.status_code)
print(r.text)
What is wrong in this code? Please assume that correct api key and secret provided.
I am getting the following response.
401
{"error":{"message":"Signature not valid.","name":"HTTPError"}}
https://www.bitmex.com/app/apiKeysUsage
I have a query to an job board API using Python Requests. It then writes to a table, that is included in a web page. Sometimes the request will return no data(if there are no open jobs). If so, I want to write a string to the included file instead of the table. What is the best way to identify a response of no data? Is it as simple as: if response = "", or something along those lines?
Here is my Python code making the API request:
#!/usr/bin/python
import requests
import json
from datetime import datetime
import dateutil.parser
url = "https://data.usajobs.gov/api/Search"
querystring = {"Organization":"LF00","WhoMayApply":"All"}
headers = {
'authorization-key': "ZQbNd1iLrQ+rPN3Rj2Q9gDy2Qpi/3haXSXGuHbP1SRk=",
'user-agent': "jcarroll#fec.gov",
'host': "data.usajobs.gov",
'cache-control': "no-cache",
}
response = requests.request("GET", url, headers=headers, params=querystring)
responses=response.json()
with open('/Users/jcarroll/work/infoweb_branch4/rep_infoweb/trunk/fec_jobs.html', 'w') as jobtable:
jobtable.write("Content-Type: text/html\n\n")
table_head="""<table class="job_table" style="border:#000">
<tbody>
<tr>
<th>Vacancy</th>
<th>Grade</th>
<th>Open Period</th>
<th>Who May Apply</th>
</tr>"""
jobtable.write(table_head)
for i in responses['SearchResult']['SearchResultItems']:
start_date = dateutil.parser.parse(i['MatchedObjectDescriptor']['PositionStartDate'])
end_date = dateutil.parser.parse(i['MatchedObjectDescriptor']['PositionEndDate'])
jobtable.write("<tr><td><strong><a href='" + i['MatchedObjectDescriptor']['PositionURI'] + "'>" + i['MatchedObjectDescriptor']['PositionID'] + ", " + i['MatchedObjectDescriptor']['PositionTitle'] + "</a></strong></td><td>" + i['MatchedObjectDescriptor']['JobGrade'][0]['Code'] + "-" + i['MatchedObjectDescriptor']['UserArea']['Details']['LowGrade']+ " - " + i['MatchedObjectDescriptor']['UserArea']['Details']['HighGrade'] + "</td><td>" + start_date.strftime('%b %d, %Y')+ " - " + end_date.strftime('%b %d, %Y')+ "</td><td>" + i['MatchedObjectDescriptor']['UserArea']['Details']['WhoMayApply']['Name'] + "</td></tr>")
jobtable.write("</tbody></table>")
jobtable.close
You have a couple of options depending on what the response actually is. I assume, case 3 applies best:
# 1. Test if response body contains sth.
if response.text: # body as str
# ...
# body = response.content: # body as bytes, useful for binary data
# 2. Handle error if deserialization fails (because of no text or bad format)
try:
json_data = response.json()
# ...
except ValueError:
# no JSON returned
# 3. check that .json() did NOT return an empty dict/list
if json_data:
# ...
# 4. safeguard against malformed/unexpected data structure
try:
data_point = json_data[some_key][some_index][...][...]
except (KeyError, IndexError, TypeError):
# data does not have the inner structure you expect
# 5. check if data_point is actually something useful (truthy in this example)
if data_point:
# ...
else:
# data_point is falsy ([], {}, None, 0, '', ...)
If your APIs has been written with correct status codes, then
200 means successful response with a body
204 means successful response without body.
In python you can check your requirement as simply as the following
if 204 == response.status_code :
# do something awesome
I'm trying to send a string array to the bing translate api, which I have received and encoded as UTF-8.
However, when I make the call with the variable the service spits out an error:
"There was an error deserializing the object of type System.String[]. Encountered unexpected character 'ï'."
If I manually add the string array as a string not within a variable, it works. I'm using the Requests module, so I'm thinking it's encoding the url... I have no idea.
class getLanguages(apiFunctions):
def GET(self):
#get the access token
data = self.accessToken()
token = data['access_token']
codes = self.getCodes(token)
languageNames = self.getLanguageNames(token,codes)
codesList = ast.literal_eval(codes)
return languageNames + codes
def getCodes(self,token):
url = 'http://api.microsofttranslator.com/V2/Ajax.svc/GetLanguagesForTranslate'
auth_header = {'Authorization': 'Bearer ' + token}
r = requests.get(url, headers=auth_header)
return r.text.encode('utf-8')
def getLanguageNames(self,token,codes):
url = 'http://api.microsofttranslator.com/V2/Ajax.svc/GetLanguageNames'
#this is where I add the language codes string to the query
payload = {'locale': 'en', 'languageCodes': codes}
auth_header = {'Authorization': 'Bearer ' + token}
r = requests.post(url, params=payload, headers=auth_header)
return r.text.encode('utf-8')
And this is the string:
["ar","bg","ca","zh-CHS","zh-CHT","cs","da","nl","en","et","fi","fr","de","el","ht","he","hi","mww","hu","id","it","ja","tlh","tlh-Qaak","ko","lv","lt","ms","mt","no","fa","pl","pt","ro","ru","sk","sl","es","sv","th","tr","uk","ur","vi","cy"]