I'm currently trying to implement a call to the Royal Mail API usingNodeJs and SOAP; I'm having difficulty recreating the security headers implementation following an example python script
The python script is as follows
#!/usr/local/bin/python2.7
import os
import sha
import binascii
import base64
password = 'test'
CREATIONDATE = '2016-03-29T14:03:46Z'
nonce = '7715776714'
HASH = sha.new(password).digest()
BASE64PASSWORD = base64.b64encode(HASH)
digest = sha.new(nonce + CREATIONDATE + HASH).digest()
PASSWORDDIGEST = base64.b64encode(digest)
ENCODEDNONCE = base64.b64encode(nonce)
print 'NONCE = ', nonce
print 'BASE64PASSWORD', BASE64PASSWORD
print 'PASSWORDDIGEST ', PASSWORDDIGEST
print 'ENCODEDNONCE ', ENCODEDNONCE
print 'CREATIONDATE ', CREATIONDATE
Which outputs:
NONCE 7715776714
BASE64PASSWORD qUqP5cyxm6YcTAhz05Hph5gvu9M=
PASSWORDDIGEST coDzcnSZObFfrM0FY33GcfxjOj4=
ENCODEDNONCE NzcxNTc3NjcxNA==
CREATIONDATE 2016-03-29T14:03:46Z
I've re-created this using NodeJs but I seem to get a differing output - inputting the correct password in the python version and using the resulting data allows me to make a valid call to the API, using the output from NodeJs gives me an authorisation failure
Node Js Code
var createdDate, password = 'test', nonce;
createdDate = '2016-03-29T14:03:46Z';
nonce = '7715776714';
var crypto = require("crypto"),
passHash = crypto.createHash('sha1'),
digestHash = crypto.createHash('sha1');
passHash.update(password);
var HASH = passHash.digest();
console.log('NONCE ' + nonce)
console.log('BASE64PASSWORD ' + base64_encode_string(HASH))
digestHash.update(nonce + createdDate + HASH);
var digest = digestHash.digest();
var PASSWORDDIGEST = base64_encode_string(digest);
console.log('PASSWORDDIGEST ' + PASSWORDDIGEST);
var ENCODEDNONCE = base64_encode_string(nonce.toString());
console.log('ENCODEDNONCE ' + ENCODEDNONCE);
console.log('CREATIONDATE ' + createdDate);
Which outputs
NONCE 7715776714
BASE64PASSWORD qUqP5cyxm6YcTAhz05Hph5gvu9M=
PASSWORDDIGEST FRMDpkDOi1j9KB/sDHg1b7BYQgA=
ENCODEDNONCE NzcxNTc3NjcxNA==
CREATIONDATE 2016-03-29T14:03:46Z
It appears that the Sha for the HASH is identical but the second Sha (digest) gives a differing result in the NodeJs version. Any pointers to where I'm going wrong?
For reference I'm using sha library in python and crypto in NodeJs
One problem is that you're implicitly converting the HASH Buffer to a UTF-8 string which is liable to cause corrupt output when converting binary to UTF-8. Instead, you can call .update() multiple times and preserve the binary data in HASH:
digestHash.update(nonce + createdDate);
digestHash.update(HASH);
var digest = digestHash.digest();
On an unrelated note, you don't need base64_encode_string() for Buffers, as for those you can simply do buffer.toString('base64') (e.g. HASH.toString('base64')).
Related
I have an Api code from Python 2 is as below:
dataStr = request.POST['data']
data = json.loads(dataStr)
key_string = data.get('key')
seed = data.get('seed')
guid = data.get('guid')
sha1 = hashlib.sha1()
yesStr = seed + 'yes' + key_string + 'ke.yp.ad' + guid
sha1.update(yesStr)
yesValue = sha1.hexdigest()
resp = {
'data': yesValue,
'place_id': str(record.GroupNumber)
}
return HttpResponse(resp)
now I am upgrading it to python 3, but sha1.update must use an encoded string. but I don't know what the encoding format was. so far I tried
sha1.udpate(yesStr.encode("utf-8"))
sha1.udpate(yesStr.encode("ascii"))
but none of them is matching the old string. Can anyone help? thanks.
Finally I figured this out with my friend's help. The default encoding for Python 2 is 'latin1'. So what I did to fix the problem is to change
sha1.update(yesStr)
to:
sha1.update(yesStr.encode('latin1'))
and it worked in python 3.
i want some example for encryption between python and flutter to encrypt request and response body between client and server
i found some example code for AES CRT encryption , but i cant see same result in flutter and python
can anybody help me ?
UPDATE :
Flutter crypt package not have counter parameter but python Crypto.Cipher package have counter parameter
this is sample code for python:
plaintext = '123'.encode('utf-8')
key = '12345678911234567891123456789123'.encode("utf-8")
iv = '12345'.encode('utf')
iv_int = int(binascii.hexlify(iv), 16)
ctr = Counter.new(AES.block_size * 8, initial_value=iv_int)
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
ciphertext = aes.encrypt(plaintext)
print('ctr = ' + str(ctr))
print('iv = ' + str(base64.b64encode(iv)))
print('iv_int = ' + str(iv_int))
print('plaintext = ' + str(plaintext))
print('key = ' + str(base64.b64encode(key)))
print('ciphertext = ' + str(base64.b64encode(ciphertext)))
this is sample code for flutter :
final plainText = '123';
final key = encrypt.Key.fromUtf8('12345678911234567891123456789123');
final iv = encrypt.IV.fromUtf8('12345');
final encrypter = encrypt.Encrypter(encrypt.AES(key, mode: encrypt.AESMode.ctr));
final encrypted = encrypter.encrypt(plainText, iv: iv);
final decrypted = encrypter.decrypt(encrypted, iv: iv);
print('key = ' + key.base64);
print('iv =' + iv.base64);
print('encrypted = ' + encrypted.base64);
I had the same issue and posted a very similar question. Fortunately, I found the error by myself.
Here is the link: Python AES CTR Mode only encrypts first two bytes
To summarize, the problem: Dart automatically uses the PKCS7 Padding when using AES Encryption, but Python does not anything like that.
So either you set the padding in your Dart code to null e.g.:
aes = AES.new(key, AES.MODE_CTR, counter=ctr, padding: null)
or you add a PKCS7 padding to your python code with like the following:
# The message has to in bytes format
def pkcs7padding(message):
pl = 16 - (len(message) % 16)
return message + bytearray([pl for i in range(pl)])
ciphertext = aes.encrypt(pkcs7padding(plaintext))
i am trying to encode a particular string with python with pycrypto and encode the same string with nodejs with crypto.
i am getting different results in both the cases for the same input string
python code:
from Crypto.Cipher import AES
from hashlib import md5
import base64
password = 'aquickbrownfoxjumpsoverthelazydog'
input = 'hello+world'
BLOCK_SIZE = 16
def pad (data):
pad = BLOCK_SIZE - len(data) % BLOCK_SIZE
return data + pad * chr(pad)
def unpad (padded):
pad = ord(padded[-1])
return padded[:-pad]
def text_encrypt(data, nonce, password):
m = md5()
m.update(password)
key = m.hexdigest()
m = md5()
m.update(password + key)
iv = m.hexdigest()
data = pad(data)
aes = AES.new(key, AES.MODE_CBC, iv[:16])
encrypted = aes.encrypt(data)
return base64.urlsafe_b64encode(encrypted)
output = text_encrypt(input, "", password)
print output
and the nodejs code is as follows:
var crypto = require('crypto');
var password = 'aquickbrownfoxjumpsoverthelazydog';
var input = 'hello+world';
var encrypt = function (input, password, callback) {
var m = crypto.createHash('md5');
m.update(password)
var key = m.digest('hex');
m = crypto.createHash('md5');
m.update(password + key)
var iv = m.digest('hex');
var data = new Buffer(input, 'utf8').toString('binary');
var cipher = crypto.createCipheriv('aes-256-cbc', key, iv.slice(0,16));
var nodev = process.version.match(/^v(\d+)\.(\d+)/);
var encrypted;
if( nodev[1] === '0' && parseInt(nodev[2]) < 10) {
encrypted = cipher.update(data, 'binary') + cipher.final('binary');
} else {
encrypted = cipher.update(data, 'utf8', 'binary') + cipher.final('binary');
}
var encoded = new Buffer(encrypted, 'binary').toString('base64');
callback(encoded);
};
encrypt(input, password, function (encoded) {
console.log(encoded);
});
the results for both the cases is different but after decryption they both tend to give the same correct result.
what might be the issue here?
You didn't specify what different results are you getting but those two should produce same-ish result. The only difference I see is in the base64 alphabet you're using.
In Python you're calling base64.urlsafe_b64encode() which differs from the standard Base64 in what characters it uses for values for 62 and 63 (- and _ instead of + and /). To get the same result, either in Python return:
return base64.b64encode(encrypted)
Or post-process the base64 encoded string in Node.js:
encoded = encoded.replace(/_/g, '/').replace(/-/g, '+');
All this being said, as I've mentioned in my comment, never derive an IV from your password/key (or anything else deterministic and unchanging). Use a cryptographically secure PRNG for it.
i try to encrypt the text with python and then i execute my code i get an error :
import base64
import boto3
from Crypto.Cipher import AES
PAD = lambda s: s + (32 - len(s) % 32) * ' '
def get_arn(aws_data):
return 'arn:aws:kms:{region}:{account_number}:key/{key_id}'.format(**aws_data)
def encrypt_data(aws_data, plaintext_message):
kms_client = boto3.client(
'kms',
region_name=aws_data['region'])
data_key = kms_client.generate_data_key(
KeyId=aws_data['key_id'],
KeySpec='AES_256')
cipher_text_blob = data_key.get('CiphertextBlob')
plaintext_key = data_key.get('Plaintext')
# Note, does not use IV or specify mode... for demo purposes only.
cypher = AES.new(plaintext_key, AES.MODE_ECB)
encrypted_data = base64.b64encode(cypher.encrypt(PAD(plaintext_message)))
# Need to preserve both of these data elements
return encrypted_data, cipher_text_blob
def main():
# Add your account number / region / KMS Key ID here.
aws_data = {
'region': 'eu-west-1',
'account_number': '701177775058',
'key_id': 'd67e033d-83ac-4b5e-93d4-aa6cdc3e292e',
}
# And your super secret message to envelope encrypt...
plaintext = PAD('Hello, World!')
# Store encrypted_data & cipher_text_blob in your persistent storage. You will need them both later.
encrypted_data, cipher_text_blob = encrypt_data(aws_data, plaintext)
print(encrypted_data)
if __name__ == '__main__':
main()
i Get : raise TypeError("Only byte strings can be passed to C code")
TypeError: Only byte strings can be passed to C code
Maybe whom know why? and how can i fix it ? please suggest!
Writing #Jeronimo's comment as an answer here, I was stuck with this same problem too and this helped.
Append a .encode("utf-8") to whatever you are passing to cypher.encrypt() function.
cypher.encrypt(PAD(plaintext_message).encode("utf-8"))
Note: this seems to be for python 3.x. For 2.x this same solution may not work.
A bit of a weird question perhaps, but I'm trying to replicate a python example where they are creating a HMAC SHA256 hash from a series of parameters.
I've run into a problem where I'm supposed to translate an api key in hex to ascii and use it as secret, but I just can't get the output to be the same as python.
>>> import hmac
>>> import hashlib
>>> apiKey = "76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b"
>>> apiKey.decode('hex')
'v\xb0,\x0cEC\xa8^EU$fiL\xf6w\x93x3\xc9\xcc\xe8\x7f\nb\x82H\xaf-,I['
If I've understood the material online this is supposed to represent the hex string in ascii characters.
Now to the powershell script:
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$hexstring = ""
for($i=0; $i -lt $apikey.Length;$i=$i+2){
$hexelement = [string]$apikey[$i] + [string]$apikey[$i+1]
$hexstring += [CHAR][BYTE][CONVERT]::toint16($hexelement,16)
}
That outputs the following:
v°,♀EC¨^EU$fiLöw?x3ÉÌè⌂
b?H¯-,I[
They are almost the same, but not quite and using them as secret in the HMAC generates different results. Any ideas?
Stating the obvious: The key in this example is no the real key.
Update:
They look more or less the same, but the encoding of the output is different. I also verified the hex to ASCII in multiple online functions and the powershell version seems right.
Does anyone have an idea how to compare the two different outputs?
Update 2:
I converted each character to integer and both Python and Powershell generates the same numbers, aka the content should be the same.
Attaching the scripts
Powershell:
Function generateToken {
Param($apikey, $url, $httpMethod, $queryparameters=$false, $postData=$false)
#$timestamp = [int]((Get-Date -UFormat %s).Replace(",", "."))
$timestamp = "1446128942"
$datastring = $httpMethod + $url
if($queryparameters){ $datastring += $queryparameters }
$datastring += $timestamp
if($postData){ $datastring += $postData }
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$apiAscii = HexToString -hexstring $apiKey
$hmacsha.key = [Text.Encoding]::ASCII.GetBytes($apiAscii)
$signature = $hmacsha.ComputeHash([Text.Encoding]::ASCII.GetBytes($datastring))
$signature
}
Function HexToString {
Param($hexstring)
$asciistring = ""
for($i=0; $i -lt $hexstring.Length;$i=$i+2){
$hexelement = [string]$hexstring[$i] + [string]$hexstring[$i+1]
$asciistring += [CHAR][BYTE][CONVERT]::toint16($hexelement,16)
}
$asciistring
}
Function TokenToHex {
Param([array]$Token)
$hexhash = ""
Foreach($element in $Token){
$hexhash += '{0:x}' -f $element
}
$hexhash
}
$apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
$postData = "{param1: 123, param2: 456}"
$token = generateToken -uri $apiEndpoint -httpMethod "GET" -queryparameters $queryParameters, postData=postData, -apiKey $apiKey
TokenToHex -Token $token
Python:
import hashlib
import hmac
import time
try: import simplejson as json
except ImportError: import json
class HMACSample:
def generateSecurityToken(self, url, httpMethod, apiKey, queryParameters=None, postData=None):
#timestamp = str(int(round(time.time()*1000)))
timestamp = "1446128942"
datastring = httpMethod + url
if queryParameters != None : datastring += queryParameters
datastring += timestamp
if postData != None : datastring += postData
token = hmac.new(apiKey.decode('hex'), msg=datastring, digestmod=hashlib.sha256).hexdigest()
return token
if __name__ == '__main__':
apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
apiKey = "76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b";
queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
postData = "{param1: 123, param2: 456}"
tool = HMACSample()
hmac = tool.generateSecurityToken(url=apiEndpoint, httpMethod="GET", queryParameters=queryParameters, postData=postData, apiKey=apiKey)
print json.dumps(hmac, indent=4)
apiKey with "test" instead of the converted hex to ASCII string outputs the same value which made me suspect that the conversion was the problem. Now I'm not sure what to believe anymore.
/Patrik
ASCII encoding support characters from this code point range 0–127. Any character outside this range, encoded with byte 63, which correspond to ?, in case you decode byte array back to string. So, with your code, you ruin your key by applying ASCII encoding to it. But if what you want is a byte array, then why do you do Hex String -> ASCII String -> Byte Array instead of just Hex String -> Byte Array?
Here is PowerShell code, which generate same results, as your Python code:
function GenerateToken {
param($apikey, $url, $httpMethod, $queryparameters, $postData)
$datastring = -join #(
$httpMethod
$url
$queryparameters
#[DateTimeOffset]::Now.ToUnixTimeSeconds()
1446128942
$postData
)
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.Key = #($apikey -split '(?<=\G..)(?=.)'|ForEach-Object {[byte]::Parse($_,'HexNumber')})
[BitConverter]::ToString($hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($datastring))).Replace('-','').ToLower()
}
$apiEndpoint = "http://test.control.llnw.com/traffic-reporting-api/v1"
#what you see in Control on Edit My Profile page#
$apikey = '76b02c0c4543a85e45552466694cf677937833c9cce87f0a628248af2d2c495b';
$queryParameters = "shortname=bulkget&service=http&reportDuration=day&startDate=2012-01-01"
$postData = "{param1: 123, param2: 456}"
GenerateToken -url $apiEndpoint -httpMethod "GET" -queryparameters $queryParameters -postData $postData -apiKey $apiKey
I also fix some other errors in your PowerShell code. In particular, arguments to GenerateToken function call. Also, I change ASCII to UTF8 for $datastring encoding. UTF8 yields exactly same bytes if all characters are in ASCII range, so it does not matter in you case. But if you want to use characters out of ASCII range in $datastring, than you should choose same encoding, as you use in Python, or you will not get the same results.